Leave college in disappointment (8)

I haven't cracked this one yet — but the brilliant team at Times for the Times have:

to ‘come down’ is the opposite of ‘go up’ to college, I assume.