undo the nut to sit's not tight BUT it still needs to be on the threads for around half their length
then, give it some thwumps from behind with the palms of your hands, it may need a few good ones and it'll pop out

the reason you leave the nut on is so that it doesn't break your face when it comes flying off

if you are talking about the non adjustable column, they have a bolt (though it may be the other way around)