the answer to the question is lots of hard taps around the core to split it from the exhaust housing. don't go too much on one side as it could damage the exhaust turbine blades.
as said though it may need balancing, just mark the nut/shaft and wheel with a scribe and it should be fine when put back together, maybe the tiniest bit out of balance.