sounds like surge(running to the right of theturbo airflow curve) , i have the same problem in 4/5th at over 0.5 bar
never had it with my sidedraughts , so im going to change the throttle to a larger unit and build a side feed plenum

if you have a boost guage fitted , it will show a rapid left to right cycling?

problem is time. your turbo in higher gears has more time to spool up and so builds up very high boost pressures at very low air flow rates
and runs into surge

its a bastard to understand as very few people understand this , spent weeks on the net to get to this conclusion

to get around this , you may eed to either change the cold side of your turbo and go smaller or unrestrict the flow capability or your engine with headwork , high lift cam , match porting etc