To be honest with you if you arent spooling the turbo up much when you come off the throttle the vacuum pressure in the inlet manifold wont be very high, you can drive the car in such a way that the DV releases very gently, if you boot it down the road you will hear how much more violently it dumps.

Unless you are getting running problems i wouldnt worry too much