Take off the crossover pipe and visually check that the throttle is closed.

You can check for broken seals by spraying them with WD40. It would have to be a very big break to leak enough air to get to 4000rpm. Have you also checked that you haven't got any bungs/hoses off of the various pipes to the plenum?

There is also a chance it could be the brake servo leaking.

Once you've checked the throttle and for leaks with WD40 I would suggest disconnecting the various hoses and plugging the holes to eliminate the other components.