As is said you probably have an air leak. I find the best way is to systematically remove potential air leaks by removing pipes one by one and blocking the connections on the engine.

Brake servos are always a prime suspect.

Does the car run smoothly if you hold it at the lowest achievable RPM? And if youve got a vacuum gauge what does that read/do?