There isn't a problem unless it runs seriously un even. Does the idle settle down if you press the pedal and hold it down?

Basically your brakes are servo assisted and it gets it's vacuum supply from the inlet manifold...when you press the pedal, it introduces a small amount of unmetered air into the manifold as the engine has to replace the vacuum back in the servo which causes a slight uneven idle breifly...