If it was the cam it would run rough all the time (unless you have an intermittent sticky lifter), ideally remove it to measure and inspect it.

Odd that it's ok if you put your foot down, perhaps the TPS is at fault.