Firstly welcome to the forum

A few things to try

1. Have you got a boost gauge? Try checking the vacuum pipe on the gauge end and not inlet.
2. Don't just wipe the idle valve, soak it in carb cleaner overnight (12hrs+) You'll be amazed how much grit/rubbish can be in there.
3. When warm unplug the idle valve, if it's still low you can adjust the idle using the screw underneath where the inlet meets the boost hose.

try them and report back