The spacers as said allow the 4wd cams to be used in 2wd heads by spacing the cams back around 5.86mm more, and go between the cams machined journal that sits in the bearing and the rear face of the cam bearing, if they were needed and not fitted, when tightening the cams and them being drawn too far forwards they would as said hit the cam cap posts, your engine would be wrecked by now if that was the case.
It looks to me like the tensioner pulley is slightly oval, I'd be putting a dial gauge on it and checking to be safe.
The direction of tension cannot cause this walking motion yours has on the belt.
tabetha