Unless you're scared of this guy why don't you just tell the boss? Say it's causing tension among the team, and you'd like the situation regarding working hours clarified as you think some people are taking the piss?

I would.