Depends on your contract! If it says "work extra hours as requested by your manager" then yes, you have to work! If not then tell him where to poke it.