I am a Construction Surveyor and project manager, I would be very inclined to mention the clear lack of health and safety on site with the crane driver not taking any notice and there should of been a banksmen present for starters!

As regards his claim, he doesnt have a leg to stand on. If you are employed to fit the windows i.e. do the spanner work, he being the main contractor is responsible for getting it there in a safe manner and ultimately making sure you are doing it safely.

Had you been asked to read and sign a method statement before you started the works and if so did you abide by it to the letter? If not then he has failed in his duties if so and you have done as you are told/signed to its his fault.

On the other hand tell him to go fuck himself! How the hell is he going to get the money out of you? Take you to court?? LMFAO!