You can still claim from his insurance it will not matter your car is not insured.
Whether its a car or a building hes damaged its not relevent, its still your property and its still damaged.