Look at this way mate - If he gave you an envelope with the money in and you said I want to check it first, would that be unreasonable? What if he said to you - "You sold the car heres an envelope with the money - but you can't check it - and drove off?"
You wouldn't be happy if it was bits of blank of paper in the envelope would you? You would want to check it was as agreed.
Technically you are right but when dealing with people you have show some courtesy. Don't leave him a negative - sort your own attitude out.