But the guy has a point to be honest!
He has paid you via Paypal so you have the funds in your Paypal account. Is he supposed to wait until you transfer the money from Paypal to your bank account

If you don't trust Paypal then don't offer it as an alternative when it comes to payment!