If you feed the muscles you will lose minimal amounts imho.

The problem is people stop training and then stop the protein etc. Keep the intake up and there is no reason for you to wilt away massively.