That's the problem with these handles, they tend to look very tatty!
I sanded mine down so they were smooth and painted them black, seemed like a good idea at the time but wish I hadn't bothered now.
If I was going to do it again I would probably just take them off, give them a good clean and then maybe use something like PlastiKote to try and colour match the best I could.
Probably worth picking up a spare set of handles of eBay to experiment with beforehand.