I cant speak for Rolex's but I kow several people with Genuine Tags. And they wear them all the time. They get mucky and wet and they still maintain time. And the paint finish doesnt seem to to be wearing. They seem fairly hardy to me.
But I can see your point as to not ruin it by wearing it when welding for example.