I'd assume this is rubber for good reason.
Unless all UJ's etc run totally true....I'd be inclined to leave well alone otherwise it could risk damaging the rack.
Do any cars have a soild link to the rack ? Not like a PAS rack requires huge effort to turn it.