I would say 1mm minimum, my XR4x4 came with bigger front discs and the calipers were too close to the discs and made some horrible grinding noises after heavy braking. I ground the caliper away for clearance and the problem was gone.

Mark