Those longer shafts can cause problems, having said that I had the same problem for literally years, and systematically changed out everything before I finally got to changing the drivers side hub and it turned out that was the problem.

It could also be your ball joint's/suspension arm if they have some play in them.