The hub (and therefore the shim) is not completely parallel to the ground. If it was parallel to the ground it would not affect toe.

Think of it when you turn the shim 90 degrees, no everything it would normally change in camber is now changed in toe. Having it at 5 or 10 degrees (what it probably is in reality) means both camber and toe are changed.