The bearings go in different ways. Driving mine from 0-30 mph would sound like its going over a train track, but past that it would be quiet/not felt. Under breaking it would wobble/judder.

I'd change it just to be sure... change them both. You say that the bearing was in the hub you fitted, which means secondhand, true? How old was the bearing in the hub? My guess it was old and had been sitting around for a while.