Are you all on 3G or 3GS phones?
I upgraded my 3GS the other night and it works fine, all contacts there, all apps there, everything works as it should other than any photos I have put on the phone myself needing reloading due to pixelating slightly, but as said, reloading cured them.
I know the people with the 3G ones have had trouble as it appears the OS4 does not seem compatible.