They have the Thames clipper which is really handy for getting from places like Madame Tussauds, runs all the way from Waterloo to the O2 and makes various stops along the way, i think it's something like £6/£7?

From the Canary Wharf to Buckingham palace, it's roughly 6 miles.

They do signpost the tube maps quite well though and the hotel will give you a pretty good idea as well