If your're going to 'try' and run homebrew (ie, programs/games that people have written instead of stuff you buy from the shop or playstation network) you do need an older version of the firmware. Much older - Sony prevented homebrew programs from running from version 1.5 onwards I think. Some buggy versions of the firmware allow you do downgrade your firmware, ie, go from 2.0 to 1.5. Generally you can only go 'up'.

Some games that you buy require a certain minimum version of the firmware, which is usually included on the UMD. Personally, I only use my PSP for 'genuine' games, and browsing forums, so homebrew software isn't a problem for me.

The Magic memory stick you're referring to, i guess, is a MagicGate memory stick. There's nothing special about them really. MagicGate is a security feature developed by Sony for the Memory Stick range. If the device (a digital camera for example) supports MagicGate, and is told to use it, the contents of the memory stick become protected, so that any other device cannot read the contents of the stick. The stick is then 'bound' the the device that protected it.

If you want my opinion, gonnect the PSP to the internet, and update the firmware to the latest available version (5.00 I think). The memory stick you need is a Memory Stick Pro Duo - Sandisk or Sony ones are what you need. Take a look at www.mobymemory.com for some good deals and prices. Apart from other features, the latest firmware now supports the Playstation network, so you can download full games and demos direct from Sony.