EBAY!!!!!
Hard drives can be got cheaper than usual, sizes from tiny tiny upto 250gb I see on there (looking for a coupld of HDD's myself at the mo)
Got a pukka soundcard off there already (new, but half retail price) and a graffix card (you want summat like a GeForce MX440 if you wanna watch films on ya TV (not good enough for 3d gaming tho) as it has an S-Video output on it...
CD writers, Roms, DVD writers and roms also much cheaper than retail, towers are cheap too.
I paid £60 for my 15" LCD screen (new) so thats proof they can be got cheap!
Keyboard and mouse for a fiver, pair of speakers for another fiver, pretty much sorted - just need software then (again, easily got on ebay, or if you have P2P you can download it off someone

)