i sold my m5 last year and dealt it with an e46 m3.......i was actually quite pleased until i began to use it and really missed my m5.the grunt from the v8 is superb.the m3 is a bit "harsh" for everyday use.
i also did a few trackdays in my m5 and was surprisingly capable out there with uprated brake pads.
i really like both cars though but i think the m5 shades it as an all rounder.if you have e46 budget money,buy an m5 and an e36 m3 for trackdays...........mine is awesome!