go and drive both of them......both awesome cars in their own right.
m3 is too harsh for every day use in comparison to m5,very easy to potter around in the m5.
i drove my m5 to donnington-seating 4 in total comfort with tv on and cruise control.then proceeded to thrash the life out of it on track all day-surprising qute a few people.......then drove home at a lovey leisurely pace....... awesome car.

i sold my m5 and took an e46 m3 in the deal-and instantly wished i hadnt got rid of the m5.i would chose the m5,and im under 40(39!).