I recently bought my 2004 320cd M Sport. Its a lovely car, can't fault it.
Turbos are better on the '04s due to them being an uprated design (with electrical wastegate actuator). So it was really only upto 53 you need to worry about turbos. However change the crankcase breather to a vortex one just incase

.
Also, delete the swirl flaps, these are the only other major issue with the engine, blanks can be bought for £6/ea (you need four) and about 3hrs to remove the swirl flaps, fit the blanks, replace the breather and clean the EGR valve