As all have said mate (including the official ford tech book) 20w-50 is for warm climates. Not a problem when the engine is hot (SAE 50) but definately a problem when its cold. I suspect you will have more cam wear as oil will not reach the head quickly enough. But if your new cam kit is of good quality it may last longer anyway.
Secondly, there is nothing wrong with mineral however it breaks down with excessive heat and i don't think its as viscously stable as synthetic stuff, im also unsure on wear additives and minerals.... IMO Synthetic oils are superior, they have been trialled tested and proven, im fairly sure lots of people have complained of bad turbo life with mineral oil?
What oil pressure do you see cold and hot? maybe this has something to do with your crazy cam wear......
Rob,