The things is, as new fence is on your side, you've lost space in your garden now. I wouldnt put up with that.
As far as I can see, he has chopped down the trees which belonged to you, and now its cost you money. I certainly wouldnt let that stand mate...