Do a search on the land registry site that will cost you £2. This will show which side is your responsibility.

It wouldn't of hurt for them to of spoken to you about it though ...

If it's your side and it shows this on the LR then they will have remedy it to your liking. It's also criminal damage. I doubt they'll get arrested but none the less it still is.

If you need any advice then a friend of mine who is here now is a legal bod that deals with neighbour/boundary disputes.