It might...you might just knock a hole in it, or it could end up inside the waterways in the head which you then still need to get it back out.

Ideally though try and pull it out and as it's already corroded it should be easier to get something in to try that.
Oil seal pullers are pretty handy tools

But they can be awkward both to remove and get back in as they are just an interference fit. Hence the better access you have the more chance you have of not making a mess.