Alex, just having freedom to control your hours, place of work and send a substitute may be enough, but i suspect you may have to go to court to argue it.
The inland revenue has a bad record with cases like this, they normally lose, but do you have the bottle to try?