Unfortunately, from a legal standpoint - he IS trying to fix it, so is perfectly within his rights to send it off. If anything I'd be grateful that he has agreed to fix it, rather than fob you off like most dealers would.
You're within your rights to expect it to be fixed within a reasonable time frame and I would write to him (ALWAYS keep a written record of what happens and when) and if you keep it formal he's less likely to string you along. Request that it is either repaired or replaced within 14 working days and take it from there.
Chances are it won't be fixable, and he'll end out shelling out for a new one anyway - costing him more.
Totally appreciate that it's taken the shine off the purchase, but once it's fixed you'll love it, and forget all about the aggro!