small bolts at the front, i think they are black ones and the ones in the rear are blue

if thats the case then it sounds like you may have striped the threads in the ali casing
to fix it properly you would really need to get a helicoil to go in there, assuming you have the space to slot it in
otherwise box out and see what happens

i wouldn't recommend leaving it with just 3 bolts to hold it as this makes for a bit of a shaky gear change

what does the bolt look like when you remove it? has it got slivers of thread on it?