Perhaps you have switched around the left and right mechanism? Or the arm needs to be at the rear of the motor instead of the front. But I am quite sure those 4 holes are the ones to use.
The manual mechanism is mounted on the (other) 4 holes on the flat rectangle, with the big hole in the middle for the winder.