Yep, it might be some mechanism which holds the gear. I've tried again, and now I manage to select 2nd gear, and gear will stay at 2nd. I can select 4th gear also and drive with it, if I hold the gear stick at 4th. But without holding the stick, 4th gear will bounce to N. I will remove the gear box from the car check the selector as you said.

Thank you for help.