Its a tricky one mate as you can't see much and are not exactly on a standard street where you can see them and they are 20' away from the front door.

If you do start to let him out then i would make sure he fully understand the importance of staying in a group and if a stranger come over or rival kids etc he goes to the middle of the green and screams for you.

You in the meantime always have the window/balcony doors open and regularly check out for him. Also make sure he hangs around the green and doesn't wonder off around the flats.

Personally if you get to the point you feel it is unfair to keep him in, maybe let him have an half an hour without you - ok only takes a minute for something bad to happen but you can't keep them under lock and key for ever and 30 minutes is not a lot of time to sit in the house and watch out of the window for.