I've noticed this since bus lanes first started.

The problem is there are too many time variations, 24hr, 07.30 - 0930, 16.00 - 18.00 etc.
And it doesn't help when the lane changes from part time to fulltime along the same road.

Most people probably can't be bothered to check the sign, check the time and make a move into the lane.
Bus lane cameras have made it even worse for people to make the move.

It's easier just to follow the car in front.