It is possible to do yes, often resulting in the bending of the guide washer that sits between the belt and the pulley to keep the belt in line, imo damage this at your peril.
Personally I think it is a bodge that could have disastrous consequences, but each to their own, it can be done, I wouldn't.
tabetha