Yes.
This is used quite often in racing, its known as a blow off valve and is simply a valve designed to open at a set pressure. Alternatively, you need some electronic way of cutting the ignition, or warning you, such as my Dynojet AFR meter.