If they had a NON return valve, the NON return of the fuel would mean the pressure had to be there.
It's a bit accedemic when engines are using running pumps anyway, although it has an initial prime, the pump also engages upon rotation of the engine.
tabetha