One problem might be that there should be a one way valve on the fuel pump outlet (the bit that screws on), its job is to close when the pressure stops (fuel pump turns off) and keep the pressure in the lines.

Check with bridging the fuel pump first then press the flap down a couple of times to clear any air. Then try and start the car.

Usually this can also be a sign of a failing fuel pump too.

Best off getting this changed for a working one. I had a similar problem. New pump and valve and off we go!

Dodgy starting may also be air getting into the system.