Rudey, The physical location of the server doesn't have anything to do with it tbh, network traffic moves at the speed of light, so france or london makes next to no difference.
Where there is a difference is that the data you send/receive is handed over from one network to the other by routers and switches, and that's where time may be lost.
In truth, a connection to your neighbour could be slower than one to a server in the states (all connection speeds being equal).

What is bizarre stu, is that most of those reporting slow speeds say that other sites are very fast, which indeed suggests that PF is the slow link in the chain, while others say it's running perfectly fine at the same time.
This would suggest there is a routing problem from an ISP to the PF server.

It would be useful to know which ISPs people are using when reporting slow connections, and possibly their (people) geographical location.

Running this command (in a command line window - Start / run / cmd) from a PC which experiences a slow connection might yield useful information:

tracert -d www.passionford.com

doing the same again with a site that appears faster (e.g. tracert -d www.microsoft.com) may highlight significant differences.

Here's what I get from work (some IP Address hidden):
Tracing route to ns102.nuxit.net [85.12.135.102]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms xxx.xxx.xxx.xxx
2 1 ms 1 ms 1 ms xxx.xxx.xxx.xxx
3 2 ms 2 ms 2 ms 10.110.3.93
4 22 ms 23 ms 24 ms 10.50.0.1
5 17 ms 17 ms 17 ms 10.50.0.217
6 24 ms 19 ms 11 ms 10.54.13.7
7 17 ms 16 ms 16 ms 195.66.226.203
8 25 ms 25 ms 25 ms 193.19.216.153
9 26 ms 26 ms 33 ms 193.19.219.46
10 25 ms 26 ms 25 ms 193.19.216.146
11 78 ms 35 ms 21 ms 193.19.218.6
12 26 ms 25 ms 25 ms 85.12.135.102

This shows how many hops and how long it took to resolve each from my workplace to the PF server.

Phew - Hope this help.