2 with him.
1 walks for a day, then drops off 2 days water and turns round.
the second pics up one days water and so does the pilgrim, so they both have 4 again.
then the second one turns round after another day, leaving 1 days water for the pilgrim, who then has days water and is 4 days away.
Thats least number of people i can come up with anyway.