Yes, it will restrict boost pressure and prevent the turbo from spooling up quickly.

2.5" should be considered the very minimum size for that engine...You have to remember that in terms of cross sectional area a 2" exhaust isn't 20% smaller than a 2.5" exhaust as you might think, it's about 37% smaller (4.9" squared vs 3.1" squared)...That's quite a big restriction.