Sometimes mixing the exact proportion of everything may not arrive at the exact pH. I always make a more concentrated sample as a stock and adjust the pH. When I need it, I will dilute it with water and again adjust the pH.
In your scenario, adjusting pH with NaOH is common practice.
The pH of a buffer will shift towards neutral as it becomes very dilute, but I think this applies to much lower concentrations than 25 mM.
The pH of concentrated stock solutions of buffers can shift when they are diluted to working concentrations because the activity of the ions is higher in the dilute solutions than in the stock, resulting in a shift of the apparent pKa. Here is some more detailed information on that subject:
There are tables for the preparation of phosphate buffers calibrated to produce buffer solutions at 0.1 M starting with 1M stock solutions.
Another way to go about preparing a phosphate buffer at a specific pH and concentration is to mix the monobasic and dibasic phosphate solutions at that concentration while monitoring the pH continuously until the correct pH is reached.