Recommend that you update your question with the identity of your nanoparticles and matrix. Then we could provide more specific advice on compatibilizing the interfaces or links to appropriate articles.
It sounds like the particles are agglomerating during slow drying. Can you cast from a fast-drying solvent like Methanol, and use a fast drying process like spin-coating?