It will affect if pH affects the drug, the particle or both. 

Drugs have a organic functional groups like( COOH, OH, NH2) Nano particles are made of metal/metal oxide. The coordination bond to form metal and and these functional groups requires particular pH. (Kindly read theory of EDTA complexes with Co and Zn using xylenol orange and Eriochrome black T, respectively or Mg and Al ion with oxine). Simply at higher pH many of metal ions are in insoluble hydroxide form thus do not form bond with functional group of drugs. Also at very low pH the functional groups are protonated thus difficult to give pair of electron to form coordinate bond.

Yes, pH will influence the intermolecular force which plays an important role in  loading of drug into the nanoparticles.