Basically its a question of energetics. The magnitude of the repulsion between particles, be it steric or electrical double layer is multiplied by the particle radius, if the particles are small the repulsion is reduced, such that the thermal energy kT becomes significant so that eventually the repulsion between the particles becomes of the same order as kT, when this happens the particles will aggregate (agglomerate) together. Larger particles with say the same zeta potential will not aggregate as the repulsion in terms of kT is larger.
To follow up with Dr. Luckham's answer. if electrostatic approaches with the repulsion due to the overlap of interacting double layers is used for dispersion, there is only a finite interparticle energy barrier produced. Particles, nanoscale or macroscale, will undergo slow agglomeration that eventually begins to dominate the physical condition of the suspensions (e.g., sedimentation, increasing opacity, etc.). However, for nanoparticles suspesnions on the order of 10 to 100 nm, the nanoparticles can be thermodynamically stabilized by small (e.g., citrate at ~0.7nm diameter for calcium phosphosilicate NPs at ~60 nm). The size of the citrate molecule is large enough to produce an electrosteric layer. Our metric for thermodynamic colloidal stability is that discussed in Napper's book, Polymeric Stabilization of Colloidal Dispersions and in our 2011 Nanoscale paper (attached). We found that the electrosteric steric effect scales with size with even a small charged ion such as citrate imparting the combination of steric effects with electrostatic repulsion effects for thermodynamic stability. Napper discussed a relatively simple method to deduce thermodynamic colloidal stability; dry the suspension and reintroduce solvent back into the suspension. If the particles spontaneously disperse, one has achieved thermodynamic colloidal stability for that solvent with those formulated particles.
I feel that from the basics of microscopic thermodynamics, the Vander Waals force of attraction is weak for large molecules and strong foe very tiny molecules. So such tiny particles tends to cluster or agglomerate. An electrical double layer, pH also has an effect on agglomeration. Use of surfactants can help in preventing the agglomeration. I have shared what I had learnt and understood. Any change is mostly welcome to help me learn.
I would like to answer this question by a very simple way: nanosized materials, either metals, oxides, hydroxides or any other types, have high surface area. They have much more atoms in the surface compared to their bulk counterparts. The atoms in the surface of nanoparticles have free valence which make them more active for adsorption of various species or interacting with each other. The smaller the particles the more number of surface atoms and, in turn, the more reactive. That’s why nanoparticles have to be stabilized to avoid their aggregation. Let me give another simple explanation. Say, if you have a cube of 1 cm3 volume and you crash it to make millions of tiny cubes from it. In doing so, a lot of energy is spent to break the bonds between the atoms or the molecules. The produced tiny particles tend to attract each other( agglomerate) and by forming chemical bonds or any other interaction to release energy