Well, it depends on what you want to adsorb. If you want to adsorb a rather big molecule (e.g. an organic dye dissolved in water), you need to have pores whose sizes allow this molecule to penetrate the corresponding porosity. Too small will lead to a very poor adsorption because of the unsuitable pore size, but too wide will lead to the same because of the too low surface area. So there exists an optimal pore size for each compound. These comments also apply to microporous materials.
Porosity, specific surface area, charge of the of the entity to be adsorbed as well as the size of the entity are the most important criteria for adsorption mechanism involving any thing from cations to whole organic molecules. If the sample is not very porous and the surface are is low, then you will find that virtually no, or very little adsorption of an entity will take place. Similarly if the size of the entity is higher compared to the pores in the material, then the entity would be struggling to attach itself. Normally higher surface area means higher porosity and therefore these characteristics will enhance adsorption of the entity. However, other operational issues, such as pH, also play a part.