I am trying to do Roughness Measurement for Alumina templates . I am getting streaks when playing around with I-Gain & P-Gain but not a good image of publication quality.
Gain values depends od micrscope model/type and it is experimental value which has to be adjusted during measurement. For DI Nanoscope 3 value is aprox 0,7 to 2 and images were ok, but after upgrade to Nanoscope 8, gain may be even 25 :). As I said, depends on microscope, but on the sample and its preparation, too. I suggest to start measurements in TappingMode, if you do not need LFM measurement.
Very good question, and IMHO generally valid for all AFMs, and for most modes.
First and most important, contact i always prone to damage of surface and/or tip. It is quite possible that you have a very sharp tip until you approach it, or, more likely, until the first few nm of scanning (nm, not microns!). So I agree with Janusz, try tapping or "noncontact", maybe not for the highest resolution, but for testing tip damage.
Then, in case of contact mode, make sure that your tip is harder than the surface. I would guess you might even need a diamond tip because your Si or Si oxide or Si nitride could wear rapidly on alumina.
Finally, to the problem: I would recommend looking for feedback oscillations, appearing in your scan, usually something around 10 Hz to 1 kHz, very sensitive to the I gain. If they are just above the noise, reduce the I gain, until the oscillations vanish. Depending on your AFM, the oscillations can destroy components (amplifiers), but below the noise level they should be harmless. I cannot advise much on the P gain; in some AFMs, it is basically turned off, in others it behaves a bit like the I gain.
Btw, the actual gain values also depend on the cantilever, not only on microscope and sample, so you need to adjust them in each experiment.
About P-gain. It's a tool for advanced users. In some cases it helps to reduce the unwanted feedback loop oscillations caused by too high I-gain, if added in a small amount (