First of all, check somehow all papers written by Foody and Stehman. These will answear all your questions.

Generally, there is no "perfect" way. It depends on your application, classifier, study area, budget, limitations, and many more ...

First of all, check somehow all papers written by Foody and Stehman. These will answear all your questions.

Generally, there is no "perfect" way. It depends on your application, classifier, study area, budget, limitations, and many more ...

Hi Paul,

in order to have design, I prefer: (1) to chose the same number of training points per class; (2) use a minimum distance between training points (to avoid pseudo-replication and to increase generality of the training data); (3) to select the center of larger patches that are likely not mixed classes; (4) select a minimum of 30 (better 50) points per class. Point 2 cannot easily be generalized, but rather depends on the spatial structure and gradients of your study area and on the number of classes you differentiate. Point 4 is not always easy to obtain. Alternative to using the same number of training points per class, you can also chose a spatially random sample. You will now have more training points in frequent classes. This will usually give more accuracy and weight to frequent classes at the cost of rare classes (that may go undetected with such an approach). If you chose the same number of sample points per class, then you give equal weight to each class, possibly at the cost of having a lower overall mapping accuracy. If you finally decide on all 4 points for a sampling scheme, you may not be able to get the same number of points in rare as in frequent classes, because the patches of rare class are often in close distance to each other. I would then still keep up with minimum distance, and sacrifice the numbers, thus accepting smaller numbers in very rare classes.

Nick

Perhaps give us some more information:

- Hard or soft classification?

- Which classes?

- Which classifier? (Very important!!!)

- Where do you get reference data? Field work? Existing data? Airborne imagery?

This will help answearing your question.

As any sampling you'll get better results with large samples representing your population. So, make polygons just where you don't have any doubt of which class these pixels are inside. People tend to sample just "inner" space inside a "object", that behavior will create small variance and "borders" will suffer from poor classification.

My explanation is valid for max likelihood method, important to remember these methods use Normal Distribution for classification. You'll need a representative sample for optimal results. E.g:

Suppose you have two classes (A, B) and you're on the step of classification, A and B now have mean and standard deviation estimated from training set. Classification tools will "ask": This pixel has more likelihood to be from A or B? Winning likelihood should be used for classification of current pixel.

I hope you can understand without need of graphics but I can provide if needed. You can look for Bayesian classification or Bayesian spam filtering. Seems similar but with another statistical school.

A spatially contrained latin hypercube sample

Try to check the image first and then see the heterogeneous patches in the landscape.

Then select these heterogeneous patches and try to verify with some reference data

chose the same number of training points per each class representing whole landscape. The sample should cover at least 15% of land cape representing all classes.

The required number of training samples to train classifiers depends on the what classifier you going to use. For example, In maximum likelihood classification, number of training samples should be greater no of dimension of the data.

In hyperspectral image, some times it may not be possible so reduce the dimension of image by dimensionality reduction methods.

Other major concern in remote sensing images is mixed pixels. Mostly of them it is better to pick training samples from where doen't so much of mixed pixels. But It is always better to have atleast 10% of mixed pixels in your training samples, so classifiers generalization capability is better and minimize misclassification in the whole image.

I agree with Cornelius Senf , to go through the papers writted by Dr. Foody

One thing regarding classifiers: If you use a probability based classifier (i.e. ML) your sample should represent mostly pure pixel, and the classifier will generalize the class borders based on the probability distribution. If you use machine-learning techniques (i.e. SVM or RF) your sample should contain many mixed pixels as well to take into account every eventual case. This is speciallly important for SVM, but less important for RF.

Regarding the sample size: There is no standard like 50 per class. If you use a probability classifier, use a multi-nomial distribution to estimate the sample size. If you use a machine-learning classifier, your sample size needs to be much smaller. There are many studies testing the effects of sample size on machine-learning classifiers (i.e. Waske et al. 2010). Generally a balanced sample is preferable. However, if you have field data it is not worth deleting information by selecting a stratified sample from your reference data. In this case, ensemble techniques such as RF oder Bagged SVM (you could even do a bagged ML) should be preferred, because you can stratify each sample drawn for each classifier in the ensemble.

Besides the ways suggested, here is maybe another way to think about it. Usually you'll have to make an accuracy assessment after the classification, so the question is to whether you can create a dataset that can be used for both, training and validation?

One way to do this, is to run an unsupervised classification algorithm (such as ISODATA) and group the classes into your classes of interest (for example if you want to classify an image into 'forest' and 'non-forest' you would assign all ISODATA classes to either 'forest' or 'non-forest'. Into the resulting areas you then randomly sample points (if you wish using a minimum distance) and label each point individually into your class of interest.

The advantage of this method is that after the labeling you can split the dataset into 'training' and 'validation', because they were all gathered independently and by using a random sample you also likely will be able to represent the entire spectral variability of your class of interest. Feed the training data into your classification algorithm, create your SVM-model and use the validation data for an accuracy assessment. Do this 10x and select the SVM-model with the highest accuracy - this will give you a good estimate of the classification accuracy. Then classify your data.

Selecting a training dataset strictly depends on your goals and biogeographical attributes. For example, my experience is focused on Amazon deforestation: the goal is to point out trends of clearings in very large surfaces of forest. So, in this case, the sampling is not selected in systematic way (grid) or randomly, but we try to define a priori the localization of the sample points in order to increase, in cost effective way, the accuracy in the classification of target classes, and avoid redundant data about the same class (e.g., a lot of samples in forest covered areas).

I´d like to suggest you to try – if you already know your target classes- a Linear Spectral Mixture Analysis that can help you to mitigate the mixed pixel problem and increase the accuracy of your classification (See PRODES project at INPE).

All the best. Marco

Well you can take minimum 30n and maximum 70n pixels for a class ( where n is number of bands in the class) and it should be uni-model in the frequency curve.

Whow, thank you all for this very active discussion! Obviously the sampling strategy for selecting reference data is a hot topic. From all the answers I have two more questions:

1. ) Several comments mentioned that the strategy should be adapted to the classifier used later. E.g Cornelius stated that for probability based (parametric) classifiers no mixed pixels should be in the training data whereas for machine learning approaches (non-parametric) it is important to include mixed pixels. Do you know of any publications where the later case is analysed in detail?

2.) Most of the comments provided guidance to the number of trainings instances to select and the class distribution. So far there are no suggestions on how to spatially locate the sampling points / polygons and what to select at each site e.g. the mean, median, sum etc or all pixels values within the a polygon (shape? size?) ?

Supervised classification analysis involves i) generation of False Colour Composite (FCC) of remote sensing data (bands – green, red and NIR). This helps in locating heterogeneous patches in the landscape ii) selection of training polygons (these correspond to heterogeneous patches in FCC) covering 15% of the study area and uniformly distributed over the entire study area, iii) loading these training polygons co-ordinates into pre-calibrated GPS, iv) collection of the corresponding attribute data (land use types) for these polygons from the field. GPS helps in locating respective training polygons in the field, v) supplementing this information with Google Earth (http://googleearth.com) and Bhuvan (http://bhuvan.nrsc.gov.in), vi) 60% of the training data is used for classification, while the balance is used for validation or accuracy assessment.

Cornelius Senf you said that sampling for machine learning should integrate mixed pixels and for probability classifiers it should be pure classes. Do you by any chance have literature advice about this? I searched for some time now, but could not find literature stating this. I am very interested in the sampling technique though. The only hint i found was for SVM only...

Or is this rather an experience you made?

Best regards,

Lisa