I have read some papers about landslide susceptibility mapping. We can obtain the LSI based on many statistical models. Then, we can use the ROC curve to assess the quality of the reliability of the model. For example, using the frequency ratio model we may obtain the LSI range from 5.8(low susceptibility) to 67.9(high susceptibility). Then, we should have the cut point to divide each pixel into landslide(1) and non-landslide(0). I do not have idea to determine such a cut point.
I found the result from a research paper, but I could not understand how the author do.
The following are the original text:
"The success rate was calculated as follows:
First, the LSI values were calculated from LSI maps in ascending power for the cumulative total and cumulative total in descending power.
Next, each calculated cumulative value was cross-applied to analyze the cross point. The cross point is the average point with the highest R-square for the LSI value.
The areas with values larger than the LSI value of the cross point were set to the areas with high landslide susceptibility (event: 1) and the areas with low landslide susceptibility (event: 0).
As a result, the LSI maps were classified into values of 0 and 1 according to the crossing point."
Park S, Choi C, Kim B, et al. Landslide susceptibility mapping using frequency ratio, analytic hierarchy process, logistic regression, and artificial neural network methods at the Inje area, Korea[J]. Environmental earth sciences, 2013, 68(5): 1443-1464.
I do not understand what is "cross-applied"? And where is R square coming from?
Hope anyone to give some advice.
Thank you very much!
Dear colleague,
I also can't understand properly what the authors of the cited paper describe...
The "cut point" should be the threshold of a LSI that separates best
the 0 (no landslides) pixels from the 1 (landslides) objects (e.g. pixels).
Taking the lowest observed LSI for mapped landslides, however,
is probably not a good choice, because it may come at the expense
of getting a LARGE number of predicted landslides (=> false positives).
Then, you could go for the LSI that covers, say, 95% of observed landslides
(i.e. the 5% percentile of the LSI values on observed landslide pixels).
In any case, this might imply a large number of false positives;
and: the lower the cut point is, the larger the number of false negatives will become.
So you need a critical LSI value as cut point that minimises the "overall error".
If you look at an ROC curve, you have the LSI values sorted in descending order on the X axis; on the y axis, you get the percentage of mapped landslide pixels on the in ascending order.
In practise, you get the data for the ROC curve from a table exported from your GIS that contains, for each raster cell, the information landslide/no landslide [0;1] and the computed LSI (range of LSI value depends on the method you used). If you sort this table by the LSI (in descending order) and compute the cumulative percentage of mapped landslides in a new column of the table, you can plot the LSI on the x axis and the cumulative percentage of mapped landslides on the y axis.
The "ideal" ROC curve will sharply and steeply go up, reaching very high percentages of mapped landslides for a small part of the highest LSI values. The better the model, the "higher in the upper left corner" the ROC will be, and the larger will the "area under the (ROC) curve" be, as a single measure of model performance. Ultimately, any model (its ROC curve) will cover 100% of the mapped landslides, but probably at a very low LSI value (which is not suitable as cut-point because it generates lots of false positives).
In my opinion, you could find the optimum cut point by looking at the ROC and finding out where it "turns over" to the right, i.e. where you have to lower the cut point by a large amount to "catch" only a few more mapped landslide pixels.
I can recommend two papers that deal with the validation of LS models and that explain the procedures in a reproducible way I think:
Chung CF, Fabbri A. 2003. Validation of Spatial Prediction Models for Landslide Hazard Mapping. Natural Hazards 30: 451–472.
These authors try different schemes for separating training and validation data, and they suggest a "ratio of effectiveness" that explains how much the LSI cut point has to be lowered in order to "catch"/explain k percent more mapped landslides in the validation dataset.
and
Frattini P, Crosta G, Carrara A. 2010. Techniques for evaluating the performance of landslide susceptibility models. Engineering Geology 111(1-4): 62–72.
They provide a very good overview and explain two possibilities to find cutoff values: ROC-based and cost-based approaches (while "cost" is associated with the total amount of misclassification, i.e. false positives and false negatives).
Let me know if you can't access these papers.
Hope this helps.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers