Hello again,

There is an issue that does not apply to the NDVI, but that is critically important for other vegetation indices (VI). Some VI incorporate numerical constants, typically determined using reflectance data. For example, the Soil Adjusted Vegetation Index includes the value 0.5 as a factor in the denominator of the equation, and the factor 1.5 as a multiplier. These values are scaled assuming that the red and NIR spectral data are measured in reflectance units, scaled 0-1. If DN values (e.g. 0-255) are used instead, the soil adjustment will be totally ineffective. The same applies to various other VI.

Hi

If you do not perform atmospheric correction, then reflectance will be just a linear transformation of DN, and that should not affect the NDVI calculation. However, if you do correct the data for atmospheric effects, that is likely to decrease values in the red and increase values in the NIR, leading to higher NDVI values calculated with reflectance, by comparison with those calculated with DNs.

http://cwcaribbean.aoml.noaa.gov/bilko/module7/lesson3/

For the calculation of vegetation indices, it is better the data to be converted from digital number to reflectance and the steps are simple. convert  DN to radiance and then to reflectance

Hello again,

There is an issue that does not apply to the NDVI, but that is critically important for other vegetation indices (VI). Some VI incorporate numerical constants, typically determined using reflectance data. For example, the Soil Adjusted Vegetation Index includes the value 0.5 as a factor in the denominator of the equation, and the factor 1.5 as a multiplier. These values are scaled assuming that the red and NIR spectral data are measured in reflectance units, scaled 0-1. If DN values (e.g. 0-255) are used instead, the soil adjustment will be totally ineffective. The same applies to various other VI.

Thank you Professor José for your convincing answers. I did not know that the values of Red decreases and NIR increases after performing atmospheric correction. Does it mean that most of the values in Red are due to the effects of atmosphere? which would mean removing them via atmospheric correction would be a good idea as they indicate the radiance coming from the atmosphere and not from the object on the ground. Can you suggest some literature reading on this topic. Your second answer makes it quite clear that other VIs with DN values will not work at all. 

Dr. Jobin thanks for pointing out the DN to reflectance conversion. You are right, it is better to work out NDVI with reflectance values.

Mahesh, thanks for the link. 

Just adding a little bit to the argument:

http://www.digital-geography.com/ndvi-savi-calculation-using-landsat-scenes-dns-vs-reflectance-values/#.VITNFzGsXiw

I have a simple question on the same topic: To calculate NDVI and other VI, is it better to get surface reflectance or top of atmospheric reflectance? (Looks like it is easier to download the surface reflectance imagery form USGS)

Shital, the Landsat Surface Reflectance High Level Data products are a recent addition to the EarthExplorer USGS data access web portal. Of course you should use the Surface Reflectance (SR) compared to Top Of Atmosphere (TOA) Reflectance. For accurate land cover classification, Vegetation Indices etc. it would be better to use the SR. SR compensates for the adverse effects of atmosphere whereas the TOA is with the effects of clouds and atmospheric aerosols, gases etc. TOA is something which you obviously would like to avoid when you are analyzing what's on the earth's surface unless you are studying the cloud or the atmosphere itself. For NDVI and other VIs go for the SR. But do look at the specific information associated with the SR for Landsat as well as the algorithms used for it. It also contains some provisional statements. Also go through the links which I've attached here. It gives a better idea with examples on how the results are affected  if you use TOA compared to SR.

http://fromgistors.blogspot.com/2015/01/landsat-8-surface-reflectance.html

http://www.exelisvis.com/Home/NewsUpdates/TabId/170/ArtMID/735/ArticleID/13592/Digital-Number-Radiance-and-Reflectance.aspx

http://landsat.usgs.gov/CDR_LSR.php

James. I recently attempted to calculate NDVI on some recent L8 surface reflectance data, and the values do not come out in the expected -1 to 1 range (they came out as -227 to 151).  The SR values in the data (for the NIR band 5) are between -131 and 10,853.

 Any insights into why these values are so different, and out of range?  I can go back and get the original DN values and convert them to reflectance myself, but any insights will be appreciated!

Hi Phil, thanks for the question. If you check the Product Guide which comes with the Provisional L8 Surface Reflectance Guide, you can find the Surface Reflectance Specifications on page 15. I have attached the screen shot of the table as well as the complete USGS pdf guide here.

So, according to the specifications, the range of values which you will find for NIR Band will be between -2000 to 16000. But the valid range is from 0 to 10000. Therefore,  what you need to do first is to use the rescaling factor of 0.0001 to convert the SR NIR (Band 5) data values to a range between 0 and 1 by using band math in ENVI or Raster Calculator in ArcGIS or QGIS or using some other image analysis tool to perform calculations on raster pixels. After you have used 0.0001 * (Band 5) you should get values ranging between 0 and 1, which makes sense because normally reflectance values should not exceed above 1 and neither below 0. Similarly with Red Band. Once you have rescaled the pixel values of NIR and Red Band between 0 and 1, you can apply your NDVI model to get the ideal range of -1 to 1.

Hope this explanation helps a bit.

Yes, this is exactly what I've done (a few minutes after posting), but this issue of "the range is........." but the valid range is only ........... is still confusing to me.  If you rescale the actual values in the SR product, you get values from sig. less than 0 to sig more than one.  If you then/also assume that the out of range values are errors, and reassign them (I reassigned them to be 0 if they were neg. and 10000 if they were higher than 10000, and then did the rescaling), then you end up with a reflectance range from exactly 0 to exactly 1 (and then the NDVI also covers the whole possible range, or can do so easily)-----------which doesn't correspond well to any other calculated reflectance values (from earlier Landsat products for example).  do you have any idea why the SR products are giving these 'perfect' reflectance ranges when the DN-radiance-reflectance algorithms we used to use don't ?

Hi Phil, now I am curious too. Although I am in the middle of my research thesis I would like to have a look at the SR L8 scene which you are analyzing. Can you share the Landsat scene ID, the date and location so that I can download the data and have a look. I am interested to know what are these out of range values after you rescale the values. Clouds? Error?

In any case it seems it would be worth doing a comparison study between Landsat 4-7 Surface Reflectance products and Landsat 8 Surface Reflectance product. The USGS website says here http://landsat.usgs.gov/CDR_LSR.php that the 'Landsat 8 Surface Reflectance data are generated from the L8SR algorithm – this is not the same used for Landsat 4-5 and 7 data'. So already the differences begin with two different algorithms. But one would have to dig deeper to know how do the different algorithms affect the SR values. It is strange because even for the Landsat 4-7 SR products USGS provides the same rescaling and specifications table. Have you gone through the blog page by Luca Congedo https://www.researchgate.net/profile/Luca_Congedo   where he has compared L8 SR product, the DOS1 (Chavez, 1996) based SR and the TOA values http://fromgistors.blogspot.com/2015/01/landsat-8-surface-reflectance.html 

It would be also interesting to do a comparison between the SR products and the manually estimated DN-Radiance-Reflectance method. Would you kindly put up this question in Researchgate so that people who have looked at the differences can comment. We can also ask USGS directly for clarification on the discrepancies. 

Thanks for thinking about this further.  I did look at the Congedo blog but didn't see any answer to my dilemma (or just didn't recognize it). If you're still interested, the file name is LC80340322014263LGN00_sr_band5.tif (etc) (from Sept. 20, 2014).  I have ended up just resetting neg. values to 0 or the lowest positive value in the data, and replacing the 'too high' values with a neighborhood mean.  For my 'teaching about NDVI' purposes, this is good enough, though it leads to a not very accurate -1 to 1 maximum posible range of NDVI.    Yes, some time I'll do the 'manual SR calc' compared to the L8 SR product values.............but not now!  Some helpful folks at the USGS did compare their NDVI calculation for a couple of high value pixels in the scene (irrigated corn fields) and compared them to another commercially available NDVI product and found them virtually identical.........but they didn't address the 'out of valid range' issue.  Most of the 'too high' values were clouds as you anticipated, and the too low were either on the very edge (off) the scene, or in bodies of water (as others have documented elsewhere) .........best wishes!

The comments/answer provided by Prof. José M.C. Pereira are reasonable. However, one should correct DN for reflectance for data acquired in anomalous weather conditions because that might induce heterogeneity across the scene.

I have a question that iam working on snowcover area extraction , i wnat to apply NDSI for snow. can i use the same method for NDSI also  as mentioned by mr. James Varghese for NDVI ? please answer

You can use the standard NDSI product if your purpose is to determine snow cover area only for a single time image. However, if you are performing analysis for different time periods, I would suggest you to follow DN-RADIANCE-REFLECTANCE process since you will correct for the time specific atmospheric conditions that are likely to show large variation. Further, these products could be used for other types of analyses as well.

I have a question too. 

I have been using Landsat 5 TM for calculating different VI like NDVI, SRI, EVI. Do I need to change DN values of LT 5 to LT 7 to calculate those indices? Or, can I directly get it done without processing for TOA? 

Please Answer!!

Hi Mann, thanks for posting your question here. Although the previous answers provided by others should be sufficient to get an answer for your own question, nevertheless I would suggest you to try both the methods and see the difference. I think that is the best way you can learn rather than accepting the answers provided by others at face value. Just try it out. Calculate the VI based on the normal DN values and the one based on the SR product which you can directly order free of charge from USGS Earth Explorer(check attached image). In fact I would even suggest you to manually perform the DN to Reflectance conversion and calculate the VI. Finally you can compare all the three outputs, generated from the DN values, manually converted DN to reflectance values and the one derived from the USGS SR product and let us know what you find. As far as I know I haven't read anyone doing that before.

I'm having the same results with the SR data values as some others here, for an area in Central America. I have cloud top data, for example, in band 4 that are with values higher than 10,000 (see attached). So, the workaround with these would be to zero out the clouds, shadow, and water by using the "cfmask" layer. I create masks using that layer such that the unwanted data have 0s and the original land data has a 1, and then I multiply the mask times those scaled SR data and get values no greater than 0.86, which are normal.

I forgot to mention one important point, that Fmask is not perfect, and in fact remnant shadows and clouds run rampant through some imagery, these cause bogus values within the 0-0.1 range for shadows, and 1-1.7 range for clouds. So while my workaround works for the lions share of these problems, it is not completely perfect, and a likely complete fix is to only accept data from 0.1-1.0.

Thanks Peter for your insights on the topic and solutions. Your comments will be helpful for many of us.

Atmospheric corrected image provide higher accuracy as compared to direct DN calculation. 

Yes, my dear friend.

It's incorrect please check the equation of radiometric corrections befor than image prossesing.

Our paper "A survival guide to Landsat preprocessing" may be helpful for some of these questions:

Article A survival guide to Landsat preprocessing

Stephen Chignell , Thank you very much for your valuable contribution, that synthesis is great.

If you do not need to compare or merge data of different areas and/or date of acquisition, then you can live with DN values only. In all other cases you should change the relative DN values to absolute radiometric values, e.g. spectral radiance using the calibration relation for respective radiometer and an atmospheric correction.

@Jan Kolar, your point is well taken. I started this discussion 4 years ago when I was just beginning my Master's degree. This is a basic concept that should be taught in undergraduate or at least in post graduate studies related to Remote Sensing and its Applications. Fast-forward to the present, I kept this page running so that new students and young researchers can benefit from the entire discussion. The 100+ followers and 10+ recommendations to this discussion is proof that this is a common question and a popular topic.

However, I would like to add from my current research on the subject that, apart from calibration and atmospheric corrections, there are other issues that affect Vegetation Indices. Take for example the case of high altitude Cirrus cloud contamination in Landsat 8 scenes. Optically thick clouds are anyway required to be masked before any land/water surface related applications but Cirrus clouds are tricky because they can be invisible in the visible bands but need a 1.38 µm band for detection. Due to the benefit of this band in MODIS, they introduced the same band in Landsat 8. If Landsat 8 images are not corrected for the cirrus effects in addition to atmospheric corrections, the resulting Vegetation Indices such as NDVI would be misleading. Imagine all the research done using previous Landsat sensors that did not have a cirrus detection band!

On DN vs. surface reflectance values for NDVI, values will definitely be affected and cannot be compared across time or sensor if atmospheric correction (surface reflectance) is not applied. Use the Landsat "level 2" products. I just rushed down the wrong path for months before finally figuring out the problem. Good luck. On a separate note...

@James Varghese, oh boy, can you elaborate anymore on the cirrus band available with Landsat 8? This seems to suggest that there is no way to detect cirrus clouds in Landsat 5 or 7 images and, therefore, no way to know if the NDVI calculated from Landsat 5 or 7 is accurate. Is that correct? This seems to have really serious implications for time-series analysis. Thanks a lot.

James Misencik, As far as I know USGS does provide operational products like quality assessment (QA) band for identifying pixels with the presence of clouds in previous Landsat 1-7 (MSS, TM & ETM+) satellite images. But the QA does not mention anything about the presence or absence of cirrus clouds in Landsat 1-7. Gao & Li (2017) state that previous Landsat satellite images are frequently contaminated by cirrus clouds. However, what percentage of Landsat 1-7 satellite images are cirrus contaminated and their quantified effect on derived products such as NDVI are open ended research questions, I think. Anyway, you can refer to the Gao & Li (2017) paper that I've attached here. They describe a process to remove the cirrus effects from Landsat 8.

(https://www.mdpi.com/2072-4292/9/8/834/htm)

James Varghese thanks a lot, good info.

James Varghese thanks for leaving this discussion open for new students (like me) to read it. I was wondering if you have any expertise on Sentinel-2 data. My idea was to make a workflow in jupyter notebooks to assess vegetation (with a simple NDVI subtraction) of the same month in different years. However, I notice that I am getting 1C level data which is TOA but not BOA corrected. For what i understand I do need to atmospherically correct this images to compare them right? why is the darkest pixel not a good approach (I usually read I should do another more complex one using Sen2Cor but I am unsure that I can implement this in jupyter notebooks). If you have any ideas they will be highly appreciated!

Frida Ruiz Mendoza, apologies for the late response. I was occupied with field work.

So regarding your question, I think as USGS provides Level 2 Surface Reflectance (SR) Landsat products, ESA also provides ready-to-use Level-2A Bottom Of Atmosphere (BOA) Sentinel 2 MSI products. Check attached link.

Regarding the question of which approach/method/atmospheric correction algorithm is best for your study, you can follow the RG discussion link and check out the useful article by Majid Nazeer et al. (2014). For more info. on the DOS method you can get in touch with its creator Pat S. Chavez, Jr.

As far as the implementation is concerned, I've never used Jupyter notebooks myself, so I'm not the right person to comment on it. However, I think there should be several open-source alternatives where you can do processing, analysis and display of remote sensing data. In this context you can check out Google Earth Engine (GEE). I would also suggest you to take a look at 'NDVI Anomaly' Maps (Follow the attached NASA EO link) which is a better representation for changes in NDVI rather than a simple subtraction method.

All links are attached below. Hope this helps you in someway.

• https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/product-types/level-2a
• https://www.researchgate.net/post/Which_is_best_Landsat_7_and_8_Atmospheric_Correction_DOS_Dark_Object_Subtraction_Vs_FLAASH_in_ENVI
• https://earthobservatory.nasa.gov/features/MeasuringVegetation/measuring_vegetation_3.php
• https://www.youtube.com/watch?v=KFuihGDHE-s
• https://earthengine.google.com/

Many thanks for Stephen Chignell , its good work!