I want to imply trend data analysis on these data( Mann-Kendall Test). So can i use all the stations regardless the % missing data? or should i eliminate some of them.
Another questions for experts in Mann-Kendall test.. how can i define the variables of the equations to finally calculate Z(standard normal test statistic) and then determine the rend of the data?
All methods I know of for statistically coping with missing data (imputation, etc.) assume that the missingness is random. Ideally it's missing with respect to all variables, but it must at least be missing with respect to the other variables of interest. In your case, if there is any reason to think the missing years differ nonrandomly from the nonmissing years (rain gauge overflowed, road was impassable due to mud...) then data from those stations may not be usable.
I have similar problem with missing data for various climate stations in the Pacific Northwest in America (over 100 years).
My approach is to
a) use all of the available data to produce a result
b) elminate the missing data to see if that changes the result significantly
c) try to interporate the missing data if there are other nearby stations and see how that result compares
d) build a selection function for missing data
so do it more than one way. I will say that 10 years is not a sufficient baseline to establish " normal" precip patterns.
Dear Abeer,
As mentionned in the previous anwer, try different solutions and make tests to see the robustness of your results. You can also use optimal interpolation (a not too bad solution) if your met stations are not too far from each other. But be cautious! Another solution would be to use satellite observations (ex TRMM) after a robust comparison with non-missing data and if your observation period correspond to the availability of TRMM data.
Concerning the second part of your question, you have to compute yearly mean anomalies before computing your tend with a Mann-Kendall test; in any case use de-seasonalized data. Moreover, when calculating your statistics, take into account ties for correcting the variance.
Best regards,
G.C.
Dear Dr. G. Caniaux
how can i specify ties in my precipitation data? i am new in trend analysis.
Dear Abeer,
Ties are repeated values in your series.
In the Mann-Kendall trend analysis you have to follow the steps:
First you rank your values by ascending order. Then compute the Mann-Kendall's sum (the sum of the sign between elements of the series) and then take into account the ties for correcting the variance of the ties (the variance without correction is simply: n(n-1)(2n+5)/18 ). From this you compute the Mann-Kentdall statistics (depending on large >10 or small
All methods I know of for statistically coping with missing data (imputation, etc.) assume that the missingness is random. Ideally it's missing with respect to all variables, but it must at least be missing with respect to the other variables of interest. In your case, if there is any reason to think the missing years differ nonrandomly from the nonmissing years (rain gauge overflowed, road was impassable due to mud...) then data from those stations may not be usable.
Maia is correct, but I would add that if data from stations with data missing not at random (MNAR) is not included that will bias the results also. Multiple imputation or full information maximum likelihood imputation is a better approach regardless. For MNAR one needs to model the likely mechanism producing missingness and hence bias - which is non-trivial.
Have you considered Metropolis-Hastings algorithm to generate the missing data?
Milind - I don't think that is sufficient. The difficulty is probably in setting up a model that accounts for MNAR data. One option would be to use Bayesian methods to add information about the missing data as priors. This kind of model would then be estimated using MCMC methods (of which Metropolis-Hastings is one kind). Without the priors it would, I think, be equivalent to full information maximum likelihood approaches.
One note about randomness.
In general, precipitation is NOT random, and that creates special problems when trying to correct for missing values - this is why adjacent station interpolation is needed, okay if your looking at large scale patterns, not okay, if your interested in small scale patterns.
I would think any complete case analysis would be a great loss of information. Imputation methods as other suggested would be much better. You could build a nice spatial-temporal model for rainfall (but rainfall usually has nasty distributional properties - Tweedie models have been successful for some) . As G.B. alluded to. The model will give you not just point predictions but uncertainty which you could include in the next analysis stage.
The severity of the problem is location dependent. Is the rain rather convective (rather local-scale) or is it large-scale advective? In any case, you should compare stations with missing data to other stations around. It makes also a difference whether you look on (sub-)daily or monthly to seasonal sums. So could you explain a bit more what you are looking at and which region?
- Badges
- Science topic