AWS ?

AWS=Automated(or automatic) Weather Stations

Isabella

One simple way to start is to do a scatter plot comparing the two measurements where

both types of data are available; then draw some preliminary conclusions how to proceed.

If necessary apply neural learning approaches.

Toda Raba Pinhas!

you can first calculate the correlation between the two measurements and see if there is good correlation between the two time series (R> 0.6 at least). if yes , then calculate the mean bias between them by considering the manually obtained data as reference , then use this bais value to adjust the AWS data with respect to the reference one.

Hi Abdoulwahab, the problem is those series are not parallel. We have the digitazed daily maximum winds upto say 1990, and since then from the automatic stations.

Hi Everybody, I'm thankful to everyone who contributed to this discussion. But after my burdening research on the issue, parallel to this discussion, I'll likely go with the Kruger et al. 2013, "Strong winds in South Africa: Part 2" (J. S.African Inst. Civ. Eng. 55(2)): "The wind data could be divided into two distinct periods – the period of the Dines anemograph and the AWS anemometers – between which significant discrepancies were evident. The exposure and quality of data from the Dines anemographs are also suspect, with the consequence that only AWS data was utilised in the analysis."

Good morning,

One option is to  try to look into quantile-quantile plots for wind speeds from the two datasets. This should show if the both datasets are coming from same statistical distribution (they should, as locations are close) and do they need any adjustment. If the datasets are from same distribution, but adjustment is needed, it should be possible to derive simple regression coefficients and apply to one of the datasets, thus receiving a homogeneous time series. One issue here could be, what dataset to consider 'ground truth' and what dataset to adjust, I guess AWS might be more reliable.

Hopefully adjustment won't be needed at all. If the data was collected at same location, on same heights, with same averaging periods and with calibrated instruments, there shouldn't be systematic error in the data, just random errors associated with observations. 

Hope this helps

Thank you David, I've considered the quantiles or range frequencies or even spectra options, and still do not reject such comparisons. However, the problem is all of them speak in too general mode, while for my goals (Building Code wind speeds) each particular gust = e.g. 35 m/s, when being proved as reliable or not, may influence the 50-y etc. wind speed (needed for Building Code). In another words, there is a huge return period difference between the "true" wind speed being a bit lower (e.g. 34 m/s) or a bit higher (e.g. 36 m/s) than measured 35 m/s. And here is a point where my  'ground truth' is, as you say. There are the papers on comparison between Munro and Dines anemographs, both have their correction factors different from those of the AWS data, and because the global trend is to continue with AWS only, I'll apparently go rather with shorter "AWS only" data series (while using methods for multiplying maxima depicted from them) then with building a long Munro+Dines+AWS series.

Before attempting any comparison and combining both types of  data  records,  it is esse ntial to bring both the data records to a  common mode. It  is suggestive to look into the details of sampling rate of the base data in both types of data  records.  Manual mode of acquisition is generally  2-minute  average known as quasi-steady state wind.  AWS  data sampling is either  instantaneous data at 120th second ( 1 record per two minutes)  or may be average over  120 seconds duration.  

 Checking the above basic details of base data, bring down both the data records to a common mode like 2-minute average and then compare both types of data for verifying the differences with a simple time based  plot of U & V and further  statistical analysis. 

Since your wind measurements are non-overlapping in time, from different instruments and even at different locations (even if "close"), I would not even try to construct a "homogeneous" time-series over the full time-period without lots of "health warnings" about the possible discrepancies at the very least.  After various statistical tests (as suggested by David Darbinyan) and checking on basic details (as described by Appa Rao), it may even turn out that both types of observation may be considered "ground truth" to a "good enough" level of accuracy, and allow you to combine them into a "homogeneous" series in a very simple direct way.

My own suggestion would be to break all your data into 10-20 year sections (i.e., the length of your shortest AWS dataset), and plot "number of events exceeding the x-axis value" on the y-axis as a function of "gust-speed" (or similar independent variable) on the x-axis.  All curves should start with large y-values for small x-values, drop steeply as x is increased, then level off as x-values get larger. 

You can extrapolate those kind of "histograms" to estimate the return period for extreme events of any given size.  That way you can estimate the size of the "30-year storm" from 20 or even 10 years of data.

If such histograms for your different datasets over different time-periods are very similar, you can have some confidence that (a) the "climate" hasn't changed and (b) both manual and AWS measurements are equally reliable.  Alternatively, the climate has changed but the manual and AWS measurements differ in just the right way to mask that change - possible, but unlikely.

If such histograms are clearly different, you can simply take the more conservative one (i.e., the one which predicts the shortest return period for strong events) as a basis for setting your building codes.

As with all data, the objective analyses need to be tempered and interpreted with human intuition and common sense!

Thanks, Isabella. In this case, it does seem more relevant to use the shorter, but true observed AWS dataset, rather than longer time-series of with artificially corrected values, especially the extremes.

Suggestion by O'Brien can be adopted, particularly to check the return period.  Considering return period and  time constant  will be helpful in estimating the gust period. 

Thank you Prof. Appa Rao, indeed you have underlined a very important issue, especially for the high quantiles - a steady state assumption resolution. Together with a response time issue thoroughly investigated in the e.g. Australian Extreme Wind Project, these two are needed  to be checked versus reliability of the measurements in a whole spectrum of the strong winds, from short-term gust fronts to well-developed storms.  Thank you.

Dear Dr. Enda William O'Brien, thank you for your interesting and very common-sensible advice. I'll likely try it. Best regards, Isabella.

In addition to the great advice given by Enda (hello Enda!) you could also use standardization methods to convert the measurements to a standard terrain for comparison purposes.  To do that you would need longer term averages (at least 10 min) and would need to have an estimate of the upstream fetch roughness for each site as a function of wind direction (using G earth these can be qualitatively estimated or you can use gust factors from a long data set).  Assuming neutral stability (a good estimate in strong winds) you could then use some conversion methods to go from one roughness to another.  One method is to  use a profile eqn. and your roughness estimate to go to the top of the PBL, and then go back down using the standard roughness you want to use (e.g. open terrain).  Vickery et al have a paper discussing this that was in J. Climate and Applied Meteorology.  Good Luck!