Mu'taman -

A cross sectional survey can be very important for statistical agencies providing Official Statistics, such as the energy data provided by the US Energy Information Administration (EIA), where i worked, in reports such as the Electric Power Monthly, or the Natural Gas Monthly, where information provided from such surveys is of substantial economic importance.  Other statistical agencies, such as the Bureau of Labor Statistics (US) provide important cross sectional aggregate data of an economic nature. But with regard to the statistical rigor, there is often huge pressure to emphasize timeliness over accuracy. I never agreed with that, as I think it better to provide less information that is useful and accurate, than to try to produce too much on such fast schedules, where results may be misleading due to inaccuracy.  Many agencies produce such results without even publishing estimated standard errors.  There should be much greater emphasis on producing various metadata for all Official Statistics.

For some weekly petroleum surveys at the EIA, I proposed a practical way of producing relative standard errors, and improving estimation as well, but there was not enough interest in statistics - just an emphasis on publishing as much economic information as possible.  This has been plagued with inaccuracy which has drawn criticism from economists trying to use the data.  Finally a team was set up to use methodologies i had developed over many years to consider this problem.  The idea was to show confidence intervals for changes from one week to the next, each week using cross sectional surveys, and trying to straighten out the confusion between cross sectional predictions and the inappropriate time series forecasting information that was partially confused with it.  (Of course nonsampling error, such as measurement error and frame error are very important as well.)  Some great results were obtained, but momentum from uninformed management and others is difficult to overcome. I retired over a year ago, and I don't think the EIA has done much with that study.  Fortunately I was able to implement other production for some other surveys over a number of years, with the help of a few cooperative statisticians, subject matter specialists and managers.  One such 'cooperative' manager, however, told one of the people in my group that statistics was not very important.  (So you can see what i had to work with.)  Also, when I retired, the head of the Office of Energy Statistics, which is responsible for EIA's cross sectional surveys, was someone who reportedly, repeatedly said that the EIA is not a statistical agency. However, President Obama said that it is, and it is listed as one of the largest statistical agencies in the US Government.   The problem in many cases is that many of the economists there seem to have forgotten about statistics and econometrics, or were likely never interested.  

I was also on a team that mentored a graduate class that looked into US  governent agency policies for publishing the accuracy of results in official publications, for a number of agencies, and the results were appalling.  It is commonplace to provide results using a larger number of "significant digits" than could possibly be justified.  Without producing estimated standard errors and other metadata, users are certain to be mislead. 

I interpreted your question as asking about the usefulness of cross sectional surveys for economic data, and the link with "statistical" accuracy for results from those surveys. My experience over the last quarter of a century of my career was directly involved with this for Official Statistics reported by a US government agency, and I can say "Yes, such surveys are an important tool for collecting economic data, and yes, accuracy is important, but arguably, accuracy and measures of accuracy are not always given enough emphasis."  Where I worked, management started putting emphasis on quick turn-around information (or I'd say, too often, possibly misinformation), and in my opinion, seemed to think that competing with 'newspapers,' as one person put it, was more important than the official statistics which is the unique responsibility of that agency to produce.  So ... the ideal may be said to be far from the reality, in my experience.

Jim 

Dear Prof James

I am very grateful your comprehensive view. It sounds very interesting career you have with a variety of challenges. I surprised of the uninformed management in the economics point of view. I would like if can please provide me your paper of the methodology for estimating the relative standard errors. 

Is it accurate to use the beta coefficient to explain the economic significance in the cross-sectional studies? 

 Warm regards, 

Mu'taman Jarrar

I am retired now, so it's easier to speak more freely.    :-)  

Regarding beta, for the classical ratio estimator (CRE), which is sometimes used in survey statistics for continuous data when linear regression through the origin is appropriate, beta is then very important. It may sometimes be considered a measure of growth. 

My ResearchGate profile has many papers in this area if you are interested in considering relative standard errors, based on the estimated variance of the prediction error for estimated totals from finite populations.  This is related to the estimated variance of the prediction error for individual data points, as found in econometrics texts, and in SAS PROC REG, the STDI is the square root of that.  I know that Maddala showed this for econometrics.

To see how RSEs and estimates of beta and the CRE may be used, information is spread throughout many papers on my profile, but maybe a good place to start would be

 https://www.researchgate.net/publication/261474159_Some_Proposed_Optional_Estimators_for_Totals_and_their_Relative_Standard_Errors_for_a_set_of_Weekly_Quasi-Cutoff_Sample_Establishment_Surveys

and the first several pages of

 https://www.researchgate.net/publication/261586154_Using_Prediction-Oriented_Software_for_Survey_Estimation

where the distinction between the estimated variance of the prediction error between individual cases, and estimated totals from a finite population, is made.

Or you might try

 https://www.researchgate.net/publication/261474011_The_Classical_Ratio_Estimator_%28Model-Based%29

Or

 https://www.researchgate.net/publication/263036348_Properties_of_Weighted_Least_Squares_Regression_for_Cutoff_Sampling_in_Establishment_Surveys

Or

 https://www.researchgate.net/publication/262143393_A_Note_on_Estimating_Impact_of_Data_Collection_Mode_Change_for_a_Given_Key_Response_Item

which shows a use for beta you might want to see.

Or something else that might be useful for you, looking at titles and abstracts.

Also, there are reference and bibliography lists in these papers that might help. 

I had to do a lot of work in a hurry with a great deal after hours or on weekends, as papers were not often encouraged, but I think the four featured on my profile might be better written than the average.

Also,

 https://www.researchgate.net/publication/261534907_WEIGHTED_MULTIPLE_REGRESSION_ESTIMATION_FOR_SURVEY_MODEL_SAMPLING

may be of interest.

Cheers! 

Article Some Proposed Optional Estimators for Totals and their Relat...

Article Using Prediction-Oriented Software for Survey Estimation

Article The Classical Ratio Estimator (Model-Based)

Article Properties of Weighted Least Squares Regression for Cutoff S...

Technical Report A Note on Estimating Impact of Data Collection Mode Change f...

Conference Paper WEIGHTED MULTIPLE REGRESSION ESTIMATION FOR SURVEY MODEL SAMPLING

Note that in survey statistics for finite populations, especially for household surveys, the usual approach is through probability/design-based sampling.  That has problems, however that can be aided by regression modeling.  You could consider the following:

Särndal, CE., Swensson, B. and Wretman, J. (1992), Model Assisted Survey Sampling, Springer-Verlang. 

Brewer, KRW (2002), Combined survey sampling inference: Weighing Basu's elephants, Arnold: London and Oxford University Press.  

Further, in

Cochran, W.G(1977), Sampling Techniques, 3rd ed., John Wiley & Sons,

the CRE discussed there uses the design-based approach, as usual, but he has a few pages (158-160) on the model-based (regression) approach.

In chapter 4 (I think it's chapter 4) of

Lohr, S.L(2010), Sampling: Design and Analysis, 2nd ed., Brooks/Cole,

she does a good job of explaining the difference between the usual design-based CRE, and the model-based version I used a great deal, which is more compatible with econometrics.

I just thought I'd let you know about the difference.

Thank you prof. James... alot of different percpective.. I should read first, then I will tell you  any further progress or questions...

My concern is how to show economic significance from the regression equation, espically when the variables is subjective. Then highlight the effectiveness of the model in USD!

I should read first.. may the answer already highlighted in your papers.. 

Warm regards

Perhaps you are thinking of econometrics models that use statistical data from statistical agencies, or some other 'found' data?  Econometrics models are more complex, though no model should be more complex than necessary.  You should consider both the accuracy and the "representativeness" of any input data. The model should be based on your economic theory, and tested using test data wherever possible, and also using 'real time' measures of accuracy that apply. 

When you say economic vs statistical "significance," do you mean the impact/importance of the economics involved, which may or may not be well supported by 'statistical accuracy'?  Well, it is true that knowing something very accurately that has low impact on the economy is a different problem than having inaccurate information on something important. In the former case, why bother?  In the latter case, if you get misleading results, it's even worse.  Only by having reasonably accurate results for something that is meaningful, are we making progress. The meaningfulness of a study may be a more qualitative concern, and the accuracy may be more quantitative.

If you mean that for the application you have in mind, that it is not clear what are the important variables/regressors to be used, and/or the 'best' model format to use, I'd think that, within the confines of subject matter theory, you should experiment. But remember that you should not overfit a model based on a given data set, and you may rethink your model whenever you have new evidence.

Statistical agencies providing Official Statistics can have a huge variety of users who need a huge variety of data. Economic data that are considered important by one set of users may not be considered important by another.  It is the function of economists to theorize what it is that they want, and the statistical agency should provide those data as timely and accurately as possible.  These data may be the end result, or more likely the econometrician may want them as input data.

For the economists who use these resulting data in econometric models, they are expecting the input data to be fairly accurate. If they use aggregate data at some level, which already has both sampling and nonsampling error, they need to be aware of that. If individual response data from the statistical agency is used in the econometric model, it may have substantial nonsampling error.

Results from the econometric models need to be tested for accuracy as well. Here, just as in model-based estimation, you may also estimate for the variance of the prediction errors,  and use test data to examine your econometric model. 

Note that there has been work done by Wayne Fuller and others to more directly account for measurement error in estimating regression coefficients. 

Perhaps a book like this may be of help:

Maddala, G.S(2001), Introduction to Econometrics, 3rd ed., Wiley.

(There is a fourth edition with Lahiri.) 

I hope that at least some of this relates to what you had in mind.

Best wishes - Jim