I have come across many situations that only the p-value was the determinant of the significance in econometrics. I wonder if there are any other methods for weighting the significance issues of the model and the variable.
When reporting pValue we do not report the probability % of the observed (F(X)), but the un-observed error level. Is there an alternative? If you do not want to report 1 - F(X) < 5%, then report F(X) > 95%.
Is there other indicator, such as effect size, than we do not have to deal with % probability? No. Even using the effect size, we still has to end up with reporting % significance level.
Whatever model you present or proposed, it is an estimate. The % probability tells us "what is the probable result under that model?" No matter how sensible the model may appear, if it could only explain small % of the data, or the likelihood that what is claim to occur is less than what is expected (95%), than the model fails.
Is there an alternative? Should there be one? We do not fix something that is not broken. We start looking for alternative when the existing tool does not work or that an alternative could offer something better. In this case, we are walking in a unit circle from point 0 to 1. Speaking in % probability is the best language in this case. There are only two options: F(X) is the % observed compare this to the specified % confidence interval or 1 - F(X) is the % error compare this to the specified error level.
It depends on the model! In general, aside from the p-value, people mostly use the Wald test or sometimes the likelihood ratio test for the overall model and the t-test for individual parameter estimates.
First you must have an economic question or theory that you wish to work on. You then formulate this as a model or equation in mathematical/statistical terms. You must also have a data set that provides measurements of the variables in your data set
You then use the data set to carry out specification tests on the model. In this way you can establish that the model and data are consistent. For this you test for heteroskedasticity, no serial correlation, stability of the model, functional form, missing variables etc. If you pass these tests you can say that the model and the data are consistent. I am not sure what you mean by saying that a model is significant. I think that the idea of data consistency or congruence is important at this stage. You should adopt the most general form of the model at this stage as modelling from general to specific is better than generalising a more specific model. You can not establish the truth of your model using econometrics. Your data set may be consistent with several different models and your inferences are made on the assumption that the model estimated is a good representation of the underlying true situation.
When you have a data consistent model you can look at the estimates of the estimated coefficients and other parameters. Significance can have two meanings at this stage. An estimated coefficient may be statistically significant but so small as to be of little importance in the model. You can include or exclude this coefficient as you wish. Coefficients may not be statistically significant. This does not always mean that the coefficient is zero. You may have too little data to establish the significance of the item. If it is correctly signed according to your theory you might consider leaving it in. If the coefficient is statistically significant but of the wrong sign you should reconsider the data consistency of your model. Any causal relationships are estimated given the model. If you have two alternative models one may be more data consistent than the other but you can not say use econometrics to say that one is true and one is false.In summary, in applying econometrics an evaluation of the significance of the model depends on the underlying economic theory as on the significance of various estimates.
In summary, in applying econometrics an evaluation of the significance of the model depends on the underlying economic theory as on the significance of various estimates.
Grey System Theory (GST) has its own set of models to validate a model, it's significance and reliability of its results. Under uncertain conditions and insufficient data, its results are acknowledged to be superior to that of traditional statistical methods. Also, the Law of Large Number and probability distribution is not a matter of concern in GST. Try it, you would be amazed! Prof. Jafar Rezaei, the founder of BWM method, recently said, GST is effect but overlooked approach.