You can correlate the difference between Yreal-Yprediction model 1 and Yreal-Yprediction model 2 and doing ANOVA test.

R^2 and R^2 adjusted should give you a pretty good idea on goodness of it.

Thank you for your valuable answers.

I am confused by whether the OP tries to test goodness of fit for residuals to one particular distribution or sample data to the regression model.

I am not saying to use R^2 adjusted alone, which however is a quick way to assess goodness of fit for data to the regression model. He also need to test all those three conditions for OLS classical assumptions on residuals: 1) Normality (Chi-square), 2) Autocorrelation (Ljung-Box Q), and 3) Heteroskedasticity (Breusch-Pagan or Koenker). If any of those three conditions are violated, the OP then need to take the non-linear regression route. It is noted that real-world data are often non-linear. For more advanced ODA method to cope with such problem, please refer to Paul's blog here: http://odajournal.com/.

However, I do not prefer f-test for its inability to differentiate individual coefficient's impact on the system. For significance interpretation, look at both confidence interval and p value of estimated coefficients.

Finally, I suggest you read the r-squared discussion by Professor Robert F. Nau:

http://people.duke.edu/~rnau/rsquared.htm

Also, take a look at those goodness of fit tests for sample data distribution:

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm

Thank you. I think I want to test the goodness of fit for data to the regression model. Let me explain this. I have strength and stiffness values of a range of material types. Now I want to plot stiffness vs strength curve and want to DEVELOP a regression equation between them so that this relationship can be used to determine stiffness of a material for a given strength value.

I have plotted the curve for 24 data points and obtained a fitting relationship like Y=5000*X (R square = 0.97). Now how significant is this relationship statistically? Reviewer says R-square alone is not enough to understand how good a regression equation is.

I tried to apply f-test like this: we have relationship between f-value and R-square. so I can calculate f-value= 711 for degrees of freedom in this case (df1=1 & df2=22). Now I compared this computed f-value to the critical f-value = 7.94 for 1% significance level (p=.01). As computed f-value > critical f-value, this can be said it is significant for 1% confidence level.

I incorporated this statistical analysis in one of my submitted journal article. Is it correct in terms of statistical point of view? Or any suggestions?

Thanks in advance for your cooperation.

Even your model is statistically significant from F-test or t-test, we do not know how much is  the prediction error when the model was used to determine stiffness of a material for a given strength value. Thus, we presented a statistical index called mean prediction error (MPE) to assess a model. MPE is related to SEE and sample size. it can be formulated as MPE=ta*(SEE/Mean)/n^0.5*100, where ta is the t-value at confidence level a with n-p degrees of freedom, n is the number of samples, p is the number of parameters, SEE is the standard error of estimate, Mean is the mean value of samples. In addition, if possible, validation is helpful for model evaluation.

Hi Dalim,

Thanks for your detailed explanation. The reviewer is correct, because R^2 or adjusted R^2 (preferred) is used for data fit to regression, while coefficient p-value is for coefficient significance. For example, p-value of X coefficient less than 0.01 will be considered highly significant, less than 0.05 be significant, and larger than 0.05 but less than 0.1 be partially significant not very much.

Best regards,

Richard

Dear Dr Juehue Shi, can you please explain about the p-value of X coefficient and how it is determined.

Thanks everyone for your answers. Now, I have a better idea on the statistical signification of a regression equation.