Dear Dalim... As far as I get you you talk about two different things:

1.) Wenn you want to "test" goodness fo fit you want to determine whether your model describes your data adequately. This is a prerequisite that you can trust the conclusions and or predictions you draw from your model. There are several ways to do this. I usually apply graphical exploration of the model (e.g. using residual (scatter and qq plots) and leverage plots). Important questions e.g. in standard linear regression are to ask are if your residuals are radmonly and normal distributed, since these are key assumptions. Also you have to make sure that you dont have strong influential data-points. In glms it is key to test for overdispersion and also goodness of fit.

2.) When you want to test whether the relationship you modelled ist significant the test to apply depends on your modelling framework. In a standard linear regression in R usually t-tests are applied to see if the model parameters are significantly different from zero and a F-test is used  to test if a model as a whole is "sigificant" (using these tests also assumes normality). In a glm frame-work sometimes Wald t-test are applied. To sum up depending on your question the answer is different. Other ways are Likelihood ratios tests.

Finally, I would not recommend to do it in Excel. Rather, in case you are willing to work yourself into it, I would recommend using the R-software (http://www.r-project.org/) e.g. in combination with maybe a bit more convenient interface such as R-Studio (http://blog.rstudio.org/). R allows you to do all kinds of statistical analyses  from very easy and standard to whatever comes to your mind.

For R plenty of free manuals are available online. Just google "R Statistics filetype:pdf" and you will find some. A good Web-page is Quick-R www.statmethods.net

Good luck & Enjoy!

Dear Dalim... As far as I get you you talk about two different things:

1.) Wenn you want to "test" goodness fo fit you want to determine whether your model describes your data adequately. This is a prerequisite that you can trust the conclusions and or predictions you draw from your model. There are several ways to do this. I usually apply graphical exploration of the model (e.g. using residual (scatter and qq plots) and leverage plots). Important questions e.g. in standard linear regression are to ask are if your residuals are radmonly and normal distributed, since these are key assumptions. Also you have to make sure that you dont have strong influential data-points. In glms it is key to test for overdispersion and also goodness of fit.

2.) When you want to test whether the relationship you modelled ist significant the test to apply depends on your modelling framework. In a standard linear regression in R usually t-tests are applied to see if the model parameters are significantly different from zero and a F-test is used  to test if a model as a whole is "sigificant" (using these tests also assumes normality). In a glm frame-work sometimes Wald t-test are applied. To sum up depending on your question the answer is different. Other ways are Likelihood ratios tests.

Finally, I would not recommend to do it in Excel. Rather, in case you are willing to work yourself into it, I would recommend using the R-software (http://www.r-project.org/) e.g. in combination with maybe a bit more convenient interface such as R-Studio (http://blog.rstudio.org/). R allows you to do all kinds of statistical analyses  from very easy and standard to whatever comes to your mind.

For R plenty of free manuals are available online. Just google "R Statistics filetype:pdf" and you will find some. A good Web-page is Quick-R www.statmethods.net

Good luck & Enjoy!

You can correlate the difference between Yreal-Yprediction model 1 and Yreal-Yprediction model 2 and doing ANOVA test.

R^2 and R^2 adjusted should give you a pretty good idea on goodness of it.

Thank you for your valuable answers.

Just a word of caution... I would not recommend to use R2 (adjusted or not) values to assess goodness of fit. As said, "the goodness of fit of a statistical model describes how well it fits a set of observations." (quote from wikipedia: http://en.wikipedia.org/wiki/Goodness_of_fit). For this you need to assess whether the chosen model describes your data adequately, ideally you know this before you collect the data. A posteriori you can test and compare different models. Different from this, R2 gives you - some - indication to what extent the variance in the rsponse variable is explained by the explanatory variable(s). However, the same degree of R2 can be related to various patterns in your data and you can even achieve a high R2 with a model fitted to a data-set that not at all is well described by the chosen model (see thee famous Anscombe example: http://de.wikipedia.org/wiki/Anscombe-Quartett). Ok so I would recommend to use a combination of graphical exploration (see plots I mentioned above) with maybe statistical test (you can e.g. in standard linear regression test if your model residuals are nomal distributed).

I am confused by whether the OP tries to test goodness of fit for residuals to one particular distribution or sample data to the regression model.

I am not saying to use R^2 adjusted alone, which however is a quick way to assess goodness of fit for data to the regression model. He also need to test all those three conditions for OLS classical assumptions on residuals: 1) Normality (Chi-square), 2) Autocorrelation (Ljung-Box Q), and 3) Heteroskedasticity (Breusch-Pagan or Koenker). If any of those three conditions are violated, the OP then need to take the non-linear regression route. It is noted that real-world data are often non-linear. For more advanced ODA method to cope with such problem, please refer to Paul's blog here: http://odajournal.com/.

However, I do not prefer f-test for its inability to differentiate individual coefficient's impact on the system. For significance interpretation, look at both confidence interval and p value of estimated coefficients.

Finally, I suggest you read the r-squared discussion by Professor Robert F. Nau:

http://people.duke.edu/~rnau/rsquared.htm

Also, take a look at those goodness of fit tests for sample data distribution:

http://www.itl.nist.gov/div898/handbook/eda/section3/eda35f.htm

Thank you. I think I want to test the goodness of fit for data to the regression model. Let me explain this. I have strength and stiffness values of a range of material types. Now I want to plot stiffness vs strength curve and want to DEVELOP a regression equation between them so that this relationship can be used to determine stiffness of a material for a given strength value.

I have plotted the curve for 24 data points and obtained a fitting relationship like Y=5000*X (R square = 0.97). Now how significant is this relationship statistically? Reviewer says R-square alone is not enough to understand how good a regression equation is.

I tried to apply f-test like this: we have relationship between f-value and R-square. so I can calculate f-value= 711 for degrees of freedom in this case (df1=1 & df2=22). Now I compared this computed f-value to the critical f-value = 7.94 for 1% significance level (p=.01). As computed f-value > critical f-value, this can be said it is significant for 1% confidence level.

I incorporated this statistical analysis in one of my submitted journal article. Is it correct in terms of statistical point of view? Or any suggestions?

Thanks in advance for your cooperation.

Even your model is statistically significant from F-test or t-test, we do not know how much is  the prediction error when the model was used to determine stiffness of a material for a given strength value. Thus, we presented a statistical index called mean prediction error (MPE) to assess a model. MPE is related to SEE and sample size. it can be formulated as MPE=ta*(SEE/Mean)/n^0.5*100, where ta is the t-value at confidence level a with n-p degrees of freedom, n is the number of samples, p is the number of parameters, SEE is the standard error of estimate, Mean is the mean value of samples. In addition, if possible, validation is helpful for model evaluation.

Hi Dalim,

Thanks for your detailed explanation. The reviewer is correct, because R^2 or adjusted R^2 (preferred) is used for data fit to regression, while coefficient p-value is for coefficient significance. For example, p-value of X coefficient less than 0.01 will be considered highly significant, less than 0.05 be significant, and larger than 0.05 but less than 0.1 be partially significant not very much.

Best regards,

Richard

Dear Dr Juehue Shi, can you please explain about the p-value of X coefficient and how it is determined.

Thanks everyone for your answers. Now, I have a better idea on the statistical signification of a regression equation.  

Hi Dalim,

P-value is from t value, which should be provided by most statistical software. Often the smaller p is, the better the more significant the coefficient will be. P-value can also be used for hypothesis test and you need either a very small p-value to reject the null hypothesis or a very large one to accept the null.

I suggest you read the attached article and those: 

http://en.wikipedia.org/wiki/P-value

http://support.minitab.com/en-us/minitab/17/topic-library/basic-statistics-and-graphs/hypothesis-tests/tests-of-means/what-is-a-t-value/

http://www.dummies.com/how-to/content/what-a-pvalue-tells-you-about-statistical-data.html

http://en.wikipedia.org/wiki/Student%27s_t-test

http://www.gla.ac.uk/sums/users/jdbmcdonald/PrePost_TTest/pandt3.html

Best regards,

Richard