I am evaluating the relationship between the amount of a nutrient intake and animal growth rate. Three regression models were used to quantify their relationship- linear, quadratic and piece-wise (or known as broken line; with one knot) regression. Three questions
1. What criterion should I look at to decide the best fit model? The R2 was 0.694, 0.704 and 0.710 for the three regression models.
2. How should I compare the quadratic model with the piece-wise regression model?
3. In addition, does anyone know the synthax for computing Bayesian Information Criterion (BIC) for piece-wise regression models (under the Non-linear regression function in IBM SPSS software)? The syntax for computing BIC in linear regression models is known https://www.ibm.com/support/pages/mallows-cp-akaike-aic-amemiya-pc-or-schwarz-bayesian-criterion-sbc
Thank you
R-Square values, Mallows Cp, Mean Square Error and minimum number of independent variables giving good/significant prediction of dependent variable are the good criteria to select the best model.
There is no single number you could use telling you what a better model is.
If the model has a particular purpose, e.g. prediction, then you might look for the bias and the error of predictions - over the range of possible predictor values. You might consider the simplicity of the model, too (maybe a quadratic model is easier to fit than a broken-line model, but the coefficients of a broken-line model are easier to interpret practically etc.). Inspecting and interpreting residual diagnostic plots may be a first step, but ultimately you must check the performance on a new, independent data set.
If the models represent different underlying theories (I don't suppose that this is the case here; it seems to just took some models that seemingly "fit" the observed data), the AIC is sufficient to identify the model (theory) that fits the data best.
Dear Fan Liu,
Although I have not worked on IBM software but worked on some other platforms. Based on that I would like to put up my opinion:
1. The R2 value consideration: The more closest to 1 is the best fit model. However, in your case all three are nearly same. Little bit improvement seen in the piece wise model.
2. The other parameters are not shown in your questions i.e. the value of RMSE, MSE, Total Positive Rate, Total Negative Rate, Fallout and Recall. Please look in to that features too.
3. Tryout different models(Course tree, SVM, KNN, Guasians etc.) taking 20% of test data.
4. You may also look for an Optimised Ensemble Bags Regression Model with Bayes' or Random Search. I am highly sure that you may get extremely better results. Wherein, you can seek better R2 and lower RMSE values.
Please share your experiences after trying out these steps.
Thank you.
Just as Prof. Jochen_Wilhelm2 , wrote, "the AIC is sufficient to identify the model (theory) that fits the data best," but you are definitely using Bayesian Approach, and you were also right with the choice of BIC as a prime criteria for the best model (for 'the growth rate'). Be they are Linear, piecewise, or quadratic, the R-square(s) (&adjusted) and the BICs are (more than minimumly sufficient) and meaningful for fitting evaluation and model comparison, hence selection!
{BIC is like AIC and Mallow's Cp, but it comes from a Bayesian argument and was believed, "it tends to choose smaller models than AIC is." The BIC selection criteria can be requested through SPSS command syntax, i.e., by adding SELECTION to the /STATISTICS subcommand following your regression syntax, METHOD=Piecewise, etc]. Mallow's CP, Akaike Information Criterion (AIC), the Amemiya Prediction Criterion, and the Schwartz Bayesian Criterion[BIC] will then appear in the Model Summary table! "These criteria currently cannot be requested through the Regression dialog box." as explained here_ https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.ibm.com/support/pages/mallows-cp-akaike-aic-amemiya-pc-or-schwarz-bayesian-criterion-sbc&ved=2ahUKEwi7h6WB6urrAhWcWhUIHSHHBUEQFjABegQIDhAI&usg=AOvVaw1Spflvaq7OKsbzA9mhsdsU
Finally, you might consider looking at the performance in terms of the regression_line and Residual plots, (and importantly plot bar of the errors for each model as you compare in your report). Goodluck. 👍
Cite
1 Recommendation
Ahmad Abubakar Suleiman
Universiti Teknologi PETRONAS
A model with higher R2 value is chosen as the best model.
Cite
Marcin Brdys
PQ
please see publication describing regression models and their weight parameterization to forecasting combinations ...
Cite
James R Knaub
Retired US Fed Govt/Home Research
Fan Liu -
Jochen said "Inspecting and interpreting residual diagnostic plots may be a first step, but ultimately you must check the performance on a new, independent data set." For those two important areas, you can research the terms "graphical residual analysis," and "cross-validation."
One of the features you may discover in a graphical residual analysis is heteroscedasticity. This is a feature, not a bug. It is to be expected that larger predicted values will be associated with larger sigma of estimated residuals. (Please see https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity and examples in https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity.)
Further, a graphical residual analysis is a way to consider model fit, and a cross-validation is a way to see if you have overfit to your particular sample. Graphics are very important, but you have to remember that they use the data you have, and if you try to weave a model too closely through the data points, you will likely overfit so that your model does not work as well for the rest of the population.
Cheers - Jim
Cite
4 Recommendations
Andrey Davydenko
JSC "CSBI"
Let me recommend this framework for the comparison of alternative statistical models:
Conference Paper Data Formats and Visual Tools for Forecast Evaluation in Cyb...
Initially presented for time series data, but can be easily adapted to the standard regression analysis settings.
Cite
Salwa A. Mousa
Al-Azhar University
HI FAN LIU YOU CAN USE THE FOLLOWING: R2adj., KS, AIC, CAIC, and BIC are useful as generally indicative of goodness of fit GOOD LUKE
Cite
3 Recommendations
Yunus Shukor
Universiti Putra Malaysia
First I would use primary growth models for organism growth rate such as modified Gompertz, modified logistics etc
see
Umar, A. M., & Abd Shukor, M. Y. (2020). Modelling the Growth of Nile Tilapia (Oreochromis niloticus) on Fed Diets Formulated from Local Ingredients in Cages. Bulletin of Environmental Science and Sustainable Management (e-ISSN 2716-5353), 4(1), 1-6.
The best model would be based on several error functions (BIC among them) that must include those that give a penalty for no of parameters used.
The growth rates obtained can then be used for secondary modelling exercises such as Monod (effect of substrate concentrations to growth rates), Haldane (inhibitory effects)
see
Halmi, M. I. E., Shukor, M. S., & Shukor, M. Y. (2014). Evaluation of several mathematical models for fitting the growth and kinetics of the catechol-degrading Candida parapsilopsis: Part 2. J Environ Bioremed Toxicol, 2(2), 53-57.
If you are using several different types of nutrients and wishes to know which would be the best judging on the growth rates, then you want to do ANOVA test (simplest) to the specific growth rates (>2 means).
see
https://www.graphpad.com/guides/prism/7/curve-fitting/reg_comparing_fits_with_anova.htm
Of course , nonlienar regression relies upon the data (residuals) to be normally distributed (do normality tests)
see
Halmi, M. I. E., Shukor, M. S., Masdor, N. A., Shamaan, N. A., & Shukor, M. Y. (2015). Testing the normality of residuals on regression model for the growth of sludge microbes on PEG 600. Journal of Environmental Microbiology and Toxicology, 3(1), 14-17.
One of many assumptions of parametric-based nonlinear regression is that the within-group variances of the groups are all the same (exhibit homoscedasticity). If the variances are different from each other (exhibit heteroscedasticity), then the model is not statistically adequate to explain the data. Several tests are available to check the equality of variances on residual data from three or more samples and include those of Cochran, Bartlett, Brown–Forsythe and Levene. Ive used Bartlett and Levene
see
Shukor, M. Y. (2016). Bartlett and the Levene’s tests of homoscedasticity of the modified Gompertz model used in fitting of Burkholderia sp. strain Neni-11 growth on acrylamide. Bioremediation Science and Technology Research, 4(1), 18-19.
and
Nonlinear regression using the least square method normally uses the assumption that data points do not depend on each other or the value of a data point is not dependent on the value of preceding or proceeding data points. You can use Durbin-Watson test
see
Shukor, M. S. (2014). Test for the presence of autocorrelation in the Buchanan model used in the fitting of the growth of the catechol-degrading Candida parapsilopsis. Journal of Environmental Microbiology and Toxicology, 2(2).
Cite
Alfredo Hernández-Llamas
Centro de Investigaciones Biológicas del Noroeste
Fan Liu posted his question more than one month ago, but he never responded to the feedback. That is why I removed my comments. I am still receiving notifications when new answers are added, and I would suggest stopping sending more of them or Fan Liu removing his question. It is pointless and a waste of time to keep open this chat. Does someone know how I can block receiving notifications on Fan’s query? I will appreciate that.
Best,
Alfredo
Cite
1 Recommendation
Fan Liu
Rivalea Australia
Hello Dr Alfredo Hernández-Llamas,
I feel sorry you are disappointed not receiving a reply from me. I read every answer coming through-some of them are helpful (including yours, appreciated). I should acknowledge answers from everyone earlier. Thank you for your replies to my questions.
Two reasons that I am not ready yet to conclude my questions. First, different model selection indexes lead me to choose different regression models for my data. Following the highest R2 will choose linear-break line model (with one-knot), but following the lowest BIC (penalizes on the increased parameters such as the added quadratic term or the knot) I was redirected to select the most simple linear model even with the lowest R2. The simple linear model may be lack of the biological meaning. To this point, I cannot really tell what most appropriate selection index is. Second, my last question regarding the syntax for computing BIC criterion of linear-break line model in SPSS has not been answered. I wish someone could answer particularly on whether the one-knot linear-break line model should draw more penalty on degree of freedom or number of parameters for calculating BIC when compared with a simple linear regression model.
Also I would like to keep the question open, because I believe it would be useful for someone else who may have the same question in future.
Thank you.
Kind Regards,
Fan
PS: I saw that you are still following the question, maybe that is why you keep receiving the alerts when new comments added to my questions. Maybe un-follow the question will cease future alerts.
Cite
1 Recommendation
Alfredo Hernández-Llamas
Centro de Investigaciones Biológicas del Noroeste
Dear Fan,
Thanks for responding. I just realized that ResearchGate offers the possibility of blocking notifications “When someone answers a question I follow”. However, if I use this option I will not receive notifications of answers added to other chats I am interested in following. So, it seems I will still receive new notifications on your question.
Good luck with your research.
Best,
Alfredo
PS. And yes, it is a good idea to acknowledge answers letting know that at least you received them. Chats with no feedback are pointless.
Cite
1 Recommendation
Abdu Manga
Bayero University, Kano
You are suppose to select the model with the highest coeefficient of determination ie R squared and the lowest error mean square to be the best fitting model
Cite
1 Recommendation
Muhammad Aslam Khan
University of Agriculture Faisalabad
Following criteria should be used to select the best fit regression model:
1. The coefficient of Determination (R-square) should be more that 0.80.
2. Error Mean Square should be minimum.
3. Mallows Cp (number of variables in the model) should be minimum.
Cite
1 Recommendation
Obey Gotore
Akita Prefectural University
aic, bic, mse, rmse, X2 among others for non-linear models. For linear models include R2 and the adjusted R2 values
Cite
Carlos Araújo Queiroz
Universidade NOVA de Lisboa
Maybe you can consider the recursive least squares algorithm (RLS) with forgetting factor (RLS-FF). RLS is the recursive application of the least squares (LS) regression algorithm, so that each new data point is taken in account to modify (correct) a previous estimate of the parameters from some linear (or linearized) correlation thought to model the observed system. The method allows for the dynamical application of LS to time series acquired in real-time. As with LS, there may be several correlation equations with the corresponding set of dependent (observed) variables. For the RLS-FF algorithm, acquired data is weighted according to its age, with increased weight given to the most recent data. The correlation parameters are updated gradually.
Application example ― I have applied the RLS-FF algorithm to estimate the parameters from the KLa correlation, used to predict the O2 gas-liquid mass-transfer, hence giving increased weight to most recent data:
Thesis Controlo do Oxigénio Dissolvido em Fermentadores para Minimi...
Cite
Koen Van de Moortel
independent math & physics tutor
Good question.
Goodness of fit is not the best criterion because you can have a good fit with a totally wrong function. The only good criterion is which model describes the best the actual physical situation. If you don't have that, I suggest: extrapolate the model and judge how realistic this is. See examples:
www.lerenisplezant.be/fitting.htm
Cite
James R Knaub
Retired US Fed Govt/Home Research
As I noted above, on Sept 19, 2020, in short, "...a graphical residual analysis is a way to consider model fit, and a cross-validation is a way to see if you have overfit to your particular sample."
You can compare model performances for a given sample on the same scatterplot using a graphical residual analysis for each model. You could then do the same for a second sample on a second scatterplot. That could be a form of cross-validation. But you want to be fairly sure not to miss any important parts/features of the population or subpopulation which you are trying to model.
As I suggested earlier, you can research the terms "graphical residual analysis," and "cross-validation."
Cite
Similar questions and discussions
How to get milisecond duration of an audio file?
Question
2 answers
- Tariq Sm
I have an audio file running in the format of mm:ss, but i want an audio file to run in this format mm:ss:msms(milisecond). I have used this format of converting by VLC media, but it is not running. LINK: https://www.youtube.com/watch?app=desktop&v=A9yq8qT0hqY
So is there any way I can get audio file in miliseconds also?
It will be a great favor
How do I report the results of a linear mixed models analysis?
Question
46 answers
- Subina Saini
1) Because I am a novice when it comes to reporting the results of a linear mixed models analysis, how do I report the fixed effect, including including the estimate, confidence interval, and p-value in addition to the size of the random effects. I am not sure how to report these in writing. For example, how do I report the confidence interval in APA format and how do I report the size of the random effects?
2) How do you determine the significance of the size of the random effects (i.e. how do you determine if the size of the random effects is too large and how do you determine the implications of that size)?
3) Our study consisted of 16 participants, 8 of which were assigned a technology with a privacy setting and 8 of which were not assigned a technology with a privacy setting. Survey data was collected weekly. Our fixed effect was whether or not participants were assigned the technology. Our random effects were week (for the 8-week study) and participant. How do I justify using a linear mixed model for this study design? Is it accurate to say that we used a linear mixed model to account for missing data (i.e. non-response; technology issues) and participant-level effects (i.e. how frequently each participant used the technology; differences in technology experience; high variability in each individual participant's responses to survey questions across the 8-week period). Is this a sufficient justification?
I am very new to mixed models analyses, and I would appreciate some guidance.
Is it possible to get non significant results in post hoc test when we got the significant result in ANOVA ?
Question
82 answers
- Umesh Wadgave
I got significant results after ANOVA test, but when I applied post hoc Tukey's test, I obtained non significant results. I could not reason it out.
How to assess goodness of fit for a non-linear model?
Question
45 answers
- Igor Shuryak
Suppose a non-linear smooth function is fitted to some data (e.g. means and standard errors for cell survival after various radiation doses). What are some useful ways to assess goodness of fit for the model, without comparing to other models?
Journal Citation Report & Journal Impact Factors 2023
Discussion
1 reply
- Pawan Kumar Rose
What are the Expectation ? Seems most journals are dipping.
Impact factor 2023
Discussion
21 replies
- Sunil Kumar Sahu
Here is list of Impact factor 2023.
Journal Citation Reports 2023
Is MDPI and its all journals are predatory??
Discussion
5 replies
- Krishna Dhakal
Hello everyone,
Recently, I came to a link that has listed all the MDPI journals as predatory. I have a few papers with MDPI journals and it really concerned me. How valid is this information? Please provide your views on this.
Link to the information:https://predatoryreports.org/news?fbclid=IwAR2_gvwBeI8uI_FDuCZxmnNOqGr4r2zV6IWJCdeMaLXjHEOZpFjSsfeAC_w&blog=y
Related Publications
Bayesian sparse regression models of high dimensional data
Article
- L. Bottolo
- Sylvia Richardson
Statistics for Ecologists: A Frequentist and Bayesian Treatment of Modern Regression Models By JohnFieberg (Ed.). 2022. Self‐published online. pp. 526. $0.00 online:(https://fw8051statistics4ecologists.netlify.app/)
Article
- Toryn Schafer
BAYESIAN INDIFFERENCE PROCEDURES: REFINEMENTS AND EXTENSIONS
Article
- Melvin R. Novick
This report is a technical supplement to the University of North Carolina, Institute of Statistics, Mimeo Series Report No. 345, which dealt with the specification of indifference distributions for a wide range of statistical models and within the context of a Bayesian logical probability model. In this paper a more refined approach to indifference...
Got a technical question?
Get high-quality answers from experts.
Top contributors to discussions in this field
Ca Dr. Gaurav Bhambri
Nidhal Kamel Taha El-Omari
Anuraj Nayarisseri
Daniel Wright
Kelvyn Jones
Join ResearchGate to find the people and research you need to help your work
- 25+ million members
- 160+ million publication pages
- 2.3+ billion citations
or
- Join for free
- LoginEmail Tip: Most researchers use their institutional email address as their ResearchGate login
PasswordForgot password?
Keep me logged inor
Continue with Google
Welcome back! Please log in.
Email · HintTip: Most researchers use their institutional email address as their ResearchGate login
PasswordForgot password?
Keep me logged in
or
Continue with Google
No account? Sign up
- Badges
- Science topic
- Similar topics
- Biological Science
- Molecular Biology
- Biomolecules
- Nucleic Acids
- RNA