I have a repeated measures design with about 16 cases and 5-6 points of measuring. Sometimes, 1-4 full cases or some points of measure are missing. (The measures are 20 numerical and categorical data taken from questionnaires.)
The clue is: It's a small dataset with holes in it, but the 16 cases are all that even exist. So they fully represent reality whereever they're complete. I wanted to run logistic regressions with up to 6 predictors. Can I do that? I know about the many problems such small datasets have for regression analysis - but what if there aren't any more cases in reality? What does this mean to my statistical analysis, and how much do the missing data matter? Many thanks
Assuming your sample is this small, would be good to try constructing different models. Due to accuray issues, it will be difficult to choose a model with confidence.
Assuming your sample is this small, would be good to try constructing different models. Due to accuray issues, it will be difficult to choose a model with confidence.
Your dataset isn't a complete one(ie., Not balanced) given that all cases or respondents are just 16, and the layers of points(units) measured varying "1-4 full cases (or some points of measure are missing)" and others 5-6.
If there are no more cases, a bootstrap approach can help you in obtaining the sampling distribution of the responses and also the confidence interval of interest. But, for repeated measure and 20 categories , your potential model would lack power, would pose interpretation difficulty. Perhabs a model reduction and using PCA, also taking care of the missing values could be of help the small number of respondents not withstanding.
On your statement, " I wanted to run logistic regressions with up to 6 predictors." , that is ok so far each predictors are not repeated measure ( a mixed effects model or GEE should be used instead or mixed model for logistic regression which isn't straight forward in SPSS or just go for repeated measure ANOVA) and you need far above 10 points/respondents for each predicting variable! ONLY that it is subject to weither your response variable/s type (Dependent Variable/s or DV) are ordinal or dichotomous and your specified variables meet the assumptions for the use of logistic regression { i.e., independence of errors, linearity in the logit for continuous variables, absence of multicollinearity, and lack of strongly influential outliers [converting the continuous predictors to standardized scores, values below -3.29 or greater than 3.29 are suspect outliers.]}.
And, If the number of observations is lesser than the number of features, Logistic Regression should not be used, otherwise, it may lead to overfitting. R2 would be misleading here, Adjusted R2 advised.
For ordinal DV, the key assumption in ordinal regression is that the effects of any explanatory variables are consistent or odds are proportional across the different thresholds, or cumulative points (which SPSS calls this the assumption of parallel lines.)
In conclusion, "You can run a Generalized Estimating Equation model for a repeated measures logistic regression using GEE (proc genmod in SAS). It has a repeated statement, and can run equivalent models to a model in Mixed with a repeated statement." Karen Grace-Martin, Mixed Models for Logistic Regression in SPSS)
And if your data wasn't the total population under study, and for valid generalization (not just for the study and sample obtainable) some authority advised that, "for observational studies that involve logistic regression in the analysis, this study recommends a minimum sample size of 500 to derive statistics that can represent the parameters in the targeted population."
NBCI (2020) retrieved at at https://www.google.com/url?sa=t&source=web&rct=j&url=https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6422534/&ved=2ahUKEwju_pfDmt_tAhUEiFwKHbIlDD8QFjADegQIBBAE&usg=AOvVaw2VyGRTQ7Al6q0hsytOottU
See the following links on ordinal logistic regression and repeated measure.
1. https://www.google.com/url?sa=t&source=web&rct=j&url=https://stats.idre.ucla.edu/r/dae/ordinal-logistic-regression/&ved=2ahUKEwj-p7S1lN_tAhXIT8AKHdMUBlQQFjARegQIEhAC&usg=AOvVaw2M6M0dW8UMC886Sd9QGLV0
2. Using Generalized Estimating Equations to Fit a Repeated Measures Logistic Regression. https://www.ibm.com/support/knowledgecenter/SSLVMB_23.0.0/spss/tutorials/gee_wheeze_intro.html
3. https://www.google.com/url?sa=t&source=web&rct=j&url=https://statistics.laerd.com/spss-tutorials/ordinal-regression-using-spss-statistics.php&ved=2ahUKEwj-p7S1lN_tAhXIT8AKHdMUBlQQFjANegQIDxAB&usg=AOvVaw3wdS98r3Oes7piTW43kG9x
It's not possible to obtain reliable statistical estimates based on a sample containing 16 cases (especially if some of them are incomplete).
However, you can use additional judgmental inputs (prior information) to modelling. This is a (relatively) complicated task. More methods/references can be found in relevant chapters of the following thesis:
Thesis Integration of judgmental and statistical approaches for dem...
I don't understand what happened to my answer, I responded here 3 days ago!
So again:
Thank you very much for all your insights, advice, and links. Looking forward to reading up on things and trying stuff out.
I know power will be low and the small sample size will remain a problem in some regards.
But what do you think about another piece of advice I got? It was that I couldn't or shouldn't use inferential statistics at all, only descriptive statistics, and these would tell me enough. I don't quite understand this because inferential stats have a different purpose. Is my sample really to small for ALL inferential stats?
Also, I was recommended to do partial least squares analysis. It's a type of regression and I assume it's a version of the OLS (ordinary least squares) anaIysis. Is anyone here familiar with it?
@Andrey Davidenko: prior information - does this involve Bayesian methods?
- Badges
- Science topic