My study is a feasibility RCT with 3 groups (one Intervention, one placebo, one usual care). Its aim is the determine the effect size for the main RCT in the future. However, the missing data is problematic with the drop-out rate is 35.9%. The main reason to drop out was participants too tired or too busy to follow the study protocol. No significant difference was found regarding drop-out rate of the 3 study groups. The baseline assessment indicates groups are comparable. No significant was found regarding characteristic and baseline assessment between participants who stayed and those who drop-out. Please advise any statistics method to handle missing data for my study. Suggestion with sample explanation are appreciated because statistic is really not my advantage. Thanks
My suggestions:
1. Significance testing for baseline differences should not be performed. If the randomisation is setup correctly, there in no need to do this testing. For details, please see: Article Testing for baseline differences in randomized controlled tr...
2. You may want to change the protocol to better suite the schedule and physical ability to complete it.
Regarding missing data can you tell us more about the nature of the missing data. Is it an entire session? Is it the follow-up tests?
Regarding sample size calculations. You can use G*Power software (http://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower.html) to do a sample size calculation for your future RCT using data from the feasibility study.
Hello Usman,
Missingness can be problematic depending upon the underlying mechanism – most often described under Rubin’s terminology (Rubin, 1976), but particularly well described in Enders very readable book (Enders, 2010). They are typically referred to as Missing at Random (MAR), Missing Completely at Random (MCAR), and Missing Not at Random (MNAR). MAR and MCAR are considered ignorable missingness (i.e., generally safe to continue).
In the perfect world, your data will be MCAR (i.e., not bias due to missing data) and there is a test for this (i.e., Little’s MCAR test; R. J. Little, 1988), however this is rarely upheld. MNAR primarily means your missingness is related to your outcome variable and means your missingness is problematic in justifying your results – they are likely biased by the missing data.
Most researchers make claims … and try to justify MAR, which means your probability of missing data is related to other variables you have measured, but not your outcome variable. Basically, this means that the missing data has the potential to be estimated (to some degree) thus reducing the bias.
There are general approaches to making the “argument” for MAR as there is no direct method of testing the underlying information.
(1) In order to understand the cause of missing data, it is useful to study predictors of missingness. Search within your collected variables to see if they are related to missingness. Often this means creating a “flag” variable (1=missing, 0=not missing) for missingness on a variable. See approach used in article (Durik, Vida, & Eccles, 2006, p. 387). They use a table to show correlations of missingness for each variable in study, then make an argument for MAR or MCAR. A very good example of how to explain and show.
If such a (correlated) variable is already incorporated in your analyses, then good. If not, many programs allow incorporating the “related to missingness” variable without being part of your model/regression (e.g., Mplus has “Auxiliary”).
(2) Modern estimation methods such as FIML/ML and multiple imputation are often invoked as they require weaker assumptions about the cause of missing data; that is parameter estimates with less bias and greater power (Enders, 2010; T. D. Little, Jorgensen, Lang, & Moore, 2014; Schafer & Graham, 2002). Keep in mind that they still require the assumption of an ignorable response mechanism (MCAR or MAR) … but everyone puts them down as a method of acknowledging the bias of missingness. “Even if MAR does not hold, estimation using ML assuming MAR is likely to perform better than using listwise deletion.” (Muthén, Muthén, & Asparouhov, 2016). For simulation studies showing this (see simulaiton studies in Muthén et al., 2016, p. 465). Once the missing data gets higher than 5% and certainly over 10%, “we recommend using inclusive implementation of MI or FIML…” (T. D. Little et al., 2014, p. 154).
(3) Others suggest comparing several analyses, such as complete cases and the model-implied methods to examine the sensitivity of results (Pigott, 2001).
Hope this is of some help and wishing you well with your research,
Tim
Refs:
Enders, C. K. (2010). Applied missing data analysis. New York, NY: The Guilford Press.
Little, R. J. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198-1202.
Little, T. D., Jorgensen, T. D., Lang, K. M., & Moore, E. W. G. (2014). On the joys of missing data. Journal of Pediatric Psychology, 39(2), 151-162. doi:10.1093/jpepsy/jst048
Muthén, B., Muthén, L. K., & Asparouhov, T. (2016). Regression and mediation analysis using Mplus. Los Angeles, CA: Muthen & Muthen.
Pigott, T. D. (2001). A review of methods for missing data. Educational Research and Evaluation, 7(4), 353-383. doi:10.1076/edre.7.4.353.8937
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581-592. doi:10.2307/2335739
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147.
Oops!! That was meant to be addressed to Hoang Huong! Apologies ...
Thanks Tim Edward Powers for your value reply. Now I am going test to see whether the missing data in my study is MAR or MNAR.
Dear Usman Rashid
The missing data in my study refer to participant who decided to discontinue the intervention. For those patients, they already done the baseline assessment but do not return the post intervention and follow up assessment. There were 3 timepoint for assessment T0 (baseline) T1 (post-intervention) T3 (post folllow-up) . Those patients did not have the date of T1 and T3.
Thanks for your recommendation ;)
You are welcome Hoang Huong . Following two articles address the issue of loss-to-follow-up.
1. Dettori, J. R. (2011). Loss to follow-up. Evidence-based spine-care journal, 2(01), 7-10.
2. Herbert, R. D., Kasza, J., & Bø, K. (2018). Analysis of randomised trials with long-term follow-up. BMC medical research methodology, 18(1), 48.
Tim Edward Powers No worries Tim. Thank you for the useful information.
- Badges
- Science topic
- Similar topics
- Academic Journals