I have a data of 100 yes/no type answers collected from 100 students, which forms a matrix of 100X100. I have around 200 missing data, and the missing data cannot be ignored since it very badly affects the scoring system leading to neglecting large samples. These 100 questions constitute both positive scoring (yes-1 mark, No-0 mark) and negative scoring (yes- 0 mark, No-1 mark). How this missing data should be handled?
As a first approach, I would consider using some kind of KNN classification on the missing data, using the complete answers (individuals) as the training set.
The first step is to know if reasons for which the data is missing can be related to the data itself or not. There are three possibilities:
1. Data missing total at random: The reason for having missing data is not related to the data itself (handling mistake, lost by the post, etc.).
2. Data missing at random: Their is no association between missing data and the variables of interest (exposure, outcome).
3. Data not missing at random: Their is a reason to believe missing data is in relationship with the variables of interest.
Replacing data can only be done for situation 1 and 2. For situation 3, you would have to analyze full data and discuss limitations to internal validity due to missing data.
To replace data, I suggest using multiple imputation. This constructs multiple datasets replacing missing data by random values. This can be done selecting sub-samples from which to draw the values from. Under the null hypothesis, I would suggest drawing from the subgroup with the characteristic of interest only. If there are no difference, this should not change anything -> but if there is, this will conserve existing differences between groups. You then need to replicate your analysis for all sub-datasets and infer results from the dispersion of these results over these.
If you are using STATA, type -help mi- in the command box for further explanations.
Hello, your name is so long I will not attempt to spell it out here. If you go to my website on the favourite links page there is a link to a statistical textbook, and in that book you can search for case-wise versus list-wise deletion of data.
Regards, Deborah Hilton Statistics Online
http://sites.google.com/site/deborahhilton/
Dear sir,
I have attached one article, please read it, and i have sent the links of books.read.
links
CK Enders - 2010 - books.google.com
RJA Little, DB Rubin - 2014 - books.google.com
PD Allison - 2001 - books.google.com
Good Luck
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics