I have a binary classification problem with a data set that is missing a large amount of data. I have been reading papers about data imputation, but haven't reached a solid conclusion about which method is best. Note that I will most likely be using SVM for classification.
Just one more addition to this topic: before using any algorithm, please be sure that missing data are really missing :). I mean NULL values are not necessarily 'i do not know' values, it also could be 'not applicable' (e.g. number of children of a 6 year old male), or there is no data (yet) (e.g. birth year of the last child). So, sometimes ignorance is the best algorithm. That is probably not the case here; just a warning to do data understanding part of your job as well and find out why data are missing.
I suggest you to explore the powerful Bayesian Network using learning with incomplete data set.
I agree with Younes; perhaps SVM is not the best method to use here. Bayesian classification methods have no problem with missing data. If you can say what type of data you're classifying we could be able to give you additional help in what type of bayesian network to use.
The data is coming from the patients and mostly relies on the survays taken by the patients. The missing data comes from the fact that many people have not completed any or some of the survays (there are different types of survey) . I am not familiar with bayesian networks (i am a student and we will cover this topic in about a month) and was planning to use an imputation method (perhs LASSO) and use either SVM or logistic regression for classification. Now does it mean that by using bayesian networks I won't need to perform data imputation?
Yes, that is correct, if using a bayesian method then imputation will not be needed.
If you are doing this for a school project and it is not research-oriented, then it would suffice to use imputation. LASSO would work well here. This website is mostly for research-oriented questions.
You do not have to be Bayesian to deal with incomplete data. The well-known expectation-maximization (EM) algorithm can handle it as well.
Bayesian method is a way to model you problem, from the complete o incomplete data, you will be able through EM algorithm to learn how to classify a given proposition such as P(q/Data) estimating of course the Maximum likelihood of the corrupted data, by q is meant the parameter to estimate (your query to Bayes Network you designed to solve you problem). Be aware that the classification concern just discrete variables (it's your case), regression instead concern continuous variables...
use EM. other option could be replace the missing data by the corresponding feature average value.
EM essentially does maximum likelihood estimation. Bayesian Network is not necessarily Bayesian. Hard-core Bayesianism does not use EM algorithm, they average over all possible values of the missing entries instead.
Thank you for the inputs.
The amount of missing data is large, for some features more than half of the data is missing. I haven't tried Bayesian but filling up the data with K-nearest neighbor or LASSO didn't generate the best result (SVM did a better job than logistic regression), but still has a large error (24%). BTW, logistic regression (sigmoid) can be used for discrete prediction as well. In fact, one advantage of using it is that it can give you how close to 1 or 0 you are (i.e. 0.7or 0.2), which is desirable in this study. I was just wondering if there are better methods for imputation or I can get a better result with the missing data.
p.s. this is a research-project and the data is real
Thank you
better use a kernel function that can handle missing data then apply svm.
Kernel functions for missing data?
see for instance http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.59.8282
Maybe, searching for 'Transductive learning' could also be of some help
Just one more addition to this topic: before using any algorithm, please be sure that missing data are really missing :). I mean NULL values are not necessarily 'i do not know' values, it also could be 'not applicable' (e.g. number of children of a 6 year old male), or there is no data (yet) (e.g. birth year of the last child). So, sometimes ignorance is the best algorithm. That is probably not the case here; just a warning to do data understanding part of your job as well and find out why data are missing.
If you have adequate numbers of complete data you can use machine learning approach to predict missing fields !
Article Machine Learning Based Missing Value Imputation Method for C...
- Badges
- Science topic