Recent research showed that all semi-supervised learning algorithms are not capable of outperforming supervised learning algorithms that work on just the small amount of labeled data.
Guo, Y., Niu, X., & Zhang, H. (2010). An Extensive Empirical Study on Semi-supervised learning. Proceedings of IEEE International Conference on Data Mining, 10, 186–195.
I agree that it's not a guarantee that using extra unlabeled data is going to improve performance. Especially in algorithms which build a model first, label unlabelled points, and then use those as new data, things can go wrong. Some more explanation and examples of performance decreasing with extra data can be found here: http://www.sciencedirect.com/science/article/pii/S0167865513000913
In principle, semi-supervised learning can help when labeled data is scarce, however, there is an important prerequisite: that the distribution of examples, which the unlabeled data will help clarify, be relevant for the classification problem.
In a mathematical formulation, one could say that the knowledge on p(x)
that one gains through the unlabeled data has to carry information that is useful
in the inference of p(y|x). If this is not the case, semi-supervised learning will not
yield an improvement over supervised learning and it may actually reduce performance. Some additional details for the assumptions that need to hold for semi-supervised learning to be beneficial can be found here:
http://mitpress.mit.edu/sites/default/files/titles/content/9780262033589_sch_0001.pdf
In order to be able to have a more reliable semi-supervised learning algorithm, one needs, above all, to have a more reliable *structural* form of data representation. In other words, the more *adequate* the chosen structural data representation is, the easier and more reliable is the semi-supervised learning task.
I'm guessing the only way to assure we have an " adequate" form of data representation is to have as much unlabeled data as possible. Or is there another way around?
But then again, how much unlabeled data can be termed as enough? Sigh!
Aardra, the adequate form of data representation has nothing to do with the amount of data you collect.
But then how else would you assure an adequate form of data representation?
By assuring that the labeled data provides relevant information to the model?
If you could please substantiate. Thanks.
I guess if the features you collect are irrelevant to your classification problem, it does not matter how much data you have, whether labeled or unlabeled, it will not be an adequate representation.
An important point when discussing the "reliability" of semi-supervised algorithms (especially the ones which can be initialized with very few seed labeled instances) is how to select a subset of labeled data for a particular problem. Performance can be very sensitive to the initial labeled instances selected. This is particularly the case with minimally supervised relation extraction settings. However, I haven't seen much "robustness" analysis on semi supervised algorithms conducted with this aspect in mind.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Artificial Intelligence