I want to know the characteristics of sets of convex hull classifiers and what is different with H alone?
A linear combination of a set of points (or elements or classifiers or other functions) say x1, x2, ...xn is the set of points generated as a weighted sum i.e. w1x1, w2x2, ...wnxn (e.g. for two non-collinear points in 2-d you can generate whole space R2 choosing different values of w's ) . An affine combination or an affine hull is a set of points that is generated by ensuring that the sum(all weights) = 1 (e.g. for x1,x2 in R2, an affine hull is a line passing through those two points). Whereas a convex hull is a further restricted subset by constraining that Sum(all weights)=1 and (every weight)>=0 (for x1,x2 in R2 that results in the line-segment between x1 and x2).
In ensemble/boosting methods where you have a set of base classifier and you often search for a best combination (weighting scheme) of those, a convex hull helps in restricting the search space since ConvexHull
Dear Muhammad. You gave the exact definition of the convex hull concept as it defined in convex analysis. But in boosting, it have a slightly different meaning. To explain it, consider the simplest classification model, where objects should be classified into 2 classes. In this case, any classifier can be considered as a mapping from the space of objects (feature space) into the finite set {-1, 1}, where 1 stands for the first class, and -1 for the second. Generally speaking, a formal convex combination of a family of classifiers does not a classifier itself. Therefore, in machine learning, by convex combination of classifiers (or just a committee) is called a composition of linear weighted combination (with normed negative weights, according to your definition) and the signum function. Simply put, such a combination makes classification decisions according to majority voting of the initial classifiers.And by convex hull of a family of basic classifiers is called a set of such 'committes'.
Thanks, Arash. The sentense in image coincides with the classical convex hull definition given by Muhammad. But, to give more complete explanation, it should be considered in context. Could you upload the full paper (chapter, etc.)?
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification