Concept drifting is an issue of all machine learning, I wouldn't consider it as ensemble-learning-specic problem.

Cornerstone of ensemble-learning is problem of diversity. The ideal ensemble systems should be accurate as possible and make errors on different parts of the input space => classifiers should be diverse. Although this definition is logic and clear, there are number of different measures of diversity and their relationship with ensemble accuracy is not straightforward.

The methods of construction of the ensemble systems that attempt to achieve the highest diversity by intuitive way, i.e. by weakening the weights of similar classifiers such as AdaBoost, work significantly better than the basic methods such as simple majority vote approach. Surprisingly, the experiments proved that the measuring of diversity and trying to explicitly use it during the iterative process of construction of the ensemble system has not worked nearly as well.

There are several possibilities how to achieve the highest diversity and build up classifiers with different decision boundaries:

- Different training datasets: Training dataset is partitioned into several subsets and each subset is used for learning of one classifier. The partitioning can be performed through resampling techniques such as bagging or bootstrapping which draws training data subset with replacement. Another possibility is to use methods which divide training data subsets without replacement, such as jack-knife method.

-Different training parameters: Each type of classifier usually offers to set up some initial parameters. For example: In case of multilayer perceptron neural networks, it can be number of layers / nodes or number of error goals.

- Different types of classifiers: Certain types of classifiers are more or less suitable for different types of problems. However, there is usually not only one suitable type of classifier at all, so choice of several of them and combination of their outputs could be also way how to build up set of classifiers with different decision boundaries.

I would recommend you to study at least several general articles about ensemble learning, i.e.:

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1688199&tag=1

http://dl.acm.org/citation.cfm?id=1713730

For deeper understanding of principles and concepts, read this excellent book:

http://dl.acm.org/citation.cfm?id=975251

Hope it helps!

Concept drifting is an issue of all machine learning, I wouldn't consider it as ensemble-learning-specic problem.

Cornerstone of ensemble-learning is problem of diversity. The ideal ensemble systems should be accurate as possible and make errors on different parts of the input space => classifiers should be diverse. Although this definition is logic and clear, there are number of different measures of diversity and their relationship with ensemble accuracy is not straightforward.

The methods of construction of the ensemble systems that attempt to achieve the highest diversity by intuitive way, i.e. by weakening the weights of similar classifiers such as AdaBoost, work significantly better than the basic methods such as simple majority vote approach. Surprisingly, the experiments proved that the measuring of diversity and trying to explicitly use it during the iterative process of construction of the ensemble system has not worked nearly as well.

There are several possibilities how to achieve the highest diversity and build up classifiers with different decision boundaries:

- Different training datasets: Training dataset is partitioned into several subsets and each subset is used for learning of one classifier. The partitioning can be performed through resampling techniques such as bagging or bootstrapping which draws training data subset with replacement. Another possibility is to use methods which divide training data subsets without replacement, such as jack-knife method.

-Different training parameters: Each type of classifier usually offers to set up some initial parameters. For example: In case of multilayer perceptron neural networks, it can be number of layers / nodes or number of error goals.

- Different types of classifiers: Certain types of classifiers are more or less suitable for different types of problems. However, there is usually not only one suitable type of classifier at all, so choice of several of them and combination of their outputs could be also way how to build up set of classifiers with different decision boundaries.

I would recommend you to study at least several general articles about ensemble learning, i.e.:

http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1688199&tag=1

http://dl.acm.org/citation.cfm?id=1713730

For deeper understanding of principles and concepts, read this excellent book:

http://dl.acm.org/citation.cfm?id=975251

Hope it helps!

Dear Jaroslav,

Thank you very much for your excellent reply. It helped me a lot to understand the problems associated with EoC.

However, I am bit confused about the type of ensemble creation. Would you suggest any good article about taxonomy of ensemble?

The main types of ensemble learning are described in chapter 3 of dr. Polikar article (mentioned in my first answer). Although the choice of ensemble is the subject of discussion and it is somehow problem-specific, I usually prefer Adaboost since its good resistance against overfitting. Beside the reading of the articles, I recommend you to try several types of ensemble systems on your training & testing datasets and evaluate the results. If you are not using any machine learning platform, I suggest you to use WEKA - although there are other possibilities, see discussion: https://www.researchgate.net/post/Whats_your_basic_tool_suite_for_the_academic_research_on_machine_learning