First, apologies for my poor English. I am working with incremental learning on concept of drift problem.
Standard versions of the classifier have not incremental learning. There are some algorithms for incremental learning, for instance into Weka API there are implementation of Hoeffding tree that have update classifier method.
But my question is this. We have a classifier builded with train dataset. To make a pseudo-incremental learning with this classifier is correct reset the classifier and rebuild it with train dataset plus new instance for each new instance?
For instance:
J48
Dear Albert,
Folami is right:
1. such method is very costly, therefore if you need the incremental tree model, then better use VFDT or CVFDT (if drift occurs) or any model from the dt family - see a short review on wikipedia https://en.wikipedia.org/wiki/Incremental_decision_tree
I reccomend you also work http://www.jmlr.org/papers/v11/ben-haim10a.html
2. if you really need the C4.5, then maybe you may speed up the retratining by using parallel mode or use parallel dt learning method as SLIQ or SPRINT modification.
3. Backing to the idea of Folami, you may also use so-called drift detector which informs you if the changes in data distribution are so significant thet model should be rebuild.
best regards,
Michal
P.S. If you would like to discuss some issue contact with me [email protected]
.
this will be terribly costly (retrain a model for each new instance !) : if your inter-arrival time for instances is smaller than the training time, you cannot keep up with your stream
you might mitigiate this by retraining only when you have gathered a large enough batch of new instances
moreover, your training set increases without bound and therefore your training time ... to say nothing of your memory resources
these are precisely the difficulties "proper" incremental learning aims at solving : keep up with a fast stream with low memory requirement
.
so ... "pseudo" incremental, yes in the sense that you will have a model which changes in time as more data comes in, but this implementation does not consider the "true" problem of incremental learning (again, keep up with a fast stream with low memory requirement)
.
Adding to what Fabrice said, it may be worthwhile exploring more than quantity of new training data but in addition, some other metric, which can capture the "information gain" or model improvement given the new data. If you for example set your incremental training to retrain after receiving 1k new samples, it may be an exercise in futility if the new data adds nothing to the model.
If you can find a computationally efficient method to evaluate how much information is added to the training set by the new data, then you can retrain based on both volume and the gain metric.
Dear Albert,
Folami is right:
1. such method is very costly, therefore if you need the incremental tree model, then better use VFDT or CVFDT (if drift occurs) or any model from the dt family - see a short review on wikipedia https://en.wikipedia.org/wiki/Incremental_decision_tree
I reccomend you also work http://www.jmlr.org/papers/v11/ben-haim10a.html
2. if you really need the C4.5, then maybe you may speed up the retratining by using parallel mode or use parallel dt learning method as SLIQ or SPRINT modification.
3. Backing to the idea of Folami, you may also use so-called drift detector which informs you if the changes in data distribution are so significant thet model should be rebuild.
best regards,
Michal
P.S. If you would like to discuss some issue contact with me [email protected]
You may also be interested in knowing the MOA software environment, an extension of Weka for online learning.
http://moa.cms.waikato.ac.nz/details/classification/classifiers-2/
http://www.jmlr.org/papers/volume11/bifet10a/bifet10a.pdf
Read the following paper for true incremental learning approach:
Article Parametric Decomposition of Sample Space for Classification
- Badges
- Science topic
- Similar topics
- Computer Science
- Data Mining