Consider I have 3 groups: 1, 2 and 3. In each group I have 3 species. 1A, 1B, 1C; 2D, 2E, 2F and 3G, 3H, 3I. It is possible to predict where groups occur given some predictors. However, species in group 2 are more "generalistic" and have the tendency to occur in the other groups. While I could weigh group 1, 2 and 3 lets say 2/1/2, I actually want to weigh all species 2 times more except for 2D and 2F, which I only want to weigh 1 time. Lets say I predict group 1 at a specific site, and at this site the species composition is 1A, 1B, 2D this prediction is for 2/3 correct, but I do not see the occurrence of species 2D occurring in group 1 as very problematic if the weights of the species in this group was 2/2/1 than the model would be 4/5 correct. Is there any way to perform this with random forest models?
Yes, it is possible to do this, but I'm not sure if there is a library out there that has this functionality implemented. You might have to code it yourself. The idea would be something like this:
Random Forests is basically a collection of decision trees. You train each tree on a random subset of instances and features. Once trained, you make a prediction with every tree and predict the average response of each tree.
To have the behaviour that you describe, you would need to modify how the individual trees are learnt. Usually you train a tree by choosing which node to split. To do this, you compute a 'splitting measure' for each feature (gini index is a common option). If you have five features, then compute the gini index per every feature, and then split the tree on the feature with the lowest index. Then repeat for the remaining features...
For computing this gini-index you can assume that your three can make one out of three possible predictions, each prediction contains 3 entries:
(1) [ 1A, 1B, 1C]
(2) [ 2A, 2B, 2C]
(3) [ 3A, 3B, 3C]
The ginni index is computed as G= 1 - (p_1)^2 - (p_2)^2 - (p_3)^2
Usually you compute p_i as the proportion of elements of each class that you have in every node, after splitting. In your case, you would get this number using the computation that you described in your question. In other words, you would be using a 'weighed proportion'.
Roberto Vega To simplify this for myself, you mean than I would need to adjust my gini, given I known which species and corresponding weights are in the specific groups? So, multiplying the proportion (pi) of the group with the weights assigned to the species included in the proportion (group): G= Ʃ[(1 - (p1*w1-p2*w2- p3*w3)2 ) + (1 - (p1*w1-p2*w2- p3*w3)2 )], where + denotes the split-point. Then in a formula expression for an R code this could be y*w ~ xi+...+xn. Where y indicaters the groups and w the weight of the species in that group. The solution is than rater simple although, I am not sure about the way getting there.
Roberto Vega after further pondering on this "issue", this "solution" only impacts the location of split-points not the classification error. Or, am I wrong?
Wim Kaijser you would be finding the splitting points that minimize the weighted classification error, so it will have an impact in the classification error. As for the other question, I'm not very familiar with R, so I don't know exactly how to write the proper expression there. If I had to solve this problem, I would create my own implementation of a decision tree in python using the custom version of the gini index (or any other splitting measure)
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification