How can final set of features be selected from multiple rank lists obtained by different ranking method?
It's a hard problem, but a simple solution might work well. This paper looked at a range of different methods in one context (ranked gene lists from microarray experiments) and found that the mean of the ranks was as good as anything else, and was reccomended due to the ease of computation:
http://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS13/paper/viewFile/5893/6111
It's a hard problem, but a simple solution might work well. This paper looked at a range of different methods in one context (ranked gene lists from microarray experiments) and found that the mean of the ranks was as good as anything else, and was reccomended due to the ease of computation:
http://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS13/paper/viewFile/5893/6111
.
mean rank == Borda method
(simple and you have to live with the Condorcet paradox)
depending on the application domain, different heuristics will perform differently !
.
Thank you Fabrice and Jonathan for providing your suggestions.
Actually I'm facing this problem very frequently and commonly used Borda method. I feel the inherent limitation of Borda method is fixed weights applied to the ranking of elements while preparing common list. I personally feel that an ideal solution to this problem would be an algorithm implementing Borda method but with varying weights changing/optimizing continuously like weights in ANN by feed back while evaluating several ranked lists.
Any suggestions regarding how I can proceed with this idea? or any major drawback in this idea?
.
if you have a fast way to evaluate a ranking a posteriori, this may be a way to go and you might be interested in the attached paper
also, it is often (depending on the context, once again) useful to weight top results more than the others in the spirit of NDCG and then go for Borda aggregation
http://http://en.wikipedia.org/wiki/Discounted_cumulative_gain
.
@Walid An approach I have used to choosing a cutoff goes something like this:
Assuming that each method is reasonably good at choosing a significantly 'best' subset, then make a 'best' subset from each method, and look at the characteristics of these sets - how much do they overlap? How much does each method agree with others? Do some methods produce large and some small 'best' lists?
Then make a cutoff such that the resulting 'best' subset only contains members considered significant by at least k of the different ranking methods. If you choose k=1 (each member must be considered significant by at least one ranking method) then you get a nice broad subset (maximum sensitivity). If you choose k=total number of ranking methods then you get the most confident subset (maximum specificity).
You could use the same FDR threshold for each, adjusted depending on whether you want a sensitive or specific test?
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Learning