Hello all,
I would like to ask about how can I perform wilcoxon rank test to test two optimization algorithms. If I run algorithm PSO 50 runs and GA 50 runs also and the two algorithms are repeated 1000 times, Should I take the best results of 50 runs or all iterations, if I use ranksum ?
I think you should first read the attached paper about "practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms". It is the best source.
It is better to check this paper and its references more and more.
you can use SPSS and MATLAB ([p,h,stats]=ranksum(x,y)).
you can also check these links and find your answer. Perhaps it helps.
https://www.mathworks.com/help/stats/ranksum.html
https://www.researchgate.net/post/How_might_I_perform_two-sided_wilcoxon_signed-rank_test
I think you should first read the attached paper about "practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms". It is the best source.
It is better to check this paper and its references more and more.
you can use SPSS and MATLAB ([p,h,stats]=ranksum(x,y)).
you can also check these links and find your answer. Perhaps it helps.
https://www.mathworks.com/help/stats/ranksum.html
https://www.researchgate.net/post/How_might_I_perform_two-sided_wilcoxon_signed-rank_test
You can make it in matlab using ranksum function:
Example:
AlgoA=[0.78, 0.76,0.73,0.89,0.76,0.76,0.73,0.87,0.78,0.70]
AlgoB=[0.88, 0.62,0.83,0.83,0.76,0.76,0.83,0.72,0.81,0.72]
pValue=ranksum(AlgoA, AlgoB)
The result is: 0.9088
Best Regards,
Ibrahim
You may also use Python & NumPy - https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wilcoxon.html - which I greatly prefer.
Dear Yousef Abdi,
According to your explanation below:
If p-value >= 0.05 , both of the results of algorithms are almost the same. [If the results are completely same you will get 1 for p-value ! ]
If p-value < 0.05 , the results of algorithms are significantly different.
If p-value equal for example 1.23*10^-23. Any algorithm be best in this situation and Any value of p that indicates that the algorithm is best.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Algorithms