I have carried out a work on object detection specifically to detection of pedestrians using YOLO v1, v2, v3, v4 and tiny YOLO v1, v2, v3 and v4 algorithm and proposed a new variant of YOLO algorithm. The employed and proposed variants of YOLO have been evaluated using precision, recall, f1 score, AP for each class and map value.
I submitted the same work to a journal and in response reviewer has asked for carrying out statistical analysis using Friedman Aligned Rank Test, Wilcoxon Test, Quade Test etc and find out the P value.
Is it possible to apply such tests on an object detection problem? Here the training and testing have been done on images. How to apply such statistical tests. Please guide.
I am not sure if all the tests you mentioned ( I assume they are nonparametric tests) apply to your problem. But if you are testing the effect of one or more variables on a system then you must be able to perform statistical tests on those variables. Just try considering your accuracy, f1-score, precision, and recall as dependent variables and assume your object detection model as a black box. This way you can compare the values of inputs with output, or the output values of different object detectors.
I have used paired sample t-tests (a parametric test) on the YOLO model by comparing its accuracy and speed while training with different input data, and based on that experience I am answering your question that any other tests may also be possible if you can correctly identify what variables you want to test and what association you want to explore in your model. Also, I used SPSS software to perform the statistical tests. Hope this helps...
You may need to understand your problem and also improve your knowledge of these statistical tests to correctly choose your input and output data with a relevant test for your problem.
Read this https://www.scribbr.com/statistics/statistical-tests/ to increase your knowledge about parametric and nonparametric tests.
Akhil Kumar I'm not sure what was the goal of your research and whether you tested the same group of people in different conditions or compared different groups. The choice of statistical method depends on it.
Akhil Kumar
Yes, it is possible to apply statistical tests such as Friedman Aligned Rank Test, Wilcoxon Test, Quade Test, etc. on an object detection problem. These tests are commonly used to compare the performance of multiple algorithms and to determine the statistical significance of the results.
To apply these tests, you would first need to compare the performance of each variant of YOLO (YOLO v1, v2, v3, v4, and tiny YOLO v1, v2, v3, and v4, as well as the proposed variant) on a specific evaluation metric, such as precision, recall, F1 score, or Average Precision (AP). The data for each metric can be organized in a table, with each row representing a different variant of YOLO and each column representing a different image or set of images used for evaluation.
Next, you would perform the Friedman Aligned Rank Test to determine if there is a significant difference in the performance of the variants of YOLO. This test involves ranking each variant of YOLO based on its performance on each evaluation metric, and then comparing the rankings to determine if there is a significant difference.
If the Friedman Aligned Rank Test indicates a significant difference, you can then use the Wilcoxon Test or Quade Test to compare pairs of variants of YOLO to determine which algorithms perform significantly better than others. These tests involve comparing the performance of each variant to a reference algorithm, and determining the P-value, which indicates the likelihood of observing the results if there was no difference in performance between the algorithms.
It is important to note that these statistical tests require a large sample size in order to accurately determine the significance of the results. If the sample size is small, the results may not be reliable, and it may be necessary to collect additional data to increase the sample size.
Thanks for all the replies. I have done this long back. Thanks for your valuable suggestions.
- Badges
- Science topic