For a specific application, I don't have access to the ground truth. I had to train a predictive model and its accuracy is about 70%. I give the output of each algorithm to the predictive model and it predicts how much this output is good. Is it correct to compare algorithms using this approach?
It might not be able to measure the actual accuracy of the algorithms, but if your dataset is large enough you might be able to use it to compare the algorithms. So you won't know how well each algorithm performs, but you will know which algorithm is likely to perform better on the real ground truth. Still, that is crude because what it actually measures is how the algorithm you test is aligned with the model you used. In any case, that will not work when your algorithm's accuracy is close to or higher than the accuracy of your model.
A simulated annealing aporoach may be helpful.
Article Feature-based tuning of single-stage simulated annealing for...
- Badges
- Science topic
- Similar topics
- Geoscience
- Asia
- India