We are working on a predictive model using Lasso to evaluate the validation cohort using publicly available data.
1. ROC-AUC 0.800 (95%CI 0.551-1.000)
2. ROC-AUC 0.750 (95%CI 0.350-1.000)
3. ROC-AUC 0.575 (95%CI 0.317-1.000)
I am wondering how to describe the evaluation of (2): please let me know if I can describe (1) and (2) as being well predicted statistically. Since the lower 95% CI limit is 0.35, despite the high ROC-AUC of 0.75, shouldn't we describe the evaluation of the model positively? thank you so much for your support.
Do you try to implement the parameter tuning via a Grid Search? With the Grid search and an additional line of code, the program will automatically select the optimal model. To make your algorithm more transparent you can obtain a graph or list of all models run and their values for each parameter and evaluation metric. After this stage, you run your final model with the optimal parameters and you can interpret it and take decisions according to your research question(s).
As far as I know, does matter what method involving ML, if the AUC value is higher than 0.8, the model consider as has good predictive model.