Hello people, I have a dataset of inhibitors as binary labels ( Zeros - Inactive , Ones - Active ). I have my ML/AI model working, now I would like to know how many are the best inhibitors out of these. Could anyone help me what should I do and what can be done to resolve my problems?
TIA
#DrugDesign #ML #AI #DataScience #DrugDiscovery
Dear Thiliban Manivarma , If you have used an ensemble decision tree such as a Random Forrest or a gradient boosting algorithm such as the XGBClassifier, you have the built in method "feature_importances_" that will give you the feature importance against your Label variable as a plot.
Alternatively, you can also use correlation to find the correlation between the features and the Label and also the cardinality of the features themselves.
It sounds like you're working on a binary classification task with a focus on identifying the best inhibitors. Here's a step-by-step approach to help you assess and further refine your model to get the results you need:
1. Model Diagnostic Assessment:
- Confusion Matrix: Construct a confusion matrix to elucidate true positive, true negative, false positive, and false negative categorizations from model predictions.
- Performance Metrics: Determine precision, recall, F1-score, and AUC-ROC to critically assess model accuracy and effectiveness.
- ROC Curve Analysis: This graphical representation delineates the compromise between sensitivity and specificity.
- Threshold Refinement: Many algorithms conventionally employ a 0.5 threshold for classification. An adjusted threshold may be imperative to either augment recall or precision. Such adjustments might be pivotal for precise inhibitor identification.
2. Examination of Feature Significance:
- Should the model possess inherent capabilities (e.g., tree-based methodologies), it's pertinent to scrutinize feature importance scores, offering insights into the most influential features for active inhibitor prediction.
- For models devoid of direct feature significance outputs, one might consider employing techniques such as Permutation Importance or SHAP values.
3. Model Refinement Strategies:
- Resampling: In the presence of a class imbalance in the dataset, methodologies such as oversampling, undersampling, or the Synthetic Minority Over-sampling Technique (SMOTE) should be explored.
- Hyperparameter Optimization: Techniques encompassing grid search or random search should be invoked for optimal hyperparameter tuning tailored to the task at hand.
- Cross-Validation Strategy: Implementation of k-fold cross-validation is advised to yield a comprehensive model performance assessment.
4. Inhibitor Ranking Framework:
- Probabilistic Outputs: Rather than binary outcomes, it is beneficial to procure probability scores from the model. Inhibitors manifesting elevated probabilities of activity might be deemed as the most potent.
- Subsequent Analysis: Following the demarcation of paramount inhibitors, a deeper analysis is advocated, potentially emphasizing their molecular characteristics or mechanistic pathways.
Best regards,
Samawel JABALLI
Lakshminarayana Rao Malyala thanks for your time and suggestions. I will try to do that and let you know here.
Samawel Jaballi I get the overall idea of what you are saying, I appreciate it :)