I'm a master's student in Political Science researching presidential interruptions; and, in order to analyse these events, I'm running logistic regression. The thing is, I'm not sure if I should include performance metrics (confusion matrix and precision-recall curve for a model with 18 positive and 86 negative events) in the final paper. It's not a machine learning type of work, and I've never seen a researcher in my field using these kinds of metrics. Are they important for a social science research not related to data science?
Thanks!
some researchers use their own thinking to calculate these metrics manually. but according to my opinion, we should use standard formulas to estimate these numbers rather than just manual calculations in Excel. secondly, every machine learning model needs verification and validation on the basis of these metrics so it should be considered. Thank you.
Breno,
In general, the metrics such as ROC, AUC and the indexes of Precision, Sensitivity, F1 recall etc., have been seen to be implemented for machine learning classification results.
That being said, it is not limited/restricted to machine learning at all. As you describe, you have two classes of data. Thus, you will end up with a 2x2 confusion matrix (TP, TN, FP and FN) which can be used to interpret the performance of LR for better understanding. So, using Precision, Recall etc is certainly plausible
Moving to the second point, AUC gives the rate of successful classification a model (which, in your case is the Logistic Regression). And to be frank, LR is susceptible to overfitting. Thus, you can use AUC and/or ROC to demonstrate that.
And how useful are these indexes in Social Science research, I cannot say as I am not from the field. But my honest guess would be, by using such indexes, you can make it easier to understand your work for a reader.
EDIT: If you're looking to publish some of your findings in a paper, I would suggest you to stick to the ones used in your field. Because by using new indexes without justifying why you're using them might create confusion for the reviewers
PS: You can find some more information here: https://stats.stackexchange.com/questions/71946/overfitting-a-logistic-regression-model
Good luck!
Attatched are examples of what I did in a couple of similar situations. Best, David Booth
- Badges
- Science topic