I have a training set consisting of 21 compounds. Here is a short code to calculate LOO q2:
from sklearn import linear_model
lm = linear_model.LinearRegression()
from sklearn.model_selection import cross_val_score
cvs=cross_val_score(lm, X_train, y_train, cv=21)
mean_cross_val_score = cvs.mean()
mean_cross_val_score
but it gives a mistake: UndefinedMetricWarning: R^2 score is not well-defined with less than two samples.
Please, help me to calculate q2 value for leave-one-out cross-validation using scikit-learn! I need to calculate q2 for SVR and linear regression models.
Dear Mohammed,
thank you for the comment. I know how to perform leave-many-out cross validation, but I need LOO validation.
Hello Andrey,
By definition, it is not possible to measure r2 from LOO since it scores only one instance in each iteration then averages all scores. However, r2 requires at least two instances to calculate.
A simple solution, you can collect predictions from each iteration then you can calculate r2 together.
I hope the following code helps:
from sklearn import linear_model
from sklearn.model_selection import cross_val_predict
from sklearn.metrics import r2_score as r2
# Assuming you have X_train and y_train in a correct form
lm = linear_model.LinearRegression()
y_pred = cross_val_predict(lm, X_train, y_train, cv=len(X_train))
r2_score = r2(y_pred,y_train)
Note: Since linear regression is resistant to overfitting you can train and test using all data.
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Programming Languages
- Python