.

sorry, i did not read your question carefully enough ; the following references are recent works around L1O

.

(there is a short historical note in https://www.researchgate.net/profile/Massimiliano_Pontil/publication/228597518_Leave-one-out_error_and_stability_of_learning_algorithms_with_applications_Stability_of_Randomized_Learning_Algorithms_Source/links/0912f50fd2561b4db2000000.pdf but it does not seem to point to specific medical applications)

.

https://arxiv.org/abs/0907.4728

A survey of cross-validation procedures for model selection

Sylvain Arlot, Alain Celisse

Used to estimate the risk of an estimator or to perform model selection, cross-validation is a widespread strategy because of its simplicity and its apparent universality. Many results exist on the model selection performances of cross-validation procedures. This survey intends to relate these results to the most recent advances of model selection theory, with a particular emphasis on distinguishing empirical statements from rigorous theoretical results. As a conclusion, guidelines are provided for choosing the best cross-validation procedure according to the particular features of the problem in hand.

.

https://arxiv.org/pdf/1608.06412.pdf

Stability revisited: new generalisation bounds for the Leave-one-Out

Alain Celisse, Benjamin Guedj

The present paper provides a new generic strategy leading to non-asymptotic theoretical guarantees on the Leave-one-Out procedure applied to a broad class of learning algorithms. This strategy relies on two main ingredients: the new notion of Lq stability, and the strong use of moment inequalities.

Lq stability extends the ongoing notion of hypothesis stability while remaining weaker than the uniform stability. It leads to new PAC exponential generalisation bounds for Leave-one-Out under mild assumptions. In the literature, such bounds are available only for uniform stable algorithms under boundedness for instance. Our generic strategy is applied to the Ridge regression algorithm as a first step.

.

http://math.univ-lille1.fr/~celisse/Papers/knn_celisse_maryhuard.pdf

Theoretical analysis of cross-validation for estimating the risk of the k-Nearest Neighbor classiffer

Alain Celisse, Tristan Mary-Huard

The present work aims at deriving theoretical guaranties on the behavior of some cross-validation procedures applied to the k-nearest neighbors (kNN) rule in the context of binary {0,1}-classification. Here we focus on the leave-p-out cross-validation (LpO) used to assess the performance of the kNN classifier. Remarkably this LpO estimator can be efficiently computed using closed-form formulas derived by Celisse and Mary-Huard (2011).

We describe a general strategy to derive exponential concentration inequalities for the LpO estimator applied to the kNN classiffier. This relies on deriving upper bounds on the polynomial moments of the centered LpO estimator by first deriving such bounds for the leave-one-out (L1O) estimator. Such results are obtained by exploiting the connection between the LpO estimator and U-statistics as well as by making an intensive use of the generalized Efron-Stein inequality. One other contribution is the extension to the LpO of the consistency results previously established by Rogers and Wagner (1978) for the L1O as an estimator of the risk and/or the error rate of the kNN classifier.

.

(... and other references from the same group of authors)

.

Article Leave-one-out error and stability of learning algorithms wit...

Fabrice, thank you for the nice overview. Will be looking into the references.