Hi David,

An overall interpretation:

Seems like your model has best performance with 10 instances (approximately) where λ=0.01, and adding additional instances will not enhance the performance that much. Knowing that, this understanding is based on the evaluation of cross-validation method only. Because building a prediction model based on the training set is a sort of cheating. 

HTH.

Samer

Hi David. I agree with Samer. However, generally, neural networks requires high number of samples. If you have more the 12 I think you should try. I do see that at 10 the performance is better than 12, but that does not mean 30 or 40 would be worse.

Regarding Lambda, Usually lambda range should be small (depending on your regularisation algorithm) like between 1e-10 to 1 

All the best

.

as said above, your Lambda must be small ! (0.01 may be fine, 10 is certainly not reasonable)

with so few training samples, you'd better not increase your hidden layer

.

what follows may add some confusion : with so few training samples, i would rather use a grnn than a mlp

if you want to have a look at grnn, you may start here

https://minds.wisconsin.edu/bitstream/1793/7779/14/ch2.pdf

(grnn is available in R ... among others)

.

Thank you Samer, Ahmad and Fabrice for your comments and suggestions.

I totally agree with you that the size of the training set in this example is very limited. I took this data set because I already had it from some exercise of the machine learning Coursera course (highly recommended for novices like me) where I had to use neural network to solve a logistic regression problem. In "real life", my data set is going to be larger. 

The reason why I was also plotting the learning curve for lambda=10 was for pure comparison with  the results with lambda=0.01. I totally agree that the results for lambda = 10 do not look good. My question there was whether the learning curve for lambda = 0.01 was ok. I saw a good trend of both cross-validation and training sets.

Regarding the validation curve (error vs. regularization parameter lambda), I was trying to identify the value of lambda where I reach the minimum of the cross validation error, but it is not clear to me whether there is a good candidate for that minimum.

Again, thanks for your comments and suggestions.

Best,

David

Dear David,

Your validation curve (error vs. regularization parameter lambda) suggests that the regularization term (lambda) does not seem to help with improving the validation error of your neural network model; that is, with no regularization (at lambda = 0) the model yields least training and validation errors. This might be due to the simple nature of your model and dataset used. Generally, regularization terms are more important for large models that have high propensity to over-fit. For simple models, regularization terms may even lead to under-fitting when not cautiously used! In any case, very small values of lambda are used, say in the range 10-3 - 10-5.

Hi everyone, 

I have updated the learning and validation curves using a much larger dataset. The question is reformulated at:

https://www.researchgate.net/post/Learning_curve_and_validation_curve_in_neural_network

Any comments and suggestions are more than welcome.

Thanks in advance,

David