Dear Sami Belkacem ,

First of all, I'm not considering the essence of your data (chronologically ordered). Regarding the grid search, you can do this using sklearn library. See the links:

Note that in hold-out you will need to build an additional set (validation set) required by grid search. This can be done using the train_test_split in sklearn:

https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html

I hope this be useful for you.

Dear Sami Belkacem,

I agree with Artur Jordão, but I recommend you read the paper:

"Cross-validation pitfalls when selecting and assessing regression and classification models" (https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-6-10)

This interesting paper explains some common pitfalls in the cross-validation process and how to perform parameter tuning, it can be very helpful in your problem.

I hope this be useful for you.

Dear Artur Jordão

Thank you for your answer, I already know the links you provided.

As far as I know, the first two links are for Grid search with k-fold cross-validation.

The last link is for the Hold-out validation but without Grid search.

I am asking about the Grid search with the Hold-out validation.

Best regards,

Sami Belkacem ,

I'm not understanding the problem. For instance, apply hold-out to generate the sets of training, validation and testing. Then employ grid search (learning stage) on training and evaluate each possible configuration using validation. Finally, use the best configuration on the testing set.

The steps above can be executed using the train_test_split and grid search from sklearn.