To develop a prediction model using different machine learning techniques, we need to divide our samples in calibration and validation samples. On what grounds we can divide the those samples.
see https://www.r-bloggers.com/2020/03/simulating-your-model-training-for-robust-ml-models/
I would recommend stratified sampling to avoid unbalanced dataset and bias issues.
Please see the following links. Scikit-Learn guide provides detailed explanation and analysis related to sampling and dataset splitting.
1. https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html
2. https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedShuffleSplit.html
3. https://scikit-learn.org/stable/modules/cross_validation.html#stratified-shuffle-split
I hope that I provided informative and beneficial information.
Good Luck!
- Badges
- Science topic