In the context of machine learning and statistical analysis, separating a test dataset from the original dataset is a crucial step in model validation. It ensures that a model's performance is robust and generalizes well to new data.

• The original dataset is typically split into two parts: the training set and the test set. The model is trained on the training set and then tested on the unseen data (test set) to evaluate its performance.
• The purpose of using a test dataset is to prevent overfitting, where the model performs well on the training data but poorly on new, unseen data. This process provides a more accurate assessment of the model’s predictive power.
• Conclusion: Properly splitting the dataset is crucial to ensure the model is not only accurate but also reliable in real-world applications. In sustainability analytics, for instance, this could be vital for predicting environmental impacts or assessing long-term financial risks.

From my side, Your approach to handling the dataset and model training/testing is generally solid, with a few key considerations for improvement. You started with 12 lakh records, selecting 1 lakh for analysis and creating a test set of 10,000 records. You divided the remaining 90,000 into 9 batches for training (9,000 records) and validation (1,000 records), performing parameter tuning on the validation sets before testing on the separate test dataset.

To enhance your methodology, ensure that your test set is completely independent from training and validation data to avoid data leakage. Consider using k-fold cross-validation for more robust evaluation and make sure to evaluate your model on the reserved 20% test set after training on the full dataset. Additionally, track performance metrics like accuracy and F1-score throughout the process. Overall, your method is well-structured; addressing these considerations will strengthen your model evaluation. Good luck!

Thank you Nur RIZQIYAH Febriani