In ANN, what is the difference between test data and validation data?
I notice in many training or learning algorithm, the data is often divided into 2 parts, the training set and the test set.
My questions are:
what is the difference between validation set and test set?
Is the validation set really specific to neural network hidden layer only ?
Validation data: you check error of your training and stop training if e.g. loss is increasing on validation data. So your training is optimized to the validation data as well (but not trained).
Test data: you test your final result. Training has nothing to do with it.
To make this really clear, there are three different data sets in ANN. Training, Validation and Testing, in the same order.
Training is the data set used to adjust the weights of the nodes to build the model initially. This data is crucial as it should reflect all of the heterogeneity of the problem. Usually this proportion is around 70%, but varies with the nature of the dataset and the model.
Validation is a step to arrest overfitting or under fitting of the model against the next set of data (usually around 20-30%). This procedure where the weights are checked for accuracy in the model prediction. There will be no adjustment in the weights. However, based on these results, the training procedure and parameters could be optimized. For example, the learning rate could be tweaked to achieve high precision. This data can emanate from the same set of data as long as it is properly randomized and proportioned.
The test data is used to check the final performance of a developed model. The data set ideally should be independent of the training and validation set. The model is now allowed to predict with inputs from a new set of data. Therefore, the results generated will reflect the model's success/failure. This will be your real world implementation. When your model under performs with a new data rounds of calibration procedures then follow.
Hope this helps.
- Badges
- Science topic