For making a linear regression of a training set for a test set, the test error increases when the sample size is reduced.
Is it correct that this simply occurs because there is more data available and thus the regression can be made more precisely or is there a different underlying reason?
The sample size and overfitting can be issues. To combat the problem, you can use leave-one-out cross-validation or k-fold cross-validation.
The error term is estimated by the expression, Sum of Squares for Error divided by the Degrees of Freedom for Error. Assuming that the variability of the data set is somewhat stable, this means that the size of the error term is dependent on the degrees of freedom for error. The degrees of freedom for error is the sample size reduced by the number of parameters in the model. Thus, the larger the sample size, the larger the degrees of freedom for error (df(error)). Since the df(error) is in the denominator of the estimate, the larger it is, the smaller the error term will be. Thus, the best way to increase the sensitivity of your test is to reduce the error term by increasing the sample size.
A very good reference is : https://b-ok.cc/book/5243562/0e8c4b
Best wishes, David Booth
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training