What would be the suitable ML methods for a low dimensional dataset e.g. 3000 x 5 (3000 records, 4 input variables and a single output)? Will Bayesian regularization methods work well due to its ability in regularizing?
For smaller datasets SVM can produce better results... By default SVM in MATLAB uses linear kernel but you can change the kernel like Gaussian, Polynomial, RBF or any custom in order to produce even better results. Hyperparameters optimization is a key to yields good output in ML model. With SVM you can optimize Box Constraint, Gamma Value, Polynomial order etc.. For automatic searching of hyperparameters you can use Random Search, Grid Search or Bayesian Optimization.
Regularization is mostly used to prevent overfitting and Bayesian regularization is mostly used with neural network. In SVM you can optimize the hyperparameters to prevent over fitting. You can read my answer on hyperparameters optimization here https://www.researchgate.net/post/Suitability_of_evolutionary_algorithms_for_hyperparameter_optimization
- Badges
- Science topic