The achievement of high-quality results of deep learning is possible by appropriately setting of its parameters. The number of hidden layers and the number of neurons in each layer of a deep machine learning have main influence on the performance of the algorithm. Some manual parameter setting and grid search approaches somewhat ease the users' tasks in setting these important parameters. But, these two techniques can be very time-consuming.
in my opinion is the best metaheuristic technique for optimization of Support Vector Machine (SVM) because Hyper-parameter tuning for support vector machines has been widely studied in the past decade. A variety of metaheuristics, such as Genetic Algorithms and Particle Swarm Optimization have been considered to accomplish this task. Notably, exhaustive strategies such as Grid Search or Random Search continue to be implemented for hyper-parameter tuning and have recently shown results comparable to sophisticated metaheuristics. The main reason for the success of exhaustive techniques is due to the fact that only two or three parameters need to be adjusted when working with support vector machines. We analyze two Estimation Distribution Algorithms, the Univariate Marginal Distribution Algorithm and the Boltzmann Univariate Marginal Distribution Algorithm, to verify if these algorithms preserve the effectiveness of Random Search and at the same time make more efficient the process of finding the optimal hyper-parameters without increasing the complexity of Random Search.
TPE is one of the popular hyper-parameter optimization algorithm widely used now in DL. You can find a python implementation to go with keras at https://maxpumperla.com/hyperas/
One other algorithm that may of of interest for hyper-parameter optimization in both SVMs and NNs is CMA-ES (https://en.wikipedia.org/wiki/CMA-ES)
depend on my work experience with SVM, I think SVM is the best, because it gives maximum separating hyperplane
hi,
you can use grid search approach to set and optimize the hyper parameters
Article Parameter investigation of support vector machine classifier...
This paper will explain in details your question.
You can use any multi-objective evolutionary algorithms to optimize the parameters of SVM and MLP
Ant Colony Optimization and Particle Swarm Optimization techniques, you can read this article to get more information.
J. Blondin and A. Saad, "Metaheuristic techniques for Support Vector Machine model selection," 2010 10th International Conference on Hybrid Intelligent Systems, Atlanta, GA, 2010, pp. 197-200. doi: 10.1109/HIS.2010.5600086
you don't need to apply any metaheuristic methods to optimize MLP just read the following paper
PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks
For SVMs, the optimal hyper-parameter tuner is an Estimation of Distribution Algorithm (EDA), according to this work: Article Optimal Hyper-Parameter Tuning of SVM Classifiers with Appli...
There is no free lunch theorem, it depends on the nature of the data under consideration.
Best regards
- Badges
- Science topic