Neural networks may provide the single platform for many statistical applications like time series forecasting, pattern recognition, classification, function approximation etc. but some conventional models out performing the neural networks. Neural networks are alternative methods for the conventional methods. Are these neural networks sub-optimal procedures? How to improve the precision of the neural networks models when it is used for function approximation.
I think that the answer makes no sense because there is no single "neural network" model. There is a huge range of classes of neural networks, each coming with their own families of learning algorithms and possible variations. The fact that the the classical three-layered architecture trained with back-propagation is the most diffused does not mean than it is also optimal.
Just to give a couple of examples, deep learning architectures have the current highest accuracy over a large set of benchmarking problems, some of which you can find here:
http://deeplearning.net/datasets/
Moreover, deep learning models can also be used to create hierarchies of features, something that very few supervised learning algorithms can do.
Similarly, Extreme Learning Machines can be shown to outperform the classical Support Vector Machines on many problems [1].
Moreover, when you want non-linear dynamic approximators, it is difficult not to think of recurrent neural networks, particularly of newer models such as the Long-Short Term Memory or the Echo State Networks [2].
And I am listing here only the fields over which I have some experience, but there are literally dozens more, e.g. bio-inspired spiking networks and so on. See also this related questions on the recent developments on NNs:
https://www.researchgate.net/post/Which_are_the_new_trends_in_the_development_of_ANNs
[1] Huang, Guang-Bin, et al. "Extreme learning machine for regression and multiclass classification." Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on 42.2 (2012): 513-529.
[2] Jaeger, Herbert, and Harald Haas. "Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication." Science 304.5667 (2004): 78-80.
I agree with Yarnold. But, too many parameters of the network may hinder the accuracy or optimality of the network. There is a chance to learn the network efficiently, but the output may mislead with the overtraining problem. So, I am feeling the neural networks always competitors to the conventional methods but not optimal. Choosing the right network model from a class of neural networks models on the same data set is always a big problem and selection of the model based on the minimum error (loss function) empirically, will yield always a sub-optimal model. There should be a complete set of all possible models then only one can obtain an optimal model, but it is practically, time consuming process. Still some more explanation needed.
- Badges
- Science topic