I am often faced with the question of "what are the advantageous/disadvantages of ANN over conventional statistical techniques?" by newcomers interesting to apply ANN in their research.
As we know, ANN provides a new alternative to MLR and some other conventional statistical techniques which are often limited by strict assumptions of normality, linearity, variable independence, one pass approximation, curse of dimensionality, etc. Is there any rigorous comparison among these two?
The no free lunch theorem states that uniformly averaged over all target functions the expected error is the same for any two algorithms. Nonetheless, there are other reasons for stating that there are advantages of ANN over other algorithms. For example, the ANNs show graceful degradation were you may have noisy input data or even the removal of units and the ANN still functions. Another advantage is the inherent distributed nature of ANNs which allow better implementations across a distributed environment.
From a personal perspective the advantage that I see is the flexibility in changing the encoding of the data to fit different statements of the problem in an ANN.
The disadvantage of ANNs is that it is non trivial to extract what is the process going underneath with regards to the decisions the ANN is making (it can be done but it is not as straightforward as other algorithms. Troubleshooting implementations of the algorithm will also be tricky ).
"One of the reasons for neural networks success story is the fact that these are universal approximator. However, one can argue that also polynomial expansions posses this property. In 1993 Barron showed that neural networks can avoid the curse of dimensionality in the sense that the approximation error becomes independent of the dimension of the input space. Models that are based on MLPs will be able to better handle larger dimensional input spaces than polynomial expansions, which is an interesting property towards many real-life problems where one has to model dependencies between several variables." (excerpted from Least Squares Support Vector Machines book: http://www.esat.kuleuven.be/sista/lssvmlab/ )
In the last decade due to the importance of the research topic Numerous Studies were concentrated on many linear and nonlinear regression equations. Modeling by using artificial intelligence (AI) has been a very active research area. According to previous studies, Although, AI techniques have proved their superior capability over traditional modeling methods and so Artificial Neural Network (ANN) was one the successful choice that used for prediction problems, it has the some following limitations:
ANN does not provide information about the relative significance of the various parameters.
A common criticism of neural networks is that they require a large diversity of training for operation. The knowledge acquired during the training of the model is stored in an implicit manner and hence it is hard to come up with reasonable interpretation of the overall structure of the network.
In addition, ANN has some intrinsic disadvantages such as slow convergence speed, less generalizing performance, arriving at local minimum and over-fitting problems.
Dear @Abdosattar Feizbakhsh,
Thank you for your answer. Although ANNs may have disadvantageous as you explained, they have many good advantageous, too. For example, in higher dimensional settings ANNs are able to represent complex non-linear behaviors. So when there is no information on the form of the function to be predicted in advance and the task of specifying the functional form from the data is computationally complex/ expensive, then ANNs are superior over polynomial expansions used in linear regressions, as the errors for the later are more dispersed (than ANN ones). Further more, ANN is a non-parametric model, thus eliminates the error in parameter estimation, while most of statistical methods (MLR, etc.) are parametric models that need higher background of statistic.
The main advantage of ANN models is the weighting part which make closes to daily life application
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Regression