Why would one use neural networks instead of regression. I know neural networks have much more computation load and I was wondering what could be the possible benefits of this approach?
Both techniques have merits and de-merits. With small data set a simple regression model may be good. But when we have nonlinearities and big data set, ANN outperform and achieve a better fit and forecast than the regression models. As already pointed out ANNs are versatile and more flexible and can be used for modeling (estimation, regression, prediction,...) as well as classification, ... They are known as "universal approximator". However, ANNs are “black box” and there is a difficulty in interpreting the possible relationships between parameters. but the significance of explanatory variables and expected predictive capability can be readily be explained by regression models.
Please check the following thread for similar question.
https://www.researchgate.net/post/Can_anyone_explain_how_a_neural_network_model_is_better_than_a_regression_model
Hello Aboozar Roosta,
Artificial neural networks are not only used for regression but also for many other tasks like classification or unsupervised learning (autoencoder, neuroscale) . The main strength of neural networks is, that they can use nonlinear basis functions (so called activation functions) which allow to handle nonlinear, multidimensional dependencies in your data. Often, in case of regression, neural networks use linear regression in the final layer.
Remember that there are many other technikes to cope with non-linearity. For example you could use k-Nearest-Neighbors or gaussian process regression.
One nice thing about neural networks is, that they can be trained sample by sample which can be very handy if you have a very large data set. Also you don't have to keep every Trainingsample in memory like in the case of k-Nearest-Neighbors.
I hope my answer helps you! Have a nice day!
Regards,
Stephan
Dear Aboozar Roosta,
your expression "neural networks instead of regression" is a little bit misleading.
As Stephan already pointed out, NNs can be used for regression.
I assume that you think of "method of least squares" when you write "regression". Depending on the learning rule (backpropagation), neural networks are also based on least squares. If your problem is nonlinear and you do not know the model function (function which maps input data to output data), then you could use an NN hoping that its nonlinear mapping is suitable for your problem.
Please have a look at the textbook "Data Fitting and Uncertainty" (2nd edition). There you can find a nice example showing the relation between an NN and the method of least squares.
Kind regards,
T.Strutz
http://navatrump.de/DFU/dfu.html
In General you have the benefit of non -linearity using ANN. It depends on your problem.
Both techniques have merits and de-merits. With small data set a simple regression model may be good. But when we have nonlinearities and big data set, ANN outperform and achieve a better fit and forecast than the regression models. As already pointed out ANNs are versatile and more flexible and can be used for modeling (estimation, regression, prediction,...) as well as classification, ... They are known as "universal approximator". However, ANNs are “black box” and there is a difficulty in interpreting the possible relationships between parameters. but the significance of explanatory variables and expected predictive capability can be readily be explained by regression models.
Please check the following thread for similar question.
https://www.researchgate.net/post/Can_anyone_explain_how_a_neural_network_model_is_better_than_a_regression_model
The main difference is that NN can approximate any function under some conditions. See for example the "universal approximation theorem", https://en.wikipedia.org/wiki/Universal_approximation_theorem
Two main tasks in machine learning are regression and classification. NN are capable of both of them. For the regression with NN you have only to choose limiter that is different than step function.
The learning ability of Neural Network makes it popular for different kind of pattern classification problem.
Thanks for all your answers, they helped me better understand the problem and where to look for my answers.
well linear regression is just a special case of NN where the layer reduced to one, and activation ruction being linear.
- Badges
- Science topic