Some Machine Learning methods have versions both for Classification and Regression. For example, SVM method (Support Vector Machine) has realizations SVC (Support Vector Classification) and SVR (Support Vector Regression). Method RVM (Relevance Vector Machine) also may be used both for Classification and Regression. What about Deep Learning, Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and other NN–based methods? Is it possible to use them not only for Classification, but also for Regression? Thanks a lot for your answer beforehand. Regards, Sergey.
Dear Surgey,
Can you please brief me the problem that you are solving. Based on the nature of the problem at hand, the suitability of using Deep Learning can be judged.
Dear Surgey,
Can you please brief me the problem that you are solving. Based on the nature of the problem at hand, the suitability of using Deep Learning can be judged.
Dear Sobhan, thanks for rapid answer. Our task is following : To predict (today) occupancy of the some hotel on some target date (e.g., 19.03.2018) based on historical data (occupancy and rates at the previous dates) and current data ( current/today occupancy and rates for the target date). In my opinion, it is classical regression task.
From the universal approximation theorem (Cybenko, 1989) we know that a NN with even a single hidden layer and sigmoid activation can approximate any continuous function (with the accuracy of this approximation depending on the number of neurons in the layer) - therefore in theory we can approximate any function, be it a classification or regression task. The crucial choice here is that of a loss function to optimize, as ultimately this is what defines the task that we want to learn. If the loss function imposes a notion of distance on the model, we are talking about a regression task - a typical choice would be an L2 (Euclidean) loss. For example, a deep convolutional autoencoder could be thought of as an example of a regression model.
A few reading primers:
- http://ufldl.stanford.edu/tutorial/supervised/LinearRegression/
- https://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Belagiannis_Robust_Optimization_for_ICCV_2015_paper.pdf
- Cybenko, G. (1989) "Approximations by superpositions of sigmoidal functions" https://doi.org/10.1007%2FBF02551274
Of course yes. Indeed in regression, you will have only one node in the output layer , unlike the classification where you will have number of nodes in output layer equivalent to number of categories. Which software are you using? I can recommend some libraries based on the software or I can get knowledge from you if you are using new library
yes, you can, any feedforward ANN can be built under the Deep Learning paradigm, in fact we have used DL with a backpropagation ANN for simultaneus prediction of two random variables and it works very well. The manuscript is under review roght now. I will let you know once it is available.
Short answer will be yes with aid of a single neuron that has a W and B which is powerful enough to estimate a model known to be a linear problem such as: y=ax+b. In this case the W would be a and B would be b.
For more complex problems, more complex models are needed.
Complexity of problem can be solved by more Weights (W) and more Biases (B) that are computed with nonlinear activation at each neuron such as ReLu or Tanh, Meaning that you would need more layers and more neurons at each layer combined with nonlinear activation functions. Such models are known to be a complex of many linear models.
Sure one can use ANNs to do solve regression problems. For example, In terms of network structure, there just has to be a final layer without any nonlinear activation function. Or with the one that handles the target value range.
Dear Sergey Porotsky, unfortunately I am not aware of the available Matlab libraries for doing so. If you are willing to work with R or python I can guide you for some available libraries for doing so. There are some libraries on R that are capable of doing conventional deep learning models for regression/classification in a timely manner.
Dear Saleh, thanks a lot for your participation, but I prefer to use Matlab. Regards, Sergey.
- Badges
- Science topic