It is right that sigmoid function gives an output between 0 to 1. Regarding to the activation function, transformation of time series data is needed. It would be better if you look at the range of activation function before you decide to transform data. And reverse transformation is a way to restore the data to the initial scale (original data)

i guess you want to keep the sigmoid in the hidden layer(to obtain a nonlinear model), but probably you want to use a linear activation function in the output layer. In this way the values won't be bounded between 0 and 1.

If you are dealinb with time series forecasting, you might also want to use a recurrent neural network instead of a BPNN.

You can apply hyperbolic tangent function at hidden as well as at output layer.

You can have the best of both worlds by rescaling the target (=output) values from [0.2,0.8] or from [0.3,0.7], leaving the basic multilayer perceptron settings intact. This keeps the backpropagated error within limits, unlike the case of unbounded linear output transfer functions, and focuses the training on the almost-linear part of the sigmoid function. Which is not to say that all your problems will vaporize.

Sigmoids in the hidden layers and linear in the output layer. Also, try to normalize your inputs and outputs.

K thanks for ur help guys.

I will try that nw,but if i apply different activation function like for model 1:sigmoid & linear model2:tanh & linear to want extend it will affect my network prediction ability or there will not be any significant difference?

First, normalize the input data and bring it to [-1 ... 1]. Then apply the bipolar sigmoid function for activation.

You have two immediate options:

- normalize your output data and keep the sigmoid/hyperbolic tangent

- go with a linear activation function in the output layer (keeping sigmoid/ht in the hidden layers)

there is some evidence that the latter may hinder convergence a bit, but it is a more general solution (normalized outputs may go out of range given new data)

there is no need that I can see to normalize your input data or to use a bipolar scheme, although both are very recommended strategies for almost all BPNN applications.

Use Sigmoid and Normalize and suggest to use RNN

The sigmoid's input should be kept between limits. Since for large negative values of x, sigma(x) degrades to zero and for large positive values to 1.

Definetely the output layer should be linear. For the hidden layer, if you want to try something different, use the derivative of the sigmoid s'(x)=s(x)(1-s(x)). This function tends to zero at both ends (+/- Infinity).

It is right that sigmoid function gives an output between 0 to 1. Regarding to the activation function, transformation of time series data is needed. It would be better if you look at the range of activation function before you decide to transform data. And reverse transformation is a way to restore the data to the initial scale (original data)

For predictive models, it is necessary to consider whether the model input variables are only endogenous or exogenous variables also owns. For the case of exogenous variables predictive model is much better to use radial basis functions, as these provide a hyperplane relations between these variables and their delays. For the case of endogenous varaibles only is important to work with neural networks without activation functions, especially when it involves working with large numbers, but it is important to note that according to Cantor's theorem Dedekind, neural networks have a higher stability learning when all the numbers for both input and output are dimensioned in the interval [0,1]. In the case of wanting to get much higher numbers to the unit, you should create a separate module for this translation, and the neural network that learns directly from these numbers.

I fully agree with "Lambert Schomaker"

I respect your views and your knowledge in this field but why to increase coding by normalizing and again d-normalizing at output. Can we have or think of some thing different thing that will allow us to get rid of this normalization. May be a different model?

I fully agree to "Stefano Rovetta" suggestions and recommendations. I did a prediction model for monthly, weekly rainfall using BPNN with the sigmoid activation function and it works. Of course, do not forget to "unnormalize" the predicted output.

Which activation function should be used in a prediction model?

This is an important issue. The activation function is a key to BPNN. If your expected output is like 231.54, then you maybe choose the activation function max(0,x) (please see the works of Hinton 2010 and 2012). In classification model, max(0,x) is better than sigmoid function.

To answer which activation function is the best is based on what type of data that you are dealing with? Definitely sigmoid is meant for data that range from [0,1] or in other words you have to normalise the data to that kind of range in order to make sure the processing runs smooth. and yet once you finish then you have to de-normalise it to it normal range. So any kind of data can be fit with sigmoid. But from my experience tanh is much faster than sigmoid since it deal with data that range from [-1,1]. anyway there is no rule of thumb in neural networks, everything is heuristic and need to do some simulation befor jump to the conclusion....anyway sharing is caring...

Correctly said by others: you should normalize your data first and re-do to original scale after training the net. By normalizing every inputs to the net get similar chance during weight's update. Otherwise some inputs would have no chance to participate in update process (I could state more advantages here, if i had more time!). I think for prediction, you are better off using nonlinear activation function for hidden layer(s) and a linear one for the output layer.

Agree with Dr. Omid. And what you have chosen(sigmoid activation function) is also right. For the prediction problem choose an appropriate normalization method for your input data and retrive back your output to the normalized scale to get your actual outputs. This will solve your problem.

As mentioned before, the choice of activation functions for an ANN is based primarily on the data available. From my experience, the best approach is to use a multitude of activation functions(AF) and with a heuristic to try different combinations. Also, you can consider AF per layer (all neurons in the same layer have the same type of AF) or per neuron (each neuron can have its specific AF)

Dear Prashant Chavan,

Sigmoid function gives an output between 0 to 1.  It would be better if you made a normalization of your data, may be an MinMax normalization is a good solution for your case. An MinMax normalization transfert all your input an output  on [0,1], and after this you can back using an denormalization for reels values (origine value).

Best regards

Recently, ReLU got a good attention for deep models as an activation function. Then, normalization process is also utilized to put your input an output in the range of [0,1].