You can use LSTM model (matlab and python/lassagne tutorial links below)

https://github.com/huashiyiqike/LSTM-MATLAB

https://github.com/Lasagne/Recipes/blob/master/examples/lstm_text_generation.py

Hi Chinedu,

This can be accomplished by tuning:

a) The number of units in the hidden layer.

b) The ridge parameter.

HTH.

Samer

Hi Chinedu,

probably the reason for your funny results is related to the so called bias-variance tradeoff.

On the one hand you are looking for a model which is able to fit your observed training data, which were generated by a more or less complicated process (e.g. financial markets). For this reason, the model needs a certain complexity. With a growing complexity your model gets more and more sensitive to your training data, which can be seen as some kind of model variance. 

On the other hand, you want a model which generalizes well. Typically, the set of training samples is limited. Some dependencies between input and output variables are visible very well, but others are not. Partially because you don't have all information (noise with unknown reason) and partially because some events in your time series occur not often enough to infer a proper relation. In such cases it is often better to restrict the model complexity to a certain degree (similar to Occam's razor). The more you restrict your model, the more systematic errors (bias) appear which could have been prevented with a more complex model. In simple terms, the reason for let's say a broken vase can be suspected in mysterious beings from another world (high model complexity, which could explain almost every broken object in the world), in the dear pet cat (moderate complexity, but much more likely) or it just fell down (low complexity, again not likely and also no dependency to anything and no chance to predict or prevent that).  

It is therefore necessary to find the right measure between model complexity (variance) and generalizability (bias), which explains, why it is called bias-variance tradeoff.

As Samer already suggested you can lower the number of hidden neurons or use some kind of regularization (ridge, weight decay, ...) to lower your model complexity. It is very helpful to split your data in a training, validation and test set (Also in variants like 5x2 cross validation or 10-folds). Usually several training cycles are performed where the model complexity is varied. The lowest error in the validation set indicates which model complexity fits your needs. This is called model selection or hyperparameter optimization, where the hyperparameter is the number of hidden neurons, the value of the ridge regularization parameter or something similar. After you found the best model complexity train data and validation data is used together to train the final model, which can be tested with the remaining test data set.  

There is a certain connection to Bayesian modelling. From this point of view the model complexity is restricted by your prior assumptions of the model parameters. As an example, ridge regression can be seen as an Bayesian estimation of linear regression parameters where the prior probability distribution of all possible regression parameter is a normal distribution with expectation zero and some variance. This prior variance affects the value of the ridge regularization parameter. 

Hopefully this can help you and I did not tell you to much stuff you already know.

Best wishes

One can think that every prediction made corresponds to the output of a certain system, generally, a dynamic one. To pretend to predict such a system for a very long period without changing the preciting model poses several  issues: 

a) the system itself can change faster than the data collection rate   used  to predict its output, in other words, you are unable to gather enough information as time passes  as to predict the outputs of the system. So you should simplify it, in order to have a  simpler model... but this could render it useless. It is a problem of complexity  and bias,  as Vogt pointed. 

b)the system, deterministic or stochastic, generally will be chaotic if it corresponds to a real world one, so it has a limit of predictability in time (the horizon of predictability, H); you can´t predict  more than H timesteps in the future. The prediction errors accumulate  (when the model is used in closed loop) and this reduces even more the ability to predict . There are algorithms to evalute H.

c) If you want to predict values that depends of  other separated  more than 100 timesteps, LSTM could be a good option, as Ramachandra suggested,  or else seek a training algorithm that preserves such dependencies; not only the topology of the network is important, it is also fundamental the training algorithm used.

Hope that these words  helped 

Good luck,

 Horacio

I sincerely appreciate all response to this question. They were so much helpful in no small measure.