Thank you Mahboobeh for your comment.

Dear Nabil,

as you might know, the rer are probably hundreds of studies that address times series forecasting (in most cases applied to stock market). however i think you should know that it has been reported that most of these techniques are useless in practice.

kindly note that of those following points must be carefully investigated in order to decide whether it fit your problem or not. in other words, there is many question must be answered in order to decide which model/algorithm to adopt.

for example: if you have a time series that has probably non-linear correlation to other variables (non-linear multi-variable model) then GP is a good tool. reference [1] is one of most advanced practical research for using GP to forecast financial time series 

[1] Tsang, E., & Garcia-Almanza, A. (2011). Evolutionary Applications for Financial Prediction: Classification Methods to Gather Patterns Using Genetic Programming. VDM Verlag Dr. Müller.

another example concerning "Pattern recognition"

usually this is used for classification purposes. it might works in case such as: "is the today's close price will go up or down?". 

under same category "Pattern recognition" you may find the M5P algorithm (model tree) useful. it build a decision tree whose leafs are linear regression. hence, it's a pattern recognition algorithm with linear regression capability.

reference [2] provides a survey of different state of the art methods (ANN, HMM,..) used for stock forecasting.

[2] Iqbal, Z., Ilyas, R., Shahzad, W., Mahmood, Z., & Anjum, J. (2013). Efficient Machine Learning Techniques for Stock. International Journal of Engineering Research and Applications, 3(6), 855-867.

hope this helps. if you need further details about a specific subject, i will be glad to help.

Dear Amer, lots of thanks for your reply.

When talking about "a arbitrary non-stationary time series forecasting" :

+ Do you think that "guessing" the complex mathematical function that is behind a given time series is interesting?

+ Are we sure that such mathematical function does exist?

Thanks.

Dear Amer Backach,

I can't access this reference for free:

[1] Tsang, E., & Garcia-Almanza, A. (2011). Evolutionary Applications for Financial Prediction: Classification Methods to Gather Patterns Using Genetic Programming. VDM Verlag Dr. Müller.

Any help?

Dear Nabil,

if you have a non-stationary time series then you can NOT use GP. but if you can turns it stationary (e.g. by differentiating, logarithm transformation,..) then you may get benefit from GP indeed. did you tried ARIMA model? 

any mathematical function is only approximation. bear in mind that it's always possible that a white noise may present. 

if you are using R package you may use "decompose" function which gives a good in-depth view of you data. (seasonality, trend, randomness)

mainly we believe that the behavior of a time series is due to something which will repeat if it's made by human.

for example. in stock market there is an algorithm that determine the bid/ask prices. theoretically, provided that this algorithm  will have same input then it will have same output. so in this case forecasting this "output" is sort of reverse engineer. in addition, if we assume that a human will take same action (buy/sell) in similar situations. that may cause periodicity, which can be detected using an appropriate machine learning algorithm.

in that case the answer of your question is 'yes' we can, to some extent, believe that such pattern does exist. of course it's not going to be 100% accurate. but it could be good enough.

Dear Nabil,

if you are interested in GP application in financial forecast you may use the attached file. the reference that i mention above -[1] Tsang, E., & Garcia-Almanza, A. (2011)- is based on the attached file.

Time series forecasting is used in many areas of specialization. Irrespective of the area of specialization, time series forecasting can be divided into two broad categories; univariate and multivariate time series forecasting. If you have a univariate time series, you can use ARIMA method and if you have multivariate time series, you may use VAR / VECM method. Both methods have one property in common, the current value of the variable in consideration depends on past values of itself and those of other model variables, as well their past error terms. Many texts are available, but I recommend 'Time Series Econometrics' by Walter Enders. Among the available software packages, I recommend 'EViews'. Fitting the data means, how well the estimated model can predict  in-sample values of the variable concerned. If the in-sample forecast trajectory of the variable almost coincides with the actual trajectory, you have a satisfactory model fit. You can compare alternative models in terms of in-sample forecasting superiority using various criteria; 'Root mean square test', 'Theil inequality test' etc. I hope, it helps.

Dear Amer,

I don't think that GP CANNOT be used in non-stationary time series forecasting. Do you have any related reference?

Dear Nabil,

for the best of my knowledge the best paper that describe how GP can fit non-stationarity in time series is:

http://people.cs.umass.edu/~sniekum/pubs/EvoTAMD.pdf

however, i am not convinced with their described model. they use many restrictions that may not hold in various real-world problem. 

let's back to the initial definition of non-stationary time series. simply it's a time series having its mean and variance changing over time-period. as such any sort of regression (either linear or symbolic or ..) can only try to reduce a fitness function (RMSE, MAPE, ..). by doing this, GP can not distinguish between "periods" where the time series behave differently from other periods. 

anyway, i would love to know that there is a practical solution to adapt GP to non-stationary process.

thank you for sharing any new perspectives.

Dear Nabil,

My suggestion will be for you to have a little historic perspective about the evolution of time series prediction. It is in this context that the relationship of the elements you mention becomes evident. I particularly like the first chapter of the book  by Gershenfeld, N. A., & Weigend, A. S. (1993). The future of time series. Xerox Corporation, Palo Alto Research Center. It is certainly clarify the relationship of time-series prediction with pattern recognition techniques. Curve fitting, genetic programming, polynomial expansion, neural networks, etc,  all can be seen in this context as alternatives to build a predictive model from the data. Will that be possible? One may say yes, but always the model will have limitations, i.e., will be an approximation of the probability distribution function if the data are purely stochastic, or may exhibit sensitivity to initial conditions if the data is chaotic.

Best regards.

How to measure the stationarity (or non-stationarity) of a time series? I think it is a non trivial question in time series forecasting.

If you consider a non-stationary time series (a realization of a pure random walk), i think that it is possible to determine a stationary process that approximates well the time series (not the pure random walk process).

Dear Nabil,

i think you already know that there exist some statistical test to test stationarity in time series (e.g. KPSS, ADF, ...). if it's stationary than it will appears like a white noise.

however, even-though your time series is stationary (or seems to exhibit a random walk process) it still can be predicted. 

i am facing similar case in my research. it turns out that my time series is stationary (white noise) and the ARIMA model could not help at all.

however, i searched for other variables those may affect the dependent variable; and i get encouraging results. probably you can do so.

Dear Nabil,

Fuzzy time series is related to forecasting,SVM is ok for forecasting.

If my requirement is the following:

Given a time series T(0..t-1), provide an accurate forecast of T for a given horizon (t, t+n). We have NO prior idea about the properties of the time series.

Is there a computerized procedure that can fulfill this requirement?

If you desire you can see my recent work titled "Genetic programming for photovoltaic plant output forecasting" on Solar Energy in 2014.

Thanks you for your reply. Download in progress.

If you are interested in univariate time series data predictions, have a look in PSF algorithm and it's R Packages available at :

CRAN: https://cran.r-project.org/web/packages/PSF/index.html

GitHub: https://github.com/neerajdhanraj/PSF

How to use: https://www.researchgate.net/publication/304131481_PSF_Introduction_to_R_Package_for_Pattern_Sequence_Based_Forecasting_Algorithm

and

https://www.researchgate.net/publication/304580701_Introduction_of_seasonality_concept_in_PSF_algorithm_to_improve_univariate_time_series_predictions

For further details contact me at: http://www.neerajbokde.com

Article PSF: Introduction to R Package for Pattern Sequence Based Fo...

Article Introduction of seasonality concept in PSF algorithm to impr...