I had fitted a neural network for a dinamyc process. I used a matrix containing a complete data set for cross validation and it showed great results. Then I wanna predict the all process just using initial conditions with the same neural network. It showed a great fitting in the first 3 or 4 values. From that point on, the error start to grow enormously.
For whole dynamic process learning (as opposed to a short-time forecasting/prediction of time-series), prefer such solutions based on ODE modeling:
- Chen et al., " Neural Ordinary Differential Equations ", 2019 - https://arxiv.org/pdf/1806.07366.pdf
- Heinonen et al., " Learning unknown ODE models with Gaussian processes ", 2018 - https://arxiv.org/pdf/1803.04303.pdf
- D. Barber, " On solving Ordinary Differential Equations using Gaussian Processes ", 2014 - https://arxiv.org/pdf/1408.3807.pdf
You can download the WEKA and experiment yourself with noise filters. http://weka.sourceforge.net/doc.dev/weka/filters/unsupervised/attribute/package-summary.html
I suggest you that use ANFIS model for accurate prediction. You can study the following article for more information:
Article An efficient method for classifying motor imagery using CPSO...
@Alexander Osherenko, I don't see any noise filters in the page that you linked, except one that adds noise, which is opposite of what the question asked. They are all different filters that Weka offers, none specifically deals with the question.
Am I missing something?
@Tom Arjannikov, do you any recommendation besides noise filter for solving this trouble??
Tom Arjannikov I would download the WEKA sources https://www.cs.waikato.ac.nz/ml/weka/downloading.html and investigate how they add noise. Once I know how it works I will implement my own noise filter and integrate it in WEKA. Section 5.2.4 in the manual:
http://statweb.stanford.edu/~lpekelis/13_datafest_cart/WekaManual-3-7-8.pdf
@Roberto Eloy Hernández Regalado, you mentioned that your data comes from a dynamic process. To me, this means time-series data. If this is the case, you may consider using wavelets to de-noise your data.
@Tom Arjannikov the trouble is most of my data is taken by lab measurements, for which I can't measure the electrical signal for equipment involved. Data measure by sensors is high quality. Even data can interpreted as time series data. But the error I'm looking for reduce is the which is generates for several successive prediction by neural network. This is I introduce initial process conditions and I start to predict, so the imput for prediction for second state is output from first state prediction and this goes on until final process conditions. So my trouble is that from state 3 and on error starts to grow and in state 20 predicted value lacks of physical meaning.
Roberto Eloy Hernández Regalado, that makes sense to me. Your initial error is amplified as you go along. If you can reduce the initial noise in your data, you can reduce the amount of error that's propagated through your system.
There have been many successful applications to de-noise input signal using wavelet transform. For example, that's the current state-of-the-art when dealing with fNIRS signal.
I am unclear about something when you say "successive prediction". Do you mean as a stack of neural networks, where the prediction of one (along with some features) is fed into the next one? Or, do you mean successive time steps, but the same network, where its prediction is from timestep t-1 is used at timestep t?
Either way, if the network is predicting incorrectly, and this error is used as input to the next network or step, you could try weighting your input to lower the emphasis on the prediction from the previous one and raise the importance of other input features. Additionally, you may find "classification in presence of label noise" literature interesting, here's a link to get you started: Article Classification in the Presence of Label Noise: A Survey
.I hope this helps :)
@Stéphane Breton your answer is highly useful. But let's say I wanna implement the neural network as an stimator of parameters which are difficult to stimate of an ODE. How could I do it?
Roberto Eloy Hernández Regalado, in the first place, a Gaussian process is a [kernel-based] neural network. Refer to:
- Lee et al., " Deep Neural Networks as Gaussian Processes ", 2018 - https://openreview.net/pdf?id=B1EA-M-0Z
Next, proceed as described by M. Heinonen in section 3-Learning from multiple initial conditions from:
- Heinonen, " Learning nonparametric differential equations with operator-valued kernels and gradient matching ", 2014 - https://arxiv.org/pdf/1411.5172.pdf
- Badges
- Science topic