Let f:Rn \to R has sufficient degree of smoothness, where R is the set of real numbers and n is a positive integer. What kind of function approximators do you suggest for approximation purposes? Based on Weierstrass Approximation Theorem, one can use linear in parameter neural networks with polynomial basis functions to approximate any smooth function to any desired degree of precision. The good side of linear in parameter neural networks is the possibility of using least squares to train the neural network (in case the basis functions are linearly independent) which reduces the training time significantly. However, finding the best basis functions is through trial and error which is not always easy. Multi-Layer Perceptron (MLP) neural networks are known to be global approximators but training them is time consuming and needs proper training patterns. Fuzzy modeling also needs trial and error to find the best membership functions. ANFIS, wavelets, .... What do you suggest for function approximation and why.
Dear Tohid,
I suggest to take a look at links and attached files on topic.
Approximating a nonlinear function by a linear function - Math Insight
https://mathinsight.org/approximating_nonlinear_function_by_linear
Nonlinear function approximation using radial basis function neural ...
https://www.researchgate.net/.../3967745_Nonlinear_function_app...
Nonlinear function approximation: Computing smooth ... - ResearchGate
https://www.researchgate.net/.../222637262_Nonlinear_function_a...
A spiking neural network architecture for nonlinear function ...
https://ieeexplore.ieee.org/document/788132
Best regards
Many thanks Professor Lafifi for your input. I appreciate your response.
One intesting alternative could be the radial base neural networks.\
They are easy to implement and offer good results.
Dear Tohid
As long as you have different real sets, then you could use ANFIS for nonlinear function approximation. Check out my paper on ANFIS usage for a proper energy load forecasting.
Hope this helps.
Many thanks Dr Erik Cuevas and Dr. Hisham Hussein .
I tried to simulate some of the methods discussed in these posts. My major focus was the required time for training and the accuracy of the function approximators for approximating a selected nonlinear function on a compact domain of interest. The function approximators were selected as Multi Layer Perceptrons, RadiaL Basis, and linear in parameter Polynomials. For training the MLP and RBF neural networks, I used Matlab neural network toolbox and in order to improve/compare the training methods, methods explained in Article Adaptive neural network model based predictive control for a...
were also used. Here is a quick summary of the results.
Let the target function as F: R^2 \to R^4 where
F (x_1,x_2) = [ 2* x1^2+x2+sqrt(x1^2+x2^4 ) *sin(x1 ) * cos(x2 ),
exp(x1+x2 )-sin(x1 * x2 )+ x1^3,
x1+x2+ logsig(x1) + logsig(x2),
x1 * x2^3+ tansig(x1 ) + tansig(x2)]^T ;
The domain of training was selected as \Omega = {(x1,x2)| -2
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training