Dear all,
I am trying to use Back propagation for incorporating learning in expert system. However, the expert system has different constraints like bound constraints , inequality constraints. Is there any defining algorithm for optimization technique for it.
Thanks in advance for the help
Raihan
Depends on what kind of contraints you have. The back propagation algorithm defentily allows the consideration of constraints as it is for example done in ANN L2 regularization ( https://en.wikipedia.org/wiki/Regularization_(mathematics) ). But this is rather a soft constraint (you can change the importance of the constraint while training). As back propagation is a gradient based method, then for hard equality constraints you should check out lagrange Multipliers (https://en.wikipedia.org/wiki/Lagrange_multiplier). The easiest way to consider constraints in optimization is the penalty method (https://en.wikipedia.org/wiki/Penalty_method), where you can easily consider constraints using a penalty function. The hardest part here is to define a good Penalty function.
Dear Raihan Ul Islam
You can try meta-heuristic algorithms such as GA, PSO, ... instead of back-propagation algorithm.
https://en.wikipedia.org/wiki/Metaheuristic
https://www.researchgate.net/post/How_can_I_model_a_constraint_in_the_meta-heuristic_algorithms_eg_GA_PSO_GSA_etc
Hi Thieu Nguyen ,
Thanks for your idea. i have tried that one. But one to compare the results with back propagation.
I forgot to mention Raihan Ul Islam : a combination of the above stated methods is the augmented lagrangian method, which can be more efficient than the other ones.
Hi Christian Gletter , Thanks for the info. Is there any article or book that i can refer in my research article to refer it.
Hi Raihan Ul Islam
I think that "Optimization Theory and Methods" by Wenyu Sun and Ya-Xiang Yuan has all you need.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning