If you build a parameter learning algorithm based on the Lyapunov stability theorem for updating the parameters of an adaptive fuzzy controller, how to determine the cost function and Lyapunov function? Is there a physical connection between them?
In fact, in general, there is no physical link between the cost function and the Lyapunov function the first determines the direction of optimality (criterion expressed by a functional) while the second defines the stability within the meaning of Lyapunov. In some cases where we have linear systems with uncertain parameters, the search for a Lyapunov function can be formalized in an optimization problem and, when this one is convex, there are efficient resolution algorithms. There are also methods for carrying out a looping so that a Lyapunov function, chosen in advance, guarantees stability.
Lyapunov function is a point-wise measure of energy, whereas, cost-functional is an interval-based measure of energy. In this sense, you may connect them by assuming Lyapunov function as an explicit function in time which could get a negative decay-rate assuming V=x^2, and then a closed-form dissipation as; V_dot=-K*V(t) for stabilizing a system, see my pre-print at the URL:
Preprint Hyper-exponential Global Feedback Stabilization of Nonlinear Systems
Meanwhile, optimal quadratic cost functional, minimizes energy as a sum of the squared state and control signal during a time-interval as; J=int(x^2+u^2).dt, from zero to T. Please see the paper:
Nonlinear Optimal Control: A Control Lyapunov Function and Receding Horizon Perspective (1999)
The control performance could be different through the two scopes, and the energy consumption could be also different. Lyapunov stabilization, minimizes energy-like function as V_Lyapunov=x^2, while in optimal control we minimize the integral of quadratic state as V_opt=int(x^2.dt). This is like the difference of proportional controller and integral controller.
I think you have got the answer because both researchers Saeb AmirAhmadi Chomachar and Mohamed-Mourad Lafifi already have explain well with useful description.