Dear all
I am using a recursive least squares (RLS) estimator to update the parameters teta(k) which is a n by m matrix ( teta(k) has n rows and m columns). The equations for the RLS are:
P(k)=(1/lambda)*P(k-1)-(1/lambda)*P(k-1)*Phi(k-1)*inv(( lambda*eye(n)+ Phi(k-1)’* P(k-1)* Phi(k-1)))* Phi(k-1)’*P(k-1)
teta(k)= teta(k-1)+(x(k)- teta(k-1)* Phi(k-1))* Phi(k-1)’* P(k)
As an example, suppose that n is 2 and m is 5 (teta(k) is a matrix with 2 rows and 5 columns) and I want to have the following inequality constraints for teta(k):
abs(teta(1,1)(k))+ abs(teta(1,2)(k))
Iman, I can see two ways to cope with your inequality constraints:
1) You may use a coordinate transformation introduced in such a way that the constraints are satisfied automatically for any values of the new coordinates. For example, consider p and q such that θ11 = (atan p - atan q) / π, θ12 = (atan p + atan q) / π.
2) You may treat the least squares as a constrained optimization problem. As in any other problem of this kind, you have the cost function defined in a region bounded by several lines. The minimum may be reached either within the region, where the derivatives are zero, or on any of those lines. Each of these cases should be treated separately. Since the single constraint |θ11| + |θ12| = 1 defines four straight lines, you should solve five optimization problems and then compare their results. With two constraints, the total number of optimization problems is even larger.
Vasiliy, thank you for the suggested ways, I prefer to not use constrained optimization problem and I want to work on the first way which is using a coordinate transformation. Could you please refer me to any resources or materials for the first way? I want to know which coordinate transformation should I consider when I have more parameters(θ11, θ12, θ13, ...)?
Dear Iman,
I suggest you study the following paper:
The article begins with a review of the online passive aggressive algorithm (OPAA), an adaptive linear regression algorithm from the machine learning literature. We highlight the strengths and weaknesses of OPAA and compare it with other popular adaptive regression techniques such as moving window and recursive least squares…
I think the use (OPAA) may result in the scores for you, here attached file.
Best regards
Iman, as for coordinate transformations, I think it is rather a general idea, not a systematic method. I have invented the transformations with arctangents specifically for your case. You may take them as an example and generalize to more difficult cases. The only universal requirement is that they must be smooth and invertible.
Anyway, don't be afraid of the constrained optimization approach. Similar problems arise in many areas, and various solution methods and tools exist under the names like "Nonlinear programming", "Quadratic programming", "Active set methods". Even the Matlab Optimization Toolbox has a dedicated function (see the link below). Unfortunately, my personal experience with these tools is very limited.
http://www.mathworks.com/help/optim/ug/quadprog.html
The recursive least squares algorithm (RLS) is the recursive application of the well-known least squares (LS) regression algorithm, so that each new data point is taken in account to modify (correct) a previous estimate of the parameters from some linear (or linearized) correlation thought to model the observed system. The method allows for the dynamical application of LS to time series acquired in real-time. As with LS, there may be several correlation equations and a set of dependent (observed) variables. A particularly clear introduction to RLS is found at: Karl J. Åström, Björn Wittenmark, "Computer-Controlled Systems: Theory and Design", Prentice-Hall, 3rd ed., 1997.
Years ago, while investigating adaptive control and energetic optimization of aerobic fermenters, I have applied the RLS algorithm with forgetting factor (RLS-FF) to estimate the parameters from the KLa correlation, used to predict the O2 gas-liquid mass-transfer, while giving increased weight to most recent data. Estimates were improved by imposing sinusoidal disturbance to air flow and agitation speed (manipulated variables). This investigation was reported at (MSc Thesis):
Thesis Controlo do Oxigénio Dissolvido em Fermentadores para Minimi...
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Control Theory
- Control Systems