Hi,
I would like to ask if there is a way to find the weighted coefficients a_i of a regression model with the following form:
y = a1*x1 + a2*x2 + a3*x3 + a4*x4 + (a3*a5)*x5 + (a4*a5)*x6.
As you can see, the model is linear in terms of x1 (predictors), but it is non-linear in terms of a_i coefficients. Any suggestions/comments would be greatly appreciated.
I may not understand the question. Where does a3, a4, and a5 come from? You mention regression, so from a least-squares regression view point, the coefficients for x1, x2, x3, x4, x5, and x6 would be determined such that
SIGMA[(y-yhat)2] is minimized. So the coefficients would be determined empirically from a least-squares fit from the data (the x and y matrix). I would also assume that there needs to be an error term in the model.
Perhaps I am missing something.
Hello Constantinos,
As John Morris 's reply, I'm unclear as to why you'd propose multiplicative constants for some IVs that were derived from the regression coefficients of other IVs; the fact that they are constants (for x5 and x6) would make no difference in the estimated values of the standardized regression coefficients (a5, a6). Perhaps you could elaborate on why such a model would be desirable.
More common, perhaps, would be interactions of IVs (e.g., x5 = x3 * x4).
Good luck with your work.
Hi,
thank you both for your responses. The reason why I get the above expressions is because I want to find the coefficients of the following expression:
y = a1 x1 + a2 x2 + a3 ( x3 + a5 x5) + a4 (x4 + a5 x6)
where a5 is not known and occurs in the last two parenthesis. There is a physical meaning on this choice (a5 represents the rate of energy transfer in a fluid system). As a result, it is desirable to keep the same rate at both terms. If this was not the case, I could solve the following expression:
y = a1 x1 + a2 x2 + a3 x3 + a4 x4 + a5 x5 + a6 x6.
Initially I thought that i could solve the regression model by using different fixed values for a5 (parametric study) and take the optimal combination of the remaining coefficients, using a range of values representative of physical constraints. But i thought that it might be interesting to let the algorithm decide what the best value is in the whole range.
This is certainly an unusual question, and one that I am still not sure I understand. For the moment, lets stick with standardized values so we have no intercept; that can be remedied later, or it may be of no concern to your theory. Regression will provide ONE weight for each of the variables, x1, x2, x3, x4, x5, and x6. Although I am not sure if this answers your question let us do a bit of transformation. Your original model was:
y = a1*x1 + a2*x2 + a3*x3 + a4*x4 + (a3*a5)*x5 + (a4*a5)*x6
Let's rewrite that as:
y = a1*x1 + a2*x2 + a3*x3 + a4*x4 + w5*x5 + w6*x6, wherein I have now have one weight "w" for x5 and x6. Let's now fit that model with regression, getting coefficients a1, a2, a3, a4, w5, and w6. As you now have a3 and a4, according to your theoretical model, could have two empirical estimates for a5 as,
a5 = w5/a3, and
a5 = w6/a4.
I leave it to you to consider the difference in these estimates. Does that help, or do I still have no idea what you are asking?
If I understood, here is an specific case of imposing conditions to estimators of a model, so we need to rearrange the model and apply some constraints. Rearranging the model and invert the expression, the parameters (or estimators) should be a function of variables included in the system.
look this simple example that I found in a R forum:
𝑦=𝑏1𝑥1+𝑏2𝑥2+𝑏3𝑥3,
subject to 𝑏1+𝑏2+𝑏3=0. Simply re-express 𝑏3 as 𝑏3=−𝑏1−𝑏2, which is to say you are trying to build a model of the form
𝑦=𝑏1𝑥1+𝑏2𝑥2−(𝑏1+𝑏2)𝑥3=𝑏1(𝑥1−𝑥3)+𝑏2(𝑥2−𝑥3).
y=b1x1+b2x2-(b1+b2)x3 = b1(x1-x3)+b2(x2-x3)
So create new variables 𝑥̃ 1=𝑥1−𝑥3, and 𝑥̃ 2=𝑥2−𝑥3, and perform your regression using these transformed variables as the independent variables.
Cheers
- Silvio
Silvio,
I don’t know if Constantinos is still attending to answers to this question, so this may be moot, however, I believe that I need to respond.
If I understand your point, the problem with this route is that if estimated with OLS regression, which was my understanding of the initial question, a constraint for b1 + b2 + b3, or any other set of weights, to sum to zero is not possible as this simply will, in general, not be the case.
John
Thanks John,
It's a hard question to explain with simple example when the regression variables are not classified as endogenous and exogenous to compose restrictions to a model. The initial question is not clear to me.
I found the Constantino's question because I need to put constraints to model estimators using variables there're not initially used in the original model. In a simple linear regression, using OLS or not, this procedure is quite simple to do algebraically whether the model contain few parameters to estimate. So, the condition in last answer could work, I guess but I didn't tested yet.
- Silvio
Silvio and John, thanks for your replies.
Silvio, in my case, the difference with your example is that: (a) there is not constraint among the a-coefficients and (b) I consider products between the coefficients, not addition. What I have eventually done, is that I have considered a modified expression:
y= a1 x1 + a2 x2 + a3 x3* + a4 x4*, where x3*= x3 + a5*x5, x4*=x4+a5*x6 (similar idea with what you have suggested) and I simply conducted a parametric study on a5 until I found the combination of {a1,a2,a3,a4} that has the optimal performance. The performance had a non-monotonic behaviour on the variations of a5.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Graphs