Suppose we have computed the list of coefficients for the regression $ y = \Sigma_{k=1}^p a_ik x_k $ where we have N measurements values, i.e. N vectors (y,X=[x1,...,xp]). $y$ is the objective, while the xi are the variables. The a_ik coefficients are obtained e.g. with the least squares method.
When we receive a new set of measurements e.g. M additional vectors (y',X') , is it possible to compute, or have a nice approximate, of the updated coefficients a_ik ?
You might want to check your notation -- I don't think we need a subscript "i" for a_ik. [There is a technique where there are locally weighted coefficients, called GWR (Geographically Weighted Regression) but I am assuming this is not what you mean.]
What you describe is a bit like what is called a hold out sample. There are N obs that you run the least square, and then, more usually, you might see how well the fitted model does at predicting (accurately) the results from the added M cases.
I do not know a short cut though if you want to recompute the results for the N+M combined group.
If you were writing your own code with loops etc, you could store the intermediate computations from the first N cases, and then add on the results from the added M cases.
Dear Gabriel Thomas, if you have already computed the model coefficients using the total number of observations/vectors and you felt that the accuracy of the model is not quiet OK as a result of inadequate vector/measurements, the later vectors/observations/measurements have to be added to the formal/previous vectors/measurements to increase the total number of observation equations. Then, apply least squares to solve for a new set of the model coefficients.
Hi all,
thanks for considering this question and for the notations remark by Morton.
The point is to compute the coefficients for the whole set of data, using the coefficients of the known set and these of a new collection of data.
Having e.g. 10000 data and coeff $y = a_1x1+ a_2x_2+...+a_kx_k+ b_k$, if I receive a new set of 100 values, I don't want to compute with 10100 data but use my new 100 data and the known coefficients. Is there a matrix method to get the results, from partial computation ?
One common component here is the product X'X which enters into the term to be inverted or manipulated in some way. (Called sum of squares.)
When you add a few rows, the elements of the new sum of squares matrix are the old elements plus the new sums of squares from the added rows.
See the attached small example; "yellow" = original data;
"blue" = including the new data by adding the original data + the incremental data
- Badges
- Science topic