I have 623 observations with one continuous dependent variable and 13 independent variables (continuous, categorical, and ordinal), defined based on researcher experience and literature review. I considered planning to do several regression analysis to estimate the dependent variable and study the predictive factors (if they are positive, negative, and their magnitude) on it. Data are provided for 10 years. Since the latest observations are more important, I’m interested in using weighted observations. How can I approach this problem and validate my approach? Thank you.
As far as I know, exponential weighting for observations would be the most suitable way of using the weighted regression if you want to make recent observations more valuable.
Hello Amir,
Andrey is right. Apply exponential smoothing for your data. This is easily done, you should only think of proper weights for the corresponding points in time.
Best regards!
Thank you Andrey and Partizio, Are you aware of some references that they applied and I can read those?
Hello again,
I have to admit I don't have any direct source I could recommend,sorry. The method is pretty straightforward, though.
You may take a look at this:
http://people.duke.edu/~rnau/411avg.htm
Under
"Brown's Simple Exponential Smoothing (exponentially weighted moving average)"
it is quite well explained how this thing works.
Best regards!
Brown's exp. smoothing is not exactly the weighted linear regression.
If you have many explanatory variables, I would suggest to start with this one:
http://www.itl.nist.gov/div898/handbook/pmd/section1/pmd143.htm
As I understand, what you need here is the recursive least squares algorithm:
https://en.wikipedia.org/wiki/Recursive_least_squares_filter
The core element is the "forgetting factor" which gives exponentially less weight to older error samples.You will need to find a suitable value for this coefficient.
I think you need to have a valid basis for why you think the more recent observations should be considered more important. For example, in the relationship between advertising and sales, a logical case can be made that recent advertising has a greater effect on sales than advertising in the past. Without solid justification, it would simply be fishing for relationships. It is better to use time period dummies to account for temporal variation, if you are not interested in the time part of it per se.
@ Andrey, my concern is how I can justify and validate the wights that I'm using in the weighted linear regression.
@ Girish, In fact, the dependent variable (a productivity index) was significantly improved over the study period (10 years). Therefore, the year of the observation was considered as a predictor variable which had a positive impact on the dependent variable. However, the year itself doesn’t have any effect on the dependent variable per se; rather, it is the other factors occurring in the same time period which result in improvements. Hence, the latest observations are more close to the reality and more important. On the other hand, there are 10 years and there will be 9 dummy variables which is a lot for my data set.
You can use out-of-sample MAE to evaluate the effectiveness of using the weighted regression vs using the standard regression.
The "forgetting factor" can also be found using some kind of out-of-sample validation.
- Badges
- Science topic