Hi Idris, to be honest with you, I am not sure what the problem is, because I don't know how you are analyzing the data. In the sort of growth modeling I do, this wouldn't be a problem, because I would have a time variable, which I can code however I want, so I could code it in years 1, 2, 3, 4, 5, 7, 9, 11, or something like that. Alternatively, I could recode it so the full interval was 2 years and code the data where I had an even year as .5, so 1, 1.5, 2, 2,5, 3, 4, 5, etc. I am guessing your method doesn't allow that. The problem with interpolating a trend is that you are giving the interpolated points the same credibility as the observed points, and that isn't good. That's why we have multiple imputation, because it adds error to ensure the imputed data are not treated as being as reliable as the observed data. Although it seems radical, if you have a lot of years, you could try throwing out all the even years and see how that effects your results. I'd probably rather do that than the simple interpolation of the even years. Bob

I suggest you to plot a trend of whatever data you have. looking into that you might find out whether the data are following a consistent trend or not. if yes you should be able to interpolate to find your intervals. hope this helps

Hi,

It is very difficult because you cannot make a trend of the phenomenon in a long period (many years) if you just have even yearly data.

You can find a relative of measured data with some factors (also measured) and using simple linear regression to find a function about the amount of waste produced depended on some others factor (daily or monthly). Then, if you have data of others factor in the odd years, the amount of waste produced (daily or monthly) could be re-predicted by known functions. As these data, you can relative estimate the amount of waste produced for odd years.

Best regards,

Vinh

Hi Idris, to be honest with you, I am not sure what the problem is, because I don't know how you are analyzing the data. In the sort of growth modeling I do, this wouldn't be a problem, because I would have a time variable, which I can code however I want, so I could code it in years 1, 2, 3, 4, 5, 7, 9, 11, or something like that. Alternatively, I could recode it so the full interval was 2 years and code the data where I had an even year as .5, so 1, 1.5, 2, 2,5, 3, 4, 5, etc. I am guessing your method doesn't allow that. The problem with interpolating a trend is that you are giving the interpolated points the same credibility as the observed points, and that isn't good. That's why we have multiple imputation, because it adds error to ensure the imputed data are not treated as being as reliable as the observed data. Although it seems radical, if you have a lot of years, you could try throwing out all the even years and see how that effects your results. I'd probably rather do that than the simple interpolation of the even years. Bob

I Agree with Robert on this.

A fun thing that I've been pondering about though: Say, I have 50 data points. out of which the 10'th and 30'th data points are missing. I take the first 40 as training set and test out the data with the remaining 10. Now,

Firstly I interpolate and find the data for the 10th and 30th data point. I fit a model and now try predicting the 41'st data point.

Now, with the newly predicted data, I again re iterate and re estimate the 10th and 30th data points. Now, with the newly interpolated values I again predict the 41st data point continuing this process until a local minimum is reached. This local minimum corresponds to the data model that well agrees with the three unknown data points. (10'th , 30'th and 41'st). but note, there have been no new addition of information. hence,

Is there any use to this? And under what conditions will this be useful?

Obviously, If there really was an underlying definite model whose parameters were not known, then it could be argued that the above method is useful. But in most statistical problems, the underlying data model is a mere assumption based on the observations ( The data model that best fits the observation ) Which might never be true, because the inherent model as in the stated example might change based on the company's structure and innovation.

Does interpolation help even in cases like this?

Following up on Robert's response:

I also usually would use a trajectory model for a situation like this. Uneven spacing is not an issue -- especially with many times of observation -- and there would be no need to discard data.

I don't see multiple imputation helping, though, as you have 100% missing on your variables in the off years. The exception to this statement would be if there's a meaningful scaling of time other than calendar year and you could consider this along the lines of an accelerated cohort design.

Pat

Dear Abdul M Ziyath,

Yes, we need to develop a model based on the factors influencing the waste generation and then use it predict the waste generation for the missing years! However, I am not sure whether there are the information on those influential factors.

Dear Vu if I have just even years I could take them but the company changed their style. I also think I made mistake to asking question, I didn’t mention days or months because I just have a data and I am assuming it is represent the year. As Mr. Abdul said, I don’t have that information. I used interpolation before ask the question, thanks to Mr. Babak and Mr Robert, I realize I was doing correct. I didn’t have many years. So simple interpolation was best idea, I think. Thanks everybody to figure out the solution.

Dear Idris, imputation of data is very controversial in statistic (as said in the posts here before). Simple interpolation could give you some values for yearly produced wastes on odds years, but to which extent you can rely on it? Very difficult to reply, if you are not considering the others variables influencing waste production. One simple solution is to compare year by year the following data (that the company for sure will have on its side): turnover (as a complementary parameter, since it is influenced by many factors year by year, i.e. inflation, variable costs, etc.), incoming raw materials, outcoming finished products, and others specific indicators of process of the company you are analyzing. With this information you can be more confident on the imputation fitness. On the other side the solution could be more simple do not care of waste production in the odds years and carry out the analysis on the real dataset you have. In this case if the years dataset is quite large you can make some consideration on the missing values. I think that to decide what is the best choice (imputing missing or not imputing missing) is also dependent on the scope of the analysis and the presence of the above complementary information to improve your the data fitting model.

I hope this help.

V.

You can impute missing data by using the average value of the surrounding data to fill into the missing years; or as others have suggested, do a linear regression and impute the missing data that way. But if your data varies wildly year to year, this could be a problem.

Hi Idris!

I would suggest that you overlap the graphical presentation of all data which you collected with that of the third proposal made by Robert. The overall trend of the graphical data presentations should still look the same. If so, I guess that result would make you more confident to use Robert's third suggestion so that you will not need to add imaginary data(which will truly add error to your analysis).

Google on "Survival Statistics" as that offers methodologies for how to deal with incomplete and or heavily biased data.

If you have adequate numbers of complete data you can use machine learning approach to predict missing fields !

Article Machine Learning Based Missing Value Imputation Method for C...

Thank you all to giving ur precious opinions