I have generated a number of data sets that follow a specific distribution. They represent different layers of a certain system but they should be correlated with predetermined correlation coefficient without affecting their statistical distribution.
Hence, I am thinking of arranging every two subsequent data sets to impose their corresponding correlation parameter. Any ideas of how to do that, especially in R, will be appreciated.
If you are content with producing simulated data sets with a two-dimensional normal distribution you may proceed hierarchically as follows:
1. Generate the independent x according to N( mX, sX2)
2. Generate the dependent y according to N(a + b*x, (1-r2)*sY2)
3. Repeat steps 1 and 2 until you have enough pairs (x, y).
You can do that in any software.
The joint distribution of the generated data is N(mX, mY, sX2, sY2, r).
Thank you Manfred for your response. However, I did generate my data sets and I need to arrange one of them such that their dot product satisfies a certain coefficient. To clarify, say that I have two vectors X & Y, I need to arrange data elements of Y such that X.Y=c, where X is a known vector and c is a predetermined scalar.
I think Manfred's suggestion makes sense. But in this case, you will change your both original x and y values.
Hiba, Please try to explain your question or give a context a little bit more, so people can suggest you clearly.
Sorry for the ambiguity:)
Okay, lets say that I start with real data sets that are highly correlated and I want to generate data sets that have the same statistical distribution and correlation coefficient. Hence I generate two data sets X & Y that follow the same specific predetermined gamma distribution. These data sets should have certain correlation coefficient c, where c is derived from original data. The generated data doesn't have that same correlation c since the data sets are ordered differently. I can fix set X and arrange set Y to give me my desired correlation. How can it be done?
I hope that my problem is better explained. Any help is appreciated.
Make sure that you check out randomization as well. The difference is that bootstrap randomly samples with replacement while randomization randomly samples without replacement. The bootstrap will reduce the importance of extreme values in your data. This is good if the extreme values are a result of mistakes. This is bad if the extreme values are a natural part of the underlying distribution.
How is this randomization carried out Timothy?
What if I just compare my original data set B to my generated set Y as such; for(i in 1:n) compare B[i] to Y elements from [1 ...n] where n is the length of the data set. The Y[] with minimum difference, min(comp) gets assigned as Y[i]. Is that efficient?
Dear Timothy
it is only about illustrating the correlation so the extreme values should be of minor importance (anyway, in case of extreme values, correlation is not really appropriate);
for other purpose, rerandomisation might be better than Bootstrap even though I think that both methods are relatively poor with rare events as they usually are not represented in the original sample so that they will not show up neither in the Bootstrap distribution nor in the rerandomisation.
Hiba, you should use R to do either the Bootstrap or rerandomisation.
- Badges
- Science topic