.

imagine you have 3 individuals a, b, c

for a and b, correlations are negative

for c, correlation is positive

assume the correlations are of the same order of magnitude : on the population {a, b, c}, you would conclude that the correlation is negative on {a, b, c}

.

now, imagine that S = {a, b, c} is a sample taken from a large population P and that you want to estimate the correlation on P given the sample S

- a first possibility is that this sample has been obtained by a simple random sampling ; then all sampled individuals a, b, c have the same inclusion probability and you would again conclude that correlation is negative in the population P (with all care due to the uncertainties brought by the sampling process of course)

- a second possibility would be that population P is known to be made up of three mutually exclusive sub-populations, A, B and C, with the following ratios (10%, 10%, 80%) ; assume a is drawn randomly from A, b from B and c from C, giving the sample {a, b, c} ; obviously, c represents much more individuals from the population than a or b : 8 times more in fact and if you want to estimate the correlation in the population P, you have to take that into account and it is quite possible that the weighted correlation will turn out positive (since the contribution of c is positive), while the correlation for the sample S (no weighting) is negative.

.

.

imagine you have 3 individuals a, b, c

for a and b, correlations are negative

for c, correlation is positive

assume the correlations are of the same order of magnitude : on the population {a, b, c}, you would conclude that the correlation is negative on {a, b, c}

.

now, imagine that S = {a, b, c} is a sample taken from a large population P and that you want to estimate the correlation on P given the sample S

- a first possibility is that this sample has been obtained by a simple random sampling ; then all sampled individuals a, b, c have the same inclusion probability and you would again conclude that correlation is negative in the population P (with all care due to the uncertainties brought by the sampling process of course)

- a second possibility would be that population P is known to be made up of three mutually exclusive sub-populations, A, B and C, with the following ratios (10%, 10%, 80%) ; assume a is drawn randomly from A, b from B and c from C, giving the sample {a, b, c} ; obviously, c represents much more individuals from the population than a or b : 8 times more in fact and if you want to estimate the correlation in the population P, you have to take that into account and it is quite possible that the weighted correlation will turn out positive (since the contribution of c is positive), while the correlation for the sample S (no weighting) is negative.

.

it depends on data

Also, i agree with Fabrice Clerot

May be in your case it will a good idea to start with classical null hypothesis or significance testing? Proving first of all that data tells you something you expect due to some reason in it, not due to the change. May be estimating p-value can help you. You also have to be sure that you data is distributed normally or otherwise. By knowing how data is formed you may chose between different statistics (t-Students, etc.) to reject or retain your null hypothesis based on your data. After knowing "more" about your data you can try to apply correlations. By the way, some of your "problems" may be due to something called "Simpson's Paradox". But this is just my assumptions, it's hard to tell without looking at data. 

Good luck!

Dear colleagues Thank you for the contribution

As a general rule I avoid weighting like the plague. Sometimes I have no choice, as in Fabrice's example. The problem is that I can use weights to force the data to conform to any hypothesis I choose to make.

Here is an alternate strategy to Fabrice's problem. Not better, just different.

Population A and B: sample 100 people from each group.

Population C: sample 800 people.

You now have a sample from each population that is the same proportion of the total population. For populations A and B I would average sets of ten people. For population C I would average sets of 80 people. I now have ten estimates for each population, and I proceed with other analyses without weighting. The problem with this method is that this method has hidden more of the within population variability for population C than for either of the other two populations. A partial solution is to discuss differences in outcome for this method versus Fabrice's.

You could try a resampling method (bootstrap, Jackknife, randomization test).

You may just have to play with your data for a while.

Of course if the problem is that you have 2 people from each of population A and B, and 800 from population C, then you have a sample size issue.

@ Clerot: Thank you for the information