I'm trying to replicate a study. I have 165 observations of a proportion variable (the distribution is skewed to the right as many proportions are low). I need to report the CI. I've applied arcsin(sqrt), computed the mean and SE of the transformed proportions, got the CI, and transformed back to the original scale.
Now, the thing is, that the total proportion of the sample does not fall into that interval. (I feel it has to do with the lower bound of the binomial, that's why the study is stretching those values by arscin(sqr)).
But how should I properly state this result?
For a binomial random variable, you can directly estimate the 95% CI without the need to transform to arcsin. Links below may help
http://www.mwsug.org/proceedings/2008/pharma/MWSUG-2008-P08.pdf
http://dept.stat.lsa.umich.edu/~kshedden/Courses/Stat485/Notes/binomial_confidence_intervals.pdf
http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval
Michal said: "I have 165 observations of a proportion variable (the distribution is skewed to the right as many proportions are low)."
It's not clear to me what your data file looks like. Do you have a single variable, where each case (row) is a proportion (value between 0 and 1)? Or is it 165 observations of a dichotomous variable (e.g., values of 1 or 0 only)? If it is the first situation, do you want a CI for each of the 165 cases? If yes to that, do you have the numerator & denominator used to compute each proportion? Or do you want just one CI for some pooled estimate of all of the proportions?
Thank you for clarifying.
Hi, thanks for interest. It's 165 proportions (between 0 and 1) computed as counts out of n=20 for each case.
I want to compute CI for that proportion for whole sample.
I tried comparing sample variance, skewness and kurtosis to that of the theoretical binomial with the same p. My data seem to be somewhat underdispersed and the transformation did not normalize it anyway. So what would you suggest?
It seems to me that you have poisson data. Your 'exposure' is 20 observations and your observed frequency is the poisson rate. In Stata, if my memory serves, you would calculate the confidence interval by having two variables
count = the count of events
exposure = 20 in all cases
. ci count, poisson exposure(exposure)
If you trust that your observed count x out of n follows a Binomial distribution B(n,p), you can find a confidence interval for the proportion p by solving equation (9) and (10) in the attached report. I have also attached the Excel workbook, which I used for calculating the confindendence intervals in Table 1 of the report. As described in section 7, you can calculate the confidence interval using n=20x165=3300 and x=sum(xi). Note that all this works only if your 165 experiments can be regarded as random sampling from a common batch of something having a specified property or not.
Thanks, I understand your ideas. The study I'm replicating used the approach with arcsin(sqrt(x)). But as it had proportions around .30, it worked out well. I have proportions around .05 with many zeros.
I'd explain more about the data if you minded:
20 is the length of record in minutes. It's divided into minutes and each minute is determined whether a certain signal (a tweet of a passing bird) did occur or did not. Thus the variable is only made binomial artificially. At least it's how the data are stored.
You may be interested in this article, which says that "The arcsine is asinine". ;-) The authors argue that the logit transformation ought to be used instead.
http://www.ncbi.nlm.nih.gov/pubmed/21560670
From Michals description of the experiment, I would (as Ronán also suggested) model the passing of the special bird by a Poisson process with rate parameter p (birds per minute). This means that the number x of tweets counted in a time interval T (= n minutes) would follow a Poisson distribution with parameter pT. In the worksheet I sent you earlier, I actually calculated a classical confidence interval for such a rate parameter (under the heading "Poisson distribution"). If you use n=165x20=3300 and x=165 in that worksheet, you get more or less the same confidence interval for the rate parameter as for the Binomial model. If you apply the Bayesian method described in my report and assume a noniformative prior for the parameter pT in the Poisson distribution, the posterior distribution for pT given the observed count x is a gamma distribution with shape parameter alpha=x+1 and rate parameter (inverse scale parameter) beta=1. In the attached Excel workbook I have calculated a confidence interval for p given x=165 in time interval T=3300 minutes. Not surprisingly, the confidence interval is very similar to that calculated from the Binomial model. The reason is that for large n and small p the Binomial distribution B(n, p) is approximately equal to a Poisson distribution with parameter np.
I think you must be clear about the type of data you are having.Accordingly your transformation is to be done.
- Badges
- Science topic