I have a panel dataset of 120 different countries measuring a variable over three periods. This variable indicates the percentage of 1000 respondents in each country that answered yes to the question. I am considering this representative of the probability that a respondent in each country will answer yes to that question. The dataset currently has a bimodal distribution across countries, with the results concentrated around 0.0-0.10 and 0.9-1.0. To transform this into a normal distribution, I am using a logit transformation employing the function
log[𝑝/(1−𝑝)]
where p is the probability of a respondent answering yes. However, in three of the countries, 100% of the respondents answered yes, resulting in a logit function that cannot be calculated of
log[1/(1−1)]
What should I do with these three countries in the sample? Is there a legitimate way to lower their values from 1.0 so that they can be used in the formula?
I will also be averaging the panel data over the three periods to create a cross-sectional data set. The three countries of concern have values of 1.0 in a single period, with their results being less than 1.0 in the other two periods, meaning their average probability would be less than 1. Would it be appropriate for me to average the probability values across the periods prior to employing the logit transformation? an example of these two options is formulated below.
Log{{[p1/(1-p1)+p2/(1-p2)+p3/(1-p3)]}/3}
Or
{log[𝑝1/(1−𝑝1)]+log[𝑝2/(1−𝑝2)]+log(𝑝3−(1-𝑝3)}/3
Dear Zachary Brower ,
The only problem due to the use OLS is that it can yield predictions above 1 and below 0. If you are not interested in prediction, you will be safe using OLS (with appropiate standard errors, etc.). Consistency in OLS does not require normality.
I would not average the data and estimate my model using a cross-section. A panel allows you to perform models that can be more consistent and precise. For instance, you can estimate fixed-effect models using the panel. Furthrmore, random-effects panel models will be always better than a cross-sectional OLS or logit model.
Best,
José-Ignacio
If you do your modelling in a generalised linear model it is the underlying probability (or proportion) (sometimes known as the expected or predicted values) that is transformed in the logit link function not the observed values, hence no problem. The modelling in its binomial form will also take account of the inbuilt heterogeneity - greater residual variation around 0.5, least as approach 0 and 1. It is not usually a Normal distribution that is assumed for the stochastic part of the model.
Logit models are often used with binary (0,1)data and when that is the case the empirical logit is either minus or plus infinity (seemingly impossible to analyse), but things again work in the modelling framework.
GIven the repeated cross sectional nature of your data (I am presuming that it is not the same 1000 that respond each time) you may be interested in this:
Article Investigating the Macro Determinants of Self-Rated Health an...
Why should you do this instead of using the raw data?
Method Non-linear regression -Why you shouldn't take the logarithms...
I guess that if all respondents of a country answered "yes", the sample size of these countries is quite small. I would merge those country with one sharing similar characteristics.
If I was using random effects analysis I would not merge them, the use of the denominator in the binomial takes account of such variability.
Don't transform the response. Use an appropriate statistical model (binomial or quasi-binomial) instead.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA