My colleagues and I conducted a study on the prevalence of gambling addiction in Ohio .The study was based on a household telephone survey of 3,600 Ohioans (18+) to get baseline prevalence data before the casinos opened. We used multistage random area probability samples (selection of individual with a zip code), and sampled people from the entire state (1,200) and over-sample people from each of the regions where casinos were going to be located (600 x 4). Data was weighted on age, race and gender in order to more closely reflect the population and enable us to make generalizations from the sample data about the population of adults in Ohio and the four county clusters. Originally, we analyzed weighted data from each of the regions and state separately, but we were wondering about combining all of the data into one analysis in order to run more sophisticated analyses.
My question is this: If we combine the data, should we “rake” it so that the areas with overrepresentation aren’t biasing the analysis? If yes, how should we do this/what are the resources to learn the raking technique?
I think you're definitely have the right goal of wanting to analyse all available data together in the one analysis.
I guess the answer to your question depends on whether you want to take the perspective of having the goal of (1) calculating a proportion or mean of your population (in which case raking is convenient), or (2) whether you approach it as modelling (in which case you should be most concerned with including all relevant predictors). Both stances, if implemented carefully, should be approximately equivalent.
(1) should be feasible if you can specify a 'region' factor, that will enable you to adjust the population estimate via raking, same as you would do for the demographic variables. As for (2), it would be handled in much the same way by including the region factor as a predictor. In (2) you would generate your estimate of state prevalence by producing predictions from the model on a 'grid' of all permutations of the predictors for which information is available, and then weighting appropriately (based on census data) before aggregating. As you can see, in either case you incorporate information based on region,along with the demographics.
Your goal should be to produce a good uncertainty estimate, as well as an unbiased prevalence mean estimate. If you're comfortable in R, I can give you some pointers to appropriate packages for both (1) and (2), depending on which way you want to go.
Hi there,
I'm an epidemiologist, and I've been watching from the sidelines how the Behavior Risk Factor Surveillance Survey (BRFSS) in the U.S. recently converted from their old weighting style to raking. You should really check out what they do now, because they have quite a bit of doco on it since everyone who uses the dataset needs to understand it. They rake on very similar variables, only for the nation. They are a SAS shop, so will have canned SAS code; if you prefer R, our colleague above is a better source for code. I'm thinking you are trying to do (1) above (since that is what BRFSS is doing with the nation), and so I figured you might get some good ideas from how they did theirs. Here is a more "layperson" description of their new raking methodology, but if you go to their site they will have tons of doco: http://health.mo.gov/data/brfss/BRFSSweightingmethod.pdf
The State of Mass will probably do something similar to your state to get a baseline on gambling addiction prevalence, as we just voted in the first casino here, and there is (logically) public talk about doing that baseline study. Hence, I'm interested to see your results! Good luck.
-Monika
Exactly what is the question you are trying to answer? If it is just to get at gambling addiction, then your answer is the prevalence and how it is distributed among age, gender, race and region. What "additional" answers do you want?
The right answer is a nuanced one:
- do not rake it when you do a regression so as to appropriately see the strength of the relationships between variables
- rake when you extrapolate the results and do inferential work. Here you need to show a balanced picture of what goes on in a population, what is the magnitude of some phenomena, etc.
Interesting ideas from everyone. The first thing that popped into my head: why not weight according to over-sampled / not over-sampled region? But you might want to take a look at
www.fcsm.gov/07papers/Gaines.VII-C.pdf
which discusses methods for analyzing responses in multiple race groups. Proximity to casinos is another kind of grouping, and this reference may be helpful.
Dear Rick,
You asked a good question about the data of a valuable study.
You can calculate "propensity score" to better match your cases, and then rake the data. This will help to generalize the results.
Regards,
YJN
Hi Rick,
Combining the data will give you more explanatory power and greater likelihood of finding significant differences between groups, especially if you have numerous regions or regions with large confidence intervals.
Raking is necessary if you believe the sample is disproportionate to your population and/or response rates across influential variables (e.g., age, gender, etc.). Here is a paper discussing some of the issues and how to do it.
Good luck.
http://www.amstat.org/sections/srms/Proceedings/y2004/files/Jsm2004-000074.pdf
Darren
- Badges
- Science topic