Laws governing telephone contacts by survey firms have made it difficult to maintain a double panel approach toward prevalence studies. For that reason, we're planning to drop the phone sampling and use a weighted online panel instead. In our prior studies, the phone panel has been more conservative and less risk-taking than the online panels and we have controlled for that by weighting. I am interested in learning how others have controlled for that difference when using an online panel only?
I agree on the limited value of telephone surveys relative to online panel surveys, especially if you do not have a regulated mandate and substantial funding, such as a census agency.
However, I don't have an exquisite answer for weighting panel samples. Nowadays, my approach to population surveys (not just prevalence types) has been to establish quotas for age, gender, and geographic distribution that reflect best available census estimates (2021 in Canada) as well as other authoritative provincial sources. I think you could potentially include other quotas for employment, education, household income, ethno-cultural identity, etc. This must be balanced against the difficulty and cost of reaching your 'target' population.
More recently I have been focused on building large cross-sectional samples at the same time as longitudinal ones. This typically involves a target sample of around 5,000 with survey administration taken at 6 month intervals. For example, one survey I am overseeing has a x-sectional 'super-sample' of around 9,000 unique respondents who completed surveys 6 months apart--the repeated measures sample is around 2,000 with another wave coming up in March. This approach works for me because the questionnaire doesn't change and I place higher value on flexibility and the ability to have greater resolution in geographic areas with smaller populations--even a seemingly representative sub-sample in a smaller populated area/community would give me pause if the n was very low. The longitudinal sample derived from this process may also provide more conservative estimates of key outcomes.
Another limiting issue with survey panels is that many companies predominantly recruit people into their sampling pool in English, which I think has the potential to exclude important key-informant groups.
Good luck with your design process and thanks for your question--it's important and could benefit from more discourse.
Note: My work is no longer focused on gambling.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication