In survey research we often do extensive cognitive pretesting and focus groups to understand how respondents think, in their everyday lives, about the complex and formal ideas and topics we want to measure. I'm doing some research now that doesn't have time for cognitive testing, but I thought someone here might have leads that would help.
The question: How do everyday people think about their "public program participation" (e.g., foodstamps, Medicaid)?
I'm thinking of two aspects here:
1) What terms do they use. For example, do people on "food stamps" use that term or do they know it as SNAP (or in CA "CalFresh"). I'm not really interested in what they are formally called (that's online), but I want to know how everyday people talk about them.
2) Are these questions really sensitive? We tend to talk about them as if they are, particularly if asked early in a survey, but do people on food stamps or Medicaid feel a stigma associated with it. Any stigma or sensitivity would probably be related to the same sensitivity mechanism that leads to high income item nonresponse. I haven't done a search to see if program participation gets similarly high item NR rates. From my experience, we tend to think survey questions are much more sensitive than they actually are to people, but sensitivity involves a lot of contextual issues like mode, placement in the interview, etc.
References to how other surveys ask about these topics would be helpful of course, but I'm really looking for research (qual or quant) on terminology use in everyday life and assessments of sensitivity. Use examples would be helpful if they are examples where the questions are asked in a screener interview (or within the first few minutes of an interview) and on the phone, rather than later in the interview and in other modes.
Thanks in advance for any leads.
This is very interesting question, Matt. I'll check in with my colleagues who focus on this population and see if they know of some studies that can answer your question. I know from some studies my students have worked on, that the question is usually posited to the recipients in more general terms, such as "food stamps" and "welfare", etc.
Matt, the Pew Research Center has asked questions about Medicaid and Medicaid use in a number of surveys. Same for Food Stamps. Go to their website and do a "Poll Questions" search. For example, this is what you get if you input "Food Stamps" as a keyword: http://www.pewresearch.org/question-search/?keyword=Food+stamps&x=13&y=9
Like anything else you want to screen for, far less stigma if in a "which of the following, if any" format than yes/no or a list with only the items of interest. I've asked up front at times if the stated topic in the intro fit so as to create a smooth transition.
So depending on the focus of the rest of the questionnaire, you could ask about sources of payment for groceries, health care that apply to them, or types of public programs in which they are enrolled, including Medicare, Medicaid (nomenclature varies by state; here in WA we would ask about "AppleHealth or Medicaid" to be clear, even though we code them the same way).
The approach to use also depends on how you are targeting/recruiting your respondents to begin with. If we were standing in the Pike Place Senior Center, these questions would be laughable and phrased completely differently. I was presuming above that you plan to use a general phone sample.
Thanks all. Ariel, I'll be curious to hear what you find. To give a little more context, we're thinking of asking these in a screener to be used for nonresponse adjustment and adaptive/responsive design of the data collection. They won't be used for reporting official rates, and so don't need to be strict, officially-defined inclusion screeners (i.e., they can be "Does anyone in this household use foodstamps?" as an example). We want the questions to be quick and comfortable (compared to the detailed survey questions that are often asked on this, and are often pretty hard to answer, particularly when long and complicated definitions are used).
I did a quick search here: http://wwwn.cdc.gov/qbank/QuestionSearch.aspx A colleague developed this at NCHS a few years back, and it links cognitive pretesting results results for questions in federal surveys. Searching "foodstamps" didn't work, but picking topic "Public Assistance Programs" found some results and some of this may be helpful.
If I find more, I'll update you.
I am awaiting a response. I'll let you know as soon as I hear back.
It seems that "public assistance programs" seems a bit esoteric, but perhaps lay-persons will understand that term better than we give them credit for...
Go Bruins!
Hi Matt: No research, just a few thoughts based on personal experience:
1) As Ariel L. says, abstract/aggregate terms like 'public assistance programs' by themselves tend not to work, because respondents key on specific terms. I see that problem a lot with phrases like 'public health.' It might work to use wording like "... any of the following public assistance programs" and then administer a checklist of specific items.
2) So, what specific names should be used? I suspect that many members of the public think in terms of the concrete mechanism by which they receive the benefits (i.e., where the rubber hits the road). A major current vehicle is the "EBT card" used in all states to provide benefits electronically. So, I would be inclined to include that term instead of (only) the antiquated term "food stamps."
Gordon,
I love when something I said in a meeting is later backed-up by your independent assessment :) But I didn't think of adding "EBT card" as an option, which probably is more current than "foodstamps" (which itself would be better than "public assistance"). In CA, we have a program called "CalFresh," which is our food stamps program. Despite it's catchier and more casual name, I've been wondering whether people who use really think of it by that name.
The bigger/other question we had was whether asking about these programs is too sensitive for a screener. For those of us with stable careers and incomes, being on food stamps would be a sensitive issue (particularly if our colleagues found out). But I'm not sure if it is for people who live in poorer communities and may be on food stamps and other assistance mechanisms off-and-on, and, more importantly, are surrounded by peers who also are. These kinds of sensitivity issues generally arise when there are comparisons across so-called "class." My family was never on food stamps as a child, but I had friends who were, and my dad was on unemployment at least once that I can remember. So my hunch from personal experience is that this is one topic that's only sensitive in our own minds (i.e., from our world view). Maybe something to study more someday :)
In the end we went with a question on medical conditions and one on MediCal (our state Medicaid program). They're under IRB review right now.
Matt - this is an old source and you may have seen it already, but it might be a good starting point: 2010 ACS Content Test Evaluation Report Covering Food Stamps/SNAP
Also, the 2015 National Survey on Drug Use and Health (page 736) has the following "introduction" to the question:
"The Supplemental Nutrition Assistance Program, or SNAP, formerly known
as food stamps, provides assistance for buying food. A special card is issued
which can be used to buy food in grocery stores."
It then goes on to ask if SNAP benefits were received and notes that WIC or free/reduced cost lunches should be excluded.
http://www.census.gov/acs/www/Downloads/library/2012/2012_Loveless_01.pdf
http://www.samhsa.gov/data/sites/default/files/NSDUH2015MRB/NSDUHmrbCAIquex2015.pdf
- Badges
- Science topic
- Similar topics
- Statistics
- Survey