Your response rate is low but not unusual for a mail-out survey, and a low response rate does not automatically indicate bias. But a bigger problem would be trying to do statistical analysis on an N of only 65. With such a small sample, your analyses will lack what is known as "power," so you will have a hard time producing significant results. Still, you might try some simple correlations and t-Tests to find out if there are any particularly strong relationships in your data.

If you do find some interesting results, this could lead to a mixed methods approach, where you frame your qualitative analysis as an investigation of those quantitative results, which is often called a "sequential explanatory design." If you cannot generate that kind of starting point, then you will need to think about what goals you will pursue with the qualitative data. Under the circumstances, it seems unlikely that what you collected would match a purely exploratory purpose, so what would be the best use for that data?

Thanks David. What do you think about the core analysis being qualitative and then simply providing means from the questionnaires for those respondents whose reflections fall into a particular coding category, without any means comparisons (such as t-test)? With this approach, the questionnaires could provide a description of those respondents who gave each categorized set of reflections.

That is why it is always advised to go for schedule rather than survey for qualitative question

If your intention is to publish in a qualitatively oriented journal, or to write an article that will primarily be reviewed by qualitative researchers, then including means on variables will not be useful (and perhaps the opposite). At a minimum, you need to ask what the added value would be for including such information.

If you do have enough quantitative data do some characterization of the different types of participants, then you might consider starting with a cluster analysis. This would sort the participants into categories that you could delve into more deeply with the qualitative data.

I hadn't thought about starting with the quantitative. I could create subgroups that are high/low on religiosity and high/low on spirituality, and then "describe" the groups using the analysis of their qualitative responses. Interesting - thanks.

Hi Cindy,

Low response rates may occur for several reasons, ranging from time constraints to a lack of interest. Armstrong and Overton (1977) examined nonresponse bias and various instances of mail survey endeavors. Armstrong's and Overton's study (1977) showed that even low response rates exhibited usefulness.

Armstrong, J. S., & Overton, T. S. (1977). Estimating nonresponse bias in mail surveys. Journal of Marketing Research, 14, 396–402.

Good luck!

--Adrian

Hi Adrian,

The Armstrong & Overton paper was indeed interesting. Maybe I could use extrapolation and include it as a subsection within my discussion section. Thanks for your thoughts.

E-mail survey response rates are generally low. Check out: Sheehan, K. B. (2001). E‐mail survey response rates: A review. Journal of Computer‐Mediated Communication, 6(2). It might be helpful.

In your case I believe you have 13 respondents out of 65, you might what to considered a mixed approach. You might even be able to understand why the response rate is so low in the process.