Hi Laura,

I would have thought that 'think aloud' methodologies, frameworks, deign and protocols would work very well with qualitative  processes. The architectural design components aside - i would argue that your suggested 'communication units' are about established narratives, beliefs. experiences, beliefs and opinions - and they always align themselves to qualitative enquiry. 

I can't see why not. I would say they're as reliable as anything else. Depends upon the context of use and will give some insight into experience, method and reason; as Dean has alluded to. Good luck ;)

I've used the Constant Comparison Analysis method (an integral part of Grounded Theory, but can be used outside of it). There's a good high-level explanation in Fram, S. M. (2013). The Constant Comparative Analysis Method Outside of Grounded Theory. The Qualitative Report, 18(1), 1–25.

I agree that you could use think-aloud protocols in qualitative research, although I think they would work best as part of a qualitative interview, rather than as your sole method of gathering data.

One area where they have been used extensively, which borders on qualitative research, is for cognitive interviewing in survey research. Gordon Willis is a leading researcher on cognitive interviewing, and you can find a summary of his ideas at:

http://www.uiowa.edu/~c07b209/interview.pdf

You might also want to read the article on Think Aloud Protocols in Wikipedia, which indicates that the earliest work was by Ericsson & Simon among others in the field of artificial intelligence -- which is certainly where I first encountered it.

Thanks for that David.  Will follow up those links.  My idea is to use them as part of a larger project which looks at the recognition of suffering; the other methods being used are observational, looking at interactions between healthcare professionals and patients.  The Think Aloud section would involve participants (healthcare professionals) being given a "vignette" based upon a fictitious case (based on observations of care derived from the other phases of the project).  I hope to use this to look at the decision-making process when presented with a complex patient scenario.  So at the moment I am trying to ascertain whether the Think Aloud method will add anything more than undertaking an unstructured interview as is often done in phenomenological research...Gut feeling is that it could be very powerful.

Hi Laura,

Hhhhmmm - it's great to get this level of feedback from you - but I am a little torn by your suggestions. Phenomenology (in its very many different schools and forms) is far more comfortable territory to me than vignette's and think aloud methods. Personally, I find 'presented' case-studies a little stilted and leading. To me, they are far less likely to  result in a rich and natural narrative than would an unfolding phenomenological interview. That said, it has to be the right phenomenological approach - and that is a whole new 'can of worms' in itself. You also infer that your phenomenological approach would be unstructured - and it could just as easily be semi-structured - which would give you a different experience and findings. 

Hi Laura, I successfully used think aloud interview protocols in my doctoral dissertation. I used Gupta's (A practical guide to needs assessment, p. 47) Sample Interview, Protocol, Script, and Questions as my base - then semi-structured and unstructured interviews (phenomenological interviews) following a retrospective verbal protocol analysis during the task under research.  A great book I used was Working minds: A practitioner's guide to cognitive task analysis by Crandall, Klein, & Hoffman (2006). They used the think aloud process extensively in a qualitative process as I recall. Hope this helps. Steve

Given your area of interest, I am presuming you have read the work of Sandra Nutley and Huw Davies. So I'll move on to talk about vignettes; then 'talk aloud' methods and phenomenological analysis.

Vignettes

First, which model of vignettes are you thinking of using.  There are two distinct models in the literature [I can give you full citations, for anything you are particularly interested in following up.] which as far as I can tell have not been differentiated by name. 

One model (which we might call the 'completion vignette') is to present a scenario and then ask the respondents what they would do next (see for example Jenkins et al (2010). "Putting it in context: the use of vignettes in qualitative interviewing " Qualitative Research 10(2): 175-198.). There are multiple models of such vignettes including some which ask for responses at different stages and locations along a decision-tree (Jenkins et al).

Completion vignettes have some similarities with methods used in cognitive research to test decision-making processes (e.g. Tversky 1969) though the 'tasks' in cognitive research tend to be quite narrow and less complex than vignettes.

The second model of vignette, (which we might call the 'discussion vignette' is to present a complete scenario and ask  respondents to discuss. 

Analysing data from either approach has issues.  The discussion vignettes have been subjected to some debate as to whether they tap into decision-making or are subject to social desirability bias (Hughes and Huby 2004; Parkinson and Manstead 1993; Spratt 2001) or not (Rahman 1996; McKeagney et al 1996). O'Dell et al (2012) argue that social desirability bias is not important if one uses a constructivist analysis as it reveals the underlying construction of such responses. Personally I think it is hard to avoid social desirability bias in this model and that you don't get a good read on probable decisions.

Completion vignettes do not appear to be as susceptible to social desirability bias and are very good at obtaining data that is not generally presented in discussions or direct questioning. However, it is important that the vignette be 'plausible' (Jenkins et al 2010; Lilienfeld et al  2000). Your approach of basing the vignette on observations is very important.  The further the vignette is from the participant's experience the less useful it appears to be.

[Given your interest in phenomenology you may be interested in Jenkins et al (op cit) argument that the underlying mechanism for vignettes is use of a 'Thou-orientation'.  I agree that orientation is crucial but I think they misinterpret Schutz ('Thou' relates to one's experience of the other's  behaviour and explicitly excludes one's interpretation of the  cognitive processes).  Instead I suggest that Schutz's notions of 'typification' and schema offer more powerful  tools for analysis.]

In analysis it is also important to realise that decisions are often inconsistent because:

a) decisions are predicated on judgments

b) judgements are highly contextual and often largely based on 'System 1 thinking' - and on heuristics. (There is a very large literature on heuristics; if you are interested, you might want to start with Kahneman 'Thinking Fast, Thinking Slow').

From a phenomenological perspective one would characterise the variability of judgements as the activation of different schemata (System 1) and one might try and analyse the responses from that perspective.

I hypothesise that the reduced social desirability bias in completion vignettes may be because it 'elicits' response from the participants, who are responding 'more intuitively' to the scenario, using implicit knowledge from a schema: rather than 'discussing' the scenario explicitly. In completion vignettes, they are more likely to be using a preponderance of System 1 thinking  - which is more responsive and less concerned with self-presentation - than system 2 Thinking.

An important caveat is that it is difficult to assert that responses to completion vignettes are accurate reflections of probable behaviour (see Jenkins op cit). There is some literature that shows that other 'elicitation techniques' are good predictors of behavior (Breivik et al) but I am not aware of any evidence for the construct validity of vignettes.

Talk aloud

I am not familiar with Gupta's use of talk aloud techniques. My experience with them is mainly in usability testing, survey design or in cognitive research.  David Morgan's reference is good: also Sudman,  Bradburn and Schwarz (1996) "Thinking About Answers", San Francisco, Jossey-Bass.   

I have found 'Talk Aloud' a particularly useful for analysis of participant tasks (such as responding to a survey question, or  an elicitation stimulus).  I would think they would also be very useful for analysing responses to a completion vignette.   The method is really good at uncovering the details of thinking that led to a particular response in a task. 

For analysis of think aloud techniques, I have used both simultaneous think aloud and retrospective think aloud; and there are advantages and disadvantages to both.  My favourite approach is to video the original participant task, sat the participant down to review the task and stop the video to ask for explanations used think aloud - also videoing the talk aloud.  I look for inconsistencies in explanation, the use of metaphor and pronouns and probe such responses quite rigorously.  For example, 'Category instance generation' is used in cognitive science.  It entails asking the participant to specify examples whenever a metaphor is used in their conversation. A metaphor is a heuristic that 'explains' something without having to do so - it is implicit knowledge.  My aim is to try and flesh out the schema being used and understand the halo of connotations that  the participant attaches to a metaphor.

Afterwards I will use different analytical methods depending on purpose.  I have used a  spread sheet of quotes and observations linking both interviews along one row.  I have also used InVivo for text analysis. 

Phenomenological Analysis 

I have found Smith, Flowers & Larkin (2009) "Interpretative Phenomenological Analysis: Theory, Method and Research" London, Sage -  a really useful primer on phenomenological research and analytical techniques.

My apologies, I misspelt Huw Davies name.

Thank you all so much for your insightful responses - much appreciated 

You might also be interested in an article I have just read:   Claire Petitmengin "Describing one's subjective experience in the second person:  An interview method for the science of consciousness"  Phenom Cogn Sci 2006 5:229-269.

The thrust of the article is not related to your work and her method would be very difficult to use.  I also have issues with her model.  However, the focus on 'how' questions is useful and some of the other probes she describes strike me as very powerful and useful. 

Interestingly she is quite scathing of  Ericsson and Simon's (1980) article on verbalisation (which she calls the 'Talk Aloud' method).  I think she is a little harsh,  Ericsson and Simon were trying to explore decision-making etc. as it happens and explicitly avoiding exploring cognition that happened in the past so their purpose and methods were quite different. They also emphasised the value of detailed descriptions of cognitive processes and focusing on specific events.

OOps again. I just noticed that I wrote the 'completion vignettes' have been criticised for social desirability bias.  It is the discussion vignettes that have been crtitcised. My apologies.