I am shortly going to be constructing a survey whose responses will be based on the output of a Delphi Study (what are the most important supervisory or organisational factors within the HFACS model which influence negative performance at the user level).
The main output of the survey will be to determine why professionals (instructors or instructor trainers) with a supervisory or organisational role commit violations (routine, operational or organisational induced) within their role. By including specific examples of what happened and showing why, the outputs will identify those areas which organisations need to target to improve safety at the user end.
There are obvious biases that are likely to bubble to the surface, not to mention sampling bias, especially considering that safety culture will influence uptake. However, I am concerned about how to make sure that I get truthful answers from the respondents as without truthful responses, the study will be flawed.
I have some references concerning Social Desirability Scales (Fisher, 1993; Uziel, 2010; De Jong et al, 2010) but any personal experiences would be gratefully received.
Any examples of how to address the ethics of potentially compromising professional integrity would also be welcome, although the survey will be totally anonymous and I will not be able to track who the individuals are - which should go some way to alleviate concerns.
This is a common difficulty in survey-based research. Two tactics have been in use for a while, and have been found to be successful to a degree:
Tactic #1 is to project the respondent's decision (which may be awkward, embarrasing, or in this case threaten professional integrity) onto a 'third party' in the same position. So instead of asking "Have you ever (done this under these circumstances)?", you can ask either: "Have you ever heard of a colleague (doing this under these circumstances)?" OR "Do you believe that any of your colleagues (do this under these circumstances)?". The Michigan Centre for Survey Research found that this removes the respondent from the "hot seat", and allows a more objective response.
Tactic #2 involves a softening-up approach that places the topic later in the Q'aire such that this topic is tackled when some relaxing of the R's defenses has taken place, and in a live-voice or face-to-face interview especially, some degree of 'trust' or rapport may have been achieved. Then the Q is approached by means of several 'lead-in' Qs to soften up the R's defences even further. Thus a first Q might ask "Do you believe practitioners are often tempted to (do this) even though it is unethical?" The next Q will ask "Have you ever been tempted to (do this)?" Then: "At that time, did you actually envision yourself (doing this)?", and finally: "And on that occasion, did you actually (do this)?"
Of course neither tactic guarantees that the R will completely reveal the truth, but it does improve the chance of getting closer to the truth than suddenly hitting the R over the head with a threatening item. After all, nothing is guaranteed in a Q'aire beyond eliciting whatever responses the respondent decides to give you at that time.
................................/ts
Good idea Tim Sly. However, do you think the choices of a question should be modified into a percentage scale, instead of "strongly agree" to "strongly disagree?" The first Q: have you ever hard about this? The follow-up question: what percent of practitioners do you believe may do this?
Yes, that would work, but for each question, think carefully what the response choices means to the respondent. This is very often ignored, and the result is a sequence of responses that is obviously ridiculous, and irritating, leading to early non-response. Sometimes a percentage scale would work and in other cases, a simple 5-point scale would be best:
... "strongly agree" - "agree" - "neither agr nor disagr" - "disagr" - "str disagr"
I have just completed an online survey study that used a number of scales. How do I deal with respondents who have answered some scales uniformly at the high end, low end or at some specific point? Do I delete the entire data of such a participant or just the data for the compromised scale. Are there are research methods papers that have explored this topic. Thanks in anticipation
- Badges
- Science topic