Grezski et al., (2015) state that extremely short response times show a low data quality through a lack of attention of the participants,
I used a cut-off point of 150 seconds based on an average of peer responses for a proposed minimum response time ( without having them taking the survey, just by showing them the survey flow and the number of items which is 34).
Is this allowed? If so, could someone refer me to a paper where this is done or what the exact term is that I am describing?
I can't use an outlier test as only the participants that took exceptionally long to fill in the survey are outliers.
Thanks in advance,
Sebastian Brom
It's difficult to recommend a specific cut-off point without knowing the data.
But let me suggest.
1) You need to log-transform response time and look at participant_id vs log(time) plot.
2) You can try a 1-D clustering algorithm like the one described hereArticle Ckmeans.1d.dp: Optimal k-means Clustering in One Dimension b...
If you identify a cluster of cases where the response time is low, you can omit it from the analysis.
It's difficult to recommend a specific cut-off point without knowing the data.
But let me suggest.
1) You need to log-transform response time and look at participant_id vs log(time) plot.
2) You can try a 1-D clustering algorithm like the one described hereArticle Ckmeans.1d.dp: Optimal k-means Clustering in One Dimension b...
If you identify a cluster of cases where the response time is low, you can omit it from the analysis.
It's a debatable issue to have a say about the inclusion or exclusion of the responses that took beyond or lesser than the alloted time. If time allotted and time taken differ with higher degree then there might be severe chances of common method variance.
Thank you both for taking the time to respond.
I was not planning on putting a lot of time into doing a log-transformation and rewriting my results as I have a week left to finish my thesis.
My supervisor (who is now unavailable) asked me if I chose 150 seconds arbitrarily or "by peers as a pre-sample average?"
I understand that it is debatable, that is why I am in need of a paper on the topic that can tell me a yes or no.
If it is not that straightforward I will not use the duration time >150 seconds as an exclusion criteria.
Thanks
Suppose you use variable x to store response times.
Basically, all you need to do is to plot a hist or boxplot (e.g., in R):
boxplot(log(x))
or
hist(log(x))
This will show you if the threshold of 150 s is good or not. I recommend 'hist'.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey