Response time (RT) measures are ubiquitous in experimental psychology. It is well-known that RT rarely fulfills the assumptions of ordinary linear models: the residual distribution typically is strongly skewed and residual variance increases by the mean (heteroskedasticity).
Theoretically, there is a member of the generalized linear models (GLM) family that would be a perfect candidate for such a situation: the Gamma model. Most experimental designs involve repeated measures, which requires modelling random effects models. So, a gamma multilevel regression would be perfect.
In the past few years, I have tried gamma regression (exponential, respectively) on several RT data sets. I was never able to obtain reasonable results. Regardless of the implementation used (lme4, blme, glmmADMB, MCMCglmm, SPSS), I was never able to get results. Almost always, the respective program would throw an error.
Has anybody had success with gamma regression of response times? Or is everybody stuck with log transformation, like me?
I'm curious why people seem to not have considered treating response-times as time-to-event data and fitting a Cox PH or an AFT model to it.
Alex,
the models you suggest are survival models, right? Do they support repeated measures?
Dear Martin,
I appreciate what you are trying to accomplish. However, it is not clear from your question what exactly the technical problem is. I can ask Mathematica to provide a fit statistic without it crashing. However, the results are nothing that I would trust.
Despite what psychologists usually pretend, RT is not a product of a singular or even a singular set of processes. At absolute best you should expect that 1-3% of scores will reflect lapses of attention/distractibility. RT for more complex tasks is vastly more complicated.
The problem with modeling RT in close detail is that distraction or delayed response coincide with the long tail. The gamma distribution cannot be stably estimated from a reasonable amount of data in this case. Some of the better models using the ex-gaussian curve; this fits most RT datasets but will not help you in your problem.
In my own case, I found that removing response sequence effects removed so much skew that I can often use conventional GLM afterwards. However, I have never managed to publish using this sort of model - resistance from reviewers was fierce.
I have used frailty models for repeated events analysis with a gamma frailty term with time to hospitalization data (here after discharge one is again at risk for another hospitalization). But it seems to me that you are measuring time to a singular event - answer to a specific question I would guess (potentially with replicates). There are various parametric and semi-parametric choices for the hazard function, and frailty terms can be used to represent the correlations between some measures. It would seen that in the case of response times to test questions there is also a competing risk at play. That is, time to incorrect, and time to correct answer models. Choice of model is most dependent on the precise question you are hoping to answer or the comparisons you are hoping to make. What are the most important comparisons for this problem?
Jeffrey, that's a good idea. Found a few implementations in R, already. Not too surprisingly, they ground on Gamma or lognormal error distribution. Let's hope that these implementations are more robust than the one I tried before.
Bruce, thank you for sharing your experience, especially the part with the reviewers ;) It was my humble presumption that the Gamma model would allow for some variation in the data generating process, like you describe, by virtue of the dispersion parameter. Was that naive? The problem really is not the lack of fit; it is the the optimization algorithms that do not converge. It is also not the absolute fidelity of the model wrt the cognitive process I am out for, I am just looking for a better approximation to the data compared to the gaussian model.
As I said earlier, I have tried several data sets. Some data I tested was, for example, the time it takes a person to find an information on a website. A more recent one is response times in a modified Stroop task. Of course, all incorrect responses are discarded.
In the meantime, I had some success with a GEE model with a gamma term. This is okay for the experimental data, but sometimes I need more flexibility in modelling the random effects, for example having them crossed for material and subject.
Use data from one subject at a time; wait till the data are stable; find the 25%, 50%, and 75% quartiles; choose the gamma with the same percentiles; this is
your initial estimate of the 'theoretical' distribution. Then, trim the real data by excluding the 'tails' (e.g., theoretical RTs below the 3rd and above 97th percentile; symmetrical trimming leaves the median unchanged); then best-fit the gamma to the remaining data using your initial estimate as a start point. (Trimming 3% follows Bruce's suggestion below).
I used grid search in matlab, rather than fminsearch, as the latter crashes, I don't know why. Mathematica does a better job. Grid search is slow but MUST converge unless that start-point parameters are ridiculous. Least-squares fits of RTs is not ideal because it puts too much weight on the longer RTs, which are often less informative. Using rate= 1/RT and best-fitting (least square) the rates is better.
I have found the Gamma fits saccade latencies very well, but motor reaction times not quite so well - if the faster RTs are fit nicely, the mid- to long- ones are not quite so good. So you may need to think about how much error you will tolerate in the 'best-fit' case. My suggestion: comparing cumulative functions, rather than binning, is the way to go; with binning, so much depends on bin width and bin location that the fit is too subjective. I hope this helps.
**
an alternative is Carpenter's LATER method of plotting the z-score of the probability
of a rate exceeding a certain value, against that value; this can yield straight
lines, and when it does, the best fit is easy to obtain by linear regression. But the model is not a gamma.
I have used the Statistica program for gamma and lognormal distributions on data of durations of precepts of ambiguous figures.
I know this is a very old topic, but this link could be useful. It does not directly concern the difficulties with Gamma mixed model GLMs, but it does give you several options for analyzing RT data:
https://lindeloev.shinyapps.io/shiny-rt/
- Badges
- Science topic