I used R's retimes::timefit to fit an exponentially modified Gaussian distribution to reaction time data. It estimates ex-Gaussian parameters based on maximum likelihood and gives AIC/BIC/log likelihood of the estimated parameters, but I would also like to check the goodness of fit. Can anyone point me to how to do this? But even if I do have a goodness of fit measure, I can already see that visually plotting observed RT histograms and the distribution curve with the estimated parameters, not all conditions for certain participants have great fit when an ex-Gaussian curve would seem like a good fit (and other times when it's clear from the distribution that an ex-Gaussian curve is not a good description, e.g. bimodal RT distribution). What do you do in these cases when it is only SOME of the conditions for SOME of the participants?
I have collapsed across different conditions in my data for each participant because keeping all the conditions would result in too few data points for each parameter estimation, so I've run timefit for these different aggregated versions of my data and chose the aggregate version resulting in the lowest overall AIC/BIC. My second question is whether this is kosher? On the other hand, the estimations with the lowest AIC/BIC also seem to have the worst fit through visual inspection. The highest AIC but visually best fit are the estimations collapsed across participant, but I need estimations per participant....Thanks very much in advance for any help!
Dear Gina,
My first answer would be that ex-Gaussian distributions are not always a good fit, and when they are not it should be treated as meaningful. Instead of assuming that the distribution fits you are examining it. But why does it not fit everywhere? No easy answers. It may not fit the underlying processes which generate the RT. Or it may be that your apparent distribution is actual a mixture produced by variations in attention or strategy. While your description is not that clear, it does sound like you are maybe mixing data from different conditions and this would invalidate the exercise from the beginning.
I may be able to give a more useful answer if you can explain what you are trying to accomplish. Why do you feel you need a good ex-G fit when it doesn't match your data? Why do you think that your interpretation will be cleaner mixing together different groups of observations?
At the end of the day if you have few repetitions per participant then why are you interested in RT distributions? That said, you might consider survival analysis on weibell distributions (clustered by participant) to explore this variability as opposed to burying it.
Thank you, Bruce, for your reply!
Yes, sorry for the generic/vague question. I meant that for certain conditions for certain subjects, the RT distribution does look like an ex-Gaussian curve would be a good fit, but I don't get a good fit from the algorithm -- my question is if, very hypothetically, all the RT distributions for all conditions in each participant look like they are ex-G in shape in the first place and it is just a problem with the algorithm estimating the parameters, what would the standard approach be? Try a different parameter estimating algorithm just for those conditions for those subjects where the first algorithm doesn't do a good job?
And yes, sometimes it does look like an ex-Gaussian curve wouldn't fit certain conditions at all for some participants, and when this is the minority, then what? Abort and try another way to capture the variations in the RT distributions? Yes, I suspected that the distributions are a mixture produced by variations in attention and strategy, and I was hoping that by modeling them as ex-Gaussian that I could try and capture/tease out these cognitive processes apart better than if I just transformed the RTs and analyzed the means of the transformed RTs through linear models or simply ANOVAs. I also wanted to do correlation analysis with the various ex-G parameters and activity in specific brain regions (for which I would need per-participant ex-G parameter estimations per condition).
The RTs are from a language study where participants had to judge the abstractness of topics and where I have two conditions nested in each other (hence by collapsing across one condition, I automatically increase the number of repetitions per less-fine-grained condition to estimate the parameters from -- and I do see that estimation is better when collapsed although the AIC is higher -- is this still invalid?). I wanted to also separate the RT distributions depending on participants' answers (on top of the 2 experimental factors), and hence sometimes the repetitions per participant are low. So maybe I just have to accept that I don't have enough minimum number of trials across the board to look at participant responses, too. But this doesn't help me with the problem that not all the RT distributions for all conditions for all individual participants are ex-G even though they are when participants are lumped together during parameter estimation....
Dear Gina,
Ok, that helps. FWIW I don't every use the ex-G in my work and have theoretical issues with it. so better and stronger responses could be had from someone who is a fan!
1. If hypothetically this appeared to be an issue with fitting data which would reasonably be expected to be described by ex-G distributions, then I would consider two step fitting. Use software that is helpful for looking at mixture distributions and try to locate the normal mode and left sided variance. Then fit the gaussian component in a second step. In fact some of the original work by A. Heathcote and A Brown worked like this. Of course there are some complications in this so some people would use a combination of resampling (bootstrapping) and smoothing to deal with the fact that the third moment (skew) and higher moments are going to be poorly estimated in any reasonable design... Nobody would advocate doing this on an individual basis, but smoothing can go a long way towards taming a heavy tailed distribution.
2. There are plenty of advocates on ex-G fitting for RTs, so you are in good company. I think that you need to spread your bets a little bit. On one side it can be argued that the participants who do not fit well with the ex-G are probably doing something behaviourally different. They can be pulled from the data, or even contrasted with the rest to see if you can prove it!
I don't think I can give a general plan for how to do this and still get published! I think that if you use smoothing and/or resampling you should probably rank people on tau and use spearman correlation instead of pearson's.
A different approach is to try to correlate brain on the ex-G parameters plus the fit statistic for each person - this helps you incorporate the variation in approach to the task into your broader analysis. In a similar vein you can use the fit as an inverse weighting parameter for the correlation on ex-G parameter rankings.
3. Whether or not it is reasonable to collapse is something the reviewers will address. To the extent that the task conditions are similar enough it is ok. But I personally wouldn't be comfortable. It will most likely help fit the data as the problem is most likely a lack of determination in the tau....
Again for what it is worth, seperating people depending on answers does not always make sense. They are doing the same task in both cases, unless you think that they are in fact working in a multi-dimensional space. (Could be!).
If you decide to go ahead with your ex-G plan - good luck! It is not unreasonable.
If you are looking for alternatives, it is hard to really provide guidance other than to say that RT modelling has a lot of options for you. I mentioned Weibull curves in a survival analysis as something more stable and better estimated. Alternately you could identify the quantiles of interest. For example p5 (the 5% point) is a good proxy for the minimum, p50 for the mode, and p90 for the tail. These points are considerably more stable but I am guessing that you may not have enough repetitions and will have to be more judicious in your choice. Just remember that it may solve what you wanted out of the ex-G and requires no fitting at all.
Hope this helps!
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R