In my study 4- and 5-year-old children had to compare quantities in different conditions. I'm not very surprised that decision times are a mess whereas the rates of errors follow a nice and predictible pattern.
In my paper I would like to refer to other studies wherein decision, solution or reaction times could not be considered as reliable either.
Have you ever published something similar or do you have a paper by someone else in mind ?
The inverse efficiency measure mentioned by Steven Samuel and Klaus Willmes may increase the variability of the data, especially when relatively few trials per condition have been conducted. This may decrease power substantially. The inverse efficiency measure can, however, be a good measure of percentage of error and response times correlate highly. See http://www.psychologicabelgica.com/articles/abstract/10.5334/pb-51-1-5/
Dear Catherine
doi:10.1016/j.janxdis.2013.11.004
See also:
http://digitalcommons.unl.edu/dcnlfacpub/44
Generally there is no reason to consider reaction times as reflecting just one process except with the simplest of repeated tasks. So would you assume that 1) each child was making the same decision in the same way? and 2) that the strategies and prior knowledge of each child were equivalent? If not then the only real reason to measure RT was probably to help identify guessing behaviour (some very fast responses and some very slow responses).
I would be very happy if you would consider sharing your data (after you have published!) - I spend a certain amount of time testing models of reaction time.
Best of luck
This is a slightly different approach to what you seem to be planning, but you might consider using 'efficiency scores' - dividing RTs by accuracy rates. I've seen a few papers using this method recently (although with adults). Here's one example:
http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=9275377&fileId=S1366728913000564
If results with these calculations support your accuracy rate results then your writeup might be a bit easier!
Steve
Hi Catherine, as Steven suggested, we also mostly use efficiency scores (RT/ACC) to account for speed accuracy trade offs. Have you checked for this? Is there a correlation between the RTs and the ACCs? In that case, you might consider using this formula. Also, which stimuli have you used? In case of non-symbolic stimuli, it is indeed possible that accuracies are more reliable. Good luck! Best, Delphine
Yes It is non-symbolic Delphine. You say accuracies are more reliable in that case. Can I find that in one of your papers ?
Hi Catherine, we don't have direct evidence for this, but as accuracies are even used in adults because there is enough variation and enough mistakes in their data, it can be assumed that this certainly will be the case in children. If you have enough variation in the accuracy data, it is logic to use these data, because the RTs don't add much information then. Gilmore, Attridge and Inglis (2011) as well as Lindskog, Winman, Juslin and Poom (2013) showed that accuracy data are reliable in adults I believe, maybe you can finc inspiration in those papers. Best, Delphine
Dear Catherine, to follow up on Steven Samual's earlier contribution, Brian Buuterworth uses the (mean adjusted) inverse efficiency measure in much of his work on numerosity precessing in young children as the most appropriate performance measure. If you search the web using "Butterworth dyscalculia screener" you will find relevant information. Best Klaus
Following from Delphine- Gilmore & Inglis have a recent paper which suggests that accuracy is the way forward for ANS tasks:
http://www.sciencedirect.com/science/article/pii/S0001691813002576
The inverse efficiency measure mentioned by Steven Samuel and Klaus Willmes may increase the variability of the data, especially when relatively few trials per condition have been conducted. This may decrease power substantially. The inverse efficiency measure can, however, be a good measure of percentage of error and response times correlate highly. See http://www.psychologicabelgica.com/articles/abstract/10.5334/pb-51-1-5/
Hi Catherine, perhaps our study is useful as an example of the problem you described. See section 2.3.3.2 in this link: http://www.sciencedirect.com/science/article/pii/S1041608013001283
Hi Catherine,
we have done reliability analyses on Number processing tasks on both accuracy and RT. The RTs on the nonsymbolic comparison task are not at all reliable. This data has been reported in
Article The association between numerical magnitude processing and m...
- Badges
- Science topic
- Similar topics
- Psychology
- Developmental Psychology