I've got some data from an experiment in which participants were asked to complete a task (i.e. placing a group of objects onto a target using a tool) as fast as they could with the minimum number of errors (dropping the objects). As time is dependant on error (dropped objects cannot be picked up again, so the experiment terminates earlier than if all objects were placed without error) I would like to combine the time and error data into a single figure by creating a time penalty for each error, but I do not want to arbitrarily select this number. Are there any classic methods for determining what value this penalty should be? I imagine it will be a combination of maximum / minimum / average time and errors.
Any suggestions appreciated.
Hi Adam,
mind that your times won't be normal distributed, so simply averaging is not a good idea. There's another conceptual problem with analyzing error data: if someone does no correctly complete the task, what did they actually do? Are there different, and well-enough defined error modes etc.?
I'd have a different suggestion. You might treat your data as censored - if someone dropped the ball at time x, he didn't handle it successfully up to that time. In fact I'm working right now on a tool to analyze time data in such situations - would be interesting to try it in your case. Can we get in touch via mail? ([email protected])
I concur with Bernard's overall assessment. The problem is that you threaten the construct validity of your assessment if you start combining things arbitrarily - although time and errors are currently confounded, you would just make the problem worse by further confounding it with your error "correction".
I would instead recommend either 1) an analysis that examines the effects of your experiment and errors separately or 2) some kind of sub-group analysis - i.e. examining the idea of "errors" in one analysis, examining the idea of "time to completion" in another - and then perhaps examining "time to completion" only for those individuals that committed no errors in another analysis. This has important implications for generalizability, but that is a conceptual argument, not a statistical one.
I don't know what your data actually look like, of course, but if it makes sense, I might suggest something like multiple regression, which will allow you to model the effect of your experimental manipulation, controlling for errors (e.g. entering errors in the first step of a hierarchical analysis so that you can partial out the effect of errors before determining the predictive power of your manipulation). If you have multiple measurements per person, you'd need an analytic approach to account for random person-wise variation (e.g. hierarchical linear modeling). But the specifics would depend on your particular data and research questions.
Hope that helps!
you need a random walk model, e.g.
Kornbrot, D. E. (1988). Random walk models of binary choice: the effect of deadlines in the presence of asymmetric payoffs. Acta psychologica, 69(2), 109-127.
Condamin, S., Benichou, O., & Moreau, M. (2005). First-passage times for random walks in bounded domains. Phys. Rev. Lett., 95, 260601. http://dx.doi.org/10.1103/PhysRevLett.95.260601
Usher, M., & McClelland, J. L. (2001). The Time Course of Perceptual Choice: The Leaky, Competing Accumulator Model. Psychological Review, 108(3), 550-592. http://www.sciencedirect.com/science/article/B6X04-46F6SSF-3/2/d329b9098ae2dd13f894711cba626456
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Human-Computer Interaction