I would say that it depends on the eye-tracker, the experience of the researcher and the context. When I have done studies with kids with Autism, with an remote eye-tracker in the lab the fall out rate was close to 5%. In this case the fall out rate is more dependent on an experienced researcher than if If you have a good remote eye tracker The only time I have been even close to 20% fall out has been in trials outdoors in extremely sunny environments. The task will also affect the quality of the measurements. A long and cognitively demanding task will result in more fall outs. Kind Regards

It depends on the eyetracker being used. Personally the worst rate I have had is around 30% but this is particularly high as the task demands were high (the video being examined lasted much longer than your typical stimuli used in eyetracking)

Thank you both for sharing your experiences. Of course context and the complexity of the task also play a role, that just makes sense. I think the experience-factor is interesting - ideally, measurements should be independent from the researcher, as always, but when using technology, there is just some know how that accumulates over time, I guess.

@Sebastian: thank you, too! Yes, of course once you have succesfully calibrated the tracker, that would be the way to go. I was thining of that first phase where you have people coming in and for some reason, you just don't get the calibration right.

I think experience is a large factor. When we started doing outdoor experiments many things happened and we learned from the mistakes (including really basic stuff, like once we found after a successful trial that the cable to the laptop got disconnected during the experiment). Sunlight is of course a big factor but we've used a large Mexican hat for participants to wear that helped (both with our SMI and Ergoneers eye-trackers). We usually do not take participants wearing glasses, this also often leads to problems.

Yes, mascara and eye-liner are a problem. Also asking people to remove it in the lab because they forgot can be a problem, because there is often some of the stuff left, some people don't like the removal at all, and also some eyes are a little irritated afterwards. Soft contact lenses up to 4 or 5 diopter ususally work fine, hard contact lenses don't work. With me, glasses work in half of the cases - I have not figured out exactly why. Also, I had a few cases where the eyes were really far away from each other and it was not easy to captue the pupil, or people just did not open their eyes enough and half of the pupil was often covered by the lid. So all in all, I just noticed, there can be a lot of issues, and thank you all for sharing your experiences!

These issues also depend partly on the population you are studying. In particular, clinical populations are very prone to data loss because (1) participants may not be able to foveate in the desired way during calibration, and (2) participants might move around a lot, despite any use of chin or head rest. Also, notably, eye trackers are more likely to work well with men than women (as men are less likely to enter the study session with mascara and eyeliner). There may be some racial disparities in tracking ability as well, in terms of differences in pupil occlusion (from different eye shapes) and the ability of the tracker to work properly with dark skin tones. Whether these things are issues will also depend largely on what method of tracking you are using, and the specifics of your setup.

People with corneal scarring, corneal reshaping (from using hard lenses--even while not wearing them), prism lenses, and other unusual visual attributes can also have difficulties calibrating, if light does not pass through the eye at the angles that the tracker expects.