My research group needs to annotate emotions through video observations with recordings of students' faces and screens as they interact with educational software. These annotations will be used in algorithms for detecting emotions. These annotations need to be made by observers and not by students themselves. We're struggling to find protocols for emotion annotation from videos. Does anyone know of any protocol with these characteristics or could to help us in this?
I encountered the Body Action and Posture (BAP) coding system a few years ago. It is mostly work of Nele Dael and colleagues and if your are doing it manually it considers the diversity of the emotion body expressions. It is worthy to take a look. This is one of the references I found, but go for the first author papers and you will find others. Good luck!
Dael, N., Mortillaro, M., & Scherer, K. R. (2012). Emotion expression in body action and posture. Emotion, 12(5), 1085–1101. https://doi.org/10.1037/a0025737
There are two relevant good papers I used as a reference for our dataset annotation (also video from facial image):
https://www.frontiersin.org/articles/10.3389/fict.2016.00027/full
https://link.springer.com/chapter/10.1007/978-3-642-15184-2_13
Also, if you haven't decided an annotation tool yet, we were quite happy with the one we used:
https://www.ncbi.nlm.nih.gov/pubmed/29308198
https://openresearchsoftware.metajnl.com/articles/10.5334/jors.ar/
The thing I like about it is that you can configure it to get the scale you need, labels, etc.
Good luck!
At the risk of appearing overly self-promoting, I also recommend CARMA for dimensional annotation. You can find the newest version at: . It has improved a lot since the paper that Anastasia linked. You may also find the following paper that discusses issues in observational measurement helpful:
Hi Tiago,
had you considered Ekman's Facial Action Coding System (e.g. https://pdfs.semanticscholar.org/eafd/a8a94e410f1ad53b3e193ec124e80d57d095.pdf) ? From the description of your research, it sounds like what you are looking for. It's very well documented.
Ekman's coding system though requires a certification, which is not easy to obtain. It takes considerable time and effort, as well as fees.. In the beginning we considered it but it was way more simple to use the CARMA and set our own dimensions. For our research for instance we evaluated the presence Joy from -8 to 8. You can set boundaries, step, evaluated emotion, it is really convenient.
Just to make clear, I am in no way related to the author/creator, and have no personal gain in promoting the software, I am just sharing my experience. I had looked exhaustively on available freeware applications for affect annotation, and at the time (haven't looked over the past year) it was the only one to serve our purpose, let alone the only one who was almost plug-n-play.
Thank you, Gabriela Baranowski Pinto . I found that the BAP coding system works with basic emotions and we are interested in nonbasic emotions like flow, confusion, frustration and boredom, which are common emotions in complex learning environments and in computer-based learning environments (our study target). In addition, we need a method of annotating emotions through video. Encoders (note-takers) need to observe recordings of students' faces and screens while they are interacting with a computer-based learning environment.
Thank you, Otto Condliffe . Ekmann's FACS considers basic emotions, but we are interested in non-basic emotions such as flow, confusion, frustration and boredom (common in computer-based learning environments, our target of study).
Thanks Jeffrey M Girard and Anastasia Pampouchidou for the sugestion. I am studying now about CARMA and verifying whether it meets, even partially, our need. We need a method of annotating emotions through video. Encoders (note-takers) need to observe recordings of students' faces and screens while they are interacting with a computer-based learning environment and we are interested in nonbasic emotions like flow, confusion, frustration and boredom, which are common emotions in complex learning environments and in computer-based learning environments (our study target).
@Tiago, FACS does indeed consider basic emotions, but there is also some precedent for using it to measure other states - confusion and frustration were measured (with specific FACS codes provided) in the following paper: Data Emote aloud during learning with AutoTutor: Applying the Fac...
- Badges
- Science topic
- Similar topics
- Psychology
- Emotion