I want to compare different facial expression recognition approaches on a (not yet) publically available dataset.
For this purpose I need at least one software that I can apply on my dataset videos out-of-the-box. A software based on a state-of-the-art method would be perfect. I also would prefer software that is free, at least for non-commercial research purposes.
I am not looking for methods, but for ready-to-use software including the trained recognition models.
Update: I do not need a fancy GUI or visualization, a simple command line application which reads an video file and writes a text file with FACS scores for each of the frames would be most helpful.
Hi,
As far as I know what you're looking for doesn't exist (yet). The links below are worth a look at, although with the caveat that I have no personal experience of any of them!
CERT http://mplab.ucsd.edu/~marni/Projects/CERT.htm
Affectiva http://www.affectiva.com/
Okao Vision http://www.omron.com/r_d/coretech/vision/okao.html
RealEyes http://www.realeyesit.com/homepage.aspx
Intraface http://www.cs.cmu.edu/~ftorre/
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Rosenwald, D., & Torre, F. D. l. (In press). Manual and automatic analysis of facial affective reactivity in major depressive disorder. IEEE International Conference on Automatic Face and Gesture Recognition, Shanghai, China. PDF
Hi,
I do have experience with CERT for detection of AUs in out-of-the-box videos. It can do what you want as you can load up a video and then export the AU activations (plus some other detection scores such as head rotation, happy, sad, etc.) into a file - although it does depend on the kind of software version you have if I recall.
Having said this, the reliability of the CERT software (like all packages out there) does have its limits. The software works best for frontal, well-lit, reasonably large-area faces. Anything else will throw off the trained models to various degrees.
I think that CERT will be a very valuable benchmark, BUT if you really are keen on reliable FACS-signals in your database, you need to invest some time in manual FACS coding.
Best
Christian
CERT is a good hint. Thank you. Update: Obviously CERT is not available anymore, because the authors of CERT want to sell its successor. Although they provide a 50% discount for universities, it is still unaffordable.
I also found two AU detectors which however do not output AU intensity scores:
TAUD: http://ibug.doc.ic.ac.uk/resources/temporal-based-action-unit-detection/
LAUD: http://ibug.doc.ic.ac.uk/resources/laud-programme-20102011/
Try OpenFace. It should be the best tool you can get for free. And I am not sure if any commercial tool is much better. (I have not tested any of those, but they probably suffer from the same problems.)
https://github.com/TadasBaltrusaitis/OpenFace
I tried OpenFace and it's excellent.. But it still isn't very accurate(but still very good).. is there any other software or package better than OpenFace.. I tried TAUD and wasn't satisfied..
We also published our code and trained model with our (winning) contribution to the ICCV 2017 Real Versus Fake Expressed Emotion Challenge, see https://github.com/fsaxen/NIT-ICCV17Challenge
The method is described in P. Werner, F. Saxen, and A. Al-Hamadi,Conference Paper Handling Data Imbalance in Automatic Facial Action Intensity...
I've made a recent search about this topic, hoping to find more open source software besides OpenFace (Tadas Baltrušaitis). However, I wasn't successful. Does anyone have further updates?
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Software