Facial expression analysis of President dr. Andrzej Duda’s talk on “Facebook Mentions” on 06-Aug-2015

Wersja po polsku tutaj.

Newly appointed Polish President’s speech on Facebook Mentions addressed to Polish internet users was analyzed with automated facial coding. The speech was streamed online on 6th of August 2015, just to 2.5 hours after the official appointment.

In this facial expression analysis, the three newest developments were combined: a) first official speech of a new Polish president who was b) talking on a new Facebook’s service and c) applying a new method – automated facial coding software, called FaceReader (2015).

The video lasted 25 minutes, 10 seconds, which allowed to analyze more than 45’000 frames, frame by frame. Such analysis by manual facial coding would take more than 60 hours to analyze (assuming that a human would code 1 frame in 5 seconds). FaceReader Software was able to provide such detailed analysis in just 30 minutes.

FaceReader Analysis: FACS, Heart Rate, Valence/Arousal & Basic Emotions

FR_view

Figure 1. Visualization of the analysis in FaceReader.

Figure above shows that FaceReader provides four different measures from the face. The numbers on the face – 2, 6, 7 and 12 with corresponding intensity ratings (the darker the color, the more intense the expression) – are Action Units (AUs) based on Facial Action Coding System (Ekman et al., 2002). The red heart with a number (in this case 84) indicated beats per minutes (bpm), which is a measure of heart rate. Next, the Circumplex Model of Affect (Russel, 1980) demonstrates valence (pleasant/unpleasant emotion) and arousal (active/inactive emotion). The expression line at the bottom indicate Basic Emotions (Ekman et al. 1969) (Happy, Sad, Angry, Surprise, Scared, Disgust)

Emotions of President Andrzej Duda

Expression_AllExpression_Neutral_Removed

Figure 2. Distribution of emotions of President Andrzej Duda.

Looking at the upper-side of the figure, one can conclude that Andrzej Duda do not use his facial expressions much and he generally has a neutral facial (87%) state during his speech.

A more detailed analysis was also performed where the neutral state is discarded. We can see at the bottom-side of the figure that there is a major difference between the emotions showed. It is observed that the most common expression of Andrzej Duda is happiness with 57% and surprise is following that with 34%. In Video 1, we can see a part of the analysis where those emotions are show. Further in Video 2, we can see how over 90 seconds, from 00:46.39 to 02:09.25 of his speech, his heart beat rate goes down from 89 beats per minute to 74 beats per minute.

Video 1 – a small part from the video with expressions of happiness

Video 2 – a small part from the video with heart beat analysis.

How FaceReader works

FaceReader is a tool to automatically analyzing facial expressions, providing users with an objective assessment of a person’s facial emotions [1], [2], [3], [4]. The image of the person is used for the analysis in a machine learning framework where a training based facial landmark tracking is performed for obtaining a 3D model of the face. The 3D model is further used to obtain the facial expressions in a supervised learning framework. The video below demonstrates how the system works. For more information please visit Noldus FaceReader website. Over 500 landmark points are tracked and analyzed for obtaining the facial expressions during the whole speech.

The FaceReader tool has been validated, showing that the accuracy for emotion recognition is as follows: 96 % – Happiness, 86% – Sadness, 76% – Anger, 94% – Surprise, 82 % – Fear, 92 % – Disgust (Lewinski et al, 2014)

FaceReader in U.S. and Turkish elections

FaceReader was also applied in analyzing political debates, in the last U.S. election, please see a video on CNN and with the latest Turkish presidential elections.

References:

Ekman, P., Friesen W.V., & Hager J. C. (2002). Facial action coding system: The manual. Salt Lake City, UT: Research Nexus.

Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86-88. doi:10.1126/science.164.3875.86

Russell, J. A. (1980). A circumplex model of affect. Journal of personality and social psychology, 39(6), 1161.

FaceReader. (2015). FaceReader: Tool for automatic analysis of facial expression: Version 6.1. Wageningen, the Netherlands: Noldus Information Technology B.V.

Lewinski, P., den Uyl, T. M., & Butler, C. (2014). Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader. Journal of Neuroscience, Psychology, and Economics, 7(4), 227.

[1] Kuilenburg, H. van, Wiering, M. and Uyl, M.J. den (2005). A Model Based Method for Automatic Facial Expression Recognition. In Proceedings of the 16th European Conference on Machine Learning (ECML-2005), October 3-7, Porto, Portugal.

[2] Marten J. den Uyl, Hans van Kuilenburg, (2005). The FaceReader: Online facial expression recognition. Proceedings of Measuring Behaviour 2005, 5th International Conference on Methods and Techniques in Behavioural Research, 589-590.

[3] H. Emrah Tasli, Amogh Gudi, and Marten den Uyl, “Remote PPG based vital sign measurement using adaptive facial regions,” in International Conference on Image Processing, 2014.

[4] H. Emrah Tasli, Paul Ivan; “Turkish Presidential Elections TRT Publicity Speech Facial Expression Analysis”; arXiv:1408.3573

Disclaimer:

All source videos and the data are publicly available for research purposes. If required, the analysis can be repeated and the results can be validated. All these results are obtained scientifically and they do not represent or reflect any personal view or political opinion. The purpose of this study is to analyses the facial expressions of the president during his speeches.

It is important to further emphasize that the analysis presented in this study shows how the president looks like. There is no intention to claim how the president would feel during the speech. This work is presented from a scientific point of view and hence, no personal option is reflected.

The original and publicly available video can be found here.

Advertisements
Standard

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s