Recognizing emotions using brain activity and facial expression analysis

Recently I wrote my master’s thesis on automatic recognition of emotions (using brain activity and face expression analysis) at Tilburg University.

Modality face off: detecting affect by analyzing electroencephalography and facial expressions

In this thesis, emotion recognition performance through encephalogram (EEG) analysis is compared to the more established performance of facial expression analysis. An experiment was conducted to assess recognition performance of five discrete affective states (fear, amusement, disgust, sadness, and neutral). Participants were shown emotion-eliciting movie fragments while their EEG and facial expressions were recorded.

The EEG modality was able to detect the emotions only to a limited extent and significantly underperformed compared to the face modality. The combination of modalities, however, significantly outperformed both the individual modalities.

These findings (1) contribute to our knowledge of the added value of EEG to facial expression recognition and (2) encourage future research to take a multimodal approach.

Here’s a link to the thesis: “Modality face off: detecting affect by analyzing electroencephalography and facial expressions

  • Master’s thesis
  • Communication and Information Sciences
  • Specialization Human Aspects of Information Technology
  • Faculty of Humanities
  • Tilburg University, Tilburg