Chapter 12. Emotion

Facial Expression and Recognizing Emotions

Jessica Motherwell McFarlane

Approximate reading time: 16 minutes

Culture influences how people exhibit emotions. A cultural display rule is a standard within a set of culturally specific norms that dictate the acceptable types and frequencies of emotional expression (Malatesta & Haviland, 1982). Consequently, individuals from different cultural backgrounds may adhere to distinct cultural display rules concerning emotions. For instance, studies indicate that people in the United States openly express negative emotions such as fear, anger, and disgust, both when alone and in company, whereas, on average, Japanese individuals tend to do so only in private (Matsumoto, 1990). Moreover, those from cultures that prioritize social harmony are more inclined to suppress emotional responses to assess the most fitting reaction for the situation (Matsumoto, Yoo, & Nakagawa, 2008).

Paul Ekman researched facial expressions for more than four decades. Despite different emotional display rules, our ability to recognise and produce facial expressions of emotion appears to be universal (Martinez, 2017; Sato, et al., 2019). In fact, even people who are blind from birth make the same facial expression of emotions, despite their never having the opportunity to observe these facial displays of emotion in other people. This would seem to suggest that the pattern of activity in facial muscles involved in generating emotional expressions is universal. In fact, there is substantial evidence for seven universal emotions that are each associated with distinct facial expressions. These include: happiness, surprise, sadness, fright (fear), disgust, contempt, and anger (Figure EM.24) (Ekman & Keltner, 1997).

 

 

 

Each of seven photographs includes a person demonstrating a different facial expression: happiness, surprise, sadness, fright, disgust, contempt, and anger.
Figure EM.11. Seven universal facial expressions. The seven universal facial expressions of emotion are shown.

There is evidence to suggest that the universality of recognition of emotional facial expressions can be modified by cultural and contextual variables. Researchers have found that cultural differences can influence the perception and intensity ratings of emotions (Engelmann & Pogosyan, 2013; Fang, Sauter, & van Kleef, 2017; Hess, Blaison, & Kafetsios, 2016). Furthermore, while some research supports the universality of emotional expression recognition, others highlight the importance of cultural specificity, suggesting a complex interplay between universal human attributes and culturally specific experiences (Wolfgang & Cohen, 1988; Matsumoto et al., 2009). This understanding emphasises the importance of considering cultural contexts in psychological research on emotion, as facial expressions are not interpreted in isolation but through a lens shaped by cultural norms and individual experiences (Gendron et al., 2020).

Do Face Masks Affect Recognition of Emotions?

Thomas and Caharel (2023) discovered that face masks significantly affect the recognition of expressions of disgust, happiness, and sadness. This impact leads to reduced accuracy and a higher rate of misinterpretations for both subtle and pronounced expressions. Conversely, the recognition of anger, fear, and neutral expressions appears to be less affected by the wearing of masks.

During the COVID-19 pandemic, which began in late 2019 and affected the world significantly in the following years, wearing face masks became a common practice globally. Governments and health organisations recommended or mandated the use of masks in public spaces to reduce the spread of the virus. This led to a widespread and prolonged period where the general population wore masks whenever they were outside their homes, especially in crowded places, public transport, and indoor settings such as stores and offices. The practice of mask-wearing varied by region and over time, depending on the severity of outbreaks and local public health guidelines. However, for much of 2020 and 2021, and into 2022 in some areas, mask-wearing was a part of daily life for billions of people worldwide. This context is important for understanding the research by Thomas and Caharel (2023), as the widespread use of masks during this period had a significant impact on social interactions, including the recognition of emotional expressions.
A person is wearing a medical mask.
Figure EM.12. Masks and emotions. Thomas and Caharel (2023) discovered that face masks significantly affect the recognition of expressions of disgust, happiness, and sadness.

Can Facial Expressions Change the Way We Feel?

Does smiling make you happy or does being happy make you smile? The facial feedback hypothesis proposes that your facial expression can actually affect your emotional experience (Adelman & Zajonc, 1989; Boiger & Mesquita, 2012; Buck, 1980; Capella, 1993; Soussignan, 2001; Strack, Martin, & Stepper, 1988).

  • Research investigating the facial feedback hypothesis suggested that suppression of facial expression of emotion lowered the intensity of some emotions experienced by participants (Davis, Senghas, & Ochsner, 2009).
  • Botox injections to paralyze facial muscles and limit facial expressions, including frowning, and they found that depressed people reported less depression after their frowning muscles were paralyzed (Havas, Glenberg, Gutowski, Lucarelli, and Davidson, 2010).
  • Other research found that the intensities of facial expressions affected the emotional reactions (Soussignan, 2002; Strack, Martin, & Stepper, 1988). In other words, if something insignificant occurs and you smile as if you just won the lottery, you will actually be happier about the little thing than you would be if you only had a tiny smile. Conversely, if you walk around frowning all the time, it might cause you to have less positive emotions than you would if you had smiled.

Can Body Language Change the Way We Feel?

Emotional expression extends beyond transient facial expressions to encompass a range of cues, including vocal tone and the subtleties of our posture. While facial expressions are often the most noticeable aspect of emotional communication, body language — encompassing our bodily positions and movements — plays a crucial role in the complex interplay of emotional expression. The stance of a person can reveal nervousness despite a smile, or the nature of a hug can communicate genuine joy or offer consolation. This is the unique language of the body in action.

Researchers have found that we are quite attuned to the emotional information conveyed by body language, even when it is not consciously perceived (de Gelder, 2006; Tamietto et al., 2009). In general, we are adept at interpreting emotional information from body language, frequently on a subconscious level (de Gelder, 2006; Tamietto et al., 2009). For example, our impression of someone’s emotional state can be significantly shaped by their body language, sometimes even more than by their facial expressions (Meeren et al., 2005). Imagine someone sharing a joyful anecdote, yet their posture is hunched and their movements are lethargic. Such incongruence (mismatch) can profoundly shift the emotional message we perceive.

Autism Spectrum Disorder and Expression of Emotions

Autism Spectrum Disorder (ASD) encompasses a range of neurodevelopmental disorders marked by repetitive behaviours and challenges in communication and social interaction. Recent research reveals that individuals with ASD often experience difficulty in recognising the emotional states of others, potentially due to an impaired ability to differentiate between nonverbal expressions of emotion, such as facial expressions (Whitaker et al., 2017). Moreover, there is a consensus that Autistic individuals with autism spectrum disorder may also struggle with expressing emotions, both through tone of voice and facial expressions (Rosenblau et al., 2016). These challenges with emotional recognition and expression are likely to contribute to the core social interaction and communication difficulties associated with ASD. To mitigate or overcome these challenges, a variety of therapeutic strategies, including educational programs, cognitive-behavioural therapies, and pharmacological interventions, have been researched and show promise in assisting individuals with ASD to process emotional information more effectively (Kerr-Gaffney et al., 2020; Santiesteban et al., 2020).

In conclusion, the emotional expression and recognition is woven with threads of cultural norms, biological underpinnings, and individual differences. Ekman’s research supports the concept of universal facial expressions, suggesting that certain emotions are expressed similarly across diverse cultures, reflecting a common aspect of human emotional experience. However, cultural display rules and recent findings on the effects of face masks illustrate that the expression and recognition of emotions can be significantly influenced by cultural norms and situational factors. The facial feedback hypothesis further reveals the bidirectional relationship between our expressions and emotions, suggesting that our faces do not merely reflect our internal states but can also shape them.

This interaction between internal experience and external expression is particularly noticeable in the context of Autism Spectrum Disorder (ASD), where challenges in emotional recognition and expression can profoundly affect social interactions. In simple terms, people with ASD often find it hard to understand and show emotions. This can make social situations challenging for them. Researchers are working on ways to help individuals with ASD better express and experience emotions. This work is crucial because it helps us learn more about emotions in general. As we get better at this, we can create a more understanding and supportive world for everyone, especially those with ASD.

Emojis, Emoticons, and Emotions

The birth of digital communication has brought about new ways in which individuals express emotions, with emojis emerging as a significant element in this landscape. Emojis are used as a digital extension of facial expressions, capturing emotions in a form that transcends the barriers of textual communication (Erle et al., 2021). While they may not convey the full range of social cues inherent in face-to-face interactions, emojis do produce neural responses similar to those observed with real facial expressions, suggesting a functional equivalence in digital contexts (Gantiva et al., 2020). Emojis can affect the interpretation of text-based messages, adding layers of emotional depth and clarity that might otherwise be absent in plain text (Holtgraves & Robinson, 2020).

Emoticons (like 🙂 or 🙁 ) and emojis (like 😊 or 😢) are more than playful extras in our digital messages; they reflect our innate human need to express and interpret emotions, a need that grows as we rely more and more on digital communication. Researchers have found that emojis can increase our understanding of indirect replies, especially when conveying negative information, by providing emotional cues that text alone may not fully communicate (Holtgraves & Robinson, 2020). For instance, a message that says “Sure, we can go with your plan 😅” uses the sweat smile emoji to add a layer of meaning that suggests the sender is agreeing but may have hesitations or is feeling awkward about the situation.

The integration of emojis into digital communication has been found to induce affective states in recipients, influencing both the emotional tone and the perceived intent of the message (Gantiva et al., 2021). This suggests that emojis do more than simply stand in for facial expressions; they actively work to maintain and enhance social relationships by performing emotion work within text-based communication (Riordan, 2017). Thompson et al. (2016) studied the effects of emoticons and found they evoke real emotions, influencing arousal and facial expressions. Their participants showed higher arousal, reduced frowning, and enhanced smiling for messages with emoticons rather than without. This intricate relationship between facial expressions, emojis, and emotions underscores the idea that our faces, and face-like emojis not only display our feelings but can also shape them (Namba et al., 2017; Recio, Schacht, & Sommer, 2014; Israelashvili, Hassin, & Aviezer, 2019).

The science behind emotions and emojis reveals a complex interplay where these digital symbols serve as a bridge between what we want to communicate and the maintenance of social relationships, effectively conveying a range of emotions and intentions that enrich our online interactions.

Summary: Facial Expressions, Body Language, ASD, and Emojis

In this section, we delve into the fascinating world of facial expressions, body language, and their profound impact on emotion recognition and expression. We start by exploring how facial expressions serve as universal signals of emotion, bridging communication gaps across different cultures. The discussion then shifts to the intriguing question of whether face masks, a common sight in recent times, hinder our ability to recognise emotions in others. This leads to an examination of the feedback theory, which suggests that our own facial expressions can influence our emotional state, highlighting the interplay between expression and feeling.

The conversation extends to body language, another pivotal aspect of non-verbal communication, exploring how our posture and movements can reflect and even alter our emotional landscape. This section also addresses the unique challenges faced by individuals with Autism Spectrum Disorder (ASD) in expressing and interpreting emotions, underscoring the importance of empathy and understanding in our interactions.

Lastly, we touch upon the modern phenomenon of emojis and emoticons, digital age tools that enrich our text-based communications by conveying emotions, intentions, and tone. This part of the discussion illuminates how these symbols have become integral to our online interactions, offering a glimpse into the evolving landscape of emotional expression.

Image Attributions

Figure EM.11. Figure EM.11 contains: Happiness Image by Hoàng Đông Trịnh Lê from Pixabay and Surprise Image by abkamta from Pixabay are used under the Pixabay Content License. Sadness Photo by Alena Darmel, Fright Photo by MART  PRODUCTION, Disgust Photo by Polina Zimmerman, Contempt Photo by Andrea Piacquadio and Anger Photo by MART  PRODUCTION are licensed under the Pexels license.

Figure EM.12. Masks and emotions by Rachel Lu is licensed under a CC BY-NC-SA license.

definition

License

Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Facial Expression and Recognizing Emotions Copyright © 2024 by Jessica Motherwell McFarlane is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

Share This Book