Connecting computing with human emotions for empathetic AI

Imagine a world where your smartphone can detect your mood just by the way you type a message or the tone of your voice. Picture a car that adjusts its music playlist based on your stress levels during rush hour traffic. These scenarios are not just futuristic fantasies.

They are glimpses into the rapidly evolving field of affective computing. Affective computing is a multidisciplinary field integrating computer science, engineering, psychology, neuroscience, and other related disciplines. A new and comprehensive review on affective computing was published in Intelligent Computing. It outlines recent advancements, challenges, and future trends.

Affective computing enables machines to perceive, recognize, understand, and respond to human emotions. It has various applications across different sectors, such as education, health care, business services, and the integration of science and art. Emotional intelligence plays a significant role in human-machine interactions, and affective computing has the potential to enhance these interactions significantly.

According to the review, research in this field covers five main aspects: basic theory of emotion, collection of emotional signals, sentiment analysis, multimodal fusion, and generation and expression of emotions.

Researchers performed a statistical analysis using a bibliometric method to improve the overall understanding of the theory, technical methods, and applications of affective computing. Bibliometrics applies quantitative methods such as mathematics and statistics to the literature of a scientific or other field and processes statistical data based on information science theory.

According to the data collected in the bibliometric study, the number of articles published in the field of affective computing has grown considerably since 1997, with a steady increase in publications until 2009, followed by rapid growth from 2010 to 2019 due to advancements in deep learning.

However, post-2019, the growth has plateaued, possibly due to a slowdown in deep learning innovation and the impact of the COVID-19 pandemic on academia. China is leading the world in publication volume, followed by the United States, India, the UK, and Germany. The field’s research is published across a wide range of journals, with journals such as IEEE Transactions on Affective Computing, Expert Systems with Applications and Knowledge-Based Systems being favored by scholars in the field of affective computing.

Affective computing research spans disciplines, including computer science, engineering, neurosciences, and other disciplines. The main research themes in this field, as identified through keyword frequency analysis, are natural language processing, facial expression recognition, human-computer interaction, affective disorder analysis, and multimodal emotion analysis.

Top institutions in this field include Nanyang Technological University, Imperial College London, and Tsinghua University. International collaborations are widespread, particularly between China and the US. The field’s scholars are primarily concentrated in Asia and North America.

Affective computing, a field that combines technology with the nuanced understanding of human emotions, is experiencing surges in innovation and related ethical considerations. Innovations identified in the review include emotion-generation techniques that enhance the naturalness of human-computer interactions by increasing the realism of the facial expressions and body movements of avatars and robots.

The development of fine-grained sentiment classification models is also advancing, allowing for a better understanding of the mixed emotions that occur in everyday life. Cognitive neuroscience provides valuable insight into the neural basis of emotions, which is crucial for designing systems that can authentically simulate emotional responses.

Meanwhile, the field is reportedly expanding its focus from individual to group-level emotions, recognizing the importance of collective affect in consumer behavior and settings such as workplaces.

In addition, researchers are addressing the challenge of cultural diversity by developing models that can adapt to various emotional expressions across different cultures. Such research involves addressing ethical and privacy concerns and advocating the establishment of international standards to govern the use of emotional data.

The researchers identified several current trends that are likely to continue. For example, the construction of large-scale, multimodal datasets is essential for training robust affective computing systems, and there is a growing trend towards collecting and analyzing data that reflects natural emotional expressions.

Multimodal fusion technology is being refined to effectively integrate information from various sources, such as facial expressions, voice and text, to improve the accuracy of emotion recognition. Additionally, the field is moving towards a knowledge-driven approach, integrating expert knowledge to compensate for data limitations and enhance computational efficiency.

Affective computing is being applied to political speeches, music, drama, and visual arts to predict emotional reactions and enhance emotional expression. This interdisciplinary approach is not only advancing the capabilities of affective computing but also opening up new avenues for research and application.

The future applications of affective computing across various domains show great promise, especially with advancements in the affective brain–computer interfaces, empathic human–computer dialogue, emotion-assisted decision-making, and affective virtual reality. Affective brain–computer interfaces, which utilize neurological signals, are instrumental in measuring and regulating emotional states, benefiting clinical diagnostics, therapy, and military applications.

Empathic human–computer dialogue seeks to create more human-like interactions by leveraging multimodal emotional cues, offering potential for elderly companionship and intelligent customer service. Emotion-assisted decision-making incorporates emotional variables into machine decision-making processes, potentially optimizing outcomes in fields like production safety and assisted driving.

Affective virtual reality plays a crucial role in constructing immersive metaverses, employing emotionally rich avatars to engage users, thus facilitating applications in virtual socialization, anchors and marketing.