Novel analytics framework measures empathy of people captured in video recordings

Empathy, the ability to understand what others are feeling and emotionally connect with their experiences, can be highly advantageous for humans, as it allows them to strengthen relationships and thrive in some professional settings. The development of tools for reliably measuring people’s empathy has thus been a key objective of many past psychology studies.

Most existing methods for measuring empathy rely on self-reports and questionnaires, such as the interpersonal reactivity index (IRI), the Empathy Quotient (EQ) test and the Toronto Empathy Questionnaire (TEQ). Over the past few years, however, some scientists have been trying to develop alternative techniques for measuring empathy, some of which rely on machine learning algorithms or other computational models.

Researchers at Hong Kong Polytechnic University have recently introduced a new machine learning-based video analytics framework that could be used to predict the empathy of people captured in video footage. Their framework, introduced in a preprint paper published in SSRN, could prove to be a valuable tool for conducting organizational psychology research, as well as other empathy-related studies.

“Our research is sparked by the recent AI revolution, which, as Huang and Rust (2021) describe, ushers in a new ‘feeling economy’ where machines handle analytical tasks and humans increasingly shoulder emotional support and interpersonal relationships,” Li Cui, one of the authors of the paper, told Tech Xplore.

“In this context, we recognize that empathy in the workplace matters more than ever, yet traditional leadership studies focus overwhelmingly on negative traits such as overconfidence or narcissism, effectively examining only one side of the coin. We see an opportunity: AI-powered video analytics now enable direct, scalable extraction of human characteristics—supplanting the indirect, biased measures of the past.”

The main objective of the recent study by Cui, Ka Chung, Lu and Zhao was to develop a realizable video analytics framework that could be used to predict the empathy of people from video recordings in which they are engaged in conversations with others. They specifically used their framework to analyze video footage of real interviews between CEOs and TV journalists.

The results of the team’s analyses allowed them to explore how empathy shapes corporate policies and affects the value of a firm or business. Notably, the analytics framework they developed is rooted in neuroscience research and theories, which suggests that people with higher levels of empathy tend to mimic the emotions expressed by others more than those who are less empathetic.

“We developed a machine learning framework that infers emotion mimicry from CEO interview QA pairs and uses it as a proxy for empathy,” explained Cui. “The framework enables a convergent measure of empathy, which we validate using multimodal signals in real-world settings. By capturing fine-grained behavioral cues, it offers an efficient, scalable, and generalizable approach to automatic empathy assessment.”

As part of their recent study, the researchers used the machine learning-based approach they developed to study the empathy and behaviors of CEOs during TV interviews, where they were asked to answer questions about their companies and their success. They found that their framework showed potential as a valid and efficient tool for approximating CEO empathy.

“While empathy has traditionally been difficult to measure at scale, our framework captures fine-grained behavioral cues—such as emotional mimicry—and validates them through real-world settings,” said Cui. “Using this measure, we find that more empathetic CEOs are associated with more prosocial corporate policies—for example, lower workplace injury rates and more equitable compensation policies.

“Moreover, CEO empathy appears to contribute to more effective crisis management, which in turn may help enhance firm value. While preliminary, these findings point to the potential organizational importance of empathy as a leadership trait. We see this as a step toward expanding how soft skills can be studied in real-world settings.”

The researchers hope that the framework they developed will further advance available FinTech and psychology research tools designed to study people’s traits, communication styles and behavior, both in organizational and everyday settings. In the future, it could also be used to further broaden the capabilities AI analytics platforms, allowing them to predict people’s empathy based on their body language.

“Our current study is an early step in exploring how video analytics can help quantify soft traits like empathy, and there’s certainly room to build further,” said Cui. “One direction we hope to pursue is applying the framework to a broader set of video contexts—such as social media clips, informal executive communications, or cross-cultural settings—to improve generalizability and understand how empathy manifests in different environments.”

As part of their future studies, the researchers also plan to refine the framework they created, with the aim of reducing any potential biases in its interpretation of nonverbal cues. Finally, they could also try to devise similar frameworks that can assess other human traits, abilities or characteristics, such as assertiveness, emotional stability and trustworthiness.

© 2025 Science X Network