Customer Service Software Senses Angry Callers

Imagine a tool that could sense your anger in a customer service call and then speed-dial a manager to help solve your problem. Such software, previously too convoluted to construct, is advancing steadily thanks to a partnership between UT Dallas and a Dallas-based call center support company called Working Solutions.

In this visual representation of recorded speech. Greater color variations suggest speech variability—an indication of anger.

The variability of human speech offers problems aplenty when using software to detect who’s angry and who isn’t, but compound that challenge with spotty cell phone quality, background noise, humming land lines or a crummy telephone receiver and the problems of detecting anger escalate significantly.

An algorithm is being developed to mathematically detect anger and overcome speech variability and telephone quality problems at the Center for Robust Speech Systems, headed by Dr. John Hansen, Distinguished Chair in Telecommunications Engineering and Department Head of electrical engineering at the Erik Jonsson School of Engineering and Computer Science.

Dr. Wooil Kim, a research assistant professor who specializes in speech recognition, is responsible for developing the algorithm.

“Emotion and stress classification in human speech is a hot issue in the research community,” Kim said. “This project is very challenging, with respect to voice and telephone line quality conditions.”

Vowels are the vocal cues that convey anger, according to Hansen. How heavily vowels are hammered away at helps determine a caller’s emotional status, but loud speech does not always translate into angry speech. Hansen says the software challenges are so complex that other researchers have given up trying to identify angry speech in open-ended communication scenarios. These complexities include background noise and the Lombard effect, or how people change their speech in noisy conditions. UT Dallas is the world leader on Lombard-based speech technology, with more published papers on the subject than any other speech laboratory.

“I worked early on identifying stress in the voice patterns of pilots flying missions over former Yugoslavia,” Hansen said. “Working Solutions—a firm that provides support for call center customers—asked us to utilize our expertise to refine this technology. We use a high-performance computing cluster with 128 networked CPUs that analyze voice data in a stream that is fed to us at UT Dallas over high-speed data lines.”

To ensure a high-quality experience for customers who call service centers, Working Solutions currently analyzes portions of recorded audio that call centers collect (i.e., “Your call may be monitored for quality assurance”). But listening to hours of audio can be cumbersome and time-consuming. Hansen’s approach is to use data mining instead. The company wants to develop a real-time software to rapidly re-direct calls from angry customers to managers who can solve problems.

“Turning a negative customer experience into a positive and memorable one can create a lasting customer relationship,” said George Platt, WS iNet executive vice president (WS iNet is an affiliate sister company of Working Solutions).

“In the contact center today, we only have the ability to identify angry customers days or weeks later, when it’s too late,” Platt said. “When we learned about what the Center for Robust Speech Systems was working on, we saw a great opportunity for collaboration. In our partnership with UT Dallas, we have been segmenting and supplying the customer call data needed to run the project, which we are confident will lead to one of the most innovative and valuable tools available to the contact center industry.”

Working Solutions’ CEO, Tim Houlne is a graduate of the UT Dallas Executive MBA program. Hansen said, “We at the Erik Jonsson School of Engineering and Computer Science really value these collaborations with local industry, given how well these projects can engage our students and their positive potential impact on the economy in the Dallas area.”

According to Hansen, any software that assists angry callers can improve customer satisfaction and, ultimately, improve the way a company is portrayed after a problem is resolved. He and Kim are evaluating the current algorithm and making necessary adjustments before commercializing the software.