Virtual reality technology developed by a University of Texas at Dallas professor is at the core of a new national initiative to enhance the teaching of children with dyslexia.
In collaboration with Scottish Rite for Children in Dallas, research professor Marjorie Zielke PhD’07 has developed an animated virtual therapist that helps children with reading, speech and language issues.
“There aren’t enough teachers who are dyslexia experts,” said Zielke, director of the Center for Simulation and Synthetic Humans at UT Dallas. “It has been a dream to create a virtual therapist that can address such an important national issue for children.”
The animated therapist, named Ms. Hallie, demonstrates through voice intonations as well as tongue and facial movements the proper ways to pronounce words. The lessons are based on a curriculum developed by Karen Avrit, director of dyslexia education at the Luke Waites Center for Dyslexia and Learning Disorders at Scottish Rite. Using specialized motion-capture recording technology, Zielke’s team recorded Avrit’s facial movements, dictation and vocal tones to create the virtual therapist.
“When you make a certain sound, the lips have to be precise, and the sound has to be precise,” Avrit said. “Dr. Zielke and her team were able to take our expectations around that and make them come to life. It was the most collaborative project I have ever worked on.”
Ms. Hallie is designed to augment instruction alongside a certified classroom teacher.
“It has been a dream to create a virtual therapist that can address such an important national issue for children.”
Marjorie Zielke PhD’07, director of the Center for Simulation and Synthetic Humans at UT Dallas
“This is an opportunity for virtual specialized instruction to really take a step forward in the education community to prove that it can be done with fidelity and accuracy, demonstrating that progress is being made by the students who need the most help,” Avrit said.
The program is being tested in four Dallas-Fort Worth-area school districts. In addition, Scottish Rite is preparing trainers throughout the country so they can share the program with educators in multiple regions.
Zielke said the goal of the research is not to replace teachers, but to complement their humanity while providing them with additional expertise.
“There are shortages of teachers and therapists, which has resulted in a lack of expertise in areas where students are particularly vulnerable and need help,” she said. “Being able to develop something that can help those children is so rewarding.”
Zielke’s work with Scottish Rite is an extension of her research using visualized artificial intelligence as a teaching tool.
To learn more about how UT Dallas is enhancing lives through transformative research, explore New Dimensions: The Campaign for UT Dallas.
Another project in Zielke’s lab involves medical students working in an augmented reality collaboration with a virtual patient named Walter who provides feedback and interacts with the students. This research, funded by a $750,000 grant from the National Science Foundation, looks at a number of virtual interactions — such as virtual peers, virtual professors, virtual patients and collaborative augmented reality paradigms — to determine the best methods for cyber-based social learning.
“We like to work in this space of virtual humans because human interactions are the interface that people are accustomed to,” Zielke said. “The gestures and facial expressions of the virtual patient create a natural kind of communication with students.”
Zielke and her team are planning next to enhance the natural language processing (NLP) of the virtual humans so they are more humanlike.
“The project data looks great. The model looks very realistic. We just need to get the NLP part working better. If we can do that, we will truly have a unique virtual human model,” she said. “Our latest results are suggesting to me that perhaps Walter works just as well as a human being. Sometimes human communication just needs to be more precise.”