Researcher Reassesses Assessments for Social Cognition Deficits

If the psychological tests used to measure social cognition deficits are so outdated that they may not yield accurate results, are those results valid?

Dr. Amy Pinkham, professor of psychology in The University of Texas at Dallas’ School of Behavioral and Brain Sciences, is co-principal investigator on a five-year grant from the National Institutes of Health for up to $3.6 million aimed at refining a battery of tasks that measures social cognitive ability and that could help predict whether teens and young adults who are at high risk of developing psychosis actually go on to develop a mental illness.

Social cognition refers to an individual’s process of responding to interactions with empathy, interpreting facial expressions and nonverbal communication, and perceiving subtexts of words, such as sarcasm and lying.

Pinkham spearheads the project, which will identify the best tasks for assessing younger people, who often find the details of antiquated tasks hard to interpret.

“If we’re going to measure something, we have to know that our measure is good. While some effective tools exist to identify people showing early symptoms of potential psychosis, the social cognition tasks used are far from ideal.”

Dr. Amy Pinkham, professor of psychology in the School of Behavioral and Brain Sciences

“It’s such an important, foundational idea: If we’re going to measure something, we have to know that our measure is good,” she said. “While some effective tools exist to identify people showing early symptoms of potential psychosis, the social cognition tasks used are far from ideal.”

Mid-to-late adolescence or early adulthood is often when disorders involving psychosis become evident. The onset of schizophrenia, for instance, varies significantly, from late adolescence up through the mid-30s, so the assessments should match that target population.

“People have been studying social cognition in this clinical high-risk group for 15 years, but we never stopped to ask whether the data is valid and reliable,” Pinkham said. “That’s why we must evaluate the measures themselves, and in schizophrenia and social cognition, we now realize that nearly every task has flaws.”

Pinkham said the assessments are often poorly applicable to a younger generation. Many were developed in the 1980s and ’90s, so stimuli are dated.

“For instance, one situation we talk about is leaving a message on somebody’s answering machine — a situation young people today don’t really encounter,” she said. “Or they mention brand names that are no longer prominent — asking a teenager today about an encounter at Blockbuster Video, RadioShack or Toys R Us doesn’t make sense.”

Pinkham said several factors make it challenging to create social cognition tasks that are universal and that will stand the test of time. For example, visual stimuli included in the tasks do not always reflect appearances, such as culturally unique clothing styles from different parts of the world, where expectations for interactions between people also may be markedly disparate.

“Even with basic, fundamental questions, people’s physical appearances and styles in society are constantly evolving, and it matters what these images communicate,” Pinkham said. “Tasks may require continual development and updating of stimuli; we don’t want to use pictures that look like they came from the ’70s.”

The ways individuals, especially young people, routinely interact in today’s technological society also need to be considered in psychological assessments, Pinkham said.

“If a test used to ask how often you talk with somebody in person, those values are all very different in the modern era of social media and texting, and especially in the wake of the COVID pandemic,” she said. “The landscape for the way that people interact is different.”

Part of the study will evaluate how a newly selected battery of tests compares to previous tests in terms of pinpointing clinical high risk of psychosis. Pinkham hopes to show which tasks are as refined as possible — instilling confidence in previous research — while also guiding potential changes moving forward.

Joining Pinkham on the project is her frequent collaborator, co-principal investigator Dr. Vijay Mittal of Northwestern University.

“The more difficulty someone has with social information processing, the more they tend to struggle in day-to-day social functioning, maintaining relationships, gaining employment and so on,” Pinkham said. “The link between social cognitive difficulty and psychosis, however, is not specific; we see social cognitive difficulties broadly across a number of disorders.

“We hope to establish here that social cognitive impairments predict conversion to psychosis and that, with better measurement, we can get a good idea of the scale of risk for these individuals, with hopes of finding methods for prevention.”