Experiments involving Dr. Seuss clarify how the brain is engaged during complex audiovisual speech perception.
The study in NeuroImage describes how listening and watching a narrator tell a story activates an extensive network of brain regions involved in sensory processing, multisensory integration, and cognitive functions associated with the comprehension of the story content. Understanding the involvement of this larger network has the potential to give researchers new ways to investigate neurodevelopmental disorders.
“Multisensory integration is an important function of our nervous system as it can substantially enhance our ability to detect and identify objects in our environment,” says Lars Ross, research assistant professor of imaging sciences and neuroscience at the University of Rochester Medical Center’s Del Monte Institute for Neuroscience.
“A failure of this function may lead to a sensory environment that is perceived as overwhelming and can cause a person to have difficulty adapting to their surroundings, a problem we believe underlies symptoms of some neurodevelopmental disorders such as autism.”
Using fMRI, researchers examined the brain activity of 53 participants as they watched a video recording of a speaker reading The Lorax. The story’s presentation would change randomly in one of four ways—audio only, visual only, synchronized audiovisual, or unsynchronized audiovisual. Researchers also monitored the participants’ eye movements. They found that along with the previously identified sites of multisensory integration, viewing the speaker’s facial movements also enhanced brain activity in the broader semantic network and extralinguistic regions not usually associated with multisensory integration, such as the amygdala and primary visual cortex. Researchers also found activity in thalamic brain regions which are known to be very early stages at which sensory information from our eyes and ears interact.
“This suggests many regions beyond multisensory integration play a role in how the brain processes complex multisensory speech—including those associated with extralinguistic perceptual and cognitive processing,” says Ross, who is first author of the study.
The researchers designed this experiment with children in mind. The investigators have already begun working with both children and adults on the autism spectrum in an effort to gain insight into how their ability to process audiovisual speech develops over time.
“Our lab is profoundly interested in this network because it goes awry in a number of neurodevelopmental disorders,” says John Foxe, lead author of this study. “Now that we have designed this detailed map of the multisensory speech integration network, we can ask much more pointed questions about multisensory speech in neurodevelopmental disorders, like autism and dyslexia, and get at the specific brain circuits that might be impacted.”
Additional coauthors are from the Albert Einstein College of Medicine and Technological University Dublin. This research was a collaboration of two Intellectual and Developmental Disability Research Centers (IDDRC), which have support from the National Institute of Child Health and Human Development (NICHD).
Source: University of Rochester
