Scientists can now pinpoint where someone’s eyes are looking just by listening to their ears.
“You can actually estimate the movement of the eyes, the position of the target that the eyes are going to look at, just from recordings made with a microphone in the ear canal,” says senior study author Jennifer Groh, a professor in the departments of psychology and neuroscience and neurobiology at Duke University.
In 2018, Groh’s team discovered that the ears make a subtle, imperceptible noise when the eyes move. In the new study, published in the Proceedings of the National Academy of Sciences, the team now shows that these sounds can reveal where your eyes are looking.
It also works the other way around. Just by knowing where someone is looking, Groh and her team were able to predict what the waveform of the subtle ear sound would look like.
These sounds, Groh believes, may be caused when eye movements stimulate the brain to contract either middle ear muscles, which typically help dampen loud sounds, or the hair cells that help amplify quiet sounds.
The exact purpose of these ear squeaks is unclear, but Groh’s initial hunch is that it might help sharpen people’s perception.
“We think this is part of a system for allowing the brain to match up where sights and sounds are located, even though our eyes can move when our head and ears do not,” Groh says.
Understanding the relationship between subtle ear sounds and vision might lead to the development of new clinical tests for hearing.
“If each part of the ear contributes individual rules for the eardrum signal, then they could be used as a type of clinical tool to assess which part of the anatomy in the ear is malfunctioning,” says lead author Stephanie Lovich, a graduate student in psychology and neuroscience.
Just as the eye’s pupils constrict or dilate like a camera’s aperture to adjust how much light gets in, the ears too have their own way to regulate hearing. Scientists long thought that these sound-regulating mechanisms only helped to amplify soft sounds or dampen loud ones.
But in 2018, Groh and her team discovered that these same sound-regulating mechanisms were also activated by eye movements, suggesting that the brain informs the ears about the eye’s movements.
In the latest study, the researchers followed up on their initial discovery and investigated whether the faint auditory signals contained detailed information about the eye movements.
To decode people’s ear sounds, Groh’s team and Christopher Shera, a professor at the University of Southern California, recruited 16 adults with unimpaired vision and hearing to take a fairly simple eye test.
Participants looked at a static green dot on a computer screen, then, without moving their heads, tracked the dot with their eyes as it disappeared and then reappeared either up, down, left, right, or diagonal from the starting point. This gave Groh’s team a wide-range of auditory signals generated as the eyes moved horizontally, vertically, or diagonally.
An eye tracker recorded where participant’s pupils were darting to compare against the ear sounds, which were captured using a microphone-embedded pair of earbuds.
The researchers analyzed the ear sounds and found unique signatures for different directions of movement. This enabled them to crack the ear sound’s code and calculate where people were looking just by scrutinizing a soundwave.
“Since a diagonal eye movement is just a horizontal component and vertical component, my labmate and coauthor David Murphy realized you can take those two components and guess what they would be if you put them together,” Lovich says.
“Then you can go in the opposite direction and look at an oscillation to predict that someone was looking 30 degrees to the left.”
Groh is now starting to examine whether these ear sounds play a role in perception.
One set of projects is focused on how eye-movement ear sounds may be different in people with hearing or vision loss.
Groh is also testing whether people who don’t have hearing or vision loss will generate ear signals that can predict how well they do on a sound localization task, like spotting where an ambulance is while driving, which relies on mapping auditory information onto a visual scene.
“Some folks have a really reproducible signal day-to-day, and you can measure it quickly,” Groh says. “You might expect those folks to be really good at a visual-auditory task compared to other folks, where it’s more variable.”
Source: Duke University
