New techniques have allowed three participants with tetraplegia to achieve peak brain-computer interface (BCI) performance within three minutes of engaging in an easy, one-step process, a new study shows.
For a BCI to be truly useful for a person with tetraplegia, it should be ready whenever it’s needed, with minimal expert intervention, including the very first time it’s used.
One participant in the study, “T5,” a 63-year-old man who had never used a BCI before, needed only 37 seconds of calibration time before he could control a computer cursor to reach targets on a screen, just by imagining using his hand to move a joystick.
While additional innovations will help to move implantable BCIs like BrainGate toward clinical availability for patients, this advance of rapid, intuitive calibration is a key one, says David Brandman, an engineering postdoctoral researcher at Brown University and lead author of the study, which appears in the Journal of Neural Engineering. It could allow future users and their caregivers to use the system much more quickly and to keep it calibrated over the long term.
“Up until now, getting going with a BCI has required oversight from a trained technician,” he says. “Our results suggest that once we begin to record from our users’ motor cortex, we can push the ‘calibrate’ button and allow the user to rapidly develop intuitive and high-quality cursor control.”
“Our new approach removes the need for technician oversight during calibration. This brings us one step closer to getting this system to people who we hope will benefit from it,” Brandman explains.
Train the computer, not the user
Senior author Leigh Hochberg, director of the BrainGate consortium and clinical trial, professor of engineering at Brown, critical care neurologist at Massachusetts General Hospital and director of the VA Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology at the Providence Veterans Affairs Medical Center, agrees that the new paper is a key advance.
“In the past few years, our team has demonstrated that people with tetraplegia can use the investigational BrainGate BCI to gain multidimensional control of a robotic arm, to point-and-click on a computer screen to type 39-plus correct characters per minute, and even to move their own arm and hand again—all simply by thinking about that movement,” Hochberg says.
“But each of these achievements required time—not to train the user, but to train the computer how to interpret the neural activity that reflects our users’ intuitively generated intended hand movements,” she says.
“Traditionally, this had required a many-minutes-long, exceedingly boring creation of a neural decoder. Here, the user is immediately engaged in building that decoder and can see cursor control evolve sometimes even in the first 30 seconds and continue to improve in just three minutes. Watching our participants do this has been really exciting,” Hochberg explains.
Without any compromise of cursor control performance, the new calibration process does away with several steps and about eight minutes of time compared to the traditional calibration method, the paper shows.
In that older approach, trial participants worked iteratively and explicitly through target acquisition attempts while a trained clinical research technician monitored and updated the software.
Rapid learning
The new decoder software uses statistical learning algorithms, Brandman says. These allow the BCI system to more quickly learn what the neural signals recorded by implants in the motor cortex of the brain are communicating about the users’ intention to move their arms and hands. This more rapid learning allowed the team to streamline calibration.
Essentially, all participants had to do was imagine moving a cursor from the center of a circle of eight targets to whichever target lit up a different color. Lingering over the target for a moment was enough to indicate selection. For about the first minute, study data show, cursor control was erratic as indicated by the wobbly paths from center to target. But by the second minute, the paths the participants traced had become much straighter.
Using one version of the decoder, for example, calibration performance saturated for T5 within about 43 seconds, for “T10,” a 34-year-old man, at around 100 seconds and for “T8,” a 55-year-old man, within 136 seconds. All three participants had sustained spinal cord injuries resulting in tetraplegia.
Each participant also performed another task in which they selected randomly appearing targets in an on-screen grid. These tests were designed to measure a “bit rate” of information akin to typing on a keyboard and to compare performance after using the traditional calibration method vs. using the new method. The results showed that participants achieved the same bit rate after the new one-step, three-minute method than they could achieve using the old multi-step, technician-supervised, 11-minute method.
“Score one for the guy in the wheelchair!”
For many other BCIs, the expectation is that the user and the research team will work together for days to weeks before the user can gain useful control of the BCI, Hochberg says. At least in theory, however, an intracortical BCI should work the very first time. With the new calibration method, the BrainGate team set out to test whether a brand-new BCI user—somebody who had never used either the BrainGate BCI or any other—might be able obtain cursor control on his first attempt.
“Of course, we only get one chance to see whether a BCI user can intuitively gain cursor control their very first time,” Hochberg says.
Brandman recalls that exact moment with T5.
“The day before his first attempt at using the intracortical BCI for controlling a computer cursor, I described to T5 that the system was going to be recording from a part of the brain that was responsible for coordinating hand and arm movement,” Brandman says. “I then asked him to suggest imagery that would be intuitive for him to use, and he suggested using a joystick.
“The first time he tried to use the system, it didn’t work. I was puzzled. But then T5 asked me, ‘When should I start?’ So I explained to him that he should start using the system with the joystick imagery he had suggested. He then rapidly gained control of the cursor and hit his first target in 37 seconds. Then he said, ‘Score one for the guy in the wheelchair!'”
Star Wars-inspired robotic hand lets pianist play again
“It was pretty amazing,” says Jaimie Henderson, principal investigator at the Stanford BrainGate site. “We asked T5 to try it out. And it just worked. He was moving the cursor around the screen as well as others who have been using the system for months or years.”
Mouse ball or joystick?
In addition to making the calibration process quicker and easier, researchers also wanted to learn more about which imagery users found most intuitive. That’s important to make the system as easy to use as possible for the user, while capitalizing on the richness and complexity of the neural signals recorded from the motor cortex, Brandman says.
So in another series of experiments, T5 performed the calibration task with each of five additional modes of attempted motion after starting out with the joystick imagery that he chose. These covered various scenarios in which some parts of the arm and hand move freely but others remain fixed. For example, in “mouse ball,” T5 imagined moving his wrist and elbow as if moving the cursor with a trackball mouse, while in “whole arm” he imagined moving his arm around in free space so that his fixed index finger could point at the on-screen targets.
Performance with each mode varied slightly, but in every case T5 achieved near-peak control within 60 seconds. In the end, after seeing his results, he decided that “mouse ball” was his new favorite.
“One of the beautiful things about using intracortical recording is that we were able to allow T5 to use multiple imageries that were intuitive for him and to allow him to see the benefits of each imagery in real-time,” says Brandman, who adds that he was also encouraged that the calibration system was robust enough to accommodate all the movement imageries that different participants preferred.
Implant lets paralyzed people type with their minds
Continuing to develop the most easily usable, reliable, and flexible system possible for people with tetraplegia is a main goal of the feasibility trial, Hochberg says.
The BrainGate collaboration includes other researchers from Brown, Stanford, and Case Western University.
The work was supported by the National Institutes of Health: National Institute on Deafness and Other Communication Disorders; the Eunice Kennedy Shriver National Institute of Child Health and Human Development / National Center for Medical Rehabilitation Research; Rehabilitation Research and Development Service, Department of Veterans Affairs; National Science Foundation; MGH—Deane Institute; Joseph Martin Prize for Basic Research / the Executive Committee on Research of MGH; Stanford BioX-NeuroVentures; Stanford Office of Postdoctoral Affairs; ALSA Milton Safenowitz Postdoctoral Fellowship; Howard Hughes Medical Institute at Stanford University; Garlick Foundation; Katie Samson Foundation; Craig H. Neilsen Foundation; Canadian Institute of Health Research (336092); Killam Trust Award Foundation; and the Brown Institute for Brain Science.
Source: Brown University
