Casey Harrell trying the BCI system for the first time (Credit: UC Regents)
Brain-computer interface yields astonishing accuracy for non-verbal individuals — using their actual voice
DAVS, Calif. — Technology is making it possible for nonverbal people to speak again. Researchers from California have developed a computer program that translates brain signals into speech. The brain-computer interface is 97% accurate, the highest accuracy achieved so far.
The goal of the technology is to help people who are nonverbal because of paralysis or have lost their ability to speak because of neurological conditions communicate with others. Think of a translator for the non-verbal brain. The brain-computer interface listens to the brain activity when a person intends to speak and converts those signals into written text. The computer then speaks the text aloud.
A study on the accuracy of the brain-computer interface program was recently published in the New England Journal of Medicine. In it, neuroscientists successfully helped a man with amyotrophic lateral sclerosis (ALS) speak again. ALS, also known as Lou Gehrig’s disease, is a condition that causes damage and loss of function in nerve cells involved in movement. Over time, people lose the ability to stand, walk, and use their hands. Difficulty speaking is another common symptom, as people lose control of muscles involved in mouth movement.
In the current study, the brain-computer interface helped a man with ALS communicate within minutes of activating the system. Casey Harrell is a 45-year-old man with ALS enrolled in the BrainGate clinical trial. Before the study, Harrell showed weakness in arms and legs, severely impaired speech, and needed people to communicate on his behalf.
“Our BCI technology helped a man with paralysis to communicate with friends, families and caregivers,” said David Brandman, a UC Davis neurosurgeon and co-senior study author, in a press release. “Our paper demonstrates the most accurate speech neuroprosthesis (device) ever reported.”
Methodology
In July 2023, Harrell was implanted with the brain-computer device on the left precentral gyrus, a brain area involved in coordinating speech. The implant included four microelectrode arrays used to record brain activity from 256 cortical electrodes.
“We’re really detecting their attempt to move their muscles and talk,” explained Sergey Stavisky, an assistant professor in the department of neurological surgery at UC Davis and co-principal investigator of the study. “We are recording from the part of the brain that’s trying to send these commands to the muscles. And we are basically listening into that, and we’re translating those patterns of brain activity into a phoneme— like a syllable or the unit of speech—and then the words they’re trying to say.”
Key Results
In the first speech data training session, the device converted Harrell’s brain signals to 50 words in 30 minutes. The system was 99.6% accurate. “The first time we tried the system, he cried with joy as the words he was trying to say correctly appeared on-screen. We all did,” recalled Stavisky.
The second session increased the computer’s vocabulary from 50 to 125,000 words. After 1.4 hours of training, the device was 90.2% accurate. Further training raised accuracy rates to 97.5%. In other words, Harrell could be understood 97% of the time.
In total, there were 84 data collection sessions over 32 weeks. Harrell used the speech for training and in spontaneous conversations. At his own pace, he successfully talked for over 248 hours in person and over video calls.
Discussion
Most machine learning programs in the past required large amounts of brain activity data, which required a lot of time and work. Even then, communication was riddled with word errors, making it harder to understand what a person was saying. This new brain-computer interface avoids this by translating signals to speech in real time, with constant system updates to keep the program working accurately. The computer voice was formatted to sound like Harrell’s pre-ALS voice from old audio samples to make the device more personalized.
“Not being able to communicate is so frustrating and demoralizing. It is like you are trapped,” said Harrell. “Something like this technology will help people back into life and society.”
According to the authors, this neurological device is more accurate than other technology and apps trying to interpret someone’s words. For Casey, a device that allows him to speak accurately 97% of the time is life-changing, allowing him to hold a conversation with loved ones for the first time in years.
The BrainGate2 clinical trial is still open for enrollment. People interested in signing up can find more information on the trial’s site.
Funding & Disclosures
The study is supported by an ALS Pilot Clinical Trial Award from the Office of the Assistant Secretary of Defense for Health Affairs, a New Innovator Award from the National Institutes of Health and managed by the National Institute on Deafness and Other Communication Disorders, a postdoctoral fellowship funded by the A.P. Giannini Foundation, a grant from the Simons Collaboration for the Global Brain, the Searle Scholar Program, the Burroughs Wellcome Fund, the University of California, Davis, a grant from the National Institute on Deafness and Other Communication Disorders, a grant from the Department of Veterans Affairs Rehabilitation Research and Development Service, the Howard Hughes Medical Institute, and the Wu Tsai Neurosciences Institute at Stanford University.
