By Cana Yao

In a scientific milestone, researchers have helped a paralyzed man speak–for the first time since 2003, when a stroke following a terrible car accident left him paralyzed and speechless.

Now, scientists have tapped into speech areas of his brain so that the man, who is known by his nickname, Pancho, can produce comprehensible words just by trying to say them. When Pancho tries to speak, a rectangular sheet of 128 electrodes implanted in his brain detects signals from speech-related sensory and motor processes linked to the mouth and larynx, then sends them through a cable to a computer.

The computer then “translates the brain activity that would have normally controlled his vocal tract directly into words and sentences” using a form of artificial intelligence, according to postdoctoral engineer David Moses, who developed this system with two graduate students, Sean Metzger and Jessie R. Liu. The translated words are then displayed on a screen.

Pancho’s first understandable sentence, “my family is outside,” marked an achievement which is part of a global effort to help tens of thousands of people who are unable to talk, but whose brains still contain neural pathways for speech. Known as speech neuroprosthesis, this advancement could go on to help many patients with various conditions which steal their ability to talk.

For many, it “is farther than we’ve ever imagined we could go,” according to Melanie Fried-Oken, a professor of neurology and Pediatrics at Oregon Health and Science University. “The urgency can’t be overstated,” added Dr. Leigh Hochberg, a neurologist who co-wrote an editorial about the study. “It’s now only a matter of years before there will be a clinically useful system that will allow for the restoration of communication.”

When Pancho first began to work with neuroscientists, they worried that the part of his brain that controls speech could be dormant.

Dr. Edward Chang, chairman of neurological surgery at the University of California, San Francisco, said that his research team “just didn’t know if it would ever really wake up in order for him to speak again.”

However, the researchers soon found out that Pancho’s brain was indeed still awake. In 50 sessions over the course of approximately a year and a half, the scientists asked Pancho to read out loud from a list of 50 common words. He also tried using the 50 words in 50 different sentences. In the 9,000 times Pancho tried to say single words, the algorithm had nearly a 50% accuracy rate. When he tried speaking in sentences, the algorithm used a language-prediction system similar to autocorrect, accurately recognizing individual words nearly 75% of the time, and perfectly decoding entire sentences more than 50% of the time.

The speech recognition algorithm is still undergoing improvements, particularly as it has difficulties when Pancho laughs, yawns, or gets distracted. Also, the algorithm sometimes confuses words with similar sounds. Dr. Chang acknowledges, “we’ve got some things to work on because we obviously want [the algorithms] to work all the time.” However, accuracy has been steadily improving during sessions as the algorithms–and Pancho’s brain–learn and change together.

Previously, Pancho communicated for years by using a pointer attached to a baseball cap to spell words on a computer, a slow and arduous method that allowed him to type only about five correct words per minute by moving his head to hit the keys one at a time. He received a head-controlled mouse last year, and, though it is an improvement from the pointer, it is not nearly as fast as the brain implant, which allows him to communicate at 15-18 words per minute. Dr. Chang said that faster decoding is possible, but it is unclear if speech neuroprosthesis will ever be able to reach the typical conversational speed of 150 words per minute. Speed is one of the main reasons that Chang’s project focuses on speaking instead of typing or writing: “it’s the most natural way for people to communicate.”

“Not to be able to communicate with anyone, to have a normal conversation and express yourself in any way, it’s devastating, very hard to live with,” Pancho emailed the Times. Later, during a session with the electrodes, he said that speech neuroprosthetics were “very much like getting a second chance to talk again.”

The team hopes to engineer more sensitive electrode implants and make their project wireless. Dr. Fried-Oken adds that as more individuals participate in studies with neuroprostheses, scientists might find variations based on the individual condition of the patient. As Dr. Moses said, Pancho “is truly a pioneer.”

Sources:

https://www.nejm.org/doi/full/10.1056/NEJMoa2027540

https://www.nytimes.com/2021/07/14/health/speech-brain-implant-computer.html