Decoding Speech to Restore the Ability to Speak

Edward Chang in the operating room.

Courtesy of UCSF Communications Office

As he prepared to implant electrode arrays onto his paralyzed patient’s brain, Edward Chang wasn’t certain the surgery he was about to perform would make a difference.

The 36-year-old man, who asked to be identified as BRAVO1, had been in a car accident at age 20, sustaining a stroke in the aftermath. As a result of his injuries, his doctors told him he would never speak again.

Chang, a neurosurgeon at the University of California, San Francisco (UCSF), had been developing a technology with the potential to intervene. However, after 15 years of severe paralysis, Chang couldn’t be sure the brain’s commands for speech were still there. Whether that machinery could survive so long without use was an open question.

During a two-hour surgery, Chang and his team implanted 128 electrode arrays over the man’s sensorimotor cortex — the part of the brain that activates when a person speaks.

Their goal wasn’t for BRAVO1 to come out of surgery capable of speaking as he had before his accident. Instead, they would record the activity in BRAVO1’s brain while he attempted — and failed — to speak and decode that activity so one day a computer could translate BRAVO1’s brain activity into intelligible speech.

The process was painstaking. Over the course of 81 weeks, Chang’s team traveled to BRAVO1’s home, recording his brain activity while he tried to say 50 English words. In the end, he could string those words together in his brain, and the computer spoke for him. It was a first: No one had ever decoded complete words and sentences from the brain of a person with paralysis.

“It was the first time that we were able to successfully decode words and sentences from someone with severe paralysis,” Chang said. Now, he is more determined than ever to advance the technique to help more people.

From basic science to biomedical application

Edward Chang, professor of neurological surgery at the University of California, San Francisco.

Photo by Mike Kai Chen

In medical school, Chang became curious about how the brain processes speech while studying the neurophysiology of rodents. Like humans, animals such as rats experience a critical developmental period when their brains become sensitive to various sounds.

“I almost instantly became fascinated, mesmerized, and fully in awe of our ability to study something like a perception,” Chang said. By putting an electrode on the auditory cortex of the animal, he observed how the brain responds to different sounds.

The experience was a defining moment. Fast forward to 2013, Chang, now a faculty member at UCSF, worked with three patients who were undergoing surgery to treat severe epilepsy and examined the areas of their brains comprising the speech sensorimotor cortex. The team characterized how those brain areas organized to control the movement of the vocal tract — the lips, tongue, jaw, and larynx — when a person speaks.

“It’s an extraordinary privilege to be able to work with patients, on the one hand, helping them with these severe neurological conditions,” he said. “On the other hand, studying how their brains work.”

In order to map the brain areas activated during seizures, Chang implanted electrodes on the patients’ cortices. Doing so provided a unique opportunity to survey regions active during functions such as speech.

Chang's team mapped how the speech sensorimotor cortex represents the coordinated vocal tract movements of the lips, tongue, jaw, and larynx that produce the sounds of speech. Producing the sounds of various words — like the “s” in sun, and the “t” in taro — requires components of the vocal tract to engage in a complex, coordinated movement. A sensor device placed on the participant’s lip, jaw, and tongue captured the movement, and Chang and his team correlated the movement patterns with the activity of the speech sensorimotor cortex. The result was a neural map of speech movement, which was published in 2018 in Neuron. In a separate study, the researchers identified the brain area that controls vocal pitch.

In addition to speech production, Chang is exploring how speech is perceived. As a person speaks, the pitch of the voice rises and falls, creating the melodic contour we hear as intonation. He and his colleagues detailed how the superior temporal gyrus (STG) encodes the onset and changes in the speech amplitude, leading to our perception of words.Unlike earlier work suggesting the STG is a static filter of auditory signals, they found the STG is “a very active, dynamic readout of our conscious perception,” Chang said. 

Together, this burgeoning map of the brain became the blueprint for developing a speech brain-computer interface, or “neuroprosthesis” — the technology that helped BRAVO1 and others turn thoughts into speech.

Future of the neuroscience of speech

Chang received the 2025 Gruber Neuroscience Prize for his research on how the human brain encodes speech, presented during Neuroscience 2025, the annual meeting of the Society for Neuroscience.

Society for Neuroscience, 2025

Chang’s work built on foundational research by neuroscientists like Robert Knight at the University of California, Berkeley.

In a 2006 paper published in Science, Knight and colleagues detailed the low- and high-frequency electrical oscillations in human brains as neurons process information. “We showed that we could map the whole language circuit, which is precisely tracked by high-frequency activity in the cortex,” said Knight, an expert in using intracranial electrical recording to study brain activities who is also Chang’s postdoctoral mentor.

The ability to monitor these signals using electrodes offers a window into the neurophysiology of cognition, memory, and language, enabling subsequent breakthroughs in devices to restore speech.

“Eddie has two qualities that set him apart: One, he has a combination of theoretical and empirical brilliance. He's unbelievable in terms of articulating the idea he wants to test, what the hypothesis is, and how to get the data,” said Knight. “The second thing that he has that very few people have is a laser focus on making sure that his basic science program [understanding speech perception] can help people,” he said.

In 2023, Chang and his team enabled a woman who had lost her ability to speak after a stroke convey her first words. Relying on AI and machine learning, the researchers decoded her attempted speech directly from brain activity, translating it into text, into synthesized speech in her own voice, and into the facial movements of a digital avatar. The whole development is “fascinating,” said Gopala Anumanchipalli, a computer scientist at the University of California, Berkeley, who was involved in the study.

As an engineer and Chang’s former postdoctoral mentee, Anumanchipalli witnessed how the “seamless blending of the science, engineering, and translational outlook” led to this innovative work. Anumanchipalli said it is something that brings him great excitement about the future.

Despite recent breakthroughs, Chang believes “we’re still at the very beginning stages of understanding how this really incredible human behavior works.” Ultimately, this growing body of research will transform the understanding of how the brain operates. “We want to rewrite the textbook,” he added.

As the technology and science mature, Chang envisions a future where brain-computer interfaces will become more commonplace. “The more we learn about the basic properties and physiology [of the brain], the more we can translate it to help people who have different kinds of neurological conditions,” he said.

About the Author

Kristel Tjandra

Kristel Tjandra is a science writer with a PhD in chemistry and a master's in science communication. Her writings have appeared in Science, Eos, and MIT Technology Review, among others. You can follow her work on her website: www.kristeltjandra.com.