Neuroengineers at Columbia University in New York have created a system that translates thoughts into intelligible and recognizable speech, which could mean new ways for computers to communicate directly with the brain, according to research known yesterday.
By monitoring someone's brain activity, technology can recreate words that a person can hear with unprecedented clarity.
This advancement, which harnesses the power of voice synthesizers and artificial intelligence, lays the foundation for people who can not speak, such as those living with ALS or recover from a stroke, regain the ability to communicate with the outside world.
"Our voice helps us connect with our friends, family and the world around us, so losing the power of voice due to injury or illness is so devastating," said Nima Mesgarani, principal investigator at the Institute for Mental Behavior. Mortimer Cerebral B. Zuckerman of Columbia University.
The expert considered that they demonstrated that, "with the right technology, the thoughts of these people can be deciphered and understood by any other listener."
Decades of research have shown that when people talk and even think they are talking, activity-revealing patterns appear in their brains.
A different pattern of signs also appears when we hear someone talking or imagining what we are hearing.
The research team turned to a "vocoder," a computerized algorithm that can synthesize words after receiving training on recordings of people who speak.
"This is the same technology used by Apple's Amazon Echo and Siri, which gives verbal answers to our questions," explained Mesgarani.
To teach the vocoder to interpret brain activity, Mesgarani has teamed with Ashesh Dinesh Mehta, a neurosurgeon at the Institute of Neuroscience at Northwell Health Physician Partners, who treats patients with epilepsy, some of whom must undergo regular brain surgery.
"Working with Dr. Mehta, we asked epileptic patients who had already undergone brain surgery to hear phrases uttered by different people while measuring patterns of brain activity," said Mesgarani, who noted that these neural patterns trained the " vocoder ". "
The team plans to test the more complicated words and phrases later and even aspires to have their system part of an implant similar to that used with some epilepsy patients who translate the user's thoughts directly into words.
"That would give someone who has lost their ability to speak, whether due to injury or illness, a new opportunity to connect with the world," he added.