Neuroscientists have just successfully translated handwriting-related cognitive signals into real-time text for the first time, according to a new study. The researchers’ new technique shattered the previous method, doubling the speed and allowing a paralyzed man to send a text at a rate of 90 characters per minute.
The system was designed by BrainGate to “enable people with severe speech and motor impairments to communicate by text, email or other forms of writing,” said Jaimie Henderson, Co-Director of The Neural Prosthetics Translational Laboratory at Stanford University, who is also a co-author of the study. Signals produced in the brain by thoughts and linked to handwriting were translated into text in real-time, allowing a paralyzed man to write 16 words per minute.
In 2017, Shenoy and his colleagues developed a thought-to-text conversion system that could outperform earlier techniques, which allowed monkeys to write texts at a rate of 12 words per minute. This success became the basis for further work on a brain-computer interface that surfaced later that year, which offered paralyzed people the ability to type 40 characters (about 80 words) per minute. But right now, “the current work goes beyond the 2017 article by more than doubling the punching speed of a paralyzed person, and using an entirely new and different method,” Henderson wrote in an email to Gizmodo. This is true because before that, no one had tried to capture and replicate the mental act of handwriting, and the groundbreaking experiment aimed to specifically design a never-before-seen thought-to-text system.
The subject of the experiment was a 65-year-old man who had suffered a severe and paralyzing spinal cord injury a decade earlier. “These electrodes can record the signals of a hundred neurons” and then process the signals by computer “to decode the brain activity associated with writing individual letters,” says Henderson.
In the middle of the experience, the man tried to move his paralyzed hand as he would when physically writing words. In his mind, he visualized the act of “writing letters on top of each other with a pen on a yellow notepad”. While he was doing this, a decoder typed in every letter he imagined writing while it was “identified by the neural network,” Henderson added. The “greater than” symbol was used to indicate spaces between words, which became the only way to detect the author’s desire or intention to leave a space between letters.
We cannot stress enough how moving and downright touching this project is. We’re still a long way from hearing doctors come up with this new brain-computer interface to help the average paralyzed victim, but the potential of this system could one day transform the way we write forever.
Reported by: Ehtisham
Written by: Imaaz Nadeem