For the first time, a single brain implant can now read a paralyzed person’s attempts to speak and gesture at the same time, turning them into text and the gestures of a digital avatar. According to the research paper published in Nature, the device uses artificial intelligence models to read the brain and transform its impulses into speech and body language. 

Current brain-computer interfaces (BCI) can decode only one thing at a time. You can either reliably translate speech or you can reliably translate gestures, but not both. They stumble because the brain areas involved in talking about something while moving your hands as you speak partly overlap.

Scientists from the University of California, San Francisco (UCSF), have now showed that their new system handles speech and gestures together thanks to artificial intelligence. Right now, it can handle only small vocabularies and two participants for simultaneous decoding. It’s also imperfectly accurate. But it’s a big step forward, because all humans communicate not only with language but also with how our bodies move. Hand gestures, like facial expressions, are an integral part of the experience of interacting with other people.

But this is only the first step toward a brighter future. “Our longer-term goal is to move toward what we think of as a ‘whole-body BCI’: a single brain interface that can restore multiple forms of communication and movement,” Samantha Brosler, one of the lead authors of the research paper, tells Fast Company .

Source link