Brain implant translates paralyzed man's thoughts into text with 94% accuracy

Previous threads:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27134049" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27134049</a><p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27157369" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27157369</a>

&gt; In tests, the man was able to achieve writing speeds of 90 characters per minute (about 18 words per minute), with approximately 94 percent accuracy (and up to 99 percent accuracy with autocorrect enabled).<p>I&#x27;d be interested in knowing how this metric changes over time as the user gains more experience with the BCI device. The article mentions that researchers recorded his neural activity while he was thinking about writing letters. Would the man eventually find that the system is more accurate or faster when he instead learns how to think &quot;the thought that generates the letter A in my BCI device&quot;? Fascinating stuff all around.

System diagram (how it works):<p><a href="https:&#x2F;&#x2F;github.com&#x2F;fwillett&#x2F;handwritingBCI&#x2F;raw&#x2F;main&#x2F;systemDiagram.png" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fwillett&#x2F;handwritingBCI&#x2F;raw&#x2F;main&#x2F;systemDi...</a><p>--<p>Code and data (for replicating results offline):<p><a href="https:&#x2F;&#x2F;github.com&#x2F;fwillett&#x2F;handwritingBCI" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;fwillett&#x2F;handwritingBCI</a><p>--<p>Published paper (you can find its full contents online):<p><a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-021-03506-2" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-021-03506-2</a>

Interesting that the user has to think of specific letters to spell out a word. I guess the 26 distinct English letters are much easier to parse and separate than the untold thousands of words, especially when those words could be used in different contexts.<p>I bet the next step wouldn&#x27;t be to even parse words to make a sentence. It seems to me the next low hanging fruit would be to enable this machine to parse common ideas. I wonder how complex it would be to translate full sentences like &quot;Good Morning&quot;, &quot;I gotta take a dump&quot;, or &quot;I&#x27;m hungry&quot;. It doesn&#x27;t seem like it would be that much of a leap, since the user already has to imagine the idea of different letters. Admittedly I have no idea how different those concepts are, or how they would express themselves in the brain to be interpreted by the machine.

I&#x27;ve spoken to Abe Caplan, one of the Braingate researchers a few times online on clubhouse. <a href="https:&#x2F;&#x2F;clubhousedb.com&#x2F;user&#x2F;abecaplan" rel="nofollow">https:&#x2F;&#x2F;clubhousedb.com&#x2F;user&#x2F;abecaplan</a> <a href="https:&#x2F;&#x2F;www.researchgate.net&#x2F;scientific-contributions&#x2F;Abraham-H-Caplan-33403213" rel="nofollow">https:&#x2F;&#x2F;www.researchgate.net&#x2F;scientific-contributions&#x2F;Abraha...</a> He said it did not need a high resolution to work well off the top of my head (it did not need to be very accurate, it does not have to be a single method of working, there are many places it can be linked and still work, this refers to the signal processing on the BCI), companies like neuralink often repackages these research projects with slick marketing as original but it was simply rebuilding this project with another interface (they used clunkier probes while neuralink wants to do an implant, but that was before they filed a patent for an implant as well).<p>This says it requires an implant, but I am not sure if its true, his contributions are much older though and they might not be the same as the current Braingate research, but they were also for aiding with disabled people with controlling prosthetics, and also with signal processing and calibration to the user, they have made text input before inplants were required, so I don&#x27;t see why its required, its benefit is being more convenient than setting up the probes or as a wearable. <a href="https:&#x2F;&#x2F;www.frontiersin.org&#x2F;articles&#x2F;10.3389&#x2F;fnins.2012.00072&#x2F;full" rel="nofollow">https:&#x2F;&#x2F;www.frontiersin.org&#x2F;articles&#x2F;10.3389&#x2F;fnins.2012.0007...</a>

I&#x27;m curious how his imagined writing compares to the handwriting he had before he was paralyzed. Was it always that messy or are the BCI controls difficult to use? For example, his comma seems exaggerated, as though he had to imagine grand gestures to get the SW to recognize it as a valid character. But on the other hand, his &quot;m&quot; and &quot;n&quot; look fairly normal. It&#x27;s possible that&#x27;s just what his handwriting looks like.

Interesting that the imagined letters are single-stroke, with no pen lifting, somewhat like most of the PalmOS graffiti letter forms.

Very cool. The opportunity for people with physical impairments to continue communicating - and convey their ability to still experience the world - gives us the ability to help these people lead happier lives.

How do you measure accuracy? Is there another way the man can communicate?<p>edit: The researchers compared the BCI output to a prompt that T5 was supposed to restate. I was thinking that T5 was communicating without prompts in the experiment. This isn&#x27;t my idea of translation accuracy, but you&#x27;ve got to have some baseline.

&gt; In this case, the man – called T5 in the study,<p>i wonder if it&#x27;s a coincidence?<p><a href="https:&#x2F;&#x2F;huggingface.co&#x2F;transformers&#x2F;model_doc&#x2F;t5.html" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;transformers&#x2F;model_doc&#x2F;t5.html</a>

This is the metaverse we should be funding, not phones strapped on bulky blindfolds.

Something even more impressive, IMHO, from the same source: <a href="https:&#x2F;&#x2F;www.sciencealert.com&#x2F;a-brain-implant-has-allowed-a-blind-woman-to-see-simple-2d-shapes-and-letters" rel="nofollow">https:&#x2F;&#x2F;www.sciencealert.com&#x2F;a-brain-implant-has-allowed-a-b...</a><p>It&#x27;s Star Trek like technology (think TNG series&#x27; Geordi La Forge).

The title is clickbait. This isn&#x27;t &quot;reading thoughts&quot;, it&#x27;s reading motor movements, which is a different part of the brain than cognition. Most people will assume thoughts = cognition reading the title.

I wonder how something like this would function with chorded typing? A keyboard like this [1] with only one key per finger I would imagine would be relatively easy (easier than handwriting even? Since it&#x27;s not limited to only your finger muscles - you attach every easily controllable muscle in the body to a button on the &#x27;keyboard&#x27;, and it seems to be a binary value whether a finger is clicking down or not, instead of what letter your hand is writing) for a brain implant to register. And a lot faster than handwriting.<p>[1] - <a href="https:&#x2F;&#x2F;www.gboards.ca&#x2F;product&#x2F;ginni" rel="nofollow">https:&#x2F;&#x2F;www.gboards.ca&#x2F;product&#x2F;ginni</a>

Total dystopian thought, but I wonder if this could ever be used to extract info people unwillingly, as in some interrogation scenario. I mean, you have a lot more control over what you say than what you think.

This is freaking awesome. Those mind-control drone pilots we have today mean there&#x27;s an exciting near-future for mind-machine interfaces. It will be fantastic not just for people who are paralyzed, but also perhaps we will have better prostheses, and maybe in the metaverse we will have additional appendages to do more tasks.<p>I am looking forward to our new world.

I wonder if it’d be faster if you imagined typing instead of writing (obviously would require the patient to be a proficient typist).

The next truly massive tech revolution will have to be something even we tech peeps reject. And I think bodily embedded stuff would do the trick.<p>It could be amazingly effective though if this is where we’re at already. Imagine the speed and enjoyment increase for anything from typing to gaming to driving a car. You’d get completely left behind if you rejected it.

I can imagine a pretty good horror&#x2F;scifi movie enabled by this.

This is really awesome. Imagine getting paid to think one day...

if someone spoke 94% of a second language we would say that person is fluent in that second language. this is great news

Can this be used for interrogation technique?

This is very very cool and interesting.

His first words were &quot;I need more computing power&quot;

Amazing

This opens the door to bad-thought-monitoring. Of course it will become required equipment for all citizens. Mark my words

&gt; (Don&#x27;t think this nurse is hot! Don&#x27;t think this nurse is hot! Holy shit she&#x27;s hot.)<p>&gt; (You can do this)<p>&gt; (OMG, is it on already?)<p>&gt; (You can do this)<p>&gt; (What a nice bottom)<p>&gt; (Don&#x27;t think bottom you idiot)<p>&gt; (She&#x27;s entering the room, think something else, now, fast, bunny bunny)<p>&gt; (Bunny)<p>&gt; (Bunny)<p>&gt; (Bunny, you got this)

It wouldn&#x27;t be particularly hard for a trained person to block their thoughts to a brain implant, if the police were trying to read their mind.

This too will be weaponized. How? Capture a North Korean official, put one of these in his brain and siphon off all the juicy intel. Capture a narco trafficker. ditto. Capture a terrorist. ditto. Have we made the world better? It seems that whenever we develop ANY device for improved information processing, it disrupts the world we live in and displaces the uniquely human style of information processing-rendering humans less necessary. It is paradoxical. We make our lives &#x27;easier&#x27; and human talent is lost.