YT YouTube2Text

YouTube2Text → Transcripts → Panel: Neurotechnology The Role of AI in Brain Computer Interfaces (2024)

Panel: Neurotechnology The Role of AI in Brain Computer Interfaces (2024) — Transcript

by MIT AI Conference · 6,662 words · 957 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01this promises to be a very
0:03intellectually stimulating panel no pun
0:05intended and we have an amazing lineup
0:08of panelists today but I'm just going to
0:11introduce our awesome moderator we have
0:13Alyssa welli here today she is a
0:15freelance journalist freelance science
0:17journalist and it's very exciting
0:19because unlike a lot of journalists I
0:21meet Who start learning about a field
0:22she got a PhD in neural technology then
0:24decide to go be a journalist and so
0:26she's uh fairly qualified to moderate
0:29this panel what's also fun about her is
0:32that she is the ultimate survivalist she
0:35was out in the wilderness for 7 Days in
0:37the Colorado mountains with her dog and
0:42most fun fact she's getting married in
0:44May with that I'll turn it over to
0:47Alyssa thank you so much hey everybody
0:50we're excited to be here I'm really
0:51excited to moderate this panel thank you
0:53all for coming we've got really amazing
0:56experts here today and the best part
0:58about this panel is that we're going to
1:00talk about this awesome field of neural
1:02technology in the context of AI from
1:05kind of two big perspectives both from
1:08the research perspective and also from
1:11the commercial perspective we've got
1:12here um two people people from synchron
1:15got Peter you and Tom Oxley here Tom is
1:18the CEO of synchron and Peter is the
1:21scientific director of um neuroscience
1:23and algorithms and then we also have
1:25from the research side we have Miriam
1:28Shi from University of of Southern
1:30California um where she is the uh
1:32professor and chair of electrical and
1:34computer engineering um as well as a
1:37couple of other affiliations and
1:39biomedical engineering and other um
1:41things so we've got a really great panel
1:43of experts and I'm excited to get into
1:45this awesome um this awesome topic so to
1:48start off maybe we can kind of just set
1:51the stage here of what neural technology
1:54is Maran would you be okay with starting
1:56us off just tell us what is
1:58neurotechnology and then what is your
2:01research specifically in
2:03neurotech okay great well thanks for the
2:05introduction it's really fun to be here
2:07today um so neurotechnology is kind of
2:10an interface with the brain and the goal
2:13is that you would record electrical
2:16activity patterns in the brain or other
2:19potential magnetic patterns or Optical
2:21pattern any kind of modality that can
2:23measure the brain and then you would try
2:26to use that to decode an internal state
2:29of the brain or a behavioral State and
2:33then use that um as potential means to
2:36provide therapy or treat brain disorder
2:39so let's take an example let's say
2:40somebody who is paralyzed they still can
2:44think about and plan a movement but they
2:46can't communicate that to their muscles
2:48so what if you can record from their
2:50brain try to decode what movement they
2:53want to perform and then translate that
2:55into a robotic arm movement or a cursor
2:57movement then we can actually restore
3:00function in that paralyzed patients so
3:03that is what neurotechnology is that
3:05interface with the brain that allows us
3:07to decode and interpret brain signals
3:11and originally when the field started
3:14about 20 years ago uh the focus was on
3:17movements how to we decode movements for
3:19the obvious purpose of restoring
3:22function in paralyzed patients have
3:24we've come quite a long way since then
3:26we have companies like synchron that are
3:28commercializing the products but but
3:31recently we've also been thinking about
3:34brain computer interfaces for a new
3:36generation of applications for example
3:40for mental health conditions can we also
3:43think about decoding emotions and moods
3:46and if we can do that let's say in
3:49somebody who has major depression then
3:51guide them as to um the mode of therapy
3:54or whether they should see their
3:56physician so you can think of
3:57neurotechnologies that these interface
3:59es with the brain that try to infer uh
4:03internal and behavioral states to do
4:05something good for the patient as
4:07potentially alternative therapies thank
4:10you yeah and so Tom can you can you just
4:12tell us a bit about synchron and and
4:14maybe answer the same question what is
4:15neurotech but then from your perspective
4:17from the commercialization
4:19side yeah so uh hi everyone Tom C of
4:23synchron I'm a neurologist by training
4:25and I'd heard about uh well W to
4:28neurology uh in medicine it's you know
4:31an interesting area of medicine very
4:33interesting uh often the joke from other
4:36surgeons who were saying that well you
4:37can diagnose everything in the brain but
4:39you can't treat anything so I think in
4:41medicine has traditionally been a field
4:43that you can't do much about and yet
4:45with a as much as a mortality morbidity
4:48burden as say cardiac so one of the
4:51biggest causes of of Health burden that
4:53we um there can't do much about so
4:55neurotechnology is um uh health and
4:58consumer Tech uh technologies that uh
5:02interface with the brain or the nervous
5:04system and I'd categorize them into I
5:06guess there's brain computer interface
5:08that's implantable or not implantable
5:10there are medical applications there are
5:12consumer
5:13applications traditionally in medical
5:15implants there have been implants that
5:17uh write stimulate deep brain
5:20stimulation clear devices for
5:22hearing and now this new wave uh sens is
5:25going into the brain that do more read
5:28so that's a read WR analogy
5:30so we're one of a group of companies
5:32you've probably heard of neuralink Elon
5:34musk's company we're one of about six or
5:36seven now that are building an
5:37implantable brain computer
5:39interface uh it's been around for 30
5:43years but none have made it to Market
5:45and that's because it's very complicated
5:47technology um needs to go into the brain
5:50it needs to give a steady amount of
5:52information out it needs to be safe it
5:53needs to work for a lifetime and then it
5:55needs to interface with products that
5:57are useful um Maran mention paralysis
6:00the field started out controlling
6:02robotic limbs but there was really no
6:03market for that now the field is looking
6:06to take uh input from the brain decode
6:10it and then control things that are
6:12widely available that we all get used to
6:14that can improve your ability to live
6:16independently and that's really
6:17smartphone control that's where the fa
6:19that's where the field is starting so we
6:21have a device uh we're the only company
6:23that doesn't have to crack open the
6:25skull to get into the brain so we build
6:27technology that's delivered through the
6:29blood vessel which can record
6:31information without having to penetrate
6:33that bloodb brain barrier um so it kind
6:36of looks like a pacemaker it goes up
6:38into the brain and it's meant to control
6:39an iPhone for people whose arms don't
6:41work yeah awesome and Peter could you
6:44just tell me a little bit about how does
6:46synchron like Leverage AI maybe you know
6:50in the you know behind the scenes and
6:52then also like how the patient is
6:54interacting with their interface sure hi
6:57thanks for having me here um the way syn
6:59leverages AI first of all it's in the
7:01neural decoding side so obviously I'm
7:04grossly simplifying this but at the end
7:06of the day we really want to model uh
7:09what your brain is doing in terms of
7:11what the patient intending to Output in
7:12terms of uh Behavior so um machine
7:16learning and the latest advances in uh
7:19deep learning and AI are one of the many
7:21tools that we utilize to tackle this
7:23problem um so obviously Mariam LA's also
7:27very familiar with this and that's the
7:28primary way that that we levage AO but
7:31also we are trying to commercialize this
7:33technology um BCI has been demonstrated
7:36for decades now we know we can decode
7:39these behavioral intentions directly
7:41from the brain so I think um the Spotify
7:44VP was talking about tokenizing
7:46different various things so if you can
7:48think about Snippets of brain activity
7:50to be a token then we can predict the
7:51next thing to think you know this
7:53patient is trying to move uh his right
7:56hand you know forward or left or to a
7:59diagonal Direction that's all well but
8:01how do you now make it useful at the end
8:03of it as a commercial product um and I
8:05say commercial because that's the best
8:07way to actually disseminate this very
8:09useful technology to people who need it
8:11totally paralyzed patients and people
8:13suffering from priess so in that and
8:16interface we can now utilize all these
8:18AI tools that effectively reduce complex
8:21choices down so reduces that high
8:25dimension of you know your desktop um
8:27interface into something that might you
8:29know just a few choices so now if you
8:32mix that as well as simple and robust
8:35behavioral output that the patient can
8:36directly out from the brain then now the
8:39experience gets a lot easier and even if
8:41you're totally paralyzed or have lots of
8:43moto difficulties and have other
8:45problems going being able to interface
8:47with your digital realm becomes lot
8:48simpler so those are the way that we're
8:51thinking about leveraging a and you know
8:53various places trying to really
8:55commercialize this
8:57technology yeah Mariam can you talk a
8:59little about that I mean are you using
9:02off-the-shelf AI Tools in your lab or
9:06are you kind of starting from scratch
9:08building up like how does how do you use
9:11AI in your research yeah so we also use
9:14AI primarily to kind of model these very
9:18complex brain activity pattern and try
9:20to infer some hidden information now
9:23unfortunately we cannot use
9:25off-the-shelf AI tools because the
9:27challenges related to the brain and
9:29brain signals and brain data are very
9:31distinct so if you think about it brain
9:34signals are these time series data
9:36they're not as structured as language
9:39there's a lot of heterogeneity from one
9:42subject to the next the recording
9:44modalities are distinct there's a lot of
9:47complexity and noise so the question is
9:50how do we develop AI architectures that
9:53can model these distinct multimodal
9:56heterogeneous brain signals and infer
9:58some hidden information from them so
10:01that's one challenge the other challenge
10:04is the sparity of brain data right
10:07unlike chat GPT or large language model
10:10that had the entire internet to train on
10:13we don't have that much data we don't
10:15have an internet of brain data so can we
10:18develop architectures that can be
10:20trainable with the amount of data that
10:22we have finally we want to build Clos
10:25Loop systems that run in real time and
10:28we have to take into account that
10:30whatever we develop that inference
10:33algorithm needs to be able to run in
10:35real time in an implantable device so
10:38that also imposes constraints on our
10:41compute architectures and inference
10:43architecture and training costs so we
10:45have to take all of this into account
10:48and therefore while we can of course
10:50take advantage of some of the conceptual
10:53advances in large language models we
10:55can't really use them off the shelf we
10:58have to develop our architectures that
11:00are for these challenges related to
11:03building brain computer interfaces and
11:05innovate also on the AI
11:07front you're you're nodding your head a
11:09lot Peter what what kind of things were
11:11you thinking about when she was um
11:13discussing this I mean all of those
11:14things that Mariam just mentioned are
11:17the difficulties of commercializing uh
11:19um this technology so you know you and I
11:22all different um I hope you can see that
11:25and even myself this morning before I
11:26had a coffee versus after that all
11:28changes your brain St
11:29but the whole thing is now we're at a
11:31really exciting stage where um we can
11:34have these devices um inside of patients
11:38chronically and therefore now we're
11:40collecting this really rich set of data
11:41really for the first time in history so
11:44we're at this really interesting pre PR
11:46precipice of um this you know new AI
11:49technology coming but also now we are
11:52starting to being able to collect the
11:55large sets of data um data from um
11:57various contexts um so that's really
12:00exciting so I hope it excites future AI
12:03scientists and Engineers CU now there's
12:05new set of data that's going to start
12:07building from now on and it didn't
12:08really exist before so we've got 10
12:11people that people that implant now uh
12:15our longest is 5 years I think CL so
12:18maram just mentioned uh Building
12:21architecture with a small data set in
12:23order to be able to train so we're now
12:25getting ready for the next sort of we're
12:27scaling our clinical Tri as we go so how
12:30have you
12:31prepared how have
12:33you this is your this is your
12:35performance this our performance review
12:37how have you prepared your architecture
12:40with regard to what Mar said about
12:41training using the data set that we have
12:43preparing for the next next set of
12:44patients yeah I mean I think first thing
12:47to do is trying to be a really good
12:49scientist um so this is experimental
12:52preparation I'm I'm trying to um treat
12:55this even in the commercial setting as
12:57trying to a really good tightly
12:58controled stud
13:00um and in doing so we're building a lot
13:02of infrastructure inh house um to be
13:04able to curate this data as well for BCI
13:07specifically so we're not trying to you
13:09know do a quick thing where um
13:10leveraging some other tools we're trying
13:12to really build purpose build tools to
13:14build this data set well from the start
13:16and of course we're at the very
13:17beginning stages but that's how we're
13:19preparing so that once we start to
13:21curate this data set and the and the
13:23Floodgate so to speak breaks then we're
13:25ready to you know ready to really um
13:27take advantage of this this brand new
13:30data set for the first time and of
13:32course you know a lot of things we think
13:34about is biod data so you have to be
13:36careful about how you um collect and
13:39store and be a good custodians of the
13:41data
13:43too and mam can you talk a little bit
13:45about you've you've kind of pivoted in
13:47the BCI space you did a lot of still do
13:49motor research but then you're also
13:51moving into this new space I mean is
13:54there can you kind of pull apart the
13:56differences in how you use AI or even
13:58just algorith Ms in general in these two
14:01lines of
14:02research yeah so again for those in the
14:05audience who are not as much into this
14:07domain so if you think about a BCI for
14:09motor the idea is you would decode an
14:12intended movement and translate that to
14:14the movement of let's say a computer
14:16cursor or or some computer interface
14:19really if you think about who is
14:21controlling that cursor it's the person
14:25themselves right it's the brain that is
14:27uh acting as the controller of that
14:29movement when you think about regulating
14:32an abnormal state of the brain let's say
14:35a depression severity symptom really the
14:39question is a little bit distinct you
14:41want to decode for example the symptom
14:44State at this time but you want to go
14:46beyond that and actually regulate and
14:49change that symptom state so that you
14:51can help the patient feel better so the
14:53distinction is essentially you have
14:55again a closed loop system but you want
14:58the Reg regulation in addition to the
15:01decoding to happen on brain signal so
15:03there's a lot of interesting inter
15:06intersection between now control theory
15:08another branch in engineering as well as
15:11AI how do we develop AI algorithms that
15:14can not only infer let's say an
15:15emotional state but also go beyond that
15:18and regulate uh that state so that's
15:21really the idea decode mental States and
15:24then regulate them when um somebody for
15:26example is depressed so that's kind of
15:28the distinction from the modeling
15:30perspective what what mental states have
15:33you decoded so we've decoded um a mood
15:36state so far we've shown for the first
15:39time that we can actually decode mood
15:40from Human intracranial brain activity
15:43and when I say mood the operational
15:45meaning of that is a state that's
15:47related to both depression and anxiety
15:50symptoms and we then look at different
15:52subscales so anxiety alone depression
15:55alone we can also decode those subscales
15:57of mood so now you can you can extend
16:00that you can say okay can I uh in the
16:02case of say obsessive compulsive
16:04disorder can I decode obsessions and
16:07compulsions from brain activity um in
16:09the case of addiction can I uh decode
16:12the craving uh level from the brain such
16:15that I can intervene I can tell the
16:17patient hey go see your um I don't know
16:19physician or give them uh take your pill
16:22right now so that's the idea they're
16:24very distinct mental States but what
16:28they're all kind of sometimes comorbid
16:30even in some of these psychiatric
16:32disorders and those are the ones that we
16:34try to decode are you presenting the
16:37patient and so you've got a closed loop
16:39deep bra stimulation you're looking for
16:40features and then that modulates how the
16:42stimulation comes in which is meant to
16:44impact the mood have you um presented to
16:48the patients your version of their mood
16:51to them rather than separate to the
16:53delivery of the therapy and has that had
16:55an impact no we have not done that so
16:58what we try to do so far is to try to n
17:01just track their naturalistic variations
17:04in their mood States we don't try to
17:06induce it we don't try to I don't know
17:10involve them in a task that changes that
17:12because the brain networks and the Brain
17:15representations for naturalistic
17:17variations May potentially be distinct
17:19than induc States so like watching a sad
17:21movie is very different than having
17:24depression right so you want to really
17:25track those clinical symptom States so
17:28that you can intervene uh correctly at
17:30that do you track what movies they're
17:32watching no I don't so how do you solve
17:35the contextual of it yes so so very
17:39early days right so I mean if mot B
17:42started 30 years ago we're like within
17:44five years of starting but at this point
17:46what we are doing is we are tracking
17:49their mood States within a 10day period
17:51when they're in the hospital so the
17:53context doesn't really change much but
17:56let's say down the road when they're in
17:58the and their at home environment
18:00there's of course going to be contextual
18:02changes so the idea is that then you
18:04would be able to also record data during
18:07those context changes and perhaps that
18:10latent representation of your deep
18:12learning architecture has as part of it
18:15the context also encoded and therefore
18:18it can figure out not only the state but
18:21also the context such that it can better
18:24capture the mental state at that time
18:29it's exciting really exciting stuff and
18:31I'm curious to know when you think about
18:33the future of neurotech when you think
18:34about the future of synchron or just in
18:36general the field where do you see it
18:39headed and what role do you see AI
18:41playing Tom maybe you can take
18:44that um so right
18:47now uh well the first incarnation of
18:51what the FDA is calling implantable
18:53brain computer interface is focused on
18:56motor motor system so the motor system
18:58is the control of movement um but you
19:01can decode the control of movement
19:04without moving because you can either be
19:06paralyzed or you can intend to move so
19:08there's a whole control system up here
19:10that is completely separate to what gets
19:12expressed so the idea is if you can have
19:14a reliable uh measure of motor and
19:18tension you can convert that into a
19:20digital system that can enable you to
19:22engage in a digital system so that's the
19:24first
19:25Incarnation um you've probably you might
19:28have seen Eddie Chang's lab in um in
19:30UCSF has done a lot of there's been a
19:32lot of talk of speech decoding but
19:35they're not decoding the content of
19:37speech they're decoding the way that the
19:39motor system controls the vocal cords to
19:41create phon names so it's actually a
19:44decoding of the attempted vocalization
19:46of speech um so it's still even though
19:49it's being called Speech it's all still
19:51in the domain of motor decoding and so
19:54Eddie the concept of motor decoding for
19:56phones is super interesting I think
19:57that's going to be um a great space I'm
19:59not sure how big the market is for it
20:01but we think that the market uh the need
20:05the huge need for people who are
20:06paralyzed is an ability to live
20:10independently so 3 million people in the
20:13US maybe I think we think there's about
20:1412 million globally who have some
20:17medical
20:18condition uh can't move their body it
20:21might be a stroke ALS muscular distopy
20:23multiple sclerosis arthropathy uh muscle
20:27disease whole range of conditions where
20:29your brain's working can't move your
20:30body if you can enable them to control a
20:34digital ecosystem in a way that stops
20:36them going to a care facility stops
20:38having a caregiver stops really having
20:40anyone else in the room so they can get
20:42their privacy and autonomy back that's
20:44the huge need and I think there's huge
20:47capacity for this technology to help
20:49people become more
20:51autonomous um right now in the early
20:54stages so neuralink removing a bit of
20:56bone um so pics Precision or doing a
21:00smaller cut cutting a bit of bone we're
21:02getting a little bit we're all getting a
21:03little Slither of the motor cortex and
21:06Pete and I think that the growth over
21:09time is going to be the growth of BCI
21:11over time is going to be the ability of
21:13companies to build technology that
21:15scales to more and more regions of brain
21:17in a way which doesn't disturb or damage
21:19brain that's going to be the Improvement
21:22of the I don't know you can speak to the
21:24tokenization but how we improve our
21:25models is by getting access to more and
21:28more
21:29brain data and then building these
21:31Foundation models that enable motor
21:33inference that enables you to now
21:35Control Systems so you know we're
21:37building Bluetooth controls but you know
21:40Steve Jobs hated the big iPhone because
21:43he thought your hand should be able to
21:45reach any part of the phone and so when
21:48the big iPhone came out he hated that
21:49because he's like well you have to use
21:50two hands to control it now the point
21:52I'm making is that all of the technology
21:54we use is built around the limitations
21:56of our physical manifestation
21:59about you know our bodies so I think BCI
22:01is going to break through that over time
22:04because we're not going to have that
22:05constraint of what are the constraints
22:07of your physical manifest like physical
22:09body and so you know so that's going to
22:12be like phase one is motor and
22:16eventually you know I think that gets to
22:19um from small device control to
22:21three-dimensional control robotics
22:22control and TI World control you know
22:25Total Recall you're walking around and
22:27there's a screening everywhere and
22:29you're able to engage with things
22:30without touching anything but then I
22:32think what mariam's doing for everyone
22:34in the audience Mariam is world world
22:36leading uh person in as academic
22:40scientist in decoding um emotional
22:43states from brain impls and I think
22:46that's going to be the second wave um
22:48and I asked that question like he
22:50showing the patients because I'm still
22:52grappling uh my evenings with the ethics
22:56of uh decoding uh emotional states
23:00because I don't I don't know what you do
23:01with that information in a way that's
23:03you know it raises a whole bunch of uh
23:05ethical questions that we have to deal
23:07with but
23:09um I'm crapping on here but I'm going to
23:12say I think what'll come in that wave is
23:15the limitation of our ability as humans
23:17to connect with each other in an
23:18emotional way and in a communication way
23:20we can we we're good at using words um
23:22words don't really capture what we're
23:24saying I think that BCI is going to
23:27enable a level of human uh communication
23:30that is not possible with our
23:32bodies and also from a clinical
23:34perspective right I mean I completely
23:37agree the first phase is going to be
23:38moror and there's a huge need for that
23:40as Tom mentioned but if you think about
23:43psychiatric disorders the number of
23:46patients who are impacted order of
23:48magnitude larger it's a huge societal
23:51burden leading cause of disability
23:53worldwide so anything that we can do
23:56would be of huge impact pack for these
23:59patients and um if you think about the
24:02ethical implications there's a lot of
24:04neuro ethicist who are actually thinking
24:06about this but if you think about 20% of
24:09major depression patients who have
24:12essentially no options no drug Works
24:14therapy doesn't work they're at risk of
24:17suicide I would argue it it would be
24:20unethical for us not to do anything and
24:22there are very smart people think about
24:24the neuroethics of this similar thing
24:27for speech PC eyes right so people have
24:30shown that you can decode silent speech
24:33when somebody is just thinking about
24:35saying something they can also decode
24:37that and that raised a whole set of
24:39ethical questions as well so it's not
24:41limited to emotions but all kind of
24:44neurotechnologies and bcis and I think
24:46it's it's imperative on all of us and on
24:49neuro ethicist to really think deeply
24:51about datacal implications and solutions
24:54for that while we push the technology
24:56forward to help these tens of millions
24:59of
25:00patients yeah and and what you're
25:03mentioning too is also it relies upon
25:05someone having the implant in the brain
25:08too right right right right and this is
25:10at this point frankly for the most
25:13severe cases when there's really no
25:16other option drug has failed therapy has
25:19failed or or in the case of paralysis
25:21like they really have no remaining
25:23movement or EMG that can be utilized
25:26that's the kind of BC that h and us and
25:29the field is is building so this is
25:31really for very severe cases where there
25:34are really no Alternatives and the hope
25:37is that this would improve the quality
25:39of life for these people as alternative
25:41and potentially revolutionizing
25:44therapies yeah and Peter I want to go
25:46back a little bit to when we were
25:48talking about um building AI tools to
25:51help with um you know preparing for the
25:53next patient right you've done 10 is
25:56that right we've got
25:5910 yeah 10 so far 10 patients so far
26:01okay so I mean you know there's a lot of
26:03energy and a lot of enthusiasm for
26:05building AI tools that are useful to
26:07certain industries you know maybe not
26:10people that are necessarily in the
26:11industry but want to be you know useful
26:13for certain things I mean are there
26:14already gaps that you've noticed that
26:17would be really useful tools in your
26:20right in your work yeah I mean so I'm
26:23going to speak in hypotheticals
26:25here so there are couple ways that we
26:27can tackle this problem one is that you
26:29know we collect lots of data do the
26:31traditional machine learning we are able
26:33to model the brain well enough and that
26:34generalizes across people or the
26:36different patients different hours
26:38that's one way so far how BC has been
26:41working is is you have a very dedicated
26:44PhD student or a post do working really
26:47personally with the patients they almost
26:48like have a PT um and and maram you and
26:51I are talking yesterday at dinner or
26:53maybe it was a backstage today um it
26:55takes special skills not just science
26:58scientific um and Engineering press but
27:00also that badside manner and um how the
27:05patient is taking in the experiments and
27:07you know the what the brainet looks like
27:09um so another alternative way is can AI
27:13help to kind of do that training right
27:16so um we'll see whether both are
27:19required or I hope the first one works
27:21just well enough because it's a much
27:23simpler um simpler uh solution but in
27:27another world um
27:28these um increasingly intelligent tools
27:32could perhaps help in these uh
27:34personalization right so we talked about
27:36the Spotify before that was really
27:37interesting it's a single um single
27:41company single brand but very well
27:43customized across all the different
27:45individuals so I can kind of see
27:47something like that happening um if it
27:49turns out that after Decades of research
27:51that human brain it are really
27:53individualized at that level or maybe
27:55we're struggling with um really finding
27:57the latent space represents everyone
27:59because um we're talking about what do
28:01you think about the future of neurotech
28:02and BCI is if I talk in 5 to 10 years
28:05time I'm hoping that we can create a
28:07synthetic virtual sorry virtual nervous
28:09system so this is part of your body it
28:11should be near your body uh whether it's
28:14a wearable or implantable your algorithm
28:16should bearn uh running kind of with you
28:19and not somewh remotely so um now in
28:23this sense how do you retrain these
28:24models and um you know who does it um so
28:28I think in that sense the air tools come
28:29really handy and CU you're kind of
28:31working with your PT to make your stuff
28:32better make your body better that's why
28:35Tom and I always insist that this is on
28:36your prosthetic it's not an assistant
28:38device we're trying to restore function
28:40of the body so that's how I think about
28:49it what do you think about a virtual
28:50nervous system virtu virtual nervous
28:53system that's just Dro I mean it's a
28:54fancy you know it's a fancy term I me
28:57it's cool I I mean the way I think about
28:59the virtual nervous system is
29:01essentially a foundation model for the
29:03brain so I mean as you probably this
29:06audience knows we have Foundation models
29:07for language right what if we can
29:10actually build one put all of our data
29:12together from different modalities
29:13whether is electrical activity magnetic
29:16Optical Imaging put it all together to
29:18come up with a foundation model of the
29:20brain that then we can use Downstream to
29:23to replicate various functions that
29:25somebody has whether it's movement
29:27whether it's their emot
29:28with any kind of action they have so to
29:31me that is kind of a virtual nervous
29:33system where you have modeled the entire
29:35brain and then you can figure out the
29:37mechanisms and then use it to enhance or
29:41restore
29:42function is that what raid Hoffman did
29:44this
29:45morning I don't think at the level of
29:48the nervous system at the level of
29:50behavior right that interesting well and
29:53one of the things that's we talked about
29:55before that's a little bit tricky about
29:56neurot tech is just the fact fact that
29:58there isn't as much data as we have as
30:01like large large linkage model we can't
30:02just go scrape you know the internet and
30:05then we have all of this data that we
30:07can build off of we're using very select
30:09finite amounts of data and very precious
30:11data that you know each patient each
30:13thing so as we get more and more data
30:17you know through new companies through
30:18new efforts just through longer patients
30:21I mean what does that open up for you in
30:24the future like what do you do with all
30:25that extra data yeah so I mean the dream
30:28is that you would combine all that data
30:31and train a deep learning architecture
30:34that can actually generalize from person
30:37to person and perhaps there's some
30:40fine-tuning needed but overall the model
30:43would generalize and that would really
30:46uh from an individualization point of
30:48view there's always going to be
30:49heterogenity but perhaps there's also a
30:52lot of biological commonality and if you
30:54put all the data together and come up
30:56with a foundation model we have already
30:58already captured the commonality and
31:00then if you want to just personalize the
31:02model to an individual patient we just
31:04need a little bit of fine tuning but we
31:07don't start from scratch so I think
31:09there's a lot of excitement that we can
31:11find these shared latent representation
31:14across modalities of measurement from
31:16the brain to come up with Foundation
31:18models and these latent each set of them
31:21would relate to a given function and we
31:22can get it Downstream how close are we
31:25to that um it's hard to put a
31:28we we're not there but we are not too
31:31far I think I think um I think we are
31:35maybe like to the first like really
31:38Foundation model maybe five years away
31:40five years okay okay well we built one I
31:43mean we've so I think the productization
31:45of that is out of the box it's got to
31:47work straight away now maybe the feature
31:49set is not as sophisticated as what it
31:51could be but why don't you speak to that
31:53cuz we've you've gone through that
31:54journey of yeah um to to to be
31:57successful commercial it's got a scale
31:59and we can't have a neuroscientist in
32:01each person's home so how have you
32:03solved that where are we up to with that
32:06uh yeah so of course we want this
32:09technology to work at mass for all of
32:11our patients and you don't want to spend
32:13hundreds of hours to make it work and I
32:15think to maram when we say foundational
32:17model I think you could also it varies
32:19in terms of how much Dimension are you
32:21trying to represent so one thing we can
32:24do is reduce their problem you know to
32:26be simpler for our own benefit at the
32:29benefit so that the patient can use
32:30stuff straight out of the box so
32:32something like general motor intention
32:35so what doesn't matter if you're trying
32:36to move your hand or a foot or whatever
32:39um uh We've we we've built a much
32:42simpler version of the model that you
32:43can just detect that right so even if
32:46the patient tried to move their hand or
32:47the right hand doesn't matter if it was
32:49patient one or patient two um we can at
32:51least very simple simply detect that
32:53onset or offset so um but then of course
32:57once we want to get to the stage of
32:59trying to decod the entire body you know
33:02complet um so the output that comes out
33:04of our model versus a motion capture
33:07suit being equal is kind of the the
33:10future that I dream of but of course
33:12capturing that um uh a foundational
33:15model will be a lot more difficult but
33:17you know we're trying to get there and
33:19we we started like we've we've you know
33:22we're setting navigate and click as the
33:24minimal viable um performance of the
33:26interaction method that's the put from
33:28those motor intents so you've got to be
33:29able to um choose a area on the screen
33:32and select it um as a minimum use case
33:35but we also come to realize like oh you
33:38have to turn the thing on so like we all
33:42take that for granted you swipe it it
33:43looks to your face it unlocks or you put
33:45your passcode in that's you know our
33:47users can't do that so we're now having
33:49to generate these variable signatures
33:51out of the model that enable the user to
33:53wake up turn on the system activate an
33:56app choose what they want to do na in
33:58select make some options and then and
33:59then shut it again so all these
34:02different you know models that you're
34:03using have these different outputs that
34:05serve these basic things that we all
34:07take for granted yeah I think these are
34:08the things that become apparent when
34:10you're trying to commercialize the
34:11things so you know we all remember butt
34:13dialing you know sitting on your phone
34:14and calling someone and you're talking
34:16and that's not what you want and
34:17something like the neural interface is
34:19you know it's on it's recording all the
34:21time if you think about it it's just
34:23electricity conducting through and then
34:25if it's connect to your phone you don't
34:26want it to make you know bunch of butt
34:28dial so trying to think about this we're
34:30really thinking about the actual
34:32commercial applications um how will the
34:34user actually use it and it's an
34:36interesting space to be in BCI at this
34:38time because traditionally you don't
34:40really think about well L Academy would
34:42think think about it that way you're in
34:43a control setting um you run experiments
34:46you check their bit rate and see how
34:49well they go um and and home use
34:52obviously a lot this really excellent
34:54group thinking about these obviously
34:55brain gate uh Ramsey Ramsey lab they're
34:57all thinking about at home use but at a
34:59commercial scale you know it's supposed
35:02to work for everyone so how how can we
35:03make a simple robust Solutions that's
35:05going to work for everyone is a
35:07challenging challenging thing I think
35:09one dream would be that we at at the end
35:11of the day at the end of five years or
35:13however long it takes we'll really have
35:15a multimodal AI model that would it
35:18doesn't matter whether you have the
35:20synchron device or you have I don't know
35:23some UT array recordings or you have
35:25magnetic recording or Optical recording
35:27you can really use the same model the
35:30multimodal model to extract those latent
35:33representations and those latent
35:35representations would not only give you
35:37movement but give you emotions or G give
35:40you thoughts or any kind of state you're
35:43interested in we are very far from there
35:45but once we have enough data and we
35:47think about these architectures
35:49carefully perhaps we can go uh towards
35:51those multimodal models just like text
35:53and images that we can already do here
35:56for NLP application
35:58yeah I'm looking through some of the the
36:00questions that we've submitted we've
36:01just got a minute left and and there are
36:03a few questions about ethical concerns
36:05um and I'm curious to know you know what
36:07do you say to somebody who's worried
36:10about the intersection of you know Ai
36:14and neurot Technology given that it is
36:16an invasive medical device Tom do you
36:18want to take that yeah it's really
36:20important I think there are there's a
36:22UNESCO development there's a um world
36:25economic Forum development there's a wh
36:28there's a lot of global bodies that are
36:30now thinking through um neuro ethics um
36:34in my head it's divided in in this
36:36dichotomy of on the one hand privacy
36:39concerns and on the other hand a
36:42requirement for extended autonomy so
36:44it's privacy and autonomy moving between
36:46the two I think I think what happened
36:49with Facebook and you know what we've
36:51learned with Google is that data sharing
36:53third parties like that's a breach if
36:55that happened in BCI it's not going to
36:57be acceptable so I think there's going
37:00to be a question around how data is
37:02shared um that gives consideration to
37:04protecting privacy but extending
37:06autonomy and it's not it's going to be
37:10hard great okay well we are right out of
37:14time thank you so much thank you all so
37:16much for sharing your thoughts and and
37:18your expertise so thank you all thank
37:20you
37:22[Applause]

About this transcript

This page contains the full transcript of Panel: Neurotechnology The Role of AI in Brain Computer Interfaces (2024) by MIT AI Conference, generated from the public captions YouTube serves with the video. The transcript has 6,662 words across 957 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.