YT YouTube2Text

YouTube2Text → Transcripts → Robots and the Search for Universal Intelligence | Columbia AI Summit

Robots and the Search for Universal Intelligence | Columbia AI Summit — Transcript

by Columbia Data Science Institute · 11,117 words · 1,601 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01please welcome president Dr Katrina a
0:07Armstrong so grateful to be here with
0:09all of you and I'm delighted to welcome
0:12you to the last session of Colombia's AI
0:14Summit I want to acknowledge Shifu Chang
0:17garud dangar and Janette wing for their
0:20expertise vision and Leadership and also
0:22thank everybody all the Columbia
0:24Scholars and researchers who have
0:26participated in this extraordinary event
0:29I ALS o want to really welcome our
0:31distinguished guest Sami Haden who's
0:34come from Munich to deliver the keynote
0:37address finally I just want to thank
0:39everybody who's worked so hard to put
0:41this together students faculty staff I
0:44know it takes a village and I'm
0:46incredibly grateful I think of course as
0:49we know we've come to discuss artificial
0:52intelligence but I will say that for me
0:55we've also really recognized the power I
0:58think of University or Colombia
1:01intelligence so as I travel around this
1:04campus and I write I think so often to
1:07all of you so much of what matters to us
1:11is our mission as a
1:13university I don't think I have to share
1:16with anybody here that we are in the
1:19Epic Center of what can feel like a
1:21seismic shift in how the world sees our
1:25universities and our approach to higher
1:28education it can feel sometimes like so
1:31many things are happening at once all of
1:34them of existential importance but I
1:38think for all of us even as we rely on
1:40AI to bring data together to help us
1:44understand a new way of seeing the world
1:47I think for all of us as we navigate
1:49these times we also have to stay
1:51grounded in who we are as people for me
1:55that means staying grounded and being a
1:57physician and a teacher being able to
2:00look at the things in front of us and
2:02understand honestly what we can do
2:05better focus on how we can fix things
2:08and move forward make decisions that are
2:11essential for our progress but never
2:14ever take our eye off of what we mean to
2:18society to our students to our patients
2:22and to all of those who benefit from the
2:24remarkable work that goes on here I know
2:28that this requires courage of all of us
2:31at this moment and I hope that this day
2:34these
2:35experiences have helped you see the
2:38incredible opportunity the incredible
2:41privilege that we have to be here and be
2:44part of this work I cannot imagine a
2:48better example of Colombia's unique
2:51ability to contribute to society than
2:54these sessions and these discussions
2:57today I know that critically important
2:59important questions are likely to come
3:01of this how should we think about guard
3:04rails for public consumption of artistic
3:07production and news content generated by
3:10AI how do we generate a coherent
3:13framework for investments in the
3:15computing power that's needed for us to
3:18really unleash ai's full promise without
3:22exacerbating the climate change we see
3:25around us what will this mean for our
3:28labor markets for our students here
3:31today and what they will do to lead
3:33fulfilling and meaningful lives how can
3:36we ensure that we main focused on safety
3:39and ethics but as I stand here and think
3:43about the importance of those questions
3:45it is also deeply reassuring me to know
3:48that there is no institution better
3:51poised to work on those questions to
3:54move us forward to provide light and
3:57midst Darkness than this great great
4:00great
4:01University we have people from
4:04nanotechnologists just explaining
4:06robotics ins surgeons to leaders in
4:10architecture and urban planning
4:12policymaking International Affairs
4:15Social Work legal Scholars there is no
4:18other university like this and so I am
4:21so grateful to be able to be part of
4:23this both for the individuals who are
4:25coming together but particularly for the
4:28collective good that that we can all do
4:31as we take on these challenges I'm
4:33excited to join you in this journey and
4:36thank you again for being here thank you
4:38very
4:39[Applause]
4:47much thank you president
4:50Armstrong uh good afternoon good evening
4:53everyone I'm shiuan of Columbia
4:55engineering school it's really my
4:58privilege my honor here to introduce our
5:00Kino speaker at this first AI Summit at
5:04Columbia University on this beautiful
5:06campus this event has been the
5:08combination of our Collective effort to
5:11bring together all the amazing activity
5:14in the area of AI across the whole
5:16university and you have seen many
5:18exciting session today on three campuses
5:21here
5:22today as you know Columbia AI initiative
5:26is centered around this unique approach
5:28called AI Plus X it highlights our cross
5:32disciplinary breadth and depth in so
5:35many different field engineering science
5:38Humanity medicine business public health
5:42law policy architectures journalism Arts
5:45unit every discipline you can find on
5:48campus in Academia our goal is to
5:51leverage our Collective Strength to
5:54advance our core mission of the
5:57University Research education and
6:00Innovation I hope today you have been
6:02able to enjoy many exciting session in
6:05all this wonderful Dimension research
6:07education and innovation in the space of
6:10AI today I'm very lucky to have the one
6:14of the world leading Pioneer and really
6:18sour leader in robotics and embody AI
6:22with us and I'm sure uh our speaker will
6:25explain what embod AI is and why it's so
6:28important Samy has been conducting
6:30groundbreaking research in this area of
6:33EMB body AI which many of us are called
6:35the next Frontier of AI this area of
6:39embod AI is an area focused on building
6:43intelligence systems that interact with
6:45physical environments no matter what
6:47scale you name it molecular level Atomic
6:51level atmosphere level and underwater in
6:55the space everywhere physical
6:57environment and this system that engage
7:00learning and adapting semi Invasion that
7:03an embod AI system can work along humans
7:06not to replace human is to help human to
7:09expand and extend our ability
7:12particularly our ability to solve the
7:15real world problems he holds degree in
7:18electrical
7:19engineer engineering computer science
7:21and technology management from tum
7:25technolog Technical University of Munich
7:27in Germany he received his doctor degree
7:30from
7:31rwth aan University join Godfrey willham
7:35lepes University of Anover and then
7:38become professor and director of the
7:40center of digital technology and
7:42management in
7:44tum he has founded several startup
7:47company and very very welln and
7:49co-founded a German robotic company
7:51called franka Amica GM I'm not sure
7:54that's the right pronunciation is so
7:57well known his invention of the panda
7:59robotic arm was named one of the Time
8:01Magazine 50 best invention in
8:052018 and was featured in National
8:09Geographic his patent for the tactile
8:12robot is part of the Milestone met in
8:14Germany Connection in German patent and
8:17trademark office Sami has also served in
8:20many important initiative and also on
8:23Expert panel around AI in 2020 he was
8:26appointed chairman of the Bavarian AI
8:29Council for his work he has received
8:31many distinguished award including godri
8:35willham liance prize and with his
8:38colleagues the German future prize he
8:41was elected to German National Academy
8:43of Science leap Dina in
8:4520121 without further Ado please join me
8:48in welcoming Samy
8:56hin thank you so much din Chang for the
8:59the kind introduction and president
9:02Katrina for Armstrong for for um doing
9:05such an amazing um job here at Colombia
9:08for interdisciplinary AI research so not
9:12only technology but really AI that that
9:15um kind of does the right thing for
9:19society I think it's very inspiring to
9:21think about the use of of AI and I try
9:24in my uh keynote address to talk about
9:27um not only robots and the search for
9:29Universal intelligence but what embodied
9:30AI means and how can we really benefit
9:33from it but also why is there such a
9:35challenging Endeavor that lies ahead of
9:37us so I think it is safe to say that one
9:41of the most pivotal events um in 2016
9:45took place where leol was beaten by um
9:48Google deep mines um um AI engine in go
9:53similarly to uh almost
9:56um 20 years before not not quite Gary
9:59kaspar of my childhood um kind of uh
10:02star and and uh um in in chess against
10:06deep blue from IBM so we see that many
10:09of the AI um advances over the last
10:12years have really um marked superhuman
10:15performance as we hear a lot but I think
10:17what is also really clear is that the
10:20year 2023 especially January marked kind
10:22of a pivotal moment because jgpt opened
10:25up to the public and we could actually
10:27use generative AI
10:29for the first time widespread in in
10:32society it was quite amazing what people
10:34have been using it for but in particular
10:36what kick started this this um this uh
10:39January event in 2023 at the same time
10:42uh it's also well known that AI has
10:44really become um widely adopted as a new
10:47scientific discipline so with 2024
10:49physics Nobel Prize being awarded to jof
10:52Hinton and John hopfield for
10:54groundbreaking um contributions to to
10:57artificial intelligence in particular um
10:59neural
11:00networks um at the same time I think
11:02it's it's very clear that we still have
11:05a long way to go so if you think of AI
11:07to be basically um centered around the
11:10two lines of self-awareness and
11:12embodiment then it is pretty clear that
11:15the things we see nowadays are still
11:17largely happening in the either digital
11:19space or using offline um physical data
11:23for example for the alpha fold mechanism
11:26or other Foundation models um that that
11:28are used for prediction um in various
11:31applications however if you really want
11:33to become um or to to to build AI
11:36systems of very high self-awareness and
11:39acting in the physical world and the
11:41Holy Grail of a complete embodied AI
11:44system we recently heard CH Nang saying
11:46physical AI um I think in the basically
11:50being the same thing so an AI system
11:53that has a physical body that is
11:54interacting with the physical world and
11:56is able to learn adapt and interact um
11:59with this world including the humans
12:01living within it oh thank you so
12:09much was a long
12:12flight so what does embodiment mean
12:15similar to the human here the famous
12:17pulus depiction um we as uh as
12:20roboticist build not only the systems
12:22the morphology of the technology but in
12:24particular we also build the brain the
12:26central nervous system so the control
12:28system that is able to give the
12:30embodiment the right ability to interact
12:32and Inter and act in the physical world
12:35so this kind of uh co-evolution of the
12:37intelligence and embodiment is what
12:39makes these systems so complex because
12:41they actually as the question how can um
12:44synthetic systems interact in the
12:46physical world in a world that is hardly
12:48predictable and in particular is largely
12:51unknown to the physical agent we call a
12:53robot um so in this today's talk I going
12:56to talk about two major points the
12:58second one being much smaller but I
13:00think giving a little bit of an Outlook
13:01of um where embodied AI can take us in
13:04the larger scale of of the discovery
13:06autonomous knowledge Discovery but first
13:08of all I want to talk about um the
13:10building of the embodied AI system so
13:12the robot itself for roboticist embodied
13:14Ai and Robotics are very similar to each
13:16other so we have a hard time of making a
13:18clear distinction therefore bear with me
13:21of using robots and embodied AI almost
13:23as interchangeable words because for the
13:26fundamental um approaches there is very
13:28little difference uh in fact so we have
13:30seen a dramatic increase in robot
13:32deployment over the last years um it's
13:35it's clearly uh exponential especially
13:37last two years we have seen even um
13:39humanoid companies coming out of nowhere
13:42so we see humanoid robots now all over
13:43the place so there's exponential growth
13:45in the embodiment but I will argue that
13:47there is still a lot of um um things to
13:49do before these embodiments really
13:51become truly smart and truly intelligent
13:53to physically interact in our physical
13:56world but what does an embodied AI
13:58system need first first of all it needs
13:59to understand it has a body so this
14:01robot you see is basically born without
14:04a program it just has its sensor it has
14:06the ability to move and what you see
14:08here is a robot that from scratch is
14:09able to experience its own sensory
14:12perception without knowing where along
14:14its body these sensors are located but
14:16it builds its own mental map of its um
14:19kind of spatial Dimensions so what we
14:21call the the self essentially so a k
14:24kinematic map of its own limbs of the
14:26motions and this purely based on machine
14:29learning running on the on the real-time
14:32data that you see in the system being
14:34used to then build a very accurate map
14:36with closed eyes so based on internal
14:38perception alone so basically you
14:40closing your eyes moving around and
14:42understanding the spatial Dimension that
14:44you have and the movement that you can
14:46execute with that so this um um kind of
14:49understanding of the uh embodiment
14:52itself then interacting with the with
14:54the um physical world is the fundamental
14:56first step in building a truly embodied
14:58AI
14:59um obviously we want to build systems
15:01and uh and embodied AI system that are
15:04able to regardless of the topology
15:05regardless of of the morphology even to
15:07build their own mental map of their
15:09existence so A system that from scratch
15:11is able to essentially without so
15:14bootstrapping the problem of creating
15:16the self um independent of the
15:18morphology whether it's a humanoid a
15:19crawler or any other topology of
15:21morphology of robotic um existence so
15:24this obviously is a very fundamental um
15:26question to answer and um has been been
15:29quite a quite a problem for many years
15:32um at the same time robotics goes back
15:33quite some time but I think it is safe
15:36to say that many of the things I going
15:37to talk about today are not really kind
15:39of related to these kind of robots so
15:41this is actually the first industrial
15:42robot that was entering a physical
15:44reality or industrial reality in the
15:461950s the unimate system um as all of
15:49you can see this is a robot that is very
15:52heavy very dangerous and obviously none
15:54of us wants to interact with that system
15:56so I think it's clear that this would
15:59not be the rosie um or um the the Johnny
16:02that we would like to see in our um
16:05kitchen environment probably lots of
16:06things would basically break especially
16:09humans around operating them so we want
16:12to obviously have different uh systems
16:13to really realize that kind of um future
16:16so where physical interaction between
16:18the human and the robot are as safe and
16:21intuitive as they can get obviously not
16:23only with the elderly with us but in
16:25particular with children so the robots
16:27have to be extremely safe to really be
16:29operating around us we know that um
16:31obviously in in the physical space
16:34there's very different dangers from the
16:35purely digital domain especially the
16:37physical harm that can happen due to the
16:40interaction of physical machines with
16:42the human body so this basic interaction
16:45problem has been researched for almost
16:4640 45 years in particular AMA K get
16:49herzinger Sano and others um in the US
16:52in Europe but also in Japan have been
16:54laying the foundations in the early 80s
16:57so of robots that can actually in
16:59principle physically interact with the
17:00world so the bases were there in the
17:02'90s in the 80s actually however only
17:04recently we were able to build robots
17:07that are truly able to do similar motor
17:09skill tests that seem to you and me very
17:11trivial but actually robots that are
17:13able to operate for example a key and
17:16insert it blindly with a sense of touch
17:18and compliance into a hole things that
17:20you and me can do with three or four
17:22years 5 years maybe but robots were not
17:24able to do them autonomously only until
17:27recently
17:30so coming back to the question of um
17:33what a robot really can do kind of asked
17:36also the question can we connect
17:37basically what robots can do and can we
17:39categorize that and how do we
17:40fundamentally understand that robots
17:42have certain innate abilities and how
17:45can we classify understand and make sure
17:47that for the vast skills that these
17:49robots are supposed to do in the
17:50physical real world they have to connect
17:52basically the innate ability not only
17:54the self with the possibility so that um
17:57of of app um so with the possibility of
18:00um the application that these robots can
18:02can execute so he basically asked the
18:04question of an embodiment system how can
18:06we categorize these robots and their
18:08morphology and capability in a sense
18:10that we can then automatically reason
18:12about them and understand given that I
18:14can see a robot what can that robot
18:15truly do so we took some inspiration
18:18from charvin uh Charles Darin's Tree of
18:20Life to connect basically the capability
18:23of a robot with its morphology to then
18:25fundamentally classify um similar to the
18:27Darwin uh um sorry um birds that kind of
18:33connect the capability of the system and
18:35the morphology meaning that the body has
18:37a certain functionality timed and and
18:39tied together with it with its
18:41intelligence so the algorithms of
18:42learning and adaptation to then
18:44fundamentally do a classification that
18:46gives you the answer to what can that
18:48system really do and what can it learn
18:50and what can it execute so coming from
18:53the perspective of the machine we
18:54introduced recently the tree of robots
18:56that basically gives a fundamental
18:58classification attempt to understand not
19:01only the morphology of the system
19:02ranging from its scale to the technology
19:05behind it to the um kind of
19:07morphological topology of the system
19:10that is ranging from drones to to
19:13humanoids to to manipulators but in
19:15particular measure fundamentally their
19:17capability which is important to then
19:19really run datadriven algorithms on
19:21these systems so we for example
19:23introduced the first time a fundamental
19:26Fitness um classification similar to
19:29what we know from from The evolutionary
19:30theory that actually automatically now
19:33classifies what the abilities behind
19:36robots are so typically we see a robot
19:38we think we know what they can do but
19:39fundamentally their capability is not
19:41yet measurable so this kind of
19:43fundamental understanding of their
19:46innate ability has now been understood
19:49and can actually build the fundamental
19:51for this tree of robot that then can
19:53serve kind of the Encyclopedia of skills
19:56of arbitrary morphologies and robots
19:59and if these robots can do all these
20:01things that I we are promising so they
20:03can interact with us they can help us at
20:05home they can help us in in U at work in
20:09helping us to to stay at home also for
20:11for elderly uh people um then obviously
20:14we also need to ensure that these
20:16systems are fundamentally safe I
20:17mentioned that already and someone who
20:19foresaw that very early on was Isaac
20:21azimov who in 54 was proposing his
20:24famous three laws of robots and the
20:26first one was a robot may not injure a
20:28human being or through inaction allowing
20:29human being to come to harm and what is
20:32the reality that we see today is this
20:34actually already there or what where are
20:36we and what do we have to do just
20:38recently we have seen a humanoid robot
20:40um almost giving a headbutt to an um
20:44person in in an incident in China and
20:47obviously these kind of situations are
20:49very very dangerous so this system is
20:51very fast very dangerous it could have
20:53caused a lot of harm so we were lucky in
20:55in in some form that nothing happened
20:57there so this reminded me to my own PhD
20:59thesis that asked the question can we
21:01actually build safe robots and how can
21:03we approach azo's first law and as a
21:06true scientist we like to do self-
21:10experiments so this was the first robot
21:12that was actually able to become um and
21:16I sometimes do the joke that this was
21:18actually my evil twin brother and
21:20obviously yeah he's not with us anymore
21:23um but in reality this kind of laid the
21:26foundation for physical interaction in
21:27the real world so actually now we see an
21:29entire field we call human robot
21:30collaboration cobots tactile robots so
21:33robots that can physically interact with
21:35humans have become reality because of
21:37these basic experiments because we
21:38showed that you can build lightweight
21:40robots that are intrinsically safe for
21:43the interaction with humans and in fact
21:45these robots are so safe that they can
21:47detect and react to collisions and
21:48contacts with the environment blindly at
21:50milliseconds so thousand times a second
21:52these systems can monitor their
21:54interaction with the with the world and
21:55here you can see how the system is so
21:57gentle that even even with a nail at the
21:59end effect of the robot is able to stop
22:01without um destroying the balloon in
22:03this example a few years before I again
22:06laid the foundation for these kind of
22:07reflex senses so the sense of touch of
22:09robots again in another self experiment
22:12so I wanted to obviously build robots
22:14that are able to help us in the kitchen
22:16and we wanted to see whether our
22:18Collision Handling Systems are so safe
22:20that nothing can happen in these um
22:23experiments so you can see in a way my
22:26thesis was a bit um different
22:32still next question is obviously now
22:35that we have robots that that uh in in
22:37particular is safe and interactive but
22:38can we actually make them do things such
22:40as um our uh you know this is actually
22:43my daughter when she was about four I
22:45always do these experiments with my kids
22:47so they they tend to be part of my
22:49science and back then I wanted to
22:51understand can I make robots actually
22:53learn to interact with the World by
22:55solving the key into whole problem um
22:58and uh you know she was making a big
23:01mess in our living room so I took the
23:03opportunity and said uh and her name is
23:06franka by the way so she's the the name
23:08that that gave the name to the company
23:10um and and back then I I kind of asked
23:12her so fanka if if you do such a mess in
23:15our living room let's do an experiment
23:17with you so here's a key go to the door
23:19and and see whether you can open it in
23:20less than 10 trials if you if you
23:22succeed then you don't have to clean
23:25up she ran as quick as you could I mean
23:27you can imagine she ran to the door and
23:30in particular I was kind of wondering
23:31whether it's you know is it too simple
23:33to do it in one trial or does it take
23:35forever she did it in s seven trials so
23:38obviously not a big data problem so
23:39somehow in her body there was this
23:42innate intelligence that made her able
23:45to um insert the key into the hole in
23:47seven trials so very few trials until
23:49she was actually able to do it so and
23:52feel fast forward to today if we now use
23:55just purely digital AI so standard
23:56reinforcement learning standard deep
23:58reinfor enforcement learning um which we
24:00see on the right side and kind of
24:02compare it with embodied AI systems that
24:04are able to understand basically the
24:06physical environment and what does it
24:08require for a robot to actually insert
24:11an object into um into another object
24:13and made them um and in particular under
24:16the under the understanding that the
24:17robot is able to kind of understand
24:19whatever that means in a technical sense
24:21that it is embodied so the interaction
24:24is is more than just a purely digital
24:26Pro event then we can see that there's a
24:29dramatic increase in performance in fact
24:30it's orders of magnitude on the left
24:33side you see how in very few trials so
24:35similar to my little daughter after a
24:36few trials this robot is nowadays able
24:39within five six trials sometimes 1
24:41minute 2 minutes um able to converge
24:44even optimally and and insert the key
24:46into the hole on the right side you see
24:48a pure Brute Force attack end to endend
24:50deep reinforcement learning actually
24:52takes sometimes weeks until we find
24:54solutions for the first time so this
24:56really un kind of I think under lines
24:58the notion that physical existence
25:00embodiment are really important to be
25:02built into the AI system so that we can
25:04really make them um perform in in in a
25:07manner that really achieves human level
25:09performance not only in cognitive tasks
25:11in chess playing and and and go but in
25:13fact in physical tests that seem trivial
25:15to you and me but they are not trivial
25:17for robots and embodied AI systems so
25:20and in uh what we call Tech manipulation
25:22there are so many things that have not
25:24been yet addressed so we need to kind of
25:26bootstrap that whole problem because
25:28it's infinitely many problems we still
25:29need to solve only very few things have
25:32been automated by robotic systems and
25:34how can we really uh do that what we
25:36have been introducing recently is kind
25:38of a structural machine learning
25:39approach that tries to essentially um
25:43understand what um Teti interaction
25:45really is coming from the question of um
25:47taxonomic understanding um in order to
25:50to manipulate something you need to
25:52essentially understand what kind of
25:54interaction happens between the embodied
25:56AI system and the physical object it's
25:58man ulating which is basically um a
26:00power interaction so a physical system
26:02interacts with another physical
26:03interaction with another physical
26:05partner exchanging energy therefore
26:07power so you need to understand what
26:09kind of twists and wrenches so motion
26:11and forces are interacted and if you do
26:13a kind of structured analysis of that
26:15problem you can actually come up with a
26:17machine learning pipeline that
26:18fundamentally seems to be much much
26:21better suited to to tackle that embodied
26:23AI tactile manipulation problem so the
26:25interaction with the world so if you
26:27express
26:29that physics matter in the machine
26:31learning pram itself it seems that
26:33fundamentally that structure um
26:35increases the speed of the learning
26:36process dramatically compared to a pure
26:39uninformed endtoend approach so and what
26:42you can then do is in fact not only in
26:44the lab interesting things but you can
26:45solve industrial tests so this kind of
26:47AI is as of today running in real
26:50Factory uh um floors in the world so you
26:54can now automate for example recycling
26:56tests that at the moment are really a
26:57big problem in E-Waste management we
27:00cannot recycle all the electronics waste
27:02that we are producing now we have been
27:04building the first robots that are
27:05actually able to disassemble electronic
27:08waste segregate the different uh Parts
27:10into the right bins and then feed them
27:12into the recycling system you can even
27:15now do autonomous manufacturing of
27:18machines themselves so you see on the
27:19left side a very complex and for the
27:22Layman very unintuitive um manufacturing
27:26object that is being assembled on the
27:28right side you see um on the left side
27:30it's the manual assembly on the right
27:31side you see how the embodied AI system
27:33is now learning autonomously how to
27:35manifacture this very strangely looking
27:38um metal piece and just within a few
27:41trials I think it's about 20 trials the
27:43robot is able to insert and and made and
27:46assemble really this very complex
27:48assembly process something that couldn't
27:50be done until recently and with this
27:52kind of embodied AI system that are able
27:54to reach now industrial performance we
27:57can actually now built autonomously
28:00machines so this was actually a gripper
28:02for production of um these kind of U um
28:06beverage machines so very complex
28:08Machinery can now be automatically
28:10manufactured by robots at a scalable and
28:13high productivity um um kind of scale so
28:18but how does that scale fundamentally do
28:20different tasks do we now have to retach
28:21do we have to retrain every new skill
28:25that would be infinitely many it sounds
28:27very unlikely that this is possible so
28:29how can we tackle that problem what we
28:31have been looking at is can we
28:32essentially build a learning system
28:33which we call a collective learning
28:35system that's about 100 robots that are
28:37trying to learn all the skills from a
28:40standard tool set that you can buy in in
28:42a tool shop and these robots are now
28:44training and interacting um and
28:46exchanging the knowledge that you know
28:48based on this representation I was
28:49talking about so the AI system kind of
28:52exchanges in this network of Agents the
28:54performance and the way that they do
28:57different tests so each one of them is
28:58doing a different test I won't go into
29:00details which ones they are and then
29:01they raise their hand when they are done
29:03essentially and you can see instead of
29:05weeks and months of training these
29:08systems are able to sort or solve the
29:11entire usage of a toolbox that you can
29:14buy in any kind of standard commercial
29:16Tool Time uh tool shop and then use that
29:19system to then solve entirely abstract
29:22tests that have not been trained and
29:23learned before and what is most
29:25importantly is there is not a single GP
29:28involved in that system so this is all
29:30running on standard CPUs because of that
29:33um kind of the representation allows
29:35that you can actually do standard
29:36optimization algorithms instead of doing
29:39very very intensive GPU calculation so
29:42that system the whole AI system runs
29:44about at I think 1 kilow for the 100
29:46robots that's the total um energy that
29:49is required for that kind of AI all
29:52right so obviously one of the next
29:53questions you would be asking is can we
29:55extend to the cyberspace so can we
29:57actually now do these things in the
29:58physical world yeah that's great but it
30:00costs it's very intensive um it's not
30:03scalable so how can we make use of the
30:05physical simulating world and can this
30:07be transferred to the to the physical
30:09Real World what you see here now is the
30:11key into a hole in a in a recent paper
30:14we published that is really about
30:17learning in the digital space how to
30:19insert and the question can this then be
30:20transferred into the physical world we
30:22call this the Zim tooreal transfer so
30:24simulation to reality a little bit like
30:26Neo when he was learning kung fu so is
30:28the is the Matrix really um kind of
30:30close enough to the physical world so
30:32that the robot can learn in this digital
30:34space and in one example we see here we
30:37see a very abstract um robot hand
30:40learning in the physical space how to
30:42open or how to turn the crank we call
30:44that problem and turning the crank is an
30:46example how to open a door for example
30:48and what you see here is now the same AI
30:50system running not anymore just for
30:52robot arms but now for hands so very
30:55articulated systems and what we can see
30:57now is that the system is training in
30:59various um different scenarios in the in
31:02the digital world and what we were able
31:05to achieve is with this AI system now a
31:08direct transfer to the physical world so
31:10the system converges in the physical
31:12world and with direct transfer is what
31:14you see now the robot has never touched
31:16any door in its life and immediately is
31:19able to grasp a door open uh um or even
31:22adapt to this huge Arrow you just saw
31:24that was about 7 cm and is opening the
31:26door without having ever ever trained
31:28with a physical door so now the next
31:30step is obviously how do you feed that
31:32back into the digital simulation that
31:34you can basically bootstrap or Kickstart
31:37that kind of evolution between the Cyber
31:39and physical embodied AI so kind of the
31:41embodied AI scales now into the digital
31:43world and vice versa which I think is
31:45kind of hopefully giving a bit Food For
31:47Thought especially if we think about
31:48highly networked systems that could
31:50learn from each other and distribute
31:51knowledge almost like in network
31:53Cascades all right so this is now even
31:56not only possible for very basic
31:58insertion prompts but we can now do
32:00single shot transfer between various
32:02scenarios and I think you have seen also
32:04various of the videos that are out in
32:05the in the wild at the moment of robots
32:07doing all kinds of things in the in the
32:09uh in the digital in the physical space
32:12um which I think gives also rise and
32:15then hopefully some kudos to kind of
32:17what made that possible so similar to
32:19the let's say um gpus in in in re in
32:23deep learning we had finally a platform
32:26that was able to um scale our Global
32:29efforts in robotics learning which was
32:31the Franco robot that has been becoming
32:33kind of the state-of-the-art system for
32:35all kinds of machine learning approaches
32:36in the world and this is the the the
32:38company that I founded many years ago
32:40been out for many years as well so but
32:42nowadays it's kind of the standard
32:44approach and standard platform
32:45especially for AI and ml research in the
32:47world so if you see what Publications
32:49are out there they basically all are due
32:52to the open platform the highight data
32:54um interfaces and the kind of
32:56state-ofthe-art
32:57A system that that now the world has
32:59access to um so I really think that this
33:02kind of combination of of physical
33:04platforms that that are open that scale
33:06but also that combine um the right
33:09state-ofthe-art um have have been really
33:11important to robotics which kind of
33:13underlines that embodied AI cannot just
33:16be solved by pure digital aspect so we
33:19actually need Hardware platforms that do
33:21what we need so we need our own gpus you
33:23could say so and as already said what
33:26was really fantastic for us was in
33:29Germany to be recognized all you know in
33:31the in the um over the ocean so I was
33:34kind of um it was interesting that even
33:36in 2024 there's a New York startup
33:39called escape and they use that robot
33:41now for for um massage robots that was
33:45kind of interesting and again became a
33:47Time invention of the year so it's the
33:48same robot but in a different clothing
33:50you could say all right and what was
33:52actually really cool to see was that the
33:54entire world is now using these robots
33:56at all the big labs and the world to to
33:58kind of really make robots learn to
34:00interact to manipulate so it's kind of
34:02an extension of what I could do in my
34:04own lab but now actually the entire
34:05world is kind of you know being part of
34:07that initiative and and using these
34:09systems to create I think a fantastic
34:11kind of Wikipedia of skills you could
34:13say that then ultimately hopefully feeds
34:16back into the tree of robots that we
34:17building that then all robots could
34:19basically be kind of feeding into so but
34:23now I was talking about one robot
34:24embodiment what about all the other
34:25robot embodiments there's so many new
34:27ones there's countless ones in the
34:29future so what can we do with all the
34:31humanoids that are happening now I mean
34:33are we always needing 10 20 years until
34:35new robots are there so how can we
34:37systematize that process it's really
34:38hard to build a robot it costs a lot of
34:40money and it's very hard to to scale
34:43that that thing and now we even have
34:44robot Rockets right I mean what I'm
34:46seeing as robotic is basically a
34:48gigantic robot hand catching a rocket
34:50right I hope all of you see that as well
34:52so basically I see robotics problems all
34:53over the place um and however the Quest
34:56for this fun fundamental embodied AI has
34:58kind of fundamental problems as well in
35:01particular the AI that we see today is
35:03very far from performance let alone
35:05optimality so often these kind of things
35:08we see there they come from very very
35:10deep understanding of Dynamics control
35:12optimization it's not straightforward to
35:14generate data so we actually have a
35:16really a fundamental problem in the in
35:18the data access to to these systems and
35:21then also there's the fundamental
35:22question how can we systematically
35:24design a system that is able to do what
35:26we want it to do typically we build
35:28robots and then we see what they can do
35:30so that's not a very structured approach
35:32so we cannot really um you know with
35:34with algorithms we can do so many things
35:36that these robots were never designed
35:38for but can we foresee that can we
35:40actually now fundamentally build designs
35:42that do what we actually want them to do
35:45so the um the actual challenge number
35:47one is data scarcity as an interaction
35:49generates data and AI drives interaction
35:51so it's a chicken and neck problem we
35:53don't have the internet readily
35:54available the robots have to generate
35:56the data to have skills but they need
35:58skills to generate the data so how can
36:00we solve that problem and then second is
36:03the synthesis of the machines um so
36:06which is fundamentally asking the
36:08question can we build a robot compiler
36:09and a make engine that basically
36:11synthesize automatically machine based
36:13on specification so and I think it's
36:16really important to to state that this
36:17is not a a solution will not just come
36:19from um fundamental modelbased
36:22approaches uh design or AI itself I
36:25think it's really important that all of
36:26them work together so fundamentally the
36:29answer that I would propose is is deeply
36:30rooted in physics and and and models
36:33machine learning and optimality and how
36:35can we bring these these fields together
36:37so what we have been working on is what
36:39we call the optimal motion framework so
36:41fundamentally A system that is able to
36:44use very costly generated motions that
36:47typically come from optimal control
36:48Frameworks I'm going to talk about this
36:49in a second and then encode them into AI
36:52system so that even the AI then can
36:53generate optimal outputs so optimal
36:56control in real time would be the
36:58optimal kind of AI You could argue and
37:00then the second one would be um an
37:02embodiment compiler and maker so
37:04basically from a phenomenology so let's
37:06say I want the robot to throw a ball to
37:08automatically synthesizing the robot
37:10that can throw a ball and hopefully in
37:12an optimal manner okay so let's go for
37:15the second one first so let's say we
37:16want to develop a robot that is throwing
37:18a ball then you can obviously have
37:21multiple embodiment so you don't see
37:22them visually the difference but I hope
37:24you can see that this one is much more
37:26doing a typical human type of throw the
37:29second one is is more doing a s swing up
37:31motion and then throwing the ball but
37:33fundamentally doing the same thing
37:35however the morphology behind them is
37:37fundamentally different so typically it
37:39would take you 10 years to build this
37:40robot 10 years to build that robot and
37:42then in the end you find out oh the
37:44second one is not doing what I want to
37:46do so can we actually revert that
37:49process so what we have been doing is
37:51really to use optimal control it's the
37:53only slide with with equations I just
37:55want to show you one thing um so what we
37:58really believe in is that the idea of
38:02physics optimality and machine learning
38:04has to be integrated why because we can
38:07formulate goals so the cost function
38:09that's the upper equation is able to
38:11give us a notion of what is
38:15quality the second one in an
38:17optimization framework is we can plug in
38:20the embodiment itself so we now have an
38:22input for hypothetical embodiments that
38:24we could kind of come up with we could
38:26use AI systems even to generate new
38:28embodiments but the optimal control
38:30framework gives us the idea this is
38:32where you can plug them in all right and
38:35then you can actually solve these
38:36problems and depending on the embodiment
38:38on the left one you have a rigid robot
38:40it does everything with brute force on
38:42the right side you see a much more
38:44gentle and much more elegant motion
38:46that's actually a robot that has
38:47intrinsic elasticity similar to humans
38:49you know the elastic elements in our
38:51body make it possible that we can store
38:53and release energy both do the same task
38:57left one with very high power the right
38:59one with more Elegance Grace and Energy
39:02Efficiency and this obviously is only
39:04due to the very different mechanisms
39:05behind it so that is great but can we
39:08systematize that I won't go into details
39:10but what we have been recently proposing
39:12is basically a way to go from a purely
39:14observation to then propos Dynamic
39:16phenomena that drive automatically
39:19cascaded um kind of proposals of
39:22different mechanisms templates and
39:23anchors that ultimately can then be
39:25synthesized as a physical robot and that
39:27actually render exactly the phen let's
39:31say the phenomenon that you want to
39:32render yeah sounds almost like a like a
39:36specification you tell the system like a
39:39almost like a search engine that can
39:40make robots hey build me the robot that
39:42does XYZ and then uh the robot is
39:44compiling and synthesizing this a
39:46student of mine is right now founding a
39:48company on on on things like that all
39:50right so what was the result right it's
39:53spectacular optimal embodiment right so
39:56this is a Fant fantastic robot it's able
39:58to so we put all that knowledge into the
40:00robot and the robot is actually not able
40:02to throw why because we as it the wrong
40:06thing because there's another way to
40:07throw which is much better so the
40:09embodiment actually shows what it can do
40:11and what it cannot do it's not just the
40:13Brute Force attack so as you and me we
40:16are built to do certain things so one
40:19solution this is the overhand throw it's
40:22really bad at but if you do the lower
40:24hand throw it does a fantastic job in
40:27fact effect and why am I saying it's a
40:29fantastic job because if I compare it to
40:31the state-of-the-art of all existing
40:32robots till today that have tried
40:34similar tasks and you normalize them by
40:37numbers of degrees of freedom that you
40:39can use and the size obviously throw is
40:41is depending on length right then you
40:44can see that there is the standard rigid
40:46robots and we have a new type of robot
40:49that really are using elastic and
40:50inertial energy transfer so this is a
40:53huge jump in performance just by
40:54embodiment so we can see if
40:57factor of three on the embodiment in
41:00performance of these systems that really
41:02shows how much the body is important to
41:05to be exploited and not just to be
41:07controlled the way and I think this is
41:09really a fundamental um statement for
41:11embodied AI so if we now go for the
41:14second Quest the optimal motion
41:15framework so now we can synthesize the
41:16robot but we need a lot of time to
41:18actually do the Motions I just showed
41:20you typically takes one or two days to
41:21generate one of these motions one or two
41:24days just for one motion so I want to
41:26throw a ball let's say 6 M I need 2 days
41:29to compute that trory throw it and then
41:32I have to wait for another 6 days
41:33depending on how accurate I want to
41:35model that so how can we get that to a
41:37millisecond and the answer is the
41:39following I come back to the optimal
41:41control this is this to be solved is
41:44quite an extensive problem especially
41:46due to its high non-convexity non
41:48linearity many constraints and so on and
41:50so forth but you can spend a lot of time
41:52of doing that so you can generate
41:53offline data so that's kind of the
41:55learning optimization this needs a lot
41:57of of time and now the wishful thinking
41:59would be can we encode that information
42:02into a beautiful machine learning system
42:06and then generalize such that it does it
42:08in real time at Optimal Performance that
42:11would be the optimum optimal AI you
42:13could generate right and we have been
42:15proposing a new method that basically
42:17uses um a specific form of of machine
42:19learning called Dynamic movement
42:20Primitives to then basically solve
42:23offline several optimization problems
42:25then encode them into the dynamic
42:27movement Primitives it's essentially
42:29latent space representation and
42:30dynamical systems and then we are able
42:32to generalize in real time and what you
42:35can then do is to compute a directory of
42:38a highly complex physical nonlinear
42:40system instead of um hours or days
42:45essentially in real time so what you see
42:46here on the right side is now the the
42:48bin is basically being moved and
42:50instantaneously the robot is throwing
42:52into the bin and we could show that the
42:54optimal manifold of the tasp SP can be
42:58approximated with these machine learning
43:00algorithms absolutely accurately so
43:02basically this means you could use
43:04optimal control theory to generate data
43:07that is optimal to then learn how to
43:09optimally execute the right skills which
43:12I think opens up a fully New Perspective
43:14on how to really mix model-based
43:16algorithms with machine learning okay so
43:19now for the last two or 3 minutes I
43:21hopefully have um I want to now go to
43:25the second topic which is much much
43:27shorter um which I want to talk about is
43:30the AI for science part so I've been
43:32talking a lot about embodied Ai and how
43:33to build robots and so on that's all
43:35beautiful but what also is interesting
43:37and interests me maybe even more is how
43:40can we use these fundamental insights
43:42into acceleration of uh knowledge
43:45Discovery itself so um basically that
43:48was the second Nobel Prize right the
43:50second Nobel Prize was given for
43:52groundbreaking contributions to protein
43:54science and um kind of accelerated by I
43:57and I think it's quite important to
43:59state that this really shows what kind
44:02of magnificent tool AI is so it has been
44:04used for prediction of of uh proteins it
44:08has been generalized to even more
44:09complex proteins so there was I think
44:11three versions of alpha fold and it
44:13showed really the power that if you have
44:14the right data if you if you do the
44:16right experiments and you choose them
44:18very carefully that the maximum Insight
44:21can be generated then you can use
44:23machine learning to redo the right
44:25prediction so the jaliz I showed you in
44:27the optimal control framework so what
44:29happens if we now could use AI in a Clos
44:32Loop fashion to even accelerate the
44:35experiment generation process not only
44:37the aftermath but actually the
44:39experiment itself and that's where we
44:41need robots right where we need embodied
44:43AI if you don't have robots and embodied
44:45AI to fundamentally
44:47scale the knowledge Discovery process
44:50this will always be a decade long effort
44:53of uh data generation and then AI kinds
44:56of kicks in so how can we you know
44:59really minimize the experimental part as
45:01well and that's kind of what I call the
45:03AI accelerate Discovery chain where we
45:05kind of want to do the experiment
45:07capabilities to automate we want to have
45:09automatic State measurements so every
45:10microscope has to become a robot
45:12essentially and then obviously learning
45:14and planning phase of the experiment to
45:16really have the full acceleration in the
45:19processes so there has been lots of work
45:21on that in the recent five six years on
45:23the automation of science mostly lab
45:26automation then some ideas of of you
45:28know doing some some next best guesses
45:30GPS for for experiment planning however
45:33I think it's still a major quest to
45:35fully automate these processes and
45:37really have a kind of a Jarvis you could
45:39say like you know the Jarvis of Iron Man
45:41I would love to have one probably in my
45:43lab so can we actually do that so let me
45:45give you an example what I can see where
45:47robotics can really make a big
45:49difference so let's say you want to
45:50interact with a cell As Natural as it
45:53gets right shaking hands with a cell how
45:55can we do that how can we use robotics
45:58AI to fundamentally make this an
46:00immersive process today you would use an
46:02atomic Force microscope you tell it
46:05basically where to go the force
46:06microscope goes there sorry here and
46:09then you know pushes on the be Cy level
46:12on the cell and then you can look at it
46:14you can look at the data that's about it
46:15right it's it's more like a passive
46:19process next thing that people would now
46:22be doing is obviously oh wait where is
46:25it is to basically built a more complex
46:28cockpit so you can do data augmentation
46:30and all kinds of things but what I was
46:32thinking is why don't we use T robotics
46:36to these kind of cases maybe some of you
46:38have heard that in surgery we use robots
46:40to do minimally invasive surgery in
46:42space robotics we use robots in on the
46:44space station or um even on on on Mars
46:48you you know interacting via Tel
46:49robotics with the operator we are using
46:52drones over kilometers we can nowadays
46:55do surgery across continents so why not
46:58using that idea to go down in scale but
47:02in in kind of in in scale not in not in
47:05distance so and this kind of idea is
47:07something we're building right now in
47:09the lab we're basically using um a
47:11heptic input device that is connected to
47:14an atomic Force microscope um that is
47:16also connected to an multi- electrode
47:18micro electrode array to then basically
47:20do an electr heptic interaction and go
47:23in a feedback loop so telepresence with
47:26individual cells so you can stimulate
47:28the cell you feel the cell so you
47:30basically use robotic technology to
47:33interact now across scales in the Clos
47:36Loop fashion and with this kind of idea
47:38you can obviously now also robotize
47:40other devices such as the hotography um
47:43system so microscope that now becomes
47:46essentially a drone so it becomes
47:48instead of just being a passive Observer
47:51the human looks into the microscope then
47:53maybe have a little bit of processing we
47:55can actually automate the entire the
47:57entire microscope to be an autonomous
47:59search and and and uh and charge system
48:02so what you see here is a fully
48:04autonomous workflow the robot microscope
48:06basically searching for the cells
48:08building the 3D structures and then
48:10automatically um detecting the right
48:12cell structures and then being able to
48:14track them to monitor them so that you
48:16can then basically build them into a
48:18fully autonomous
48:20workflow um at the same time Bion
48:22optimization has been really
48:24instrumental to do optimization from
48:27Black Box to even gray box system so you
48:29can now again I'm coming back to the
48:30physics modeling so what we see now is
48:33that we can really accelerate the next
48:35best hypothesis planning via modern
48:37machine learning algorithms um and
48:40really accelerate by fusing domain
48:43knowledge such as certain constraints in
48:44in essay optimization for example and
48:46other kind of processes that then can
48:48really um accelerate the the the the
48:50planning process and only do the
48:52experiments that promise the maximum
48:55output of of what we want to have
48:57and I think what what this really gives
48:59us is a fundamental new tool of
49:01knowledge Discovery because what we have
49:03seen so far is if you if you see the
49:05x-axis is the generality of of things
49:07the Y AIS the prediction effort what we
49:09have seen and what the Nobel Prize has
49:11been given for is a prediction of a
49:14model with very very high intensity of
49:16data so a lot of data was generated and
49:18it's a it's still a rather narrow field
49:21so I think the future really lies in
49:22fully automating the processes and
49:24really kind of use physics informed um
49:27machine learning methods together with
49:30um the right experiment planning to
49:33really build kind of more and more aarto
49:35front in terms of the generality and
49:37prediction efforts that we can see and I
49:39think what is really the the the promise
49:41that will that I think can really scale
49:44lab sciences and other experimental
49:46Sciences is that the smarter the
49:47machines get the faster we will actually
49:50see the right experiment to be to be
49:52done and I think that really this idea
49:55of an AI Accel Discovery system and you
49:57see here is a fully automated robotic
50:00laboratory we've been building that is
50:01doing all the standard lab uh LAB Works
50:04and only with two students we can now
50:06run an entire laboratory in a fashion
50:09that I would say usually would need a
50:10100 people to run that in manual work so
50:13you can fundamentally scale science by
50:16embodied AI systems you can use it for
50:18synthesis processes you can do it for
50:20you know I mean carting charting um um
50:24and and navigating through through um
50:27Cellular Systems you can use it for the
50:28learning and planning but I think most
50:30importantly it is the first blueprint to
50:33actually think about how can we
50:35fundamentally automate knowledge
50:37Discovery if not even the scientific
50:39method itself we see that many many
50:41methods nowadays solve partial problems
50:44of the scientific method right from the
50:46hypothesis generation to the selection
50:49to the experiment planning I was talking
50:51about the parameterization and then
50:52execution but also in the evaluation and
50:55testing I think we can see many many
50:57contributions at the moment and I think
50:59this brings me to the conclusion of
51:00today so embodied AI is not only
51:03robotics is not only the Quest for
51:05Universal intelligence but I think it's
51:07even the automation of the scientific
51:09method itself that fundamentally can
51:11help us to correct the key to Universal
51:14intelligence thank you very
51:16[Applause]
51:23much thank you
51:26all right really fascinating and I'm
51:30sure we have a lot of question we want
51:31to expand and then to push the envelope
51:34to understand more let me just take a
51:35remaining six minutes to just uh extend
51:38a little bit on uh some Dimension so one
51:41of the last piece you mention on this
51:43Universal knowledge Discovery is using
51:46the AI or embody AI to assist human
51:50right in this very intellectual process
51:52of knowledge Discovery so make me wonder
51:57in this intellectual process what is the
51:59best role human can
52:01play in the scientific or in general
52:04Universal knowledge
52:07Discovery I mean in a way I think the
52:09human will always be the first one
52:11asking the right question so there's the
52:13creative process of of right asking the
52:15question I think I will always be the
52:17Executioner you could say and um also
52:20where especially in the scientific
52:23discovery problem um the setting
52:26boundaries setting you know intangible
52:28knowledge parameterizing the AI to to
52:32use domain knowledge will always remain
52:34um the human uh scientist's
52:37responsibility and at the same time I
52:39think the human scientist might at some
52:42point
52:43become you know he's spending more time
52:46on actually on the core science of
52:48things than to spend the time in the lab
52:52like I used lots of my time to to build
52:56Rob was to program them to iterate I
53:00feel that with these kind of tools we're
53:03not writing binary code anymore we will
53:05write
53:06basic an analogy as a computer scientist
53:10so we will not write anymore on Machine
53:11level but we will write on abstract um
53:13kind of languages and um I think with
53:16this we can paralyze the the scientific
53:19discovery process itself for us as
53:20scientist um DaVinci died with a long um
53:24to-do list and I think we can reduce the
53:26to-do list a little bit if you can
53:28paralyze our scientific question it's
53:29amazing right and it's you can reduce
53:32the tool that we human have to learn but
53:34then you can focus on maybe higher level
53:37hypothesis
53:39formulation right or maybe
53:41counterfactual reasoning to see if I
53:43don't do this then if what will be the
53:44outcome yeah so you show very
53:47interesting example that you work with
53:50your daughter on to test human
53:53capability of putting the key in the
53:55hole so may me wonder that in this AI
54:00assisted assisted scientific
54:04process we have a lot of students here
54:07in the audience also and I'm sure they
54:09know they all know how to put key in the
54:11hole I'm sure and but then the question
54:13in this AI assisted Paradigm what would
54:16be the skill and the knowledge set in
54:20the era of this embodi Ai No matter they
54:23are scientist or they user they domain
54:26what is the best way they can prepare
54:28themsel in this era of embod AI not only
54:32digital AI
54:34right um I mean it depends it's a
54:37question that probably is not a single
54:39answer um I think it's it's really
54:42important to to be either prepared to
54:47become a developer of embodied AI or do
54:50you become a user of embodied AI so a
54:52domain embodied AI expert I feel that if
54:55you for example let's say healthcare
54:58workers or even medical doctors I think
55:01they have so much to benefit from these
55:04Technologies and training them early on
55:07to be able to not only use the current
55:09state of the art but also give their
55:11active input into what exactly the right
55:13technology is we should be developing
55:15should be a core competency because it
55:17typically takes a long time to bring
55:20these two Fields together until the
55:22right solution can be created my
55:24experience has been that um it took it
55:26takes often a year or two until you
55:28fundamentally speak the same language
55:31and if if you know in in the education
55:33of our young talents we can prepare them
55:37to be
55:38early not only adopters but but
55:41communicators and and and and discussion
55:43partners that can really guide that
55:45process much faster that would be a
55:46fantastic um ability that I would have
55:48loved to have earlier on to yeah that's
55:51the first one and if you think about um
55:53being a developer I think it's really
55:55important to first of all the current AI
55:59doesn't solve the embodied AI problem
56:02it's really very very hard problems
56:04still ahead of us that um require a lot
56:08of deep mathematical knowledge lots of
56:10engineering knowledge but also the
56:13bridge to the data science so I think
56:15that's the so really being open to that
56:17what I see at the moment is a lot of
56:19dogmatism I think it's really the future
56:21will be more in in integrating these
56:23approaches and be prepared to that
56:25that's I think the two main take home
56:28messages I can give so if we extend from
56:31that concept of integration or combine
56:36uh expertise domain uh Knowledge from
56:39different discipline right as I
56:40mentioned at the introduction Columbia
56:43AI unique concept is called AI plus X so
56:47we are very proud that we're very active
56:49in advocating this cross disciplinary uh
56:52collaboration across all different area
56:55of AI
56:57so what in your view what is the best
56:59way to conduct cross disciplinary AI
57:02particularly in the embodied physical AI
57:04space for University of Colombia I mean
57:07to actually solve real problems that you
57:09that that have meaning I think one of
57:11the main problems I mean think about
57:14environment think about sustainability
57:16or health there's like real problems
57:18that AI can can help solve so if you
57:21think about um you have a wonderful
57:24colleague Sunil aaral Rehabilitation
57:26robotics expert in Colombia he has his
57:29lab in in the Colombia hospital so I
57:32think this is the way that these things
57:33can be done this is unique I I don't
57:35think uh there is many places in the
57:37world where you can have this kind of
57:38intense handshake and this is why so
57:40many of the things he has been
57:42developing translated into into clinical
57:45reality and I think this statement is
57:48probably true for many other including
57:50Financial sectors even so um I think
57:53this is uh the unique opportunity that I
57:56see here in Colombia that it's a you
57:58know one of the world's most prestigious
58:01universities but
58:03also from technology to Natural Sciences
58:06to social sciences to financials
58:10economics I think there's there's all
58:12this unique Pockets that can connect
58:16with each other and really you know from
58:18the domain to the solution to really
58:20build systems I truly think that many of
58:23the of the you know many of my
58:25fundamental more fundamental insights
58:28typically came from solving real
58:30problems let me give you an example um
58:32just today in the morning while I was
58:34actually working on my talk which I by
58:35the way never gave before so I really
58:37tried to to do a few things that I
58:39didn't talk about before um we were
58:41submitting two papers one was um very
58:45theoretical paper about solving optimal
58:48control for nonlinear systems very hard
58:51to explain why this is important but it
58:53solves a lot of problems for finance for
58:56robotics for neurom mot control for you
58:58know many many systems but how did we
59:01come up with that Solution by asking the
59:03following question in a rehabilitation
59:05um scenario you have a robot that is
59:07supposed to move your arm and this robot
59:10is supposed to to isolate the right
59:12muscle to do the right rehab motion and
59:15from asking the question from the motion
59:18to the right muscle we found out that
59:20there was no mathematical solution to
59:22that and this led to 5 years of basic
59:25research so one fundamental paper that
59:27was basically Rehabilitation neuromotor
59:30control and the other one really
59:32optimization and optimal control Applied
59:33Mathematics basically I think this is uh
59:36this is wonderful about our fantastic
59:39and it's really foundational Theory can
59:41impact all different application exactly
59:43so you mention but the but the question
59:45was really not coming from the theory
59:47that's my point it was I had no question
59:49on the theoretical side first and I
59:51think this was
59:53uh I don't know that has been my
59:55experience usually are it's really
59:57amazing you are able to abstract down to
59:59a common Foundation right optimal
1:00:02control or optimal embodiment so that
1:00:04you can quickly real in real time decide
1:00:06what is the best design exactly so if I
1:00:08extend that question further right you
1:00:10mentioned Rehabilitation collaboration
1:00:12between science engineering and Medicine
1:00:15we at Colombia has so many session as
1:00:17you see today in finance in creactivity
1:00:20in policy so if they ask you to pick one
1:00:23area in the next five years years or 10
1:00:26years and AI is moving so fast here what
1:00:30is the most exciting application through
1:00:33this cross diser application you a pick
1:00:36for the next 10 years for my own for my
1:00:38own sake I would probably uh um I would
1:00:42put pick the Natural Sciences because I
1:00:44think the Natural Sciences at Columbia
1:00:46are quite amazing so there's there's
1:00:48really leaders in many fields and um I
1:00:51see them to to I mean I had brilliant
1:00:53discussions for example with Jim H I
1:00:56enjoyed this a lot and uh and he has
1:00:58also this this idea of how can you use
1:01:01Ai and Robotics for for his own material
1:01:04synthesis and finding new new materials
1:01:07I think this is intriguing and I'm sorry
1:01:09to say that for all the other fields
1:01:11which are probably equally interesting
1:01:14but there are so many things you could
1:01:15be doing I I think there's a a great
1:01:18great Focus here about health um mental
1:01:22health is also one of the big Focus
1:01:23areas I learned in Colombia um I think
1:01:27it's quite quite spectacular what what
1:01:29could be done there but um in the end
1:01:32what what drives me at the moment is
1:01:33really this basic science question yeah
1:01:36and um and I'm I'm a bit of in in my
1:01:39private life I'm I'm doing a lot of Arts
1:01:41and Music so I would love to be engaged
1:01:44there yeah we have a session today about
1:01:46Ai and arts and creativity also that
1:01:48sounds fantastic and Jim Hong you
1:01:50mentioned is a faculty member in the
1:01:52mechanical engineering but he has really
1:01:54benefited a lot through the cross
1:01:56disciplinary collaboration between
1:01:57engineering and natural science in
1:02:00Material Science and Quantum in Nano so
1:02:03that's one of the very promising area
1:02:05that certainly embody AI will be very
1:02:08absolutely absolutely and also with the
1:02:09data Science Institute I think you have
1:02:11a pretty amazing institution that that
1:02:13really goes across many many disciplines
1:02:16and um I think this is I mean also the
1:02:19Neuroscience uh in in Colombia is quite
1:02:22quite amazing to to work with I'm sure
1:02:25and and I mean embodied AI Neuroscience
1:02:27also have very very intense relation
1:02:29with each other so I think um we can we
1:02:33can probably continue for the next two
1:02:34hours I do have two or three other
1:02:36question but the clock show have please
1:02:38close the program and really wonderful
1:02:42amazing presentation no matter is the
1:02:45tree of a robot the safety and also the
1:02:48this optimal control design and also the
1:02:51universal knowledge Discovery taking
1:02:53advantage of a cross disciplinary
1:02:55strength at colia really fit so
1:02:58perfectly aligned with our mission and
1:03:00vision thank you so much for your talk
1:03:02thank you for joining us today

About this transcript

This page contains the full transcript of Robots and the Search for Universal Intelligence | Columbia AI Summit by Columbia Data Science Institute, generated from the public captions YouTube serves with the video. The transcript has 11,117 words across 1,601 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.