YouTube transcript (PeRJkvzIEts) — Transcript
Full transcript
- 0:05I'm just going to go ahead and introduce
- 0:07our guest speakers and [clears throat]
- 0:09I'm very excited to have talk about
- 0:12their use of AI as part of their
- 0:15academic research.
- 0:17First, we're going to hear from Mark. He
- 0:19is a professor of English and I won't
- 0:23read his whole bio, but just look at the
- 0:25size of it. We should all be very
- 0:26impressed. And this will be true for all
- 0:29our speakers. Uh Megan is joining us
- 0:31from uh Frontiers Technology Lab in the
- 0:35Law School. So, um the whole focus of
- 0:38their work is on using AI in the
- 0:41furtherance of practice of law.
- 0:44And Marcus joins us from the depart-
- 0:47department of bioengineering
- 0:49and he uses uh models as part of his um
- 0:53work in understanding the cell and
- 0:55bioengineering.
- 0:56Uh so, without further ado, I will hand
- 0:59things off to our first speaker, Mark.
- 1:01>> Thank you all so much for um I guess
- 1:03coming to your class. Thank you more for
- 1:06having me and giving me the opportunity
- 1:07to talk to you a little bit about what I
- 1:10do.
- 1:11Um so, I am a professor in the English
- 1:13department, although technically I'm a
- 1:15professor of digital humanities. Um I
- 1:17also run a research center called the
- 1:20Center for Spatial and Textual Analysis
- 1:21over on the top floor of Wallenberg, um
- 1:24which has a very robust internship
- 1:26program uh starting uh I I about 25
- 1:30undergraduates that we hire per year to
- 1:31work on our projects, um as well as the
- 1:34associate director of data science for
- 1:36artistic and cultural analysis. So, all
- 1:39of that's a long way of saying that if
- 1:40anything I say today resonates, there
- 1:43are plenty and plenty and plenty of
- 1:44opportunities um for you to do this for
- 1:47yourself and I would be thrilled to talk
- 1:49to all of you about that.
- 1:52Um so, what kind of work do I do? Well,
- 1:54I think of it as the um meeting of data
- 1:57science and humanities.
- 2:00A kind of reading at scale, um
- 2:02particularly in my work where I think
- 2:04about large analyses of corpora. Um and
- 2:08that looks like a number of different
- 2:09things. It can look like the analysis of
- 2:11thousands and thousands of texts,
- 2:13looking for machine-recognized patterns
- 2:17uh that we've trained uh the different
- 2:19kinds of models to find for us. Or it
- 2:21can look like the in-depth analysis of a
- 2:23very specific text, in this case just
- 2:26one novel, but along the axes that are
- 2:29just unaccessible to a human reader.
- 2:31Here the result uh of a model that was
- 2:33changed to recognize all of the
- 2:35different genres except for the one that
- 2:36that book is actually written in. Um
- 2:39this looks different in a number of
- 2:41different hands. There are There are
- 2:42mapping projects that use GIS data um to
- 2:46think, for example, about geographical
- 2:47investment in a corpus of novels, or
- 2:50even things that look at early uh
- 2:52modern, in this case, correspondence
- 2:54networks uh trying to find patterns. Um
- 2:58but overall, since the subject of today
- 3:00is artificial intelligence, um this
- 3:02obviously comes into play in my world uh
- 3:06a lot, both as a method but also as an
- 3:09object of analysis. And the project that
- 3:11I'm going to share with you very quickly
- 3:13um will talk about uh how we how we
- 3:17think about it. Um
- 3:19and so what I really want to sort of
- 3:20hammer home is that for us, artificial
- 3:22intelligence has a place within the
- 3:24pipeline. Certainly, machine learning
- 3:26has a big place in the pipeline, but
- 3:28actual generative models have a weird um
- 3:31relationship, particularly with
- 3:33humanities research. Um and so I want to
- 3:36think about this carefully, right? We
- 3:38need to deploy it, but as a kind of
- 3:40middle layer between a strong
- 3:42statistical foundation for what we do
- 3:44with corpora, and a human-led
- 3:46understanding,
- 3:48um in my case, of literature, but a
- 3:49human interpretation
- 3:51of both the underlying data and what the
- 3:53model is doing. Um and so the project
- 3:56that I I wanted to kind of show to you
- 3:58as an example of the kind of work that I
- 4:00do um is called Style in the age of AI.
- 4:03It really takes a hard look at what we
- 4:06think of as AI style. And as you can
- 4:09see, this is a subject of a lot of think
- 4:12pieces recently.
- 4:13People arguing that AI style is ruining
- 4:16us, that we're all adopting it, that
- 4:18it's making our writing bad. But what I
- 4:20want to
- 4:21think about with this project is that
- 4:23it's not just the case of style as
- 4:26style. What we can think about in the
- 4:27humanities is the relationship between
- 4:30style and meaning. And so this project
- 4:33really takes a hard look at what happens
- 4:36when generative AI style infiltrates the
- 4:41way in which we make meaning. And for
- 4:42humanities, this is particularly
- 4:44important because in the sciences, we
- 4:46can think about meaning being made with
- 4:48numbers, right? It's the the material of
- 4:50research is the data.
- 4:52But in the humanities, we make meaning
- 4:54with words. If you read like a
- 4:56traditional English paper, not one of my
- 4:58papers, but a traditional English paper,
- 5:00what you find is a lot of words and
- 5:02meaning emerging from that. So what does
- 5:04that mean when we think about using
- 5:06generative AI to communicate it? So in
- 5:09this particular project, we're asking,
- 5:11do disciplines have style? That is, do
- 5:15historians write like a historian? Do
- 5:17psychologists write like a psychologist?
- 5:19Do computer scientists write like a
- 5:21computer scientist? Regardless of what
- 5:23they're writing about. And so for this
- 5:26particular project, we got a big corpus.
- 5:28This is just a pilot version of the
- 5:30larger project I'm currently running. So
- 5:32this is about 2,000 articles. We're
- 5:33currently working on a 10,000 article
- 5:35sample across 18 disciplines.
- 5:38And we wanted to compare style and
- 5:41meaning. So a a really simple way to
- 5:43sort of access aboutness is a topic
- 5:45model. Here, each dot represents an
- 5:46article in our corpus.
- 5:48It's placed within this TSNE graph shows
- 5:51roughly what it's similar to topic-wise
- 5:54in terms of its topical composition. And
- 5:56the disciplines come out really nicely.
- 5:58In particular, I want you to take a look
- 6:00at these two,
- 6:02philosophy and computer science. Nothing
- 6:04quite so far apart.
- 6:07Now we want to compare this to how these
- 6:10two disciplines, not just what they
- 6:11write about, but what they actually
- 6:14how they write. So not what they write
- 6:15about, but how they write. And so we can
- 6:18also take the same corpus of 2,000
- 6:20articles and we can classify using only
- 6:24what I'm calling stylistic features.
- 6:25This is just grammar along with things
- 6:28like sentence length, number of named
- 6:30entities, etc. etc. So no information on
- 6:33what the articles are about, just how
- 6:35they're written, and we get to about a
- 6:3743%
- 6:39success rate. That doesn't sound great
- 6:41and in fact we're working on improving
- 6:43it, but when you consider that, you
- 6:44know, at chance we'd be at 5%, this
- 6:46isn't that bad. It turns out that even
- 6:48at a rough level, anthropologists write
- 6:50like anthropologists.
- 6:52So what does this mean in terms of AI?
- 6:54What we decided to do is test an
- 6:58LLM model, which are particularly noted
- 7:01for doing interesting stylistic things,
- 7:03by doing what we're calling a style
- 7:05transfer. That is by carefully prompting
- 7:07here a local model, we said, "Okay, take
- 7:10those two farthest apart fields, right?
- 7:12What happens when you get an LLM to
- 7:14write a philosophy article, which you
- 7:16see on the left, as if it were a
- 7:18computer science article, which you see
- 7:20on the right?" So that is a philosophy
- 7:22article written like a computer
- 7:24scientist.
- 7:25In our initial experiments using this
- 7:28local model, it turned out quite well.
- 7:30It's not perfect yet. If we reclassify
- 7:33it using the same model, most of what
- 7:35used to be philosophy is now classified
- 7:37as computer science, but but there's
- 7:39still a long tail there that's that
- 7:40we're still working on. The more
- 7:42important bit, interrogating style and
- 7:44meaning, is if we go back to a different
- 7:48topic model.
- 7:49Again, same corpus of 2,000 articles
- 7:51except this time we included all of the
- 7:55philosophy articles that we rewrote as
- 7:58computer science. And so just to
- 7:59reorient you, there is in gray our
- 8:01philosophy, in green our computer
- 8:03science, we suddenly have that new
- 8:06group, which is our philosophy written
- 8:10as computer science. And again, this is
- 8:11not based on grammar at all. This is
- 8:13purely content words. And what it
- 8:15suggests is that the original philosophy
- 8:19articles that we rewrote using LLM's
- 8:21probing deeply into what they do with
- 8:23style, actually moves the content
- 8:26from philosophy just a smidge, but
- 8:29noticeably and meaningfully towards
- 8:31computer science, suggesting there is in
- 8:33fact a relationship between style and
- 8:36meaning at least as we can find
- 8:37computationally. And so this is just a a
- 8:40rough overview, a very quick overview of
- 8:43the kind of project that we do, the way
- 8:45we interrogate how LLM's are working,
- 8:48how we use them, but how we think about
- 8:50the kind of work they do in making
- 8:52meaning particularly in the humanities.
- 8:55Thank you very much.
- 8:57>> Hello everyone. So I'm Megan. I'm the
- 9:00executive director of a research lab at
- 9:02the law school. We're called the Legal
- 9:03Innovation Through Frontier Technology
- 9:05Lab. And so since it's quite the
- 9:07mouthful, we call it Lift Lab. And part
- 9:09of what we do is we think about applied
- 9:11AI in the legal domain, in particular in
- 9:14private practice. So in large law firms
- 9:16as well as corporate legal departments.
- 9:18How many of you are familiar with what
- 9:20lawyers actually do?
- 9:24Oh, quite some hands. Well, actually our
- 9:27lab basically asks this question, how do
- 9:29you define quality in the legal space?
- 9:31And largely it's because there is a
- 9:33sense of kind of alchemy behind what
- 9:35attorneys do. Um in economics terms it's
- 9:37kind of a credence good. You're not
- 9:39always sure exactly what you are
- 9:41purchasing.
- 9:42Um our lab kind of looks at a couple of
- 9:44pillars including evaluation, uh
- 9:47delivery of legal services, as well as
- 9:50legal education and training, and
- 9:51finally kind of what we call advanced
- 9:53methods. In evaluation, we're not only
- 9:56thinking about, you know, how closely
- 9:58does AI do legal work, but actually what
- 10:01exactly is important about evaluation in
- 10:04a domain where ground truth quality is
- 10:07not really there.
- 10:08How many of you, you know, are more
- 10:10familiar with the fact that benchmarks
- 10:12kind of look into say rubrics for
- 10:14example?
- 10:15I assume those that have technical
- 10:18backgrounds. Yeah, so in legal what we
- 10:20call a judgement rich domain, this is
- 10:22incredibly difficult because the model
- 10:25of the firm is that it indexes heavily
- 10:27on the individual. I will choose to work
- 10:30with Megan, I will choose to work with
- 10:32Mark, and we build sort of these
- 10:34long-term relationships. So how do we
- 10:36actually understand what exactly is the
- 10:38value that the profession has to offer?
- 10:41And so some of the research that we
- 10:42recently released known as judgement
- 10:44bench reckons with how we should think
- 10:46about evaluation. And so there's
- 10:48comparison between rubrics versus using
- 10:50preference ranking. So if you have two
- 10:52versions of something, which one is
- 10:54better? And it turns out in our paper
- 10:56that um
- 10:57preference rankings had far outperformed
- 11:00rubrics because the lawyer intuition,
- 11:03something about that was incredibly
- 11:05powerful. So some of the work that we're
- 11:07thinking about is if you have a large
- 11:09preference data set, can we actually
- 11:11inductively gather a rubric from it for
- 11:14post training? The second kind of pillar
- 11:16that we look at is the delivery of legal
- 11:18services. So this is kind of a domain
- 11:21where many of the
- 11:23the sort of primary business model that
- 11:25they work with is the billable hour. And
- 11:28as you know, the billable hour kind of
- 11:30runs into tension when you have AI that
- 11:32is vastly improving productivity. And so
- 11:35when you have, you know, lawyers that
- 11:37typically charge say 10 hours for this
- 11:39particular work product, and AI could
- 11:41make it simplified to about two hours,
- 11:44what does that necessarily mean for
- 11:46their margins? And of course, nobody is
- 11:48crying tears for big law departments
- 11:50that are already earning a ton of money,
- 11:53but at least I think it allows us to get
- 11:55a sense, you know, when it comes to high
- 11:58demand knowledge work, are we thinking
- 12:01about where these revenue models are
- 12:03going? And more importantly, what is the
- 12:05AI exposure in the work all together?
- 12:08And so some of the projects that we're
- 12:09thinking about here is we're doing an
- 12:11observational study. How many of you're
- 12:13familiar with any of the legal AI
- 12:15providers like Harvey or Legora?
- 12:20Great. Great to know that they're
- 12:21somewhat popular. Um
- 12:24basically, we're going to be working
- 12:25with some of these legal AI providers to
- 12:27get an inside look exactly at how law
- 12:30firms are using these providers. So,
- 12:33what are sort of the edit histories we
- 12:34can go into, how much of certain work is
- 12:37being edited on top of or intervened
- 12:39with humans, and how much of it is
- 12:41actually never intervened at all.
- 12:44Through this, we're going to try to
- 12:45understand is there no intervention
- 12:47because agents have gotten to a place
- 12:50where they are perfectly fine with
- 12:52executing legal work, or is it that this
- 12:55work is simply not that important and
- 12:57that we actually have to push uh up the
- 12:59high-value work further up the pipeline.
- 13:02Our third pillar is actually something
- 13:04that I'm most sort of passionate about,
- 13:06which is the legal education and
- 13:08training aspect. So, in the legal
- 13:10profession, a lot of what you learn in
- 13:13law school, there tends to be a bit of a
- 13:15disconnect into practice.
- 13:17Um a lot of lawyers become the amazing
- 13:20lawyers that they are because they
- 13:22learned by doing. Um and a lot of this
- 13:25experiential learning is coming into
- 13:27tension because the traditional sort of
- 13:30everyday day-to-day work is going away.
- 13:33So, what happens when the work that you
- 13:36had a lot of muscle reps into and you
- 13:38dedicated a lot of young attorneys to
- 13:40help support kind of naturally
- 13:42vaporizes. How do you train in that
- 13:44regard? So, we're working on a lot of
- 13:46AI-based simulation training. We have in
- 13:49particular a couple of platforms. So, we
- 13:50built a multi-agent negotiation
- 13:52simulator. We also built a deposition
- 13:55simulator. How many of you have seen
- 13:57Suits? I feel like your generation likes
- 14:00Suits.
- 14:01Um
- 14:02basically, if you've ever seen them do a
- 14:05deposition poorly, I think, um you get
- 14:08to practice. And actually, this tool is
- 14:10open source. So, in case you're curious,
- 14:12you can all uh try it on our lab's
- 14:15website. Um, the idea is is that in a
- 14:18profession like the legal space, because
- 14:20you're spending all this time in a
- 14:22high-stakes environment, you often can't
- 14:25afford to lose. There are a lot of
- 14:26senior attorneys right now that still
- 14:28are kept up at night by those small
- 14:30mistakes that they made when they were
- 14:32juniors. And so we are trying to kind of
- 14:34not only help support kind of the
- 14:37training in areas that might go away,
- 14:40but we're also kind of creating new
- 14:41environments. So you can imagine, um,
- 14:44you can actually do a deposition as many
- 14:46times as you like. Basically our
- 14:48platform is you can upload any discovery
- 14:50document or any witness statement. It
- 14:52creates an AI witness on the fly, and
- 14:55then you can actually adjust their
- 14:57personalities. We work with some senior
- 14:59litigators here where, um, there's 14
- 15:02typical archetypes that they've
- 15:04imagined: combative, evasive, forgetful.
- 15:07And you can play on any of these
- 15:09particular sort of configurations. It
- 15:11dials the personality all the way up or
- 15:14down, and so you can get the most
- 15:16caricatured version of a combative
- 15:18witness. And the goal is that through
- 15:20your line of questioning that you're
- 15:22able to steer and get the responses that
- 15:24you need. Um, as we were building this
- 15:27out, one of the uh, attorneys that we
- 15:29actually worked with, they told us this
- 15:31kind of funny anecdote that continues to
- 15:33haunt them, which was a deposition that
- 15:35they had done as a young attorney. And
- 15:37it turns out that they could not
- 15:39actually get a single impeachment. So in
- 15:42the legal context, that means sort of
- 15:44locking in that particular fact you want
- 15:46for your case.
- 15:47Um, they couldn't get a single
- 15:48impeachment because the opposing counsel
- 15:50was incredibly distracting. So the
- 15:53opposing counsel with every question
- 15:55that this attorney said did a weird
- 15:57laugh, like
- 15:59And because of that, um, he was
- 16:01incredibly distracted, and then he had a
- 16:04dramatic failure in being able to
- 16:06complete the deposition. We want to
- 16:08actually use simulation as an
- 16:09opportunity not only for as a
- 16:12pedagogical tool for rehearsal, we also
- 16:15want to simulate what should not happen
- 16:17and what we want to avoid happening. And
- 16:19so, that's kind of one aspect that we've
- 16:20been thinking about. And the last pillar
- 16:22is what we call the advanced methods.
- 16:24This is kind of a hodgepodge. It's a
- 16:26grab bag. It's really kind of in short,
- 16:30what's made available for the legal
- 16:32profession in light of generative AI.
- 16:34So, one example is that we've been
- 16:36working on something called partner
- 16:38personas or legal personas. And this is
- 16:40basically the personalization of an LLM
- 16:43to behave like the specific individual.
- 16:46For those of you who are in CS, maybe
- 16:48you're familiar with Simile
- 16:50from Michael Bernstein and their lab
- 16:52that's since become a company.
- 16:55Here, what we're really fascinated by is
- 16:57can we create almost AI twins of senior
- 17:01partners.
- 17:02So far in some of our research, what we
- 17:04discovered was that AI is very good at
- 17:06role-playing, but it really kind of
- 17:08lends itself into stereotyping and
- 17:10caricatures really quite quickly. How do
- 17:13you actually steer at a granular level
- 17:16an LLM to become exactly like the
- 17:19individual. So, if I was a negotiator
- 17:21and you say that I'm an aggressive
- 17:22negotiator, it doesn't mean that I'm so
- 17:25aggressive that I will haunt your
- 17:26dreams. It means that actually I tend to
- 17:30lean more aggressive, and it could be
- 17:31circumstantial depending on, you know,
- 17:34what type of specific where I am in the
- 17:37pipeline of the negotiation, and what I
- 17:40want to do to represent my clients. So,
- 17:41here we kind of think a lot about can
- 17:44you actually use numerical steering to
- 17:46get that level of granularity. So, all
- 17:49in all, if there is any interest in kind
- 17:51of working in sort of a knowledge-rich
- 17:53space, or you have more questions, feel
- 17:55free to reach out.
- 17:57>> I wouldn't say like well, quick
- 18:00introduction. So, I'm a
- 18:02I'm the chair of the bioengineering
- 18:03department, which is just over here.
- 18:08Let me see real quick.
- 18:10And
- 18:11you know, just like a lot of us so we
- 18:13are doing plenty of plenty of AI stuff
- 18:16here. It was great. It's actually I love
- 18:17that I'm kind of together with other
- 18:19corners of campus because that's where
- 18:20you get to get to really generate ideas
- 18:23and think of new things. But, um
- 18:25I've got kind of I can I'm going to tell
- 18:27you kind of my background and how I'm
- 18:28thinking about it and then really
- 18:29instead of saying what I'm currently
- 18:31doing too much, I'm going to talk more
- 18:33about questions I have and what I'm
- 18:34thinking about. Um
- 18:37so I think a lot about disease and I
- 18:39think you know, it can be really easy if
- 18:42you read the papers, you know, you'll
- 18:43always read about a cancer gene or an
- 18:45Alzheimer's gene or this or that gene.
- 18:47And yet when it comes to the cures for
- 18:49all those things, they aren't there,
- 18:51right? So, you might wonder why and the
- 18:53reason is that they're not you know,
- 18:54cancer's not a one gene problem. Right?
- 18:57It's like a hundred or a thousand of
- 18:59genetic but also epigenetic and other
- 19:01environmental factors and they all come
- 19:03together in these non-intuitive ways.
- 19:06And so you'll never be able to combat
- 19:08like a complex disease like that unless
- 19:11you learn to embrace its multi-factorial
- 19:14complexity. So, that's what I'm all
- 19:15about.
- 19:17And for this reason, people have pointed
- 19:19to whole cell simulation
- 19:22uh and just trying to simulate things
- 19:23like a tumor or or different kind of
- 19:25cells as a grand challenge of this
- 19:26century and my lab's been working in
- 19:29this for years. We actually built the
- 19:30first computer model of a cell that
- 19:32takes all the gene content into account
- 19:35and uses it to make predictions about
- 19:36phenotype. I'll show you kind of a
- 19:39kind of like a cartoon but this is based
- 19:40on real simulation data where we can
- 19:42have in this case a bacterial cell
- 19:43that's growing out and it's replicating
- 19:45its DNA and it's dividing at a regular
- 19:47rate. One simulation of one cell
- 19:50generating one time gives us about a gig
- 19:52of data, right? And so now we can throw
- 19:54the cell into another environment, a
- 19:57bunch of sensors are going to get
- 19:58triggered, uh the gene expression will
- 20:00all change, metabolism will re-route
- 20:03itself to this new change and the the
- 20:05net effect of it all is that the cells
- 20:06will grow faster and bigger and and uh
- 20:09start even parallel processing their
- 20:11DNA. All this cool stuff happens.
- 20:14And uh I wouldn't call this AI per se,
- 20:16more like artificial life which people
- 20:18don't talk about as much, right? Like
- 20:20but but that's kind of the space I see
- 20:22myself in, artificial life
- 20:24and
- 20:26and also how AI interacts with
- 20:29artificial life. So we got involved in
- 20:31deep learning
- 20:33over a decade ago now where we were kind
- 20:34of amongst the pioneers of trying to
- 20:37automatically look at a microscopy image
- 20:39stack and be able to give you all of the
- 20:41cells that are in there and be able to
- 20:43track them over time so that we can see
- 20:45the individuality of cells, how they
- 20:47interact with each other.
- 20:49>> [snorts]
- 20:49>> And this was a lot of fun
- 20:52and at the same time I was actually at
- 20:55Google X as a consultant. And working on
- 20:59a project that now is now is known,
- 21:01right, A-life. And at the time that I
- 21:03was sitting next to this guy who only
- 21:05went by Hash. I now know his name is
- 21:07Jonathan Ross. He works at Nvidia. But
- 21:09he had made these first TPUs, tensor
- 21:12processing units, maybe you've heard of
- 21:13those. They ran at about a thousand
- 21:15times the speed of the fastest GPUs and
- 21:18they were the secret of how AlphaGo was
- 21:22able to beat Lee Sedol. It wasn't known
- 21:24that these TPUs existed but without them
- 21:26each turn at that Go table would have
- 21:29taken about a year for the computer to
- 21:31process. And so just watching all this,
- 21:34I'd spend my lunch breaks like with
- 21:35those guys talking about AlphaGo and it
- 21:38gave me this first question that I just
- 21:40want to pose to you, right, as the
- 21:42future of our world, right? What
- 21:44audacious human aspirations are only now
- 21:47conceivable with this new technology?
- 21:49That's something I think about a lot.
- 21:51Not just about this technology, by the
- 21:52way, right? I've had these exact same
- 21:54feelings
- 21:55when we first learned that we could take
- 21:57regular cells and dedifferentiate them
- 21:59to induce induce pluripotent stem cells,
- 22:02right? I've had these same feelings when
- 22:03I heard about gene editing
- 22:05with CRISPR, right? So these big
- 22:08technologies, I think it's an important
- 22:09question to ask, like what now is
- 22:11possible? Not just at a low level but at
- 22:14like the highest level. No one could
- 22:16ever do this and now they can.
- 22:20Then the next thing is education, you
- 22:21know, that's been really fascinating to
- 22:23watch. I'm sure for you guys, too, for
- 22:25better or for worse, right? Um
- 22:27I don't know if you saw this recent
- 22:29report by MIT talking about education
- 22:32and AI. Uh I love these principles. I
- 22:34think the whole thing is really
- 22:35fascinating, but it's fun. We uh you
- 22:38know, this summer my TAs and I worked on
- 22:40creating, you know, an AI tool that
- 22:42would help students to learn my class.
- 22:44Like
- 22:45I wrote the textbook, so it seemed like
- 22:46we could just put everything that I had
- 22:48ever generated into this into this
- 22:50helper. And then, of course, only to
- 22:52find from this MIT report that students
- 22:55who uh who used tools like that not only
- 22:58had worse learning outcomes and learned
- 23:00less,
- 23:01but actually were more depressed, felt
- 23:03more isolated, and felt more anxiety,
- 23:05right? So, then we we actually had a
- 23:07vote on this and decided not to give
- 23:09this beautiful tool out uh just for
- 23:12those very reasons. We didn't want to do
- 23:13more harm than good. But, I would ask
- 23:15based on that, like
- 23:17and again, maybe a little bit
- 23:18independent from AI, but AI is right in
- 23:20the center of it, right? How can we
- 23:22ensure
- 23:23that you receive the best possible
- 23:26education that anyone could ever give,
- 23:28right? In the history of the world, cuz
- 23:30that's really what where we are and what
- 23:32you deserve.
- 23:34The third question kind of relates to
- 23:36this fun thing. I've I've been really
- 23:38interested in what these faith and other
- 23:40like ancient traditions have been how
- 23:41they've been reacting to AI. Some of you
- 23:43might know that the first letter that
- 23:45that this new pope wrote
- 23:47was a long treatise on AI. And in
- 23:50particular, AI in terms of safeguarding
- 23:52the human person, which is a fascinating
- 23:55uh phrase.
- 23:57And then I was uh honored to be recently
- 23:59in a in a brief uh in a in an
- 24:02interaction with uh Father Michael
- 24:04Baggott representing the Vatican, who
- 24:06had just come to meet with Anthropic
- 24:07leaders. Some of you might have seen the
- 24:09article in the New York Times talking
- 24:10about AI and transhumanism and how how
- 24:14they are thinking about things at the
- 24:15Vatican. I would love to hear how others
- 24:18uh both faith and non-faith but
- 24:20traditions are thinking about these
- 24:22things. But, I guess the question there
- 24:24that they are asking, as far as I can
- 24:25tell, is what can we do to promote human
- 24:28flourishing and the highest ideals of
- 24:30civilization using this technology?
- 24:33Okay, also a very, very interesting
- 24:35question and again, I probably think of
- 24:37each of these multiple times a day.
- 24:39Final one and then I'll be done.
- 24:42Uh this is an article I found in nature
- 24:44a couple years ago and the headline just
- 24:46grabbed me. Scientists need more time to
- 24:50think. I was like, [laughter] yes, I
- 24:52need more time to think. Uh my my kids
- 24:55would probably say I need to any time A
- 24:56time to sound like I'm thinking. My even
- 24:59my mom sometimes talks about superficial
- 25:02intelligence, which is kind of what
- 25:03she's talking about here. So, uh
- 25:06artificial superficial, I still feel
- 25:07good about it. But, um
- 25:09scientists need more time to think and
- 25:11the big thing here that that the point
- 25:13they make is that people have become
- 25:15more and more productive with
- 25:17productivity tools. Email has made us
- 25:19more productive.
- 25:21But, it's actually given us less time to
- 25:23think.
- 25:24It's given us less time to internalize,
- 25:27to reconcile seemingly opposing
- 25:29evidence. It's given us less time to do
- 25:32all of that stuff and so the final
- 25:33question that I
- 25:34would love to hear about here or think
- 25:36about is, you know, how can we give
- 25:38scientists more time to think?
- 25:41How can we give doctors more time to
- 25:43care?
- 25:45How can we give students more time to
- 25:46explore?
- 25:48Right? How can we give humans more time
- 25:50to connect? So, those are my obsessions
- 25:53surrounding AI and I can just leave it
- 25:55there.
- 25:56>> All right, that was an exciting
- 25:58introduction to what um
- 26:00expertise we're getting from our
- 26:02panelists. I'd like to invite them up
- 26:04now to take uh these seats in the front
- 26:07and I will ask some questions, we'll do
- 26:10some Q&A, and um then we'll open it up
- 26:14for audience Q&A.
- 26:16All right, so the first question that uh
- 26:18I would like to put to each of you is
- 26:21when you're thinking about applying AI,
- 26:25how do you balance thinking about what
- 26:29the capabilities and limitations of the
- 26:31technology are today
- 26:34with
- 26:35uh trying to
- 26:37predict out into the future a little bit
- 26:40where the technology might be 5 years
- 26:43down. So, if you're, you know, building
- 26:45a tool or starting kind of a research
- 26:48agenda that might involve AI, how do you
- 26:51balance
- 26:52the present versus
- 26:55um possible improvements or change in
- 26:56the future.
- 27:02>> Uh I can take a stab.
- 27:04Um one thing that happened for me when I
- 27:06started at Google X way back when, um I
- 27:09was just thinking, you know, being
- 27:11academic, I was wondering if I wanted to
- 27:12do that, but one thing that they said to
- 27:14me that I thought was really fascinating
- 27:16was um
- 27:18you should just assume here that your
- 27:20compute and memory are infinite, and you
- 27:22should not develop based on any
- 27:25consideration of what today's capacity
- 27:27is.
- 27:28Um
- 27:29that was a huge uh mind shift for me
- 27:33because I think very often I I do feel
- 27:35limited by the technology. I'm thinking,
- 27:37oh, could we actually do this, right?
- 27:39But, I think it really gave me
- 27:41permission to think
- 27:43much bigger, just kind of feeling like
- 27:46uh you know, okay, things will be in
- 27:48play. If I create this, at some point
- 27:49the right the right situation will be in
- 27:52place where we could actually use it for
- 27:53something.
- 27:54And uh and it was, right? So, I think in
- 27:56this case just the fact of kind of
- 27:59opening your mind to the to this idea
- 28:01that somewhere out there somebody's
- 28:03invented something that runs a thousand
- 28:04times faster or something like that is a
- 28:07is a pretty big impetus to to go big, I
- 28:11think.
- 28:12>> [clears throat]
- 28:15>> I will say from my perspective, um if
- 28:18the last 5 years have shown us anything,
- 28:21it's that
- 28:23this is this is largely unpredictable,
- 28:26right? Like if you if it's fascinating
- 28:28to watch the discourse evolve around uh
- 28:31large language models since 2001,
- 28:34when we thought, "Oh, well, these are
- 28:37okay writing and if they can string
- 28:39together a sentence, but they're never
- 28:40going to
- 28:41no one's ever going to fool the or
- 28:42mistake this for a human."
- 28:45And just the leaps that have happened
- 28:48in ways that we still don't really
- 28:51completely understand,
- 28:54I think make
- 28:56projecting out, especially 5 years, a
- 29:00kind of a kind of fool's game. But,
- 29:03there are things that are constant,
- 29:05right? And that's that's us. We don't
- 29:07know how eventually we'll change in
- 29:10response to the in response to the
- 29:11technology, but we don't change quite as
- 29:15fast as as LLMs do, right? The human
- 29:18remains a kind of constant in the room.
- 29:20And the way in which we interact with
- 29:22them,
- 29:23there are going to be some stable
- 29:25aspects to that. So, certainly in my
- 29:28teaching and in my research, when we
- 29:30think about this, we tend to center the
- 29:34human experience in interacting with the
- 29:37models,
- 29:39and looking at how that that has
- 29:42changed, but but that's a kind of change
- 29:44that I think we can we can actually
- 29:46grasp and and do something with
- 29:47regardless of what happens in the next 5
- 29:49years.
- 29:51>> I think I really resonate with both of
- 29:52your points. In particular, Mark, your
- 29:54discussion about the human experience.
- 29:57At least in the legal domain, I think
- 29:58we've barely even scratched the surface.
- 30:01So, Legal Tech, which now has evolved
- 30:04into Legal AI, has for many, many years
- 30:07focused on a question around
- 30:09standardization and trying to look for
- 30:11what is the golden standard. And so, a
- 30:13lot of the work that's been done
- 30:15historically was about sort of labeling
- 30:17what would be the best quality or best
- 30:20markers of a particular contract or a
- 30:23particular legal document. And the issue
- 30:25is is that, as I mentioned, so much of
- 30:28the business model indexes so heavily on
- 30:30the individual. And that a lot of it is
- 30:33thriving in the space of what lawyer
- 30:35intuition and the process behind it is.
- 30:38And we actually haven't even been able
- 30:40to really unpack when an attorney says,
- 30:43"I know it when I see it." what that
- 30:45necessarily entails. And so, when it
- 30:48comes to kind of the technological
- 30:50advancement, what we're seeing actually
- 30:52is largely a cultural disconnect between
- 30:54how things are operating internal to the
- 30:57profession, internal to a law firm or a
- 30:59corporate legal department, and actually
- 31:01how the technology's interfacing with
- 31:03these people.
- 31:05And so, some of the things that we've
- 31:07built so far, for example, in the
- 31:09education space, like the simulations,
- 31:11really we're thinking about how do you
- 31:13act how do lawyers train and impart
- 31:15wisdom from one person to another? How
- 31:18do we actually extract tacit knowledge?
- 31:21And what's been really interesting is
- 31:22that as the models get better, we
- 31:24actually are able to have a better sense
- 31:27of collaboration.
- 31:29Um just as an example, we built this
- 31:31tool called the Socratic Mentor, and it
- 31:33really is meant to reflect, for example,
- 31:36uh that process between PhD advisers and
- 31:39their PhD students. And it's really
- 31:41questioning, breaking down your research
- 31:43problems. We actually experimented with
- 31:46some senior attorneys on using this
- 31:49tool. And basically what it does is they
- 31:51throw out, you know, why they make
- 31:53certain decisions, and then this tool is
- 31:56quizzing and quizzing and breaking down
- 31:58actually one senior attorney had said
- 32:00that they felt like they were being
- 32:01picked apart like a Costco chicken. Um
- 32:04but and the result is that as these
- 32:07models just got better and better, they
- 32:09felt like they were really being drilled
- 32:12and asked to distill down, "Why is it
- 32:14exactly that I do this? How do I tether
- 32:17the quality of my work to specific
- 32:19experiences that I've had?" And so, I
- 32:22think it's pretty exciting that, you
- 32:23know, it's not necessarily a direct
- 32:25correlation like this technology is
- 32:27going to answer to this work. I think
- 32:30it's about that is really the large
- 32:31focus of the industry where it's about
- 32:33AI executing on legal work, but I think
- 32:36where the sort of magic and research is
- 32:38going is how can AI help attorneys
- 32:42better understand why they do the things
- 32:44the way they do.
- 32:48>> I think that's a great segue to the next
- 32:49question I wanted to ask, which is um so
- 32:53something we spend a lot of time in this
- 32:55class doing is trying to cultivate
- 32:57within each person a a well-developed
- 33:01moral compass about what kinds of things
- 33:05is appropriate to use AI for and what
- 33:08should really be handled by the human
- 33:11person.
- 33:12And um something that we will in the
- 33:17second half of the class be reaching
- 33:20into a little bit that I'm hoping I can
- 33:23invite you to reach into is
- 33:26um beyond one's own moral compass
- 33:30there are expectations of others,
- 33:33co-workers,
- 33:35clients, bosses,
- 33:38community members, who however much we
- 33:42may have
- 33:44you know, put effort into being
- 33:46thoughtful and responsible and ethical
- 33:49in our own choices about AI use, others
- 33:52may have different conclusions and
- 33:54different expectations.
- 33:56And so that kind of mismatch could come
- 33:58up, for example, between a student and
- 34:01an instructor where a student thinks I
- 34:03you know, that they have made a
- 34:05responsible choice to
- 34:06only have very limited participation of
- 34:09AI in editing an essay that they wrote
- 34:11for homework, for example, but maybe the
- 34:13instructor feels like even that much is
- 34:15inappropriate or something. So
- 34:17um another you know, example of that
- 34:20that came up um as I was doing research
- 34:22for this class was in a conversation
- 34:25with someone who's um a leader of a a
- 34:28major global faith or
- 34:30and um was talking about how
- 34:34they're putting a lot of thought into
- 34:36what things that is appropriate in in of
- 34:39sort of office efficiencies that could
- 34:41be achieved by AI.
- 34:43And what do what does the flock
- 34:46consider, you know, things that they
- 34:48expect that their leaders are doing
- 34:52through
- 34:53divine inspiration or personal uh touch.
- 34:58And if there's a mismatch there in
- 35:00expectation
- 35:02and that is discovered, it could be
- 35:04catastrophic for someone's, you know,
- 35:06faith. So,
- 35:08um how do you think about that within
- 35:12the
- 35:14the human circles that you operate, how
- 35:17to
- 35:18um govern yourself and manage
- 35:21expectations with others?
- 35:26>> Can I pull the class on something?
- 35:29So,
- 35:30uh
- 35:31you know, probably all of you have asked
- 35:33a professor for a letter of
- 35:34recommendation.
- 35:36How would you feel? How many of you
- 35:37would be in favor of them using AI to
- 35:40write that?
- 35:42Yeah, right. Can I see
- 35:44>> [laughter]
- 35:44>> How many of you would really be troubled
- 35:46and probably choose someone else if you
- 35:47knew that this professor was going to be
- 35:48using AI?
- 35:50Yeah. Right. That's interesting, right?
- 35:53I think that's really interesting.
- 35:55Uh I think communication's really
- 35:57important, but this is this is a good
- 35:59example for me because I have made the
- 36:00decision that I am not going to
- 36:03write letters using AI, right? Like I
- 36:05I will use it I will not use it to
- 36:07write, right? I will use it to basically
- 36:09hold information and I can ask it
- 36:11questions. It'll remind me, you know,
- 36:13remind me where they did that
- 36:14internship, you know, something like
- 36:16that, but not um not in a way that I
- 36:19lose my voice and my personal connection
- 36:22to the to the student. So, then
- 36:25to me, at least, the question becomes,
- 36:27okay,
- 36:28how can I use AI not to save time with
- 36:32this letter because I think that's what
- 36:34most professors might be thinking when
- 36:36they're using AI. How can I save time?
- 36:38Instead, it's how can I write a higher
- 36:40quality letter that gives this gives a
- 36:42better
- 36:44view of this student and really
- 36:46showcases how I know them and what my
- 36:48experience is with them so that someone
- 36:50else can read it and really know
- 36:53that [clears throat] student and my care
- 36:54and concern for that that student and
- 36:56our relationship. That's a very
- 36:58different question to ask AI, right? Or
- 37:00a very different approach to take, but I
- 37:02think that's kind of goes to what you
- 37:03were saying. Like are we just using it
- 37:05to save time? Like write me 100 student
- 37:07like I'm writing them all right now. So
- 37:09I'm thinking about it a lot.
- 37:10Uh
- 37:10>> [laughter]
- 37:11>> I've been tempted.
- 37:13But [clears throat] um but I think this
- 37:14idea of saying no, what I want and I I
- 37:18tell this to Claude all the time. What I
- 37:20want is quality. I do not care as much
- 37:22about saving my time. I want higher
- 37:25quality and I want my the way I define
- 37:27quality is is humans first. So um
- 37:31or let's say planet first. Real entities
- 37:33first. Sometimes it's also animals
- 37:35first. But um but that idea I think is
- 37:38is important. So maybe I'll just throw
- 37:39that out as kind of a fun way to pull
- 37:40the class on how they feel. It might
- 37:42also change how you think about, you
- 37:44know,
- 37:45what you are presenting to a professor,
- 37:46you know, how how we might all feel in
- 37:48terms of what that means about our
- 37:49relationship.
- 37:51>> Can I actually ask a follow-on question
- 37:53to that? Um to poll everybody, how many
- 37:56of you have had professors
- 37:58respond to you, you can write whatever
- 38:01you want in this recommendation letter
- 38:02and I'll sign it.
- 38:06Would you prefer that approach or
- 38:09knowing sort of transparently that your
- 38:13professor that's writing your
- 38:14recommendation letter has had some
- 38:16support from AI?
- 38:21>> You have to ask who prefer Who Who
- 38:23prefers uh
- 38:26you writing
- 38:26>> writing it.
- 38:31>> Would you use AI to write it?
- 38:33>> [laughter]
- 38:37>> I think that these questions are
- 38:38starting to get kind of quite difficult
- 38:40at least from our standpoint. Um we
- 38:44recently hear in the legal profession
- 38:45that a lot of young attorneys are
- 38:47really, really scared about using AI in
- 38:50practice. And because they could be a
- 38:53breath away from being the headline of a
- 38:55very salacious newspaper title like
- 38:59Sullivan Cromwell attorney like has
- 39:01these, you know, hallucinated citations
- 39:04in their legal briefs. And so
- 39:06incidentally, I spoke to actually the
- 39:08Sullivan Cromwell team about that
- 39:10specific incident. And there were
- 39:11actually seven attorneys eyes on that
- 39:15particular brief, and it still slipped
- 39:18their radar. Um, and so now I think
- 39:21there's been kind of this tension where
- 39:23a lot of partners and they're looking at
- 39:25a lot of usage internally in AI,
- 39:27especially on Legora or Harvey or any of
- 39:29the legal AI providers. Um,
- 39:32the highest usage comes from leadership
- 39:34and partnership.
- 39:36And actually the lowest usage is from
- 39:38the young attorneys. And what's been
- 39:40interesting is like how do we get
- 39:42actually the partnership to encourage
- 39:44back some of the younger attorneys to
- 39:45use these platforms in a responsible
- 39:47way. Um, and some partners have started
- 39:50to write documentation of exactly how
- 39:52they've been using AI and kind of using
- 39:54that as a way to kind of soft guide a
- 39:58lot of the younger attorneys. I think
- 40:00it's really our responsibility kind of
- 40:03with familiarity with using AI that we
- 40:05show that direction um, a lot to our
- 40:08students uh, in the legal space, to the
- 40:11younger attorneys. Because honestly, it
- 40:13is really incredibly unclear sometimes
- 40:16to only uh, rely solely on your
- 40:18independent kind of moral compass
- 40:20because the models are just moving so
- 40:22quickly. And even our conception of the
- 40:25work that has to be done is going to
- 40:27look so different. And so I think there
- 40:29is this kind of balance where um, you
- 40:32know, is it the educator that will just
- 40:34use the exact same exam, exact same
- 40:36material over and over again? Or is it
- 40:38that it's our responsibility to think
- 40:40through like how we should be teaching
- 40:42the class entirely differently?
- 40:45>> I would I think I'd echo both of what
- 40:48you just said because for me
- 40:51I remember learning as a young computer
- 40:53science student before I found my way
- 40:55into to to literature.
- 40:58Um,
- 40:58remember learning about the Turing test,
- 41:00right? The gold standard for half a
- 41:02century or more actually in how to, you
- 41:06know, judge when we have achieved some
- 41:08kind of artificial intelligence. The
- 41:10thing I remember thinking about it at
- 41:12the time is that this is a test based on
- 41:15deception, right? Like the gold standard
- 41:17test for years and years. We're more
- 41:19complicated now, but as we're thinking
- 41:21about how we've trained these models,
- 41:24what we're what what the ideal has
- 41:26always been is that we want them to fool
- 41:29humans into thinking that they're
- 41:32humans. Now, again, we've moved beyond
- 41:34that ethically, but still somewhere at
- 41:36the core is is is the kind of factoid,
- 41:39the kind of nugget, the kind of ethical
- 41:41quandary that these are machines, these
- 41:44are machines trained to deceive us. Um,
- 41:48and how we use them obviously can can
- 41:51can have positive or negative ethical
- 41:52ramifications, but the way to get around
- 41:55I think all of the moral quandaries that
- 41:57that you've posed is radical
- 41:59transparency, right? Like in that case,
- 42:02I don't have to depend on my own morals
- 42:06covering what I do with AI in my
- 42:08classroom, in my lab, etc. Because if I
- 42:11am 100% transparent about how I'm using
- 42:13it and where I'm using it, other people
- 42:15can make up their own minds and we can
- 42:17have a conversation about that. I think
- 42:19where ethically we run into trouble is
- 42:22when, you know, we have a professor that
- 42:25writes 100 recommendation letters using
- 42:27AI, but pretends that they've written
- 42:29them themselves or or vice versa, the
- 42:31student who, you know, hands in a paper
- 42:34um generated that that they say they've
- 42:37done. So, I think I mean again, it sort
- 42:39of comes back to the human. I think
- 42:41transparency allows us to have these
- 42:44conversations in a way that opens up
- 42:48these ethical questions to a larger to a
- 42:50larger audience.
- 42:54>> Another perfect segue. The next thing I
- 42:56wanted to talk about is transparency and
- 43:00developing cultural community norms as
- 43:04well as
- 43:06just very specific infrastructure
- 43:08supporting
- 43:10transparency and disclosure around AI
- 43:13use. So, if each of you could share if
- 43:18you observed any
- 43:20of that kind of work happening in your
- 43:24fields either in how
- 43:27research conferences are incorporating
- 43:31opportunities for
- 43:34places for people to disclose exactly
- 43:37how they've used AI
- 43:39or if you have any thoughts about you
- 43:43know what we should expect say from
- 43:47political advertisements that use AI
- 43:49videos
- 43:51and should they have a little
- 43:54you know
- 43:54>> [laughter]
- 43:54>> thing at the bottom of the screen saying
- 43:56that what should we expect from media
- 43:59and what should we expect from each
- 44:02other
- 44:03is you know a future recommendation
- 44:06letters submission form web page going
- 44:09to have a little box that
- 44:11says describe your AI use here what what
- 44:14kind of infrastructure and societal
- 44:17adjustments do we need to make to
- 44:19support transparency.
- 44:25>> I guess I would say and here I'm going
- 44:26to get here I'm going to get heretical.
- 44:28This is something I would say in this
- 44:29classroom but I would never say in my
- 44:31own department. Or any of my own
- 44:33conferences.
- 44:36I think one of the places interestingly
- 44:38where we run into problems is like the
- 44:40outright ban on AI the kind of blanket
- 44:43across the board this should never be
- 44:45used for any reason.
- 44:47And I think there are situations in
- 44:49which its use is deeply a problem.
- 44:52And and I say this again coming from a
- 44:54from a field that has a tendency to to
- 44:55do those bands. That's it's sort of
- 44:57going back to blue books as a way to to
- 44:59get out from under it. But when we do
- 45:01that, what we're doing is we're
- 45:03incentivizing people to still use it,
- 45:06but not say anything about it, right?
- 45:08Um, and I think it comes the best work
- 45:10I've seen in my field um, around its use
- 45:16in a in a I think ethical and open way
- 45:18that that really incentivizes
- 45:19transparency is just is to figure out
- 45:22what we care about and and what we care
- 45:26less about. So, for example, in my
- 45:28field, in my very specific like digital
- 45:30humanities, computational literary
- 45:31studies, we care very much about writing
- 45:35as a thought process and writing as a
- 45:36human um,
- 45:38uh, created object. Um, and so it would
- 45:41be a problem if people started
- 45:43submitting, you know, AI-written um,
- 45:45articles to journals because as I said,
- 45:47that's where the meaning sort of
- 45:48happens.
- 45:50But on the coding side, a lot of the
- 45:53people in my field came to doing this
- 45:56kind of computational work from a
- 45:57humanities background, not from a CS
- 45:59background. And so AI has given um, a
- 46:03large number of people access to really
- 46:06complicated coding techniques that they
- 46:08didn't have before. And so, for example,
- 46:11in uh, a number of the journals in my
- 46:13field, um, you're expected to disclose
- 46:16your use of AI. But in the in the sort
- 46:19of fine print, um, it says where that
- 46:23use is acceptable and where it's not,
- 46:24right? Don't write the paper with AI.
- 46:27If you did use AI, you know, Claude code
- 46:30to help develop the the software behind
- 46:33it, you reveal that, credit it, cite it,
- 46:36um, and then move on. The blanket ban
- 46:39encourages, I think, a kind of bad
- 46:40acting. Sort of drawing the lines around
- 46:43here's where it's okay gives people the
- 46:46freedom, I think, to be transparent
- 46:48about it.
- 46:50>> I mean, I can't really speak in the
- 46:53broader context, but I'd say internal to
- 46:56our lab, we do have an AI policy
- 46:59and some of the things that the faculty
- 47:02director, Julian Nyarko, and I have
- 47:04thought about as we put it into writing
- 47:06was we do very much want them to be
- 47:09using the state of the art,
- 47:12but we do want to know why you use it
- 47:14and how you're using it. And
- 47:15importantly, we just want them to answer
- 47:17one single question, which is can you
- 47:20explain to us the design and the
- 47:22specific critical choices that you had
- 47:24to make as you were using this
- 47:26technology at every juncture.
- 47:29I think a lot of what we were seeing
- 47:31before we put the AI policy into place
- 47:33was that, you know, it's very quick for
- 47:36you to just throw it into, you know,
- 47:38Codex, Claude Code, whatever it might be
- 47:41and for you to produce
- 47:43everything and kind of just submit it
- 47:45during our research meetings. But when
- 47:47we actually drilled in on what the
- 47:48impact was to the research, why you made
- 47:51specific design choices, all of that
- 47:54kind of fell apart. And so, again, a lot
- 47:57of what we think is the responsibility
- 47:59of the researcher going forward is this
- 48:02design question, is the architecture
- 48:04questions. The execution, we are going
- 48:07to be well aware that that's going to be
- 48:09handled much better and much faster and
- 48:11much more productively using these
- 48:13tools, but again, like you have to
- 48:16really wrestle with the much harder
- 48:18problem, which is can you build that
- 48:20foresight into why you're doing these
- 48:22why you're making these decisions.
- 48:25>> Yeah.
- 48:27It's the
- 48:28I mean, there's a lot of interesting
- 48:29These are great questions, Cynthia. Um
- 48:32And what Mark was saying about the
- 48:33Turing test is being fundamentally
- 48:35deceptive, I've just kind of still been
- 48:36turning this one over my head. It's very
- 48:38interesting. I think, you know, when I
- 48:40think about my biggest worry for for
- 48:42like your generation right now
- 48:44educationally and what we're trying to
- 48:45provide, it's kind of related to this
- 48:48because the Turing test performance is
- 48:51strong through college. Like I think you
- 48:54could probably ask, you know, you can
- 48:56ask for help solving integrals and you
- 48:58can ask for all these anything where
- 49:00there's been a lot of homework problems
- 49:01made or whatever, probably you can get
- 49:03those answers. And so it's possible for
- 49:05some people, you know, and I actually
- 49:07have a kid who's at Stanford right now
- 49:09and someone told him once over dinner or
- 49:10something like I'm going to cheat
- 49:12basically on every everything for the
- 49:13rest of my time at Stanford and I'm
- 49:14never going to do a problem again. And
- 49:17which is kind of crazy, right? But the
- 49:20the damage there is that that person
- 49:21doesn't realize that that basically
- 49:23[clears throat] that means they'll leave
- 49:24Stanford with nothing nothing internal
- 49:27to show for it. And then they'll go out
- 49:29into a world where we're actually super
- 49:31bad at the Turing test. Like none of the
- 49:33problems that I am working on right now
- 49:35could you just like
- 49:37ask Claude to solve them, right? So it's
- 49:39it's interesting. I feel like there's
- 49:41this weird cliff where it's like AI
- 49:43could be ridiculously helpful to get you
- 49:45through school without thinking at all.
- 49:48And then it will become increasingly
- 49:50unreliable as soon as you walk off of
- 49:52this campus and try to do something new.
- 49:55So that is a really crazy thing that we
- 49:57have to figure out and I think
- 50:00to your points like you have to you
- 50:02definitely have to learn how to use it.
- 50:04I mean it would be crazy not to do that.
- 50:05But at the same time you have to learn
- 50:07everything else that you still
- 50:09that you came here for, right? That you
- 50:11came here for. Not just not just the
- 50:12badges and plus five and whatever the
- 50:14video game version of what Stanford the
- 50:16Stanford experience is. Not just you
- 50:18know, these these these little awards.
- 50:21In my lab we talk a lot about so we
- 50:23could talk about several places where
- 50:25this is problematic. The whole journal
- 50:26publishing system. Right now one of the
- 50:29biggest problems with AI is that you can
- 50:30just generate like it's like the most
- 50:32incredible like garbage generator
- 50:34in the history of civilization
- 50:36[laughter]
- 50:37and people can you know, I heard
- 50:39somebody just recently say who very very
- 50:42high place, oh now in five minutes I can
- 50:44just write a 50 page report and I send
- 50:46it to all my staff. It doesn't feel like
- 50:47they appreciate it though and I was
- 50:48like, no they don't
- 50:50because they're not idiots. But the
- 50:53but this idea of being able to generate
- 50:55so much. So I tell my lab again, I feel
- 50:57like transparency, quality, like really
- 51:01engaging deeply and kind of thinking
- 51:03about what your thought process is.
- 51:05Those are all really important. Uh in my
- 51:08lab we've established some things that I
- 51:10consider to be rules. I think we're
- 51:11still kind of getting there, but one is
- 51:14uh if you save all this time with AI,
- 51:17you have to give some back. And I say
- 51:19this is where I want you to give it
- 51:20back. Like if you save 90% of your time,
- 51:23I want to give 20% back to correct
- 51:26checking and checking yours and other
- 51:28people's work in this lab because there
- 51:30is such a high chance of error uh that
- 51:33arises, right? So just checking,
- 51:35double-checking,
- 51:37uh you making sure that we all agree
- 51:39that what goes out is still the highest
- 51:40quality. And I also say that they should
- 51:42be spending 20%
- 51:45I call it walking in the redwoods, just
- 51:47going out and thinking, thinking deeply
- 51:49about big problems. Like what's the most
- 51:51important thing you could possibly be
- 51:53doing right now with this time you're
- 51:55saving? It is not Wordle.
- 51:58It is not Sudoku.
- 52:00Not even the hard Sudoku.
- 52:02Um so those are interesting uh and
- 52:05important to recognize. And then the
- 52:07other thing that I think they should be
- 52:08giving back time to is just
- 52:11talking with other brilliant people who
- 52:14want to think about the deepest things.
- 52:16I was at a um I had the great
- 52:18opportunity to be at uh after-hours at
- 52:22the Museum of Modern Art in New York uh
- 52:25listening to a docent there talk about
- 52:28scientific revolutions. Like how do
- 52:30major revolutions happen in thought?
- 52:34And she talked about Picasso. It was
- 52:36amazing. Like I guess he he painted the
- 52:38first abstract art, and it was so crazy
- 52:41that nobody his agent wouldn't sell it,
- 52:43nobody liked it, so he retreated to
- 52:45Cubism. Isn't it crazy to think that
- 52:47Cubism was [laughter] like a retreat? A
- 52:49retreat from what he wanted to do,
- 52:51right? So she talked about these
- 52:53revolutions and what's required, and
- 52:55there's one part I will never forget
- 52:57because she talked about how important
- 52:59it was while there are people of
- 53:01extraordinary talent who can come up
- 53:03with things
- 53:04By and large, it's people interacting uh
- 53:07who are bright and excited mostly in
- 53:10social terms. And she said if they
- 53:12continue meeting socially, and this is
- 53:14like a direct quote, eventually
- 53:17the constraints of civilization begin to
- 53:19break down and radicalism can emerge.
- 53:22>> [laughter]
- 53:23>> Like literally my wife, you can ask her.
- 53:25Like in the middle of the night I'd wake
- 53:26up and be like, "Radicalism can emerge."
- 53:29Like that's what I want, right?
- 53:30[laughter] I want us to be breaking
- 53:31paradigms and doing new stuff. And so I
- 53:34think if we achieve those things, then
- 53:37AI will have done hopefully what we
- 53:39created it for, not to deceive ourselves
- 53:41or anyone else, but to enable us to do
- 53:44something that benefits others including
- 53:47ourselves that inspires, right? That
- 53:49brings something new, that creates
- 53:51connection, and where radicalism in
- 53:54thought and brand new paradigms can
- 53:56emerge that no one has even imagined
- 53:58right now because they didn't have the
- 53:59tools to do it.
- 54:03>> Okay, I feel like that's a good applause
- 54:05line, everyone.
- 54:11>> [applause]
- 54:11>> Um
- 54:12and I will run this mic over to you now
- 54:15if you would like to raise your hand and
- 54:17ask a question. All right, that's the
- 54:18first one I saw.
- 54:21>> Gosh, did you really make run?
- 54:23>> [laughter]
- 54:25>> So something that we're asked often in
- 54:27this class every week is to reflect on a
- 54:29skill that we have that we
- 54:33>> [laughter]
- 54:33>> that we don't want to lose um due to AI
- 54:36and how we're going to kind of foster
- 54:38that skill moving forward. And you know,
- 54:40hindsight 20/20, we can't really see in
- 54:42the future and know what we need, but
- 54:45all of you guys work with people our age
- 54:47and undergrads and grad students. What
- 54:49skills do you see um that you think are
- 54:52really important for us not to let
- 54:53degrade throughout these years in
- 54:55college and grad school?
- 55:02>> I there's like the easy answer and then
- 55:04there's like I'm trying to think of a
- 55:05much more clever but I mean obviously,
- 55:06critical thinking, right? Like in my
- 55:08field,
- 55:09um
- 55:10regardless of what you think that is, um
- 55:12the ability to interpret
- 55:16for yourself,
- 55:18um relying on your own mind, your own
- 55:20experiences, your own ideas, and
- 55:23bringing that to an object. That can get
- 55:27you not just far, that can get you just
- 55:29about anywhere. And that's kind of the
- 55:30seductive promise, right? Um even if you
- 55:32treat something like Claude as just a
- 55:34chatbot conversationalist, you start
- 55:36asking it questions, you're outsourcing,
- 55:38you're doing that cognitive outsourcing
- 55:40of exactly that faculty onto it. And
- 55:43that's that's fine if you've spent, you
- 55:46know, 40 years of your life honing that.
- 55:50But if you haven't, that's that's
- 55:52dangerous.
- 55:56>> So, in the legal space, a big thing that
- 55:59is being discussed right now is
- 56:01acquiring judgment. Uh and I'm sure that
- 56:03this is across every single vertical,
- 56:06um but a lot of people don't really
- 56:09narrowly define what is judgment and
- 56:11what that entails. And so, some of the
- 56:14things I've been thinking a lot is
- 56:16creating a necessary reference library
- 56:18of experiences. And so, I often share a
- 56:21lot with the students that work very
- 56:23closely with us is
- 56:25it's incredibly fine for you to go out
- 56:29and
- 56:31I don't know if your generation says
- 56:32this often, but it's like touch grass is
- 56:34something we often see.
- 56:37I think that's going to be really,
- 56:39really important. Um so, law school
- 56:42actually in the US is a second sort of
- 56:45post-grad degree. In other
- 56:46jurisdictions, people learn law like
- 56:48from bachelor's and onward.
- 56:51I think what's really fascinating about
- 56:54learning law in the US is a lot of
- 56:56people come with mixed experiences. And
- 56:59I hope that, you know, your experience
- 57:01is not just speed-running through school
- 57:04with all the cheat codes, it's that you
- 57:06are going to be present and bringing
- 57:09kind of even the wildest personal
- 57:12adventures, that's really going to make
- 57:15and start allowing you to create these
- 57:16good analogies, good references.
- 57:19And so maybe the ultimate thing is
- 57:21storytelling and building the right
- 57:23narratives.
- 57:25Yeah.
- 57:29>> I mean, I I think you're
- 57:31what you've been doing Megan to try to
- 57:33encode like expertise and intuition and
- 57:36judgment is really interesting. One of
- 57:38my students was asked this question who
- 57:40I consider to be really great with
- 57:42cloud code etc. She's fantastic. Um
- 57:46And she said scientific taste is going
- 57:48to be one of the most important things.
- 57:50Just like what do you solve and what
- 57:52don't you solve, right? What do you
- 57:54decide to work on? How do you make your
- 57:56mark? Uh I was I was at MIT giving a
- 58:00talk once and one of the students said,
- 58:02"It sounds like all you're talking about
- 58:03is solving unsolvable problems."
- 58:06And I said, "Yeah." That's all I'm ever
- 58:08going to talk about from now on because
- 58:11we live in the greatest age in history
- 58:13for solving unsolvable problems. And I
- 58:15could name them across the board like
- 58:17across basically every discipline.
- 58:20Things that people never ever thought
- 58:22had a chance. You know, the physical
- 58:24limit of light microscopy, right? Solved
- 58:26here, right? Like there there are many
- 58:28things.
- 58:29You know, optogenetics where our
- 58:30department has just been celebrating all
- 58:32last week cuz of cuz of Karl Deisseroth
- 58:34and the Nobel Prize. Like these things
- 58:35that nobody thought had a chance.
- 58:38Uh
- 58:39That's where I would expect a large
- 58:41portion of this group to be working,
- 58:43right? I I hope you will. I hope you
- 58:46will choose unsolvable problems to
- 58:48break. And that's going to
- 58:50require
- 58:53you know, some courage like an
- 58:54intellectual courage. It's definitely
- 58:56going to require having learned a lot of
- 58:57stuff, right? I I would say intellectual
- 59:00ambidexterity, the ability to take this
- 59:02field and this field and kind of know
- 59:03enough about both that you can you can
- 59:06pit them against each other or bring
- 59:07them together. Um
- 59:10I think the ability to work as part of
- 59:12an excited team that's it's huge. These
- 59:14habits of mind and they'll matter more
- 59:17than than the exact ideas. Like I I
- 59:20I used to teach this thing about
- 59:22engineering in medical like
- 59:24physiological systems and one year
- 59:26literally I said to the students, "Look,
- 59:27if you know anyone from last year, could
- 59:29you just go tell them that everything I
- 59:31taught them
- 59:32about stem cells turned out to be
- 59:34wrong."
- 59:35Like just in one year it changed that
- 59:37fast, right? AI is like that, right? So,
- 59:40you'll you'll want to learn to just be a
- 59:43a continually curious learner who's
- 59:46engaged and, you know, just trying stuff
- 59:49out.
- 59:53>> All right, the way in the front.
- 1:00:06>> So, I'm aware that AI has definitely led
- 1:00:08to kind of the quality of education
- 1:00:11among students and how students are like
- 1:00:13allocating the time for education. And
- 1:00:15but I also recognize that is very
- 1:00:17efficient when you are working on
- 1:00:19something that maybe you don't have the
- 1:00:20skills to do. So, what do you think is
- 1:00:23the boundary between using AI as a tool
- 1:00:26for like building knowledge and just for
- 1:00:28example, you might prompt it like, "Can
- 1:00:30you teach me how to build like a code
- 1:00:33that does a certain kind of task?"
- 1:00:35Versus just telling the AI and prompting
- 1:00:37it to do that task.
- 1:00:42>> I have I have a couple thoughts. It's a
- 1:00:43It's a great question and I love it cuz
- 1:00:45it's really like, you know, directly
- 1:00:48focused to this group. Um,
- 1:00:51kind of similar to the the
- 1:00:52recommendation letter question. Here's
- 1:00:54another example that might be
- 1:00:55interesting. Let's say that we we all
- 1:00:57signed up for like a rec class, right?
- 1:00:58We're going to go to the weight room and
- 1:01:00learn how to lift weights. And so, we
- 1:01:02went to the rec room and for 1 hour we
- 1:01:03just were watch robots lifting the
- 1:01:05weights.
- 1:01:06>> [laughter]
- 1:01:07>> Right? So, what would that accomplish?
- 1:01:10Right? What would that accomplish?
- 1:01:12Uh, Uh, I think it's important to
- 1:01:14recognize that you're lifting those
- 1:01:15weights for a reason. It's not because
- 1:01:17we we live somewhere where people have
- 1:01:19to be moving weights, right? We're we're
- 1:01:21lucky. We're just doing it for our own
- 1:01:23personal health, our own personal
- 1:01:25benefit. And hopefully it's it's obvious
- 1:01:28from that example that having a robot do
- 1:01:30it is
- 1:01:32absurd, right? It's completely absurd.
- 1:01:35Uh, I think in the same way we will have
- 1:01:38to figure out ways to work with AI that
- 1:01:40that aren't basically having a robot
- 1:01:42lift our weights.
- 1:01:44Um,
- 1:01:45I'll say, like I was very reluctant to
- 1:01:47write anything with AI and especially to
- 1:01:49edit anything generated with AI. Like I
- 1:01:51see recommendations letters written with
- 1:01:53AI, for sure. I also see, um,
- 1:01:56whole papers written, people send me
- 1:01:58stuff, and I always, if I can detect it,
- 1:02:01which is probably 100% using this neural
- 1:02:04network, all of you have it. It's still
- 1:02:06the best one for now.
- 1:02:09Uh, so, um,
- 1:02:11then I just absolutely won't won't edit
- 1:02:13or read it, especially if it's like 100
- 1:02:14pages, which is typically one of the
- 1:02:16tells. Um,
- 1:02:18>> [laughter]
- 1:02:18>> but I did start writing I did decide,
- 1:02:21exactly to your question, um, I decided
- 1:02:24I wanted to learn how I could write
- 1:02:26better, like higher quality. And so I
- 1:02:28did, uh, over a couple weeks ago I
- 1:02:31started writing one of our papers, and I
- 1:02:33told AI I don't want like I told Claude,
- 1:02:35I don't want you to write this for me.
- 1:02:36It's all going to be my voice, but I do
- 1:02:38want you to help me.
- 1:02:40And so I gave it the flow of the paper
- 1:02:41that I was considering. And then I
- 1:02:43started as I as I was trying this more
- 1:02:45and more, I realized something, and I'll
- 1:02:47just share with you what I did.
- 1:02:49I wrote a paragraph, and then what I
- 1:02:51said was, "Okay, I want for each
- 1:02:54sentence in this paragraph, I want you
- 1:02:56to tell me
- 1:02:57the three things that I should be citing
- 1:03:00that I haven't thought about. I want you
- 1:03:02to tell me the three things that would
- 1:03:04most destroy this argument. And I want
- 1:03:08you to tell me three things like an
- 1:03:10analogy to another field that I've never
- 1:03:12thought about before."
- 1:03:14And so we just went through like that.
- 1:03:16Like and I would we had a running total
- 1:03:18of papers. I said, "I don't trust you
- 1:03:20with papers. I'm just going to read
- 1:03:21these and I'm going to tell you if
- 1:03:22they're wrong." And so, it started
- 1:03:24looking them up and paying paying a
- 1:03:26little more attention. And so, long
- 1:03:28story short, after 2 days, I had written
- 1:03:32four paragraphs
- 1:03:34with AI. And
- 1:03:37um
- 1:03:38probably the best paragraphs I have ever
- 1:03:40written. Right? All written by me, but
- 1:03:43with a lot of just thinking back and
- 1:03:45forth and reading new papers.
- 1:03:47To this point about a professor asking
- 1:03:50you to write your own letter, I could
- 1:03:51have written those four paragraphs
- 1:03:53probably in an hour because I know what
- 1:03:55I knew what I had in my head, and I
- 1:03:57basically know all the citations that I
- 1:03:59would consider to be normal for my
- 1:04:01field.
- 1:04:02But, having this like foil that's really
- 1:04:04pushing me to go harder was was
- 1:04:07powerful. And I think it's a it's almost
- 1:04:09like an inside-out approach of what what
- 1:04:12I often see in the AI space, right?
- 1:04:15Really, what happened was I was just
- 1:04:17having guided deep thinking for two for
- 1:04:202 days, right? And I came up with these
- 1:04:22four paragraphs that were
- 1:04:24were very meaningful and intense. So, so
- 1:04:26maybe that's a thought. Like, if if
- 1:04:28there's a way for students to almost
- 1:04:30think of the AI is like
- 1:04:32"Okay, now we've got the weights, but I
- 1:04:34somehow I'm going to be able to develop
- 1:04:35even instead of lift higher weights,
- 1:04:37develop stronger muscles because of my
- 1:04:40use, then that's a really good news."
- 1:04:45>> For me, it goes back to something I said
- 1:04:47earlier, which is that deciding what you
- 1:04:49care about, right? You can't No one can
- 1:04:52do everything all the time. Uh you know,
- 1:04:56some of us are STEM majors, some of us
- 1:04:58are other kinds of majors. Um I can
- 1:05:00appreciate organic chemistry, but I'm
- 1:05:02not going to sit and solve, you know, a
- 1:05:04bunch of problems.
- 1:05:06Um and so, for example, I'm designing
- 1:05:09So, in Cesto, we have a critical gaming
- 1:05:10lab, and I'm working with um a colleague
- 1:05:13to design uh a class uh that that
- 1:05:16critically thinks about video games. And
- 1:05:18one of the activities that we really
- 1:05:19want the students in this class to do um
- 1:05:22is to design their own game according to
- 1:05:25you know the principles that we're
- 1:05:27talking about thinking narratively
- 1:05:29thinking
- 1:05:30gameplay wise etc.
- 1:05:32But it's not a CS class right like we do
- 1:05:35not want to teach them the code to
- 1:05:37create that but we want the experience
- 1:05:39them to experience having created it and
- 1:05:42then actually playing through it and
- 1:05:44that's where
- 1:05:46AI can really really help because there
- 1:05:49is a dedicated AI model for creating
- 1:05:52fairly simple games
- 1:05:54and the emphasis in the classes is on
- 1:05:56the creation process and something that
- 1:05:58thoughtful creation process not on the
- 1:06:01coding. So in that particular instance
- 1:06:03offloading
- 1:06:05the actual mechanical creation of the
- 1:06:08game to an engine which is not something
- 1:06:11that we're teaching or testing in the
- 1:06:12class that has enabled us to do so much
- 1:06:16more than we would have been able to do
- 1:06:18otherwise with the things that we
- 1:06:20actually care about.
- 1:06:22Whereas I can imagine in in a very
- 1:06:24different class you could have almost
- 1:06:25the opposite but in all cases it it if
- 1:06:29we use it I think cleverly and and are
- 1:06:31really clear about what we want to do in
- 1:06:33a like a class or in a research context
- 1:06:35it can expand rather than contract what
- 1:06:39we can do.
- 1:06:41>> I I think that YouTube Coursera has
- 1:06:44basically democratized education in a
- 1:06:46lot of ways that AI is basically
- 1:06:49accelerating or furthering.
- 1:06:51Something that Marcus you mentioned
- 1:06:52earlier is at the end of the day it
- 1:06:54boils down in my mind to curiosity and
- 1:06:57in this case it's not just about the
- 1:06:58first step of learning how to do
- 1:07:00something that is outside of
- 1:07:03your typical sort of skill set but it's
- 1:07:05what you want to do afterwards with it.
- 1:07:08It could be very simple like I just need
- 1:07:10to figure out how to fix my toilet and
- 1:07:12become a plumber for the day if I don't
- 1:07:14have you know access to that but I think
- 1:07:18if it's something where you want to tap
- 1:07:19into a neighboring field maybe it
- 1:07:22encourages you to actually speak with
- 1:07:24colleagues that are more well-versed in
- 1:07:26this space because you took that step
- 1:07:28and started sort of the beginner
- 1:07:30foundational skills in that realm. So, I
- 1:07:32think it's, you know,
- 1:07:34a two-step process.
- 1:07:36I wouldn't treat it as, you know, a
- 1:07:37one-and-done.
- 1:07:40>> Yeah, I think we have time for one more.
- 1:07:44All right, I'll go ahead and pass.
- 1:07:53>> Hi, thank you all for coming. I have a
- 1:07:55specific question for Dr. Covert. Is
- 1:07:58that correct? Yeah.
- 1:08:01It's on like one of the points on one of
- 1:08:03of your AI's obsessions like regarding
- 1:08:06like AI in education.
- 1:08:08I'm very curious about the educational
- 1:08:10tools and like particularly which
- 1:08:12features or design choices like may have
- 1:08:15contributed to the negative outcomes and
- 1:08:17your thoughts on AI for learning in
- 1:08:19general. And like what features or
- 1:08:22design choices could have been
- 1:08:23implemented so students could like
- 1:08:25quote-unquote like lift their own
- 1:08:27weights because I know like I'm in a CS
- 1:08:30class right now and
- 1:08:32the way that we're allowed to use AI for
- 1:08:35our learning is that they give us the
- 1:08:36agents MD file which we're allowed to
- 1:08:39put in whatever like AI we want to use
- 1:08:41that put like guardrails on our
- 1:08:43learning. So, I'm just curious about
- 1:08:45your thoughts on like educational AI
- 1:08:47tools in general.
- 1:08:49>> Yeah, that's a great. Well, wait. Wait.
- 1:08:51Give her back the mic. What's your name?
- 1:08:54Nice to meet you and feel free to call
- 1:08:55me Marcus. No one calls me Dr. Covert.
- 1:08:58>> [laughter]
- 1:08:59>> Okay, great. Like I think this is such
- 1:09:02an interesting question and it just
- 1:09:04gives me kind of a chance to do a little
- 1:09:06bit of a rah-rah moment for Stanford in
- 1:09:08the sense that I mean what you're
- 1:09:10hearing here are, you know, some
- 1:09:13incredible minds in English and in law
- 1:09:16and and I happen to know like Cynthia
- 1:09:18and others are so dedicated to making
- 1:09:20sure that you get the most out of out of
- 1:09:23AI and your education, you know, which
- 1:09:26is the more important thing in this
- 1:09:28rapidly developing field. So, I think
- 1:09:31it's it's pretty exciting to be here at
- 1:09:33a time where things are in flux and you
- 1:09:36have a chance to just be right here
- 1:09:38watching it. So,
- 1:09:40I grew up here. I grew up a mile away
- 1:09:42from here, right? So, right by Midtown,
- 1:09:44if any of you know that um
- 1:09:46So, and I remember when I was young, uh
- 1:09:50web search was like a big thing, right?
- 1:09:52And one thing that was really
- 1:09:53interesting is
- 1:09:55you would never even know this, but
- 1:09:56there was like Lycos and WebCrawler and
- 1:09:58Ask Jeeves. There were like a million
- 1:10:00and then Google showed up with the new
- 1:10:01PageRank algorithm and they all just
- 1:10:04disappeared, right?
- 1:10:05>> [laughter]
- 1:10:06>> So, I do I can easily imagine that what
- 1:10:09we're seeing right now is just like a a
- 1:10:11pre-shock of kind of the the the
- 1:10:14earthquake, let's say, that's going to
- 1:10:16happen in AI.
- 1:10:18And watching everybody kind of test and
- 1:10:21having people who
- 1:10:23who have innovated as far as you can in
- 1:10:25this space, who are not only doing that
- 1:10:27in their labs, but also in the
- 1:10:28classroom, is something very unique that
- 1:10:31you have at Stanford.
- 1:10:33When I first got here and got a speech
- 1:10:35about teaching from the provost, they
- 1:10:37said, you know, we want you to be a
- 1:10:39great teacher, but we also want to make
- 1:10:41sure you never forget that you have the
- 1:10:44power to give Stanford students the
- 1:10:45ability to know what happened this
- 1:10:47morning in the cutting edge of the field
- 1:10:50and you should share it with them. And
- 1:10:51so, that's that's my recommendation to
- 1:10:54you is go take those classes where
- 1:10:56people are literally just leaning in so
- 1:10:59hard creating this new frontier and
- 1:11:02that's going to be the best way to just
- 1:11:04be engaged and energized and it's
- 1:11:07something you can only get really on
- 1:11:08this campus in many ways. So,
- 1:11:12>> All right. Well, can we get a round of
- 1:11:13applause for everyone?
About this transcript
This page contains the full transcript of YouTube transcript (PeRJkvzIEts) , generated from the public captions YouTube serves with the video. The transcript has 12,383 words across 1,894 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.