YouTube2Text

YouTube transcript (PeRJkvzIEts) — Transcript

12,383 words · 1,894 segments · language en · Watch on YouTube

Full transcript

  1. 0:05I'm just going to go ahead and introduce
  2. 0:07our guest speakers and [clears throat]
  3. 0:09I'm very excited to have talk about
  4. 0:12their use of AI as part of their
  5. 0:15academic research.
  6. 0:17First, we're going to hear from Mark. He
  7. 0:19is a professor of English and I won't
  8. 0:23read his whole bio, but just look at the
  9. 0:25size of it. We should all be very
  10. 0:26impressed. And this will be true for all
  11. 0:29our speakers. Uh Megan is joining us
  12. 0:31from uh Frontiers Technology Lab in the
  13. 0:35Law School. So, um the whole focus of
  14. 0:38their work is on using AI in the
  15. 0:41furtherance of practice of law.
  16. 0:44And Marcus joins us from the depart-
  17. 0:47department of bioengineering
  18. 0:49and he uses uh models as part of his um
  19. 0:53work in understanding the cell and
  20. 0:55bioengineering.
  21. 0:56Uh so, without further ado, I will hand
  22. 0:59things off to our first speaker, Mark.
  23. 1:01>> Thank you all so much for um I guess
  24. 1:03coming to your class. Thank you more for
  25. 1:06having me and giving me the opportunity
  26. 1:07to talk to you a little bit about what I
  27. 1:10do.
  28. 1:11Um so, I am a professor in the English
  29. 1:13department, although technically I'm a
  30. 1:15professor of digital humanities. Um I
  31. 1:17also run a research center called the
  32. 1:20Center for Spatial and Textual Analysis
  33. 1:21over on the top floor of Wallenberg, um
  34. 1:24which has a very robust internship
  35. 1:26program uh starting uh I I about 25
  36. 1:30undergraduates that we hire per year to
  37. 1:31work on our projects, um as well as the
  38. 1:34associate director of data science for
  39. 1:36artistic and cultural analysis. So, all
  40. 1:39of that's a long way of saying that if
  41. 1:40anything I say today resonates, there
  42. 1:43are plenty and plenty and plenty of
  43. 1:44opportunities um for you to do this for
  44. 1:47yourself and I would be thrilled to talk
  45. 1:49to all of you about that.
  46. 1:52Um so, what kind of work do I do? Well,
  47. 1:54I think of it as the um meeting of data
  48. 1:57science and humanities.
  49. 2:00A kind of reading at scale, um
  50. 2:02particularly in my work where I think
  51. 2:04about large analyses of corpora. Um and
  52. 2:08that looks like a number of different
  53. 2:09things. It can look like the analysis of
  54. 2:11thousands and thousands of texts,
  55. 2:13looking for machine-recognized patterns
  56. 2:17uh that we've trained uh the different
  57. 2:19kinds of models to find for us. Or it
  58. 2:21can look like the in-depth analysis of a
  59. 2:23very specific text, in this case just
  60. 2:26one novel, but along the axes that are
  61. 2:29just unaccessible to a human reader.
  62. 2:31Here the result uh of a model that was
  63. 2:33changed to recognize all of the
  64. 2:35different genres except for the one that
  65. 2:36that book is actually written in. Um
  66. 2:39this looks different in a number of
  67. 2:41different hands. There are There are
  68. 2:42mapping projects that use GIS data um to
  69. 2:46think, for example, about geographical
  70. 2:47investment in a corpus of novels, or
  71. 2:50even things that look at early uh
  72. 2:52modern, in this case, correspondence
  73. 2:54networks uh trying to find patterns. Um
  74. 2:58but overall, since the subject of today
  75. 3:00is artificial intelligence, um this
  76. 3:02obviously comes into play in my world uh
  77. 3:06a lot, both as a method but also as an
  78. 3:09object of analysis. And the project that
  79. 3:11I'm going to share with you very quickly
  80. 3:13um will talk about uh how we how we
  81. 3:17think about it. Um
  82. 3:19and so what I really want to sort of
  83. 3:20hammer home is that for us, artificial
  84. 3:22intelligence has a place within the
  85. 3:24pipeline. Certainly, machine learning
  86. 3:26has a big place in the pipeline, but
  87. 3:28actual generative models have a weird um
  88. 3:31relationship, particularly with
  89. 3:33humanities research. Um and so I want to
  90. 3:36think about this carefully, right? We
  91. 3:38need to deploy it, but as a kind of
  92. 3:40middle layer between a strong
  93. 3:42statistical foundation for what we do
  94. 3:44with corpora, and a human-led
  95. 3:46understanding,
  96. 3:48um in my case, of literature, but a
  97. 3:49human interpretation
  98. 3:51of both the underlying data and what the
  99. 3:53model is doing. Um and so the project
  100. 3:56that I I wanted to kind of show to you
  101. 3:58as an example of the kind of work that I
  102. 4:00do um is called Style in the age of AI.
  103. 4:03It really takes a hard look at what we
  104. 4:06think of as AI style. And as you can
  105. 4:09see, this is a subject of a lot of think
  106. 4:12pieces recently.
  107. 4:13People arguing that AI style is ruining
  108. 4:16us, that we're all adopting it, that
  109. 4:18it's making our writing bad. But what I
  110. 4:20want to
  111. 4:21think about with this project is that
  112. 4:23it's not just the case of style as
  113. 4:26style. What we can think about in the
  114. 4:27humanities is the relationship between
  115. 4:30style and meaning. And so this project
  116. 4:33really takes a hard look at what happens
  117. 4:36when generative AI style infiltrates the
  118. 4:41way in which we make meaning. And for
  119. 4:42humanities, this is particularly
  120. 4:44important because in the sciences, we
  121. 4:46can think about meaning being made with
  122. 4:48numbers, right? It's the the material of
  123. 4:50research is the data.
  124. 4:52But in the humanities, we make meaning
  125. 4:54with words. If you read like a
  126. 4:56traditional English paper, not one of my
  127. 4:58papers, but a traditional English paper,
  128. 5:00what you find is a lot of words and
  129. 5:02meaning emerging from that. So what does
  130. 5:04that mean when we think about using
  131. 5:06generative AI to communicate it? So in
  132. 5:09this particular project, we're asking,
  133. 5:11do disciplines have style? That is, do
  134. 5:15historians write like a historian? Do
  135. 5:17psychologists write like a psychologist?
  136. 5:19Do computer scientists write like a
  137. 5:21computer scientist? Regardless of what
  138. 5:23they're writing about. And so for this
  139. 5:26particular project, we got a big corpus.
  140. 5:28This is just a pilot version of the
  141. 5:30larger project I'm currently running. So
  142. 5:32this is about 2,000 articles. We're
  143. 5:33currently working on a 10,000 article
  144. 5:35sample across 18 disciplines.
  145. 5:38And we wanted to compare style and
  146. 5:41meaning. So a a really simple way to
  147. 5:43sort of access aboutness is a topic
  148. 5:45model. Here, each dot represents an
  149. 5:46article in our corpus.
  150. 5:48It's placed within this TSNE graph shows
  151. 5:51roughly what it's similar to topic-wise
  152. 5:54in terms of its topical composition. And
  153. 5:56the disciplines come out really nicely.
  154. 5:58In particular, I want you to take a look
  155. 6:00at these two,
  156. 6:02philosophy and computer science. Nothing
  157. 6:04quite so far apart.
  158. 6:07Now we want to compare this to how these
  159. 6:10two disciplines, not just what they
  160. 6:11write about, but what they actually
  161. 6:14how they write. So not what they write
  162. 6:15about, but how they write. And so we can
  163. 6:18also take the same corpus of 2,000
  164. 6:20articles and we can classify using only
  165. 6:24what I'm calling stylistic features.
  166. 6:25This is just grammar along with things
  167. 6:28like sentence length, number of named
  168. 6:30entities, etc. etc. So no information on
  169. 6:33what the articles are about, just how
  170. 6:35they're written, and we get to about a
  171. 6:3743%
  172. 6:39success rate. That doesn't sound great
  173. 6:41and in fact we're working on improving
  174. 6:43it, but when you consider that, you
  175. 6:44know, at chance we'd be at 5%, this
  176. 6:46isn't that bad. It turns out that even
  177. 6:48at a rough level, anthropologists write
  178. 6:50like anthropologists.
  179. 6:52So what does this mean in terms of AI?
  180. 6:54What we decided to do is test an
  181. 6:58LLM model, which are particularly noted
  182. 7:01for doing interesting stylistic things,
  183. 7:03by doing what we're calling a style
  184. 7:05transfer. That is by carefully prompting
  185. 7:07here a local model, we said, "Okay, take
  186. 7:10those two farthest apart fields, right?
  187. 7:12What happens when you get an LLM to
  188. 7:14write a philosophy article, which you
  189. 7:16see on the left, as if it were a
  190. 7:18computer science article, which you see
  191. 7:20on the right?" So that is a philosophy
  192. 7:22article written like a computer
  193. 7:24scientist.
  194. 7:25In our initial experiments using this
  195. 7:28local model, it turned out quite well.
  196. 7:30It's not perfect yet. If we reclassify
  197. 7:33it using the same model, most of what
  198. 7:35used to be philosophy is now classified
  199. 7:37as computer science, but but there's
  200. 7:39still a long tail there that's that
  201. 7:40we're still working on. The more
  202. 7:42important bit, interrogating style and
  203. 7:44meaning, is if we go back to a different
  204. 7:48topic model.
  205. 7:49Again, same corpus of 2,000 articles
  206. 7:51except this time we included all of the
  207. 7:55philosophy articles that we rewrote as
  208. 7:58computer science. And so just to
  209. 7:59reorient you, there is in gray our
  210. 8:01philosophy, in green our computer
  211. 8:03science, we suddenly have that new
  212. 8:06group, which is our philosophy written
  213. 8:10as computer science. And again, this is
  214. 8:11not based on grammar at all. This is
  215. 8:13purely content words. And what it
  216. 8:15suggests is that the original philosophy
  217. 8:19articles that we rewrote using LLM's
  218. 8:21probing deeply into what they do with
  219. 8:23style, actually moves the content
  220. 8:26from philosophy just a smidge, but
  221. 8:29noticeably and meaningfully towards
  222. 8:31computer science, suggesting there is in
  223. 8:33fact a relationship between style and
  224. 8:36meaning at least as we can find
  225. 8:37computationally. And so this is just a a
  226. 8:40rough overview, a very quick overview of
  227. 8:43the kind of project that we do, the way
  228. 8:45we interrogate how LLM's are working,
  229. 8:48how we use them, but how we think about
  230. 8:50the kind of work they do in making
  231. 8:52meaning particularly in the humanities.
  232. 8:55Thank you very much.
  233. 8:57>> Hello everyone. So I'm Megan. I'm the
  234. 9:00executive director of a research lab at
  235. 9:02the law school. We're called the Legal
  236. 9:03Innovation Through Frontier Technology
  237. 9:05Lab. And so since it's quite the
  238. 9:07mouthful, we call it Lift Lab. And part
  239. 9:09of what we do is we think about applied
  240. 9:11AI in the legal domain, in particular in
  241. 9:14private practice. So in large law firms
  242. 9:16as well as corporate legal departments.
  243. 9:18How many of you are familiar with what
  244. 9:20lawyers actually do?
  245. 9:24Oh, quite some hands. Well, actually our
  246. 9:27lab basically asks this question, how do
  247. 9:29you define quality in the legal space?
  248. 9:31And largely it's because there is a
  249. 9:33sense of kind of alchemy behind what
  250. 9:35attorneys do. Um in economics terms it's
  251. 9:37kind of a credence good. You're not
  252. 9:39always sure exactly what you are
  253. 9:41purchasing.
  254. 9:42Um our lab kind of looks at a couple of
  255. 9:44pillars including evaluation, uh
  256. 9:47delivery of legal services, as well as
  257. 9:50legal education and training, and
  258. 9:51finally kind of what we call advanced
  259. 9:53methods. In evaluation, we're not only
  260. 9:56thinking about, you know, how closely
  261. 9:58does AI do legal work, but actually what
  262. 10:01exactly is important about evaluation in
  263. 10:04a domain where ground truth quality is
  264. 10:07not really there.
  265. 10:08How many of you, you know, are more
  266. 10:10familiar with the fact that benchmarks
  267. 10:12kind of look into say rubrics for
  268. 10:14example?
  269. 10:15I assume those that have technical
  270. 10:18backgrounds. Yeah, so in legal what we
  271. 10:20call a judgement rich domain, this is
  272. 10:22incredibly difficult because the model
  273. 10:25of the firm is that it indexes heavily
  274. 10:27on the individual. I will choose to work
  275. 10:30with Megan, I will choose to work with
  276. 10:32Mark, and we build sort of these
  277. 10:34long-term relationships. So how do we
  278. 10:36actually understand what exactly is the
  279. 10:38value that the profession has to offer?
  280. 10:41And so some of the research that we
  281. 10:42recently released known as judgement
  282. 10:44bench reckons with how we should think
  283. 10:46about evaluation. And so there's
  284. 10:48comparison between rubrics versus using
  285. 10:50preference ranking. So if you have two
  286. 10:52versions of something, which one is
  287. 10:54better? And it turns out in our paper
  288. 10:56that um
  289. 10:57preference rankings had far outperformed
  290. 11:00rubrics because the lawyer intuition,
  291. 11:03something about that was incredibly
  292. 11:05powerful. So some of the work that we're
  293. 11:07thinking about is if you have a large
  294. 11:09preference data set, can we actually
  295. 11:11inductively gather a rubric from it for
  296. 11:14post training? The second kind of pillar
  297. 11:16that we look at is the delivery of legal
  298. 11:18services. So this is kind of a domain
  299. 11:21where many of the
  300. 11:23the sort of primary business model that
  301. 11:25they work with is the billable hour. And
  302. 11:28as you know, the billable hour kind of
  303. 11:30runs into tension when you have AI that
  304. 11:32is vastly improving productivity. And so
  305. 11:35when you have, you know, lawyers that
  306. 11:37typically charge say 10 hours for this
  307. 11:39particular work product, and AI could
  308. 11:41make it simplified to about two hours,
  309. 11:44what does that necessarily mean for
  310. 11:46their margins? And of course, nobody is
  311. 11:48crying tears for big law departments
  312. 11:50that are already earning a ton of money,
  313. 11:53but at least I think it allows us to get
  314. 11:55a sense, you know, when it comes to high
  315. 11:58demand knowledge work, are we thinking
  316. 12:01about where these revenue models are
  317. 12:03going? And more importantly, what is the
  318. 12:05AI exposure in the work all together?
  319. 12:08And so some of the projects that we're
  320. 12:09thinking about here is we're doing an
  321. 12:11observational study. How many of you're
  322. 12:13familiar with any of the legal AI
  323. 12:15providers like Harvey or Legora?
  324. 12:20Great. Great to know that they're
  325. 12:21somewhat popular. Um
  326. 12:24basically, we're going to be working
  327. 12:25with some of these legal AI providers to
  328. 12:27get an inside look exactly at how law
  329. 12:30firms are using these providers. So,
  330. 12:33what are sort of the edit histories we
  331. 12:34can go into, how much of certain work is
  332. 12:37being edited on top of or intervened
  333. 12:39with humans, and how much of it is
  334. 12:41actually never intervened at all.
  335. 12:44Through this, we're going to try to
  336. 12:45understand is there no intervention
  337. 12:47because agents have gotten to a place
  338. 12:50where they are perfectly fine with
  339. 12:52executing legal work, or is it that this
  340. 12:55work is simply not that important and
  341. 12:57that we actually have to push uh up the
  342. 12:59high-value work further up the pipeline.
  343. 13:02Our third pillar is actually something
  344. 13:04that I'm most sort of passionate about,
  345. 13:06which is the legal education and
  346. 13:08training aspect. So, in the legal
  347. 13:10profession, a lot of what you learn in
  348. 13:13law school, there tends to be a bit of a
  349. 13:15disconnect into practice.
  350. 13:17Um a lot of lawyers become the amazing
  351. 13:20lawyers that they are because they
  352. 13:22learned by doing. Um and a lot of this
  353. 13:25experiential learning is coming into
  354. 13:27tension because the traditional sort of
  355. 13:30everyday day-to-day work is going away.
  356. 13:33So, what happens when the work that you
  357. 13:36had a lot of muscle reps into and you
  358. 13:38dedicated a lot of young attorneys to
  359. 13:40help support kind of naturally
  360. 13:42vaporizes. How do you train in that
  361. 13:44regard? So, we're working on a lot of
  362. 13:46AI-based simulation training. We have in
  363. 13:49particular a couple of platforms. So, we
  364. 13:50built a multi-agent negotiation
  365. 13:52simulator. We also built a deposition
  366. 13:55simulator. How many of you have seen
  367. 13:57Suits? I feel like your generation likes
  368. 14:00Suits.
  369. 14:01Um
  370. 14:02basically, if you've ever seen them do a
  371. 14:05deposition poorly, I think, um you get
  372. 14:08to practice. And actually, this tool is
  373. 14:10open source. So, in case you're curious,
  374. 14:12you can all uh try it on our lab's
  375. 14:15website. Um, the idea is is that in a
  376. 14:18profession like the legal space, because
  377. 14:20you're spending all this time in a
  378. 14:22high-stakes environment, you often can't
  379. 14:25afford to lose. There are a lot of
  380. 14:26senior attorneys right now that still
  381. 14:28are kept up at night by those small
  382. 14:30mistakes that they made when they were
  383. 14:32juniors. And so we are trying to kind of
  384. 14:34not only help support kind of the
  385. 14:37training in areas that might go away,
  386. 14:40but we're also kind of creating new
  387. 14:41environments. So you can imagine, um,
  388. 14:44you can actually do a deposition as many
  389. 14:46times as you like. Basically our
  390. 14:48platform is you can upload any discovery
  391. 14:50document or any witness statement. It
  392. 14:52creates an AI witness on the fly, and
  393. 14:55then you can actually adjust their
  394. 14:57personalities. We work with some senior
  395. 14:59litigators here where, um, there's 14
  396. 15:02typical archetypes that they've
  397. 15:04imagined: combative, evasive, forgetful.
  398. 15:07And you can play on any of these
  399. 15:09particular sort of configurations. It
  400. 15:11dials the personality all the way up or
  401. 15:14down, and so you can get the most
  402. 15:16caricatured version of a combative
  403. 15:18witness. And the goal is that through
  404. 15:20your line of questioning that you're
  405. 15:22able to steer and get the responses that
  406. 15:24you need. Um, as we were building this
  407. 15:27out, one of the uh, attorneys that we
  408. 15:29actually worked with, they told us this
  409. 15:31kind of funny anecdote that continues to
  410. 15:33haunt them, which was a deposition that
  411. 15:35they had done as a young attorney. And
  412. 15:37it turns out that they could not
  413. 15:39actually get a single impeachment. So in
  414. 15:42the legal context, that means sort of
  415. 15:44locking in that particular fact you want
  416. 15:46for your case.
  417. 15:47Um, they couldn't get a single
  418. 15:48impeachment because the opposing counsel
  419. 15:50was incredibly distracting. So the
  420. 15:53opposing counsel with every question
  421. 15:55that this attorney said did a weird
  422. 15:57laugh, like
  423. 15:59And because of that, um, he was
  424. 16:01incredibly distracted, and then he had a
  425. 16:04dramatic failure in being able to
  426. 16:06complete the deposition. We want to
  427. 16:08actually use simulation as an
  428. 16:09opportunity not only for as a
  429. 16:12pedagogical tool for rehearsal, we also
  430. 16:15want to simulate what should not happen
  431. 16:17and what we want to avoid happening. And
  432. 16:19so, that's kind of one aspect that we've
  433. 16:20been thinking about. And the last pillar
  434. 16:22is what we call the advanced methods.
  435. 16:24This is kind of a hodgepodge. It's a
  436. 16:26grab bag. It's really kind of in short,
  437. 16:30what's made available for the legal
  438. 16:32profession in light of generative AI.
  439. 16:34So, one example is that we've been
  440. 16:36working on something called partner
  441. 16:38personas or legal personas. And this is
  442. 16:40basically the personalization of an LLM
  443. 16:43to behave like the specific individual.
  444. 16:46For those of you who are in CS, maybe
  445. 16:48you're familiar with Simile
  446. 16:50from Michael Bernstein and their lab
  447. 16:52that's since become a company.
  448. 16:55Here, what we're really fascinated by is
  449. 16:57can we create almost AI twins of senior
  450. 17:01partners.
  451. 17:02So far in some of our research, what we
  452. 17:04discovered was that AI is very good at
  453. 17:06role-playing, but it really kind of
  454. 17:08lends itself into stereotyping and
  455. 17:10caricatures really quite quickly. How do
  456. 17:13you actually steer at a granular level
  457. 17:16an LLM to become exactly like the
  458. 17:19individual. So, if I was a negotiator
  459. 17:21and you say that I'm an aggressive
  460. 17:22negotiator, it doesn't mean that I'm so
  461. 17:25aggressive that I will haunt your
  462. 17:26dreams. It means that actually I tend to
  463. 17:30lean more aggressive, and it could be
  464. 17:31circumstantial depending on, you know,
  465. 17:34what type of specific where I am in the
  466. 17:37pipeline of the negotiation, and what I
  467. 17:40want to do to represent my clients. So,
  468. 17:41here we kind of think a lot about can
  469. 17:44you actually use numerical steering to
  470. 17:46get that level of granularity. So, all
  471. 17:49in all, if there is any interest in kind
  472. 17:51of working in sort of a knowledge-rich
  473. 17:53space, or you have more questions, feel
  474. 17:55free to reach out.
  475. 17:57>> I wouldn't say like well, quick
  476. 18:00introduction. So, I'm a
  477. 18:02I'm the chair of the bioengineering
  478. 18:03department, which is just over here.
  479. 18:08Let me see real quick.
  480. 18:10And
  481. 18:11you know, just like a lot of us so we
  482. 18:13are doing plenty of plenty of AI stuff
  483. 18:16here. It was great. It's actually I love
  484. 18:17that I'm kind of together with other
  485. 18:19corners of campus because that's where
  486. 18:20you get to get to really generate ideas
  487. 18:23and think of new things. But, um
  488. 18:25I've got kind of I can I'm going to tell
  489. 18:27you kind of my background and how I'm
  490. 18:28thinking about it and then really
  491. 18:29instead of saying what I'm currently
  492. 18:31doing too much, I'm going to talk more
  493. 18:33about questions I have and what I'm
  494. 18:34thinking about. Um
  495. 18:37so I think a lot about disease and I
  496. 18:39think you know, it can be really easy if
  497. 18:42you read the papers, you know, you'll
  498. 18:43always read about a cancer gene or an
  499. 18:45Alzheimer's gene or this or that gene.
  500. 18:47And yet when it comes to the cures for
  501. 18:49all those things, they aren't there,
  502. 18:51right? So, you might wonder why and the
  503. 18:53reason is that they're not you know,
  504. 18:54cancer's not a one gene problem. Right?
  505. 18:57It's like a hundred or a thousand of
  506. 18:59genetic but also epigenetic and other
  507. 19:01environmental factors and they all come
  508. 19:03together in these non-intuitive ways.
  509. 19:06And so you'll never be able to combat
  510. 19:08like a complex disease like that unless
  511. 19:11you learn to embrace its multi-factorial
  512. 19:14complexity. So, that's what I'm all
  513. 19:15about.
  514. 19:17And for this reason, people have pointed
  515. 19:19to whole cell simulation
  516. 19:22uh and just trying to simulate things
  517. 19:23like a tumor or or different kind of
  518. 19:25cells as a grand challenge of this
  519. 19:26century and my lab's been working in
  520. 19:29this for years. We actually built the
  521. 19:30first computer model of a cell that
  522. 19:32takes all the gene content into account
  523. 19:35and uses it to make predictions about
  524. 19:36phenotype. I'll show you kind of a
  525. 19:39kind of like a cartoon but this is based
  526. 19:40on real simulation data where we can
  527. 19:42have in this case a bacterial cell
  528. 19:43that's growing out and it's replicating
  529. 19:45its DNA and it's dividing at a regular
  530. 19:47rate. One simulation of one cell
  531. 19:50generating one time gives us about a gig
  532. 19:52of data, right? And so now we can throw
  533. 19:54the cell into another environment, a
  534. 19:57bunch of sensors are going to get
  535. 19:58triggered, uh the gene expression will
  536. 20:00all change, metabolism will re-route
  537. 20:03itself to this new change and the the
  538. 20:05net effect of it all is that the cells
  539. 20:06will grow faster and bigger and and uh
  540. 20:09start even parallel processing their
  541. 20:11DNA. All this cool stuff happens.
  542. 20:14And uh I wouldn't call this AI per se,
  543. 20:16more like artificial life which people
  544. 20:18don't talk about as much, right? Like
  545. 20:20but but that's kind of the space I see
  546. 20:22myself in, artificial life
  547. 20:24and
  548. 20:26and also how AI interacts with
  549. 20:29artificial life. So we got involved in
  550. 20:31deep learning
  551. 20:33over a decade ago now where we were kind
  552. 20:34of amongst the pioneers of trying to
  553. 20:37automatically look at a microscopy image
  554. 20:39stack and be able to give you all of the
  555. 20:41cells that are in there and be able to
  556. 20:43track them over time so that we can see
  557. 20:45the individuality of cells, how they
  558. 20:47interact with each other.
  559. 20:49>> [snorts]
  560. 20:49>> And this was a lot of fun
  561. 20:52and at the same time I was actually at
  562. 20:55Google X as a consultant. And working on
  563. 20:59a project that now is now is known,
  564. 21:01right, A-life. And at the time that I
  565. 21:03was sitting next to this guy who only
  566. 21:05went by Hash. I now know his name is
  567. 21:07Jonathan Ross. He works at Nvidia. But
  568. 21:09he had made these first TPUs, tensor
  569. 21:12processing units, maybe you've heard of
  570. 21:13those. They ran at about a thousand
  571. 21:15times the speed of the fastest GPUs and
  572. 21:18they were the secret of how AlphaGo was
  573. 21:22able to beat Lee Sedol. It wasn't known
  574. 21:24that these TPUs existed but without them
  575. 21:26each turn at that Go table would have
  576. 21:29taken about a year for the computer to
  577. 21:31process. And so just watching all this,
  578. 21:34I'd spend my lunch breaks like with
  579. 21:35those guys talking about AlphaGo and it
  580. 21:38gave me this first question that I just
  581. 21:40want to pose to you, right, as the
  582. 21:42future of our world, right? What
  583. 21:44audacious human aspirations are only now
  584. 21:47conceivable with this new technology?
  585. 21:49That's something I think about a lot.
  586. 21:51Not just about this technology, by the
  587. 21:52way, right? I've had these exact same
  588. 21:54feelings
  589. 21:55when we first learned that we could take
  590. 21:57regular cells and dedifferentiate them
  591. 21:59to induce induce pluripotent stem cells,
  592. 22:02right? I've had these same feelings when
  593. 22:03I heard about gene editing
  594. 22:05with CRISPR, right? So these big
  595. 22:08technologies, I think it's an important
  596. 22:09question to ask, like what now is
  597. 22:11possible? Not just at a low level but at
  598. 22:14like the highest level. No one could
  599. 22:16ever do this and now they can.
  600. 22:20Then the next thing is education, you
  601. 22:21know, that's been really fascinating to
  602. 22:23watch. I'm sure for you guys, too, for
  603. 22:25better or for worse, right? Um
  604. 22:27I don't know if you saw this recent
  605. 22:29report by MIT talking about education
  606. 22:32and AI. Uh I love these principles. I
  607. 22:34think the whole thing is really
  608. 22:35fascinating, but it's fun. We uh you
  609. 22:38know, this summer my TAs and I worked on
  610. 22:40creating, you know, an AI tool that
  611. 22:42would help students to learn my class.
  612. 22:44Like
  613. 22:45I wrote the textbook, so it seemed like
  614. 22:46we could just put everything that I had
  615. 22:48ever generated into this into this
  616. 22:50helper. And then, of course, only to
  617. 22:52find from this MIT report that students
  618. 22:55who uh who used tools like that not only
  619. 22:58had worse learning outcomes and learned
  620. 23:00less,
  621. 23:01but actually were more depressed, felt
  622. 23:03more isolated, and felt more anxiety,
  623. 23:05right? So, then we we actually had a
  624. 23:07vote on this and decided not to give
  625. 23:09this beautiful tool out uh just for
  626. 23:12those very reasons. We didn't want to do
  627. 23:13more harm than good. But, I would ask
  628. 23:15based on that, like
  629. 23:17and again, maybe a little bit
  630. 23:18independent from AI, but AI is right in
  631. 23:20the center of it, right? How can we
  632. 23:22ensure
  633. 23:23that you receive the best possible
  634. 23:26education that anyone could ever give,
  635. 23:28right? In the history of the world, cuz
  636. 23:30that's really what where we are and what
  637. 23:32you deserve.
  638. 23:34The third question kind of relates to
  639. 23:36this fun thing. I've I've been really
  640. 23:38interested in what these faith and other
  641. 23:40like ancient traditions have been how
  642. 23:41they've been reacting to AI. Some of you
  643. 23:43might know that the first letter that
  644. 23:45that this new pope wrote
  645. 23:47was a long treatise on AI. And in
  646. 23:50particular, AI in terms of safeguarding
  647. 23:52the human person, which is a fascinating
  648. 23:55uh phrase.
  649. 23:57And then I was uh honored to be recently
  650. 23:59in a in a brief uh in a in an
  651. 24:02interaction with uh Father Michael
  652. 24:04Baggott representing the Vatican, who
  653. 24:06had just come to meet with Anthropic
  654. 24:07leaders. Some of you might have seen the
  655. 24:09article in the New York Times talking
  656. 24:10about AI and transhumanism and how how
  657. 24:14they are thinking about things at the
  658. 24:15Vatican. I would love to hear how others
  659. 24:18uh both faith and non-faith but
  660. 24:20traditions are thinking about these
  661. 24:22things. But, I guess the question there
  662. 24:24that they are asking, as far as I can
  663. 24:25tell, is what can we do to promote human
  664. 24:28flourishing and the highest ideals of
  665. 24:30civilization using this technology?
  666. 24:33Okay, also a very, very interesting
  667. 24:35question and again, I probably think of
  668. 24:37each of these multiple times a day.
  669. 24:39Final one and then I'll be done.
  670. 24:42Uh this is an article I found in nature
  671. 24:44a couple years ago and the headline just
  672. 24:46grabbed me. Scientists need more time to
  673. 24:50think. I was like, [laughter] yes, I
  674. 24:52need more time to think. Uh my my kids
  675. 24:55would probably say I need to any time A
  676. 24:56time to sound like I'm thinking. My even
  677. 24:59my mom sometimes talks about superficial
  678. 25:02intelligence, which is kind of what
  679. 25:03she's talking about here. So, uh
  680. 25:06artificial superficial, I still feel
  681. 25:07good about it. But, um
  682. 25:09scientists need more time to think and
  683. 25:11the big thing here that that the point
  684. 25:13they make is that people have become
  685. 25:15more and more productive with
  686. 25:17productivity tools. Email has made us
  687. 25:19more productive.
  688. 25:21But, it's actually given us less time to
  689. 25:23think.
  690. 25:24It's given us less time to internalize,
  691. 25:27to reconcile seemingly opposing
  692. 25:29evidence. It's given us less time to do
  693. 25:32all of that stuff and so the final
  694. 25:33question that I
  695. 25:34would love to hear about here or think
  696. 25:36about is, you know, how can we give
  697. 25:38scientists more time to think?
  698. 25:41How can we give doctors more time to
  699. 25:43care?
  700. 25:45How can we give students more time to
  701. 25:46explore?
  702. 25:48Right? How can we give humans more time
  703. 25:50to connect? So, those are my obsessions
  704. 25:53surrounding AI and I can just leave it
  705. 25:55there.
  706. 25:56>> All right, that was an exciting
  707. 25:58introduction to what um
  708. 26:00expertise we're getting from our
  709. 26:02panelists. I'd like to invite them up
  710. 26:04now to take uh these seats in the front
  711. 26:07and I will ask some questions, we'll do
  712. 26:10some Q&A, and um then we'll open it up
  713. 26:14for audience Q&A.
  714. 26:16All right, so the first question that uh
  715. 26:18I would like to put to each of you is
  716. 26:21when you're thinking about applying AI,
  717. 26:25how do you balance thinking about what
  718. 26:29the capabilities and limitations of the
  719. 26:31technology are today
  720. 26:34with
  721. 26:35uh trying to
  722. 26:37predict out into the future a little bit
  723. 26:40where the technology might be 5 years
  724. 26:43down. So, if you're, you know, building
  725. 26:45a tool or starting kind of a research
  726. 26:48agenda that might involve AI, how do you
  727. 26:51balance
  728. 26:52the present versus
  729. 26:55um possible improvements or change in
  730. 26:56the future.
  731. 27:02>> Uh I can take a stab.
  732. 27:04Um one thing that happened for me when I
  733. 27:06started at Google X way back when, um I
  734. 27:09was just thinking, you know, being
  735. 27:11academic, I was wondering if I wanted to
  736. 27:12do that, but one thing that they said to
  737. 27:14me that I thought was really fascinating
  738. 27:16was um
  739. 27:18you should just assume here that your
  740. 27:20compute and memory are infinite, and you
  741. 27:22should not develop based on any
  742. 27:25consideration of what today's capacity
  743. 27:27is.
  744. 27:28Um
  745. 27:29that was a huge uh mind shift for me
  746. 27:33because I think very often I I do feel
  747. 27:35limited by the technology. I'm thinking,
  748. 27:37oh, could we actually do this, right?
  749. 27:39But, I think it really gave me
  750. 27:41permission to think
  751. 27:43much bigger, just kind of feeling like
  752. 27:46uh you know, okay, things will be in
  753. 27:48play. If I create this, at some point
  754. 27:49the right the right situation will be in
  755. 27:52place where we could actually use it for
  756. 27:53something.
  757. 27:54And uh and it was, right? So, I think in
  758. 27:56this case just the fact of kind of
  759. 27:59opening your mind to the to this idea
  760. 28:01that somewhere out there somebody's
  761. 28:03invented something that runs a thousand
  762. 28:04times faster or something like that is a
  763. 28:07is a pretty big impetus to to go big, I
  764. 28:11think.
  765. 28:12>> [clears throat]
  766. 28:15>> I will say from my perspective, um if
  767. 28:18the last 5 years have shown us anything,
  768. 28:21it's that
  769. 28:23this is this is largely unpredictable,
  770. 28:26right? Like if you if it's fascinating
  771. 28:28to watch the discourse evolve around uh
  772. 28:31large language models since 2001,
  773. 28:34when we thought, "Oh, well, these are
  774. 28:37okay writing and if they can string
  775. 28:39together a sentence, but they're never
  776. 28:40going to
  777. 28:41no one's ever going to fool the or
  778. 28:42mistake this for a human."
  779. 28:45And just the leaps that have happened
  780. 28:48in ways that we still don't really
  781. 28:51completely understand,
  782. 28:54I think make
  783. 28:56projecting out, especially 5 years, a
  784. 29:00kind of a kind of fool's game. But,
  785. 29:03there are things that are constant,
  786. 29:05right? And that's that's us. We don't
  787. 29:07know how eventually we'll change in
  788. 29:10response to the in response to the
  789. 29:11technology, but we don't change quite as
  790. 29:15fast as as LLMs do, right? The human
  791. 29:18remains a kind of constant in the room.
  792. 29:20And the way in which we interact with
  793. 29:22them,
  794. 29:23there are going to be some stable
  795. 29:25aspects to that. So, certainly in my
  796. 29:28teaching and in my research, when we
  797. 29:30think about this, we tend to center the
  798. 29:34human experience in interacting with the
  799. 29:37models,
  800. 29:39and looking at how that that has
  801. 29:42changed, but but that's a kind of change
  802. 29:44that I think we can we can actually
  803. 29:46grasp and and do something with
  804. 29:47regardless of what happens in the next 5
  805. 29:49years.
  806. 29:51>> I think I really resonate with both of
  807. 29:52your points. In particular, Mark, your
  808. 29:54discussion about the human experience.
  809. 29:57At least in the legal domain, I think
  810. 29:58we've barely even scratched the surface.
  811. 30:01So, Legal Tech, which now has evolved
  812. 30:04into Legal AI, has for many, many years
  813. 30:07focused on a question around
  814. 30:09standardization and trying to look for
  815. 30:11what is the golden standard. And so, a
  816. 30:13lot of the work that's been done
  817. 30:15historically was about sort of labeling
  818. 30:17what would be the best quality or best
  819. 30:20markers of a particular contract or a
  820. 30:23particular legal document. And the issue
  821. 30:25is is that, as I mentioned, so much of
  822. 30:28the business model indexes so heavily on
  823. 30:30the individual. And that a lot of it is
  824. 30:33thriving in the space of what lawyer
  825. 30:35intuition and the process behind it is.
  826. 30:38And we actually haven't even been able
  827. 30:40to really unpack when an attorney says,
  828. 30:43"I know it when I see it." what that
  829. 30:45necessarily entails. And so, when it
  830. 30:48comes to kind of the technological
  831. 30:50advancement, what we're seeing actually
  832. 30:52is largely a cultural disconnect between
  833. 30:54how things are operating internal to the
  834. 30:57profession, internal to a law firm or a
  835. 30:59corporate legal department, and actually
  836. 31:01how the technology's interfacing with
  837. 31:03these people.
  838. 31:05And so, some of the things that we've
  839. 31:07built so far, for example, in the
  840. 31:09education space, like the simulations,
  841. 31:11really we're thinking about how do you
  842. 31:13act how do lawyers train and impart
  843. 31:15wisdom from one person to another? How
  844. 31:18do we actually extract tacit knowledge?
  845. 31:21And what's been really interesting is
  846. 31:22that as the models get better, we
  847. 31:24actually are able to have a better sense
  848. 31:27of collaboration.
  849. 31:29Um just as an example, we built this
  850. 31:31tool called the Socratic Mentor, and it
  851. 31:33really is meant to reflect, for example,
  852. 31:36uh that process between PhD advisers and
  853. 31:39their PhD students. And it's really
  854. 31:41questioning, breaking down your research
  855. 31:43problems. We actually experimented with
  856. 31:46some senior attorneys on using this
  857. 31:49tool. And basically what it does is they
  858. 31:51throw out, you know, why they make
  859. 31:53certain decisions, and then this tool is
  860. 31:56quizzing and quizzing and breaking down
  861. 31:58actually one senior attorney had said
  862. 32:00that they felt like they were being
  863. 32:01picked apart like a Costco chicken. Um
  864. 32:04but and the result is that as these
  865. 32:07models just got better and better, they
  866. 32:09felt like they were really being drilled
  867. 32:12and asked to distill down, "Why is it
  868. 32:14exactly that I do this? How do I tether
  869. 32:17the quality of my work to specific
  870. 32:19experiences that I've had?" And so, I
  871. 32:22think it's pretty exciting that, you
  872. 32:23know, it's not necessarily a direct
  873. 32:25correlation like this technology is
  874. 32:27going to answer to this work. I think
  875. 32:30it's about that is really the large
  876. 32:31focus of the industry where it's about
  877. 32:33AI executing on legal work, but I think
  878. 32:36where the sort of magic and research is
  879. 32:38going is how can AI help attorneys
  880. 32:42better understand why they do the things
  881. 32:44the way they do.
  882. 32:48>> I think that's a great segue to the next
  883. 32:49question I wanted to ask, which is um so
  884. 32:53something we spend a lot of time in this
  885. 32:55class doing is trying to cultivate
  886. 32:57within each person a a well-developed
  887. 33:01moral compass about what kinds of things
  888. 33:05is appropriate to use AI for and what
  889. 33:08should really be handled by the human
  890. 33:11person.
  891. 33:12And um something that we will in the
  892. 33:17second half of the class be reaching
  893. 33:20into a little bit that I'm hoping I can
  894. 33:23invite you to reach into is
  895. 33:26um beyond one's own moral compass
  896. 33:30there are expectations of others,
  897. 33:33co-workers,
  898. 33:35clients, bosses,
  899. 33:38community members, who however much we
  900. 33:42may have
  901. 33:44you know, put effort into being
  902. 33:46thoughtful and responsible and ethical
  903. 33:49in our own choices about AI use, others
  904. 33:52may have different conclusions and
  905. 33:54different expectations.
  906. 33:56And so that kind of mismatch could come
  907. 33:58up, for example, between a student and
  908. 34:01an instructor where a student thinks I
  909. 34:03you know, that they have made a
  910. 34:05responsible choice to
  911. 34:06only have very limited participation of
  912. 34:09AI in editing an essay that they wrote
  913. 34:11for homework, for example, but maybe the
  914. 34:13instructor feels like even that much is
  915. 34:15inappropriate or something. So
  916. 34:17um another you know, example of that
  917. 34:20that came up um as I was doing research
  918. 34:22for this class was in a conversation
  919. 34:25with someone who's um a leader of a a
  920. 34:28major global faith or
  921. 34:30and um was talking about how
  922. 34:34they're putting a lot of thought into
  923. 34:36what things that is appropriate in in of
  924. 34:39sort of office efficiencies that could
  925. 34:41be achieved by AI.
  926. 34:43And what do what does the flock
  927. 34:46consider, you know, things that they
  928. 34:48expect that their leaders are doing
  929. 34:52through
  930. 34:53divine inspiration or personal uh touch.
  931. 34:58And if there's a mismatch there in
  932. 35:00expectation
  933. 35:02and that is discovered, it could be
  934. 35:04catastrophic for someone's, you know,
  935. 35:06faith. So,
  936. 35:08um how do you think about that within
  937. 35:12the
  938. 35:14the human circles that you operate, how
  939. 35:17to
  940. 35:18um govern yourself and manage
  941. 35:21expectations with others?
  942. 35:26>> Can I pull the class on something?
  943. 35:29So,
  944. 35:30uh
  945. 35:31you know, probably all of you have asked
  946. 35:33a professor for a letter of
  947. 35:34recommendation.
  948. 35:36How would you feel? How many of you
  949. 35:37would be in favor of them using AI to
  950. 35:40write that?
  951. 35:42Yeah, right. Can I see
  952. 35:44>> [laughter]
  953. 35:44>> How many of you would really be troubled
  954. 35:46and probably choose someone else if you
  955. 35:47knew that this professor was going to be
  956. 35:48using AI?
  957. 35:50Yeah. Right. That's interesting, right?
  958. 35:53I think that's really interesting.
  959. 35:55Uh I think communication's really
  960. 35:57important, but this is this is a good
  961. 35:59example for me because I have made the
  962. 36:00decision that I am not going to
  963. 36:03write letters using AI, right? Like I
  964. 36:05I will use it I will not use it to
  965. 36:07write, right? I will use it to basically
  966. 36:09hold information and I can ask it
  967. 36:11questions. It'll remind me, you know,
  968. 36:13remind me where they did that
  969. 36:14internship, you know, something like
  970. 36:16that, but not um not in a way that I
  971. 36:19lose my voice and my personal connection
  972. 36:22to the to the student. So, then
  973. 36:25to me, at least, the question becomes,
  974. 36:27okay,
  975. 36:28how can I use AI not to save time with
  976. 36:32this letter because I think that's what
  977. 36:34most professors might be thinking when
  978. 36:36they're using AI. How can I save time?
  979. 36:38Instead, it's how can I write a higher
  980. 36:40quality letter that gives this gives a
  981. 36:42better
  982. 36:44view of this student and really
  983. 36:46showcases how I know them and what my
  984. 36:48experience is with them so that someone
  985. 36:50else can read it and really know
  986. 36:53that [clears throat] student and my care
  987. 36:54and concern for that that student and
  988. 36:56our relationship. That's a very
  989. 36:58different question to ask AI, right? Or
  990. 37:00a very different approach to take, but I
  991. 37:02think that's kind of goes to what you
  992. 37:03were saying. Like are we just using it
  993. 37:05to save time? Like write me 100 student
  994. 37:07like I'm writing them all right now. So
  995. 37:09I'm thinking about it a lot.
  996. 37:10Uh
  997. 37:10>> [laughter]
  998. 37:11>> I've been tempted.
  999. 37:13But [clears throat] um but I think this
  1000. 37:14idea of saying no, what I want and I I
  1001. 37:18tell this to Claude all the time. What I
  1002. 37:20want is quality. I do not care as much
  1003. 37:22about saving my time. I want higher
  1004. 37:25quality and I want my the way I define
  1005. 37:27quality is is humans first. So um
  1006. 37:31or let's say planet first. Real entities
  1007. 37:33first. Sometimes it's also animals
  1008. 37:35first. But um but that idea I think is
  1009. 37:38is important. So maybe I'll just throw
  1010. 37:39that out as kind of a fun way to pull
  1011. 37:40the class on how they feel. It might
  1012. 37:42also change how you think about, you
  1013. 37:44know,
  1014. 37:45what you are presenting to a professor,
  1015. 37:46you know, how how we might all feel in
  1016. 37:48terms of what that means about our
  1017. 37:49relationship.
  1018. 37:51>> Can I actually ask a follow-on question
  1019. 37:53to that? Um to poll everybody, how many
  1020. 37:56of you have had professors
  1021. 37:58respond to you, you can write whatever
  1022. 38:01you want in this recommendation letter
  1023. 38:02and I'll sign it.
  1024. 38:06Would you prefer that approach or
  1025. 38:09knowing sort of transparently that your
  1026. 38:13professor that's writing your
  1027. 38:14recommendation letter has had some
  1028. 38:16support from AI?
  1029. 38:21>> You have to ask who prefer Who Who
  1030. 38:23prefers uh
  1031. 38:26you writing
  1032. 38:26>> writing it.
  1033. 38:31>> Would you use AI to write it?
  1034. 38:33>> [laughter]
  1035. 38:37>> I think that these questions are
  1036. 38:38starting to get kind of quite difficult
  1037. 38:40at least from our standpoint. Um we
  1038. 38:44recently hear in the legal profession
  1039. 38:45that a lot of young attorneys are
  1040. 38:47really, really scared about using AI in
  1041. 38:50practice. And because they could be a
  1042. 38:53breath away from being the headline of a
  1043. 38:55very salacious newspaper title like
  1044. 38:59Sullivan Cromwell attorney like has
  1045. 39:01these, you know, hallucinated citations
  1046. 39:04in their legal briefs. And so
  1047. 39:06incidentally, I spoke to actually the
  1048. 39:08Sullivan Cromwell team about that
  1049. 39:10specific incident. And there were
  1050. 39:11actually seven attorneys eyes on that
  1051. 39:15particular brief, and it still slipped
  1052. 39:18their radar. Um, and so now I think
  1053. 39:21there's been kind of this tension where
  1054. 39:23a lot of partners and they're looking at
  1055. 39:25a lot of usage internally in AI,
  1056. 39:27especially on Legora or Harvey or any of
  1057. 39:29the legal AI providers. Um,
  1058. 39:32the highest usage comes from leadership
  1059. 39:34and partnership.
  1060. 39:36And actually the lowest usage is from
  1061. 39:38the young attorneys. And what's been
  1062. 39:40interesting is like how do we get
  1063. 39:42actually the partnership to encourage
  1064. 39:44back some of the younger attorneys to
  1065. 39:45use these platforms in a responsible
  1066. 39:47way. Um, and some partners have started
  1067. 39:50to write documentation of exactly how
  1068. 39:52they've been using AI and kind of using
  1069. 39:54that as a way to kind of soft guide a
  1070. 39:58lot of the younger attorneys. I think
  1071. 40:00it's really our responsibility kind of
  1072. 40:03with familiarity with using AI that we
  1073. 40:05show that direction um, a lot to our
  1074. 40:08students uh, in the legal space, to the
  1075. 40:11younger attorneys. Because honestly, it
  1076. 40:13is really incredibly unclear sometimes
  1077. 40:16to only uh, rely solely on your
  1078. 40:18independent kind of moral compass
  1079. 40:20because the models are just moving so
  1080. 40:22quickly. And even our conception of the
  1081. 40:25work that has to be done is going to
  1082. 40:27look so different. And so I think there
  1083. 40:29is this kind of balance where um, you
  1084. 40:32know, is it the educator that will just
  1085. 40:34use the exact same exam, exact same
  1086. 40:36material over and over again? Or is it
  1087. 40:38that it's our responsibility to think
  1088. 40:40through like how we should be teaching
  1089. 40:42the class entirely differently?
  1090. 40:45>> I would I think I'd echo both of what
  1091. 40:48you just said because for me
  1092. 40:51I remember learning as a young computer
  1093. 40:53science student before I found my way
  1094. 40:55into to to literature.
  1095. 40:58Um,
  1096. 40:58remember learning about the Turing test,
  1097. 41:00right? The gold standard for half a
  1098. 41:02century or more actually in how to, you
  1099. 41:06know, judge when we have achieved some
  1100. 41:08kind of artificial intelligence. The
  1101. 41:10thing I remember thinking about it at
  1102. 41:12the time is that this is a test based on
  1103. 41:15deception, right? Like the gold standard
  1104. 41:17test for years and years. We're more
  1105. 41:19complicated now, but as we're thinking
  1106. 41:21about how we've trained these models,
  1107. 41:24what we're what what the ideal has
  1108. 41:26always been is that we want them to fool
  1109. 41:29humans into thinking that they're
  1110. 41:32humans. Now, again, we've moved beyond
  1111. 41:34that ethically, but still somewhere at
  1112. 41:36the core is is is the kind of factoid,
  1113. 41:39the kind of nugget, the kind of ethical
  1114. 41:41quandary that these are machines, these
  1115. 41:44are machines trained to deceive us. Um,
  1116. 41:48and how we use them obviously can can
  1117. 41:51can have positive or negative ethical
  1118. 41:52ramifications, but the way to get around
  1119. 41:55I think all of the moral quandaries that
  1120. 41:57that you've posed is radical
  1121. 41:59transparency, right? Like in that case,
  1122. 42:02I don't have to depend on my own morals
  1123. 42:06covering what I do with AI in my
  1124. 42:08classroom, in my lab, etc. Because if I
  1125. 42:11am 100% transparent about how I'm using
  1126. 42:13it and where I'm using it, other people
  1127. 42:15can make up their own minds and we can
  1128. 42:17have a conversation about that. I think
  1129. 42:19where ethically we run into trouble is
  1130. 42:22when, you know, we have a professor that
  1131. 42:25writes 100 recommendation letters using
  1132. 42:27AI, but pretends that they've written
  1133. 42:29them themselves or or vice versa, the
  1134. 42:31student who, you know, hands in a paper
  1135. 42:34um generated that that they say they've
  1136. 42:37done. So, I think I mean again, it sort
  1137. 42:39of comes back to the human. I think
  1138. 42:41transparency allows us to have these
  1139. 42:44conversations in a way that opens up
  1140. 42:48these ethical questions to a larger to a
  1141. 42:50larger audience.
  1142. 42:54>> Another perfect segue. The next thing I
  1143. 42:56wanted to talk about is transparency and
  1144. 43:00developing cultural community norms as
  1145. 43:04well as
  1146. 43:06just very specific infrastructure
  1147. 43:08supporting
  1148. 43:10transparency and disclosure around AI
  1149. 43:13use. So, if each of you could share if
  1150. 43:18you observed any
  1151. 43:20of that kind of work happening in your
  1152. 43:24fields either in how
  1153. 43:27research conferences are incorporating
  1154. 43:31opportunities for
  1155. 43:34places for people to disclose exactly
  1156. 43:37how they've used AI
  1157. 43:39or if you have any thoughts about you
  1158. 43:43know what we should expect say from
  1159. 43:47political advertisements that use AI
  1160. 43:49videos
  1161. 43:51and should they have a little
  1162. 43:54you know
  1163. 43:54>> [laughter]
  1164. 43:54>> thing at the bottom of the screen saying
  1165. 43:56that what should we expect from media
  1166. 43:59and what should we expect from each
  1167. 44:02other
  1168. 44:03is you know a future recommendation
  1169. 44:06letters submission form web page going
  1170. 44:09to have a little box that
  1171. 44:11says describe your AI use here what what
  1172. 44:14kind of infrastructure and societal
  1173. 44:17adjustments do we need to make to
  1174. 44:19support transparency.
  1175. 44:25>> I guess I would say and here I'm going
  1176. 44:26to get here I'm going to get heretical.
  1177. 44:28This is something I would say in this
  1178. 44:29classroom but I would never say in my
  1179. 44:31own department. Or any of my own
  1180. 44:33conferences.
  1181. 44:36I think one of the places interestingly
  1182. 44:38where we run into problems is like the
  1183. 44:40outright ban on AI the kind of blanket
  1184. 44:43across the board this should never be
  1185. 44:45used for any reason.
  1186. 44:47And I think there are situations in
  1187. 44:49which its use is deeply a problem.
  1188. 44:52And and I say this again coming from a
  1189. 44:54from a field that has a tendency to to
  1190. 44:55do those bands. That's it's sort of
  1191. 44:57going back to blue books as a way to to
  1192. 44:59get out from under it. But when we do
  1193. 45:01that, what we're doing is we're
  1194. 45:03incentivizing people to still use it,
  1195. 45:06but not say anything about it, right?
  1196. 45:08Um, and I think it comes the best work
  1197. 45:10I've seen in my field um, around its use
  1198. 45:16in a in a I think ethical and open way
  1199. 45:18that that really incentivizes
  1200. 45:19transparency is just is to figure out
  1201. 45:22what we care about and and what we care
  1202. 45:26less about. So, for example, in my
  1203. 45:28field, in my very specific like digital
  1204. 45:30humanities, computational literary
  1205. 45:31studies, we care very much about writing
  1206. 45:35as a thought process and writing as a
  1207. 45:36human um,
  1208. 45:38uh, created object. Um, and so it would
  1209. 45:41be a problem if people started
  1210. 45:43submitting, you know, AI-written um,
  1211. 45:45articles to journals because as I said,
  1212. 45:47that's where the meaning sort of
  1213. 45:48happens.
  1214. 45:50But on the coding side, a lot of the
  1215. 45:53people in my field came to doing this
  1216. 45:56kind of computational work from a
  1217. 45:57humanities background, not from a CS
  1218. 45:59background. And so AI has given um, a
  1219. 46:03large number of people access to really
  1220. 46:06complicated coding techniques that they
  1221. 46:08didn't have before. And so, for example,
  1222. 46:11in uh, a number of the journals in my
  1223. 46:13field, um, you're expected to disclose
  1224. 46:16your use of AI. But in the in the sort
  1225. 46:19of fine print, um, it says where that
  1226. 46:23use is acceptable and where it's not,
  1227. 46:24right? Don't write the paper with AI.
  1228. 46:27If you did use AI, you know, Claude code
  1229. 46:30to help develop the the software behind
  1230. 46:33it, you reveal that, credit it, cite it,
  1231. 46:36um, and then move on. The blanket ban
  1232. 46:39encourages, I think, a kind of bad
  1233. 46:40acting. Sort of drawing the lines around
  1234. 46:43here's where it's okay gives people the
  1235. 46:46freedom, I think, to be transparent
  1236. 46:48about it.
  1237. 46:50>> I mean, I can't really speak in the
  1238. 46:53broader context, but I'd say internal to
  1239. 46:56our lab, we do have an AI policy
  1240. 46:59and some of the things that the faculty
  1241. 47:02director, Julian Nyarko, and I have
  1242. 47:04thought about as we put it into writing
  1243. 47:06was we do very much want them to be
  1244. 47:09using the state of the art,
  1245. 47:12but we do want to know why you use it
  1246. 47:14and how you're using it. And
  1247. 47:15importantly, we just want them to answer
  1248. 47:17one single question, which is can you
  1249. 47:20explain to us the design and the
  1250. 47:22specific critical choices that you had
  1251. 47:24to make as you were using this
  1252. 47:26technology at every juncture.
  1253. 47:29I think a lot of what we were seeing
  1254. 47:31before we put the AI policy into place
  1255. 47:33was that, you know, it's very quick for
  1256. 47:36you to just throw it into, you know,
  1257. 47:38Codex, Claude Code, whatever it might be
  1258. 47:41and for you to produce
  1259. 47:43everything and kind of just submit it
  1260. 47:45during our research meetings. But when
  1261. 47:47we actually drilled in on what the
  1262. 47:48impact was to the research, why you made
  1263. 47:51specific design choices, all of that
  1264. 47:54kind of fell apart. And so, again, a lot
  1265. 47:57of what we think is the responsibility
  1266. 47:59of the researcher going forward is this
  1267. 48:02design question, is the architecture
  1268. 48:04questions. The execution, we are going
  1269. 48:07to be well aware that that's going to be
  1270. 48:09handled much better and much faster and
  1271. 48:11much more productively using these
  1272. 48:13tools, but again, like you have to
  1273. 48:16really wrestle with the much harder
  1274. 48:18problem, which is can you build that
  1275. 48:20foresight into why you're doing these
  1276. 48:22why you're making these decisions.
  1277. 48:25>> Yeah.
  1278. 48:27It's the
  1279. 48:28I mean, there's a lot of interesting
  1280. 48:29These are great questions, Cynthia. Um
  1281. 48:32And what Mark was saying about the
  1282. 48:33Turing test is being fundamentally
  1283. 48:35deceptive, I've just kind of still been
  1284. 48:36turning this one over my head. It's very
  1285. 48:38interesting. I think, you know, when I
  1286. 48:40think about my biggest worry for for
  1287. 48:42like your generation right now
  1288. 48:44educationally and what we're trying to
  1289. 48:45provide, it's kind of related to this
  1290. 48:48because the Turing test performance is
  1291. 48:51strong through college. Like I think you
  1292. 48:54could probably ask, you know, you can
  1293. 48:56ask for help solving integrals and you
  1294. 48:58can ask for all these anything where
  1295. 49:00there's been a lot of homework problems
  1296. 49:01made or whatever, probably you can get
  1297. 49:03those answers. And so it's possible for
  1298. 49:05some people, you know, and I actually
  1299. 49:07have a kid who's at Stanford right now
  1300. 49:09and someone told him once over dinner or
  1301. 49:10something like I'm going to cheat
  1302. 49:12basically on every everything for the
  1303. 49:13rest of my time at Stanford and I'm
  1304. 49:14never going to do a problem again. And
  1305. 49:17which is kind of crazy, right? But the
  1306. 49:20the damage there is that that person
  1307. 49:21doesn't realize that that basically
  1308. 49:23[clears throat] that means they'll leave
  1309. 49:24Stanford with nothing nothing internal
  1310. 49:27to show for it. And then they'll go out
  1311. 49:29into a world where we're actually super
  1312. 49:31bad at the Turing test. Like none of the
  1313. 49:33problems that I am working on right now
  1314. 49:35could you just like
  1315. 49:37ask Claude to solve them, right? So it's
  1316. 49:39it's interesting. I feel like there's
  1317. 49:41this weird cliff where it's like AI
  1318. 49:43could be ridiculously helpful to get you
  1319. 49:45through school without thinking at all.
  1320. 49:48And then it will become increasingly
  1321. 49:50unreliable as soon as you walk off of
  1322. 49:52this campus and try to do something new.
  1323. 49:55So that is a really crazy thing that we
  1324. 49:57have to figure out and I think
  1325. 50:00to your points like you have to you
  1326. 50:02definitely have to learn how to use it.
  1327. 50:04I mean it would be crazy not to do that.
  1328. 50:05But at the same time you have to learn
  1329. 50:07everything else that you still
  1330. 50:09that you came here for, right? That you
  1331. 50:11came here for. Not just not just the
  1332. 50:12badges and plus five and whatever the
  1333. 50:14video game version of what Stanford the
  1334. 50:16Stanford experience is. Not just you
  1335. 50:18know, these these these little awards.
  1336. 50:21In my lab we talk a lot about so we
  1337. 50:23could talk about several places where
  1338. 50:25this is problematic. The whole journal
  1339. 50:26publishing system. Right now one of the
  1340. 50:29biggest problems with AI is that you can
  1341. 50:30just generate like it's like the most
  1342. 50:32incredible like garbage generator
  1343. 50:34in the history of civilization
  1344. 50:36[laughter]
  1345. 50:37and people can you know, I heard
  1346. 50:39somebody just recently say who very very
  1347. 50:42high place, oh now in five minutes I can
  1348. 50:44just write a 50 page report and I send
  1349. 50:46it to all my staff. It doesn't feel like
  1350. 50:47they appreciate it though and I was
  1351. 50:48like, no they don't
  1352. 50:50because they're not idiots. But the
  1353. 50:53but this idea of being able to generate
  1354. 50:55so much. So I tell my lab again, I feel
  1355. 50:57like transparency, quality, like really
  1356. 51:01engaging deeply and kind of thinking
  1357. 51:03about what your thought process is.
  1358. 51:05Those are all really important. Uh in my
  1359. 51:08lab we've established some things that I
  1360. 51:10consider to be rules. I think we're
  1361. 51:11still kind of getting there, but one is
  1362. 51:14uh if you save all this time with AI,
  1363. 51:17you have to give some back. And I say
  1364. 51:19this is where I want you to give it
  1365. 51:20back. Like if you save 90% of your time,
  1366. 51:23I want to give 20% back to correct
  1367. 51:26checking and checking yours and other
  1368. 51:28people's work in this lab because there
  1369. 51:30is such a high chance of error uh that
  1370. 51:33arises, right? So just checking,
  1371. 51:35double-checking,
  1372. 51:37uh you making sure that we all agree
  1373. 51:39that what goes out is still the highest
  1374. 51:40quality. And I also say that they should
  1375. 51:42be spending 20%
  1376. 51:45I call it walking in the redwoods, just
  1377. 51:47going out and thinking, thinking deeply
  1378. 51:49about big problems. Like what's the most
  1379. 51:51important thing you could possibly be
  1380. 51:53doing right now with this time you're
  1381. 51:55saving? It is not Wordle.
  1382. 51:58It is not Sudoku.
  1383. 52:00Not even the hard Sudoku.
  1384. 52:02Um so those are interesting uh and
  1385. 52:05important to recognize. And then the
  1386. 52:07other thing that I think they should be
  1387. 52:08giving back time to is just
  1388. 52:11talking with other brilliant people who
  1389. 52:14want to think about the deepest things.
  1390. 52:16I was at a um I had the great
  1391. 52:18opportunity to be at uh after-hours at
  1392. 52:22the Museum of Modern Art in New York uh
  1393. 52:25listening to a docent there talk about
  1394. 52:28scientific revolutions. Like how do
  1395. 52:30major revolutions happen in thought?
  1396. 52:34And she talked about Picasso. It was
  1397. 52:36amazing. Like I guess he he painted the
  1398. 52:38first abstract art, and it was so crazy
  1399. 52:41that nobody his agent wouldn't sell it,
  1400. 52:43nobody liked it, so he retreated to
  1401. 52:45Cubism. Isn't it crazy to think that
  1402. 52:47Cubism was [laughter] like a retreat? A
  1403. 52:49retreat from what he wanted to do,
  1404. 52:51right? So she talked about these
  1405. 52:53revolutions and what's required, and
  1406. 52:55there's one part I will never forget
  1407. 52:57because she talked about how important
  1408. 52:59it was while there are people of
  1409. 53:01extraordinary talent who can come up
  1410. 53:03with things
  1411. 53:04By and large, it's people interacting uh
  1412. 53:07who are bright and excited mostly in
  1413. 53:10social terms. And she said if they
  1414. 53:12continue meeting socially, and this is
  1415. 53:14like a direct quote, eventually
  1416. 53:17the constraints of civilization begin to
  1417. 53:19break down and radicalism can emerge.
  1418. 53:22>> [laughter]
  1419. 53:23>> Like literally my wife, you can ask her.
  1420. 53:25Like in the middle of the night I'd wake
  1421. 53:26up and be like, "Radicalism can emerge."
  1422. 53:29Like that's what I want, right?
  1423. 53:30[laughter] I want us to be breaking
  1424. 53:31paradigms and doing new stuff. And so I
  1425. 53:34think if we achieve those things, then
  1426. 53:37AI will have done hopefully what we
  1427. 53:39created it for, not to deceive ourselves
  1428. 53:41or anyone else, but to enable us to do
  1429. 53:44something that benefits others including
  1430. 53:47ourselves that inspires, right? That
  1431. 53:49brings something new, that creates
  1432. 53:51connection, and where radicalism in
  1433. 53:54thought and brand new paradigms can
  1434. 53:56emerge that no one has even imagined
  1435. 53:58right now because they didn't have the
  1436. 53:59tools to do it.
  1437. 54:03>> Okay, I feel like that's a good applause
  1438. 54:05line, everyone.
  1439. 54:11>> [applause]
  1440. 54:11>> Um
  1441. 54:12and I will run this mic over to you now
  1442. 54:15if you would like to raise your hand and
  1443. 54:17ask a question. All right, that's the
  1444. 54:18first one I saw.
  1445. 54:21>> Gosh, did you really make run?
  1446. 54:23>> [laughter]
  1447. 54:25>> So something that we're asked often in
  1448. 54:27this class every week is to reflect on a
  1449. 54:29skill that we have that we
  1450. 54:33>> [laughter]
  1451. 54:33>> that we don't want to lose um due to AI
  1452. 54:36and how we're going to kind of foster
  1453. 54:38that skill moving forward. And you know,
  1454. 54:40hindsight 20/20, we can't really see in
  1455. 54:42the future and know what we need, but
  1456. 54:45all of you guys work with people our age
  1457. 54:47and undergrads and grad students. What
  1458. 54:49skills do you see um that you think are
  1459. 54:52really important for us not to let
  1460. 54:53degrade throughout these years in
  1461. 54:55college and grad school?
  1462. 55:02>> I there's like the easy answer and then
  1463. 55:04there's like I'm trying to think of a
  1464. 55:05much more clever but I mean obviously,
  1465. 55:06critical thinking, right? Like in my
  1466. 55:08field,
  1467. 55:09um
  1468. 55:10regardless of what you think that is, um
  1469. 55:12the ability to interpret
  1470. 55:16for yourself,
  1471. 55:18um relying on your own mind, your own
  1472. 55:20experiences, your own ideas, and
  1473. 55:23bringing that to an object. That can get
  1474. 55:27you not just far, that can get you just
  1475. 55:29about anywhere. And that's kind of the
  1476. 55:30seductive promise, right? Um even if you
  1477. 55:32treat something like Claude as just a
  1478. 55:34chatbot conversationalist, you start
  1479. 55:36asking it questions, you're outsourcing,
  1480. 55:38you're doing that cognitive outsourcing
  1481. 55:40of exactly that faculty onto it. And
  1482. 55:43that's that's fine if you've spent, you
  1483. 55:46know, 40 years of your life honing that.
  1484. 55:50But if you haven't, that's that's
  1485. 55:52dangerous.
  1486. 55:56>> So, in the legal space, a big thing that
  1487. 55:59is being discussed right now is
  1488. 56:01acquiring judgment. Uh and I'm sure that
  1489. 56:03this is across every single vertical,
  1490. 56:06um but a lot of people don't really
  1491. 56:09narrowly define what is judgment and
  1492. 56:11what that entails. And so, some of the
  1493. 56:14things I've been thinking a lot is
  1494. 56:16creating a necessary reference library
  1495. 56:18of experiences. And so, I often share a
  1496. 56:21lot with the students that work very
  1497. 56:23closely with us is
  1498. 56:25it's incredibly fine for you to go out
  1499. 56:29and
  1500. 56:31I don't know if your generation says
  1501. 56:32this often, but it's like touch grass is
  1502. 56:34something we often see.
  1503. 56:37I think that's going to be really,
  1504. 56:39really important. Um so, law school
  1505. 56:42actually in the US is a second sort of
  1506. 56:45post-grad degree. In other
  1507. 56:46jurisdictions, people learn law like
  1508. 56:48from bachelor's and onward.
  1509. 56:51I think what's really fascinating about
  1510. 56:54learning law in the US is a lot of
  1511. 56:56people come with mixed experiences. And
  1512. 56:59I hope that, you know, your experience
  1513. 57:01is not just speed-running through school
  1514. 57:04with all the cheat codes, it's that you
  1515. 57:06are going to be present and bringing
  1516. 57:09kind of even the wildest personal
  1517. 57:12adventures, that's really going to make
  1518. 57:15and start allowing you to create these
  1519. 57:16good analogies, good references.
  1520. 57:19And so maybe the ultimate thing is
  1521. 57:21storytelling and building the right
  1522. 57:23narratives.
  1523. 57:25Yeah.
  1524. 57:29>> I mean, I I think you're
  1525. 57:31what you've been doing Megan to try to
  1526. 57:33encode like expertise and intuition and
  1527. 57:36judgment is really interesting. One of
  1528. 57:38my students was asked this question who
  1529. 57:40I consider to be really great with
  1530. 57:42cloud code etc. She's fantastic. Um
  1531. 57:46And she said scientific taste is going
  1532. 57:48to be one of the most important things.
  1533. 57:50Just like what do you solve and what
  1534. 57:52don't you solve, right? What do you
  1535. 57:54decide to work on? How do you make your
  1536. 57:56mark? Uh I was I was at MIT giving a
  1537. 58:00talk once and one of the students said,
  1538. 58:02"It sounds like all you're talking about
  1539. 58:03is solving unsolvable problems."
  1540. 58:06And I said, "Yeah." That's all I'm ever
  1541. 58:08going to talk about from now on because
  1542. 58:11we live in the greatest age in history
  1543. 58:13for solving unsolvable problems. And I
  1544. 58:15could name them across the board like
  1545. 58:17across basically every discipline.
  1546. 58:20Things that people never ever thought
  1547. 58:22had a chance. You know, the physical
  1548. 58:24limit of light microscopy, right? Solved
  1549. 58:26here, right? Like there there are many
  1550. 58:28things.
  1551. 58:29You know, optogenetics where our
  1552. 58:30department has just been celebrating all
  1553. 58:32last week cuz of cuz of Karl Deisseroth
  1554. 58:34and the Nobel Prize. Like these things
  1555. 58:35that nobody thought had a chance.
  1556. 58:38Uh
  1557. 58:39That's where I would expect a large
  1558. 58:41portion of this group to be working,
  1559. 58:43right? I I hope you will. I hope you
  1560. 58:46will choose unsolvable problems to
  1561. 58:48break. And that's going to
  1562. 58:50require
  1563. 58:53you know, some courage like an
  1564. 58:54intellectual courage. It's definitely
  1565. 58:56going to require having learned a lot of
  1566. 58:57stuff, right? I I would say intellectual
  1567. 59:00ambidexterity, the ability to take this
  1568. 59:02field and this field and kind of know
  1569. 59:03enough about both that you can you can
  1570. 59:06pit them against each other or bring
  1571. 59:07them together. Um
  1572. 59:10I think the ability to work as part of
  1573. 59:12an excited team that's it's huge. These
  1574. 59:14habits of mind and they'll matter more
  1575. 59:17than than the exact ideas. Like I I
  1576. 59:20I used to teach this thing about
  1577. 59:22engineering in medical like
  1578. 59:24physiological systems and one year
  1579. 59:26literally I said to the students, "Look,
  1580. 59:27if you know anyone from last year, could
  1581. 59:29you just go tell them that everything I
  1582. 59:31taught them
  1583. 59:32about stem cells turned out to be
  1584. 59:34wrong."
  1585. 59:35Like just in one year it changed that
  1586. 59:37fast, right? AI is like that, right? So,
  1587. 59:40you'll you'll want to learn to just be a
  1588. 59:43a continually curious learner who's
  1589. 59:46engaged and, you know, just trying stuff
  1590. 59:49out.
  1591. 59:53>> All right, the way in the front.
  1592. 1:00:06>> So, I'm aware that AI has definitely led
  1593. 1:00:08to kind of the quality of education
  1594. 1:00:11among students and how students are like
  1595. 1:00:13allocating the time for education. And
  1596. 1:00:15but I also recognize that is very
  1597. 1:00:17efficient when you are working on
  1598. 1:00:19something that maybe you don't have the
  1599. 1:00:20skills to do. So, what do you think is
  1600. 1:00:23the boundary between using AI as a tool
  1601. 1:00:26for like building knowledge and just for
  1602. 1:00:28example, you might prompt it like, "Can
  1603. 1:00:30you teach me how to build like a code
  1604. 1:00:33that does a certain kind of task?"
  1605. 1:00:35Versus just telling the AI and prompting
  1606. 1:00:37it to do that task.
  1607. 1:00:42>> I have I have a couple thoughts. It's a
  1608. 1:00:43It's a great question and I love it cuz
  1609. 1:00:45it's really like, you know, directly
  1610. 1:00:48focused to this group. Um,
  1611. 1:00:51kind of similar to the the
  1612. 1:00:52recommendation letter question. Here's
  1613. 1:00:54another example that might be
  1614. 1:00:55interesting. Let's say that we we all
  1615. 1:00:57signed up for like a rec class, right?
  1616. 1:00:58We're going to go to the weight room and
  1617. 1:01:00learn how to lift weights. And so, we
  1618. 1:01:02went to the rec room and for 1 hour we
  1619. 1:01:03just were watch robots lifting the
  1620. 1:01:05weights.
  1621. 1:01:06>> [laughter]
  1622. 1:01:07>> Right? So, what would that accomplish?
  1623. 1:01:10Right? What would that accomplish?
  1624. 1:01:12Uh, Uh, I think it's important to
  1625. 1:01:14recognize that you're lifting those
  1626. 1:01:15weights for a reason. It's not because
  1627. 1:01:17we we live somewhere where people have
  1628. 1:01:19to be moving weights, right? We're we're
  1629. 1:01:21lucky. We're just doing it for our own
  1630. 1:01:23personal health, our own personal
  1631. 1:01:25benefit. And hopefully it's it's obvious
  1632. 1:01:28from that example that having a robot do
  1633. 1:01:30it is
  1634. 1:01:32absurd, right? It's completely absurd.
  1635. 1:01:35Uh, I think in the same way we will have
  1636. 1:01:38to figure out ways to work with AI that
  1637. 1:01:40that aren't basically having a robot
  1638. 1:01:42lift our weights.
  1639. 1:01:44Um,
  1640. 1:01:45I'll say, like I was very reluctant to
  1641. 1:01:47write anything with AI and especially to
  1642. 1:01:49edit anything generated with AI. Like I
  1643. 1:01:51see recommendations letters written with
  1644. 1:01:53AI, for sure. I also see, um,
  1645. 1:01:56whole papers written, people send me
  1646. 1:01:58stuff, and I always, if I can detect it,
  1647. 1:02:01which is probably 100% using this neural
  1648. 1:02:04network, all of you have it. It's still
  1649. 1:02:06the best one for now.
  1650. 1:02:09Uh, so, um,
  1651. 1:02:11then I just absolutely won't won't edit
  1652. 1:02:13or read it, especially if it's like 100
  1653. 1:02:14pages, which is typically one of the
  1654. 1:02:16tells. Um,
  1655. 1:02:18>> [laughter]
  1656. 1:02:18>> but I did start writing I did decide,
  1657. 1:02:21exactly to your question, um, I decided
  1658. 1:02:24I wanted to learn how I could write
  1659. 1:02:26better, like higher quality. And so I
  1660. 1:02:28did, uh, over a couple weeks ago I
  1661. 1:02:31started writing one of our papers, and I
  1662. 1:02:33told AI I don't want like I told Claude,
  1663. 1:02:35I don't want you to write this for me.
  1664. 1:02:36It's all going to be my voice, but I do
  1665. 1:02:38want you to help me.
  1666. 1:02:40And so I gave it the flow of the paper
  1667. 1:02:41that I was considering. And then I
  1668. 1:02:43started as I as I was trying this more
  1669. 1:02:45and more, I realized something, and I'll
  1670. 1:02:47just share with you what I did.
  1671. 1:02:49I wrote a paragraph, and then what I
  1672. 1:02:51said was, "Okay, I want for each
  1673. 1:02:54sentence in this paragraph, I want you
  1674. 1:02:56to tell me
  1675. 1:02:57the three things that I should be citing
  1676. 1:03:00that I haven't thought about. I want you
  1677. 1:03:02to tell me the three things that would
  1678. 1:03:04most destroy this argument. And I want
  1679. 1:03:08you to tell me three things like an
  1680. 1:03:10analogy to another field that I've never
  1681. 1:03:12thought about before."
  1682. 1:03:14And so we just went through like that.
  1683. 1:03:16Like and I would we had a running total
  1684. 1:03:18of papers. I said, "I don't trust you
  1685. 1:03:20with papers. I'm just going to read
  1686. 1:03:21these and I'm going to tell you if
  1687. 1:03:22they're wrong." And so, it started
  1688. 1:03:24looking them up and paying paying a
  1689. 1:03:26little more attention. And so, long
  1690. 1:03:28story short, after 2 days, I had written
  1691. 1:03:32four paragraphs
  1692. 1:03:34with AI. And
  1693. 1:03:37um
  1694. 1:03:38probably the best paragraphs I have ever
  1695. 1:03:40written. Right? All written by me, but
  1696. 1:03:43with a lot of just thinking back and
  1697. 1:03:45forth and reading new papers.
  1698. 1:03:47To this point about a professor asking
  1699. 1:03:50you to write your own letter, I could
  1700. 1:03:51have written those four paragraphs
  1701. 1:03:53probably in an hour because I know what
  1702. 1:03:55I knew what I had in my head, and I
  1703. 1:03:57basically know all the citations that I
  1704. 1:03:59would consider to be normal for my
  1705. 1:04:01field.
  1706. 1:04:02But, having this like foil that's really
  1707. 1:04:04pushing me to go harder was was
  1708. 1:04:07powerful. And I think it's a it's almost
  1709. 1:04:09like an inside-out approach of what what
  1710. 1:04:12I often see in the AI space, right?
  1711. 1:04:15Really, what happened was I was just
  1712. 1:04:17having guided deep thinking for two for
  1713. 1:04:202 days, right? And I came up with these
  1714. 1:04:22four paragraphs that were
  1715. 1:04:24were very meaningful and intense. So, so
  1716. 1:04:26maybe that's a thought. Like, if if
  1717. 1:04:28there's a way for students to almost
  1718. 1:04:30think of the AI is like
  1719. 1:04:32"Okay, now we've got the weights, but I
  1720. 1:04:34somehow I'm going to be able to develop
  1721. 1:04:35even instead of lift higher weights,
  1722. 1:04:37develop stronger muscles because of my
  1723. 1:04:40use, then that's a really good news."
  1724. 1:04:45>> For me, it goes back to something I said
  1725. 1:04:47earlier, which is that deciding what you
  1726. 1:04:49care about, right? You can't No one can
  1727. 1:04:52do everything all the time. Uh you know,
  1728. 1:04:56some of us are STEM majors, some of us
  1729. 1:04:58are other kinds of majors. Um I can
  1730. 1:05:00appreciate organic chemistry, but I'm
  1731. 1:05:02not going to sit and solve, you know, a
  1732. 1:05:04bunch of problems.
  1733. 1:05:06Um and so, for example, I'm designing
  1734. 1:05:09So, in Cesto, we have a critical gaming
  1735. 1:05:10lab, and I'm working with um a colleague
  1736. 1:05:13to design uh a class uh that that
  1737. 1:05:16critically thinks about video games. And
  1738. 1:05:18one of the activities that we really
  1739. 1:05:19want the students in this class to do um
  1740. 1:05:22is to design their own game according to
  1741. 1:05:25you know the principles that we're
  1742. 1:05:27talking about thinking narratively
  1743. 1:05:29thinking
  1744. 1:05:30gameplay wise etc.
  1745. 1:05:32But it's not a CS class right like we do
  1746. 1:05:35not want to teach them the code to
  1747. 1:05:37create that but we want the experience
  1748. 1:05:39them to experience having created it and
  1749. 1:05:42then actually playing through it and
  1750. 1:05:44that's where
  1751. 1:05:46AI can really really help because there
  1752. 1:05:49is a dedicated AI model for creating
  1753. 1:05:52fairly simple games
  1754. 1:05:54and the emphasis in the classes is on
  1755. 1:05:56the creation process and something that
  1756. 1:05:58thoughtful creation process not on the
  1757. 1:06:01coding. So in that particular instance
  1758. 1:06:03offloading
  1759. 1:06:05the actual mechanical creation of the
  1760. 1:06:08game to an engine which is not something
  1761. 1:06:11that we're teaching or testing in the
  1762. 1:06:12class that has enabled us to do so much
  1763. 1:06:16more than we would have been able to do
  1764. 1:06:18otherwise with the things that we
  1765. 1:06:20actually care about.
  1766. 1:06:22Whereas I can imagine in in a very
  1767. 1:06:24different class you could have almost
  1768. 1:06:25the opposite but in all cases it it if
  1769. 1:06:29we use it I think cleverly and and are
  1770. 1:06:31really clear about what we want to do in
  1771. 1:06:33a like a class or in a research context
  1772. 1:06:35it can expand rather than contract what
  1773. 1:06:39we can do.
  1774. 1:06:41>> I I think that YouTube Coursera has
  1775. 1:06:44basically democratized education in a
  1776. 1:06:46lot of ways that AI is basically
  1777. 1:06:49accelerating or furthering.
  1778. 1:06:51Something that Marcus you mentioned
  1779. 1:06:52earlier is at the end of the day it
  1780. 1:06:54boils down in my mind to curiosity and
  1781. 1:06:57in this case it's not just about the
  1782. 1:06:58first step of learning how to do
  1783. 1:07:00something that is outside of
  1784. 1:07:03your typical sort of skill set but it's
  1785. 1:07:05what you want to do afterwards with it.
  1786. 1:07:08It could be very simple like I just need
  1787. 1:07:10to figure out how to fix my toilet and
  1788. 1:07:12become a plumber for the day if I don't
  1789. 1:07:14have you know access to that but I think
  1790. 1:07:18if it's something where you want to tap
  1791. 1:07:19into a neighboring field maybe it
  1792. 1:07:22encourages you to actually speak with
  1793. 1:07:24colleagues that are more well-versed in
  1794. 1:07:26this space because you took that step
  1795. 1:07:28and started sort of the beginner
  1796. 1:07:30foundational skills in that realm. So, I
  1797. 1:07:32think it's, you know,
  1798. 1:07:34a two-step process.
  1799. 1:07:36I wouldn't treat it as, you know, a
  1800. 1:07:37one-and-done.
  1801. 1:07:40>> Yeah, I think we have time for one more.
  1802. 1:07:44All right, I'll go ahead and pass.
  1803. 1:07:53>> Hi, thank you all for coming. I have a
  1804. 1:07:55specific question for Dr. Covert. Is
  1805. 1:07:58that correct? Yeah.
  1806. 1:08:01It's on like one of the points on one of
  1807. 1:08:03of your AI's obsessions like regarding
  1808. 1:08:06like AI in education.
  1809. 1:08:08I'm very curious about the educational
  1810. 1:08:10tools and like particularly which
  1811. 1:08:12features or design choices like may have
  1812. 1:08:15contributed to the negative outcomes and
  1813. 1:08:17your thoughts on AI for learning in
  1814. 1:08:19general. And like what features or
  1815. 1:08:22design choices could have been
  1816. 1:08:23implemented so students could like
  1817. 1:08:25quote-unquote like lift their own
  1818. 1:08:27weights because I know like I'm in a CS
  1819. 1:08:30class right now and
  1820. 1:08:32the way that we're allowed to use AI for
  1821. 1:08:35our learning is that they give us the
  1822. 1:08:36agents MD file which we're allowed to
  1823. 1:08:39put in whatever like AI we want to use
  1824. 1:08:41that put like guardrails on our
  1825. 1:08:43learning. So, I'm just curious about
  1826. 1:08:45your thoughts on like educational AI
  1827. 1:08:47tools in general.
  1828. 1:08:49>> Yeah, that's a great. Well, wait. Wait.
  1829. 1:08:51Give her back the mic. What's your name?
  1830. 1:08:54Nice to meet you and feel free to call
  1831. 1:08:55me Marcus. No one calls me Dr. Covert.
  1832. 1:08:58>> [laughter]
  1833. 1:08:59>> Okay, great. Like I think this is such
  1834. 1:09:02an interesting question and it just
  1835. 1:09:04gives me kind of a chance to do a little
  1836. 1:09:06bit of a rah-rah moment for Stanford in
  1837. 1:09:08the sense that I mean what you're
  1838. 1:09:10hearing here are, you know, some
  1839. 1:09:13incredible minds in English and in law
  1840. 1:09:16and and I happen to know like Cynthia
  1841. 1:09:18and others are so dedicated to making
  1842. 1:09:20sure that you get the most out of out of
  1843. 1:09:23AI and your education, you know, which
  1844. 1:09:26is the more important thing in this
  1845. 1:09:28rapidly developing field. So, I think
  1846. 1:09:31it's it's pretty exciting to be here at
  1847. 1:09:33a time where things are in flux and you
  1848. 1:09:36have a chance to just be right here
  1849. 1:09:38watching it. So,
  1850. 1:09:40I grew up here. I grew up a mile away
  1851. 1:09:42from here, right? So, right by Midtown,
  1852. 1:09:44if any of you know that um
  1853. 1:09:46So, and I remember when I was young, uh
  1854. 1:09:50web search was like a big thing, right?
  1855. 1:09:52And one thing that was really
  1856. 1:09:53interesting is
  1857. 1:09:55you would never even know this, but
  1858. 1:09:56there was like Lycos and WebCrawler and
  1859. 1:09:58Ask Jeeves. There were like a million
  1860. 1:10:00and then Google showed up with the new
  1861. 1:10:01PageRank algorithm and they all just
  1862. 1:10:04disappeared, right?
  1863. 1:10:05>> [laughter]
  1864. 1:10:06>> So, I do I can easily imagine that what
  1865. 1:10:09we're seeing right now is just like a a
  1866. 1:10:11pre-shock of kind of the the the
  1867. 1:10:14earthquake, let's say, that's going to
  1868. 1:10:16happen in AI.
  1869. 1:10:18And watching everybody kind of test and
  1870. 1:10:21having people who
  1871. 1:10:23who have innovated as far as you can in
  1872. 1:10:25this space, who are not only doing that
  1873. 1:10:27in their labs, but also in the
  1874. 1:10:28classroom, is something very unique that
  1875. 1:10:31you have at Stanford.
  1876. 1:10:33When I first got here and got a speech
  1877. 1:10:35about teaching from the provost, they
  1878. 1:10:37said, you know, we want you to be a
  1879. 1:10:39great teacher, but we also want to make
  1880. 1:10:41sure you never forget that you have the
  1881. 1:10:44power to give Stanford students the
  1882. 1:10:45ability to know what happened this
  1883. 1:10:47morning in the cutting edge of the field
  1884. 1:10:50and you should share it with them. And
  1885. 1:10:51so, that's that's my recommendation to
  1886. 1:10:54you is go take those classes where
  1887. 1:10:56people are literally just leaning in so
  1888. 1:10:59hard creating this new frontier and
  1889. 1:11:02that's going to be the best way to just
  1890. 1:11:04be engaged and energized and it's
  1891. 1:11:07something you can only get really on
  1892. 1:11:08this campus in many ways. So,
  1893. 1:11:12>> All right. Well, can we get a round of
  1894. 1:11:13applause for everyone?

About this transcript

This page contains the full transcript of YouTube transcript (PeRJkvzIEts) , generated from the public captions YouTube serves with the video. The transcript has 12,383 words across 1,894 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.