YouTube2Text

Sam Altman - You no longer need to write prompts. — Transcript

by prabhakar varma · 7,577 words · 1,085 segments · language en · Watch on YouTube

Full transcript

  1. 0:00you with like an affordable amount of
  2. 0:03spend on tokens, you can do what a 100
  3. 0:06person incredibly great engineering team
  4. 0:10would do as a startup and that was just
  5. 0:12totally impossible. That was like not in
  6. 0:13the set of options for a startup and now
  7. 0:17it is. So, so I think what you can take
  8. 0:19on uh the level of ambition you can
  9. 0:21have, the speed of which you can move,
  10. 0:23the amount of stuff you can do at once
  11. 0:25uh is just totally different. And um
  12. 0:27does that change the shape of the
  13. 0:29problems you feel like you'd assign at
  14. 0:32the end of the class for people to
  15. 0:33attack, you know, at the end of that
  16. 0:35quarter if you were teaching it again? I
  17. 0:38don't think assigning problems to attack
  18. 0:40ever works because if you like if I can
  19. 0:43think of a problem, if I can think of
  20. 0:44like a really great startup idea, uh if
  21. 0:46it's like obvious enough to me, uh then
  22. 0:49it's probably obvious to a lot of
  23. 0:51people. when we started OpenAI, we were
  24. 0:52we were like the uh you know, one of
  25. 0:54maybe generously speaking four AGI
  26. 0:57efforts in the world, right? And you
  27. 0:59want to find something like that. And
  28. 1:00I'm sure that there exists something
  29. 1:02today that just wasn't possible at all
  30. 1:04pre like automated coding era uh that is
  31. 1:08totally nonobvious that will be, you
  32. 1:10know, a multi-t trillion dollar market
  33. 1:13soon. Uh and that only four companies
  34. 1:14are working on right now. But I don't
  35. 1:16know what that is. It's much more likely
  36. 1:17you all know what that is than I know
  37. 1:19what that is. I just, you know, my brain
  38. 1:20is like taken over by OpenAI. Um, but
  39. 1:24you know, the kind of idea someone can
  40. 1:25assign you to work on is probably not
  41. 1:27what you want.
  42. 1:28>> Yep. Um, okay. So, that that's fair. Um,
  43. 1:32but I think it would be helpful since
  44. 1:33this is a systems class to maybe uh
  45. 1:36reason about a particular problem that
  46. 1:38you have to reason through so that they
  47. 1:40can then apply the shape of the
  48. 1:42techniques used to break down from a
  49. 1:44systems perspective that problem into
  50. 1:45solutions to their own problem.
  51. 1:47>> Yeah. Um and a and a concept that uh you
  52. 1:50had started to tease in the class you
  53. 1:52know back in 2014 and then uh clearly
  54. 1:54you've talked about publicly over the
  55. 1:55years is um scale right scale is its own
  56. 1:59beast. It's it it's you know quantity is
  57. 2:01its own quality what scale as a concept
  58. 2:04has been something it seems like you've
  59. 2:07um empirically investigated in all kinds
  60. 2:10of ways over the last 10 years. So um
  61. 2:12could you help help us first unpack like
  62. 2:14what you mean by scale now 10 years
  63. 2:16later how would you deconstruct that as
  64. 2:18a systems design uh attribute to apply
  65. 2:21whether it's a as as a tool um can we
  66. 2:24start there?
  67. 2:25>> Yes. Uh so I don't know why the
  68. 2:29following observation is true. I offer
  69. 2:32no theory
  70. 2:35that I find satisfying to explain it.
  71. 2:37And that makes me a little bit nervous
  72. 2:39to suggest you follow it, but I'm going
  73. 2:42to anyway because empirically it does
  74. 2:44seem to be true, which is all of the
  75. 2:47most interesting things I have observed
  76. 2:49in my career in watching other uh things
  77. 2:52happen. All of the most interesting ones
  78. 2:55uh have had something to do with
  79. 2:59emergent properties that scale or scale
  80. 3:01continuing to provide returns far beyond
  81. 3:04what the consensus thinks will work. And
  82. 3:07this obviously happens with like scaling
  83. 3:08laws for AI models. Um but this happens
  84. 3:12with uh you know getting more smart
  85. 3:15people together to think about one
  86. 3:16problem. This h in a in a research
  87. 3:18setting. Um this happens with uh
  88. 3:22companies and the sort of economy of
  89. 3:24scale. You can get all the in all these
  90. 3:25different ways. I really learned this at
  91. 3:27Y Combinator when uh it became clear to
  92. 3:30me that everybody was saying, "Oh, Y
  93. 3:32Combinator's gotten too big. It should
  94. 3:34shrink. We should fund less companies
  95. 3:35per batch." you know, the best times of
  96. 3:36Y Combinator when it was like 10
  97. 3:38companies per batch. And a lot of like
  98. 3:40very smart people were saying this and
  99. 3:45and it was like tempting because it
  100. 3:46would have been like much less work and
  101. 3:47the theory was that, you know, the best
  102. 3:49companies are always kind of obvious and
  103. 3:51then you fund the rest and it's not as
  104. 3:52helpful. Um, but a huge part of the
  105. 3:54magic of what made YC work were uh was
  106. 3:58the sort of the network effects inside
  107. 4:00of the batch and that was an emergent
  108. 4:02property at scale that just hadn't been
  109. 4:04discovered before. No one had tried to
  110. 4:05fund startups
  111. 4:07at scale in the same way and and thus no
  112. 4:10one had ever happened upon this
  113. 4:12observation of when you do that um
  114. 4:15there's
  115. 4:16there's something important that happens
  116. 4:18that just didn't exist at all at the one
  117. 4:20or 1/100th of a scale.
  118. 4:23There's a bunch of other examples like
  119. 4:25this. Uh I
  120. 4:29and I'll skip them in the interest of
  121. 4:31time, but I I would say again I offer no
  122. 4:35explanation for why, but empirically
  123. 4:36speaking, when you find a time that you
  124. 4:39can push on,
  125. 4:41you can push something to a scale people
  126. 4:42have not tried before and it's already
  127. 4:44working in some interesting way at the
  128. 4:46smaller scale. More often than not, that
  129. 4:48seems to be a good idea. And it also
  130. 4:50seems to be something that
  131. 4:54most people don't do enough.
  132. 4:56>> And I don't offer an explanation for
  133. 4:58this either, but like in, you know, when
  134. 5:00we were like, we're really going to
  135. 5:01scale AI models. Um, all of the like
  136. 5:04geniuses in the field, most of them
  137. 5:06were, oh, this isn't really working. You
  138. 5:07know, that's barely a scientific result.
  139. 5:09It's not interesting that it gets better
  140. 5:10at scale. You've already shown that. Why
  141. 5:12keep scaling it? I mentioned the YC
  142. 5:14example. Um, I've seen a lot of
  143. 5:18startup founders where they're like,
  144. 5:19"Well, you know, there might be
  145. 5:21something interesting that would happen
  146. 5:22if I scaled this up, but I'm a little
  147. 5:25worried about it for non-specific
  148. 5:26reasons." And again, looking back at
  149. 5:28like a huge data set of people that have
  150. 5:31scaled their companies in all these
  151. 5:34different ways. There's almost always
  152. 5:35interesting stuff there. So, I think
  153. 5:37directionally that's like an interesting
  154. 5:39thing to push on and
  155. 5:42severely underexplored.
  156. 5:44Um on the systems design part of that uh
  157. 5:50I think one reason people don't do it as
  158. 5:53much is stuff breaks uh at an
  159. 5:57accelerating rate and in an
  160. 5:58unpredictable way as you scale it
  161. 6:00>> and if you are going to really scale
  162. 6:02something um
  163. 6:05it's always like a little bit broken.
  164. 6:07there are always like very smart people
  165. 6:09who say why you shouldn't do this you
  166. 6:11know don't get too ambitious don't get
  167. 6:12too big let's try this smaller and so
  168. 6:15breaking that down as a systems problem
  169. 6:17I use the thing of when we were like
  170. 6:19scaling up AI models there was
  171. 6:21technically can we do this at all this
  172. 6:22seems crazy like no one had ever thought
  173. 6:24about trying to do a run across 10,000
  174. 6:26or 100,000 GPUs and that was going to
  175. 6:27require stacks of engineering talent um
  176. 6:30there was the capital requirements and
  177. 6:32what was going to take to do this and
  178. 6:34like how is there ever going to be a
  179. 6:35business how can you think about taking
  180. 6:37this risk
  181. 6:38uh there was this sort of like cultural
  182. 6:39stuff of researchers saying well if
  183. 6:40we're going to get all this compute why
  184. 6:42do we put it all into this one project
  185. 6:43we're not going to learn something why
  186. 6:44not divide it up among all these all
  187. 6:46these projects and this also happens in
  188. 6:49kind of every area I've looked at almost
  189. 6:51every area for scale and breaking it
  190. 6:53down into the sort of each difficult
  191. 6:57area or each reason not to do it and
  192. 6:59trying to address them one at a time
  193. 7:01that's been really important
  194. 7:03>> um
  195. 7:04I'm going to push on that a little bit
  196. 7:06because
  197. 7:07there's very few people who've been able
  198. 7:08to sort of repeatedly scale new products
  199. 7:12and systems the way uh the OpenAI team
  200. 7:14has over the years. But it seems like
  201. 7:16one of the issues is there there are all
  202. 7:19these prior conditioning sort of mental
  203. 7:22models and expectations humans have and
  204. 7:24you said things break and one of the
  205. 7:26things it seems often breaks that's hard
  206. 7:29the hardest to refactor is is human the
  207. 7:32human side of the the systems design
  208. 7:35right wherever there's human
  209. 7:36implementers or there's uh human
  210. 7:38participants in that. And so what have
  211. 7:40you learned about humans at scale? Like
  212. 7:41organizing humans at scale to
  213. 7:43participate in a system that may not be
  214. 7:45uh like just a redo of some past system
  215. 7:48that they they get naively on at a
  216. 7:51priority on first blush. Um
  217. 7:55I think like clear [snorts]
  218. 7:57a clear goal, a clear plan to get there.
  219. 8:01uh and like a clear
  220. 8:06answer to the way that you're going to
  221. 8:08get there and kind of how you're going
  222. 8:09to make decisions along the way. That's
  223. 8:10that's very important. So, um you know,
  224. 8:14if we go back to the example of when we
  225. 8:15decided to scale up models, there were a
  226. 8:17lot of people who were like, "Ah, this
  227. 8:19isn't really going to work. It's going
  228. 8:20to have these problems. It's also not,
  229. 8:21you know, we need a more diversified
  230. 8:23portfolio." But once we say, "No, we're
  231. 8:25going to make a bet on scaling deep
  232. 8:26learning." Like, that's our thing. If
  233. 8:27we're wrong, we'll fail. But we're going
  234. 8:29to do that. here's why we're going to do
  235. 8:30that. Here's what we believe about what
  236. 8:32the state of the world could be like if
  237. 8:33we get there. Uh that's very powerful.
  238. 8:37And then
  239. 8:39for whatever reason, um we did not
  240. 8:42evolve to be good at thinking about
  241. 8:45exponentials.
  242. 8:47People have a hard time imagining that
  243. 8:50scaling laws are going to continue
  244. 8:52exponentially, that revenue will grow
  245. 8:53exponentially, that an organization can
  246. 8:55take on exponential complexity. And in
  247. 8:59my experience, it takes a lot of time to
  248. 9:01really reason through first principles
  249. 9:03with people about why why that can
  250. 9:05happen. Can we take two examples uh to
  251. 9:08walk through that? The first being Chach
  252. 9:10and the second being Codeex. You know,
  253. 9:12both of these have transformed. Can can
  254. 9:14everyone hear? I'm going to try to
  255. 9:15project it. Yeah. Okay. Um so let let me
  256. 9:19put in a frame and you can challenge
  257. 9:21both the assumption and then we can
  258. 9:22hopefully reason to example what
  259. 9:24happened in the case of Chad GP. You
  260. 9:26know for a long time in scaling of
  261. 9:27models a big mental block that seemed to
  262. 9:31be prevalent in the space is what are
  263. 9:33these things going to be useful for this
  264. 9:35is you know it's a research uh sort of
  265. 9:37solution solution chasing a problem
  266. 9:39research first approach it's not a
  267. 9:41product um and then you know chat GPD
  268. 9:44came out and it proved to the world that
  269. 9:46you know that chat experience was a
  270. 9:48killer app for general models um at
  271. 9:51scale for consumers and then a couple of
  272. 9:54years later It's clear that coding has
  273. 9:57been the killer enterprise app. So what
  274. 9:59how would you compare and contrast the
  275. 10:01systems you guys used to discover those
  276. 10:03use cases, ship them, scale them,
  277. 10:05monetize them? Any any salient learnings
  278. 10:08from those two systems?
  279. 10:09>> Yes. Um
  280. 10:12so we had made GPT3 and
  281. 10:16we needed to make money because we
  282. 10:17wanted to go scale up to you know a
  283. 10:19billion and multi-billion dollar
  284. 10:20computers and we had GBT3 and it was
  285. 10:22kind of interesting. It was a cool demo,
  286. 10:23but we couldn't figure out a product to
  287. 10:25build around it. And we had been
  288. 10:27thinking thinking, we just couldn't do
  289. 10:28it. We had tried a few things. They they
  290. 10:30hadn't worked. Um, and so we knew the
  291. 10:32models were going to get better, but we
  292. 10:33also wanted to like start a revenue
  293. 10:35engine sooner. And we said, well, since
  294. 10:38we can't figure out what product to
  295. 10:40build, we're just going to put this into
  296. 10:41an API, and we're going to hope that
  297. 10:43somebody else can figure out what
  298. 10:44product to build. And so, we launched in
  299. 10:46like, I don't know, something in the
  300. 10:48summer of 2020 the GPD3 API. And
  301. 10:53initially it kind of got no traction at
  302. 10:56all. And then about a month later,
  303. 10:59randomly, as far as we can tell, it went
  304. 11:02viral on Twitter. On the same day, uh a
  305. 11:04few different developers kind of found
  306. 11:06got it to do something cool, posted it,
  307. 11:07other people started trying and and then
  308. 11:10like a lot of people started trying the
  309. 11:13API. Um but it was shockingly bad. If
  310. 11:16you go back and use GBT3 or 3.5 um you
  311. 11:21will be astonished at how bad the models
  312. 11:23were then uh relative to the amount of
  313. 11:25excitement they generated at the time.
  314. 11:27Uh so people tried all these things and
  315. 11:29really the only business that people got
  316. 11:31to work in a significant way with GPT3
  317. 11:35was copyrighting. Um and that was like
  318. 11:38not that great and not that exciting and
  319. 11:39we were kind of like you know ah it's
  320. 11:41just going to have to wait for a better
  321. 11:42model. But although no, that was the
  322. 11:45only business that was working,
  323. 11:47developers had figured out how to like
  324. 11:49put in a prompt and get and be able to
  325. 11:51chat with it. And we saw this a lot like
  326. 11:54more people were using they couldn't get
  327. 11:58the API to work for their business, but
  328. 12:00they were using their API key to just
  329. 12:01chat. And we said, well, we can build a
  330. 12:03good chatbot. People clearly want that.
  331. 12:05And we had a new model. We actually had
  332. 12:07GP4 done, but we had a new model we were
  333. 12:09ready to release in between called 3.5.
  334. 12:12And we had figured out a new kind of
  335. 12:13post training where we could get the
  336. 12:15models to do like a good job with
  337. 12:16instruction following so make it easier
  338. 12:18to chat with. And we said, well, you
  339. 12:21know, the API is not working great.
  340. 12:24Maybe it was like a 10 or a $20 million
  341. 12:26run rate kind of business, but there is
  342. 12:28this thing that people love. Uh, and
  343. 12:31under the YC principle of see what your
  344. 12:33users love and do that, we said we'll
  345. 12:34we'll build a chatbot around it. And we
  346. 12:36put that out and we still didn't think
  347. 12:38it was going to do that well. Uh there
  348. 12:40was it was really meant as like a
  349. 12:41research demo uh to convince other
  350. 12:44people that they should build chat light
  351. 12:45products and pay us for the API
  352. 12:48but that went like crazy viral and
  353. 12:50another thing I had learned from YC is
  354. 12:52when something really starts growing and
  355. 12:54it's not very good you have like a
  356. 12:56guaranteed hit on your hands and so we
  357. 12:58had like five days where the traffic
  358. 13:01would shoot up fall off and everybody be
  359. 13:03like well that was just a hype cycle but
  360. 13:04then the next day it would get to a
  361. 13:06higher peak fall off again later in the
  362. 13:07day people would say that's a hype
  363. 13:09cycle.
  364. 13:10By the fourth or fifth day, I was like,
  365. 13:11I know how this works. I know what's
  366. 13:13going to happen. Like, we have the
  367. 13:14potential here
  368. 13:16>> at a killer product. Um, and we knew we
  369. 13:20could make it much better. We knew we
  370. 13:21could, we knew we had GPT4. We knew we
  371. 13:23could keep scaling. Um, but by that
  372. 13:28fifth day, we got everybody together and
  373. 13:30said, "This is an emergency. This is a
  374. 13:32good kind of emergency, but we have to
  375. 13:34build a company and a product all at
  376. 13:36once." Mhm.
  377. 13:37>> Uh we then had like 2 months of crazy
  378. 13:40scaling. Uh and then we said, you know,
  379. 13:43we have to figure out a business model
  380. 13:44later. For now, we're just going to
  381. 13:46charge people so that we don't like run
  382. 13:48out our compute bills. But that's
  383. 13:49obviously not the long-term answer. That
  384. 13:51also turned out just to work. Um and
  385. 13:54that was the story of Chach. And then
  386. 13:55there was so much utility that people
  387. 13:58just had not gotten over the activation
  388. 14:00energy of find that that has worked
  389. 14:02really well. Um and then codeex.
  390. 14:06Actually the plan before chatbt was that
  391. 14:08we were going to go all in on code.
  392. 14:10>> Um we knew these models could write
  393. 14:11code. Uh we knew that they could be
  394. 14:14really and we knew that that would be
  395. 14:16like a valuable area. But then we had
  396. 14:17this incredibly exciting thing happen.
  397. 14:19Um but our kind of internal belief at
  398. 14:21the time was that coding was how these
  399. 14:25models would control things on computers
  400. 14:27and robots were how these models would
  401. 14:30control things in the physical world.
  402. 14:31And if you made a smart enough model
  403. 14:33that had sort of the actuators of
  404. 14:34writing code and robot and driving a
  405. 14:37robot, you could then kind of actually
  406. 14:40get this intelligence to do stuff for
  407. 14:41you in the world.
  408. 14:42>> So, uh, then it took us a while to get
  409. 14:44there. And then I think Codeex got
  410. 14:47really good by early this year, but with
  411. 14:505.5 is when we saw this real inflection
  412. 14:52point where people are now like doing
  413. 14:55just incredible things with it. And um
  414. 14:58you know that we earlier in the class
  415. 15:00we've talked about how the capabilities
  416. 15:02pipeline uh is starting to look is
  417. 15:05starting to become somewhat more legibly
  418. 15:07standard across different research
  419. 15:09groups. You got you know pre-training,
  420. 15:11mid-training, post- training. Then you
  421. 15:12got the RL and supervised feedback loop.
  422. 15:14Is do you think that's roughly like the
  423. 15:16shape of the pipeline that allowed
  424. 15:18codeex to you know go through a
  425. 15:20capability jump and that will basically
  426. 15:21stay stable now and consistent or are we
  427. 15:23going to go through a major rewrite of
  428. 15:24that pipeline? I think that is
  429. 15:26definitely the current pipeline. I
  430. 15:28expect we will go through a major
  431. 15:29rewrite. I don't know when it'll happen
  432. 15:30or exactly how. Um, but
  433. 15:34it is a little odd to me that it's so
  434. 15:37happens as a pipeline and doesn't quite
  435. 15:40feel like the optimal solution. Um, what
  436. 15:44would be an optimal solution in your
  437. 15:46head?
  438. 15:46>> I think that's a research problem for
  439. 15:48the AIS to figure out. Um, I think we're
  440. 15:50at a point where and we've set this goal
  441. 15:52that by September of this year, we will
  442. 15:54use 500,000 A100 equivalent GPUs, like a
  443. 15:57lot of computing power, as an AI
  444. 15:59research intern, and by March of 2028
  445. 16:01that we will have a full endto-end very
  446. 16:04talented researcher like figuring out
  447. 16:06complete new architectures. Um, so I
  448. 16:09think we are going to get like with the
  449. 16:11current pipeline, the current
  450. 16:12architectures, I think we're going to
  451. 16:14get over the line of when AIs can do
  452. 16:16incredible, incredible work. Um, you
  453. 16:19know, one of the things that you you
  454. 16:22just described there
  455. 16:25is is you you we we've been talking a
  456. 16:27lot in the class about systems,
  457. 16:29frameworks and analogies to make
  458. 16:30concepts from one domain legible to
  459. 16:32other people who may not have all the
  460. 16:33context in another. And that sometimes
  461. 16:36because of the translation problem, you
  462. 16:39know, reasoning by analogy is not
  463. 16:40helpful because then errors compound.
  464. 16:42>> Yeah. Um right there you said you know
  465. 16:44our goal is to try to use it as an AI
  466. 16:46intern which obviously is a very useful
  467. 16:48metaphor within the context of you know
  468. 16:50Silicon Valley a class that understands
  469. 16:52how these pipelines work and so on and
  470. 16:54then as as you scale actually that
  471. 16:56metaphor globally people who might not
  472. 16:57have all that context go start
  473. 16:59analogizing these models in ways that
  474. 17:00they shouldn't be like how should we
  475. 17:02think about the limits of of that of of
  476. 17:06what are the limits to scale of um what
  477. 17:08are the product analogies the research
  478. 17:11analogies you find most useful
  479. 17:12within the valley and which one of the
  480. 17:14what have you found about the limits of
  481. 17:17those analogies scaling and now how do
  482. 17:19you navigate between those two problems?
  483. 17:23I I've been very interested in studying
  484. 17:25how like
  485. 17:28I I think what is happening is we are we
  486. 17:30are in the process of creating a new
  487. 17:31utility. This doesn't happen very often.
  488. 17:33you know, electricity is utility,
  489. 17:34internet's a utility, there water, I
  490. 17:37guess there's not a lot of these. Uh,
  491. 17:38and so there are not a lot of examples
  492. 17:40that we can study for good metaphors or
  493. 17:43learnings about how to explain this to
  494. 17:46the world. Um, but I was recently
  495. 17:49looking at what happened when
  496. 17:50electricity became a utility. And it's a
  497. 17:54good analogy for many reasons. It's
  498. 17:55imperfect, of course, too. But the
  499. 17:58electricity companies, at least the ones
  500. 17:59I could find information about, they
  501. 18:01didn't talk about selling electricity
  502. 18:02cuz no one knew what that was or why
  503. 18:04they wanted. It sounds like very scary.
  504. 18:05It's this thing that's like going to
  505. 18:06come into your house and it can kill you
  506. 18:08in this like gruesome way and you, you
  507. 18:10know, it feels sort of like very
  508. 18:12different than the world before. Uh, and
  509. 18:15maybe they tried to sell electricity or
  510. 18:17market electricity at first. I don't
  511. 18:19know. But in any case, that didn't work.
  512. 18:20And then what they started marketing
  513. 18:23selling to people was light at night.
  514. 18:25You know, we are going to what you are
  515. 18:27getting from us is not electricity. It's
  516. 18:29light at night. By the way, you can use
  517. 18:31the same thing that lets you get light
  518. 18:33for all these other things. But people
  519. 18:34are like, well, why would I want that?
  520. 18:35And they're like, well, you know, it'll
  521. 18:37wash your clothes for you someday. And
  522. 18:38no, no, it won't. I can't. That's too
  523. 18:40far of a jump for me,
  524. 18:41>> right?
  525. 18:41>> Um, so I don't know what our analogy
  526. 18:46for this should be. Um, but I suspect
  527. 18:49that even if even if we're totally right
  528. 18:52and intelligence is going to become this
  529. 18:54new utility that every company, every
  530. 18:57customer, uh, every government just
  531. 18:59needs access to and is going to use in
  532. 19:01all sorts of incredible ways and you
  533. 19:03will have like a OpenAI token
  534. 19:05subscription that you will plug into
  535. 19:06everything and use to access everything
  536. 19:08and you have running for you all the
  537. 19:09time and doing this amazing stuff. I
  538. 19:12kind of don't think at least right now
  539. 19:14the right way for us to analogize that
  540. 19:15is we're selling intelligence because
  541. 19:17people are just like somehow not
  542. 19:19resonating. I don't know what our
  543. 19:21equivalent of we're selling you light at
  544. 19:24night is going to be. But I think if
  545. 19:26we're going to become a new utility, we
  546. 19:28need to find a way to explain to the
  547. 19:30world what it means to have this like
  548. 19:32intelligence pike that you can just do
  549. 19:34whatever you'd like with
  550. 19:35>> it. So um one
  551. 19:39question that has emerged an emerging
  552. 19:41property of this class of of having a
  553. 19:43diversity of different speakers is that
  554. 19:45the utility analogy has come up several
  555. 19:47times but in reference to different
  556. 19:48things. So Jensen likened ut like
  557. 19:51compute to a utility um and why there
  558. 19:54should be access and so on and talked
  559. 19:56about how Stanford should pull budget
  560. 19:58and so on and and and procure that as a
  561. 20:00utility for everybody on campus whereas
  562. 20:02you just likened the intelligence part
  563. 20:04to a util are both of these things true
  564. 20:06is one of them true one is one more
  565. 20:07likely to be true how should people
  566. 20:08reason about compute as a utility versus
  567. 20:10tokens as a utility and and by comput I
  568. 20:12mean here chips versus tokens does that
  569. 20:15make sense
  570. 20:16>> I think as a consumer as like a business
  571. 20:19or an individual um you will think in
  572. 20:22something closer to tokens or probably
  573. 20:24even one level up from tokens. I don't
  574. 20:27think you'll care very much about you
  575. 20:29know where the hardware is, what
  576. 20:31particular chip it is, what's powering
  577. 20:33it. I think that stuff will be
  578. 20:34abstracted out and what you will care
  579. 20:36about is when you're interacting with
  580. 20:39the system. Um
  581. 20:41can you use it a lot? Is it cheap? Is it
  582. 20:43doing a good job? Um so right now it's
  583. 20:46like tokens. it may get as we move into
  584. 20:49a world where we all just have like this
  585. 20:51constant agent running for us being
  586. 20:53useful to us all of the time. Um you may
  587. 20:55think about it as even one level up. But
  588. 20:57yeah, my my guess is is you when you
  589. 21:00like pay for your cell phone bill,
  590. 21:02you're like, "All right, I'm buying
  591. 21:04access to airtime and some number of
  592. 21:07gigabytes and you know, it's going to do
  593. 21:08all these things and I'll use all these
  594. 21:09apps and whatever else." But like what
  595. 21:11you think about paying for the kind of
  596. 21:13internet utility in this case is just
  597. 21:15like access to the whole system and the
  598. 21:19particular hardware at the base station
  599. 21:21and how it connects to the internet. You
  600. 21:23don't think about that as much. Um I
  601. 21:25know I could nerd out about utility
  602. 21:27infrastructure for a long time, but I
  603. 21:28want to make sure we switch a little bit
  604. 21:29to being relevant for the students.
  605. 21:31Usually we have uh questions, but we're
  606. 21:33not hearing those today unless you're
  607. 21:35comfortable. Oh, okay. Great. How about
  608. 21:37that? Improv. Okay. Uh so one final
  609. 21:41question to start getting the creative
  610. 21:42juices flowing is um the final project
  611. 21:44for this class
  612. 21:46183 is the oneperson frontier lab. So
  613. 21:49everybody here is working on projects
  614. 21:51where they're simulating being an
  615. 21:52individual uh as a lab with access to
  616. 21:55all the right tools. They've got
  617. 21:56hundreds of thousands of dollars of
  618. 21:57credits from Cloudflare. I think we've
  619. 21:59got some open AI tokens maybe. But
  620. 22:01there's a bunch of comput disposal. Um,
  621. 22:04what would you, if you were in the
  622. 22:06class, what would you be working on for
  623. 22:07your oneperson frontier lab project?
  624. 22:10First of all, I think that's an awesome
  625. 22:11project. Um,
  626. 22:14I think this is top of mind because uh
  627. 22:18you we we were just like talking about
  628. 22:20utility frame frameworks. I think
  629. 22:23there's a lot of very smart people
  630. 22:24working on uh great training ideas and
  631. 22:28we're going to have incredible models.
  632. 22:30No matter what you all do, we're going
  633. 22:32to have incredible models. I promise
  634. 22:33here uh like pretty quickly but
  635. 22:37I I think we have not invested enough in
  636. 22:39being able to deliver at scale huge
  637. 22:42amounts of cheap intelligence. So maybe
  638. 22:44I would go work on like the inference
  639. 22:45part of the stack [clears throat]
  640. 22:46>> and how are we going to get this
  641. 22:48incredible intelligence to be cheap and
  642. 22:50abundant? Uh I think that's
  643. 22:52underinvested and and and I think all of
  644. 22:54the frontier labs are going to have to
  645. 22:56become inference companies to a
  646. 22:58significant degree.
  647. 22:59Um, okay.
  648. 23:02It might be too late to pivot your
  649. 23:03projects, but better late than never.
  650. 23:05>> Work on whatever you want to work on.
  651. 23:06[laughter]
  652. 23:08>> Uh, okay. Let's start doing questions
  653. 23:09and I'm going to moderate and try to be
  654. 23:11not, you know, please try to be
  655. 23:13productive and not spicy, etc. Remember,
  656. 23:15it's a CS class, but up to you Sam is
  657. 23:18fine.
  658. 23:19>> Oh, we've got questions. Oh, perfect.
  659. 23:21All right. First one, the questions
  660. 23:23about your views on Yan Lun's view that
  661. 23:25LM are a dead end. Um,
  662. 23:29first of all, in terms of achieving
  663. 23:31human level intelligence, these models
  664. 23:33have already far surpassed human
  665. 23:35intelligence in some ways and then
  666. 23:37they're wildly worse than others. Like
  667. 23:39for example, they seem much worse than
  668. 23:41people are at very long horizon
  669. 23:46kind of high judgment signal and tasks.
  670. 23:50Um on the other hand yesterday we had
  671. 23:54one of our models uh discover or
  672. 23:57disprove a conjecture one of the airish
  673. 23:59problems that had smart people had
  674. 24:01worked on for a long time and a lot of
  675. 24:03people a lot of smart scientists I don't
  676. 24:05know if lun was one of them or not had
  677. 24:07even quite recently said something like
  678. 24:09that was not going to happen. Uh and
  679. 24:11then like the model just did it and you
  680. 24:13know now you have all these
  681. 24:14mathematicians saying like is math over?
  682. 24:16What does this mean for our field? So
  683. 24:18clearly LLMs are capable of figuring out
  684. 24:22new knowledge and clearly they are
  685. 24:24capable of doing some things that some
  686. 24:26intelligence tasks that humans just
  687. 24:28can't do. Um they're going to scale much
  688. 24:31further. So how much better and what
  689. 24:32distribution of the tasks they can do
  690. 24:34better than humans? We'll find out. But
  691. 24:36I suspect it's a lot. And the you know
  692. 24:38in terms of this like lack of a belief
  693. 24:41in the exponential we were talking about
  694. 24:42earlier. Um I think the field was
  695. 24:45honestly held back by a generation of
  696. 24:48scientists who just were way too certain
  697. 24:50on what wouldn't what what scaling was
  698. 24:52not going to produce and then some
  699. 24:54people just looked at the graphs and
  700. 24:55said well it looks like it's continuing
  701. 24:57beautifully let's keep going. Um
  702. 25:00I think world models are clearly
  703. 25:03important and to
  704. 25:06we'll need that for things like
  705. 25:07robotics. Uh but betting
  706. 25:10against LLM scaling at this point
  707. 25:15uh feels quite misguided to me.
  708. 25:19Does it get annoying to be the I told
  709. 25:20you so guy?
  710. 25:21>> No. I mean
  711. 25:24there are these like Twitter trolls that
  712. 25:26you know for years have just been like
  713. 25:28it's not going to work. It's not going
  714. 25:29to work. This is so dumb. Like you know
  715. 25:30this is a fraud. This company is going
  716. 25:32to fail. This research approach is going
  717. 25:33to fail. And I used to get more bothered
  718. 25:35by them. But I don't even like feel the
  719. 25:37I told you so at this point. It's like
  720. 25:38you were just like never.
  721. 25:40>> You're still going on about it. Like the
  722. 25:42data is
  723. 25:45>> quite strong on our side.
  724. 25:48>> It I don't think it'd be that fun to say
  725. 25:49I told you so. And also the fact that
  726. 25:51you're like still saying we're wrong
  727. 25:52doesn't really bother me.
  728. 25:53>> I think there's that kind of move on.
  729. 25:55>> There's that saying that like insanity
  730. 25:56is doing the same thing over and over
  731. 25:58again when presented with data that is
  732. 26:00not working and they keep repeating
  733. 26:01that. And in a sense it's it's it's a
  734. 26:03form of insanity. I think
  735. 26:05>> I I think there's something that happens
  736. 26:07which is if you make your identity about
  737. 26:09a particular
  738. 26:11thing is going to work or not work
  739. 26:14and you associate yourself with that
  740. 26:16belief and then the science or the
  741. 26:18empirical results disprove you and
  742. 26:21you're like too hung up on your
  743. 26:23identity. You can't let it go. You can't
  744. 26:24see the truth.
  745. 26:25>> Yeah.
  746. 26:26>> And I think this is like a important
  747. 26:28reminder in both directions.
  748. 26:29>> Yeah.
  749. 26:31>> How do you see education?
  750. 26:33Um, it clearly has to super adapt and I
  751. 26:37am worried. I I thought by now it would
  752. 26:39have. Um, the the I think if we continue
  753. 26:43to teach and evaluate students
  754. 26:46as if we were in a pre- agi world, um,
  755. 26:50it's not going to work and it is going
  756. 26:51to lead to like atrophy of learning how
  757. 26:53to think or whatever. And I thought that
  758. 26:55was going to be obvious enough that I
  759. 26:57wasn't that worried. You know, when
  760. 26:58ChhatgBT launched, I was like, "Yeah,
  761. 26:59we're going to have one year of like
  762. 27:01students like cheating and not learning
  763. 27:03that much. And then the educational
  764. 27:05system is just going to redesign itself.
  765. 27:06And there's and we're going to teach
  766. 27:08people so much better." You know, people
  767. 27:09are going to really
  768. 27:11get projects where they have to they
  769. 27:14have to use AI to be able to do it, but
  770. 27:15they still have to like stretch their
  771. 27:17brain more and think more and figure out
  772. 27:18new things to do. And honestly, I
  773. 27:22struggle to point to any significant
  774. 27:24systemic change that I've seen in the
  775. 27:26education system at large in the three
  776. 27:29and a half years since Chad launched.
  777. 27:30And I that was a prediction error for
  778. 27:32me. I thought I thought that would have
  779. 27:33happened. So I have no doubt that we can
  780. 27:38uh like we have done with every other
  781. 27:39technological leap before redesign how
  782. 27:42education works so that you still have
  783. 27:44to learn how to think. And there will be
  784. 27:46some things like I I I am a person who
  785. 27:49thinks by writing and I write a lot of
  786. 27:52stuff that I never show anyone else but
  787. 27:54it's still important to me to figure
  788. 27:55something out and so I'm grateful that I
  789. 27:57I learned to write. People say the same
  790. 27:58thing about programming. Um so there
  791. 28:01will be some things that we teach people
  792. 28:02to do that machines can do better just
  793. 28:04because it's helpful to teach them the
  794. 28:08meta skill of thinking and learning and
  795. 28:09that makes sense. But there are a lot of
  796. 28:11other things where we should just
  797. 28:12totally teach totally change how we
  798. 28:15teach or how we learn or how we
  799. 28:16evaluate. And
  800. 28:19if we don't do that, I think there will
  801. 28:20be like significant atrophy in people's
  802. 28:23critical thinking skills.
  803. 28:24>> Uh question is what was your favorite
  804. 28:26class and what what do you wish you had
  805. 28:27taken while when you were at Stanford?
  806. 28:29>> Does Stanford still do intros?
  807. 28:32I did like all the I did like three
  808. 28:33intros a quarter my freshman year like
  809. 28:36and I loved all of them. Uh they were
  810. 28:38all super different. Uh I but looking
  811. 28:42back the fact that I
  812. 28:45was able to get such a broad exposure to
  813. 28:48stuff and h have like a a very shallow
  814. 28:50understanding of lots of different
  815. 28:51fields was an incredible thing. If it
  816. 28:54had not been for that I just would have
  817. 28:55taken like CS and physics classes which
  818. 28:57still would have been great. But um I I
  819. 29:01think more about the stuff the classes I
  820. 29:04took that were like totally random and
  821. 29:06unrelated to what I do now but in some
  822. 29:08important way
  823. 29:10gave me a perspective then I do I think
  824. 29:12I would have like learned a program no
  825. 29:15matter what. Uh so I and I didn't think
  826. 29:19that at the time I was like kind of like
  827. 29:20yeah you know this is this stuff is all
  828. 29:23cool but it's mostly going to be about
  829. 29:24like learning CS. Um, I only did two
  830. 29:27years of school. Uh, so there was a lot
  831. 29:29of stuff I wanted to take that I didn't
  832. 29:30get to. Um, but that's kind of the
  833. 29:32surprising thing.
  834. 29:34My question is, what is your spiciest
  835. 29:37take of all,
  836. 29:39right?
  837. 29:43I I think with more time to think uh I
  838. 29:46could come up with a much
  839. 29:48spicier one, but um
  840. 29:52I think AI is just going to keep going.
  841. 29:55And
  842. 29:57I think this is considered
  843. 30:00I don't I don't think this is like
  844. 30:01widely believed yet. And I think if this
  845. 30:03were widely believed, there would be
  846. 30:05like significantly more reverberations
  847. 30:07that are happening through society right
  848. 30:09now. And maybe I don't have a spicer
  849. 30:11tag. Actually, maybe this is the high
  850. 30:12order bit that if AI progress continues
  851. 30:15on the exponential that it's on for
  852. 30:18another
  853. 30:20it's been three and a half years since
  854. 30:21tragic if even if it were another three
  855. 30:22and a half years on that same trajectory
  856. 30:24the world
  857. 30:26the potential the way that society
  858. 30:29what's society is capable of are just
  859. 30:32completely different. Well, let me try
  860. 30:33to prompt more thinking tokens on that
  861. 30:35one. um you you have if we treated you
  862. 30:39as a model like as a frontier model and
  863. 30:42you have some inherent capabilities and
  864. 30:44we're going to we're going to try to
  865. 30:44elicit capabilities that people don't
  866. 30:47know about for the next few minutes. Um
  867. 30:49one of them is that you've been
  868. 30:50postrained now on you you've been
  869. 30:52continuously RL on OpenAI as well as the
  870. 30:55external feedback loop of the world on
  871. 30:56what doesn't work and what works and
  872. 30:58doesn't work. So now if we're going to
  873. 31:00treat you as a prediction engine for a
  874. 31:01sec, the prompt is what are the three
  875. 31:04most likely forks of the universe you
  876. 31:06see over the next 10 years and what is
  877. 31:09your what is your probability assessment
  878. 31:10on each of those? Does that make sense?
  879. 31:13One that feels very important is uh like
  880. 31:18how much is this technology going to be
  881. 31:20very widely democratized versus how much
  882. 31:22is it going to sit in a few companies. I
  883. 31:26I think a world there are all of these
  884. 31:28reasons why you could imagine the
  885. 31:29default is that this gets concentrated
  886. 31:31to a few companies and they become like
  887. 31:35you know a significant fraction of the
  888. 31:36wealth on earth that would obviously be
  889. 31:38terrible and we work super hard to push
  890. 31:40against that but I think that's going to
  891. 31:42require like the will of the world to to
  892. 31:44really avoid um because there is a sort
  893. 31:47of a tractor state there and I think
  894. 31:48part of the reason that we need to push
  895. 31:50to this kind of utility model of the
  896. 31:52world is that a it's quite unstable and
  897. 31:55quite and will feel quite unfair if a
  898. 31:57few companies have all this. But B, I
  899. 31:59think there's a real alignment failure
  900. 32:01and a very fragile world. Uh, and the
  901. 32:03best way to get to a world we want that
  902. 32:06represents like everybody winning and
  903. 32:07everybody's values being represented,
  904. 32:09everybody having agency is to just put
  905. 32:11push this technology out into the world.
  906. 32:14Um, but there will be a very strong
  907. 32:15argument against that around sort of
  908. 32:17safety and stability. And I think that
  909. 32:20will be a big fork and it's very
  910. 32:21important and I encourage all of you in
  911. 32:23your careers to push hard that this is a
  912. 32:25technology
  913. 32:27it can bring us an incredible sci-fi
  914. 32:29future life can be unbelievably much
  915. 32:31better we are going to incur some risk
  916. 32:33to get there but the risk of keeping
  917. 32:35this concentrated in a handful of
  918. 32:37companies even though we would be one of
  919. 32:38these companies is not something we
  920. 32:40should tolerate so I think that would be
  921. 32:41a big fork uh in terms of probability I
  922. 32:44think it's
  923. 32:47the world should have such an interest
  924. 32:49in it happening this way that I think
  925. 32:51it's like 80% we end up on the
  926. 32:53democratic path but there will be a very
  927. 32:56strong safety message and you know there
  928. 32:58will be a lot of power seeking people
  929. 33:00who who want to concentrate the power
  930. 33:02and
  931. 33:04one of the problems with forecasting
  932. 33:07this or that you have and we all have as
  933. 33:10humans is once you make that forecast
  934. 33:12then you have agency to affect the
  935. 33:14forecasts right and the forecast for
  936. 33:15>> well I mean we're clear on what we're
  937. 33:17going to use our agency before like this
  938. 33:18is what we believe in. We think that uh
  939. 33:21you know we're going to do everything we
  940. 33:22can to push it in this direction. We
  941. 33:24just we see the forces in the other
  942. 33:26direction. Maybe a related fork uh
  943. 33:30there's a lot of talk about like future
  944. 33:31economic models and are we going to do
  945. 33:33universal basic income? Are we going to
  946. 33:35have everybody gets to like own a slice
  947. 33:37of every company? Like are we going to
  948. 33:39is it capitalism with no change? Is it
  949. 33:42like full-on communism? There's like a
  950. 33:44lot of talk about this. One thing that I
  951. 33:46think is not talked about much is how
  952. 33:49specifically how we distribute compute.
  953. 33:52>> So maybe a lot of the economy can work
  954. 33:56in a way that it's going to work. And
  955. 33:57I've actually I've become much less of a
  956. 33:58even short-term jobs doomer. I've always
  957. 34:00been optimistic we find new things to do
  958. 34:02but this may not be as disrupted as I
  959. 34:05originally thought in the short term. Um
  960. 34:07but we are seeing comput shortages now.
  961. 34:11I can imagine them getting much worse
  962. 34:13and I can imagine compute being like the
  963. 34:15most important utility that people need.
  964. 34:18Uh so if the price of compute from a
  965. 34:20supply and demand perspective gets way
  966. 34:21out of whack then I think there will be
  967. 34:23a very interesting fork about what it
  968. 34:26means to equitably distribute compute.
  969. 34:28So you said two very interesting things
  970. 34:30there which you said on the economic
  971. 34:32side we might have need universal basic
  972. 34:34income. Everybody owns a piece of
  973. 34:36shares. You know, one of the speakers in
  974. 34:38this class is um Nikolai Tangen who runs
  975. 34:42the Norwegian sovereign wealth fund.
  976. 34:43He's awesome.
  977. 34:44>> He's awesome. You know, this Norwegian
  978. 34:46sovereign wealth fund owns 1.5% of all
  979. 34:48publicly traded companies on the planet.
  980. 34:50They also have effectively universal
  981. 34:51basic income. You could argue there's
  982. 34:53flavors of this already today because
  983. 34:55you know the largest employer now in the
  984. 34:56United States is the government and you
  985. 34:58could argue like large sections of that
  986. 35:00or a way for the government to
  987. 35:01redistribute income from taxpayers. So
  988. 35:03are these solutions that actually need
  989. 35:05to be novel or just reimplemented for
  990. 35:08this era? How do you think about the
  991. 35:10novelty of those solutions where we
  992. 35:11often you know in Silicon Valley make
  993. 35:13have this tendency to be like reinvent
  994. 35:16you know old things from first
  995. 35:17principles and so should we just look to
  996. 35:19existing systems and tweak them. Um
  997. 35:20yeah, I don't think that these things
  998. 35:22require
  999. 35:23deeply new ideas. Although I will say um
  1000. 35:28I am much more excited about people
  1001. 35:31having some sort of ownership stake than
  1002. 35:33a fixed monthly cash dividend,
  1003. 35:35>> right?
  1004. 35:36>> Um and I I funded like a big universal
  1005. 35:40basic income study a while ago. I've
  1006. 35:43also watched what happens when people
  1007. 35:45like invest in startups and I know which
  1008. 35:47model I think like hits human psychology
  1009. 35:49better. So what I would love to see is
  1010. 35:52as leverage in the world shifts from
  1011. 35:56labor to capital which I think is going
  1012. 35:58to keep happening
  1013. 35:59that we find a way to have something
  1014. 36:03like a citizens wealth fund in the
  1015. 36:04country or in the world eventually where
  1016. 36:07you like you basically own a slice of
  1017. 36:09capitalism right own slice of these
  1018. 36:10companies. And then on the second fork
  1019. 36:12there on compute bottlenecks, you said
  1020. 36:15uh at some point when compute prices get
  1021. 36:17out of whack between January and this
  1022. 36:19year, my my current understanding is
  1023. 36:21based on data we've seen that H100
  1024. 36:23prices and Blackwell prices, the spreads
  1025. 36:26between long-term reservations and spot
  1026. 36:28is like 5x.
  1027. 36:30>> I don't know if it's that high anymore.
  1028. 36:32I think it got a little better, but
  1029. 36:33yeah,
  1030. 36:33>> tell me.
  1031. 36:34>> Or if you can even find H100s cuz
  1032. 36:36they're pretty much all gone for this
  1033. 36:37year. Does that sound right?
  1034. 36:39>> No argument. There's a gigantic computer
  1035. 36:42shortage. Yeah. So that that's a good
  1036. 36:44example of an of a systems problem right
  1037. 36:46now that's live. At least to some folks
  1038. 36:49it feels like co you know for the
  1039. 36:51compute era like all the toilet paper's
  1040. 36:53gone.
  1041. 36:53>> Yeah.
  1042. 36:55>> Why are people not freaking out about
  1043. 36:56this?
  1044. 36:57>> Well, I think people assume we will make
  1045. 37:00big inference gains on the hardware we
  1046. 37:01have. Uh I also think there is a tsunami
  1047. 37:03of hardware coming. But maybe the demand
  1048. 37:06tsunami is even bigger and I think
  1049. 37:08people should be freaking out somewhat
  1050. 37:09>> and and would you say it's fair like how
  1051. 37:12long are we going to exist in a comput
  1052. 37:13shortage
  1053. 37:14at least you know based on current data
  1054. 37:17you have
  1055. 37:19>> I think like other you you can't talk
  1056. 37:22really about like worldwide demand for
  1057. 37:24electricity without talking about the
  1058. 37:26price like it's there's an extremely
  1059. 37:29different demand about how much energy
  1060. 37:31people use in the world if the price
  1061. 37:32comes down by a factor of 10 or goes up
  1062. 37:34by a factor factor of 10 and I think AI
  1063. 37:38is like that too.
  1064. 37:40uh the
  1065. 37:42if we can make models
  1066. 37:47sufficiently smart and a sufficiently
  1067. 37:50low cost, I think demand is like kind of
  1068. 37:52uncapped. And so in some sense, as long
  1069. 37:54as we can continue to make progress on
  1070. 37:56this, there will be a shortage forever
  1071. 37:58and things will be bid among above what
  1072. 38:01the price we think we think the price
  1073. 38:03should be. Even though people are
  1074. 38:04getting better, smarter, more whatever
  1075. 38:07intelligence just because you can use
  1076. 38:09like
  1077. 38:11if we make really great personal agents
  1078. 38:13and you can have 10 of them running and
  1079. 38:14working for you all the time or 100 and
  1080. 38:16you know you want the hundred. I think
  1081. 38:19it's a lot of inference.
  1082. 38:21>> Awesome. With that, I'm going to give
  1083. 38:22you the swag for the class which is
  1084. 38:28[applause]
  1085. 38:28>> Thank you for coming. Thank you.

About this transcript

This page contains the full transcript of Sam Altman - You no longer need to write prompts. by prabhakar varma, generated from the public captions YouTube serves with the video. The transcript has 7,577 words across 1,085 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.