Sam Altman - You no longer need to write prompts. — Transcript
Full transcript
- 0:00you with like an affordable amount of
- 0:03spend on tokens, you can do what a 100
- 0:06person incredibly great engineering team
- 0:10would do as a startup and that was just
- 0:12totally impossible. That was like not in
- 0:13the set of options for a startup and now
- 0:17it is. So, so I think what you can take
- 0:19on uh the level of ambition you can
- 0:21have, the speed of which you can move,
- 0:23the amount of stuff you can do at once
- 0:25uh is just totally different. And um
- 0:27does that change the shape of the
- 0:29problems you feel like you'd assign at
- 0:32the end of the class for people to
- 0:33attack, you know, at the end of that
- 0:35quarter if you were teaching it again? I
- 0:38don't think assigning problems to attack
- 0:40ever works because if you like if I can
- 0:43think of a problem, if I can think of
- 0:44like a really great startup idea, uh if
- 0:46it's like obvious enough to me, uh then
- 0:49it's probably obvious to a lot of
- 0:51people. when we started OpenAI, we were
- 0:52we were like the uh you know, one of
- 0:54maybe generously speaking four AGI
- 0:57efforts in the world, right? And you
- 0:59want to find something like that. And
- 1:00I'm sure that there exists something
- 1:02today that just wasn't possible at all
- 1:04pre like automated coding era uh that is
- 1:08totally nonobvious that will be, you
- 1:10know, a multi-t trillion dollar market
- 1:13soon. Uh and that only four companies
- 1:14are working on right now. But I don't
- 1:16know what that is. It's much more likely
- 1:17you all know what that is than I know
- 1:19what that is. I just, you know, my brain
- 1:20is like taken over by OpenAI. Um, but
- 1:24you know, the kind of idea someone can
- 1:25assign you to work on is probably not
- 1:27what you want.
- 1:28>> Yep. Um, okay. So, that that's fair. Um,
- 1:32but I think it would be helpful since
- 1:33this is a systems class to maybe uh
- 1:36reason about a particular problem that
- 1:38you have to reason through so that they
- 1:40can then apply the shape of the
- 1:42techniques used to break down from a
- 1:44systems perspective that problem into
- 1:45solutions to their own problem.
- 1:47>> Yeah. Um and a and a concept that uh you
- 1:50had started to tease in the class you
- 1:52know back in 2014 and then uh clearly
- 1:54you've talked about publicly over the
- 1:55years is um scale right scale is its own
- 1:59beast. It's it it's you know quantity is
- 2:01its own quality what scale as a concept
- 2:04has been something it seems like you've
- 2:07um empirically investigated in all kinds
- 2:10of ways over the last 10 years. So um
- 2:12could you help help us first unpack like
- 2:14what you mean by scale now 10 years
- 2:16later how would you deconstruct that as
- 2:18a systems design uh attribute to apply
- 2:21whether it's a as as a tool um can we
- 2:24start there?
- 2:25>> Yes. Uh so I don't know why the
- 2:29following observation is true. I offer
- 2:32no theory
- 2:35that I find satisfying to explain it.
- 2:37And that makes me a little bit nervous
- 2:39to suggest you follow it, but I'm going
- 2:42to anyway because empirically it does
- 2:44seem to be true, which is all of the
- 2:47most interesting things I have observed
- 2:49in my career in watching other uh things
- 2:52happen. All of the most interesting ones
- 2:55uh have had something to do with
- 2:59emergent properties that scale or scale
- 3:01continuing to provide returns far beyond
- 3:04what the consensus thinks will work. And
- 3:07this obviously happens with like scaling
- 3:08laws for AI models. Um but this happens
- 3:12with uh you know getting more smart
- 3:15people together to think about one
- 3:16problem. This h in a in a research
- 3:18setting. Um this happens with uh
- 3:22companies and the sort of economy of
- 3:24scale. You can get all the in all these
- 3:25different ways. I really learned this at
- 3:27Y Combinator when uh it became clear to
- 3:30me that everybody was saying, "Oh, Y
- 3:32Combinator's gotten too big. It should
- 3:34shrink. We should fund less companies
- 3:35per batch." you know, the best times of
- 3:36Y Combinator when it was like 10
- 3:38companies per batch. And a lot of like
- 3:40very smart people were saying this and
- 3:45and it was like tempting because it
- 3:46would have been like much less work and
- 3:47the theory was that, you know, the best
- 3:49companies are always kind of obvious and
- 3:51then you fund the rest and it's not as
- 3:52helpful. Um, but a huge part of the
- 3:54magic of what made YC work were uh was
- 3:58the sort of the network effects inside
- 4:00of the batch and that was an emergent
- 4:02property at scale that just hadn't been
- 4:04discovered before. No one had tried to
- 4:05fund startups
- 4:07at scale in the same way and and thus no
- 4:10one had ever happened upon this
- 4:12observation of when you do that um
- 4:15there's
- 4:16there's something important that happens
- 4:18that just didn't exist at all at the one
- 4:20or 1/100th of a scale.
- 4:23There's a bunch of other examples like
- 4:25this. Uh I
- 4:29and I'll skip them in the interest of
- 4:31time, but I I would say again I offer no
- 4:35explanation for why, but empirically
- 4:36speaking, when you find a time that you
- 4:39can push on,
- 4:41you can push something to a scale people
- 4:42have not tried before and it's already
- 4:44working in some interesting way at the
- 4:46smaller scale. More often than not, that
- 4:48seems to be a good idea. And it also
- 4:50seems to be something that
- 4:54most people don't do enough.
- 4:56>> And I don't offer an explanation for
- 4:58this either, but like in, you know, when
- 5:00we were like, we're really going to
- 5:01scale AI models. Um, all of the like
- 5:04geniuses in the field, most of them
- 5:06were, oh, this isn't really working. You
- 5:07know, that's barely a scientific result.
- 5:09It's not interesting that it gets better
- 5:10at scale. You've already shown that. Why
- 5:12keep scaling it? I mentioned the YC
- 5:14example. Um, I've seen a lot of
- 5:18startup founders where they're like,
- 5:19"Well, you know, there might be
- 5:21something interesting that would happen
- 5:22if I scaled this up, but I'm a little
- 5:25worried about it for non-specific
- 5:26reasons." And again, looking back at
- 5:28like a huge data set of people that have
- 5:31scaled their companies in all these
- 5:34different ways. There's almost always
- 5:35interesting stuff there. So, I think
- 5:37directionally that's like an interesting
- 5:39thing to push on and
- 5:42severely underexplored.
- 5:44Um on the systems design part of that uh
- 5:50I think one reason people don't do it as
- 5:53much is stuff breaks uh at an
- 5:57accelerating rate and in an
- 5:58unpredictable way as you scale it
- 6:00>> and if you are going to really scale
- 6:02something um
- 6:05it's always like a little bit broken.
- 6:07there are always like very smart people
- 6:09who say why you shouldn't do this you
- 6:11know don't get too ambitious don't get
- 6:12too big let's try this smaller and so
- 6:15breaking that down as a systems problem
- 6:17I use the thing of when we were like
- 6:19scaling up AI models there was
- 6:21technically can we do this at all this
- 6:22seems crazy like no one had ever thought
- 6:24about trying to do a run across 10,000
- 6:26or 100,000 GPUs and that was going to
- 6:27require stacks of engineering talent um
- 6:30there was the capital requirements and
- 6:32what was going to take to do this and
- 6:34like how is there ever going to be a
- 6:35business how can you think about taking
- 6:37this risk
- 6:38uh there was this sort of like cultural
- 6:39stuff of researchers saying well if
- 6:40we're going to get all this compute why
- 6:42do we put it all into this one project
- 6:43we're not going to learn something why
- 6:44not divide it up among all these all
- 6:46these projects and this also happens in
- 6:49kind of every area I've looked at almost
- 6:51every area for scale and breaking it
- 6:53down into the sort of each difficult
- 6:57area or each reason not to do it and
- 6:59trying to address them one at a time
- 7:01that's been really important
- 7:03>> um
- 7:04I'm going to push on that a little bit
- 7:06because
- 7:07there's very few people who've been able
- 7:08to sort of repeatedly scale new products
- 7:12and systems the way uh the OpenAI team
- 7:14has over the years. But it seems like
- 7:16one of the issues is there there are all
- 7:19these prior conditioning sort of mental
- 7:22models and expectations humans have and
- 7:24you said things break and one of the
- 7:26things it seems often breaks that's hard
- 7:29the hardest to refactor is is human the
- 7:32human side of the the systems design
- 7:35right wherever there's human
- 7:36implementers or there's uh human
- 7:38participants in that. And so what have
- 7:40you learned about humans at scale? Like
- 7:41organizing humans at scale to
- 7:43participate in a system that may not be
- 7:45uh like just a redo of some past system
- 7:48that they they get naively on at a
- 7:51priority on first blush. Um
- 7:55I think like clear [snorts]
- 7:57a clear goal, a clear plan to get there.
- 8:01uh and like a clear
- 8:06answer to the way that you're going to
- 8:08get there and kind of how you're going
- 8:09to make decisions along the way. That's
- 8:10that's very important. So, um you know,
- 8:14if we go back to the example of when we
- 8:15decided to scale up models, there were a
- 8:17lot of people who were like, "Ah, this
- 8:19isn't really going to work. It's going
- 8:20to have these problems. It's also not,
- 8:21you know, we need a more diversified
- 8:23portfolio." But once we say, "No, we're
- 8:25going to make a bet on scaling deep
- 8:26learning." Like, that's our thing. If
- 8:27we're wrong, we'll fail. But we're going
- 8:29to do that. here's why we're going to do
- 8:30that. Here's what we believe about what
- 8:32the state of the world could be like if
- 8:33we get there. Uh that's very powerful.
- 8:37And then
- 8:39for whatever reason, um we did not
- 8:42evolve to be good at thinking about
- 8:45exponentials.
- 8:47People have a hard time imagining that
- 8:50scaling laws are going to continue
- 8:52exponentially, that revenue will grow
- 8:53exponentially, that an organization can
- 8:55take on exponential complexity. And in
- 8:59my experience, it takes a lot of time to
- 9:01really reason through first principles
- 9:03with people about why why that can
- 9:05happen. Can we take two examples uh to
- 9:08walk through that? The first being Chach
- 9:10and the second being Codeex. You know,
- 9:12both of these have transformed. Can can
- 9:14everyone hear? I'm going to try to
- 9:15project it. Yeah. Okay. Um so let let me
- 9:19put in a frame and you can challenge
- 9:21both the assumption and then we can
- 9:22hopefully reason to example what
- 9:24happened in the case of Chad GP. You
- 9:26know for a long time in scaling of
- 9:27models a big mental block that seemed to
- 9:31be prevalent in the space is what are
- 9:33these things going to be useful for this
- 9:35is you know it's a research uh sort of
- 9:37solution solution chasing a problem
- 9:39research first approach it's not a
- 9:41product um and then you know chat GPD
- 9:44came out and it proved to the world that
- 9:46you know that chat experience was a
- 9:48killer app for general models um at
- 9:51scale for consumers and then a couple of
- 9:54years later It's clear that coding has
- 9:57been the killer enterprise app. So what
- 9:59how would you compare and contrast the
- 10:01systems you guys used to discover those
- 10:03use cases, ship them, scale them,
- 10:05monetize them? Any any salient learnings
- 10:08from those two systems?
- 10:09>> Yes. Um
- 10:12so we had made GPT3 and
- 10:16we needed to make money because we
- 10:17wanted to go scale up to you know a
- 10:19billion and multi-billion dollar
- 10:20computers and we had GBT3 and it was
- 10:22kind of interesting. It was a cool demo,
- 10:23but we couldn't figure out a product to
- 10:25build around it. And we had been
- 10:27thinking thinking, we just couldn't do
- 10:28it. We had tried a few things. They they
- 10:30hadn't worked. Um, and so we knew the
- 10:32models were going to get better, but we
- 10:33also wanted to like start a revenue
- 10:35engine sooner. And we said, well, since
- 10:38we can't figure out what product to
- 10:40build, we're just going to put this into
- 10:41an API, and we're going to hope that
- 10:43somebody else can figure out what
- 10:44product to build. And so, we launched in
- 10:46like, I don't know, something in the
- 10:48summer of 2020 the GPD3 API. And
- 10:53initially it kind of got no traction at
- 10:56all. And then about a month later,
- 10:59randomly, as far as we can tell, it went
- 11:02viral on Twitter. On the same day, uh a
- 11:04few different developers kind of found
- 11:06got it to do something cool, posted it,
- 11:07other people started trying and and then
- 11:10like a lot of people started trying the
- 11:13API. Um but it was shockingly bad. If
- 11:16you go back and use GBT3 or 3.5 um you
- 11:21will be astonished at how bad the models
- 11:23were then uh relative to the amount of
- 11:25excitement they generated at the time.
- 11:27Uh so people tried all these things and
- 11:29really the only business that people got
- 11:31to work in a significant way with GPT3
- 11:35was copyrighting. Um and that was like
- 11:38not that great and not that exciting and
- 11:39we were kind of like you know ah it's
- 11:41just going to have to wait for a better
- 11:42model. But although no, that was the
- 11:45only business that was working,
- 11:47developers had figured out how to like
- 11:49put in a prompt and get and be able to
- 11:51chat with it. And we saw this a lot like
- 11:54more people were using they couldn't get
- 11:58the API to work for their business, but
- 12:00they were using their API key to just
- 12:01chat. And we said, well, we can build a
- 12:03good chatbot. People clearly want that.
- 12:05And we had a new model. We actually had
- 12:07GP4 done, but we had a new model we were
- 12:09ready to release in between called 3.5.
- 12:12And we had figured out a new kind of
- 12:13post training where we could get the
- 12:15models to do like a good job with
- 12:16instruction following so make it easier
- 12:18to chat with. And we said, well, you
- 12:21know, the API is not working great.
- 12:24Maybe it was like a 10 or a $20 million
- 12:26run rate kind of business, but there is
- 12:28this thing that people love. Uh, and
- 12:31under the YC principle of see what your
- 12:33users love and do that, we said we'll
- 12:34we'll build a chatbot around it. And we
- 12:36put that out and we still didn't think
- 12:38it was going to do that well. Uh there
- 12:40was it was really meant as like a
- 12:41research demo uh to convince other
- 12:44people that they should build chat light
- 12:45products and pay us for the API
- 12:48but that went like crazy viral and
- 12:50another thing I had learned from YC is
- 12:52when something really starts growing and
- 12:54it's not very good you have like a
- 12:56guaranteed hit on your hands and so we
- 12:58had like five days where the traffic
- 13:01would shoot up fall off and everybody be
- 13:03like well that was just a hype cycle but
- 13:04then the next day it would get to a
- 13:06higher peak fall off again later in the
- 13:07day people would say that's a hype
- 13:09cycle.
- 13:10By the fourth or fifth day, I was like,
- 13:11I know how this works. I know what's
- 13:13going to happen. Like, we have the
- 13:14potential here
- 13:16>> at a killer product. Um, and we knew we
- 13:20could make it much better. We knew we
- 13:21could, we knew we had GPT4. We knew we
- 13:23could keep scaling. Um, but by that
- 13:28fifth day, we got everybody together and
- 13:30said, "This is an emergency. This is a
- 13:32good kind of emergency, but we have to
- 13:34build a company and a product all at
- 13:36once." Mhm.
- 13:37>> Uh we then had like 2 months of crazy
- 13:40scaling. Uh and then we said, you know,
- 13:43we have to figure out a business model
- 13:44later. For now, we're just going to
- 13:46charge people so that we don't like run
- 13:48out our compute bills. But that's
- 13:49obviously not the long-term answer. That
- 13:51also turned out just to work. Um and
- 13:54that was the story of Chach. And then
- 13:55there was so much utility that people
- 13:58just had not gotten over the activation
- 14:00energy of find that that has worked
- 14:02really well. Um and then codeex.
- 14:06Actually the plan before chatbt was that
- 14:08we were going to go all in on code.
- 14:10>> Um we knew these models could write
- 14:11code. Uh we knew that they could be
- 14:14really and we knew that that would be
- 14:16like a valuable area. But then we had
- 14:17this incredibly exciting thing happen.
- 14:19Um but our kind of internal belief at
- 14:21the time was that coding was how these
- 14:25models would control things on computers
- 14:27and robots were how these models would
- 14:30control things in the physical world.
- 14:31And if you made a smart enough model
- 14:33that had sort of the actuators of
- 14:34writing code and robot and driving a
- 14:37robot, you could then kind of actually
- 14:40get this intelligence to do stuff for
- 14:41you in the world.
- 14:42>> So, uh, then it took us a while to get
- 14:44there. And then I think Codeex got
- 14:47really good by early this year, but with
- 14:505.5 is when we saw this real inflection
- 14:52point where people are now like doing
- 14:55just incredible things with it. And um
- 14:58you know that we earlier in the class
- 15:00we've talked about how the capabilities
- 15:02pipeline uh is starting to look is
- 15:05starting to become somewhat more legibly
- 15:07standard across different research
- 15:09groups. You got you know pre-training,
- 15:11mid-training, post- training. Then you
- 15:12got the RL and supervised feedback loop.
- 15:14Is do you think that's roughly like the
- 15:16shape of the pipeline that allowed
- 15:18codeex to you know go through a
- 15:20capability jump and that will basically
- 15:21stay stable now and consistent or are we
- 15:23going to go through a major rewrite of
- 15:24that pipeline? I think that is
- 15:26definitely the current pipeline. I
- 15:28expect we will go through a major
- 15:29rewrite. I don't know when it'll happen
- 15:30or exactly how. Um, but
- 15:34it is a little odd to me that it's so
- 15:37happens as a pipeline and doesn't quite
- 15:40feel like the optimal solution. Um, what
- 15:44would be an optimal solution in your
- 15:46head?
- 15:46>> I think that's a research problem for
- 15:48the AIS to figure out. Um, I think we're
- 15:50at a point where and we've set this goal
- 15:52that by September of this year, we will
- 15:54use 500,000 A100 equivalent GPUs, like a
- 15:57lot of computing power, as an AI
- 15:59research intern, and by March of 2028
- 16:01that we will have a full endto-end very
- 16:04talented researcher like figuring out
- 16:06complete new architectures. Um, so I
- 16:09think we are going to get like with the
- 16:11current pipeline, the current
- 16:12architectures, I think we're going to
- 16:14get over the line of when AIs can do
- 16:16incredible, incredible work. Um, you
- 16:19know, one of the things that you you
- 16:22just described there
- 16:25is is you you we we've been talking a
- 16:27lot in the class about systems,
- 16:29frameworks and analogies to make
- 16:30concepts from one domain legible to
- 16:32other people who may not have all the
- 16:33context in another. And that sometimes
- 16:36because of the translation problem, you
- 16:39know, reasoning by analogy is not
- 16:40helpful because then errors compound.
- 16:42>> Yeah. Um right there you said you know
- 16:44our goal is to try to use it as an AI
- 16:46intern which obviously is a very useful
- 16:48metaphor within the context of you know
- 16:50Silicon Valley a class that understands
- 16:52how these pipelines work and so on and
- 16:54then as as you scale actually that
- 16:56metaphor globally people who might not
- 16:57have all that context go start
- 16:59analogizing these models in ways that
- 17:00they shouldn't be like how should we
- 17:02think about the limits of of that of of
- 17:06what are the limits to scale of um what
- 17:08are the product analogies the research
- 17:11analogies you find most useful
- 17:12within the valley and which one of the
- 17:14what have you found about the limits of
- 17:17those analogies scaling and now how do
- 17:19you navigate between those two problems?
- 17:23I I've been very interested in studying
- 17:25how like
- 17:28I I think what is happening is we are we
- 17:30are in the process of creating a new
- 17:31utility. This doesn't happen very often.
- 17:33you know, electricity is utility,
- 17:34internet's a utility, there water, I
- 17:37guess there's not a lot of these. Uh,
- 17:38and so there are not a lot of examples
- 17:40that we can study for good metaphors or
- 17:43learnings about how to explain this to
- 17:46the world. Um, but I was recently
- 17:49looking at what happened when
- 17:50electricity became a utility. And it's a
- 17:54good analogy for many reasons. It's
- 17:55imperfect, of course, too. But the
- 17:58electricity companies, at least the ones
- 17:59I could find information about, they
- 18:01didn't talk about selling electricity
- 18:02cuz no one knew what that was or why
- 18:04they wanted. It sounds like very scary.
- 18:05It's this thing that's like going to
- 18:06come into your house and it can kill you
- 18:08in this like gruesome way and you, you
- 18:10know, it feels sort of like very
- 18:12different than the world before. Uh, and
- 18:15maybe they tried to sell electricity or
- 18:17market electricity at first. I don't
- 18:19know. But in any case, that didn't work.
- 18:20And then what they started marketing
- 18:23selling to people was light at night.
- 18:25You know, we are going to what you are
- 18:27getting from us is not electricity. It's
- 18:29light at night. By the way, you can use
- 18:31the same thing that lets you get light
- 18:33for all these other things. But people
- 18:34are like, well, why would I want that?
- 18:35And they're like, well, you know, it'll
- 18:37wash your clothes for you someday. And
- 18:38no, no, it won't. I can't. That's too
- 18:40far of a jump for me,
- 18:41>> right?
- 18:41>> Um, so I don't know what our analogy
- 18:46for this should be. Um, but I suspect
- 18:49that even if even if we're totally right
- 18:52and intelligence is going to become this
- 18:54new utility that every company, every
- 18:57customer, uh, every government just
- 18:59needs access to and is going to use in
- 19:01all sorts of incredible ways and you
- 19:03will have like a OpenAI token
- 19:05subscription that you will plug into
- 19:06everything and use to access everything
- 19:08and you have running for you all the
- 19:09time and doing this amazing stuff. I
- 19:12kind of don't think at least right now
- 19:14the right way for us to analogize that
- 19:15is we're selling intelligence because
- 19:17people are just like somehow not
- 19:19resonating. I don't know what our
- 19:21equivalent of we're selling you light at
- 19:24night is going to be. But I think if
- 19:26we're going to become a new utility, we
- 19:28need to find a way to explain to the
- 19:30world what it means to have this like
- 19:32intelligence pike that you can just do
- 19:34whatever you'd like with
- 19:35>> it. So um one
- 19:39question that has emerged an emerging
- 19:41property of this class of of having a
- 19:43diversity of different speakers is that
- 19:45the utility analogy has come up several
- 19:47times but in reference to different
- 19:48things. So Jensen likened ut like
- 19:51compute to a utility um and why there
- 19:54should be access and so on and talked
- 19:56about how Stanford should pull budget
- 19:58and so on and and and procure that as a
- 20:00utility for everybody on campus whereas
- 20:02you just likened the intelligence part
- 20:04to a util are both of these things true
- 20:06is one of them true one is one more
- 20:07likely to be true how should people
- 20:08reason about compute as a utility versus
- 20:10tokens as a utility and and by comput I
- 20:12mean here chips versus tokens does that
- 20:15make sense
- 20:16>> I think as a consumer as like a business
- 20:19or an individual um you will think in
- 20:22something closer to tokens or probably
- 20:24even one level up from tokens. I don't
- 20:27think you'll care very much about you
- 20:29know where the hardware is, what
- 20:31particular chip it is, what's powering
- 20:33it. I think that stuff will be
- 20:34abstracted out and what you will care
- 20:36about is when you're interacting with
- 20:39the system. Um
- 20:41can you use it a lot? Is it cheap? Is it
- 20:43doing a good job? Um so right now it's
- 20:46like tokens. it may get as we move into
- 20:49a world where we all just have like this
- 20:51constant agent running for us being
- 20:53useful to us all of the time. Um you may
- 20:55think about it as even one level up. But
- 20:57yeah, my my guess is is you when you
- 21:00like pay for your cell phone bill,
- 21:02you're like, "All right, I'm buying
- 21:04access to airtime and some number of
- 21:07gigabytes and you know, it's going to do
- 21:08all these things and I'll use all these
- 21:09apps and whatever else." But like what
- 21:11you think about paying for the kind of
- 21:13internet utility in this case is just
- 21:15like access to the whole system and the
- 21:19particular hardware at the base station
- 21:21and how it connects to the internet. You
- 21:23don't think about that as much. Um I
- 21:25know I could nerd out about utility
- 21:27infrastructure for a long time, but I
- 21:28want to make sure we switch a little bit
- 21:29to being relevant for the students.
- 21:31Usually we have uh questions, but we're
- 21:33not hearing those today unless you're
- 21:35comfortable. Oh, okay. Great. How about
- 21:37that? Improv. Okay. Uh so one final
- 21:41question to start getting the creative
- 21:42juices flowing is um the final project
- 21:44for this class
- 21:46183 is the oneperson frontier lab. So
- 21:49everybody here is working on projects
- 21:51where they're simulating being an
- 21:52individual uh as a lab with access to
- 21:55all the right tools. They've got
- 21:56hundreds of thousands of dollars of
- 21:57credits from Cloudflare. I think we've
- 21:59got some open AI tokens maybe. But
- 22:01there's a bunch of comput disposal. Um,
- 22:04what would you, if you were in the
- 22:06class, what would you be working on for
- 22:07your oneperson frontier lab project?
- 22:10First of all, I think that's an awesome
- 22:11project. Um,
- 22:14I think this is top of mind because uh
- 22:18you we we were just like talking about
- 22:20utility frame frameworks. I think
- 22:23there's a lot of very smart people
- 22:24working on uh great training ideas and
- 22:28we're going to have incredible models.
- 22:30No matter what you all do, we're going
- 22:32to have incredible models. I promise
- 22:33here uh like pretty quickly but
- 22:37I I think we have not invested enough in
- 22:39being able to deliver at scale huge
- 22:42amounts of cheap intelligence. So maybe
- 22:44I would go work on like the inference
- 22:45part of the stack [clears throat]
- 22:46>> and how are we going to get this
- 22:48incredible intelligence to be cheap and
- 22:50abundant? Uh I think that's
- 22:52underinvested and and and I think all of
- 22:54the frontier labs are going to have to
- 22:56become inference companies to a
- 22:58significant degree.
- 22:59Um, okay.
- 23:02It might be too late to pivot your
- 23:03projects, but better late than never.
- 23:05>> Work on whatever you want to work on.
- 23:06[laughter]
- 23:08>> Uh, okay. Let's start doing questions
- 23:09and I'm going to moderate and try to be
- 23:11not, you know, please try to be
- 23:13productive and not spicy, etc. Remember,
- 23:15it's a CS class, but up to you Sam is
- 23:18fine.
- 23:19>> Oh, we've got questions. Oh, perfect.
- 23:21All right. First one, the questions
- 23:23about your views on Yan Lun's view that
- 23:25LM are a dead end. Um,
- 23:29first of all, in terms of achieving
- 23:31human level intelligence, these models
- 23:33have already far surpassed human
- 23:35intelligence in some ways and then
- 23:37they're wildly worse than others. Like
- 23:39for example, they seem much worse than
- 23:41people are at very long horizon
- 23:46kind of high judgment signal and tasks.
- 23:50Um on the other hand yesterday we had
- 23:54one of our models uh discover or
- 23:57disprove a conjecture one of the airish
- 23:59problems that had smart people had
- 24:01worked on for a long time and a lot of
- 24:03people a lot of smart scientists I don't
- 24:05know if lun was one of them or not had
- 24:07even quite recently said something like
- 24:09that was not going to happen. Uh and
- 24:11then like the model just did it and you
- 24:13know now you have all these
- 24:14mathematicians saying like is math over?
- 24:16What does this mean for our field? So
- 24:18clearly LLMs are capable of figuring out
- 24:22new knowledge and clearly they are
- 24:24capable of doing some things that some
- 24:26intelligence tasks that humans just
- 24:28can't do. Um they're going to scale much
- 24:31further. So how much better and what
- 24:32distribution of the tasks they can do
- 24:34better than humans? We'll find out. But
- 24:36I suspect it's a lot. And the you know
- 24:38in terms of this like lack of a belief
- 24:41in the exponential we were talking about
- 24:42earlier. Um I think the field was
- 24:45honestly held back by a generation of
- 24:48scientists who just were way too certain
- 24:50on what wouldn't what what scaling was
- 24:52not going to produce and then some
- 24:54people just looked at the graphs and
- 24:55said well it looks like it's continuing
- 24:57beautifully let's keep going. Um
- 25:00I think world models are clearly
- 25:03important and to
- 25:06we'll need that for things like
- 25:07robotics. Uh but betting
- 25:10against LLM scaling at this point
- 25:15uh feels quite misguided to me.
- 25:19Does it get annoying to be the I told
- 25:20you so guy?
- 25:21>> No. I mean
- 25:24there are these like Twitter trolls that
- 25:26you know for years have just been like
- 25:28it's not going to work. It's not going
- 25:29to work. This is so dumb. Like you know
- 25:30this is a fraud. This company is going
- 25:32to fail. This research approach is going
- 25:33to fail. And I used to get more bothered
- 25:35by them. But I don't even like feel the
- 25:37I told you so at this point. It's like
- 25:38you were just like never.
- 25:40>> You're still going on about it. Like the
- 25:42data is
- 25:45>> quite strong on our side.
- 25:48>> It I don't think it'd be that fun to say
- 25:49I told you so. And also the fact that
- 25:51you're like still saying we're wrong
- 25:52doesn't really bother me.
- 25:53>> I think there's that kind of move on.
- 25:55>> There's that saying that like insanity
- 25:56is doing the same thing over and over
- 25:58again when presented with data that is
- 26:00not working and they keep repeating
- 26:01that. And in a sense it's it's it's a
- 26:03form of insanity. I think
- 26:05>> I I think there's something that happens
- 26:07which is if you make your identity about
- 26:09a particular
- 26:11thing is going to work or not work
- 26:14and you associate yourself with that
- 26:16belief and then the science or the
- 26:18empirical results disprove you and
- 26:21you're like too hung up on your
- 26:23identity. You can't let it go. You can't
- 26:24see the truth.
- 26:25>> Yeah.
- 26:26>> And I think this is like a important
- 26:28reminder in both directions.
- 26:29>> Yeah.
- 26:31>> How do you see education?
- 26:33Um, it clearly has to super adapt and I
- 26:37am worried. I I thought by now it would
- 26:39have. Um, the the I think if we continue
- 26:43to teach and evaluate students
- 26:46as if we were in a pre- agi world, um,
- 26:50it's not going to work and it is going
- 26:51to lead to like atrophy of learning how
- 26:53to think or whatever. And I thought that
- 26:55was going to be obvious enough that I
- 26:57wasn't that worried. You know, when
- 26:58ChhatgBT launched, I was like, "Yeah,
- 26:59we're going to have one year of like
- 27:01students like cheating and not learning
- 27:03that much. And then the educational
- 27:05system is just going to redesign itself.
- 27:06And there's and we're going to teach
- 27:08people so much better." You know, people
- 27:09are going to really
- 27:11get projects where they have to they
- 27:14have to use AI to be able to do it, but
- 27:15they still have to like stretch their
- 27:17brain more and think more and figure out
- 27:18new things to do. And honestly, I
- 27:22struggle to point to any significant
- 27:24systemic change that I've seen in the
- 27:26education system at large in the three
- 27:29and a half years since Chad launched.
- 27:30And I that was a prediction error for
- 27:32me. I thought I thought that would have
- 27:33happened. So I have no doubt that we can
- 27:38uh like we have done with every other
- 27:39technological leap before redesign how
- 27:42education works so that you still have
- 27:44to learn how to think. And there will be
- 27:46some things like I I I am a person who
- 27:49thinks by writing and I write a lot of
- 27:52stuff that I never show anyone else but
- 27:54it's still important to me to figure
- 27:55something out and so I'm grateful that I
- 27:57I learned to write. People say the same
- 27:58thing about programming. Um so there
- 28:01will be some things that we teach people
- 28:02to do that machines can do better just
- 28:04because it's helpful to teach them the
- 28:08meta skill of thinking and learning and
- 28:09that makes sense. But there are a lot of
- 28:11other things where we should just
- 28:12totally teach totally change how we
- 28:15teach or how we learn or how we
- 28:16evaluate. And
- 28:19if we don't do that, I think there will
- 28:20be like significant atrophy in people's
- 28:23critical thinking skills.
- 28:24>> Uh question is what was your favorite
- 28:26class and what what do you wish you had
- 28:27taken while when you were at Stanford?
- 28:29>> Does Stanford still do intros?
- 28:32I did like all the I did like three
- 28:33intros a quarter my freshman year like
- 28:36and I loved all of them. Uh they were
- 28:38all super different. Uh I but looking
- 28:42back the fact that I
- 28:45was able to get such a broad exposure to
- 28:48stuff and h have like a a very shallow
- 28:50understanding of lots of different
- 28:51fields was an incredible thing. If it
- 28:54had not been for that I just would have
- 28:55taken like CS and physics classes which
- 28:57still would have been great. But um I I
- 29:01think more about the stuff the classes I
- 29:04took that were like totally random and
- 29:06unrelated to what I do now but in some
- 29:08important way
- 29:10gave me a perspective then I do I think
- 29:12I would have like learned a program no
- 29:15matter what. Uh so I and I didn't think
- 29:19that at the time I was like kind of like
- 29:20yeah you know this is this stuff is all
- 29:23cool but it's mostly going to be about
- 29:24like learning CS. Um, I only did two
- 29:27years of school. Uh, so there was a lot
- 29:29of stuff I wanted to take that I didn't
- 29:30get to. Um, but that's kind of the
- 29:32surprising thing.
- 29:34My question is, what is your spiciest
- 29:37take of all,
- 29:39right?
- 29:43I I think with more time to think uh I
- 29:46could come up with a much
- 29:48spicier one, but um
- 29:52I think AI is just going to keep going.
- 29:55And
- 29:57I think this is considered
- 30:00I don't I don't think this is like
- 30:01widely believed yet. And I think if this
- 30:03were widely believed, there would be
- 30:05like significantly more reverberations
- 30:07that are happening through society right
- 30:09now. And maybe I don't have a spicer
- 30:11tag. Actually, maybe this is the high
- 30:12order bit that if AI progress continues
- 30:15on the exponential that it's on for
- 30:18another
- 30:20it's been three and a half years since
- 30:21tragic if even if it were another three
- 30:22and a half years on that same trajectory
- 30:24the world
- 30:26the potential the way that society
- 30:29what's society is capable of are just
- 30:32completely different. Well, let me try
- 30:33to prompt more thinking tokens on that
- 30:35one. um you you have if we treated you
- 30:39as a model like as a frontier model and
- 30:42you have some inherent capabilities and
- 30:44we're going to we're going to try to
- 30:44elicit capabilities that people don't
- 30:47know about for the next few minutes. Um
- 30:49one of them is that you've been
- 30:50postrained now on you you've been
- 30:52continuously RL on OpenAI as well as the
- 30:55external feedback loop of the world on
- 30:56what doesn't work and what works and
- 30:58doesn't work. So now if we're going to
- 31:00treat you as a prediction engine for a
- 31:01sec, the prompt is what are the three
- 31:04most likely forks of the universe you
- 31:06see over the next 10 years and what is
- 31:09your what is your probability assessment
- 31:10on each of those? Does that make sense?
- 31:13One that feels very important is uh like
- 31:18how much is this technology going to be
- 31:20very widely democratized versus how much
- 31:22is it going to sit in a few companies. I
- 31:26I think a world there are all of these
- 31:28reasons why you could imagine the
- 31:29default is that this gets concentrated
- 31:31to a few companies and they become like
- 31:35you know a significant fraction of the
- 31:36wealth on earth that would obviously be
- 31:38terrible and we work super hard to push
- 31:40against that but I think that's going to
- 31:42require like the will of the world to to
- 31:44really avoid um because there is a sort
- 31:47of a tractor state there and I think
- 31:48part of the reason that we need to push
- 31:50to this kind of utility model of the
- 31:52world is that a it's quite unstable and
- 31:55quite and will feel quite unfair if a
- 31:57few companies have all this. But B, I
- 31:59think there's a real alignment failure
- 32:01and a very fragile world. Uh, and the
- 32:03best way to get to a world we want that
- 32:06represents like everybody winning and
- 32:07everybody's values being represented,
- 32:09everybody having agency is to just put
- 32:11push this technology out into the world.
- 32:14Um, but there will be a very strong
- 32:15argument against that around sort of
- 32:17safety and stability. And I think that
- 32:20will be a big fork and it's very
- 32:21important and I encourage all of you in
- 32:23your careers to push hard that this is a
- 32:25technology
- 32:27it can bring us an incredible sci-fi
- 32:29future life can be unbelievably much
- 32:31better we are going to incur some risk
- 32:33to get there but the risk of keeping
- 32:35this concentrated in a handful of
- 32:37companies even though we would be one of
- 32:38these companies is not something we
- 32:40should tolerate so I think that would be
- 32:41a big fork uh in terms of probability I
- 32:44think it's
- 32:47the world should have such an interest
- 32:49in it happening this way that I think
- 32:51it's like 80% we end up on the
- 32:53democratic path but there will be a very
- 32:56strong safety message and you know there
- 32:58will be a lot of power seeking people
- 33:00who who want to concentrate the power
- 33:02and
- 33:04one of the problems with forecasting
- 33:07this or that you have and we all have as
- 33:10humans is once you make that forecast
- 33:12then you have agency to affect the
- 33:14forecasts right and the forecast for
- 33:15>> well I mean we're clear on what we're
- 33:17going to use our agency before like this
- 33:18is what we believe in. We think that uh
- 33:21you know we're going to do everything we
- 33:22can to push it in this direction. We
- 33:24just we see the forces in the other
- 33:26direction. Maybe a related fork uh
- 33:30there's a lot of talk about like future
- 33:31economic models and are we going to do
- 33:33universal basic income? Are we going to
- 33:35have everybody gets to like own a slice
- 33:37of every company? Like are we going to
- 33:39is it capitalism with no change? Is it
- 33:42like full-on communism? There's like a
- 33:44lot of talk about this. One thing that I
- 33:46think is not talked about much is how
- 33:49specifically how we distribute compute.
- 33:52>> So maybe a lot of the economy can work
- 33:56in a way that it's going to work. And
- 33:57I've actually I've become much less of a
- 33:58even short-term jobs doomer. I've always
- 34:00been optimistic we find new things to do
- 34:02but this may not be as disrupted as I
- 34:05originally thought in the short term. Um
- 34:07but we are seeing comput shortages now.
- 34:11I can imagine them getting much worse
- 34:13and I can imagine compute being like the
- 34:15most important utility that people need.
- 34:18Uh so if the price of compute from a
- 34:20supply and demand perspective gets way
- 34:21out of whack then I think there will be
- 34:23a very interesting fork about what it
- 34:26means to equitably distribute compute.
- 34:28So you said two very interesting things
- 34:30there which you said on the economic
- 34:32side we might have need universal basic
- 34:34income. Everybody owns a piece of
- 34:36shares. You know, one of the speakers in
- 34:38this class is um Nikolai Tangen who runs
- 34:42the Norwegian sovereign wealth fund.
- 34:43He's awesome.
- 34:44>> He's awesome. You know, this Norwegian
- 34:46sovereign wealth fund owns 1.5% of all
- 34:48publicly traded companies on the planet.
- 34:50They also have effectively universal
- 34:51basic income. You could argue there's
- 34:53flavors of this already today because
- 34:55you know the largest employer now in the
- 34:56United States is the government and you
- 34:58could argue like large sections of that
- 35:00or a way for the government to
- 35:01redistribute income from taxpayers. So
- 35:03are these solutions that actually need
- 35:05to be novel or just reimplemented for
- 35:08this era? How do you think about the
- 35:10novelty of those solutions where we
- 35:11often you know in Silicon Valley make
- 35:13have this tendency to be like reinvent
- 35:16you know old things from first
- 35:17principles and so should we just look to
- 35:19existing systems and tweak them. Um
- 35:20yeah, I don't think that these things
- 35:22require
- 35:23deeply new ideas. Although I will say um
- 35:28I am much more excited about people
- 35:31having some sort of ownership stake than
- 35:33a fixed monthly cash dividend,
- 35:35>> right?
- 35:36>> Um and I I funded like a big universal
- 35:40basic income study a while ago. I've
- 35:43also watched what happens when people
- 35:45like invest in startups and I know which
- 35:47model I think like hits human psychology
- 35:49better. So what I would love to see is
- 35:52as leverage in the world shifts from
- 35:56labor to capital which I think is going
- 35:58to keep happening
- 35:59that we find a way to have something
- 36:03like a citizens wealth fund in the
- 36:04country or in the world eventually where
- 36:07you like you basically own a slice of
- 36:09capitalism right own slice of these
- 36:10companies. And then on the second fork
- 36:12there on compute bottlenecks, you said
- 36:15uh at some point when compute prices get
- 36:17out of whack between January and this
- 36:19year, my my current understanding is
- 36:21based on data we've seen that H100
- 36:23prices and Blackwell prices, the spreads
- 36:26between long-term reservations and spot
- 36:28is like 5x.
- 36:30>> I don't know if it's that high anymore.
- 36:32I think it got a little better, but
- 36:33yeah,
- 36:33>> tell me.
- 36:34>> Or if you can even find H100s cuz
- 36:36they're pretty much all gone for this
- 36:37year. Does that sound right?
- 36:39>> No argument. There's a gigantic computer
- 36:42shortage. Yeah. So that that's a good
- 36:44example of an of a systems problem right
- 36:46now that's live. At least to some folks
- 36:49it feels like co you know for the
- 36:51compute era like all the toilet paper's
- 36:53gone.
- 36:53>> Yeah.
- 36:55>> Why are people not freaking out about
- 36:56this?
- 36:57>> Well, I think people assume we will make
- 37:00big inference gains on the hardware we
- 37:01have. Uh I also think there is a tsunami
- 37:03of hardware coming. But maybe the demand
- 37:06tsunami is even bigger and I think
- 37:08people should be freaking out somewhat
- 37:09>> and and would you say it's fair like how
- 37:12long are we going to exist in a comput
- 37:13shortage
- 37:14at least you know based on current data
- 37:17you have
- 37:19>> I think like other you you can't talk
- 37:22really about like worldwide demand for
- 37:24electricity without talking about the
- 37:26price like it's there's an extremely
- 37:29different demand about how much energy
- 37:31people use in the world if the price
- 37:32comes down by a factor of 10 or goes up
- 37:34by a factor factor of 10 and I think AI
- 37:38is like that too.
- 37:40uh the
- 37:42if we can make models
- 37:47sufficiently smart and a sufficiently
- 37:50low cost, I think demand is like kind of
- 37:52uncapped. And so in some sense, as long
- 37:54as we can continue to make progress on
- 37:56this, there will be a shortage forever
- 37:58and things will be bid among above what
- 38:01the price we think we think the price
- 38:03should be. Even though people are
- 38:04getting better, smarter, more whatever
- 38:07intelligence just because you can use
- 38:09like
- 38:11if we make really great personal agents
- 38:13and you can have 10 of them running and
- 38:14working for you all the time or 100 and
- 38:16you know you want the hundred. I think
- 38:19it's a lot of inference.
- 38:21>> Awesome. With that, I'm going to give
- 38:22you the swag for the class which is
- 38:28[applause]
- 38:28>> Thank you for coming. Thank you.
About this transcript
This page contains the full transcript of Sam Altman - You no longer need to write prompts. by prabhakar varma, generated from the public captions YouTube serves with the video. The transcript has 7,577 words across 1,085 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.