I thought AI was gonna get cheaper? — Transcript

Full transcript

1. 0:00Intelligence too cheap to measure is
2. 0:02AI's next frontier. According to Sam
3. 0:05Altman back in July of 2025, I'm pretty
4. 0:09sure that in all of 2025, my personal
5. 0:11use of AI would probably amount to like
6. 0:13$800 to $1,000. Versus this year, 2026,
7. 0:17any given day, I can probably pull
8. 0:19$1,000 in tokens. It seems like
9. 0:22intelligence is getting way more
10. 0:23expensive, especially now with models
11. 0:25like Fable and Astra that are literally
12. 0:27more expensive per token and are doing
13. 0:29more in it generating more tokens,
14. 0:31massively increasing the cost that
15. 0:33real-world businesses are seeing. It
16. 0:35seems like everyone's AI bill is just
17. 0:36going up and up and up with no sign of
18. 0:39stopping. So, what happened here? Why
19. 0:41aren't we achieving this new frontier as
20. 0:44Sam Altman promised us? Why isn't
21. 0:46intelligence getting cheaper? I feel
22. 0:48like I was spending a few pennies per
23. 0:49prompt last year, and now I'm spending
24. 0:51tens, if not hundreds, of dollars. What
25. 0:53went so wrong? Why are these companies
26. 0:55charging so much more? I thought it was
27. 0:57supposed to get cheaper.
28. 0:59This video is inspired by Primagen,
29. 1:01because he made a tweet referencing a
30. 1:02real promise that Sam Altman had made,
31. 1:05which is that he said AI would get 10
32. 1:07times cheaper year over year. And Prime
33. 1:09said he's a bit confused because he
34. 1:11thought that we would get this 10 times
35. 1:14cheaper AI, but Astra's out and it's way
36. 1:16more expensive. So, what went wrong? I
37. 1:20got a lot of thoughts on this one.
38. 1:22The short version of what happened here
39. 1:23is it did actually get cheaper. We're
40. 1:25just not measuring it right. Long
41. 1:27version, aka the video, will be the next
42. 1:30thing after the sponsor break. Okay,
43. 1:31look. I know this is an ad, but I just
44. 1:33want to nerd out for a minute, because
45. 1:35I've been building a lot more with
46. 1:36Railway recently and I've had so much
47. 1:38fun doing it. Let me get some bias out
48. 1:39of the way first. I've loved Railway for
49. 1:41years now. I was almost their fourth
50. 1:43hire, but I instead chose to make my own
51. 1:44startup and then my YouTube channel soon
52. 1:46after. I know this team well. I love
53. 1:48them and everything they're building is
54. 1:49awesome. Railway had already built the
55. 1:51easiest cloud for developers to build
56. 1:53and ship real full-stack applications
57. 1:55on. it. guys can everything from the
58. 1:56database to the CDN and everything else
59. 1:58in between. You could build your full
60. 2:00app from scratch on Railway. And it was
61. 2:02really nice when you needed persistent
62. 2:03servers. Where things got genuinely
63. 2:05magical is when they introduced their
64. 2:06skills, CLIs, and MCPs to our agents.
65. 2:09And now I can have agents just deploy
66. 2:11and manage full stack apps without ever
67. 2:13having to think about it. Even calling
68. 2:14these apps almost feels misleading
69. 2:16though, cuz one of those apps is a cloud
70. 2:17I built on top of Railway. Yes, the
71. 2:20cloud I've been working on for a while
72. 2:21now, Lakebed, is entirely built on top
73. 2:23of Railway and I'm so happy I did it
74. 2:25this way. I have my full production
75. 2:27environment as well as a staging
76. 2:28environment that my agents provisioned
77. 2:29for me, as well as an environment for
78. 2:31one specific PR that again, my agent
79. 2:33provisioned for me to test things out
80. 2:34end to end. When I have any questions
81. 2:36about how the infra is working or any
82. 2:38ideas for how it could work better, I
83. 2:40tell Claude or Codex about it. It
84. 2:42connects to the MCP, it spins up a new
85. 2:43environment, it figures it out, it
86. 2:45verifies the changes, it ships to prod
87. 2:47by merging on main. It will then check
88. 2:49it on the environment through the MCP
89. 2:50server and just know what's going on.
90. 2:52Ready for the spiciest part of this ad?
91. 2:54I haven't looked at this dashboard in
92. 2:56weeks cuz I don't have to, because my
93. 2:58agents can manage all of it for me.
94. 2:59Whether I'm building a one-off
95. 3:00experiment to host my own analytics like
96. 3:02I did with Slot Paws, or I'm trying to
97. 3:04put together an HTML host like I did
98. 3:05with Post Plan, or I'm building my own
99. 3:07cloud with Lakebed, my default has been
100. 3:09Railway for months now and I'm so
101. 3:11thankful that changed. If you're worried
102. 3:12about costs, don't be. I'm hosting a ton
103. 3:14of stuff on Railway including our entire
104. 3:16internal video pipeline where this ad's
105. 3:18actually going to be uploaded, and my
106. 3:20estimated costs for the month are under
107. 3:21$50 for this insane amount of usage I'm
108. 3:24getting. Ship on a platform that you and
109. 3:26your agents will love at
110. 3:27sodalite.link/railway.
111. 3:29Sorry for the break, good to have you
112. 3:30back. In order to explain why AI is
113. 3:32actually getting cheaper, we're going to
114. 3:33have to look at some numbers and talk
115. 3:35more about how models work. The first
116. 3:37thing we need to talk about is token
117. 3:39cost. Flash bang warning for artificial
118. 3:41analysis.
119. 3:42When you look at model prices over time,
120. 3:44the numbers are pretty scary. GPT-5
121. 3:47medium was $10 per million tokens out
122. 3:50and a
123. 3:51per million tokens in. Versus a model
124. 3:53like GBT-6 Astra, which is $50 per mil
125. 3:56out and 10 per mil in. That is a five to
126. 3:5910x increase on input and output tokens
127. 4:02cost. That's crazy. That's terrifying.
128. 4:04That's the exact opposite of what we
129. 4:06were promised. We were promised 10x
130. 4:08decreases in cost, but there's a 10x
131. 4:10increase. Well, there are layers to this
132. 4:12because token cost is one of the worst
133. 4:14ways to measure cost. The three things
134. 4:16we need to be considered of are first,
135. 4:18yeah, sure, token cost. I'll talk a
136. 4:19little bit about it. But the other two
137. 4:21are where most of the important parts
138. 4:22come from. The second one is how many
139. 4:25tokens it takes to complete a given
140. 4:26piece of work. And the third is the
141. 4:29types of work the models can complete.
142. 4:30Specifically, how much intelligence per
143. 4:33dollar, so to speak. So, we have token
144. 4:35cost, tokens per dollar. We have tokens
145. 4:38per task cost, how many tokens it take
146. 4:40to solve a task. And then we have cost
147. 4:42per level of intelligence. How much
148. 4:44spend does it take? How much money do we
149. 4:46have to spend to get a solution to a
150. 4:48problem at a certain level of
151. 4:50intelligence? So, yes, token costs did
152. 4:52go up for the best of the best models,
153. 4:54but let's be real here, there are also
154. 4:57cheap options that are incredible. For
155. 4:58example, a model like JLM 5.3 Flash,
156. 5:01which I absolutely adore, it's one of my
157. 5:02favorite model releases ever. This model
158. 5:04is 15 cents per million input tokens and
159. 5:0750 cents per million out. It's also on
160. 5:09sale in a lot of places for 50% off. So,
161. 5:12it's actually 7 cents per mil in and 25
162. 5:15per mil out. That means that it's about
163. 5:17a 30th the cost of GPT-5. For a 20th to
164. 5:22a 30th the price per token, JLM 5.3
165. 5:25Flash will run circles around GPT-5 for
166. 5:28a lot of real-world agentic stuff.
167. 5:30So, again, if we look at the top-tier
168. 5:33models, yes, cost per token's gone up.
169. 5:36But if you look at other places, you'll
170. 5:38see that we've also gotten really
171. 5:40capable models that are way cheaper than
172. 5:41before as well. So, yes, the top end has
173. 5:44gotten crazier, but so has the bottom
174. 5:46end. Both sides have seen meaningful
175. 5:49improvements here. But as I mentioned
176. 5:51before, cost per token is only a tiny
177. 5:54piece of the formula here. I'll use a
178. 5:56really fun example. Let's look at Gemini
179. 5:583 8 Flash.
180. 6:00On this chart, it looks like a pretty
181. 6:01good deal. It's $3.75 per million out.
182. 6:05That makes it less than half the price
183. 6:06of GPT-5 and less than a tenth the price
184. 6:09of Astra and Fable. But then we hop up a
185. 6:11little bit to cost per intelligence
186. 6:13index task, and suddenly that difference
187. 6:15gets a lot less different because 3 8
188. 6:18Flash uses so many tokens for the work
189. 6:21that it's doing. So even though it is
190. 6:23comically cheaper per token, the token
191. 6:25waste that 3 8 Flash has results in this
192. 6:27gap going from over 10x to under 3x. And
193. 6:313.5 Flash was even worse here cuz it was
194. 6:33so token hungry, getting to over half
195. 6:35the cost of 5 6 Soul, even though 5 6
196. 6:38Soul is a way more expensive model. So
197. 6:40once again, you need to factor all of
198. 6:42these things in. Hilariously enough,
199. 6:44Sonnet 5 was more expensive than Astra
200. 6:46because Sonnet just burns tokens.
201. 6:50So those are the first two dimensions to
202. 6:51consider.
203. 6:52Cost per token and the amount of tokens
204. 6:54used to solve a given task. And to be
205. 6:56clear, different tasks have different
206. 6:59ways that they work here. For example,
207. 7:01if the task is to ingest 800,000 tokens
208. 7:05and categorize them in some way,
209. 7:07the categorization and the output tokens
210. 7:09aren't going to matter as much. The
211. 7:10input tokens will, so you're always
212. 7:12paying that cost basically no matter
213. 7:14what. At that point, optimizing for the
214. 7:16cheapest model for the input token read
215. 7:18is probably the right call. If you're
216. 7:19doing agentic dev work, things like
217. 7:21caching, reliability, and token
218. 7:23efficiency suddenly matter a hell of a
219. 7:25lot more. We're also not looking at this
220. 7:28the right way. We're just looking at
221. 7:30cost right here. What we want to see is
222. 7:32cost versus intelligence. This chart is
223. 7:35much more interesting, where you can see
224. 7:37how expensive a run was and how well it
225. 7:40performed in the benchmarks. They have
226. 7:42this nice line now for the Pareto
227. 7:44frontier, which is what is the best
228. 7:45model at a given cost level. And you'll
229. 7:48see GBD-6 Astra has a significant number
230. 7:51of the spots here. And Fable ends up
231. 7:53being way more expensive all the way
232. 7:54over there, but you'll see at a given
233. 7:56level of intelligence, cost can go down
234. 7:58massively. Like if you're happy with a
235. 8:0045 to a 46 as the score here, your cost
236. 8:05will be 82 cents per task. Where if you
237. 8:07want to bump up into the 50s, suddenly
238. 8:09it's $3.26.
239. 8:11So for just a few more points, you're
240. 8:12spending three times more money. I still
241. 8:14don't think this is the best way to
242. 8:16think about it though. I want to flip
243. 8:17it. Instead of thinking about this as
244. 8:20how much intelligence can you get per
245. 8:21dollar, let's take a level of
246. 8:23intelligence and see how much it costs
247. 8:25over time. If we took at the capability
248. 8:27of GPT-5 for example, how expensive
249. 8:30would it be to get that same level of
250. 8:32intelligence today with other cheaper
251. 8:34options? I actually did a good bit of
252. 8:36research here to find what models have
253. 8:38similar scores to GPT-5. And it was
254. 8:41harder than expected because we've had
255. 8:42to overhaul almost all the benchmarks
256. 8:44that we use over the last year because
257. 8:46we saturated everything. I eventually
258. 8:48stumbled on V4 Flash being roughly
259. 8:50equivalent to 5 Codex High because the
260. 8:52handful of benches they had overlapping
261. 8:54were actually similar in score. Funny
262. 8:56enough, the Deep Seek V4 Flash model is
263. 8:58actually outperforming it in a bunch of
264. 9:00places like Humanities Last Exam, AI
265. 9:02Amnitions, which is a benchmark
266. 9:03measuring hallucination rates, but it
267. 9:05got roughly the same overall
268. 9:07intelligence index score. Deep Seek V4
269. 9:09Flash is 21 times cheaper than GPT-5 for
270. 9:13input tokens and 55 times cheaper for
271. 9:16output tokens. I couldn't find good
272. 9:18numbers to measure the efficiency
273. 9:19changes here, but even if you assumed
274. 9:21that Deep Seek was 3x less efficient,
275. 9:24the difference still nets out to around
276. 9:26a 20x improvement. So if the goal was to
277. 9:30get 10 times cheaper intelligence at a
278. 9:32given level, we blew that out the
279. 9:34[ __ ] park. We're at 20 times cheaper.
280. 9:37Thankfully, this is actually being
281. 9:38tracked by our friends at Artificial
282. 9:39Analysis. They would pick a given level,
283. 9:42which was intelligence less than 10
284. 9:44points before. That used to cost 60
285. 9:47cents per task, and that got all the way
286. 9:49down to 2 cents per task over the course
287. 9:53of about 2 years. From November of 2022
288. 9:56to July of 2024, we saw that cost go
289. 9:59down massively. But then we start
290. 10:01ranking things between 10 and 20 points.
291. 10:04Initially, they were expensive all the
292. 10:05way up to like $15 per task with O1
293. 10:07preview, cuz that model was super
294. 10:09expensive. But you can see over time, it
295. 10:12didn't take very long to drop. It was
296. 10:14September of 2024 that it cost $15 for
297. 10:17this level, and by February of 2025,
298. 10:19just a few months later, Grok-3 mini was
299. 10:22getting similar numbers at a way, way
300. 10:26cheaper price. Pennies to what were
301. 10:28dollars before. 16 cents to get a score
302. 10:32that was actually higher than O1
303. 10:33preview, cuz O1 preview only got an 11.
304. 10:35And that was September of 2024, and by
305. 10:38February of 2025, Grok-3 mini had four
306. 10:40points on top of it at under a hundredth
307. 10:43the price. Sorry, roughly a hundredth
308. 10:45the price. Pretty crazy that the price
309. 10:47has dropped by almost 99% in just 4
310. 10:49months. And Chad just made a good
311. 10:51observation. Chad said, "The drops are
312. 10:54faster each time." That is where things
313. 10:56get really, really cool. The speed of
314. 10:58this drop was a little crazy because the
315. 11:00O1 preview and O1 models were more
316. 11:02expensive than they should have been,
317. 11:04but the drop happened nearly immediately
318. 11:06after. And then we get to the 20s, where
319. 11:08O3 was the first model to break 20, and
320. 11:11from April to August that year, GPT-5
321. 11:15mini got a meaningfully higher score.
322. 11:17Actually, no, it's like the same score.
323. 11:18It's a 20 to 21.
324. 11:20But instead of costing $1.55, it only
325. 11:21cost 26 cents. Then DeepSeek 3.2 got the
326. 11:25same score for 10 cents. And now there's
327. 11:27models like Ling-3 flash that get the
328. 11:29same score for 4.7 cents. But now models
329. 11:32are even smarter. The ones in the 30s
330. 11:34started at a dollar and 90 cents with
331. 11:365.2.
332. 11:38And not long after got to literally 3
333. 11:41cents.
334. 11:42And now that we're in the 40s to the
335. 11:4350s, Opus 4.7 started us strong here at
336. 11:46$3.85 per task, which also this is a fun
337. 11:49trend.
338. 11:50Initially, the new model at the new
339. 11:52level isn't that much more expensive.
340. 11:54But then with 01, this new level cost us
341. 11:57way more. Then the next new level cost
342. 11:59us less more. Then the next one after
343. 12:01was roughly the same. The next one after
344. 12:03was slightly more expensive but barely.
345. 12:05So like when a new level of intelligence
346. 12:07happens, the cost isn't going up a whole
347. 12:10bunch then even. And yes, all the axes
348. 12:12are exponential here. The cost measure
349. 12:14here is log, so the top is $16
350. 12:17and halfway through is a dollar. So
351. 12:20remember that as we look.
352. 12:21But now we get to where we are now,
353. 12:23where you can see some of the most
354. 12:24intense battlegrounds we've ever seen in
355. 12:27LLM history, where when we got Opus 4.7,
356. 12:30which was meaningful bump hitting a 41
357. 12:32on the intelligence index, we got 5.6
358. 12:34Tera not long after with a higher score
359. 12:37at like almost half the price. Then GLM
360. 12:395.3 getting an even higher score at
361. 12:42under a third the price. And then GLM
362. 12:445.3 Flash getting it nearly the same
363. 12:46score at a hundredth the price. So while
364. 12:49it might feel bad now because we just
365. 12:52did another one of these bumps, at a
366. 12:54given level of intelligence prices do
367. 12:56drop massively, way more than 10x year
368. 12:59over year. And when I said there was
369. 13:00three things, I kind of lied cuz there's
370. 13:02one last piece that's important to know.
371. 13:04As intelligence has gotten more
372. 13:06intelligent, the ways we use it have
373. 13:09changed. We used to have the model auto
374. 13:11complete a single function for us in our
375. 13:13code base. Now we have it take a
376. 13:15screenshot of a bug and get the change
377. 13:17landed for us. The models can do so much
378. 13:20more. They're going end to end. We don't
379. 13:22have to interfere as much because they
380. 13:23stay on task better, too. So yes, our
381. 13:26bills are getting higher. I'm not trying
382. 13:27to gaslight you into thinking otherwise,
383. 13:29but it's not because the token cost went
384. 13:31up, it's because the usage went up. We
385. 13:33are using these models so much more.
386. 13:36They are running longer. They are doing
387. 13:38a bigger tasks. They are completing work
388. 13:40that we used to have to intervene with
389. 13:41all the time. The fact that we are
390. 13:43generating way more tokens per prompt is
391. 13:46success. It's because the models got
392. 13:48cheaper and better and more capable.
393. 13:50This is kind of a Jevons paradox thing.
394. 13:53When steam power got cheaper and coal
395. 13:55got more accessible, we started spending
396. 13:57much more money on all of those things
397. 13:59because it suddenly was cheap enough and
398. 14:01good enough to be usable in places it
399. 14:03wasn't before. Our bills aren't going up
400. 14:05because the models are more expensive,
401. 14:07they're going up because we're using the
402. 14:08expensive models more.
403. 14:10It really is that simple. And this is
404. 14:12where things get really fun. Like A4 in
405. 14:15chat saying, "Imagine getting Fable 5.1
406. 14:18levels of intelligence for GLM 5 3 Flash
407. 14:20pricing." It's going to happen. It's
408. 14:22absolutely going to happen because I
409. 14:24kind of danced around something earlier.
410. 14:27I was comparing 5 3 Flash and 5 3 Flash
411. 14:30pricing to GPT 5 pricing. And comparing
412. 14:33the 25 to 50 cents per mil out on 5 3
413. 14:35Flash against the $10 per mil out on GPT
414. 14:385 is unfair. It's unfair because it does
415. 14:41not do enough credit to GLM 5 3 Flash. 5
416. 14:443 Flash is a way, way better model than
417. 14:47GPT 5. I would rather use GLM 5 3 Flash
418. 14:50for everything than be stuck using GPT 5
419. 14:53for anything because 5 3 Flash has a lot
420. 14:56of the industry-wide advancements that
421. 14:58have happened in the last year. The way
422. 14:59it does vision is better. The way it
423. 15:01handles tool calls is better. Most
424. 15:02importantly, the way it manages
425. 15:05long-running work in agentic workflows
426. 15:07where the model goes and does things for
427. 15:09hours instead of seconds is comically
428. 15:12better. GLM 5 3 Flash is not just
429. 15:15cheaper for a given level of capability
430. 15:17than GPT 5, it's also just so much
431. 15:20better in every realistic way. The only
432. 15:23thing 53 flash doesn't have that GPT-5
433. 15:26does is a bit more knowledge that
434. 15:28doesn't require searching. And chances
435. 15:30are if you gave both a search tool and
436. 15:32bench them on any knowledge anything, 53
437. 15:35flash will smoke GPT-5 like order of
438. 15:38magnitude difference simply because it
439. 15:41can do the searching well. It can use
440. 15:43its tools well. It can get solutions
441. 15:45better even if the model is dumber in
442. 15:48terms of what we've stuffed into its
443. 15:49brain. Small models with good tools
444. 15:51massively outperform last year's
445. 15:53smartest models even with tools because
446. 15:55we've gotten so much better at that.
447. 15:57And we're going to see more and more of
448. 15:58these types of improvements. And this is
449. 16:00coming from the guy who not that long
450. 16:02ago said that we had peaked.
451. 16:04Two years ago I did a video back when I
452. 16:07was mostly blonde and mustached titled
453. 16:10AI isn't going to keep improving because
454. 16:13I thought we had kind of flatlined. And
455. 16:15you can see here in the bench I had
456. 16:17GPT-3.5 then GPT-4, 14, and 4.0 did not
457. 16:21meaningfully improve on the benchmarks
458. 16:22that we were looking at. That's because
459. 16:24we needed new benchmarks.
460. 16:25I was so wrong. I've never in my life
461. 16:27had a take that was that bad. And I
462. 16:31I've learned my lesson. I was so wrong
463. 16:32it's hilarious. You're going to make fun
464. 16:34of me for being a guy who doesn't know
465. 16:36what they're doing, make fun of that
466. 16:37version of me not whatever you're mad
467. 16:39about when I talk about compaction or
468. 16:40something. I be people are mad at me for
469. 16:42things that I'm right on and they are
470. 16:44not mad at me for the things I was
471. 16:45really really wrong on for some reason.
472. 16:47But yeah, the point I'm trying to make
473. 16:49is that we are still trying to find ways
474. 16:51to measure these levels of intelligence.
475. 16:52Things are so much better than I ever
476. 16:54would have guessed they would have
477. 16:55gotten way faster than I ever would have
478. 16:57expected it to happen and way cheaper
479. 16:59than I ever could have imagined it would
480. 17:00be. The issue is that these things have
481. 17:02gotten so cheap and so capable that we
482. 17:04are just pissing money all over them. We
483. 17:06are spending so much because they are so
484. 17:08useful. And that doesn't mean that we
485. 17:10failed. In fact, I would go as far as to
486. 17:13say intelligence too cheap to measure,
487. 17:16that isn't happening in the way that Sam
488. 17:18may have implied here, but his previous
489. 17:20promise of intelligence 10 times cheaper
490. 17:24has been blown out of the water. We
491. 17:26absolutely achieved that. The tasks that
492. 17:29we used AI for last year are actually
493. 17:31that much cheaper. The difference is we
494. 17:33can do bigger and tougher things with
495. 17:35it, which is, in my opinion, way more
496. 17:37exciting. All that said,
497. 17:40it's kind of crazy that you can rack up
498. 17:41$1,500 of spend in a day using these
499. 17:44models. I don't want to discount that
500. 17:45with the things I'm saying.
501. 17:47All I'm saying is that you would never
502. 17:48have been able to do that with GPT-5,
503. 17:50not because it was so much cheaper, but
504. 17:52because you just wouldn't be able to get
505. 17:53that much work done. LLM spend is tied
506. 17:56to the productivity we get from LLMs,
507. 17:58and yes, we are getting more from that.
508. 18:00Think I've said all I have to on this
509. 18:01one. I hope you can better understand
510. 18:03where our money's going now. And don't
511. 18:04worry, things will continue to get
512. 18:06cheaper, even if it doesn't look like it
513. 18:08right now. Keep your head up and be
514. 18:09excited for all the developments that'll
515. 18:11continue to make models cheaper and more
516. 18:12capable. I know I've been hyped about
517. 18:14this, and I hope you are, too. Until
518. 18:16next time,
519. 18:17peace, nerds.