OpenAI DevDay - What Actually Matters for Builders — Transcript

Full transcript

1. 0:00Okay, so open AI had their Dev Day
2. 0:02yesterday and there are a number of sort
3. 0:05of really interesting points in there.
4. 0:06What I want to do in this video is go
5. 0:08through what they announced and give you
6. 0:11my honest take on some of the different
7. 0:13things that are there. I actually think
8. 0:14there were some really interesting
9. 0:16announcements not only for open AI
10. 0:18themselves but also about how people are
11. 0:20thinking at the frontier model labs
12. 0:22themselves. All right, so first up is
13. 0:25dots. So what actually is a dot? It's
14. 0:28basically an always-on agent that lives
15. 0:31inside your chat GPT or at least chat
16. 0:34GPT and Codex for now. There were some
17. 0:37really interesting things though where
18. 0:38you can sort of text it, you can call it
19. 0:41and stuff like that. They talk about
20. 0:43that it has its own computer in the
21. 0:45cloud, its own browser. It can write and
22. 0:48test code for you. It uses the same
23. 0:51plugins that you've already connected to
24. 0:53your account. And all those sort of
25. 0:55plugins and apps and connectors, there
26. 0:58are thousands of them there. One of the
27. 1:00cool things here is you can add it to
28. 1:01Slack. So in many ways you could kind of
29. 1:04think of open AI basically covering off
30. 1:07what anyone else has done in the
31. 1:10personal agent space. So I think the big
32. 1:13thing here is that this is their first
33. 1:16or at least their more substantial foray
34. 1:19into personal agents. So the idea here
35. 1:23of where it can basically do things for
36. 1:26you, it can be probably a lot more
37. 1:27proactive than just reactive. Now it's
38. 1:31going to be interesting to see how that
39. 1:33proactiveness actually plays out. Is
40. 1:36this just a bunch of sort of cron jobs
41. 1:38going on underneath or is there
42. 1:40something a bit more to it? Obviously a
43. 1:43lot of the ideas for this have come from
44. 1:46things like Muse, like Grokbot, like
45. 1:49Instinct, which is a startup that is
46. 1:51very hot at the moment in San Francisco.
47. 1:53It's just raised about a billion dollars
48. 1:55on a $10 billion valuation for their
49. 1:57first round.
50. 1:59And of course, the other two personal
51. 2:00ones here would be Open Claw and Hermes.
52. 2:04And don't forget, while OpenAI didn't
53. 2:06buy Open Claw directly, they did hire
54. 2:09the creator of that. And you can imagine
55. 2:11he's had a lot of input on where dots is
56. 2:14actually going and how that should
57. 2:15actually work. So, the interesting thing
58. 2:17here is that this seems both be a pitch
59. 2:19that this is your personal agent, but
60. 2:21also perhaps more than your personal
61. 2:23agent. They talk about the you give it
62. 2:25responsibility. So, rather than just
63. 2:27sort of go and do a task for me, like
64. 2:30book a flight or something, this is
65. 2:32going to sit on top of things and manage
66. 2:34and monitor things, whether that's like
67. 2:36incoming bug reports, whether that's
68. 2:39testing fixes, whether that's doing a
69. 2:41whole bunch of different things. Now,
70. 2:43the interesting thing is that this is
71. 2:44actually powered by Astra. So, it's
72. 2:47powered by the top-level model that
73. 2:49they've got there. And it does seem, in
74. 2:51a very sort of Open Claw / Hermes kind
75. 2:54of way, that you're kind of expected to
76. 2:56treat this as your chief of staff. Now,
77. 2:59the usage model for this is really kind
78. 3:01of interesting and worth understanding
79. 3:02as well. OpenAI said that your dot is
80. 3:05included in your plan. So, talking to it
81. 3:08doesn't consume your token usage sort of
82. 3:11stuff. But, anything that it kind of
83. 3:13does on your behalf does. So, a dot
84. 3:16operates on top of your ChatGPT and your
85. 3:19Codex threads, but the threads that it
86. 3:21drives for you in those particular apps,
87. 3:25so in its own computer, it looks like
88. 3:27they're going to be threads that do
89. 3:28count to your token usage. So,
90. 3:30basically, talking to it is free, but
91. 3:33the work that it does for you is not
92. 3:35going to be free. I can still see that
93. 3:37this is going to be a huge win for
94. 3:39people who perhaps have never tried out
95. 3:42Open Claw, never tried Hermes. And in
96. 3:45many ways, Google tried to do this with
97. 3:47Gemini Spark after Google I/O, but it
98. 3:50seems to me that that hasn't taken off
99. 3:52as much as they would have liked,
100. 3:54perhaps in large degree because of the
101. 3:56models that were actually running it and
102. 3:59what it actually had access to. You
103. 4:01could imagine, too, that the actual dots
104. 4:03or the dot that works for you is going
105. 4:06to be a large part of the OpenAI
106. 4:08hardware that supposedly they're
107. 4:11bringing out a product either late this
108. 4:13year or next year sometime. And so, if
109. 4:16that product is like people have
110. 4:17pictured where it's some kind of, you
111. 4:19know, hockey puck or something like that
112. 4:21that you can then talk to, that you can
113. 4:23then interact with, you could imagine
114. 4:25that that is something then that people
115. 4:26are going to basically put on their
116. 4:27desk, have it sitting there on all the
117. 4:30time, being able to talk to their dot.
118. 4:33And if they feel the usage for that is
119. 4:34free for the actual conversations, but
120. 4:37they're just getting billed for anytime
121. 4:39this thing goes off and does something
122. 4:40for you, I can see that being appealing
123. 4:42to a lot of people who don't have a plus
124. 4:44account, don't have a Teams account,
125. 4:46don't have a pro account currently. And
126. 4:48like I've said in a number of videos, if
127. 4:50you're watching my videos, you're
128. 4:51definitely one of the earlier adopters.
129. 4:54I think some of the real huge growth
130. 4:55that we're going to see over the next
131. 4:57couple of years with these particular
132. 4:59companies and with AI itself is just the
133. 5:02normies or the sort of masses are going
134. 5:05to come along and start using these
135. 5:06things in their day-to-day life. All
136. 5:08right, so let's look at some of the
137. 5:09developer stuff that they launched. So,
138. 5:12that brings us to sort of agents API.
139. 5:14And this is basically their managed
140. 5:16agent runtime. So, just like we've got
141. 5:18Claude managed agents, we've got a
142. 5:20number of companies that are doing this
143. 5:22and I've made videos about these
144. 5:23specific things. You basically get a
145. 5:25harness, hosting, memory, sort of
146. 5:28multi-agent controls, and now they've
147. 5:31added computer use to that as well. I do
148. 5:33wonder how much is this a me-too
149. 5:35product, basically where they just don't
150. 5:37want to lose market share to Anthropic
151. 5:39who's been offering their Claude managed
152. 5:42agents since much earlier this year. And
153. 5:45And interesting that this does seem to
154. 5:46be the same stack that powers Codex and
155. 5:49dots. Just like the Claude managed
156. 5:52agents is using a lot of the same stack
157. 5:55that powers Claude code and co-work. So,
158. 5:57the way I think about this one is that
159. 5:59there are a lot of companies out there
160. 6:00that are using the open AI models,
161. 6:02whether that's directly, whether that's
162. 6:03through AWS or Microsoft. This gives
163. 6:06them a way to build vertical harnesses
164. 6:09so that an agent for insurance claims,
165. 6:12an agent for sort of contract reviews,
166. 6:15all of those things are going to be
167. 6:16possible. And it's basically going to be
168. 6:18running on a runtime that's kind of been
169. 6:20rubber stamped by open AI as to what
170. 6:23they're saying is the best way for
171. 6:25running their own agents.
172. 6:27The second big developer news here was
173. 6:29the whole decisions API. And I kind of
174. 6:32feel this one really wasn't a surprise.
175. 6:34I've made comments before that probably
176. 6:36most of the frontier labs have Jev
177. 6:38models in development after seeing just
178. 6:41the reactions to the whole idea of a
179. 6:44decision model. And yet again, like I've
180. 6:46said in a couple of my videos, it does
181. 6:48seem that these people are settling on
182. 6:49the name of decision models with open AI
183. 6:52actually calling their one the decisions
184. 6:55API. The idea here is that this is a
185. 6:57Luna model. So, again, you don't need a
186. 6:59huge model to do this. This is something
187. 7:01that we've looked at in my videos with
188. 7:03the open models, etc. Certainly, Luna is
189. 7:05actually a bigger size than a lot of the
190. 7:07open models that I've shown. And pretty
191. 7:09much the idea here is very similar to
192. 7:11Jev. You've got a model, you give it a
193. 7:13predefined set of options, and it
194. 7:15basically sort of picks one out.
195. 7:17Interestingly, they are launching with
196. 7:18image support. And I kind of feel this
197. 7:21is a no-brainer here. You don't need a
198. 7:22big model to do it, and you don't need a
199. 7:24huge amount of training. Open AI didn't
200. 7:27really even hide that. They said on
201. 7:28stage, it's the Luna model. And I think
202. 7:31for a lot of organizations that aren't
203. 7:33going to want to add another new
204. 7:34company, aren't going to want to run
205. 7:36something like this on prem, this just
206. 7:38becomes a no-brainer for them to
207. 7:40basically use the open AI version of
208. 7:42this. Now, another big developer
209. 7:44announcement that they had there was the
210. 7:46new model. Now, almost every year we get
211. 7:49a new model and stuff like that. And at
212. 7:51first, my gut reaction was like, "Okay,
213. 7:53GPT 6.1 Soul." Remember, Astra is the
214. 7:57really big one. Soul is their version of
215. 7:58sort of Opus. My first reaction was
216. 8:01like, "Okay, so what? Does this really
217. 8:03kind of matter here?" But, it was
218. 8:05interesting that just like Opus 5.5
219. 8:08seems to have improved massively on
220. 8:10Fable. They're saying that GPT 6.1 Soul
221. 8:13is actually getting up there on par with
222. 8:16the Astra level here. And the other
223. 8:18thing here is that this is like a fifth
224. 8:20of the actual Astra costs. So, it's $2
225. 8:24per million input tokens, $10 per
226. 8:27million output tokens, and then
227. 8:29amazingly, only 10 cents for caching
228. 8:31that input. So, when you compare that to
229. 8:34Astra, which is $10 in, $50 out, this is
230. 8:38an 80% saving for a model that's
231. 8:40supposed to be on par with that Astra
232. 8:43model. Now, it does seem that just like
233. 8:45with Fable, there are certain kinds of
234. 8:47things where if you're maxing out the
235. 8:49thinking on Astra or on Fable, the
236. 8:52second tier of models is going to be
237. 8:54just slightly not as good there. The
238. 8:57other thing that I thought was
239. 8:57fascinating here was the actual price
240. 8:59for caching. So, it's a 95% discount.
241. 9:03And we know that sort of agents reread
242. 9:05the same context over and over again,
243. 9:08the system prompt files, the
244. 9:10conversation so far. So, for agent
245. 9:13workloads, the cash price is going to be
246. 9:15the real price in here. And on a
247. 9:17personal note, I noticed this yesterday
248. 9:18with something that I was working on,
249. 9:20where I'd done 250 million tokens of
250. 9:22using the Opus 5.5 model, yet about 96%
251. 9:27of all the tokens were actually already
252. 9:29cached. So, the price wasn't anywhere
253. 9:31near what you would think it would be if
254. 9:34you were talking about 250 million
255. 9:35tokens there. So, out of this, I think
256. 9:38if you're running agents on OpenAI, Soul
257. 9:406.1 is probably going to become your new
258. 9:43default model there. All right, next up
259. 9:45that brings us to Codex. And there are a
260. 9:47few things that people have been asking
261. 9:49here for. So, Codex on Linux, multiple
262. 9:51folders in one project, Codex on your
263. 9:54phone. Sam Altman basically announced
264. 9:56those very quickly even before they got
265. 9:58to probably some of the interesting
266. 9:59things in here. Big one for me here is
267. 10:02that whole idea of Codex in the cloud.
268. 10:04So, this definitely enables you to
269. 10:06basically start a task on your phone,
270. 10:08pick it up in your browser, run with it
271. 10:10over a lot of different surfaces. And it
272. 10:13was kind of funny in Romaine's demo for
273. 10:15that being literally like, "Okay,
274. 10:17rewrite the entire back end in Rust."
275. 10:19and then just sort of leaving it and
276. 10:21walking away. Interestingly, they've
277. 10:23also brought voice into the app and to
278. 10:26the CLI. So, this is their bidirectional
279. 10:28audio powered by GPT Live. So, it's not
280. 10:31just doing sort of dictation.
281. 10:33Unfortunately, they had some hiccups
282. 10:34with that on stage. But, I guess if
283. 10:37you're like myself or most people who
284. 10:38are using these agents nowadays, you're
285. 10:40talking to them and not typing to them
286. 10:43most of the time anyway. There are also
287. 10:45some nice things that they showed around
288. 10:47computer use and allowing Codex to
289. 10:49actually use that, whether that's for
290. 10:51sort of getting app shots as it's
291. 10:52actually making something for you, and
292. 10:55some nice stuff around security of where
293. 10:57it can basically scan for
294. 10:58vulnerabilities and prepare fixes for
295. 11:01you to review as it's actually going
296. 11:03through your code base. All right, so
297. 11:05next up was one that I think is really
298. 11:07kind of interesting and this was
299. 11:08ultrafast. So, this is Astra at eight
300. 11:11times the speed of the normal API for
301. 11:14six times the price. Now, they're
302. 11:16claiming that this can do around 300
303. 11:18tokens per second. I'm guessing this is
304. 11:20basically the running on Cerebras
305. 11:23version of these models. And as we're
306. 11:26all starting to see with running these
307. 11:28agents, these agents can take hours. So,
308. 11:31I guess if you've got the budget to pay
309. 11:33for this, being able to get something
310. 11:35done at a really fast pace may be worth
311. 11:38it. I'm not totally convinced though how
312. 11:39much people will actually pay for the
313. 11:42the six times the price API. They
314. 11:45mentioned that this is out on Astra and
315. 11:47coming to Soul very soon. I guess if you
316. 11:49were to take the Soul 6.1 at six times
317. 11:52the price, it's only slightly more
318. 11:55expensive than the normal speed Astra
319. 11:57was before. So, like that, maybe it is a
320. 11:59good deal. Let me know in the comments.
321. 12:01I can certainly see that maybe there are
322. 12:02people that are quite happy to pay that
323. 12:04premium to get the tokens a lot faster,
324. 12:07etc. And it is probably going to make
325. 12:09sense where latency is the product. So,
326. 12:12for anything that you're doing sort of
327. 12:14live editing or interactive demos or
328. 12:18where there's actually a user sitting
329. 12:19there waiting for something to happen,
330. 12:22then those tokens being that much faster
331. 12:24probably is going to be worth it. And
332. 12:26this kind of ties us into one of the
333. 12:28things that I think is probably
334. 12:29controversial that they announced, that
335. 12:31there's a new Pro 500 plan. And as I was
336. 12:35watching this, I only kind of twigged
337. 12:37what it was as they were sort of rolling
338. 12:39out that there were Pro 200, Pro 100
339. 12:42plans as well. So, this is a 500 being
340. 12:46$500
341. 12:47per month here. And this is going to
342. 12:49have 25 times the usage of Plus. It's
343. 12:53also going to have access to ultra-fast
344. 12:55in ChatGPT and Codex. And interestingly,
345. 12:58that usage is going to carry over into
346. 13:00partner apps. So, that's something
347. 13:02that's going to be interesting to see.
348. 13:04Now, they reopened the Pro 200 plan. So,
349. 13:07the $200 plan, they actually closed that
350. 13:09after Astra for a while. I think they
351. 13:11just didn't want people just hammering
352. 13:13it and stuff like that. And the sad
353. 13:15thing here is and the thing that perhaps
354. 13:16is going to anger a lot of people is
355. 13:18that this has gone from being 20 times
356. 13:21Plus down to 10 times Plus here. Now, we
357. 13:25know that usage limits are something
358. 13:26that kind of fluctuates anyway, so it's
359. 13:29going to be interesting to see how this
360. 13:31actually plays out. And does that drive
361. 13:35a lot of people who were paying for the
362. 13:37200 to move to the $500 plan, or does it
363. 13:40have the opposite effect and drive more
364. 13:42people to go over to Anthropic's plan?
365. 13:45It does seem at this point that the
366. 13:46pricing is kind of steering you, and
367. 13:48that really now if you're going to go
368. 13:50for the sort of top-tier models, you're
369. 13:53going to have to be paying quite a lot
370. 13:55of money to do that. All right, so the
371. 13:56third part of the keynote was about
372. 13:58helping you to find customers. And here
373. 14:01basically Sam Altman was pretty candid
374. 14:03that they've tried different versions of
375. 14:05these kind of things before with mixed
376. 14:08results. And just to recap those, we've
377. 14:11seen a bunch of these things. They first
378. 14:12started out with ChatGPT plugins back in
379. 14:15March '23. They had GPTs for a while
380. 14:18from sort of November '23. They had the
381. 14:21GPT store. They then went and sort of
382. 14:24switched off all the original plugins.
383. 14:26Around this time last year they
384. 14:28announced apps in ChatGPT and the whole
385. 14:30sort of apps SDK. And I don't know about
386. 14:33for you, but I haven't really seen apps
387. 14:35take off here at all. So it does seem to
388. 14:38me that this is something that's still
389. 14:39definitely a work in progress. And I
390. 14:41think Sam Altman's statement of saying
391. 14:43that they'd had mixed results with what
392. 14:45they've launched is in many ways an
393. 14:47understatement here. So what have they
394. 14:50got new? So I would say the big and most
395. 14:52interesting piece here is the whole
396. 14:54sign-in with ChatGPT, where users can
397. 14:57sign in to your app and use the tokens
398. 15:00that come with their ChatGPT plan on
399. 15:03your particular app. And I think the
400. 15:05argument here is that that allows you to
401. 15:07basically ship sort of like a free core
402. 15:10app. That because you're no longer
403. 15:12having to pay for the inference of
404. 15:14tokens, you can sort of give people a
405. 15:17taste, and then you can upsell to other
406. 15:19parts of it without it costing you a
407. 15:21fortune. So I think this really opens up
408. 15:23the door to a bunch of new freemium
409. 15:25products that we'll see that use this
410. 15:28kind of thing. The challenge is going to
411. 15:30be, though, do we very soon see a
412. 15:32sign-in with Claude? And then are sites
413. 15:35going to basically have to allow people
414. 15:37to sign in with Claude or to sign in
415. 15:39with ChatGPT or to sign in with Gemini?
416. 15:42All these ideas are trying to do what
417. 15:44the Facebook login button did sort of
418. 15:46back 15-plus years ago. But, we don't
419. 15:49just have one major player here. We've
420. 15:50actually got a few different players.
421. 15:52It's also not clear what are you
422. 15:53actually offering if someone comes to
423. 15:55your site and they've only got a free
424. 15:57ChatGPT plugin.
425. 15:59Uh you then sort of stopping their
426. 16:01inference halfway through a question or
427. 16:04halfway through some particular task and
428. 16:06having to tell them, "Hey, you should go
429. 16:09and pay for ChatGPT as opposed to you
430. 16:11should pay for my product on my site."
431. 16:14All right, next up a plugin extension.
432. 16:16So, these are basically full apps that
433. 16:17run inside ChatGPT and Code X. Figma and
434. 16:21Adobe both showing versions of their
435. 16:24products running in there. And
436. 16:25supposedly ChatGPT is going to suggest
437. 16:27your plugin in the middle of a
438. 16:29conversation. I think we're still in the
439. 16:31same place as last year where we don't
440. 16:33know how it's going to suggest that I
441. 16:35use Adobe Photoshop over some other
442. 16:38hyper cooler image editor that happens
443. 16:41to be in their directory as well. Is
444. 16:43that going to come down to whichever
445. 16:44developer pays OpenAI the most? I don't
446. 16:47know. I guess we'll see with time. They
447. 16:48also launched an OpenAI marketplace with
448. 16:51a bunch of different partners including
449. 16:52people like Code Rabbit, Notion, etc.
450. 16:55And I thought it was kind of funny that
451. 16:57Sam Altman actually made the point that
452. 16:59because Base Ten, who's an inference
453. 17:01provider, is one of these partners in
454. 17:03the marketplace, you will be able to
455. 17:05access open-source models in the
456. 17:08marketplace. My bet is that's something
457. 17:10that you're not going to see on Claude
458. 17:12anytime soon in any way, shape, or form.
459. 17:15So, overall I would say that the event
460. 17:17was actually very interesting. There
461. 17:18were lots of things showing sort of
462. 17:20ideas of where things are going. I do
463. 17:22think that things like the agents API is
464. 17:24going to be something that a lot of
465. 17:26people are going to use in organizations
466. 17:28to make their hosted agents, if they're
467. 17:30already an open AI customer. I do think
468. 17:32it's going to be really interesting to
469. 17:33see how dots plays out. And 3 months
470. 17:37from now, are 10 million people using it
471. 17:39every day? Or is it something that
472. 17:40people have just basically added onto
473. 17:42the side? And it could be that dots
474. 17:44becomes a new part of co-pilot or of
475. 17:46some kind of enterprise apps, which is
476. 17:48something that they actually talked
477. 17:50about making sort of work versions of
478. 17:52these kind of helpers as well. Anyway,
479. 17:54let me know in the comments what you
480. 17:56thought was interesting there. I know a
481. 17:57lot of people are going to bag on open
482. 17:59AI and criticize everything that they
483. 18:01did. And if you've listened to the end
484. 18:03here, thank you very much cuz I know
485. 18:04this video is probably not going to do
486. 18:06very well. But I do feel it is very
487. 18:08important to see what they're thinking
488. 18:10about how things are going to play out.
489. 18:13We know that the all you can eat buffet
490. 18:15of tokens cannot go on forever as costs
491. 18:18go up. We know that lots of people are
492. 18:20trying to find the killer app for AI so
493. 18:23that you can charge a premium and not
494. 18:25just focus on selling tokens all the
495. 18:27time. And I think open AI's dev day is
496. 18:30always interesting to get a sort of
497. 18:32state of play of where things actually
498. 18:35are both for consumers, but for us as
499. 18:38developers as well. So, anyway, as
500. 18:40always, if you like the video, please
501. 18:42click like and subscribe and I will talk
502. 18:43to you in the next video. Bye for now.