I REALLY Want Open Models to Win, But They Just Don't Cut It (Yet) — Transcript

Full transcript

1. 0:00Every single time a new open model comes
2. 0:02out, like a new Kimi, GLM, or DeepSeek
3. 0:04model, I always get excited because they
4. 0:06show these really promising benchmarks
5. 0:08and they say that they match or approach
6. 0:09the strongest proprietary models for AI
7. 0:12coding. And I think to myself, just for
8. 0:14that day, "Oh man, we've finally gotten
9. 0:16to the point where open weight models
10. 0:18are as good as frontier models for AI
11. 0:20coding, so we don't have to rely on the
12. 0:22big companies anymore like Anthropic and
13. 0:24OpenAI for everything." But then I
14. 0:26actually test the model and I'm
15. 0:28disappointed every single time. But
16. 0:31exactly how these open models disappoint
17. 0:33me actually has a lot of nuance to it
18. 0:34because genuinely they are impressive
19. 0:36and they do match the performance of the
20. 0:39best frontier models on a lot of
21. 0:41different things for a fraction of the
22. 0:42cost. Like looking at Terminal Bench 4.0
23. 0:44here, comparing Kimi K3.1 Pro Max to
24. 0:47GPT-6 Astra and Claude Opus 5.5, it's
25. 0:50even a bit better in this benchmark. And
26. 0:52so the question I had to ask myself is
27. 0:54what gives? How come Kimi, Qwen, GLM,
28. 0:57all these companies can release these
29. 0:59open models with benchmarks saying that
30. 1:01they're better than the best frontiers,
31. 1:02but then when I actually try to use it
32. 1:03to build real things, it just falls
33. 1:06apart. And so that's what I want to
34. 1:07cover with you in this video, why I
35. 1:09can't use open models for all my AI
36. 1:11coding, the testing that I did so I can
37. 1:13really prove it to you, and then also
38. 1:15the solution because I really am rooting
39. 1:17for open models. I do think this is the
40. 1:20future and so I want to incorporate them
41. 1:22as much as I can and there are a lot of
42. 1:24ways to do so even if we don't hand over
43. 1:26all of our workflows. Now, don't get me
44. 1:28wrong, I love using my Codex Code
45. 1:30subscriptions. I'm using them every day,
46. 1:32but those subscriptions are not
47. 1:33guaranteed to be around forever and we
48. 1:35all know the rate limits are getting
49. 1:36worse. And so I don't want to be
50. 1:38dependent on that. And if it ever got to
51. 1:40the point where I had to pay the API
52. 1:42pricing for Opus 5.5 or GPT-6 Astra, it
53. 1:45would not be realistic for me
54. 1:47financially to be using it for most
55. 1:49things just considering the amount of
56. 1:50tokens I'm using every day. Now, there
57. 1:52are a lot of other reasons to root for
58. 1:54open models and invest into using them,
59. 1:56but we don't have to get into that right
60. 1:57now. I want to focus on the performance
61. 1:59here. This is the big dichotomy that
62. 2:01I've seen. Frontier models and open
63. 2:04models are equivalent in performance
64. 2:06when you give them shorter tasks, even
65. 2:08if they are more complicated and require
66. 2:10some deeper reasoning. And so, there's a
67. 2:12lot of different tasks that I've given
68. 2:14both frontier and open models, and I've
69. 2:16had a separate agent review them, and
70. 2:18then I've also reviewed the outputs
71. 2:19myself. And it is basically equivalent.
72. 2:22And of course, using the open models
73. 2:24here, it's a lot cheaper. And so, for
74. 2:26these kinds of tasks, in fact, for most
75. 2:29business automations and use cases for
76. 2:31agents, open models win.
77. 2:33I've even seen a lot of studies and
78. 2:35professional opinions circulating
79. 2:36recently that say things like open
80. 2:38weight models will run 90% of enterprise
81. 2:40agentic workflow steps. And they usually
82. 2:42cite within around 2 to 5 years, cuz
83. 2:44we're not at that level of mass adoption
84. 2:46yet, but we're getting there very
85. 2:48quickly. And I'm even seeing this myself
86. 2:49as I work with companies, and I see how
87. 2:51they're very cautious for token spend,
88. 2:54especially within the last half year.
89. 2:55And for a lot of these open weight
90. 2:57models, to run it locally because
91. 2:58they're dealing with sensitive data. And
92. 3:00the thing is, 9 in 10 agent workflow
93. 3:02steps are routine, and open weight
94. 3:04models handle them at parity for a
95. 3:06quarter of the price of even the
96. 3:08cheapest closed model, something like
97. 3:10Haiku 5.5. And so, the point is, where
98. 3:12we really need to have agents built into
99. 3:15larger business automations, it's just a
100. 3:16step in the workflow where we don't have
101. 3:18to have GPT-6 Astra or Opus 5.5. Most
102. 3:22things you can have a weaker model that
103. 3:25gets the job done and is a lot cheaper.
104. 3:26And that's where open weight models win
105. 3:28every time. And I say weaker, but really
106. 3:30I should correct myself, because like I
107. 3:32was saying, for the short-running tasks,
108. 3:34the open models are just as good as the
109. 3:36frontier models. Where they really fall
110. 3:38apart, and this is what matters a lot
111. 3:40for agentic coding, is for
112. 3:41longer-running tasks. If we're using
113. 3:44these open models to drive a larger
114. 3:46harness, like my AI software factory,
115. 3:48for example, that's where I see them
116. 3:50disappoint time and time again. And I'll
117. 3:51show you exactly how. So, what I have
118. 3:53here is a large simplification of my AI
119. 3:56software factory. This is generally what
120. 3:58my AI coding workflow looks like,
121. 4:00because I don't just open up Claude
122. 4:02code, Kimiko, whatever, and send off a
123. 4:04request. I have skills and rules that
124. 4:06guide myself and my agent through the
125. 4:08process of planning together,
126. 4:10implementing based on that plan, and
127. 4:11then having myself and another agent
128. 4:13review and playtest to make sure we're
129. 4:14confident for the merge. I do this for
130. 4:16every bug fix and every new feature that
131. 4:18I'm implementing. The problem here is
132. 4:21there is a lot of opportunity for drift
133. 4:23that comes naturally with any complex
134. 4:25code base and larger process you're
135. 4:27going to be working with. And this is
136. 4:29where open models fall apart. Even
137. 4:30Kimiko 3.1 Pro, which I keep referencing
138. 4:33because it is really the latest and
139. 4:34greatest open model. What it comes down
140. 4:36to, I've kind of thought of a word for
141. 4:38this. It's all about the attention. It
142. 4:40seems like open models aren't able to
143. 4:42pay attention to all of the rules and
144. 4:45context it needs to be following when we
145. 4:47have longer running processes. Like when
146. 4:49I'm working with Opus 5.5, I have all my
147. 4:52rules in a complex code base and it has
148. 4:54to understand where to integrate things
149. 4:56and what to review. It always just seems
150. 4:58to get me and get the code base. Even
151. 5:01when I have it running for a long time.
152. 5:03But it's not the case with Kimiko. It
153. 5:04does good for individual edits if I keep
154. 5:06it very, very granular, but if I really
155. 5:08try to expand the scope, it just starts
156. 5:10to forget rules that I have for it in
157. 5:12the code base. Like even the most basic
158. 5:14things in my global rules, like don't
159. 5:16touch the PRD, it's read only. I've had
160. 5:18it try to do that. It makes that attempt
161. 5:20quite a few times as I've done testing
162. 5:22over the past couple of weeks. And it
163. 5:24just constantly gets stuck because I
164. 5:26have strict guidelines for what is
165. 5:27allowed to merge in my code base. I have
166. 5:30a pretty high bar that Kimiko seems to
167. 5:32never really be able to reach without
168. 5:34iterating over and over and over again.
169. 5:35And that actually makes it so that
170. 5:37Kimiko is even more expensive than
171. 5:38Claude just because of how many tokens I
172. 5:40have to spend fixing all the little
173. 5:42problems that come up.
174. 5:44And we can even see this play out when
175. 5:45we look at the larger list of
176. 5:46benchmarks, not just the ones that
177. 5:48Kimiko wants you to see, like Terminal
178. 5:50Bench 4.0. So yeah, definitely for some,
179. 5:53Kimiko 3.1 and a lot of the other open
180. 5:56models are better or on par than the
181. 5:58frontier models. So, there's something
182. 6:00to it. But if we look at the other
183. 6:03benchmarks that are focused more on long
184. 6:05horizon tasks like Terminal Bench
185. 6:07Science, Frontier Code 2.0, there is a
186. 6:09very very big difference. This is more
187. 6:12in line with what I've seen with Kimmy K
188. 6:143.1 Pro and the other open models
189. 6:16seemingly falling flat on their face
190. 6:18compared to the Frontier models like
191. 6:19Gemini 4 Carbon and Claude Opus 5.5. And
192. 6:23so for how I actually want to
193. 6:25incorporate models in my agentic coding
194. 6:27workflows, this is more my experience,
195. 6:30not this as much as I wish Terminal
196. 6:32Bench 4.0 was accurate for my work. Now,
197. 6:35I also don't want you to be under the
198. 6:37impression that Frontier models have
199. 6:38totally solved long-running tasks.
200. 6:41There's still a struggle there. I've
201. 6:42seen this in a bunch of research
202. 6:43including my own that Frontier models do
203. 6:46still struggle on long horizon tasks.
204. 6:49It's just they struggle a lot less than
205. 6:51open models like GPT 5.6 Soul being at a
206. 6:5441.6% in this study and then Kimmy K3
207. 6:57only being at a 22.1%.
208. 7:00That is a big difference. So we still
209. 7:02need harness engineering as a whole. We
210. 7:05need the guidance in the process for
211. 7:07even the best models to get any kind of
212. 7:09accuracy with these larger AI coding
213. 7:11workflows, but even if we have a lot of
214. 7:13that guidance and process set up, it
215. 7:15still doesn't seem to help these open
216. 7:17models nearly as much.
217. 7:19The sponsor of today's video is
218. 7:21TripleTen and they build programs for
219. 7:22senior engineers who want to get really
220. 7:24good at building AI systems. AI has
221. 7:27actually been the top reason companies
222. 7:29gave for job cuts in the US this year,
223. 7:32but the need for engineering hasn't just
224. 7:34disappeared. It's shifted and most of
225. 7:36the growth in job postings has gone to
226. 7:38senior roles. You can pick up an AI tool
227. 7:41in a weekend, but getting real
228. 7:42experience managing AI systems in
229. 7:44production is really hard to do on your
230. 7:46own. And that is what TripleTen is for.
231. 7:49Their two programs, AI system
232. 7:50engineering and AI machine learning, are
233. 7:53made for engineers with a couple of
234. 7:55years of production experience. So this
235. 7:56is not a beginner program. You build
236. 7:59five projects that come together into a
237. 8:01real system like a retrieval pipeline
238. 8:03with evals and a support agent hooked
239. 8:05into Jira with MCP. And you defend your
240. 8:07projects in front of a real engineer,
241. 8:09which I really like. Triple 10's expert
242. 8:12projects are public on GitHub, too, so
243. 8:14you can see the kinds of systems that
244. 8:15you build before signing up. And the
245. 8:17instructors have public profiles you can
246. 8:19look up yourself, like Muhammad Sharif,
247. 8:21who's worked at AMD, Microsoft, and
248. 8:23Amazon. If you have a few years of
249. 8:25engineering under your belt and you want
250. 8:27to go deeper on AI systems, I'll have a
251. 8:29link to both programs in the
252. 8:31description.
253. 8:32So, everything I've covered with you
254. 8:33here is based on all my research and my
255. 8:35experience doing a lot of comparing
256. 8:37closed and open models over the course
257. 8:40of this year. But, also, I want to make
258. 8:42things interesting for you and really
259. 8:43quantify what I'm seeing. And so, I put
260. 8:46together a very controlled test using my
261. 8:48AI software factory to build the exact
262. 8:51same application with Claude, GPT-6, and
263. 8:55Kimiko 3.1 Pro. And so, every single
264. 8:57model here got its own AI software
265. 8:59factory to build the exact same video
266. 9:01game. I chose a video game because
267. 9:03they're fun for me, nice to visualize
268. 9:05for you, and I wanted something
269. 9:06reasonably complex, not just a simple
270. 9:08web app, so I can really push the limit
271. 9:10of these models.
272. 9:11So, the game that I built is called
273. 9:13Northstar. I have my real factory
274. 9:15building the full version of this right
275. 9:16now, but this is the version that I
276. 9:18built with Claude, using Opus 5.5 for
277. 9:20the planning and reviewing, and Sonnet
278. 9:225.5 for the builder, the kind of
279. 9:25workhorse. And so, this is a very, very
280. 9:28simplified version of the game because I
281. 9:29just wanted to run about 12 laps in
282. 9:32total of the factory. So, there's 12
283. 9:34tickets that I built for this game. And
284. 9:36it's pretty good overall. Like, this is
285. 9:38a really solid with the physics. I mean,
286. 9:39the combat is kind of silly with the
287. 9:41animations. Like, my guy looks kind of
288. 9:43bad here, but the full game loop is
289. 9:45working here. This was built by Claude
290. 9:47within my software factory.
291. 9:49So, here's a version of the game built
292. 9:50by GPT with 6 Astra for the planning and
293. 9:53reviewing, and then the workhorse with
294. 9:556.1 Soul. And so, the HUD looks pretty
295. 9:58ugly here, but if we actually go into a
296. 9:59raid, it looks good. So, of course,
297. 10:01there's going to be little issues to fix
298. 10:02with all of these proof of concepts
299. 10:04because I didn't spend a ton of time on
300. 10:06it, but this one had 11 tickets in total
301. 10:08to get to a rat now. So, there's some
302. 10:10hit boxes left over. It looks like some
303. 10:12debugging things. So, it doesn't look as
304. 10:14good as Claude. It also decided to do a
305. 10:16much more zoomed out view. So, each one
306. 10:17of my factories got the exact same
307. 10:19starting point for the artwork, which is
308. 10:21why it looks so similar. But again, like
309. 10:23Claude, it did a pretty good job
310. 10:24overall. My goal here was absolutely not
311. 10:26to make a polished version of the game,
312. 10:29but to run a bunch of tickets on a more
313. 10:30complex code base just to get a feel for
314. 10:33how well it was able to build out the
315. 10:34features and test its work. And then
316. 10:37finally, this is the game built by
317. 10:38Kimmy, which has a much more zoomed out
318. 10:40view like the one built by Claude. And
319. 10:42then if I start a raid here, the UI
320. 10:44actually looks kind of clean compared to
321. 10:46Claude and GPT. I like it. But the
322. 10:48problem is, and we'll get into this in a
323. 10:50little bit, it didn't build nearly as
324. 10:52much as far as the features in the game
325. 10:54because it spent so much time fixing all
326. 10:56these little things and iterating on
327. 10:58each of the individual tickets. And so,
328. 11:00for example, I don't really have time to
329. 11:01show this here, but between raids
330. 11:03there's a shop where you can upgrade
331. 11:05your character with the Claude and Open
332. 11:07AI versions, but Kimmy never got around
333. 11:09to building that.
334. 11:10So, I know this is kind of a silly
335. 11:12example, and it is just one application,
336. 11:14but I wanted to build something concrete
337. 11:16to give you some tangible numbers here
338. 11:18for what I've been seeing building
339. 11:19dozens of applications with frontier and
340. 11:21open models. And so, there's not a lot
341. 11:23of data points as far as the sheer
342. 11:25number of tickets, but I did spend tens
343. 11:27of millions of tokens with all the
344. 11:28testing I did here, really letting it
345. 11:30iterate and struggle through things so I
346. 11:32could get the real data. And this is in
347. 11:34line with everything I've seen
348. 11:35throughout this year. So, Claude did 12
349. 11:38out of 12 tickets for the game in about
350. 11:4014 hours. And so, each time it went
351. 11:42through the full loop of the factory, it
352. 11:43took about an hour, which is pretty good
353. 11:46considering how big each ticket was, and
354. 11:48it went through the full process of
355. 11:49planning, building, reviewing, play
356. 11:51testing, and merging. And then going to
357. 11:53Open AI, it took twice the length and
358. 11:56didn't even quite finish. It did 11 out
359. 11:58of 12 tickets. And so, even between
360. 12:00frontier models, there's a pretty big
361. 12:01difference here. There's a reason why
362. 12:03I'm using Claude Opus 5.5 to drive a lot
363. 12:07of my agent coding workflows right now,
364. 12:09pretty much all of them. But overall,
365. 12:11the curve looks pretty solid between the
366. 12:13two, right? Like it wasn't super stuck
367. 12:14on anything, and the end result was
368. 12:16pretty good. But, then when we get to
369. 12:17Kimmy, look at this. There's a couple of
370. 12:20points where it stalled for hours and
371. 12:22hours, and only got through five out of
372. 12:2312 tickets in 44 hours, which is why I
373. 12:26didn't have things like the shop at the
374. 12:28end of the raid. It just got so stuck
375. 12:31iterating in all these little things
376. 12:32that came up. It was so frustrating
377. 12:34because it would constantly try to do
378. 12:35things that the rules told it not to,
379. 12:37and so that's why the reviews would
380. 12:39catch it, and then it would have to
381. 12:40iterate. It was just constantly not
382. 12:42obeying everything that the factory had
383. 12:44for it. Now, I got to say, maybe I could
384. 12:46change the factory to be a bit more
385. 12:48lenient, let some of these minor issues
386. 12:50in so that at least Kimmy would look
387. 12:51closer to Claude and GPT. But, I want to
388. 12:55keep my entire review process. I've
389. 12:57spent so much time refining this in my
390. 12:59AI software factory over time that I
391. 13:01don't want to have to conform to weaker
392. 13:03models just so that more things go
393. 13:05through. When I switch between frontier
394. 13:07models, they all seem to just get my
395. 13:09factory, get my codebase, and my rules,
396. 13:11but open models, it really does seem
397. 13:13like a true flaw where for these longer
398. 13:15running tasks, they just aren't able to
399. 13:17juggle and pay attention to everything
400. 13:19they need to.
401. 13:20I also have some more interesting
402. 13:22numbers to show you. At least I find
403. 13:23this fascinating. So, the total cost of
404. 13:25building everything here with all 12
405. 13:27tickets with Claude was $255.
406. 13:30Now, obviously I use my subscription for
407. 13:32everything here. This is just what it
408. 13:33would have been with API costs, which is
409. 13:36really the only thing that I can truly
410. 13:37quantify.
411. 13:38And so, that's pretty good overall. With
412. 13:41GPT, it was $635.
413. 13:43It didn't even fully finish. It ran for
414. 13:46twice the length, but it was actually
415. 13:47more than twice as expensive as well.
416. 13:50Pretty interesting. And then with Kimmy,
417. 13:52it was $180, which really isn't cheaper
418. 13:55than Claude because it didn't actually
419. 13:57finish everything. And so, that's the
420. 13:59thing. Just because a model is cheaper,
421. 14:01doesn't mean that it's actually cheaper
422. 14:03per task because it has to iterate more.
423. 14:05If it's failing more often, it can even
424. 14:07become more expensive. That's what I see
425. 14:09with these open weight models a lot of
426. 14:11times in my coding workflows. And we can
427. 14:13see this with the very next row. Claude
428. 14:15was $21 per ticket, GPT 63, and Kimmy
429. 14:18almost $36 for every single ticket. And
430. 14:21also, I had to intervene a lot less when
431. 14:23I had Claude running the factory. So, I
432. 14:25also have a part of the system here
433. 14:27where I basically have a manager over
434. 14:29the entire workflow that alerts me when
435. 14:30there are uncertainties that need my
436. 14:32decision. Or at least the manager would
437. 14:34decide for me. And so, with that, Claude
438. 14:37most of the time is able to go through
439. 14:38things very smoothly. OpenAI, generally
440. 14:41about once per ticket, it would bring
441. 14:42one thing to my attention. But Kimmy
442. 14:44needed help for a couple of different
443. 14:46instances every single ticket it was
444. 14:48creating. And that also was pretty
445. 14:49frustrating for me. Claude is just able
446. 14:51to be a lot more autonomous overall. It
447. 14:54has a lot more confidence in its
448. 14:56implementations, which seems warranted
449. 14:58as well. Right? Like we don't want
450. 15:00unwarranted confidence where it's just
451. 15:01making a ton of mistakes, but Claude is
452. 15:03solid and it knows it is. And then the
453. 15:05last interesting thing here is I had a
454. 15:07separate agent do a blind code review of
455. 15:09the whole code base. Claude got an eight
456. 15:11out of 10, and GPT a seven out of 10,
457. 15:13and Kimmy a six out of 10. And a six out
458. 15:15of 10 is okay. I mean, Kimmy got to
459. 15:18something decent eventually. It was just
460. 15:19pretty painful to get there.
461. 15:21Okay, so at this point we have covered
462. 15:23the problem in a lot of detail. We know
463. 15:25why we can't just hand our
464. 15:27longer-running agentic coding workflows
465. 15:28over to a cheaper open model. Even if we
466. 15:31let it iterate a lot, it ends up getting
467. 15:33even more expensive than just having
468. 15:35Claude handle our software factory, our
469. 15:37coding workflow, whatever. And it's
470. 15:38going to be a lot more reliable. So,
471. 15:40with that, let's talk about the
472. 15:41solution, or at least what I'm doing
473. 15:43right now. The eventual goal, at least
474. 15:45for me, is to get to the point where I
475. 15:47can use open models for everything. And
476. 15:49I firmly believe that in the future
477. 15:51we're going to get there. We're not
478. 15:52going to be locked into using frontier
479. 15:54models for anything if we don't want.
480. 15:56It's just like we can see we're not
481. 15:58there yet. But that also doesn't mean
482. 16:01that I have to ditch the idea of using
483. 16:03open models altogether in my agentic
484. 16:05coding workflows. Something I've been
485. 16:07experimenting with a lot this year is
486. 16:09mixing models and providers, using the
487. 16:12right model for the right task depending
488. 16:14on how much it costs and how good it is.
489. 16:17And so, for example, using Opus for the
490. 16:20planning and reviewing in my larger
491. 16:21agentic coding workflow, and then using
492. 16:23Kimmy as the workhorse. And these kinds
493. 16:26of combinations have worked really well
494. 16:28for me. Now, I'll have to put out more
495. 16:30content where I really show you how to
496. 16:31set up this harness. And there is a
497. 16:33video I'll link to right here on my
498. 16:34channel where I get into these numbers
499. 16:35in a lot more detail. But at a high
500. 16:37level right now, I just want to show you
501. 16:39what it looks like to mix models and how
502. 16:41we really don't have to sacrifice the
503. 16:42reliability of the code to get a lot of
504. 16:45cost and time savings. And so, what I
505. 16:47did with this test is I had a control
506. 16:50group where I had a lot of agentic
507. 16:52coding workflows that I ran through Opus
508. 16:54for everything. It did all of the
509. 16:55planning, implementing, and reviewing.
510. 16:58And then for my other test, I had Opus
511. 17:00do the reviewing and the planning, but
512. 17:03Kimmy was the workhorse that did the
513. 17:04implementation, which is where a lot of
514. 17:06my tokens go. And what I found is the
515. 17:08results were pretty equivalent between
516. 17:10these two groups, which is incredible
517. 17:11cuz we look at the cost here. For simple
518. 17:14tasks, it was really cheap between but
519. 17:16for more complicated tasks, it was more
520. 17:18than twice the cost using Opus for
521. 17:20literally everything, even though the
522. 17:22results were pretty equivalent both for
523. 17:24simple and complex tickets that I had it
524. 17:26implement. This is crazy. It all comes
525. 17:28down to the fact that if we use the more
526. 17:30powerful model for the planning, that's
527. 17:32where we need to load in all the context
528. 17:34and follow all the rules. So, if we
529. 17:35scope that down into a plan document,
530. 17:38our weaker but still pretty capable
531. 17:39model is able to execute on that very
532. 17:41well. This is the solution right now. We
533. 17:44can still use open weight models, just
534. 17:46not for everything in our larger
535. 17:48workflows. So, that's my main
536. 17:49recommendation right now. If you want to
537. 17:51save on tokens, make your workflows
538. 17:53faster, experiment with open models, use
539. 17:55them for the workhorse that always
540. 17:57writes the code, but then you just
541. 17:59sandwich that with the planning and
542. 18:00reviewing that uses a frontier model.
543. 18:03And exactly how you set that up is going
544. 18:05to be a lot more content I put out soon
545. 18:07where I cover that. So, also let me know
546. 18:08in the comments what part of this you're
547. 18:10interested in. Like, do you want me to
548. 18:11show more data? Do you want me to show
549. 18:12you how to really set up these systems?
550. 18:14I'd be curious what you're looking for.
551. 18:16And so, I hope that you appreciate this
552. 18:18overall. The problem with open models,
553. 18:20getting into the solution, a lot more I
554. 18:21want to do on this soon. If you
555. 18:23appreciated this and you're looking
556. 18:24forward to more videos on agentic
557. 18:26engineering, open weight models, I would
558. 18:28really appreciate a like and a
559. 18:29subscribe. And with that, I will see you
560. 18:31in the next video.