I Tested DeepSeek Agent Harness Against Codex and Claude Code — Transcript

Full transcript

1. 0:00Today we're talking about the new Deep
2. 0:01Seek agent harness, which is one of the
3. 0:02most interesting AI tools I've seen in a
4. 0:04long time. What's interesting about this
5. 0:05agent is everything is a plug-in, which
6. 0:07means you can extend and [music]
7. 0:08customize basically everything,
8. 0:10including all of the UI as well. So, in
9. 0:12this video, I'll show you everything you
10. 0:13need to know about setting up this
11. 0:14[music] harness, how to install it, how
12. 0:16to use it, how to build your own
13. 0:17extensions. I'll show you an extension
14. 0:18that I built myself, and we'll also talk
15. 0:21about the two most [music] important
16. 0:22topics when it comes to this agent. If
17. 0:24you're new to the channel, my name's
18. 0:25Owen. I've been a professional software
19. 0:26engineer, and I've been leading
20. 0:27engineering teams [music] for the last
21. 0:2820 years. Everything that we talk about
22. 0:30in the video, I'll link for free in the
23. 0:32description below. So, let's [music] get
24. 0:33into it. What I want to show you now is
25. 0:35probably my favorite feature of this
26. 0:36whole harness. If you look down here,
27. 0:38I've built my own custom extension. And
28. 0:40so, this is a completely new UI for the
29. 0:44harness. [music] So, when you click on
30. 0:45this, you see a custom canband board
31. 0:47that shows all of the work that I
32. 0:49currently have running inside this agent
33. 0:51harness. [music] You can see here, this
34. 0:52is the task I'm currently running,
35. 0:54looking for bugs within the agent OS
36. 0:56code base. I've got, you know, 120 steps
37. 0:59have been run, 20% of the context
38. 1:01window. You can see here things like sub
39. 1:03agent calls. You can see here the work
40. 1:05that's archived. You can toggle things
41. 1:07on or off. You can group by project or
42. 1:10not. You know, you can see work that's
43. 1:12done, what's active. You've got a bunch
44. 1:14of statistics at the top showing you how
45. 1:16many tokens you've used, how many agents
46. 1:18are currently running, how many things
47. 1:19are waiting on you. This to me is
48. 1:22incredibly powerful because one of the
49. 1:23challenges I have with agents in general
50. 1:25and the way I work is I'm working on so
51. 1:27many things in parallel right now. I'm
52. 1:29working on maybe five or six projects
53. 1:31every day at the same time with, you
54. 1:33know, 50 60 tasks I'm working on every
55. 1:35single day. And keeping track of all of
56. 1:38that work is one of the biggest
57. 1:39bottlenecks that most of us have right
58. 1:40now. So being able to build a custom UI
59. 1:43like this, I was absolutely blown away
60. 1:45by how awesome this is. And this maybe
61. 1:47took me 30 minutes to put together. So
62. 1:49you can imagine what would be possible
63. 1:51if you just spent a bit more time really
64. 1:53polishing and if you get really creative
65. 1:55trying to do something like this within
66. 1:57claude code or codec certainly would be
67. 1:58impossible. There's no way to do
68. 2:00something like this without building
69. 2:02like a different application that would
70. 2:04hook in. And same thing with the PI
71. 2:06agent as well. Although you can extend
72. 2:07the PI agent with extensions, what you
73. 2:10can actually do there is relatively
74. 2:11limited. Whereas with this, everything
75. 2:13is customizable. You can build basically
76. 2:15anything on top of the agent harness.
77. 2:18Okay. Okay, so if you do want to install
78. 2:19this harness, it's pretty easy to set
79. 2:21up. The first thing you want to do is
80. 2:22just copy this one command over here.
81. 2:24Then I'm just going to jump back into
82. 2:26the terminal and then we're going to
83. 2:28paste this in. Sorry, I'm just jumping
84. 2:30about Windows. So we're going to paste
85. 2:31this into the terminal. Then this will
86. 2:33start up the agent harness for us. You
87. 2:35can see this is now running on port 380.
88. 2:38So we're just going to go to this in our
89. 2:39web browser. So when you first log into
90. 2:42the harness, you'll see an interface
91. 2:44that looks like this. This is running in
92. 2:45a web browser rather than an application
93. 2:47or the terminal. It looks very familiar
94. 2:50if you've used codecs before. It's the
95. 2:52same kind of interface. I recently maxed
96. 2:54out my codec subscription and I had to
97. 2:56go back to using clawed code in the
98. 2:57terminal. And honestly, it was a
99. 2:58horrible experience. I find because I'm
100. 3:01working on so many different projects
101. 3:02and so many things at once. I just find
102. 3:04terminals really difficult to use. Even
103. 3:06though I've been a developer a long
104. 3:07time, I use terminals every day. I'm
105. 3:09using multiplexes. I just don't find
106. 3:11them a good environment to work in when
107. 3:13it comes to Agentic coding. [music] So,
108. 3:14I really do like the interface here. You
109. 3:16can obviously have multiple different
110. 3:17projects running. Many many sessions and
111. 3:20it's very easy to navigate around this
112. 3:22user interface. You can obviously toggle
113. 3:24the various settings here for
114. 3:26permissions. So, you can have workspace
115. 3:27rights where you can only write within
116. 3:29your current folder or you can have full
117. 3:31access where the agent can access
118. 3:32anything outside of the current folder.
119. 3:35You also get these different modes. You
120. 3:37can use the standard mode, which is what
121. 3:39most people will be using. There's also
122. 3:41a minimal mode, which has very few
123. 3:43tools. You only get the bash tool and
124. 3:45the string replace editor tool. The
125. 3:47[music] PTC mode allows you to have the
126. 3:50agent write code essentially that will
127. 3:53allow it to customize its workflow. This
128. 3:55stands for programmatic tool calling. I
129. 3:58haven't experimented much with this
130. 3:59myself to be honest. The creator mode is
131. 4:01where you can interactively build
132. 4:03plugins and extensions. This is really,
133. 4:05really useful. And then down here I have
134. 4:07another mode enabled which allows me to
135. 4:09dispatch to claude code and codeex via
136. 4:12tool calls. So I'll just quickly show
137. 4:14you how that works. So if I click on
138. 4:15this, what I can do is just ping cla
139. 4:19code directly. So if you did want to use
140. 4:20your subscription [music] for claude
141. 4:22code or codeex, but you want this to
142. 4:24drive your coding, you can do something
143. 4:26like this. We can say, can you ping
144. 4:30claude code and codeex? They're running
145. 4:33on my local machine. and we're just
146. 4:34going to basically call them via a tool
147. 4:35call which allows you to dispatch work
148. 4:38to those tools. So a really interesting
149. 4:40concept here is you can use this deep
150. 4:41sea caris as a coordinator for multiple
151. 4:44different coding agents if you wanted to
152. 4:46explore that pattern. It's a pattern I
153. 4:48actually use myself quite a lot in my
154. 4:49own work. When you go into the settings
155. 4:51you can see you have a bunch of
156. 4:52different options. You have the modes.
157. 4:54You can obviously toggle the different
158. 4:55themes. You're able to choose different
159. 4:57model providers. So when you first sign
160. 4:59in, you need a DeepSseek API key, but
161. 5:02you can actually change the provider. So
162. 5:04you can click in here and you can see
163. 5:05you've got all of these different model
164. 5:07providers available. The one thing I
165. 5:09would say that is a downside in my
166. 5:10opinion is these are all API keybased
167. 5:13providers and I much rather use
168. 5:15subscriptions because it's much easier
169. 5:18to manage your costs and you just get
170. 5:20way more value for your money with a
171. 5:21subscription. But you do get access to
172. 5:23all of these different models. You can
173. 5:25integrate with obviously GitHub copilot
174. 5:26or deepseeek itself. You can use things
175. 5:29like open code or open router if you
176. 5:31want to try different models. If you're
177. 5:33running local AI for example, you can
178. 5:35obviously configure all of that as well
179. 5:36which is good. And then the interesting
180. 5:39area is the plug-in section. So you can
181. 5:41see here we have some plug-in
182. 5:42configuration. So you can tune
183. 5:44everything about this agent. You can
184. 5:45disable tools. Even the agent loop
185. 5:48itself has customization which is pretty
186. 5:50interesting. So you can manage things
187. 5:52like the number of parallel tool calls
188. 5:54the agent is able to run. All of this is
189. 5:56configurable and it's not really
190. 5:58configurable in any other harness. If
191. 6:00you click on the plug-in list, you can
192. 6:01see we have a whole bunch of different
193. 6:03plugins available. I'm not going to go
194. 6:04through all of these, but the general
195. 6:06idea is that each of these plugins can
196. 6:08be enabled or disabled. So things like
197. 6:10the user question plugin. Everything
198. 6:12about the agent is extensible and
199. 6:14customizable. So you can basically turn
200. 6:15off and turn on bits of the harness as
201. 6:17you want. Another really interesting
202. 6:19thing about this harness is you can
203. 6:21build your own agent presets. So imagine
204. 6:23you only want a certain subset of tools
205. 6:25enabled. So this is really interesting.
206. 6:27It's just very very customizable in
207. 6:28general. What I'm showing you now is an
208. 6:30application I'm building called agent
209. 6:31OS. This is a way of delegating and
210. 6:33tracking tasks for yourself but also for
211. 6:35agents as well. We can look at a typical
212. 6:37cambban board. This is the YouTube video
213. 6:39we're currently working on. You can see
214. 6:41here I have a bunch of subtasks I need
215. 6:43to work through like recording the video
216. 6:45etc. But you can dispatch these tasks to
217. 6:48individual agents. So for example, you
218. 6:50can either work on this yourself or
219. 6:51queue it up for an agent to work on. And
220. 6:54you can store knowledge bases as well.
221. 6:56So you can install all your context. You
222. 6:59have runners, so you can basically run
223. 7:00agents in the cloud. You can run them on
224. 7:03your virtual machine as well and
225. 7:04basically have different agents work on
226. 7:06different tasks. You can track the runs
227. 7:08that your agents are doing. Agents can
228. 7:11also communicate back to you via an
229. 7:12inbox. I'll talk about this project in
230. 7:14another video maybe. So, what we want to
231. 7:17do is go and find some bugs inside this
232. 7:18codebase with DeepSeek. All right. So,
233. 7:21what I've done is I've sent a pretty
234. 7:22terrible prompt, but I've just said
235. 7:24review the code and find any bugs we
236. 7:26need to fix. Then, we're going to use an
237. 7:28MCP server to track those bugs. And
238. 7:30then, we're going to delegate those bugs
239. 7:32off to the agent harness. And I'll show
240. 7:33you some interesting tricks you can use
241. 7:35here as well. I'm using the Deep Seek V4
242. 7:38Flash model. I like this model a lot.
243. 7:40I've been using Frontier models for most
244. 7:42of my work. Codeex is used for the
245. 7:44majority of my work personally. The
246. 7:46thing about those models is they can be
247. 7:47really really slow. So tasks with Fable
248. 7:50or you know OpenAI codeex can just take
249. 7:52a really long time to complete. I found
250. 7:54this is a very fast model. It's
251. 7:56relatively cheap and pretty capable as
252. 7:58well. You can obviously swap this out
253. 8:00for a different model. You could use V4
254. 8:02Pro if you want, but I just really like
255. 8:03the Flash model. I think it's great. All
256. 8:05right, so you can see here we found a
257. 8:06bunch of bugs in the code. The agent has
258. 8:09now created four tasks. So we created
259. 8:11four bug tasks and this is an
260. 8:13improvement task. So it's labeled
261. 8:14everything correctly. And then if we
262. 8:15click into one of these tasks, you can
263. 8:17see we have a good description, why it
264. 8:19matters, where the code is, and what the
265. 8:21fix is as well. So pretty well scoped
266. 8:24out ticket. And then we're going to go
267. 8:25into DeepS and try and just close out
268. 8:27these bugs. What I'm going to use is the
269. 8:29coordinator worker pattern here to close
270. 8:32out all of these bugs in parallel at the
271. 8:34same time. Okay. Okay, so what's kind of
272. 8:36funny is that even though I asked the
273. 8:37agent to spawn these sessions in the
274. 8:40sidebar as new sessions, the agent
275. 8:41didn't think it had the tools available.
276. 8:43I had to argue with it quite a lot and
277. 8:45then pointed out that it does have RPC
278. 8:47call abilities to spawn these sessions.
279. 8:50So the agent has the ability to do a
280. 8:52session new session.create
281. 8:55session.prompt, etc. So it has all of
282. 8:57the all of the capabilities to do this,
283. 8:59but the agent wasn't aware of them. Now
284. 9:01you can see there it's just spawned
285. 9:03these. I think you probably saw that in
286. 9:04the sidebar. You can see here now we've
287. 9:06spawned these four sessions in the
288. 9:09sidebar. They're labeled correctly with
289. 9:11the right, you know, the right ticket
290. 9:13numbers and and things like that. So,
291. 9:15what you can see now is within this
292. 9:16harness, all of the different agent
293. 9:17sessions that are running. So, we have
294. 9:19four bug fixes happening in parallel.
295. 9:21And then this agent session is
296. 9:23monitoring these other four, waiting for
297. 9:26them to be complete. This task has been
298. 9:28running a long time. It's got 40 million
299. 9:30tokens, but these are individual
300. 9:32sessions just fixing bugs in separate
301. 9:34work trees. So these sessions are
302. 9:36completely isolated. And you can see now
303. 9:38that this session is spawning a sub
304. 9:40agent. You can see these two dots
305. 9:42indicate that this is now probably in a
306. 9:43review phase, I would guess. So if I
307. 9:45click into one of these, yeah, you can
308. 9:47see here now both sub aents are running
309. 9:48in the background. We're doing a
310. 9:50readonly sanity check. So we're just
311. 9:52basically coordinating all of the work
312. 9:54here. And I just really like this
313. 9:56pattern. I find it to be really
314. 9:57effective to orchestrate a larger amount
315. 10:00of work. One other really interesting
316. 10:01thing about this harness is you get this
317. 10:03trajectory view which allows you to see
318. 10:06all of the different tool calls. If
319. 10:08you've ever built AI systems and you've
320. 10:09used tracing systems like Langfuse, etc.
321. 10:12Very very similar. You can basically
322. 10:13just see everything that the agent is
323. 10:15doing. You can search through the you
324. 10:17know the audit history. You just get
325. 10:19more visibility into what your agents
326. 10:21are doing. I'm not sure how much I would
327. 10:22need this myself or use it, but it is
328. 10:24kind of useful if you're debugging why
329. 10:26an agent took so long, if you're trying
330. 10:28to improve your process. Having access
331. 10:30to this to kind of inspect what your
332. 10:32agents are doing would be quite useful.
333. 10:34So, one thing to be aware of if you're
334. 10:36using any community plugins is to be a
335. 10:39bit careful about security. Anytime you
336. 10:41install code from the internet, there's
337. 10:42obviously a danger that that could be
338. 10:44malicious code. But this is especially
339. 10:46dangerous when it comes to plugins like
340. 10:48this because these plugins will have
341. 10:50access to your agents which have access
342. 10:52to your credentials, your whole computer
343. 10:54essentially. So I would be very careful
344. 10:56about installing third party plugins
345. 10:58personally when it comes to this agent
346. 10:59harness. So just to give you some
347. 11:01context on the pricing, this is the
348. 11:03usage that I spent or I've racked up now
349. 11:05over the past 2 days. So, I've been
350. 11:07experimenting with this harness a little
351. 11:09bit myself, trying to get familiar with
352. 11:10it and understand how it works. But even
353. 11:12just using this for like 20% of my
354. 11:15coding in the last 2 days, I've already
355. 11:17spent $2. So, if I was using this for
356. 11:19all of my coding, I I would estimate it
357. 11:21would be six or maybe $6 or $7 that I
358. 11:24would have spent in API tokens already
359. 11:26in 2 days. And so, this is why I don't
360. 11:28think API keybased billing makes any
361. 11:30sense for developers. I think you should
362. 11:32be using a subscription. and it will
363. 11:34just save you more money and you know
364. 11:35how much money you're spending each
365. 11:37month. Open AAI give you crazy limits on
366. 11:40your subscription. I find it's the most
367. 11:41generous around. I would much rather use
368. 11:43the OpenAI subscription and pay for API
369. 11:47based billing where I don't really know
370. 11:48what I'm going to get charged each
371. 11:49month. I could easily rack up a massive
372. 11:51bill and yeah, I I just don't like API
373. 11:55key based billing in general. You could
374. 11:57argue that you could use a local model.
375. 11:59So you can run Deep Seek Flash locally,
376. 12:02but in my opinion, you also need to
377. 12:03spend thousands of dollars on really
378. 12:05good hardware to make local AI work
379. 12:07right now. And it just doesn't make
380. 12:09sense financially to me personally to
381. 12:11spend thousands of dollars on expensive
382. 12:13hardware. And the models still don't run
383. 12:15as well as they would on, you know,
384. 12:17OpenAI's hardware, etc. So, you're
385. 12:19essentially paying more money for an
386. 12:21inferior model, which just doesn't make
387. 12:22any sense to me right now. So, in terms
388. 12:24of cost, I would I would really stick
389. 12:26with my subscription for codecs. I think
390. 12:29that's the most costefficient way to
391. 12:31build with AI today, but I could see me
392. 12:34using this in the future. I could see me
393. 12:35using the Deep Seek Flash model. I did
394. 12:38really like it. I thought the model was
395. 12:40very fast, and as these models get
396. 12:42better, as we get things like local AI
397. 12:44that's actually usable, then I could see
398. 12:46me using this way more often. So, I
399. 12:49really like the harness and I would come
400. 12:51back to it mainly because of this
401. 12:53ability to build a custom UI on top. I I
402. 12:57just really like this ability to have
403. 12:58this. You know, this plug-in took me 30
404. 13:01minutes, but I just really like it
405. 13:02already. So, yeah, I was pretty
406. 13:04impressed by this harness. So, I hope
407. 13:06you found the video valuable and hope
408. 13:08you enjoyed it. If you did, please
409. 13:09remember to give a like and subscribe.
410. 13:11It massively helps out the channel.
411. 13:13Check out the links in the description
412. 13:14if you want to go deeper or if you want
413. 13:16to spend more time going into agentic
414. 13:18coding, building AI systems, or
415. 13:20improving or growing your career or your
416. 13:22business with AI. Thanks again for
417. 13:24watching. I really appreciate it and
418. 13:25I'll see you in the next one. Take a