YouTube2Text

Ship your first Managed Agent — Transcript

by Claude · 7,185 words · 1,107 segments · language en · Watch on YouTube

Full transcript

  1. 0:15[music]
  2. 0:19>> All right. Hello everyone. It's great to
  3. 0:21see you all here today for our session
  4. 0:23on shipping your first manage agent.
  5. 0:25Let's go ahead and get started. My name
  6. 0:27is Isabella He. I'm a member of
  7. 0:29technical staff at Anthropic on the
  8. 0:31Applied AI team.
  9. 0:32The Applied AI team at Anthropic sits at
  10. 0:34the intersection of products, research,
  11. 0:36and our customers, which means that I
  12. 0:38get to contribute internally to products
  13. 0:39at Anthropic like Claude code and our
  14. 0:41Claude harnesses, as well as work
  15. 0:43externally with our customers that are
  16. 0:45building on top of Claude and on top of
  17. 0:47our harnesses.
  18. 0:48So, my goal today is to get you all
  19. 0:50hands-on with actually building on top
  20. 0:52of manage agents, understanding how the
  21. 0:54harness works under the hood, and
  22. 0:56getting you ready to actually ship your
  23. 0:57first incident response management.
  24. 1:00So, the quick overview of today's
  25. 1:01agenda. We're going to cover first a
  26. 1:03quick refresher of Claude manage agents.
  27. 1:06I want to talk you through a little bit
  28. 1:07about how this harness works under the
  29. 1:08hood and what makes it so special. Our
  30. 1:11team put a lot of thought into the
  31. 1:12architectural design of Claude manage
  32. 1:14agents to make sure that it runs ready
  33. 1:16and reliably for production-ready
  34. 1:18agents. So, I want to talk you through a
  35. 1:19little bit of how that works so that
  36. 1:21then when we transition to the second
  37. 1:23portion here, which is the hands-on
  38. 1:25workshop, you'll actually understand
  39. 1:27what each of the primitives you're
  40. 1:28building actually mean for your agents
  41. 1:30under the hood.
  42. 1:32So, for the majority of today's session,
  43. 1:33I want you all to actually have your
  44. 1:35laptops open, building alongside me,
  45. 1:37actually working inside of a repository,
  46. 1:39and getting you ready to actually spin
  47. 1:41up a working incident response agent.
  48. 1:44Lastly, we'll talk a little bit about
  49. 1:45beyond the basics. Today's session is
  50. 1:47the first session of a couple of other
  51. 1:49ones that will build on top of this on
  52. 1:50Claude manage agents.
  53. 1:52Specifically, right after this one, I
  54. 1:53think there's another session on
  55. 1:54dreaming, which is one of my favorite
  56. 1:56new features with Claude manage agents
  57. 1:58for self-improving agents and memory
  58. 2:00built into the harness. So, encourage
  59. 2:02everyone to dive in a little bit deeper
  60. 2:04into what else is in the box after we
  61. 2:06set you all up for success today with a
  62. 2:08quick introduction.
  63. 2:10So, let's first touch a little bit about
  64. 2:11how we got here with Claude manage
  65. 2:13agents.
  66. 2:14When we first released the very first
  67. 2:16Claude back in 2023, we released a
  68. 2:19messages API alongside access to Claude.
  69. 2:22This provided raw model access to all
  70. 2:24Claude models. This became the very
  71. 2:27first way that people could
  72. 2:28programmatically build on top of Claude
  73. 2:30and essentially gave a way for people to
  74. 2:32access tokens in and tokens out via our
  75. 2:35Claude models.
  76. 2:36This also meant that for everyone
  77. 2:38building on top of Claude models, they
  78. 2:40had to implement all the various
  79. 2:41primitives themselves. Things like
  80. 2:43context management, the actual agent
  81. 2:45loop, compaction, etc. All the
  82. 2:47primitives that come alongside making
  83. 2:49the agent work.
  84. 2:50When models were less intelligent back
  85. 2:52in the early days of let's say 2023,
  86. 2:55some of these primitives were much
  87. 2:56simpler because agents could simply do
  88. 2:57less. But, as we evolved into now with
  89. 3:00higher model intelligence and as agents
  90. 3:02are able to take on more complex tasks
  91. 3:04and actually take actions within
  92. 3:06environments and come to actually do
  93. 3:08entire tasks for humans,
  94. 3:10the primitives that come alongside
  95. 3:12context management and managing an
  96. 3:14agent's ability to execute API calls and
  97. 3:16tool calls becomes much more complex.
  98. 3:19So, that's when we moved to the agent
  99. 3:20SDK, which became a harness that allows
  100. 3:23you to programmatically call Claude
  101. 3:25code, one of our agents at Anthropic.
  102. 3:28So, Claude code is something that an
  103. 3:30agent has access to a computer and takes
  104. 3:32actions within file system. So, the
  105. 3:34agent SDK became a way for you to make
  106. 3:36Claude much more powerful by leveraging
  107. 3:38the power of Claude code within a
  108. 3:40harness.
  109. 3:41The main thing here though is that with
  110. 3:43the agent SDK,
  111. 3:45developers still had to manage hosting
  112. 3:46and scaling on their own and making sure
  113. 3:49that the agent SDK would be safe to run
  114. 3:50within their containers.
  115. 3:53That's only then evolved into Claude
  116. 3:54managed agents, which is the first
  117. 3:57harness to be able to handle scaling and
  118. 3:59production ready components for you by
  119. 4:01Anthropic,
  120. 4:02providing things like a purpose-built
  121. 4:04harness, sandboxing, observability, tool
  122. 4:06runtime, all within a managed
  123. 4:08infrastructure system. This means that
  124. 4:10developers can focus on task and agent
  125. 4:12configuration, custom tool logic, the
  126. 4:15things that actually matter for bringing
  127. 4:17domain expertise and customizability to
  128. 4:19your agents, where you're handing off
  129. 4:21the rest of all the primitives and core
  130. 4:24compute and primitives of essentially
  131. 4:26managing the basics of agent running to
  132. 4:28Anthropic.
  133. 4:30So, that brings me to managed agents as
  134. 4:33the fastest way to build production
  135. 4:35ready agents on Claude.
  136. 4:36We've seen people build 10 to 15 times
  137. 4:38faster to production with Claude managed
  138. 4:40agents by leveraging our purpose-built
  139. 4:43harness.
  140. 4:44Part of the reason why we built Claude
  141. 4:45managed agents is because is because
  142. 4:47harnesses should evolve alongside your
  143. 4:49agents.
  144. 4:50For example, back when we were building
  145. 4:52ourselves on top of models like Sonnet
  146. 4:544.5, we noticed that Sonnet 4.5 emitted
  147. 4:57a particular behavior called context
  148. 4:59anxiety. This meant that with Sonnet
  149. 5:014.5, Claude started wrapping up tasks
  150. 5:03early even when it still had room to
  151. 5:05spare in its context window.
  152. 5:07To manage that in our harness, we then
  153. 5:09added some mitigations to combat against
  154. 5:12this early stopping behavior.
  155. 5:14But, when Opus 4.5 then came out, we
  156. 5:16actually saw this behavior go away,
  157. 5:18making all that work we had done inside
  158. 5:20of the harness essentially obsolete
  159. 5:21because Claude had evolved beyond that
  160. 5:23behavior that we had built into the
  161. 5:25harness to manage.
  162. 5:26So, the takeaway there is that it's a
  163. 5:28lot of work to maintain harnesses and
  164. 5:30make sure that they actually evolve
  165. 5:31alongside your agents, which is why with
  166. 5:33Claude managed agents, we want to make
  167. 5:35it really easy for Claude and Anthropic
  168. 5:37to handle all the complexities that come
  169. 5:39with compaction, caching, things like
  170. 5:41context anxiety, all these various
  171. 5:44primitives that come with actually
  172. 5:45making agent production ready and
  173. 5:46getting the most out of Claude. So
  174. 5:48again, you can focus on the tasks,
  175. 5:50tools, and things that actually matter
  176. 5:51for building agents on Claude.
  177. 5:55So three primary resources go into
  178. 5:57building on Claude managed agents.
  179. 6:00First is the agent's endpoint, which is
  180. 6:02the persona and capabilities. This is
  181. 6:04the core system prompt that powers your
  182. 6:06agent. Essentially here, you're defining
  183. 6:08the model, the MCP servers, the skills,
  184. 6:11the various components that your agent
  185. 6:12can actually leverage when it's able to
  186. 6:14run in that agent loop.
  187. 6:16The next is the environments.
  188. 6:18You can think of this as the hands of
  189. 6:20the agent, where the previous one is the
  190. 6:22brain of the agent where the agent is
  191. 6:23thinking through what to execute, and
  192. 6:25then it's using an environment to
  193. 6:27actually have a space and a container to
  194. 6:29actually take action on your behalf.
  195. 6:32Sessions are next the way to tie
  196. 6:34together agents and environments.
  197. 6:36A single session has a spun up on an
  198. 6:39agent instance within an environment. So
  199. 6:41you can connect the two together and
  200. 6:43actually stream events back to your user
  201. 6:45and start to take action on behalf of
  202. 6:47your humans as part of a Claude powered
  203. 6:49agent.
  204. 6:51A key thing here, as I alluded to
  205. 6:53briefly before, Claude managed agent has
  206. 6:56the agent loop run server side.
  207. 6:58This means that a lot of the
  208. 6:59complexities that come with managing
  209. 7:01hosting and scaling are abstracted away.
  210. 7:03And when you close your laptop or you
  211. 7:05hit hard refresh on your agent that
  212. 7:07you're building on Claude managed
  213. 7:08agents, everything is maintained and you
  214. 7:10don't have to worry about durability,
  215. 7:12reliability, all these various aspects
  216. 7:14that usually come to bite you when
  217. 7:16you're trying to turn your agent from a
  218. 7:17prototype into production.
  219. 7:22And lastly here, before we dive into the
  220. 7:24hands-on portion, is I want to talk you
  221. 7:26through a key design decision that went
  222. 7:28into Claude managed agents.
  223. 7:32Previously, with a lot of agent
  224. 7:33harnesses, we saw the agent loop coupled
  225. 7:35tightly with tool execution.
  226. 7:38This design pattern made sense and still
  227. 7:39makes sense for some agents because you
  228. 7:41want to give the agent powerful
  229. 7:43abilities to actually take action within
  230. 7:45the environment. For instance, with
  231. 7:46Cloud Code, we want the agent to be able
  232. 7:48to access various files on your
  233. 7:50computer, take action within a file
  234. 7:51system, and therefore it makes sense for
  235. 7:53the agent to have access to all those
  236. 7:55tools spun up on every container.
  237. 7:58But, we also realized there are some
  238. 7:59constraints for this, especially with
  239. 8:01some agents where you essentially want
  240. 8:03to be able to decouple the hands from
  241. 8:06the brains of the agents.
  242. 8:08For instance,
  243. 8:09credentials and uh
  244. 8:11credentials and security became a huge
  245. 8:13concern.
  246. 8:14With the ability to have the agent
  247. 8:16access your file system, you can
  248. 8:18actually add very distinct sandboxing by
  249. 8:20decoupling these two components, where
  250. 8:22the agent is no longer able to access
  251. 8:24the actual credentials without
  252. 8:25encryption by decoupling the hands from
  253. 8:27the sandbox of the agent.
  254. 8:30The other aspect here is actually you
  255. 8:32can see huge benefits by doing these
  256. 8:34decoupling on things like time to first
  257. 8:36token and latency.
  258. 8:38Previously, with the agent loop into
  259. 8:40execution in the same box, you had to
  260. 8:42spin up containers for every single
  261. 8:43session that you're spinning up in the
  262. 8:44agent, which contributed to additional
  263. 8:47latency from a time to first time to
  264. 8:48first token perspective.
  265. 8:50But, with this now decoupled, our teams
  266. 8:52actually saw reductions in time to first
  267. 8:53token along the lines of over 90%
  268. 8:57reduction in TTFT for our P95 metrics on
  269. 9:01latency.
  270. 9:02So, here you can start to see the power
  271. 9:03of this design decision coming through
  272. 9:05from the perspective of safety,
  273. 9:07reliability, latency, and everything
  274. 9:10else that you care about when it comes
  275. 9:11to building production-ready agents.
  276. 9:15All right, so now it's time for the
  277. 9:17exciting part of today's session, which
  278. 9:19is where I want you all to open up your
  279. 9:20laptops and go to this URL here to
  280. 9:23actually clone a repository, and let's
  281. 9:26start to actually feel the magic of
  282. 9:27everything that I just talked through.
  283. 9:29So, I'm going to give everyone a second
  284. 9:31to just go over to that URL there
  285. 9:33and just spin up the repository that we
  286. 9:35have ready for you.
  287. 9:37All right, so here are some additional
  288. 9:39commands that I want you all to run to
  289. 9:40make sure this is all set up on your
  290. 9:42computers.
  291. 9:43So, the first step many of you might
  292. 9:44have done already, but just take that
  293. 9:46repository, hit the URL, get clone it,
  294. 9:50and then I want you to CD into the
  295. 9:52specific repository for the session,
  296. 9:54which is ship your first manage agent.
  297. 9:56And then, if you're on Mac, you'll see
  298. 9:57those two commands on the side, the
  299. 9:59Python and the source. Um, there's a
  300. 10:01command there for Windows as well.
  301. 10:03And you'll just do the rest there where
  302. 10:04you want to install the requirements,
  303. 10:06copy over the environment key into your
  304. 10:08.env file. Um, here you'll put in the
  305. 10:10Anthropic API key that hopefully all of
  306. 10:12you also received from the QR code for
  307. 10:14free credits earlier. And lastly, we'll
  308. 10:16just run the app.
  309. 10:17All right, let's go ahead and dive in,
  310. 10:19but as I mentioned before, let me just
  311. 10:21show everyone where these instructions
  312. 10:23are. If you go into the repository in
  313. 10:25the link and then go to ship your first
  314. 10:27manage agents, you scroll down on the
  315. 10:28read me, you'll see all the setup
  316. 10:30instructions here. So, feel free to do
  317. 10:32this, um, as we go along or even in your
  318. 10:34own time later today and continue
  319. 10:36playing around with it, but as I
  320. 10:37mentioned before, everything will be
  321. 10:39also shown on the screen to follow along
  322. 10:41with. So, do not worry if you did not
  323. 10:42have time to fully get it set up on your
  324. 10:44laptop.
  325. 10:46Without further ado, let's go ahead and
  326. 10:47dive in. So, once you run streamlit run
  327. 10:50app.py, you should be able to see a URL
  328. 10:53that looks like this and a page that
  329. 10:54looks like this.
  330. 10:56What we're doing here is we're going to
  331. 10:58be simulating an agent, um, interaction
  332. 11:00here where we have an incident that's
  333. 11:02going to come up. A lot of you who might
  334. 11:04be software engineers in the room will
  335. 11:06be intimately familiar with the pain
  336. 11:08that comes alongside incident response.
  337. 11:10If you are software engineer, you might
  338. 11:12be woken up at, let's say, 3:00 a.m. in
  339. 11:14the morning, 2:00 a.m. in the morning
  340. 11:15when you're out around on on vacation as
  341. 11:17you're on call, and this is usually a
  342. 11:19very painful portion of a software
  343. 11:21engineer's life, uh, because when you're
  344. 11:23on call, it means that if a server goes
  345. 11:24down or a service goes down, you have to
  346. 11:26be immediately the one there to respond
  347. 11:28and tackle the incident. Usually for a
  348. 11:30human, this means diving into metrics
  349. 11:32and logs and deployments. You can
  350. 11:35actually investigate what's going on.
  351. 11:37And so, what we're going to do is we're
  352. 11:39going to now have an agent run on Claude
  353. 11:41manage agents to do all this for us. So,
  354. 11:43that when we get woken up by 3:00 a.m.,
  355. 11:46we can hand it off to an agent, or maybe
  356. 11:47we don't even get woken up at all if
  357. 11:48Claude is able to do everything for us.
  358. 11:51Okay. So, let's now go ahead and dive
  359. 11:53into the code here.
  360. 11:55What we're going to open up here is we
  361. 11:57have the agent.py file on the left and
  362. 12:00the agent complete on the right. If you
  363. 12:02want to challenge yourself, you can of
  364. 12:03course try to implement everything
  365. 12:05yourself here or with Claude. Um but,
  366. 12:07what we're going to do just for
  367. 12:07simplicity's sake is just copy over
  368. 12:09various elements from the completed file
  369. 12:11onto the incomplete file one by one. So,
  370. 12:14you can see how these primitives compose
  371. 12:15our agent one piece at a time.
  372. 12:18So, let's go ahead and start off with
  373. 12:20this very first part, which is the
  374. 12:21agent.
  375. 12:23We mentioned before that the agent is
  376. 12:24the one that defines the persona and the
  377. 12:26capabilities of the agent here. So,
  378. 12:28that's model, the system prompts, and
  379. 12:30the tools in our case for our agent
  380. 12:33here.
  381. 12:34So, let me go ahead and copy over what
  382. 12:37we see there on the screen. And you can
  383. 12:39see here that we're defining the SRE
  384. 12:41agent. We're going to use Claude Opus
  385. 12:444.7 here. And I've preconfigured a
  386. 12:47system prompt and tools for the agent.
  387. 12:50We can actually take a quick look into
  388. 12:51what that system prompt and tool looks
  389. 12:53like here.
  390. 12:54For the system prompt, you can see that
  391. 12:55it's actually extremely simple for the
  392. 12:57agent that we're defining today. You can
  393. 12:59of course add more complexity and
  394. 13:00constraints here, but we actually see a
  395. 13:02very simple prompt working for our agent
  396. 13:04that we're building today. We're just
  397. 13:06telling it that it's an SRE agent. It's
  398. 13:08responsible for coming in and debugging
  399. 13:09incidents, and it has access to various
  400. 13:12tools like metrics, recent deployments,
  401. 13:14get diff. These are tools that you would
  402. 13:16want as a developer if you're actually
  403. 13:18managing an incident response as well,
  404. 13:20like the ability to actually fetch logs
  405. 13:22so you can see exactly what's going
  406. 13:23wrong.
  407. 13:24So, we're going to give those same tools
  408. 13:25and the same instructions over to our
  409. 13:27agent.
  410. 13:29So now that we've configured this on the
  411. 13:30screen and feel free for those of you
  412. 13:32who are able to spin it up on your own
  413. 13:33laptops to just follow along with
  414. 13:35exactly what I'm doing, which is copying
  415. 13:37over this portion from the right onto
  416. 13:39the left here.
  417. 13:42And then when we flip back over to the
  418. 13:43screen, what we'll see is this wasn't
  419. 13:45there until I just added that there, but
  420. 13:47we can now actually have a unique
  421. 13:49identifier attached to the agent that
  422. 13:51we're building.
  423. 13:54Okay, so that's step one. Now let's go
  424. 13:56ahead and move on to step two,
  425. 13:59which is the environment where the agent
  426. 14:01is going to actually do work in.
  427. 14:04All of you here were very lucky for
  428. 14:05those of you who were able to come
  429. 14:06yesterday as well to code with Cloud
  430. 14:08London. We actually just released
  431. 14:10yesterday the ability to bring your own
  432. 14:12containers and your own compute to Cloud
  433. 14:13Managed Agents, which means that you can
  434. 14:15actually execute the agent for the tools
  435. 14:17and the actual ability for the agent's
  436. 14:19actions to work within your own
  437. 14:20infrastructure and not just Entropic's
  438. 14:22managed infrastructure. So that's an
  439. 14:24exciting update that just came to Code
  440. 14:26with Cloud London. Um before today's
  441. 14:28purposes, you can actually see if we
  442. 14:29copy over this environment configuration
  443. 14:32here.
  444. 14:33We're defining our SRE agent to work
  445. 14:35within the Entropic cloud and here we're
  446. 14:38just giving it unrestricted access from
  447. 14:40a networking perspective.
  448. 14:41We've made Cloud Managed Agents very
  449. 14:43composable and very customizable. So
  450. 14:46this networking list here is actually an
  451. 14:48allowed list. If you want your agent to
  452. 14:50only be able to access specific sites
  453. 14:52and URLs, you can restrict this down as
  454. 14:54much as you would like.
  455. 14:55We also released um Cloud MCP tunnels,
  456. 14:59which actually also gives you the
  457. 15:00ability to run MCP servers within a
  458. 15:02private environment instead of on the
  459. 15:04public network as well. So again, just
  460. 15:06offering various components to help you
  461. 15:08make sure that your agents are as
  462. 15:10production ready and as secure as
  463. 15:11possible.
  464. 15:13So now that we've defined this
  465. 15:14environment here, let's flip back over
  466. 15:16and we just saw that environment piece
  467. 15:18come into our agent as well. So here we
  468. 15:21have unique identifier for an agent and
  469. 15:22an environment.
  470. 15:24And that will next help us as we go
  471. 15:26along with setting up the rest of our
  472. 15:28agents as we start to get into session
  473. 15:29definitions here.
  474. 15:33The next thing that we have to do is
  475. 15:34actually give our agent the ability to
  476. 15:37look at logs.
  477. 15:39With Cloud Code, that is one of the
  478. 15:41first times where we realized the power
  479. 15:42of giving the agent access to files and
  480. 15:44a file system. Here with Cloud Managed
  481. 15:47Agents, we're leveraging essentially the
  482. 15:48files API by uploading the metrics and
  483. 15:50logs to the agent. So the agent can
  484. 15:52start to run code and process through
  485. 15:53those files.
  486. 15:55So here we've attached the log here as a
  487. 15:58file for our agent. So we just also saw
  488. 16:00that populate and come through. Again
  489. 16:02here, the key takeaway is as much data
  490. 16:05as you're able to give the agent um as
  491. 16:07possible is what makes it so powerful.
  492. 16:09Context engineering is a huge portion
  493. 16:11that comes to actually making an agent
  494. 16:13powerful. And this is where we see the
  495. 16:14developers spending the majority of
  496. 16:16their time working on top of primitives
  497. 16:18like Cloud Managed Agents is managing
  498. 16:20context and managing what types of files
  499. 16:22are uploaded, how the agent processes
  500. 16:24those files. These are components that
  501. 16:26you compose yourself and are very
  502. 16:27customizable on top of Cloud Managed
  503. 16:29Agents to make it work as far and as
  504. 16:31wide as you want it to.
  505. 16:33Okay. So now let's go ahead and start to
  506. 16:35define the session that we have here.
  507. 16:38The session is going to oops, the
  508. 16:40session is going to bind the agent and
  509. 16:42the environment and also mount the log
  510. 16:45here. So you can see we're passing in
  511. 16:46the agent ID, the environment ID, and
  512. 16:49the resources that we're giving to the
  513. 16:51agent.
  514. 16:52And this is going to give it the ability
  515. 16:54to start to actually act and interact
  516. 16:56with me as a user.
  517. 16:59Let's go ahead and just complete the
  518. 17:01rest of this here so that we can
  519. 17:02actually start to run our agent.
  520. 17:05What we want to do is now also give the
  521. 17:07ability for the agent to come in and
  522. 17:09stream responses to me as we go along.
  523. 17:15There we go.
  524. 17:19Okay, and the key portion here is that
  525. 17:22when our Cloud Managed Agents runs
  526. 17:24within a single session, instead of
  527. 17:25responding in tokens in and tokens out,
  528. 17:27it actually works in units of events.
  529. 17:30Events here are things like user
  530. 17:31messages or agent tool calls, agent
  531. 17:33responses, so that every event can be
  532. 17:35logged from an observability
  533. 17:37perspective, as well as streamed back to
  534. 17:38the user for the user to see the agent
  535. 17:40responding as it calls tools and as it
  536. 17:42starts to populate responses. This is
  537. 17:44crucial for both a user experience
  538. 17:46perspective, so user starts to see
  539. 17:48things as they come through and not just
  540. 17:50when Claude finishes an entire task, and
  541. 17:52also from an observability perspective,
  542. 17:55and Claude Managed Agents actually has a
  543. 17:56very neat console built in for looking
  544. 17:59at everything the agent is doing, and a
  545. 18:00lot of observability features built into
  546. 18:02Claude Managed Agents.
  547. 18:05Okay, the last step here of just being
  548. 18:07able to put our agent together, you can
  549. 18:09start to see that our agent is actually
  550. 18:11starting to come together. We can start
  551. 18:13to create sessions and we can start to
  552. 18:15do things. Um, what we're actually going
  553. 18:17to see here though is that if I send
  554. 18:19something like hi to the agent, it can
  555. 18:21respond, um, but it doesn't actually
  556. 18:23have the ability to be able to call the
  557. 18:24various tools that we want it to yet,
  558. 18:26because we haven't connected that
  559. 18:28locally to what we want the agent to do
  560. 18:30when it calls tools like get metrics.
  561. 18:33So, the agent is ready. The agent is
  562. 18:34actually defined on the server side
  563. 18:35already. The missing piece here is just
  564. 18:38to finally give it our local tools
  565. 18:40so the agent can start to take action
  566. 18:42here on my computer or my
  567. 18:43infrastructure.
  568. 18:47Okay, so now that we have that copied
  569. 18:49over, the agent is going to be able to
  570. 18:50start to call get metrics, get recent
  571. 18:53deploys, get diffs, so it can truly
  572. 18:55start to take action in terms of helping
  573. 18:57us debug this incident.
  574. 19:00The last thing I'm going to do here is
  575. 19:01also just to make sure I give my agent
  576. 19:03the ability to delete sessions.
  577. 19:08So, that's when I come in, I can start
  578. 19:10to hit this delete button and delete
  579. 19:12sessions as I compose my agent. And this
  580. 19:15is also crucial from a security
  581. 19:17perspective. If you want to make sure
  582. 19:19that, you know, nothing is being
  583. 19:21retained for sessions that you don't
  584. 19:23want on the cloud or on your
  585. 19:24infrastructure, you can actually just
  586. 19:26come in and proactively manage how
  587. 19:28sessions are deleted. And once they're
  588. 19:30deleted, they will be also removed from
  589. 19:31every single log aspect here, so that
  590. 19:34you can truly make sure that whatever
  591. 19:36data you want managed is managed
  592. 19:37actively and proactively via Cloud
  593. 19:40Manager agents.
  594. 19:42Okay.
  595. 19:43So, with that all set up, let's go ahead
  596. 19:45and give our agent a test run here.
  597. 19:49I'm going to click the new session here,
  598. 19:51and I'm going to just go ahead and ask
  599. 19:53the agent to debug my incident for me.
  600. 20:03You can see here that because we gave
  601. 20:05the agent access to tools like
  602. 20:07sandboxing and bash, and get recent
  603. 20:10deploys,
  604. 20:11the agent is starting to really take
  605. 20:13powerful actions on my behalf here. It's
  606. 20:15come in, it's run the sandbox command.
  607. 20:17We can open this up and see what this
  608. 20:19looks like.
  609. 20:20Um we can see that it's actually coming
  610. 20:21in and looking at what the logs were
  611. 20:23added to.
  612. 20:25It's then come in and called this tool
  613. 20:27called get recent deploys,
  614. 20:29which is coming in and returning results
  615. 20:31like what the recent deployments look
  616. 20:32like, the metrics. We can see this from
  617. 20:35a user perspective if you click on the
  618. 20:37tabs here, but this is essentially the
  619. 20:39data that's actually being passed into
  620. 20:41the agent via these local tools that we
  621. 20:43define.
  622. 20:46And again, we can start to see the magic
  623. 20:47of that streaming that we implemented
  624. 20:49come through as well, because we saw
  625. 20:51these tools come in as they were being
  626. 20:53called from the agent. We saw the user
  627. 20:55prompt come in as soon as I prompted it
  628. 20:57to the agent, and the agent is actually
  629. 20:59streaming responses to me as it comes
  630. 21:01through with more token response and
  631. 21:03outputs, as well as as it calls more
  632. 21:05tool as it goes along as well.
  633. 21:11Okay, so what we're going to start to
  634. 21:12see is the agent being able to help us
  635. 21:14actually debug what's going on here,
  636. 21:16which we can see here that the incident
  637. 21:17is that there's something going wrong
  638. 21:19with our P99 latency that seems to be 10
  639. 21:22times above the baseline.
  640. 21:24The agent is coming in and debugging
  641. 21:25everything for us. Looks like it's
  642. 21:27taking another second there.
  643. 21:29So some of the major design decisions
  644. 21:31that come in here when you're designing
  645. 21:32a real site reliability site incident
  646. 21:35response agent for your systems is to
  647. 21:38think deeply about the various
  648. 21:39components that go in and the various
  649. 21:41MCP servers and skills that you want to
  650. 21:43give your agent. Here we've defined, of
  651. 21:45course, a very, very simple agent, but
  652. 21:47for lots of the SRE agents that we
  653. 21:49build, we actually also think about
  654. 21:50things like how can we give the agent a
  655. 21:52skill to actually execute and run
  656. 21:54runbooks.
  657. 21:55Runbooks are things where as teams debug
  658. 21:58incidents, they note down and document
  659. 22:00how they debug that incident so that
  660. 22:01they can do it again for a future
  661. 22:03session or future incident. You want to
  662. 22:05give the agent the same access to the
  663. 22:07materials that you would have as a human
  664. 22:09developer. So something like a runbook
  665. 22:11skill where the agent is actually able
  666. 22:12to look at example runbooks or fetch
  667. 22:15other postmortems from other incident
  668. 22:17responses. That is something that is
  669. 22:19very powerful for the agent to be able
  670. 22:21to understand how to work within your
  671. 22:23systems and debug incidents
  672. 22:25successfully.
  673. 22:29Okay, let's go ahead and take a look at
  674. 22:31the agent here.
  675. 22:34Let's see.
  676. 22:38I'm going to go ahead and just start a
  677. 22:39new session here to make sure everything
  678. 22:41is working well.
  679. 22:44All right, let's say debug
  680. 22:46my incident for me.
  681. 22:49Okay, hopefully this one works.
  682. 22:51Is anyone able to get it working on
  683. 22:52their laptops better than I have on the
  684. 22:54screen? Okay, we got some success in the
  685. 22:55room, so hopefully this will work as it
  686. 22:57goes along.
  687. 23:08>> Okay, looks like we are streaming. We're
  688. 23:09getting everything in.
  689. 23:12Let the agent go.
  690. 23:14Okay, agent is checking logs, debugging
  691. 23:16everything.
  692. 23:19So, if we just also look through some of
  693. 23:20the data here as the agent is working,
  694. 23:23the data that's actually being passed in
  695. 23:25for our agent here is all local just for
  696. 23:28our sake of our purposes for our demo
  697. 23:29now workshop that we're running today.
  698. 23:31But, with the ability for you to run
  699. 23:33your agents within a container and
  700. 23:35infrastructure, you can start to see how
  701. 23:37things like your get metrics tool that
  702. 23:38are currently pulling from JSON can be
  703. 23:40easily moved to something like DataDog
  704. 23:42or other production systems for your
  705. 23:44infrastructure from that perspective.
  706. 23:46So, everything that you see here that is
  707. 23:48currently local can be something that's
  708. 23:49easily movable into infrastructure as
  709. 23:51well via Cloud Manager agents.
  710. 24:04Okay. Let's all cross our fingers and
  711. 24:06see if this one works.
  712. 24:17Oh, there we go. Success.
  713. 24:19Okay, so the agent has come in. You can
  714. 24:21see here that if we scroll through all
  715. 24:23the tool calls, everything is persisted
  716. 24:25in the cloud from a logs perspective.
  717. 24:27All of this will also be logged in the
  718. 24:28observability console.
  719. 24:30And then the agent has come back to us
  720. 24:32with the incident response here. It says
  721. 24:34that this seems to be caused by a
  722. 24:36database pool exhaustion. Seems like a
  723. 24:39commit that someone added here from
  724. 24:41Alice to refactor the order summary
  725. 24:44builder introduced a query that then
  726. 24:47caused the pool resources to be
  727. 24:49exhausted. So, it's looking at and
  728. 24:50giving us the exact everything that went
  729. 24:52wrong from all the metrics they were
  730. 24:54able to call. It ruled out various other
  731. 24:56causes, and then it's also giving us
  732. 24:58recommended actions to take.
  733. 25:00Another key component here in a lot of
  734. 25:01other incident response management
  735. 25:03agents that we built is actually giving
  736. 25:05the agent to actually go ahead and fix
  737. 25:07everything that it's been able to find.
  738. 25:10By giving the agent then access to
  739. 25:11something like Cloud Code for instance,
  740. 25:13you can actually imagine this agent can
  741. 25:14then go into your code base, suggest
  742. 25:16fixes, put up a PR, and essentially do
  743. 25:19everything that it needs to do to help
  744. 25:21you go from initial incident all the way
  745. 25:23to fixing the root cause. So again here
  746. 25:26for demo purposes, we're stopping at
  747. 25:27just the agent giving us the recommended
  748. 25:29actions, but I want you all to imagine
  749. 25:31the possibilities of where this can go
  750. 25:33if we give our agent more tools, more
  751. 25:35ability to take actions, access to your
  752. 25:37code base, ability to put up PRs,
  753. 25:39ability to fix incidents, so that you as
  754. 25:42a human developer can just become the
  755. 25:43oversight and watch over the agents as
  756. 25:46they take action, and you no longer have
  757. 25:47to go through and do manual steps like
  758. 25:50actually following the agent's
  759. 25:51instructions here to fix the root cause
  760. 25:53of the incident.
  761. 25:55So another key component of what we've
  762. 25:57built here on Cloud Manage Agents is
  763. 25:59session persistence. So when I come in
  764. 26:02and hit hard refresh on the screen,
  765. 26:04we're seeing that the agent is listing
  766. 26:05the sessions and everything is retained
  767. 26:08from all the sessions that we just ran.
  768. 26:11We also have the previous sessions that
  769. 26:12we ran all retained in the cloud. Looks
  770. 26:14like this one actually came back to us
  771. 26:15as well.
  772. 26:17Um and the previous sessions where we
  773. 26:18just said hi, everything is retained in
  774. 26:20the cloud, and we didn't have to deal
  775. 26:21with things like database and deployment
  776. 26:23of our agent and moving it from our
  777. 26:25laptops to production. Everything is
  778. 26:27already maintained server side.
  779. 26:30We can also see the ability to delete
  780. 26:31sessions come in. So I've run that
  781. 26:33delete, and now we have that um running
  782. 26:36the session here. Now we have that
  783. 26:38removed from our list here.
  784. 26:41Another thing that I want you to take a
  785. 26:42note of, which we'll talk through a
  786. 26:43little bit in just a second, is the
  787. 26:45states of the session. Here we can see
  788. 26:47that the sessions are now idle. Just now
  789. 26:49as they they running, they were in a
  790. 26:50running state.
  791. 26:52We have the sessions managed by state
  792. 26:54here as part of that same durability and
  793. 26:56maintenance and reliability of the
  794. 26:58session. So, when I come in and ask the
  795. 27:00agent something else like, "Who are
  796. 27:02you?" I was able to easily resume the
  797. 27:05session and execute as it goes along
  798. 27:07within that same session window. So,
  799. 27:09state management here is really
  800. 27:10important to how manage agents works
  801. 27:12under the hood.
  802. 27:15All right.
  803. 27:18So, now as if we just take a quick step
  804. 27:20back and look through everything we were
  805. 27:22able to accomplish. We started with an
  806. 27:24empty agent here just built on a couple
  807. 27:26of primitives on Cloud Managed Agents.
  808. 27:28We then went and defined the agent
  809. 27:30definition, the persona, the
  810. 27:31capabilities. We gave the agent an
  811. 27:33environment. We gave the agent data and
  812. 27:36context to operate over.
  813. 27:38We then gave the agent sessions
  814. 27:40combining the agent definitions to an
  815. 27:42environment so the agent can think
  816. 27:44through which tools to call from an
  817. 27:46agent loop perspective. And then it can
  818. 27:48actually call those tools and take
  819. 27:49action on our behalf.
  820. 27:51We then came in and streamed the
  821. 27:52responses to the user into our logs,
  822. 27:54implemented some local tools as well as
  823. 27:56the ability to delete sessions. And
  824. 27:58within this Streamlit app here, we saw
  825. 28:01how that actually affected from a front
  826. 28:02end perspective how our agent was
  827. 28:04actually able to be presented to our
  828. 28:05users by adding all of these primitives
  829. 28:07together.
  830. 28:10So, now let's go ahead and move back
  831. 28:12over to the slides to do a quick recap
  832. 28:14and talk through some of the lessons of
  833. 28:15what we learned about how Cloud Managed
  834. 28:17Agents works under the hood. But
  835. 28:19hopefully for all of you who were able
  836. 28:20to actually build on your laptops, you
  837. 28:22all were able to just build a site
  838. 28:24reliability agent. So, congrats to you
  839. 28:25all.
  840. 28:27But let's go ahead and dive in a little
  841. 28:29bit here into understanding what
  842. 28:30actually happened when we put all those
  843. 28:32pieces together.
  844. 28:34The first thing we saw is that we saw
  845. 28:36session speak in events and not
  846. 28:37responses in and tokens in tokens out
  847. 28:40from a request response perspective like
  848. 28:42we see typical with things like message
  849. 28:43API or other APIs that we see.
  850. 28:46With Cloud Managed Agents, instead of
  851. 28:48just having a request response, we
  852. 28:49actually have events appended to the
  853. 28:51logs.
  854. 28:52Again, this is a huge portion of why
  855. 28:54Cloud Managed Agents is so reliable and
  856. 28:56secure, because events are coming
  857. 28:58through and just added into an existing
  858. 29:00session logs, so that it's easy to then
  859. 29:02resume a session and kick back off where
  860. 29:04you left off, and it's easy to then come
  861. 29:06in and look at everything from a log
  862. 29:08perspective. This is also really
  863. 29:09important from a reliability perspective
  864. 29:11when we separate the hands from the
  865. 29:13brain of the agent. If a container goes
  866. 29:15down, we can just spin that container
  867. 29:17back up again, and we don't have to
  868. 29:18restart the entire agent loop alongside
  869. 29:21that container.
  870. 29:24The next thing here is that we saw the
  871. 29:26ability to implement local tools.
  872. 29:29And we implemented in our workshop these
  873. 29:31local tools defined in JSON and loading
  874. 29:33them in via our local files here. We
  875. 29:36were then actually able to see how with
  876. 29:38our Cloud Managed Agents Harness, the
  877. 29:40execution of the agent is completely
  878. 29:42separate from the agent loop.
  879. 29:44We defined everything that executed
  880. 29:46locally on our laptops and our scripts,
  881. 29:48um and our agent loop ran on the cloud
  882. 29:50inside of Anthropic's managed
  883. 29:51infrastructure.
  884. 29:53Again here, especially with what we just
  885. 29:55released with bring your own compute and
  886. 29:57bring your own sandboxing here, you can
  887. 29:59swap out where you want that agent to
  888. 30:01execute its tools in your own
  889. 30:02infrastructure or on Anthropic managed
  890. 30:04infrastructure, but within your own
  891. 30:06environments and your own containers as
  892. 30:08well as you spin them up. Moving from
  893. 30:10things like loading our tools in from
  894. 30:12JSON into anywhere you want to have your
  895. 30:15tools run, like a Data Dog client using
  896. 30:17the same wire protocol, making it very
  897. 30:20easy to then go from initially building
  898. 30:22the agent for Cloud Managed Agents to
  899. 30:24then actually producing it and deploying
  900. 30:26it on production-ready infrastructure.
  901. 30:31Next thing we saw here as we thought
  902. 30:33about how our sessions are being
  903. 30:34streamed into our users and what we see
  904. 30:37from a front end perspective
  905. 30:39is that we saw when our events were
  906. 30:41being able to be streamed to our users,
  907. 30:43these were in the forms of actually
  908. 30:44things we care about as a user. We saw
  909. 30:46events come in and we saw the agent's
  910. 30:48ability to actually log everything to
  911. 30:51its observability console.
  912. 30:53And another key thing here is that as we
  913. 30:55think about how sessions are controlled
  914. 30:56in Cloud Managed Agents, you can
  915. 30:58actually think about the state as being
  916. 30:59something very powerful when you can
  917. 31:01start to take action on behalf of
  918. 31:03events.
  919. 31:05What that means is that we saw a couple
  920. 31:07of key states for sessions in CMA or
  921. 31:09Cloud Managed Agents.
  922. 31:11We went from idle to running,
  923. 31:12rescheduling if the agent needs to retry
  924. 31:14anything, or terminated if any of the
  925. 31:16sessions fail.
  926. 31:18And so the agent is able to restart from
  927. 31:20a reliability perspective, a
  928. 31:21resumability perspective, but also it
  929. 31:23can actually do some very powerful
  930. 31:25things.
  931. 31:26For instance, you can actually have a
  932. 31:27webhook run, and when an event happens
  933. 31:30from a webhook, the agent receives that
  934. 31:32webhook in and can then do something
  935. 31:34like resume a session or kickstart a
  936. 31:35specific state based on external events.
  937. 31:39So again, this powerful form of having
  938. 31:41events and sessions be the core concepts
  939. 31:43of how Cloud Managed Agents runs means
  940. 31:45that you can make it very, very easy to
  941. 31:47compose your agent however you want it
  942. 31:49to, and have the agent listen for things
  943. 31:51that happen both internally and
  944. 31:52externally via webhooks to take actions
  945. 31:55or resume your agent as you desire.
  946. 31:59And lastly here, something that we saw
  947. 32:01come through through the agent that we
  948. 32:02all built for the site reliability
  949. 32:04agent, is that everything lives in the
  950. 32:06cloud from the agent loops perspective.
  951. 32:09The conversation is persisted. When we
  952. 32:11hard refresh the page, we saw that those
  953. 32:13same sessions were maintained. And we
  954. 32:15saw that if we were able to, let's say,
  955. 32:17exit out of the agent and come back, we
  956. 32:19didn't have to manage anything from a
  957. 32:21database perspective or wire up where
  958. 32:23the agent is stored. We were just able
  959. 32:25to have all of that persisted in the
  960. 32:27cloud, again making it very, very easy
  961. 32:29to go to production-ready agents.
  962. 32:35And lastly here, I just want to talk you
  963. 32:36through we just built the very basic
  964. 32:38form of Claude managed agents. We saw
  965. 32:40what was possible with just the very,
  966. 32:42very simple primitives that we all built
  967. 32:44with, the basic level of what you can do
  968. 32:46with Claude managed agents. And already
  969. 32:48there we were able to have something
  970. 32:49that would usually take us a lot of time
  971. 32:51to spin up from a production
  972. 32:52perspective. All of compaction, caching,
  973. 32:55tool calling, all of that was handled
  974. 32:57for us there's via Claude managed
  975. 32:59agents.
  976. 33:00And even if we wanted to go beyond that
  977. 33:02to make our agent much, much more
  978. 33:03powerful,
  979. 33:04we can do things like add in skills, add
  980. 33:06in subagents, add in memory, add in
  981. 33:09outcomes. These are all core components
  982. 33:11that we offer to developers out of the
  983. 33:13box from Claude managed agents.
  984. 33:16I'll just briefly talk you through a
  985. 33:17couple of the key components, but want
  986. 33:18to encourage everyone to check out our
  987. 33:20documentation, what's publicly available
  988. 33:22on Claude managed agents, attend the
  989. 33:24session after this one on dreaming to
  990. 33:26dive in deeper onto these topics.
  991. 33:29Subagents or multi-agents is a way for
  992. 33:31you to have an orchestrator agent spin
  993. 33:33up context with other agents so that you
  994. 33:35can manage it from a context engineering
  995. 33:37perspective where subagents can then
  996. 33:39handle tasks and have their own context
  997. 33:41windows and contribute back to the main
  998. 33:42agent making it much more powerful from
  999. 33:44a parallelization perspective as well as
  1000. 33:47the ability for context management.
  1001. 33:49Memory is something that's always very
  1002. 33:50important as we're building agents. I
  1003. 33:52hear a lot of questions about how you
  1004. 33:54can build self-improving agents or
  1005. 33:55agents that learn from user corrections,
  1006. 33:57agents that start to remember user
  1007. 33:59preferences. That's where we're offering
  1008. 34:01memory and a dreaming service for Claude
  1009. 34:03managed agents out of box.
  1010. 34:05What dreaming means for managed agents
  1011. 34:07is that Claude can actually come in and
  1012. 34:08also look through its own memory logs
  1013. 34:11and determine what to keep and determine
  1014. 34:13how it can actually start to memorize
  1015. 34:15and manage context for its own memory.
  1016. 34:17So, it can actually be able to really
  1017. 34:18accurately remember which parts of your
  1018. 34:20user preferences matter and which part
  1019. 34:22of user corrections you want to retain
  1020. 34:24for future sessions you run on that same
  1021. 34:26agent.
  1022. 34:28Outcomes is another one of my favorites
  1023. 34:29where for Claude managed agents, this
  1024. 34:31means that you can actually define a
  1025. 34:32rubric for your agent outcomes. So, you
  1026. 34:35can start to think of your agent's tasks
  1027. 34:36as something where you want the agent to
  1028. 34:38reach a desired outcome instead of just
  1029. 34:40executing calls and doing things on your
  1030. 34:42behalf, but not associating that to a
  1031. 34:44result that you want.
  1032. 34:46So, with outcomes, you can define a
  1033. 34:48rubric of exactly what you want the
  1034. 34:49agent to produce, and it'll figure out
  1035. 34:51along the way which tool calls and what
  1036. 34:53it needs to do to execute towards that
  1037. 34:55final result.
  1038. 34:57Vaults is something also that I hear
  1039. 34:59come up a lot as of interest for Cloud
  1040. 35:01Managed Agents because managing user
  1041. 35:03credentials is something that's very
  1042. 35:04painful from an access management
  1043. 35:06perspective, making sure that your
  1044. 35:08agents are secure and safe to run.
  1045. 35:10So, for vaults and Cloud Managed Agents,
  1046. 35:11there's actually an encryption that
  1047. 35:13happens between where the credentials
  1048. 35:14are stored on a separate endpoint and
  1049. 35:16what the agent is actually able to
  1050. 35:18access. So, you can manage these
  1051. 35:19credentials on a per user, per session
  1052. 35:21basis, all very safely and securely. And
  1053. 35:24this relies in large part during to that
  1054. 35:26architecture that I described earlier of
  1055. 35:28how the brains and the hands of the
  1056. 35:29agent are separated, so that credentials
  1057. 35:31can be stored very securely in these
  1058. 35:33vaults. This means that you don't have
  1059. 35:35to set up your own secret stores or your
  1060. 35:37own credential stores, and you can just
  1061. 35:39rely on the built-in capability here.
  1062. 35:42There are a couple other things here
  1063. 35:43that I won't have time to go through in
  1064. 35:44depth. So, again, encourage everyone to
  1065. 35:46check them out in more detail. There are
  1066. 35:47things like the ability to do web hooks
  1067. 35:49and really make this agent run on
  1068. 35:51external events, things like detailed
  1069. 35:53and fine-grained permission policies,
  1070. 35:55the MCP servers that I mentioned where
  1071. 35:57we just released new MCP uh server
  1072. 36:00controls as well. And something that I
  1073. 36:02also love just to briefly touch on is
  1074. 36:04the console agent builder where we have
  1075. 36:06built in a lot of capability and
  1076. 36:07functionality into the default developer
  1077. 36:09console, where you can start to see a
  1078. 36:11beautiful observability dashboard come
  1079. 36:13through and other ways for you to define
  1080. 36:15Cloud Managed Agents right there on your
  1081. 36:16consoles.
  1082. 36:20So, just as a quick recap to end us off
  1083. 36:22here of what we were able to accomplish
  1084. 36:24today, hopefully everyone leaves here
  1085. 36:26with a bit of a mental model about how
  1086. 36:27manage agents actually works under the
  1087. 36:29hood and be proud of yourselves for
  1088. 36:31everyone that was able to come in and
  1089. 36:33build on your laptops and actually ship
  1090. 36:35a site reliability agent so you can all
  1091. 36:37leave here being very happy with
  1092. 36:39yourselves that you were able to come in
  1093. 36:40and save future developers hours of time
  1094. 36:42of being woken up at 3:00 a.m. or 2:00
  1095. 36:44a.m. in the morning and being able to
  1096. 36:45handle incidents for them.
  1097. 36:47And next you also learned a little bit
  1098. 36:49about where to go next for how you can
  1099. 36:50really start to unlock the power that
  1100. 36:52comes with manage agents and think about
  1101. 36:53how your agents can become super powered
  1102. 36:56with all of these additional
  1103. 36:57functionalities.
  1104. 36:59So that is where I'll end up today but
  1105. 37:01thank you all so much for coming. I'll
  1106. 37:03be around on the side.
  1107. 37:04>> [music and applause]

About this transcript

This page contains the full transcript of Ship your first Managed Agent by Claude, generated from the public captions YouTube serves with the video. The transcript has 7,185 words across 1,107 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.