Context management for Claude agents — Transcript

Full transcript

1. 0:01Every request that you send Claude has a
2. 0:03context window.
3. 0:04A million tokens sounds like a lot, but
4. 0:06it runs out faster than you think once
5. 0:08you're shipping a real agent. And this
6. 0:10is where contact management comes in.
7. 0:12It's how you stay inside the window
8. 0:14without losing what matters.
9. 0:17Context is everything Claude sees on a
10. 0:19given turn. The system prompt, the
11. 0:21message history, tool definitions, tool
12. 0:23results, attached files, skills,
13. 0:25thinking blocks. It's the input to every
14. 0:27single API call. You pay for it on the
15. 0:30way in and you pay for it on the way
16. 0:31out. And once it's full, the request
17. 0:33fails. So, the goal isn't fit
18. 0:35everything. The goal is fit the right
19. 0:37things in.
20. 0:39Anthropic publishes four patterns for
21. 0:41managing context in long-running agents.
22. 0:43Three are first-class API features and
23. 0:45one is a design pattern. One is the
24. 0:47just-in-time context. Don't load
25. 0:50everything up front. Load what the agent
26. 0:52needs now and let it pull more in via
27. 0:54tools when it asks. The compliance
28. 0:57review agent doesn't get the entire
29. 0:58building codebook in a system prompt. It
30. 1:01calls lookup building code when it needs
31. 1:03a section.
32. 1:05When a conversation runs long, Anthropic
33. 1:07server-side compaction summarizes older
34. 1:09turns into a single block. Add a context
35. 1:11management key that holds a edits and
36. 1:14then a type. Just like this. The API
37. 1:17auto summarizes when input crosses the
38. 1:18trigger threshold. Prompt caching lets
39. 1:21you mark stable parts of the request.
40. 1:23The system prompt, the tool definitions,
41. 1:25a long document, and reuses them across
42. 1:27calls at a fraction of the cost. So, if
43. 1:29your system prompt is 4,000 tokens and
44. 1:31you call it 100 times an hour, well,
45. 1:33caching is a difference between a usable
46. 1:35bill and a phone call from finance. Some
47. 1:38context needs to survive across
48. 1:39sessions. User preferences, the agent's
49. 1:42running notes, what was decided last
50. 1:44week. The recommended primitive is the
51. 1:45memory tool. Claude reason writes a
52. 1:47memories directory via tool calls. You
53. 1:50implement the storage back end client
54. 1:51side. So, this could be like a file
55. 1:53system, database, encrypted store,
56. 1:55whatever you want. Anthropic auto
57. 1:57injects a system instruction telling
58. 1:58Claude to check the memory directory
59. 2:00before starting work. In a production
60. 2:02app, you'll usually layer all four at
61. 2:04once.
62. 2:05The compliance review agent caches a
63. 2:06system prompt and tool definitions and
64. 2:09pulls building code sections in just in
65. 2:10time via lookup building code. And each
66. 2:13pattern handles a different failure
67. 2:14mode. Cost, window size, statelessness,
68. 2:17pick the ones that match what's
69. 2:18breaking.
70. 2:23Context isn't free and it isn't
71. 2:25infinite.
72. 2:26Load just in time.
73. 2:28Compact what's old.
74. 2:30And remember what has to survive across
75. 2:32sessions.
76. 2:34Four patterns but one goal.
77. 2:36Either wire these up by hand or use
78. 2:38Claude managed agents which ship with
79. 2:39caching and compaction on by default.