The NEW Way to Build a Second Brain w/ Claude (2027) — Transcript

Full transcript

1. 0:00In 1945, an engineer called Vannevar
2. 0:02Bush imagined a machine that could store
3. 0:03all of his books, records, and
4. 0:05communications. He called it an enlarged
5. 0:08intimate supplement to his memory. 80
6. 0:11years later, his wishes came true, and
7. 0:12now we have an avalanche of AI slop, big
8. 0:15flashy videos, a bunch of 3D brains
9. 0:17floating around, and whatever the hell
10. 0:18this thing is. So, why are second brains
11. 0:20so synonymous with garbage? I mean,
12. 0:22aside from the fact that they're really
13. 0:23easy marketing, they seem sufficiently
14. 0:25complex such that, you know, an engineer
15. 0:27might have to be involved in the making
16. 0:29of it. They're also very easy to sell to
17. 0:30people that don't understand how this
18. 0:32stuff works as a way to improve their
19. 0:34business productivity, you know, sell
20. 0:36more products, and so on and so forth.
21. 0:38In reality, though, I hold that personal
22. 0:40knowledge bases can be good. I use mine
23. 0:43for a business that does a fair amount
24. 0:44of money every month, and what I want to
25. 0:46do in this video is actually want to run
26. 0:47you guys through a practical way that
27. 0:49you can set one up for yourself. Best
28. 0:50part of all is it'll only take you maybe
29. 0:523 or 4 minutes. So, what is a personal
30. 0:54knowledge base really? Well, it's just a
31. 0:56searchable pile of your own stuff plus
32. 0:59AI looking that up before it answers
33. 1:01you. Now, engineers in the past would
34. 1:03call that process retrieval, and when
35. 1:05you are augmenting an AI generation with
36. 1:07it, naturally, it is retrieval augmented
37. 1:10generation, or rag, which you probably
38. 1:12heard about before. Back in the day, rag
39. 1:14was super useful and used super commonly
40. 1:16because AI models were pretty short on
41. 1:19context windows, meaning you couldn't
42. 1:20really fit a lot of things in there. And
43. 1:22if you wanted to give an AI the ability
44. 1:24to answer a wide variety of things and
45. 1:26not hallucinate, uh you sort of needed
46. 1:28to use the approach. Nowadays, we have
47. 1:30way larger context windows capable of
48. 1:32storing up to approximately 35 to 50
49. 1:36books worth for most open consumer
50. 1:37models, meaning you can literally just
51. 1:39dump everything that you want a model to
52. 1:41talk to you about inside the context
53. 1:43window, and it'll do just fine.
54. 1:44Although, if you were to store 35 books
55. 1:46worth of content, probably cost you a
56. 1:47lot of money, so that is not
57. 1:48recommended. However, the nuanced reason
58. 1:51why something like a personal knowledge
59. 1:52base makes sense, and the mechanisms I'm
60. 1:54about to show you will improve its
61. 1:56ability to answer questions for you,
62. 1:58especially on your own or proprietary
63. 1:59data, is when you have a giant stack of
64. 2:03context all in a model's context window,
65. 2:05the model assumes everything is equally
66. 2:07important. And so, if your knowledge
67. 2:09base has information in it from forever
68. 2:11ago, say, you know, a couple years ago
69. 2:13you had a certain product price and, you
70. 2:15know, your your knowledge base is being
71. 2:17consulted to help you put together a
72. 2:18proposal or something like that, it will
73. 2:20treat that price the same as it will
74. 2:22treat a new price. And that's just sort
75. 2:23of how it works. It'll be really
76. 2:25confident because to it, it's just like,
77. 2:26"Hey, piece of text here, piece of text
78. 2:28here. I don't know which one's true. I'm
79. 2:29just going to return the first and
80. 2:31easiest thing I can find." Now, a simple
81. 2:32solution to that, which I'm going to run
82. 2:33you through, is called recency
83. 2:35weighting, where you stamp every new
84. 2:37piece of information that comes into the
85. 2:38system with a date. Then, you apply a
86. 2:41little boost to things that were stamped
87. 2:43with a date that is more recent, and
88. 2:45then you apply a little bit of it a
89. 2:46decrease in power to things that were
90. 2:48stamped with a date that was a long time
91. 2:50ago. That way, AI models will
92. 2:52understand, "Okay, I should probably
93. 2:54look for things that are newer as
94. 2:55opposed to things that are older." But,
95. 2:57there's still a problem. The more text
96. 2:58that you put in, the harder it is for an
97. 3:00AI to effectively retrieve any one
98. 3:02individual thing. This is a very known
99. 3:04problem in large language models called
100. 3:06context rot, where performance tends to
101. 3:08go down the more context you have in the
102. 3:10window. So, while theoretically
103. 3:12speaking, you could add 35 books worth
104. 3:14or maybe over a million tokens of
105. 3:16content, uh in terms of practical uses,
106. 3:19one, it'll cost way too much money in
107. 3:21order for you to run that effectively.
108. 3:22Two, it'll probably take way too much
109. 3:23time because now the AI has to loop over
110. 3:2535 books worth. And three, it is less
111. 3:28likely to be accurate than if you did it
112. 3:29in the way I'm about to show you. A
113. 3:31solution to that is called retrieval.
114. 3:33Instead of sending all the data, what
115. 3:34you do is you just have it somewhere
116. 3:36else, and then you have the AI model
117. 3:37search for what it's looking for first.
118. 3:39And it can search it relatively
119. 3:40efficiently uh through other plain text
120. 3:42or through vector embeddings, which
121. 3:44aren't super important right now. This
122. 3:45is just scratching the surface, by the
123. 3:47way. The current state-of-the-art
124. 3:48solutions tend to combine all of those
125. 3:50things into one and then make them way
126. 3:52better. In my case, which is called
127. 3:54Engram, our knowledge base holds around
128. 3:5635 to 50,000 records at a time. We hold
129. 4:00all of my get history, we hold all of
130. 4:02our Claude sessions collectively as a
131. 4:04company, as well as our video
132. 4:06transcripts, and I make a lot of videos,
133. 4:08you guys probably know, so I hope you
134. 4:10can imagine that's a lot of tokens. As
135. 4:12information comes in, we recency weight
136. 4:14them to avoid the problem of, you know,
137. 4:16when we consult an AI pulling
138. 4:17information that might no longer be
139. 4:19useful. And we also periodically sweep
140. 4:21through it to remove anything that might
141. 4:23no longer be valuable to us. Then when I
142. 4:25ask Claude a question, literally any
143. 4:27question about anything, and you can
144. 4:28replace Claude with your own large
145. 4:30language model if you use a different
146. 4:31one, um it'll run an extremely fast bit
147. 4:34of inference to quickly pick a right
148. 4:36search term to consult that knowledge
149. 4:38base. Then it will search the results,
150. 4:40usually in under half a second or so,
151. 4:42before returning me a handful of best
152. 4:44matches, appending that to the prompt of
153. 4:47the query that I asked, and then using
154. 4:49that information to answer. Which sounds
155. 4:51pretty simple, but you would not believe
156. 4:52the degree to which this makes my
157. 4:54experience seamless. I never have to
158. 4:56worry about storing any information
159. 4:58anywhere. I never have to worry about
160. 5:00Claude's memory failing. I never have to
161. 5:02worry about it grabbing me the wrong
162. 5:04information because it is almost always
163. 5:06going to grab the recency weighted
164. 5:08information that I actually want it to
165. 5:10return me, not some random artifact of a
166. 5:13conversation we talked about four weeks
167. 5:14ago. So, how do you actually build
168. 5:16something like this? Well, the first
169. 5:18step is maintaining consistent context
170. 5:20pipelines into your knowledge base. So,
171. 5:23in my opinion, this step matters the
172. 5:25most because obviously knowledge bases
173. 5:27are only as good as the data. You need
174. 5:28consistent context in via pipelines that
175. 5:31refresh essentially every time an event
176. 5:33occurs that produces data within your
177. 5:35company. What you do is you define all
178. 5:38of your sources first. So, maybe your
179. 5:40email, for instance, or your website, or
180. 5:42all of the leads that you have ever had
181. 5:44conversations with, all of the Slack
182. 5:46messages in your company, all of the
183. 5:48transcripts of the calls from your
184. 5:50company phones, especially if they're
185. 5:51VoIP. And then you build a small script
186. 5:53called a connector. What that connector
187. 5:55does is it pulls in only what is new
188. 5:58data since the last run. Then it stamps
189. 6:00that data with the current date and
190. 6:03time. Example, my email connector uses
191. 6:05the Gmail API with read-only access.
192. 6:08That's important. You don't want to give
193. 6:09this thing right access because it'll
194. 6:11have total control of your life
195. 6:12otherwise. Um when I ran it the first
196. 6:14time, it pulled a full year of email
197. 6:17from my last three accounts into the
198. 6:18pipe. Now when I ran it a second time,
199. 6:20it only had to grab the time since the
200. 6:22last run. And to keep in mind, this
201. 6:24thing is running on my own schedule. I
202. 6:26set it to run once every 15 minutes. But
203. 6:29obviously, depending on compute load,
204. 6:31you guys can set it to run however you
205. 6:32want. I would highly recommend you also
206. 6:34add Claude sessions. That's even easier.
207. 6:36So if you're using ChatGPT or Claude,
208. 6:38all of those sessions are actually
209. 6:40stored on your computer as plain text
210. 6:41already. Makes them very easy to index.
211. 6:44So the key thing is they normally are
212. 6:46deleted after 30 days. This allows you
213. 6:48to consult conversations that you had
214. 6:50months ago, which honestly usually are
215. 6:52quite vital in building out ideas and
216. 6:54nuggets from bigger projects, as well as
217. 6:56not having you repeat yourself. So you
218. 6:58can literally copy and paste this video
219. 7:00transcript into your favorite agent and
220. 7:02then ask it, "Hey, what are my sources?"
221. 7:03and it could probably do a pretty good
222. 7:05job of enumerating a few of them. Not to
223. 7:06mention building out the rest of the
224. 7:08pipeline, which looks like this. Now you
225. 7:09can't actually just add the content
226. 7:11itself. As much as that would be cool,
227. 7:14the main way that this works is you turn
228. 7:16every one of these long items into the
229. 7:19long item itself, then some metadata,
230. 7:22including a list of notes about what's
231. 7:23actually in it. Usually rephrased a
232. 7:25bunch of times to really capture the
233. 7:27idea correctly. And that's what this
234. 7:29next step does, which is called letting
235. 7:32a cheat model write the summaries. You
236. 7:34know, raw text itself is really hard to
237. 7:35search, especially for AI models that
238. 7:37are going to be doing this stuff via
239. 7:39keyword matching. But if instead you do
240. 7:41two things, first, create a short
241. 7:44summary of the topic that you are
242. 7:46putting into your system, essentially of
243. 7:48the data, including the question that
244. 7:51somebody would probably type in order to
245. 7:53find it, you solve that completely.
246. 7:55Then, I'd also add what are called
247. 7:57atoms, which are exact error messages,
248. 8:00the exact fixes, any commands that work,
249. 8:02let's say if you're building some sort
250. 8:04of program or product, and decisions for
251. 8:07why the person did what they did during
252. 8:10that time period. What this allows you
253. 8:11to do is essentially consolidate all of
254. 8:13the various implicit decisions that
255. 8:16people in your company are making, and
256. 8:17then make them explicit, aka verbalize
257. 8:19them, turn them into what's called
258. 8:20institutional memory of your
259. 8:22organization. Very powerful stuff,
260. 8:24because now you have literally any
261. 8:25workflow or process you or any agent
262. 8:28could ever possibly want. Obviously,
263. 8:30this step involves another of large
264. 8:31language model. The way that I do it is
265. 8:33I use Claude Haiku 5.5 right now.
266. 8:36Before, I was using 4.5. In the future,
267. 8:38I'll probably be using 6.5. But,
268. 8:40basically, whatever the cheapest, um,
269. 8:42you know, AI model is for the provider
270. 8:44that you trust with this information,
271. 8:46and then just give it a prompt, which
272. 8:48your AI model can build from this
273. 8:49transcript, and then have it create this
274. 8:51metadata, create that list of short
275. 8:53notes, plus that list of atoms, every
276. 8:55time something makes it into the system.
277. 8:57The value here is now Claude or your
278. 8:59other agent gets to search for the
279. 9:01summaries. It doesn't even have to
280. 9:02search for the raw data itself. And
281. 9:03then, when it matches something in the
282. 9:05summary, then it's just pointed to the
283. 9:07raw data, so it actually gets the whole
284. 9:09thing. Okay, so from here on out, all
285. 9:10you have to do is actually make this
286. 9:11searchable. Uh, most people
287. 9:13overcomplicate this. They use things
288. 9:14like vector databases. Uh, you don't
289. 9:16have to use embeddings at all.
290. 9:17Literally, the simplest and easiest way
291. 9:18to do this is just plain text search.
292. 9:20Uh, and this was like crazy when I
293. 9:21figured this out. Um, you know, we read
294. 9:24a lot of different ways that people were
295. 9:25solving this problem, and obviously I
296. 9:27built a lot of systems with RAG for a
297. 9:28variety of clients up until that point.
298. 9:30But, the second I realized, "Wait a
299. 9:31second, you don't need RAG for this.
300. 9:32There are people out there that are just
301. 9:33doing this with plain text." I mean,
302. 9:34inference time went down by like 9/10.
303. 9:36The cost associated with maintaining all
304. 9:38of these vector DBs, the embeddings
305. 9:40cost, all of that went down by 9/10. And
306. 9:42yeah, we just have massive strings of
307. 9:43text basically maintaining everything.
308. 9:44The specific plain text algorithm is
309. 9:46BM25. This one's been around since the
310. 9:481990s. It just gives every document that
311. 9:51you add, which is a piece of data, um
312. 9:53like a score based off of the frequency
313. 9:55of the words that you are using to
314. 9:56search it up. Uh and turns out that's
315. 9:58all you need. You know, Claude
316. 9:59intelligently can infer the rest. I saw
317. 10:01this way too late, but Boris Cherney,
318. 10:03who's the uh guy initially behind the
319. 10:05creation of Claude Code, the product
320. 10:07that you and I use literally every day,
321. 10:09said something like, you know, plain
322. 10:11search outperformed every possible other
323. 10:13approach by a lot. And uh yeah, I mean,
324. 10:16just an insane realization that we've
325. 10:19been overcomplicating things like RAG
326. 10:20for a really long time. Anyway, aside
327. 10:22from that, just make sure to keep it
328. 10:23fresh. So, you do need to have some sort
329. 10:25of script in the background that reruns
330. 10:27and rechecks that the connector is
331. 10:28working. If the connector does not
332. 10:30working, then you're not getting new
333. 10:31data into your engram, and then
334. 10:33obviously all of it fails. This
335. 10:35obviously no reason to be using this
336. 10:36thing if it's not catching up with fresh
337. 10:37data on a regular time period. You will
338. 10:40find things like, I don't know, API keys
339. 10:41will get rotated, um some permissions
340. 10:43will change, some routes won't work
341. 10:45anymore, whatever. So, just make sure
342. 10:46that you have something in your script
343. 10:47to constantly update it. And yeah, there
344. 10:49you go. Now you have the exact same
345. 10:50second brain system that I and many
346. 10:52multi-billion dollar companies out there
347. 10:54use. This is initially built on top of a
348. 10:56second brain system I read about from
349. 10:58Cohere. Um they make obviously large
350. 11:00language model products as well as, you
351. 11:02know, actually build RAG systems. I saw
352. 11:05their ingestion pipeline, and I thought
353. 11:07to myself, how the hell can we add this
354. 11:08in our company? And I actually ran some
355. 11:10tests comparing it to like pre-existing
356. 11:12RAG approaches that I had attempted. Saw
357. 11:14it wasn't even close. I was like, "Okay,
358. 11:15we need to dive really, really deep into
359. 11:17this." If you guys want a prompt that
360. 11:19lets you rebuild this whole thing in a
361. 11:21copy-paste, um maybe you'll have an
362. 11:23easier time doing that than just copying
363. 11:25the transcript of this video, just click
364. 11:26the second link in the description and
365. 11:28you'll be taken to Maker Zero. It's It's
366. 11:30home for literally all of my resources,
367. 11:32both you know, AI resources, broad
368. 11:34educational and informational resources,
369. 11:36curriculums, roadmaps, literally
370. 11:38everything you need is just down below
371. 11:39and it doesn't cost you a cent. Um, all
372. 11:41you have to do is join and then I think
373. 11:43you need to move over to the community
374. 11:44tab and then you'll find it under
375. 11:46recently uploaded. Copy, paste, it'll be
376. 11:48on the left hand side of the page. Aside
377. 11:49from that, thank you very much. If you
378. 11:50guys like this video, please subscribe
379. 11:52and leave a comment down below telling
380. 11:53me what you guys would like me to record
381. 11:55next. I get virtually all of my ideas
382. 11:57directly from the comments these days
383. 11:58and I really appreciate it every time.
384. 12:00Catch all y'all in the next video.