YT YouTube2Text

YouTube2Text → Transcripts → Multi-Agent Memory? 🧠 Full Hindsight Guide (Hermes + Codex)

Multi-Agent Memory? 🧠 Full Hindsight Guide (Hermes + Codex) — Transcript

by Wanderloots · 7,726 words · 1,101 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00We can see we have eight memories, and
0:01click refresh. Perfect. More memories
0:02have appeared. So, what just happened is
0:04I told Codex to remember something. It
0:06stored that memory inside of Hindsight,
0:08and then Hermes was immediately able to
0:10recall that. So, that's pretty cool.
0:12>> Many of us use multiple agents, but
0:14wouldn't it be nice if we had a way to
0:15share context between them so one agent
0:18knew immediately what the other was
0:19working on? No need to copy and paste,
0:21no need to re-explain everything you're
0:23working on over and over again as you
0:25switch back and forth between agents.
0:27Hi, my name is Callum, also known as
0:28Waterloo Lutes, and welcome to today's
0:30video on how to connect a single memory
0:32tool to multiple agents. I'll be using
0:34Hindsight for memory and Codex and
0:36Hermes for the agents, but this system
0:38works for any agent. My goal is to show
0:39how cross-agent memory systems work.
0:41Even if you don't use Hindsight, Hermes,
0:43or Codex, you will still learn a lot
0:45about how to structure a memory system
0:46for multiple agents effectively. In
0:48today's video, we'll do a full setup and
0:50practical deep dive of Hindsight,
0:52connect to Hermes and Codex, organize
0:54everything with memory banks and tags,
0:55and test memory moving between agents
0:57automatically. Now, let's sync our agent
1:00memories.
1:04If you want to learn more about the
1:05theory of the memory system and how to
1:06set them up in an agent like Hermes,
1:08check out my previous video that
1:09explores the full stack. Today, there's
1:11only two things you need to understand
1:12to get started. The system itself and
1:15how it works within a chat. For the
1:16system, imagine you have a private
1:18studio that has multiple workstations.
1:20At one workstation, Codex is helping
1:21with code. At another, Hermes is acting
1:23as a personal assistant. Each
1:25workstation combines a model which
1:26provides the intelligence with a harness
1:28which provides the tools and working
1:30environment. But, the agents are missing
1:31a way to pass information and context
1:33between each other. Hindsight acts as a
1:35bridge between them. A memory system
1:37that operates like a shared notebook
1:39passed between your agents. Hindsight
1:41extracts key information from chats with
1:42both agents and organizes it so it's
1:45readable by both when they need it. Now,
1:46let's get this set up for multiple
1:48agents.
1:50If you find this video helpful, please
1:51consider liking, hyping, and subscribing
1:53as I really appreciate your support. If
1:55you're looking for more ways to support
1:56me, please consider joining my YouTube
1:58or Patreon memberships. I give my
1:59members access to insights, tips, and
2:01kits to improve their knowledge
2:03management and AI, including the
2:05Hindsight memory kit that I put together
2:06today to make this whole process easier
2:08to set up for you. You can still do
2:10everything I'm showing you today
2:11yourself, but the kit just makes it a
2:12little bit easier if you're starting
2:13from scratch to help save you some time.
2:15Okay, so this is what Hindsight looks
2:17like here. This is what we're going to
2:18be setting up and connecting to multiple
2:20agents. So, in my last video, I set this
2:22up just for Hermes, and today we're
2:23going to set up a new bank that's a
2:25global user, which is what Hindsight
2:27recommends specifically for a single
2:29user, solo developer, someone who's
2:32using multiple agents for related
2:34projects. So, that's what we're going to
2:35get set up with the goal being that
2:37instead of just having Hermes connect to
2:39this bank producing these memories here,
2:41I can have Codex and Claude code and
2:42Gemini and any agent I want to
2:45connecting here. So, let's get this set
2:46up. So, the first thing I've done here
2:48is I've created a Hindsight memory kit
2:50folder, and this is just a folder for me
2:52to organize all of the files that I'm
2:53going to use to automatically set up
2:56Hindsight and connect it to my different
2:57tools. So, I'm not going to go through
2:59all of these files right now, but this
3:01includes a few extra things that are
3:03completely optional if you want to do
3:04this. For example, I'll talk about
3:06setting up automated backups of your
3:07memory system, one-click launchers to
3:09restart your memory system if it shuts
3:11down, how we can use something like
3:13Docker Compose to make this a lot easier
3:16for you to run continuously and not have
3:18to configure every time, and a few other
3:20things that I'll get further into. For
3:21now, let's move into this memory kit.
3:23And next, I'm just checking to make sure
3:24I have all of the prerequisites that I
3:26need, so I'm going to be using Docker
3:27today, Python, Codex, and Hermes. So, we
3:30can see that everything is installed
3:33here. If you're not sure how to get any
3:34of this going, I do have a video that
3:36goes deeper into each of them, and I
3:38also ran Hermes memory provider just to
3:39show you that the current memory
3:40provider is Nemesis, which is one of the
3:43tools that I showed you how to install
3:45in the last video. We're going to be
3:46setting this up for Hindsight today.
3:54Okay, so the first thing that we need to
3:55create is an environment file, which we
3:57can see just popped up on the side here,
3:58and I'm just going to open it right here
3:59so you can see what this looks like. And
4:01the reason I want to do this is because
4:02if we go back to Hindsight for a second,
4:05we can see that this Hindsight instance
4:07is running on localhost 999, like we can
4:10see I currently have it running with the
4:11Docker container I set up last time, and
4:13I can limit access to this Docker
4:15container, limit access to this control
4:17plane by introducing a shared API key
4:20and a control plane access key. So, I'm
4:22going to introduce secrets here that
4:25then get saved inside of this dot
4:27environment file, and that means that
4:28when I restart Hindsight, I'll be able
4:31to have these secrets that exist here.
4:33So, I'm going to create a secret. I'll
4:34just show you what this looks like and
4:35then I'll clear it. So, I would just
4:36copy this, go over here, and then
4:38replace it with this long string, and
4:41save the file. So, you can do that here
4:43and here. Okay, so once we've done that,
4:44we now have this dot environment file,
4:46and this has the API key that we're
4:48going to give to Hermes Codex and Claude
4:51Code and any other connector you want
4:53for your agent, and then another one
4:54that protects the control plane. So, now
4:56our next step, I'm going to open up the
4:58Docker Compose file, and there's a few
4:59things that I want to show you.
5:02So, the first is that we need an LLM to
5:04power the way that Hindsight actually
5:07saves the information and reflects on
5:08it. Today, I'm going to quickly show you
5:10how to use OpenAI's Codex
5:11authentication. In the last video, I
5:13walked through how to do this with the
5:14local model using Ollama. So, this is
5:16where you can set the specific model you
5:18want. So, for example, I'm planning on
5:20using Tara as the default, but then I'm
5:22going to override it for retain with
5:24Luna, which is a more efficient one. So,
5:26retaining is saving or remembering, and
5:28then I can use a more powerful model for
5:30consolidation and reflecting. So,
5:32there's a lot of settings that we can go
5:33through here. I'm also leaving this as a
5:35local embedding, so this is for
5:36retrieval of information for recall. And
5:39the most important one for setting it up
5:41with Codex is that we need to set up a
5:43new environment for the authentication,
5:45and this is because if you want to use
5:46your existing Codex quota for powering
5:48these models, we need to set up a
5:50separate authentication for Hindsight to
5:51connect to. So, that's what this is
5:53showing here. So, I'll show you how that
5:54works in just a moment. And then we have
5:56two ports at the top. The first one
5:57creates the actual container that runs
5:59Hindsight, and the second one here shows
6:01the actual control plane. This is where
6:02we can see the existing memories, and we
6:04can configure a lot more about it that
6:06I'll touch on in a moment. And the last
6:07thing that's important is we have this
6:09setting here of restart unless stopped.
6:11So, this means that even if our computer
6:12shuts down, once Docker opens again, it
6:15will restart automatically. We don't
6:16have to manually configure the setup of
6:19the Hindsight. Okay. So, then this shows
6:20the Docker Compose file, which you can
6:22also run manually yourself every time if
6:24you want to. I was just thinking that it
6:26will be easier for me to set this up
6:28once, so I have this file, and then I'm
6:30just going to create a single launch
6:32Hindsight command that just becomes
6:33basically a shortcut for me to trigger
6:35this. So, now we just want to make sure
6:37that that Compose file is recognized,
6:39and that the environment file here can
6:40be seen. And if we don't get an error,
6:42then we're good to go.
6:45So, the goal here is to have Codex power
6:47Hindsight's internal memory processing,
6:49but then give the service of Hindsight
6:51its own durable login, so it can't
6:53interfere with the Codex app I'm already
6:55using anyways. So, the first step is to
6:57create a Docker volume that's actually
6:59going to let us have a dedicated Codex
7:01authentication system for Docker for
7:04Hindsight. And this is a completely
7:06separate storage area that is different
7:08than the memory database, it's different
7:10from normal Codex, it's different from
7:11all the files. This is purely just to
7:13get the authentication going, so we can
7:14sign in to Hindsight using our Codex
7:17login. Our next step is we are going to
7:19actually create the authentication
7:21inside the volume, and then click run.
7:23This is going to start the node
7:24container. It's going to download the
7:26official Codex CLI inside of it, so that
7:28we can then actually authenticate Codex.
7:31Okay, great. So, that worked. It gave us
7:32the link that we can sign in to verify
7:34this particular account, and then I'm
7:36going to take this code that it gave us,
7:38and click continue. And just a quick
7:39note, if this fails for you, you might
7:42need to change your settings inside of
7:43Chat GPT to enable device code
7:45authentication inside of Codex. Great,
7:47there we go. So, it says successfully
7:48logged in. So, now I'm going to quickly
7:50check to make sure that we have logged
7:52in correctly. And here we go, it says
7:53Codex login is ready. So, we basically
7:55just created a separate volume that has
7:58authenticated my Codex login for me to
8:00be able to now run Hindsight through my
8:02Docker container leveraging the Codex
8:05login. Again, if you want to, you can
8:06run this locally like I did in the last
8:08video or you can also just input your
8:11API key directly if you want to. If you
8:13want to do a pay per use, I just wanted
8:15to sign in using my existing Codex quota
8:17just because why not. So, now that we
8:19have Hindsight connected to Codex, so we
8:21have the LLM ready to power it, it's
8:23time for us to actually launch
8:24Hindsight. So, let's do that now.
8:30So, in the last video, I launched
8:32Hindsight just by typing in run Docker
8:34command and that's what created this
8:35here, which we can see has files, logs.
8:38This is all the information. This is
8:40where we get to the actual memory system
8:42here where we can connect to our memory
8:44bank like I was showing you. But if
8:46you're a little less technical or you're
8:47like me and I don't want to have to save
8:50that Docker command somewhere every
8:51time, we can just turn this into a
8:53single button here that will make it
8:55easy for us to launch. So, basically all
8:57of this double click button is doing is
8:58it's running a script called start
9:00Hindsight and it's running the Docker
9:02compose. So, you can also just run this
9:04by using a normal Docker compose string
9:06as well. So, I just shut down my old
9:09Docker and now I'm going to click set up
9:10Hindsight again. Here we go, it looks
9:12like it's all good. So, now if we go
9:13back to Docker container, cool. We can
9:15see that we now have this new compose
9:17stack here. So, if I click on this, we
9:19can see that it pops up right here and
9:20it takes me to the Hindsight. So,
9:22basically all this did here is it
9:24confirmed that Docker's installed, it
9:25confirmed that the dot env file was
9:28there. If Docker Desktop wasn't running,
9:30it starts it. It then runs the compose
9:32to start the container, waits for
9:34Hindsight to appear and then opens the
9:36control plane. And we can see here it
9:38actually is blocking me from being able
9:40to access it and that's because I need
9:42to put in the control plane access key
9:43that we put into the environment file.
9:45So, if you're curious, this would be
9:47what you would normally run for the
9:49Docker Compose. I just put all of this
9:51and a couple other things into its own
9:53script. So, now we can go get the
9:54environment key that I had saved before
9:56inside of my environment file here,
9:58paste it in and click sign in and here
10:00we go. I'm signed back into Hindsight.
10:02So, it still has the old bank here, but
10:04we've now added that authentication
10:05layer by this new setup that we're
10:07running inside of Docker. So, now that
10:09we have signed into Hindsight, we have
10:11our API connected, we've got our LLM
10:13connected. Our next step is to create a
10:15new memory bank for multiple AI agents
10:17to connect to.
10:20So, if you're in Hermes, for example,
10:21this is the one that I set up
10:22specifically as a Hindsight profile that
10:24I'm now going to be changing to be my
10:26default. My default is currently
10:27Mnemosyne. You can see here that I have
10:29the bank ID of Hermes. So, you don't
10:31have to create this bank directly inside
10:34of Hindsight itself where we see create
10:36new bank, it will automatically create
10:38it if you add it in here. But, why don't
10:40we create it here so that it we can just
10:42see what this looks like. You can import
10:43it from a template. I'm just going to
10:45call this one Wanderlust for my global
10:47user memory bank. Click create and there
10:49we go. So, we can see this one has no
10:51memories. If we switch over to Hermes,
10:52it loads the memories. So, we're
10:54starting with a fresh bank here. And if
10:56you want to learn more about should you
10:57have one or more banks, for example, the
10:59ability to auto create a bank for every
11:02coding project you're working on, I
11:04recommend checking out this article on
11:05the one bank or many strategy. Hindsight
11:08honestly has a ton of good documents
11:09that have taught me a lot, but the idea
11:11here is that if you want a global bank
11:14that you use for everything, it's
11:15usually for a single user, a solo dev.
11:18That's great if you want to have all of
11:19your projects kind of melding together
11:22in one memory system. If you're dealing
11:24with multiple coding projects that are
11:26completely different from one another,
11:27you might want to have a per project or
11:29per repo bank. The key here is that a
11:31bank is a hard limit where you can't
11:33access between them. Like you saw in
11:35Hermes, I could only select Hermes as
11:37that memory bank. I couldn't have
11:38multiple banks connected to that same
11:40profile. So, you could have a different
11:42bank for each Hermes profile if you want
11:44to, kind of like a per user profile. But
11:46for me personally, I'm going to go with
11:47a global one for now because I want to
11:48have one user that all of my agents can
11:51understand and work with. And it doesn't
11:52just have to be you, either. You can set
11:54it up where, for example, I have this
11:57Hindsight container running right here.
11:59I can serve this on something like
12:00Tailscale, and I'd be able to have
12:02different people connect to this single
12:04memory system where each person gets
12:06their own memory bank. So, I can be the
12:08one running this Hindsight memory
12:09container, and my fiance Taylor can
12:12connect to it with her computer, as
12:13well. That way I can have one person
12:15powering this system, but it's all
12:17easily separated by these banks. But if
12:19you want to learn more about how to
12:20connect multiple devices to a single
12:21system, I recommend checking out my one
12:23agent many devices workflow because it
12:24works very similar for Hindsight as it
12:27does for Hermes. And I'll show you in a
12:29little bit how we can also set up a bit
12:30of a soft separation where we can add
12:34what are called tags on a per project
12:35basis. So, that involves the Hindsight
12:38configuration. So, let's go take a look
12:39at that for a second.
12:42So, one thing that's nice about setting
12:43up your memory bank first inside of
12:46Hindsight is we are able to go in and
12:49modify the configuration of this bank
12:51before any agent connects to it. So,
12:53when the agent connects to it, it
12:54already has all of these settings
12:56preconfigured. So, let's go through and
12:57set up a few things. So, I'm going to
12:59click on configuration here, and I
13:01recommend keeping your extraction mode
13:03on concise to start. That just reduces
13:05the token usage and keeps your memories
13:07a little bit more targeted. And this is
13:09in the retain section. So, if we go over
13:12to the Hindsight docs for a second, we
13:13can see that this is what retain does.
13:15You have a conversation, the agent
13:17extracts facts, identifies entities,
13:19builds connections, and then adds it to
13:21the memory bank. So, basically, what's
13:23happening here is the retain pulls the
13:25information, the context from your
13:27conversations with your agent, puts it
13:29in the server, and then adds it to the
13:31memory bank, which includes all of this
13:32information that I'll go into a little
13:34bit later. Recall does the opposite. It
13:36pulls the information from the memory
13:38bank into your AI agent's context. So,
13:41this is what recall looks like, where if
13:43you ask a question to the agent, it'll
13:44query, it'll go and pull in semantic
13:46meaning, keywords, graph, and temporal
13:48information, and fuse it into one
13:51answer, encode it, and then give you the
13:53results. So, this is why I say this is
13:55one of the more powerful systems because
13:56of this recall setup right here that
13:58brings in all of these different
14:00database mechanisms to give you one
14:02answer based on different formats. And
14:04then, reflect is a little bit different
14:05again. Reflect goes and it searches
14:08through mental models and observations,
14:10and it has a little bit more of a
14:12complex system that autonomously gathers
14:14evidence, uses hierarchical retrieval.
14:16It cites sources. There's a whole loop
14:19that it goes through here. So, I
14:20recommend checking it out if you want to
14:21learn a little bit more about it. But,
14:23for reflect in particular, you need a
14:24bit more powerful of a model usually.
14:26And there's a few more that we can get
14:27into as well, like observations, which
14:30is once you have memories retained,
14:31hindsight will automatically consolidate
14:33them into observations by deduplicating.
14:35And it keeps track of its citations, its
14:38provenance the entire time here. So,
14:39it's pretty interesting. So, that brings
14:41me back to the configuration here
14:43because not only do we want to have the
14:44extraction mode set up for concise, and
14:46we can have multiple strategies. We can
14:48have a codex one, a Hermes one. You can
14:49have different things here. But, we also
14:51have this mission.
14:54So, this is one of the most important
14:55parts. You can kind of think of it like
14:57the system prompt for your retaining
14:59information. This is a way to steer the
15:01LLM without replacing extraction rules.
15:03It's just kind of a way to improve it.
15:06So, this is just an example. I recommend
15:08tweaking it, changing it, testing it out
15:10until you get one that works really well
15:11for you. Now, the next thing I want to
15:13get to is free-form entities. So, this
15:15is where we want to have entities, which
15:17are people, places, and concepts,
15:19extracted automatically. So, I'm going
15:21to leave that enabled on, and we can
15:23also get into entity labels
15:25specifically. It's an optional way for
15:28you to produce an extra label based on
15:30the information that your agent is
15:32retaining as you're having
15:33conversations. So, one that I'm
15:35interested in, for example, let's go
15:36here. I'm going to change this to text.
15:38I'm going to call this project, and I'm
15:41going to give my extractor hint of the
15:42primary named project or code base this
15:44memory concerns. Use its stable
15:46canonical name. Omit it if there's no
15:48project clear. So, basically what this
15:50means is let's say I'm working on one
15:51coding project, and it's called project
15:53A. If the folder I'm working out of is
15:55called project A, it's possible that
15:57Hindsight will automatically retain
15:59information and label it with that
16:01project name. So, what we can do is we
16:03can toggle on tag here, and this is
16:05where if we go back over to our memory
16:06bank strategy, if a bank is a recall
16:09boundary, you can't have any type of
16:11cross-bank query. So, what we can do is
16:13we can use tags inside of one bank for a
16:15soft partition that we want to be able
16:17to filter and maybe cross-reference. So,
16:20by setting up the entity label with the
16:21project, it is possible that the agent
16:23doesn't do it correctly, but this
16:24improves the likelihood that it will tag
16:27its memories that it's remembering with
16:29the project name. And as another
16:30example, we could add a second label
16:32that maybe I want to call harness, and
16:34we could say the agent harness that
16:35produced this memory, choose only when
16:37indicated, otherwise omit. And we could
16:39change this to single value. And
16:42basically what we're doing is we're
16:43pre-labeling
16:44the possible tags that could be
16:46triggered so that as we save new
16:47memories here, we can see it has user
16:49Hermes profile, for example. We can add
16:51more tags depending on the information
16:54that's coming in and try to improve the
16:57filtration system that we want to add in
16:59the memories that are being retained.
17:01Okay, so I'm going to click save changes
17:02on that.
17:05And now we're going to go down to
17:06observations, which is where we can
17:07control how facts are synthesized into
17:09durable observations, like I showed you
17:12in this workflow right here. It's a form
17:14of knowledge consolidation. So, it's
17:16effectively a way of identifying
17:18patterns in your work, which is one of
17:19the most beneficial reasons for using
17:22something like Hindsight in the first
17:23place. Going to add a mission statement
17:25here. Again, I recommend maybe chatting
17:27with your agent, seeing what your goals
17:29are here. This is going to change what
17:30your mission is. I'm just going to leave
17:31everything as the default here. Click
17:33save changes. And then we're going to go
17:34down to reflect here. And this is where,
17:37like I was mentioning, reflect is a
17:38little bit more of a complex system that
17:40runs a loop to autonomously gather
17:42evidence and then reason through all of
17:44that and produce more contextual
17:47responses. So, it's a deeper reasoning
17:49layer that connects relevant memories
17:50without pretending uncertain or outdated
17:52memories are verified facts. So, I'm
17:54going to add this as my memory
17:56statement, specifically mentioning
17:57cross-project personal and technical
17:59memory assistant. Something like stating
18:01uncertainty is valuable. And I'm just
18:02going to go with the defaults here. You
18:04can change this again if you want. I
18:05would recommend looking deeper into it.
18:07Now, we get to the next step of the bank
18:08configuration, which is memory defense.
18:10And this basically enables you to
18:11configure sensitive content to be
18:13redacted or quarantined depending on
18:15your particular preferences. So, this is
18:16just a good idea to have in there. If
18:18you don't want your agent to remember
18:19any kind of secrets, you can just turn
18:20this on. And we can choose to redact,
18:22block, or allow. So, I'll just redact it
18:24for now. Okay, so I know that was a lot
18:26of setup here with the bank
18:27configuration, but this is a one-time
18:29thing that you need to set up. And
18:30there's a lot more that you can get into
18:32here. So, I recommend again going
18:33through the docs and checking it out
18:35because depending on your particular use
18:37case, there may be more things you want
18:39to turn on here.
18:41So, now that we have Hindsight set up,
18:43it's currently running inside of this
18:44Docker container. We have access to the
18:46control plane. We have all of our bank
18:48configuration set up. Why don't we
18:50quickly test it to make sure it works?
18:52I'm just going to click add document and
18:54I'm very quickly going to add something
18:56like this. Cool. It was able to be
18:58added. We can see it has this
18:59information. Now, if we go over to
19:01recall, I can say searching memories and
19:04here we go. It was able to restore the
19:06memory and say the Wanderloot's memory
19:07system uses one shared Hindsight bank
19:09named Wanderloot's. So, that just proves
19:11that the calls are working here. If we
19:13go down to settings and over to
19:15background operations, we can see that
19:17it was able to run right here. So this
19:19means that it's connecting correctly to
19:21the LLM setup that we have given it,
19:23which is perfect. So now before we
19:25connect this to an agent, there's one
19:26more important thing I would recommend
19:28setting up, which is to enable automated
19:30backups.
19:32So I'm going to go back to my kit here
19:34for a second where I have an install
19:35automatic backups command. So I'm going
19:37to run that. It's creating a host backup
19:39directory. It's going to run it once
19:41immediately just to make sure that it
19:42all works. And yeah, it looks like it
19:44is. I went to my folder. I have
19:45something called hindsight backups and I
19:47have a zip folder right here. And
19:48basically what it's doing is it's
19:50running the built-in hindsight admin
19:52backup and then just controlling which
19:54folder I wanted it to go to. So you can
19:56also just run that manually yourself or
19:57you can set it up to run on a schedule
19:59as well. And if you want to learn more
20:00about how you can run this yourself as
20:02part of the hindsight admin, I recommend
20:04checking out the admin CLI. And you can
20:06just give this to your agent so that
20:07it's able to set this up for you in a
20:09way that you're happy with.
20:12All right, so this is me from the
20:13future. Part way through recording this
20:14video, hindsight released version 0.9,
20:16which introduces a bunch of cool
20:18elements that I'll touch on a little bit
20:19later. So I thought this would be a good
20:21opportunity to show you how to upgrade.
20:23So the first thing I did was I used my
20:25backup system that I had just installed.
20:27So we can see that it has now 33,000
20:29memories. This is after about a week and
20:30a half. So we can see I have all of my
20:32memories, they've been backed up. That's
20:33just good practice. And next what I'm
20:35going to do is I'm going to pull
20:36hindsight. Okay, there we go. It has now
20:38completed pulling the latest version.
20:40That took about a minute. So now what I
20:41can do is I can relaunch the hindsight
20:42command. And there we go. We can now see
20:44that the user interface is a bit
20:45different than what we were currently
20:47looking at, but it was able to bring in
20:49my existing bank that I had created and
20:51now all of the memories. So again, just
20:52as we continue going throughout this
20:54tutorial, remember that some of it might
20:55be on the old user interface. There's
20:57just now a couple new features like this
20:59home button, knowledge pages, and this
21:01knowledge base. So I'll touch on that in
21:03a little bit. Now it's time to actually
21:05connect this to one of our agents.
21:12Okay, so I'm first going to connect this
21:14to Hermes because it's actually quite
21:15easy. We can go to the settings in the
21:17top right or by pressing command comma
21:19and then we can go over to memory and
21:21context. I'm going to leave the
21:22persistent memory and user profile
21:24memory on because we can do that with
21:26hindsight and I'm going to switch from
21:27Nemosyne which is more of a memory layer
21:29to hindsight here. So, when I do that it
21:31brings up the hindsight settings and
21:33this is where we have two modes, local
21:34external and cloud. I'm going to leave
21:36it as local external because I have it
21:37running inside of my docker container. I
21:39talked more about this process in the
21:41last video on the pros and cons of
21:43running it yourself versus using the
21:44cloud which is a paid service. But for
21:46now, let's leave it as local external
21:48and then we get to the API key and this
21:50is where we need that second API key
21:52that we set up inside of the environment
21:54file. So, we can go get that and now
21:55we're going to set the API URL to the
21:57locally running docker container. I'm
21:59going to go down and change bank ID to
22:01Wanderer's and I'm going to leave recall
22:03budget on mid. It's up to you to control
22:05the tokens that you want to use there
22:07and that's it. It's set. So, now I'm
22:08going to go here and I'm going to
22:09restart the gateway just to make sure
22:10that the default profile picked that up
22:12properly. Now that I reset, I'm going to
22:14go back to memory and context and
22:16perfect. It looks like it's all set up
22:17still. So, the next way that we can
22:18check this is we can go back to terminal
22:20and we can type in Hermes memory status
22:23and it should pop up. Yes, we have
22:24built-in memory enabled. The provider is
22:26hindsight. It's installed and available.
22:29Perfect. That's exactly what we needed
22:30and this is just if you're running the
22:32desktop app. If you want to do this
22:33through the CLI, you can also write
22:35Hermes memory setup and this would help
22:38you get going with connecting hindsight
22:39as your provider
23:00>> it's able to connect to the LLM that we
23:02have powering it and it was able to
23:04update the user profile and memory
23:06preference. Perfect. So now if we go
23:08back over to Hindsight for a moment and
23:10go to our bank, we can see here we have
23:13a few more memories. So we can see that
23:14it's pulling up the project of Wander
23:16Lutes. Interesting. Probably because
23:17it's inside Wander Lutes tutorials
23:19folder. We can go to entities and we can
23:21see there's all of this information here
23:22that's been extracted as that
23:24information was retained. So that's
23:25perfect. Everything is working. But
23:27that's not where we want to end it here.
23:29We don't want to just have Hermie's
23:30connected to Hindsight. The point is
23:31that we're creating this shared memory
23:33engine that connects all of our tools
23:34together. So next I'm going to connect
23:36Codex. But this process also works very
23:38similarly for Claude code as well. So
23:40the goal is that Codex is now becoming a
23:43doorway into the same memory bank. And
23:45this connection that we're about to set
23:46up is completely different than using
23:48Codex to power the Hindsight system. So
23:50we have Codex now powering Hindsight and
23:52Codex is also going to use Hindsight.
23:54But for this it's a little bit
23:55different.
24:02Okay, and now just very quickly before
24:03we actually connect this to Codex. This
24:05is me from the future again. I want to
24:06mention that with the 0.9.0 update,
24:09there's actually a new way for us to
24:11connect Hindsight that makes it easier
24:13for us to connect this single memory
24:14plugin for every single coding agent. So
24:16basically what this means is there's now
24:18a single coding agent plugin and we can
24:20automatically have the repos get history
24:22and conversations flow into the memory
24:24bank in the background while you work.
24:26So previously if we installed the Codex
24:28connection with the hooks, we can
24:30uninstall that now. So I'm going to do
24:31that quickly for a moment. If you
24:32haven't installed it yet, don't worry
24:33you don't need to do this step. There we
24:34go. It's uninstalled. And now we can go
24:36and we can get this command here, this
24:38NPX command. And you can either install
24:40all to work for every single agent
24:42that's detected in your computer at once
24:44or you can list a specific one that
24:46you're going for. So for myself, I'm
24:47just going to install this for Codex
24:48right now. But you could do this for all
24:50of them. And then I have two flags here,
24:51one for the fact that we're working on a
24:53self-hosted server and that this is
24:55where we want to launch the server to.
24:57So I'm going to click enter there.
24:58Great, there we go. It just took a
24:59second. And again, what's wild here is
25:01that this installed something called
25:03Hindsight Coding Agents. So, if I would
25:05have put all here instead of just Codex,
25:08it would connect to Antigravity and
25:09Cloud Code and all of the other ones.
25:11So, we do need to make sure that the
25:13hooks are enabled in our Codex, which
25:15I'll show you in a moment. But first,
25:17before we do that, I want to take a
25:18quick look at the Coding Agent
25:19configuration file that we just
25:20installed.
25:23So, if I go to my .hindsight folder,
25:24this is my previous one that I had
25:26installed when it was just Codex, but
25:28now we're dealing with the full system
25:29here. So, we can see as part of this
25:30Coding Agent installation, we have all
25:33of the different operations needed for
25:35every Coding Agent. So, the
25:36configuration here is at
25:38.hindsight/codingagent.json,
25:40and this brings in the built-in
25:41defaults, but also gives you the ability
25:43to control the various levels. So, I'm
25:45going to open this now for a moment, and
25:47we can see that the built-in one here
25:48has just server mode self-hosted and the
25:50API URL. That's what we set when we
25:52first started up. So, I'm now going to
25:54introduce a few more layers here. I'm
25:56going to mention that we're dealing
25:58specifically with the bank ID. I'll put
26:00in the shared private key that we
26:01created in a moment here. And I also
26:03want to have dynamic bank ID set to
26:05false. This is just my preference right
26:07now because I want to have multiple
26:09projects sharing the same memory because
26:11the projects are related. If you wanted,
26:13you could have a different bank created
26:15for every single agent, which I'll talk
26:17about a little bit more later. You just
26:18need to then change your private key,
26:20which I'll do now, and then click save.
26:21And now we can move over to Codex.
26:24So, if we go to Codex settings and then
26:26go into hooks, we can see that we have
26:28three hooks that need review. So, these
26:30are the lifestyle hooks that were
26:32automatically pulled from the Hindsight
26:33configuration file. The hooks do three
26:35things. They automatically initialize
26:37memory, they recall relevant context,
26:40and they retain the completed session.
26:42If this is the first time you've
26:43installed it, this might look a little
26:44bit different. But like I mentioned, I'm
26:45updating this from the previous version,
26:47so I need to re-trust the changed files
26:50because this can run outside the
26:51sandbox, so we need to give permission.
26:53And then now I can quit and reload
26:55Codex. And alongside the hooks, what was
26:57also installed was the MCP server for
26:59Hindsight. So basically what that means
27:00is if we go over to plugins, we now have
27:02this Hindsight section installed as a
27:04plugin that gives Codex even more tools
27:07that it's able to run to work with
27:09Hindsight. For example, it helps you
27:10with bank selection. You can run
27:12specific actions. There's a whole list
27:14of available tools that come with the
27:15MCP, so I recommend checking out the MCP
27:17server documentation if you want to
27:19learn more about that.
27:21Okay, so now that I have it all
27:22configured, I've trusted all of the
27:24hooks here, we should be able to give it
27:26a prompt and have it remember something.
27:28Please remember that Project Memory Test
27:29uses purple as its temporary test color.
27:31All right, so we can see here we have
27:33these hooks that are enabled. Now if we
27:34go back to our Hindsight, we can see we
27:37have eight memories. And now if we go
27:38back here and click refresh, we can see
27:39perfect, more memories have appeared.
27:41And these ones have the context of
27:42Codex. We can see here the Project
27:44Memory Test temporary color is purple.
27:46Okay, so now let's test it inside of
27:48Hermes. What temporary color did I
27:49choose for my Project Memory Test? So we
27:51can see that it triggered Hindsight
27:53recall and it shows purple as the
27:54temporary color for Project Memory Test.
27:56That's perfect. So what just happened is
27:58I told Codex to remember something, it
28:00stored that memory inside of Hindsight,
28:02and then Hermes was immediately able to
28:04recall that. So now let's test it the
28:06other way. As a temporary test, can you
28:07please remember that my favorite animal
28:09is a lemur. So now we're triggering
28:11Hermes Hindsight retention. It just
28:13added a temporary test fact, and if we
28:14go back here and click refresh, we're
28:16going to have a few more memories. Now
28:17we have 21 total memories. So some of
28:19this is coming from these temporary
28:21tests that I'm asking. Some of them are
28:23being created because of just the
28:24questions I'm asking Codex and Hermes.
28:26But we can see here is my user's
28:28favorite animal, which I do love lemurs.
28:30They're such cool animals. I very much
28:32enjoyed seeing them in Madagascar. So
28:34now I'm going to go back to my Codex
28:36chat and say, "What is my favorite
28:38animal?" And we can see here it
28:39immediately pulled in, "Your favorite
28:40animal is a lemur," noted as a temporary
28:42test preference. So you can see just by
28:43having these basic conversations, this
28:45information is beginning to expand quite
28:47a lot. We now have 22 memories in here,
28:49and this is just the beginning. We're
28:51going to see experiences appear here.
28:52Under observations, here is also where
28:54we can get into that filtration by tags.
28:56So, you can search, but we can also
28:57filter by for Harness Hermes, and we can
29:00see all of the memories that were
29:01created by Hermes specifically. We can
29:03clear that, and we can switch over to
29:04Harness Codex, and we can see the
29:06memories that were created specifically
29:07by Codex. So, this is where, going back
29:09to that configuration, you can think of
29:11more tags that you would like to have
29:13here. Hindsight's going to continue
29:14creating those free-form entities, but
29:16you can also structure your own data in
29:19a way that you want to be able to filter
29:20by in the future.
29:27So, now that we have some memories in
29:28here, we can also run something like
29:29reflect, which we can do either directly
29:31from in here, or we can do from Hermes
29:33itself. So, I can ask Hermes, for
29:35example, "Can you please reflect on my
29:36workflow concisely?" And we can see here
29:38that it's running Hindsight reflect. And
29:40this is where it goes through and
29:41autonomously gathers evidence and
29:43reasons through the overall bank to give
29:45you a better contextual response. So,
29:47this can take a little bit longer. I
29:49have this set up on a slightly more
29:50powerful model, but we can see here,
29:51"Your workflow is durable memory first.
29:53Hindsight holds cross-session facts,
29:55decisions, and all this information."
29:57So, this is a good observation on how my
29:59system is currently working. And we can
30:01see that that reflection was now stored
30:03as an observation. So, this is where the
30:05agent gets a little smarter as it
30:06continues working, as it learns from my
30:08process and what I'm hoping for. And
30:10there's honestly so much more that we
30:12can do with this. Every conversation,
30:14every message, every chat we have with
30:16any agent that's connected to this
30:18shared memory bank is going to operate
30:19on the same memory layer, and then
30:22Hindsight will operate as the engine
30:23that's going to consolidate knowledge in
30:25different patterns and preferences that
30:27helps each agent that you connect to
30:29understand you and what you're working
30:31on a little bit better. So, that's
30:32pretty cool. This could be done for
30:34Claude Code, for Codex, for Gemini, for
30:36any agent that you want, for Hermes, for
30:38Open Claw, and it's all sharing this one
30:40single user profile, a global memory
30:43system.
30:45Okay, this is me from the future again.
30:47I now have worked with Codex and Hermes
30:50to produce tons of memories here over
30:52the last little while. This is what it
30:53looks like, which is pretty awesome. And
30:55can now show you a new feature that came
30:57in with version 9, which is knowledge
30:58pages. So we can see here on the side I
31:00have no knowledge pages yet. I can click
31:01create my first page. So basically what
31:03happens is we have all of the memories
31:04operating in the background. The
31:05knowledge pages are projections from
31:07that data that automatically update over
31:09time. So there's more information here
31:11on why it works and how to get it going.
31:12And I do want to mention that this is
31:13different than the LLM wiki I've talked
31:15about in previous videos because this is
31:17all automatically built and projected
31:19from agent memories, which means it
31:21doesn't have a true trust layer built
31:23into it like I talk about in my LLM wiki
31:25update video. This is instead a way for
31:27you to help produce mental models that
31:30automatically get projected from real
31:31files and it gives you an ever evolving
31:34summary of components, concepts,
31:36conventions, decisions, current
31:37initiatives. And for me because I'm
31:39operating on one bank here, all of these
31:41pages that I'll build inside of the
31:43knowledge area can connect my knowledge
31:44across related projects. to link them
31:47specifically by particular projects, I
31:48could do that as well. But this is
31:50effectively agent maintained working
31:52context, not a replacement for my human
31:54reviewed documentation like I have in my
31:56Obsidian system. So why don't we try
31:58building one?
32:00I'm going to try and create a new page
32:01here that's going to summarize my agent
32:04memory philosophy with the query of what
32:06is the method of operating with this
32:07Hindsight memory bank for Hermes and
32:09Codex agents, the two that I have
32:10connected here. So you can see it's
32:11generating content. We can pull in the
32:13prompt right here. We can open the file
32:15and we can get more details on the
32:16configuration and how it's automatically
32:19refreshed over time. And we can tweak
32:20all of the configuration and we can see
32:22the history here. There we go, it just
32:23finished. So we can see that the
32:25operating philosophy is to use Hindsight
32:27as the shared external memory provider,
32:29not as the harness or model. It has my
32:31understanding of the bank boundary that
32:33we just set up today and some details on
32:35how we would have a normal session flow.
32:37So basically, what this has done is it's
32:38taken all of the memories of everything
32:40that I've talked about so far and
32:42extracted from those 33,000 memories a
32:44particular format of the conversations
32:46I've had with Codex and Hermes to
32:48understand how I want to operate. It
32:51also default built in some of these core
32:53concepts, initiatives, and enhancements.
32:54So, these are something that I could
32:56modify and update over time. That's all
32:58being automatically projected from the
32:59memories I already have in my memory
33:01bank. So, I think it's pretty cool that
33:03we can have this system built here.
33:04Again, I don't personally think at the
33:06moment that this will replace the LLM
33:08Wiki that I'm building in Obsidian
33:09because that has a much greater trust
33:11layer and approval process of what makes
33:13it into the memory. But, I talk about
33:15that more in my other video if you want
33:16to learn how this system can work
33:17synergistically with the one we've just
33:19set up here and with the knowledge pages
33:21that plug into my existing LLM Wiki.
33:28We now have a single memory system that
33:30connects multiple agents together. But,
33:32this is just one layer in the memory
33:34stack. An automated system that's in the
33:36control of your agent. We still need a
33:38trusted human-in-the-loop knowledge
33:40system that helps combat AI errors and
33:42hallucinations. That's what we'll get
33:43into in the next video. We're building a
33:45trusted Wiki in Obsidian, an agentic
33:47librarian that manages your trusted
33:49knowledge separate from your automated
33:50memory system. A reminder to check out
33:52the free Patreon article that I'll link
33:53in the description and also that members
33:55get access to the hindsight memory kit
33:57I've put together to help you set up
33:59your system more easily. If you found
34:00this video helpful, please like and
34:01subscribe. I really appreciate it.
34:03Thanks again for watching and I will see
34:04you in the next video.

About this transcript

This page contains the full transcript of Multi-Agent Memory? 🧠 Full Hindsight Guide (Hermes + Codex) by Wanderloots, generated from the public captions YouTube serves with the video. The transcript has 7,726 words across 1,101 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.