Agent Memory EXPLAINED - Complete Architecture — Transcript
Full transcript
- 0:01Good morning everyone. If you want your
- 0:03agent to remember things across
- 0:04conversations about yourself, about
- 0:06itself, and what it learns about the
- 0:08world, you're going to need an agent
- 0:10memory system. And today, that's what
- 0:13we're going to be explaining. First of
- 0:14all, we're going to see what agent
- 0:16memory systems are. And then we're going
- 0:18to be taking a deep dive into one
- 0:20existing very popular memory system
- 0:22called MEM0. We're going to explore what
- 0:25stores it uses to create its full
- 0:28long-term memory. We're going to explore
- 0:30the workflows for ingestion and
- 0:32retrieval of the memories that it
- 0:34creates. And we're going to take a look
- 0:35at what it does when it just wants to
- 0:37delete one memory. And finally, I'm
- 0:39going to explain to you how you can
- 0:41build this yourself if you want to build
- 0:42it your own. And how you can run the
- 0:45whole thing with your own local models
- 0:46and some recommendations of local models
- 0:48that will be useful on every step of the
- 0:51whole thing. So without any further ado,
- 0:54let's get right into it. All right. So
- 0:57the first thing to talk about is what
- 0:59actually is long form or long-term
- 1:02memory for agents. And something to keep
- 1:04in mind is that this is not the same
- 1:06thing as conversational memory. So
- 1:09remember that LLMs are stateless
- 1:12machines. In other words, when you send
- 1:14the prompt to your LLM, your LLM is
- 1:16going to process it and give you a
- 1:19completion. If you send another prompt
- 1:21to it after that, it will not remember
- 1:23anything about the previous prompt or
- 1:25the previous completion. So, if you want
- 1:28your agent or your your assistant to be
- 1:31able to remember what it said before,
- 1:33you're going to have to send over the
- 1:35entire history of the conversation with
- 1:38your prom so that it knows what it
- 1:40talked about before. And that is
- 1:42essentially conversational memory. This
- 1:44is what happens actually when you create
- 1:46an agent. So an agent scaffold or an
- 1:48agent harness will handle that context
- 1:51for you. In other words, whenever you
- 1:53send a message to your agent, your agent
- 1:55will process it with an LLM. Then we'll
- 1:58get the response back to from the LLM
- 2:01and will append the response both your
- 2:03message and the response to the history
- 2:06that it keeps in its state. And then
- 2:09whenever there is a new message, it will
- 2:11add the new message right here and send
- 2:13the whole thing back again to the LLM.
- 2:16That's the way it will remember things
- 2:18about its conversation. And this way
- 2:20when you mention something right here,
- 2:22the agent will remember it right here in
- 2:25the conversation. But this is
- 2:27conversational memory. If you only have
- 2:30this and you, for example, you start a
- 2:32new conversation and you send a new
- 2:34message, your agent will not remember
- 2:37anything from this other conversation
- 2:39because it is in another context. So
- 2:41what we do when we want to create long
- 2:43form memory is we create another service
- 2:46right here which is going to be the
- 2:47long-term memory. And this one right
- 2:50here is going to store things from every
- 2:53single conversation you have. And on
- 2:55every turn of your agent, it is going to
- 2:57check its long-term memory to be able to
- 3:00respond with facts that it knows about
- 3:02you and about itself which will be
- 3:04stored in the long form memory which
- 3:06will be independent from the
- 3:09conversational memory. Okay. And what
- 3:11we're going to be doing today is we're
- 3:13going to be taking a look at how this
- 3:14long for memory works in mem which is
- 3:17one very popular example of long-term
- 3:20memory implementations. And we're going
- 3:22to take a look at the entire
- 3:24architecture and how it works. So let's
- 3:27get right into that. And before I
- 3:29forget, just remember that for long form
- 3:31memory to work, you're going to have to
- 3:33make it external to your conversation.
- 3:36So it should not be within the same uh
- 3:38file or the same store as the session
- 3:40itself. Uh it [snorts] should be
- 3:42persistent across sessions. In other
- 3:44words, it should be at a different place
- 3:47from where you store your sessions. And
- 3:49this also means that you can also use
- 3:51the same memory for multiple agents if
- 3:54you're going to have it centered on the
- 3:56user. So you have the same memory for
- 3:58one user like for example myself and my
- 4:01tastes and my current projects that I'm
- 4:03working on and have that memory be
- 4:05shared among different agents. So for
- 4:07example with JHPT with claude with PI
- 4:10etc. [snorts]
- 4:11And then ideally you want to give your
- 4:13agent the tools to explore its own
- 4:16memory or at least you want to do
- 4:18ingestion and retrieval automatically on
- 4:21every agent turn. Okay? [snorts] or at
- 4:24least at after each conversation. All
- 4:27right, so with that out of the way,
- 4:28let's take a look at which stores Meim
- 4:31Zero uses. And let's just store it like
- 4:33this. Let's say stores. And it uses
- 4:36three main stores. Okay, so the first
- 4:39one is going to be the main memory. And
- 4:43this one is going to be the main uh
- 4:45place where all the memories are going
- 4:47to be stored. So the memories
- 4:48themselves, the short paragraph or short
- 4:51sentences that contain the memory itself
- 4:54are going to be stored right here. So
- 4:57the memories plus their metadata. A
- 5:00memory can be something like for example
- 5:02the user prefers vegan restaurants and
- 5:05that memory is going to contain a bunch
- 5:07of metadata associated to it and it's
- 5:09going to be in this uh vector store as
- 5:11well. Uh the metadata can include things
- 5:14such as the date when the memory was
- 5:17created. Uh whether the memory belongs
- 5:20to the user or the agent. So if it's
- 5:23information for example about the user
- 5:25as we mentioned about the user
- 5:27preferring vegan restaurants for example
- 5:29or about the agent for example learning
- 5:32about current events or about a new way
- 5:34to perform some procedure etc. It's also
- 5:37going to contain the hash of the memory
- 5:40itself. And this is used strictly for
- 5:42the duplication. We're also going to
- 5:44have the leatized version of this. Um,
- 5:47and this one right here is used mainly
- 5:50for keyword uh search uh which we're
- 5:53going to be explaining in a bit. And um
- 5:55after that, it also includes the dates
- 5:57such as when it was the dates when it
- 6:00was created and updated. And it also
- 6:02create uh mentions the expiration date
- 6:05and it will only have also have the
- 6:08attribution of who created this this
- 6:11memory. So for example if you're using
- 6:12multiple agents which agent created this
- 6:15memory and that's the main memory store
- 6:17the vector store and on top of that we
- 6:20have another store called the entity
- 6:22store or the entity memory and what's
- 6:25going to happen is that me zero is going
- 6:28to extract the entities from the main
- 6:30memory. So the places, the proper names,
- 6:34the people, etc. And it's going to save
- 6:37them in this separate vector store. So
- 6:40this one right here is also a vector
- 6:42database, but every point in that vector
- 6:45database is going to be an entity. And
- 6:48each entity, we're going to have in its
- 6:51metadata that it's linked to one or more
- 6:55main memories. So for example, if my
- 6:57memory is that my favorite neighborhood
- 7:00in Paris is Mat, then the entity is
- 7:04going to be Mat and Paris and whenever I
- 7:07am asking a question about Paris, it
- 7:10will retrieve the entity about Paris and
- 7:13it will see all the memories linked to
- 7:16that entity and among which there is
- 7:19going to be my memory about my favorite
- 7:22um neighborhood in Paris. Okay, that is
- 7:24the second one. And the third one is a
- 7:27SQLite database which is used for two
- 7:31things. First of all, basically a
- 7:33logging system to keep track of all the
- 7:36history of all the changes in your
- 7:38vector stores. And second, it's going to
- 7:41keep the latest 10 messages that were
- 7:45sent to the pipeline so that it has
- 7:49access to those um most recent messages
- 7:52to figure out what the user is talking
- 7:54about when creating a new memory. And
- 7:56I'm going to explain to you a little bit
- 7:57more about this in a moment. So those
- 7:59are the three different stores that we
- 8:01have. Let's take a look at how Mezero
- 8:04uses them in ingestion and in retrieval.
- 8:07All right. So now we have a much better
- 8:09understanding first of all of what is
- 8:11long-term memory for agents and also uh
- 8:14in this particular example which are the
- 8:16stores that are used by our agent. Now
- 8:19let's talk about the ingestion part
- 8:21which is the part that is going to run
- 8:22when the agent wants to save new
- 8:25memories. Okay. So um the ingestion part
- 8:28is going to be executed or it's going to
- 8:30be run after every agent turn. So when
- 8:33you ask the agent to do something, it
- 8:35will continue the task, finish it or
- 8:38when it finishes one turn, it will take
- 8:40those messages produced by that turn and
- 8:43they will be sent to this pipeline.
- 8:45Okay, so the input in this case, it's
- 8:47going to be messages. And just uh for
- 8:51reference, there are three main ways in
- 8:54which those messages can be turned into
- 8:56memories in your database. The first one
- 9:00is going to be procedural memory.
- 9:04And this way of ingesting the memory is
- 9:07going to uh take a list of messages and
- 9:10summarize the process or the actions
- 9:12that the agent took so that in the
- 9:15future it is able to retrieve this
- 9:17memory and reproduce that exact
- 9:19procedure. This is useful if you want to
- 9:22if you finished the a if the agent
- 9:24finished doing something and you wanted
- 9:26to remember exactly what it did with
- 9:28which tool calls it uh made and which
- 9:31results it got. Uh this is the one that
- 9:34you probably want to use. But [snorts]
- 9:36uh in general I don't really uh feel
- 9:38like it's very used anymore precisely
- 9:40because you can just extract a scale
- 9:42from a from a a conversation transcript
- 9:45and that works much better. Um, the
- 9:48second way is to set the infer parameter
- 9:51to false. And what this is going to do
- 9:53is that it's going to take all these
- 9:54input messages that you're taking
- 9:56sending into the pipeline, embed them,
- 9:58and put them straight into your
- 10:01database. Just like that, the messages
- 10:03themselves. Okay. And the third way is
- 10:06to set infer equal to true. And this one
- 10:10right here is going to trigger the
- 10:12extraction with LLM pipeline, which is
- 10:14the more sophisticated and more advanced
- 10:16way to uh do memory for long-term
- 10:20agents. And this is the one that we're
- 10:22going to be focusing on today. Okay. So
- 10:26let's suppose that you have set infer to
- 10:28true and you send the input messages to
- 10:31the ingestion pipeline. What is going to
- 10:33happen after that? So the second step
- 10:35right here is going to be to load
- 10:39recent
- 10:41context. And this includes a bunch of
- 10:44things because this extraction is going
- 10:45to happen with an LLM. So what's going
- 10:47to happen is that you're going to send
- 10:49all these messages to an LLM
- 10:53and the LLM is going to export
- 10:56a structured JSON with all the memories
- 11:00that it extracted from it. Okay. And in
- 11:03order to do that, the LLM is going to
- 11:05have a very specific prompt and a very
- 11:07specific context about the user and
- 11:10about what they are talking about so
- 11:12that it is able to extract relevant um
- 11:15memories from those messages and this uh
- 11:18context includes a bunch of different
- 11:20things. So let me tell you the first of
- 11:23all um let me just give you a tour on
- 11:25this prompt. The prompt includes just
- 11:27the role of the LLM which is going to be
- 11:31memory extractor. Then also includes a
- 11:34summary of the user. So user summary. It
- 11:38is also going to include the messages of
- 11:40course that we are adding right here as
- 11:43input. It is also going to include the
- 11:45recent memories
- 11:48that were saved to the database so that
- 11:50we know what we're talking about and
- 11:52maybe some relevant memories that come
- 11:53into play. right here, it's going to
- 11:55include relevant memories. So that means
- 11:58that it is going to look into the
- 11:59database to find relevant memories to
- 12:02these messages. And this one right here
- 12:04is actually done by just flattening the
- 12:07messages. So turning the messages into a
- 12:09single uh string, embedding that list of
- 12:12me that that messages and searching the
- 12:16existing database for relevant memories
- 12:19with this particular um that are related
- 12:22to that message. And those relevant
- 12:24messages are also going to be appended
- 12:26to context. And it is also going to
- 12:28include the last n number of messages.
- 12:31Ideally some these are the 10 latest
- 12:33messages. So last 10 messages. And these
- 12:36ones right here are going to come from
- 12:38the SQLite database that we mentioned
- 12:40before. So every time this uh pipeline
- 12:43is triggered, it is going to save that
- 12:45message into the SQLite database and
- 12:49it's going to keep only the 10 latest
- 12:51messages. And this one right here is
- 12:53very useful to identify and to figure
- 12:56out what pronouns mean for example in
- 12:58that particular new message. So for
- 13:00example, let's suppose that we sent a
- 13:03new input messages a new input message
- 13:05that includes things like it is great or
- 13:09he's really good at this. Those it and
- 13:12he don't really have context about what
- 13:15they mean unless you send the past few
- 13:17messages so that the this LLM knows what
- 13:20you're talking about. So this one right
- 13:21here comes to comes from SQLite and this
- 13:24one right here come from the main vector
- 13:27DB. Then on top of that it lists the
- 13:29conversation date and the current date
- 13:32and this prompt is going to export this
- 13:35very nice JSON which is going to be the
- 13:37memory that we're going to be importing
- 13:40into our system. Okay. So let's suppose
- 13:42that we now have passed this and as a
- 13:45result we got something like the user
- 13:48prefers
- 13:50vegan
- 13:52restaurants and this is going to be the
- 13:54memory of the user. Okay, that's just an
- 13:56example. Now what's going to happen is
- 13:58that this memory is going to be saved
- 14:01into our main memory store. It's going
- 14:04to be added. It's going to uh get the
- 14:06date where it was added or updated. Uh
- 14:09whether it's from the user or the agent
- 14:11as we mentioned before. It's going to be
- 14:13hashed and uh dduplicated against the
- 14:17existing database. And of course, since
- 14:19we're using hash duplication in this
- 14:21particular case, it only works for
- 14:24memories that are the exact same
- 14:26wording. So probably not the most useful
- 14:29to dduplication system that we have, but
- 14:31that's what we have right here. and it's
- 14:33also going to contain its leatized
- 14:35version. So this sentence right here is
- 14:38going to be leatized and it's going to
- 14:39be saved alongside the memory for uh
- 14:43keyword search later on. And that is how
- 14:45it's going to be stored into the main
- 14:48main vector
- 14:51DB and also into our SQLite database as
- 14:56we mentioned before to keep track of the
- 14:57latest messages that were added to the
- 15:00pipeline. Okay. And that's essentially
- 15:03how the system is going to store the the
- 15:07new memories. And the most important
- 15:09part is the LLM based extraction. And
- 15:12this right here can be done with a very
- 15:14small LLM. If I'm not mistaken, uh,
- 15:17Mez0ero does it with GPT5 mini, but you
- 15:20can use a very small model, open source,
- 15:23and run this for free in your in your
- 15:25machine because it is very
- 15:27straightforward and very simple. going
- 15:28to be giving you some recommendations of
- 15:30models in a bit. All right, so we have
- 15:33effectively understood pretty well what
- 15:35is long-term memory, the stores that
- 15:37come into play with me zero and how the
- 15:40ingestion flow works. Now let's talk a
- 15:42little bit about retrieval and this is a
- 15:44very important part and there are
- 15:47essentially two main parts or two main
- 15:49moments where you are going to want uh
- 15:51retrieval to work. Retrieval essentially
- 15:54means uh getting relevant memories to
- 15:57whatever topic you're talking about. Uh
- 15:59so you want to retrieve those relevant
- 16:01memories in two different scenarios. The
- 16:04first one is when your agent wants to um
- 16:07explicitly search its memory and ideally
- 16:10this is exposed to the agent via an
- 16:12agent tool. So for example a tool called
- 16:15search memory database or something like
- 16:17that. That way your agent is going to be
- 16:20able to call the tool uh search it with
- 16:22a query and find relevant memories
- 16:24related to that particular query. That
- 16:27is explicit memory searching. And
- 16:29another way of searching is to have it
- 16:32be automatic on every agent turn. So
- 16:35when you send a message to the agent um
- 16:38the memory is going to trigger
- 16:39automatically. It's going to find
- 16:41relevant memories related to your query,
- 16:44append them to the to the context of
- 16:46your agent and send that to your LLM. So
- 16:48your LLM has context about what um your
- 16:53user is talking about and some previous
- 16:55conversations. Okay. [snorts] And this
- 16:57retrieval the pipeline for the retrieval
- 17:00is the same for both the tool and
- 17:02automatic retrieval. And what's going to
- 17:04happen is the following. So first of
- 17:06all, we're going to take a few
- 17:08parameters as input. We're going to take
- 17:10first of all the query of course. So
- 17:12whatever we're talking about or what
- 17:14we're using if it is the user message
- 17:17then the user message becomes the query
- 17:18itself. There might be some query
- 17:21rewriting at this step and that can be
- 17:23pretty useful in your pipeline. I um as
- 17:26far as I'm not aware mem does not do
- 17:28that by default. So you would have to
- 17:30implement that on the harness side. But
- 17:32um just keep that in mind. The query is
- 17:34the first part. The top k memories that
- 17:37you want. So how many memories do you
- 17:39want by retrieval search? So for
- 17:42example, let's say that you want 10
- 17:44memories. A threshold of the minimum
- 17:46score of memories to be added to the
- 17:49list of related memories. And finally
- 17:52the identity of who is looking for these
- 17:55memories. Okay. [snorts] So not the ID,
- 17:57the identity. So is this uh memory
- 18:00related to the agent, a memory related
- 18:02to the user or to the run itself? Okay.
- 18:05And this is very important because agent
- 18:07memory can be things about what it
- 18:10learned about the world or some
- 18:11procedures about uh how to do some
- 18:14different stuff and about the user it
- 18:17will be related to the user preferences
- 18:19the user's previous experience etc. And
- 18:23with that input what we're going to do
- 18:24is we're going to retrieve relevant
- 18:26memories for that conversation. So the
- 18:29first step right here is to embed the
- 18:32query. And in order to do that, we're
- 18:34going to use the same embedding system,
- 18:36the same embedding uh model as we used
- 18:38to create the vector database. And in
- 18:41order to do that, we can use some open
- 18:43source models. And here on the screen is
- 18:45a list of some of the most popular
- 18:47embedding models that you can use for
- 18:49this particular task. And you can run
- 18:51them locally, of course, in your
- 18:52machine. Men zero by default uses closed
- 18:55models. Just know that you can also do
- 18:57this locally completely with your own
- 18:59open- source embedded models. Once the
- 19:02uh query is uh embedded, we're going to
- 19:06use it to retrieve semantically similar
- 19:09memories in our main vector store. Okay,
- 19:12so right here we're going to search
- 19:15vector DB semantically and this is done
- 19:18with aN. Okay, so nearest neighbors
- 19:22and actually here is where your top K
- 19:25value comes into play [snorts] because
- 19:28remember that top K is the number of
- 19:30memories that you want to be added to
- 19:32your context of your agent. And let's
- 19:34suppose that you said uh that you want
- 19:36the top K of 10. So you want 10 related
- 19:39me uh memories to enrich your context of
- 19:42your user of or your or about your
- 19:44agent. And uh what's going to happen is
- 19:47that this vector search is not going to
- 19:49look for only 10 messages. It's going to
- 19:52look for many more because it's going to
- 19:53do some reranking. And reranking works
- 19:55better, of course, if you have a larger
- 19:57pool of results. And then it's going to
- 20:00select the top 10 from those uh that
- 20:03bigger pool. And in order to do that,
- 20:05what's going to happen is that it's
- 20:06going to either multiply your top k * 4
- 20:11or select uh 60 um as the minimum uh
- 20:18number of messages memories that are
- 20:20queried. Okay. So if you set a top k of
- 20:2310, you're going to actually get 60
- 20:25messages from this vector search. And
- 20:28this is very important because on top of
- 20:30this 60 messages uh there are going to
- 20:33be a couple of extra steps to uh rerank
- 20:37and to boost some memories over others.
- 20:41And here is where you have the keyword
- 20:44matching score. And on the other side
- 20:47you have the entity boost. Now, this one
- 20:52right here is essentially going to
- 20:54search for overlaps in words between
- 20:57your query and the memories within the
- 21:01that would were uh that were returned
- 21:03from here. So, uh remember that the
- 21:05memories returned from here um are the
- 21:08base pool on which there is going to be
- 21:10the reanking. None of these two steps
- 21:13can add additional memories to what was
- 21:15retrieved right here. So, the first part
- 21:17right here is keyword matching score. So
- 21:20what's going to happen is that both the
- 21:22query uh is going to be leatized and
- 21:24actually remember that all the memories
- 21:26already already contain their own lema
- 21:30in their metadata as we mentioned
- 21:31before. So we're going to compare these
- 21:33two and we're going to give it a score
- 21:36depending on how similar uh the words
- 21:38are. So word so in other words it's
- 21:41measuring the word overlap and it is
- 21:43doing that in case you're curious about
- 21:45it with BM25.
- 21:47Okay. And by the end, we're going to uh
- 21:50translate that into a score between zero
- 21:53and one. Okay, [clears throat]
- 21:56so that's going to be this score right
- 21:58here of keyword matching. And for entity
- 22:00boost, what's going to happen is that it
- 22:02is going to first extract the entities
- 22:06from the query. Then it is going to
- 22:08search the entity database for this
- 22:12query and the resulting entities contain
- 22:16memories that are linked to them. Right?
- 22:18As we mentioned before, that is how the
- 22:20entity uh memory works. So every entry
- 22:24in this vector database is an entity. So
- 22:26for example, Paris, restaurant, etc. And
- 22:30every single one of those entities is
- 22:32linked to one or more memories. Okay?
- 22:36And what's going to happen is that
- 22:38whatever entities we get from that
- 22:40vector search, we're going to take the
- 22:43memories associated to those entities
- 22:46and we're going to only select the ones
- 22:49that are in our initial 60 or our top K
- 22:54initial pool. And in order to do that,
- 22:57we're going to select those and we're
- 22:58going to give them an additional score.
- 23:00And this score right here is between
- 23:02zero and five. If you're interested in
- 23:05the exact formula of how they get this
- 23:08specific score, you can take a look at
- 23:11the screen. I will be showing it right
- 23:13now. But to keep things simple, just
- 23:16remember that whichever entity,
- 23:18whichever memories were retrieved that
- 23:20were retrieved from this entity database
- 23:23are going to get a a score and the score
- 23:26is going to be higher if there are less
- 23:29memories associated to that particular
- 23:30entity. Okay. See for example if for
- 23:33example if my entity is Paris and I have
- 23:35a thousand memories associated to that
- 23:37one then the score is going to be much
- 23:39lower. If my uh entity is Paris and I
- 23:43have just two memories associated to
- 23:44that one then it is more likely that
- 23:46those memories are related to whatever I
- 23:48am um trying to retrieve right here. So
- 23:51that's going to give us a larger score
- 23:54between 0.5 uh uh closer to 0.5. In the
- 23:58end, all these three scores are going to
- 24:00be added. Um, the vector search also is
- 24:03translated into a score between zero and
- 24:06one. So, we have from 0 to one here,
- 24:09from 0 to 1 here, and from 0 to 0.5
- 24:11here. That gives us a total of 2.5
- 24:15potential score. So, what's going to
- 24:17happen for each one of the messages in
- 24:20our initial pool is we're going to add
- 24:23both three scores. So this one, this one
- 24:26plus this one plus this one. And we're
- 24:28going to divide them over 2.5. And that
- 24:32is going to give us the final score. And
- 24:36then given that score, we're going to
- 24:38select the actual top K. And those are
- 24:43the ones that are going to be returned
- 24:45uh to the user or to your agent. Okay.
- 24:48And that's essentially how retrieval
- 24:50works. As you can see, it's a little bit
- 24:52more complex than the ingestion part. Uh
- 24:54[snorts] there are three different uh
- 24:57scores that are added together in order
- 24:59to actually rank the most relevant
- 25:01memories for that particular query. And
- 25:03something else remember I mean I already
- 25:05mentioned it but remember it's very
- 25:06important to remember that the query
- 25:09right here is any it's uh it can be
- 25:12anything and it's usually uh whatever
- 25:15you send to this pipeline. But this
- 25:17pipeline does not necessarily have a
- 25:19query rewriting system. If you want the
- 25:22retrieval to work better, I would
- 25:23recommend that you add a query rewriting
- 25:25system right here on the side of your
- 25:28harness. So whenever the user asks about
- 25:30something, you ideally have a small LM
- 25:34translate that into a better query so
- 25:37that you can get better results right
- 25:38here in the retrieval. [snorts] And
- 25:41there you go. This is essentially
- 25:43everything that you need to know about
- 25:45how MEMS works. And let's take a look
- 25:47very quickly at some models that you may
- 25:49want to use in each one of these stages.
- 25:52Uh if you want to do this with
- 25:54completely local setups with open source
- 25:56models. So if you go to hugging phase,
- 25:58we have right here if you go into
- 26:01models, you're going to see a bunch of
- 26:02different models. uh I would recommend
- 26:05that if you want to use an open- source
- 26:08model for the extraction part which is
- 26:10one of the most important parts I would
- 26:12recommend that you choose a small model
- 26:14because extraction is a relatively
- 26:16simple uh task. So between 1 billion and
- 26:1912 billion parameters should be enough.
- 26:22I would not go be under 1 billion unless
- 26:25you fine-tune the model in order to make
- 26:27it more accurate. But um in this
- 26:30ballpark you already have good
- 26:32suggestions. And if you click on text
- 26:33generation, it's going to uh filter only
- 26:35for text generation. So for example,
- 26:37right here you can see that you can use
- 26:40some lama models, you can use Quen. Uh
- 26:43probably you could I would recommend
- 26:45something like Quen 38B. I I think that
- 26:478 billion parameters is pretty good. You
- 26:49can choose the latest version of the
- 26:52model at the time of watching. For
- 26:55embeddings, if you want to take a look
- 26:56at them as well, I recommend that you go
- 26:59also right here to models. And in tasks,
- 27:02you're going to go right here and filter
- 27:05by feature extraction. And that way, you
- 27:08will see mostly embedding models. And
- 27:10you can select the one that you like the
- 27:12best. And if you want to see a ranking
- 27:15of the embedding models, you can take a
- 27:17look at it right here. Here, which is
- 27:20one of the most reputable benchmarks for
- 27:22a bunch of different um embedding models
- 27:26for montilingual, for English, for
- 27:28different languages, etc. And here you
- 27:30have the benchmarks exactly depending on
- 27:32which one you're trying to figure out.
- 27:33So for example, let's suppose that you
- 27:35want something related specifically to
- 27:37medical. Well, here you have the
- 27:39benchmark for embedding models that
- 27:42perform better with medical contexts.
- 27:45And of course, there's always the
- 27:46possibility of fine-tuning your own
- 27:48small models. I would actually recommend
- 27:50that if you're going to be doing this in
- 27:52a long term, you fine-tune your models
- 27:55for this to work even better and to be
- 27:57able to run this locally. And uh there
- 27:59you go. That is essentially a very quick
- 28:02overview of u long form or long-term
- 28:05memory for agents with a very nice use
- 28:07case like me zero. I will be dissecting
- 28:10other memory libraries such as super
- 28:13memory for example etc to show you a
- 28:15little bit more of different ways of
- 28:18dealing with long form memory. But uh I
- 28:20hope that this was educational and uh do
- 28:22let me know if there are other things
- 28:24that you would like to see in this
- 28:25format. And thanks a lot for watching.
- 28:27It has been a great pleasure. I'll see
- 28:30you in the next one.
About this transcript
This page contains the full transcript of Agent Memory EXPLAINED - Complete Architecture by Hugging Face, generated from the public captions YouTube serves with the video. The transcript has 4,935 words across 688 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.