YT YouTube2Text

YouTube2Text → Transcripts → LangChain Full Crash Course - AI Agents in Python

LangChain Full Crash Course - AI Agents in Python — Transcript

by NeuralNine · 10,535 words · 1,463 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Today we're going to do a crash course
0:01on Langchain, the Python framework for
0:03working with and building AI agents. It
0:06makes it super simple to interact with
0:08standalone models, build complex agents,
0:10and integrate all sorts of other
0:11components like embedding models or
0:13vector stores. And all of this without
0:15caring about specific differences in API
0:18definitions of the various providers.
0:20The goal of this video today is to cover
0:21as much as possible in a short amount of
0:23time. We're going to start by talking
0:25about Langchain and its ecosystem. Then
0:27we're going to take a look at a couple
0:28of simple examples for building agents
0:31and working with standalone models.
0:32We'll learn how to work with message
0:34histories, stream responses, use tools,
0:36generate structured output, handle
0:38multimodal input, pass context, and keep
0:40track of memory. After that, we'll also
0:42build a simple rag example. And finally,
0:44we're going to take a look at
0:45Langchain's powerful middleware, as well
0:48as some interesting use cases for it.
0:49That's quite a few things to cover, and
0:51I think you can learn a lot today,
0:52especially since we're going to work
0:54with the latest version of Langchain
0:551.0. If you like this video, let me know
0:57by hitting the like button and
0:59subscribing. But now, let us get right
1:00into it.
1:06[music]
1:10>> All right. So, we're going to cover
1:11quite a lot today, which is why I'm
1:12going to try to speed this up to cover
1:14as much as possible in a concise way so
1:16we don't waste too much time here. I
1:18want to start by just briefly talking
1:20one or two minutes about Langchain, the
1:22Langchain ecosystem, and also its
1:24development history. Now, langchain
1:26itself as I mentioned is a Python
1:28framework for building and working with
1:29AI agents. The main use case or the main
1:32benefit of using Langchain is that we
1:35can use all these models and related
1:37tools in an abstract way. So, regardless
1:39of the provider, regardless of whether
1:40you're using OpenAI or Enthropic or
1:43Google, you basically have the same
1:45classes and methods everywhere. And if
1:47you have a system built on langchain
1:49that uses vector stores, embedding
1:50models, uh AI models, agents, whatever,
1:53you can easily just swap out the
1:55underlying technology and you can keep
1:57the code the same for the most part. So
1:59that's what I would consider the main
2:00selling point of Langchain that you have
2:02this abstract highlevel way of working
2:05with agents and related tooling. Now in
2:07addition to lang, we also have lang
2:09graph which is more low-level. We have
2:10more granular control. We can build
2:12complex graph-based event-driven agents
2:15or agentic systems. We're not going to
2:17cover Langraph in this video today. I do
2:19have videos on this channel where we use
2:21Langraph, but it's not going to be the
2:23topic of this crash course. And finally,
2:24we also have Lang Smith, which is like a
2:26collection of tools for observing,
2:28evaluating, monitoring, and deploying
2:30models. We're not going to cover that
2:32either. We're going to focus fully on
2:34Langchain today. Speaking of which,
2:35Langchain is now on version 1.0. So,
2:38some of you guys watching this might
2:39have already worked with Langchain in
2:41the past. Maybe you're already familiar
2:43with the package to some degree. But if
2:45you take a look at 1.0, you will notice
2:47that some things have changed. So, when
2:49I first used Langchain, the structure
2:51was a little bit different. We had
2:52Langchain, then we had Langchain core,
2:54and langchain community. And then you
2:56also had langai, langchain anthropic as
3:00separate import packages. So you would
3:01do something like from langchain core
3:04import something from langchain openai
3:07import something. Now with 1.0 it seems
3:10like if you look in the documentation
3:12everything is going through the main
3:14langchain package. So from langchain do
3:17something we import stuff. The second
3:19thing you will notice is that langchain
3:20is now much more focused on being a
3:22library for agents. Not just for
3:24integrating tools not just for using
3:26models and vector stores and all that
3:28but actually for building agents. Which
3:30means that we now have here langchain
3:32agents with create agent and in the past
3:35in my opinion it was more like langchain
3:37is this general toolkit for working with
3:39these tools and lang graph was more
3:41about the agent side. So we can still
3:43use the packages like langchain core and
3:45langchain community but for the most
3:46part it's enough to just go with the
3:48main lang package that is the modern way
3:50of working with this framework. Cool. So
3:52now that we covered that let us go ahead
3:54and set up our environment. In my case
3:56I'm going to use uv which is a rustbased
3:58python package manager. Of course, feel
4:00free to use pip or pip 3 install. Feel
4:02free to use virtual environments with
4:04virtual env or ven, whatever you want to
4:06choose. In my case, I'm going to go to
4:08my tutorial directory, say UV init, and
4:11then I'm also going to say UV at
4:13langchain. Now, one thing that's
4:14important is if you want to use lang
4:17chain together with some AI provider,
4:19which you usually want to do, you need
4:21to also provide it in square brackets.
4:23So for example, if I want to install
4:24lang chain with the openai dependencies,
4:27what I will do is I will say open AI in
4:30square brackets here. So in my case, UV
4:32add langchain openai. If you use pip,
4:34you do pip or pip 3 install langchain
4:37openai. And this will also install
4:39packages like of course openai. And you
4:42have to do that basically for all the
4:44providers that you want to use. So you
4:45want to do that for enthropic, you want
4:47to do that for mistral AI. I'm going to
4:50actually just do it now to show you how
4:52this works. Mistral AI and you also want
4:54to do this for Google genai if you plan
4:56to use all these models. Now what we're
4:58also going to need is API keys. If you
5:00want to use models from providers of
5:02course you have to authenticate
5:03yourself. So we're going to create a
5:05file called and this file will contain
5:08our keys. So these keys will be the
5:10open_appi_key.
5:12It's going to also be the Mistral API
5:15key, the Anthropic
5:18API key, and I'm not sure what the
5:20correct name is for the Google one, but
5:22I'm just going to say Google API key,
5:24even though that's probably not the
5:25correct one. So, what you want to do is
5:26you want to go to the providers if you
5:28have accounts there if you have API keys
5:30and you want to paste them here. So, I'm
5:32going to go to OpenAI console to Mistral
5:34Console, Enthropic Console, and to my
5:36Google Cloud Platform. There I'm going
5:38to take the API keys, copy paste them
5:40here. I'm not going to show how to
5:42obtain them or maybe I'm going to show
5:43them for one of them. So yeah, here for
5:45example, we go to
5:45platform.openai.com/api
5:48keys or you just go to API keys here on
5:50the left and here you can create a new
5:52secret key or API key. You do that for
5:54the various providers. Then you just
5:55copy paste the keys here like this or
5:57you use quotation marks if you want to
5:59and that is our end file. So in my case
6:01I already did this and now I'm going to
6:03install an additional package called
6:06python-.end.
6:08This is going to allow us to load these
6:10API keys into our Python script. Now, to
6:13get autocomp completion, I'm also going
6:14to activate this environment here. This
6:16is something I just have to do for Neoim
6:18to get autocomp completion. So, you can
6:19ignore that if you're not coding in the
6:21terminal. And then we're going to go
6:23into our main py file and get started
6:25with a first simple example. So, instead
6:27of just covering the concepts one by
6:29one, I'm going to show you how to build
6:30a simple agent. And we're going to cover
6:32a couple of concepts while we're doing
6:33that. So, we're going to start with the
6:35import. Let's import requests. Then also
6:38as I said from we're going to import
6:40load.n this is for loading the API keys
6:43into the environment. And then from
6:45langchain
6:47agents as we saw already in the docs
6:49we're going to import the create agent
6:51function. This is like the central
6:53function that is going to create agents
6:55that we then use to do stuff. And in
6:57this function we can do a lot of things.
6:59So we can provide middleware, we can
7:01provide tools, we can do a lot of things
7:03in that function or with that function.
7:04In addition to that, I'm also going to
7:06say now from langchain.tools
7:08import tool. This is a decorator that we
7:11can use to annotate a function to
7:13basically make that function a tool. We
7:15can also provide some uh information
7:16like a description or the name of the
7:18function. So for example, we can say add
7:20tool and I want to define a tool that is
7:23called get weather. This is going to
7:25obviously get the weather and I can
7:27provide a description here for the agent
7:29to know what this function is about or
7:31what this tool can do. In my case, I'm
7:33going to say here return weather
7:35information for a given city.
7:38Theoretically, you can also say return
7:40direct is equal to true if you want to
7:42have just the output of that tool
7:43immediately returned to the user. In
7:45this case, set this to true. We can also
7:47explicitly set it to false. I think it's
7:49the default, but just so we know it
7:51exists, we're going to set it to false.
7:53And then we're going to say now the
7:55function is get weather. It's going to
7:56get a city, which is going to be a
7:58string as input. And we're going to make
8:00it very simple here. Here I don't want
8:02to use a mock function. So I'm actually
8:03going to ping a weather API. But this
8:06one is actually open source and free to
8:08use or maybe not open source, but you
8:09don't need an account to use it. So
8:11we're going to say here response is
8:12equal to requests.get.
8:14And we're going to use https/wtr.in/
8:22city question mark format equals j1. And
8:25of course this needs to be an fstring
8:27otherwise we cannot parse or we cannot
8:30format the city into the string here.
8:31But this basically gives us a JSON
8:33object with temperature and the weather
8:35information for a given city. And all we
8:37want to do here is we want to return the
8:39JSON object that we get here. So return
8:42response.json and then the model the
8:44agent can do whatever it wants with that
8:46information or whatever it thinks is the
8:48most reasonable thing to do. So to keep
8:49it simple we're going to have just this
8:50one tool. And now we're going to say
8:52agent is equal to create agent. And the
8:54first thing we want to do here is we
8:56want to provide the model. Now the model
8:57can just be a string. For example, I can
8:59say GPT-40.
9:01Now 40 doesn't support tool use. So
9:03we're not going to use that. But I can
9:05go with 4.1 mini. For example, this
9:08would be open AI. It is automatically
9:11recognized as OpenAI. So if I don't have
9:13Langchain OpenAI installed, it's not
9:15going to work. So for this, a couple of
9:17things need to be present. One is an
9:19OpenAI API key in the environment. And
9:22the other thing is langchain open AAI
9:24needs to be installed. So these two
9:26things need to be given so that we can
9:28actually use this model. After that I
9:30can just say give me a list of tools and
9:32this list will contain just get weather
9:34as an entity. So we're not calling the
9:36function we're passing it. And then we
9:38can provide a system prompt like you are
9:41a helpful weather assistant who always
9:45cracks jokes and is humorous while
9:49remaining
9:51helpful. So just to give you an example
9:53so that you can see that the system
9:54prompt actually affects the agent and
9:56then in order to get something from this
9:58agent in order to send something to the
10:01agent I should say we can just invoke
10:02it. So I can say agent.invoke invoke and
10:05here what we do is we pass a dictionary
10:08that contains a field messages and this
10:12messages has to point so this key
10:15messages has to point to the value which
10:17is a list and this list will contain all
10:19the messages as dictionaries with ro and
10:22content as we know it from the typical
10:25API. So ro is going to be user and
10:28content is going to be whatever we want
10:29to ask for. For example, what is the
10:32weather like in Vienna? Question mark.
10:36So I need to then save this as a
10:39response. And down here I can then print
10:41the entire response. So print response
10:45or I can say if I want to get just the
10:47message, I can get from response the
10:49messages. Maybe let me do that here with
10:52square brackets and a string. And then I
10:55want to have the last message. So
10:56negative one. And then from this
10:58message, I'm interested in the content.
11:00So I can run this now. And I get a
11:02problem because because I'm of course
11:04not loading the environment variables.
11:07So I need to import load. But I also
11:10need to call it to load the data from
11:12the end file. So let's go ahead and say
11:14uvun main py. I'm doing it outside here
11:16so I can see all the output. And what
11:19you see up here is the raw response
11:22object which contains the entire message
11:24history also with all the data that was
11:26provided by the API. And as a result
11:29down here I get the actual message. The
11:32weather in Vienna right now is partly
11:33cloudy with a comfortable temperature of
11:36about 15° C and then some information
11:38about the wind speed and humidity and so
11:40on. So actually quite comprehensive.
11:42We're going to build on top of this
11:43example and extend it later on. But
11:45before we go deeper into agents, let me
11:47remove all of this. I want to show you
11:48how to use standalone models. So maybe
11:50you don't want to have an agent. You
11:52just want to interact with a simple
11:54model and that's it. And you want to do
11:56that in a more abstract way. So you can
11:58replace the models that you're using.
11:59For this you can also use lang chain but
12:01not agents. You want to use lang chain
12:03chat models. And you want to import the
12:05function init chat model. And for this
12:07we're also not going to use tools. We're
12:09just going to do it like this. And it's
12:11actually quite simple and
12:12straightforward. I just say model is
12:14equal to init chat model. Then I can
12:16provide basic parameters like again the
12:18model identifier which is going to be
12:214.1 mini again. And theoretically if I
12:24want to I can do stuff like temperature
12:270.1 for example. And once I'm done with
12:29all this I can just say response is
12:31equal to model.invoke and here I just
12:34provide a prompt now like hello what is
12:38python question mark and then I can
12:40print the entire response object or I
12:43can just print the response content. So
12:45in this case, response.content since we
12:47didn't pass a message history or
12:49conversation history. And of course, one
12:52more time, I forgot the load. So we're
12:54going to add this and run this again.
12:56And there you go. We get an answer that
12:58tells us what Python is, a highle
13:00interpreted programming language. And we
13:02also have again this entire response
13:04object if we want to access different
13:06fields like the total amount of tokens
13:08and stuff like this. But you can see how
13:10easily this is done in lenchain. And we
13:12can also just swap the model if we want
13:13to use a different one. So instead of
13:15using GPT I can say I want to use
13:16Mistral medium. I just have to provide
13:18the proper string from the website and
13:21then I can run this and everything else
13:23in the application stays the same. I'm
13:25now just using Mistral uh medium instead
13:27of GPT4.1 mini and I'm going to get the
13:30response and everything's going to work
13:31in the same way. There you go. It's a
13:33quite comprehensive response but we get
13:36the answer here from Mistl. Let me now
13:38switch back to 4.1 mini. Now if we want
13:41to pass a conversation history and not
13:43just a single prompt, we can do that as
13:45well with this list and dictionary sort
13:47of notation that we used before. But we
13:49can also import specific classes for
13:51that. I can also say from langchain dot
13:54messages import human message AI message
13:58and system message. This makes it then
14:00super simple to work with. I can just
14:02say conversation is equal to a list and
14:05in here I can say I first have a system
14:07message. For example, you are a helpful
14:10assistant for questions regarding
14:14programming. Then the second message
14:16could be something that a human asks.
14:18Like for example, what is Python?
14:22Maybe let's stay consistent here with a
14:24quotation marks. Let's use single
14:26quotations everywhere. And then we're
14:27going to say an AI already answered
14:29that. We're going to say that it told us
14:32Python is an interpreted
14:35programming language. not question mark
14:38but period and then we're going to say
14:40the human has a follow-up question which
14:42relates to the previous messages and
14:45this is when was it released question
14:48mark. So now instead of invoking on a
14:50string I can also invoke on a
14:52conversation and actually I'm missing a
14:55t here and here as well and now the rest
14:58stays the same and we have a
14:59conversation. So Python was first
15:01released in 1991 by gofo fun rosesome.
15:04Now, we saw with Mistl that responses
15:06can be quite long and we need to wait
15:08for them to be finished before we can
15:09start reading them. If we don't want
15:11that, if we want to read them in real
15:12time as they're generated, we can also
15:14stream the response. So, let us maybe go
15:16back here to the prompt and let us also
15:19change the model back to mistrol. So,
15:21here I'm going to say mistrol medium 258
15:26and we're going to ask the same
15:27question, but instead of just getting
15:28the response here, we're going to stream
15:30it. So we're not going to invoke. We're
15:31going to say model stream and we're
15:34going to iterate over this generator
15:35here to generate the chunks. So I'm
15:37going to say for chunk in model stream
15:40I'm going to print the chunk.ext.
15:43I'm going to have no line breaks after
15:45each print and I'm going to say flush is
15:48equal true equal to true so I can see
15:50the output in real time. And we're going
15:53to delete that and run this. And now you
15:56can see how this is generated in real
15:58time and I can read while it's still
16:00generating. So let us now come back to
16:01our initial example with the agent here
16:04and the weather function. We're now
16:05going to extend it to incorporate more
16:07concepts. So on the one hand I want to
16:09have structured output. I want to have
16:11the output message or summary and I also
16:13want to have some key information like
16:15the temperature or the humidity. Also I
16:18want the agent to be able to realize
16:20what location I'm asking this question
16:22from. So I don't have to specify the
16:24city. So I just want to say what's the
16:26weather like? And the agent should
16:27realize that I'm asking from a specific
16:30city based on mock database entries that
16:32we're going to provide here. And with
16:34this context, it's going to then
16:36retrieve the proper information. And
16:38finally, I would also like to add memory
16:39to this agent so it can remember that we
16:42had a conversation and we're now
16:43continuing that conversation. So we're
16:45going to add some imports for all of
16:46this. First of all, from core Python,
16:48we're going to add here from data
16:50classes the data class. And for
16:53langchain here, we're going to say from
16:54langchain.models
16:57or chat models importit
17:00chat model. In addition to tool, we're
17:02also going to import tool runtime. And
17:05finally here from lang graph. Now I said
17:08we're not going to do lang graph. We're
17:10not going to cover the langraph
17:12framework, but we're going to use uh one
17:14specific class from there. It's from
17:16checkpoint memory the inmemory saver
17:19which is going to be important for
17:21remembering the message history. Cool.
17:23So now let us create two data classes.
17:25One is going to be for the context.
17:26We're going to keep track of the user ID
17:28that the model is communicating with so
17:30that we can actually look up the
17:32location of that user from our database
17:35which we're going to just model as a
17:37match case statement in a function. And
17:39then we're also going to have a data
17:40class for the response format. So we're
17:42going to say here at data class and then
17:45it's going to be a class called context.
17:47Quite simple and it's just going to have
17:49a user ID which is a string. And then
17:50for the response format we're going to
17:52say data class response format. And here
17:56I want to have a summary which is going
17:58to be a string. I want to have a
18:00temperature in Celsius which is going to
18:03be a float. I want to have the same in
18:05Fahrenheit. And I also want to have a
18:07humidity whatever the unit is here.
18:10Cool. So now what we're going to do is
18:12we're going to add an additional tool
18:14and this tool is going to be locate
18:16user. So the name is going to be locate
18:18user and it's going to have the
18:20following description. Look up a user's
18:24now I have to use double quotations
18:26here. Look up a user's city based on the
18:31context. Now the interesting thing is
18:33we're not going to pass the user ID as a
18:35parameter. We're going to have a tool
18:38runtime which contains context. So the
18:41context is going to contain the user ID
18:42and we're going to get it from this
18:44context. How do we do that? We say
18:46deflocate user and here we have a
18:49runtime. This runtime is going to be a
18:51tool runtime and we're going to pass
18:54here in square brackets the context
18:56class the data class we just created.
18:59And here what we're going to do is we're
19:00going to get the user ID from
19:02runtime.context.
19:03And I'm going to use a match statement.
19:05So match runtime dot context dot user
19:09ID. And depending on the value, we're
19:11going to return a different city. So
19:13let's make up some cases here. If the
19:14user ID is ABC123,
19:17I'm just going to return Vienna. Another
19:19case could be if the user ID is XYZ456,
19:24then I'm going to say that we're in
19:26London. Then another case could be HJKL
19:30for Vim and then 111. That would return
19:33Paris. And if it's none of these, if
19:36it's unknown, we're going to say case
19:38default is just going to return unknown.
19:41Now, you can define unknown behavior in
19:44multiple ways. You can provide it in the
19:46description. You can even provide it in
19:47the return value itself. You can provide
19:49it in the system prompt. You can also
19:51add some custom logic. But basically, we
19:53need to somehow instruct the model that
19:55if it's unknown, just say it's unknown.
19:57Maybe it can do it automatically as
19:58well. Uh, wherever you want to put that,
20:00put that somewhere how you want to
20:02handle unknown values. But we're going
20:03to look up based on the user ID in the
20:06context. So remember the connection here
20:08we have the runtime the tool runtime
20:10passed to this tool which is based on
20:12the data class context which we defined
20:14up here which contains the user ID. So
20:17now we're going to go down here create
20:19the model to show you that we can also
20:21pass a model instance. So model is going
20:23to be init chat model GPT-4.1-
20:27mini and the temperature not this sort
20:30of temperature but the model temperature
20:33is going to be 0.3 for example and then
20:37we're going to create a checkpointer
20:39which is going to be an in-memory saver.
20:41As I said this is for remembering
20:42conversations. We're going to add to the
20:45agent invocations a thread ID and this
20:48is going to determine the conversation
20:51that we're focusing on. So we can keep
20:53asking questions about the same
20:54conversation. And now we can combine all
20:56of this into the create agent function.
20:59So we're going to say agent is equal to
21:00create agent. Model is equal to model.
21:04Tools is equal to get weather and locate
21:06user. System prompt can stay the same.
21:09Now new stuff here is context schema.
21:12This is going to be the class the data
21:13class of our context. So just context.
21:17Then also response format. I think it's
21:19not surprising that this will be our
21:20class response format. And finally,
21:23checkpointer also not surprising is
21:26going to be the checkpoint. So what our
21:28agent now does is it has access to a
21:30model GPT4.1 mini. It has access to two
21:34tools. One for getting weather
21:35information about the city, one for
21:37locating the user, so getting the city
21:39of the user based on the user ID. We
21:41have a system prompt here. We now also
21:43have the ability to pass context. So for
21:46this we use the data class context which
21:48again contains the user ID. Then we have
21:50a response format which means that our
21:51model is forced now to answer in a
21:54specific format. This format is going to
21:56be a summary string and then three
21:58floats for temperature and humidity. And
22:00finally we add memory to the model so it
22:03can keep track of conversations based on
22:05a thread ID. So now when we invoke
22:07something we also need to pass context
22:10and thread ID. For this we're going to
22:12start by saying config is equal to
22:14dictionary which is going to have a key
22:17called configurable. And this
22:19configurable is going to point to
22:20another dictionary which contains thread
22:22ID which itself points to one for
22:25example. And now for the invocation I'm
22:27going to say what is the weather like
22:28without specifying a city but I'm going
22:31to specify context and I'm going to pass
22:33the configuration. So for the
22:35configuration just config equals config
22:38and for the context we're going to say
22:39context is equal to context an instance
22:42of the data class where we set the user
22:45ID to be equal to ABC123.
22:48So that would result in Vienna again. We
22:50should get the same response. And since
22:52we're now working with this response
22:54format, since we're forcing structured
22:56output, we're going to not access just
22:58messages negative1 content. We're going
23:00to print the entire response object if
23:02we want to. Actually, I don't want to do
23:03that. I just want to get the structured
23:05output itself. And for that, I'm going
23:07to say here response structured
23:10response. And this will give us the
23:12entire response object. If I'm
23:13interested in specific parts of that, I
23:15can just say dot summary or dot
23:19temperature Celsius for example. So now
23:21I can run this and we have a problem
23:24because we're not closing this curly
23:28bracket early enough. So of course it
23:30belongs to this message history. But
23:32these are now just keyword arguments. So
23:35let me run this again. And we can see
23:37the current weather in Vienna is partly
23:39cloudy. And then I also get 15.0 zero
23:42for the Celsius temperature. Now, if I
23:45change my user ID to something else like
23:47XYZ, what was it? 456, then I should get
23:51the weather for London. There you go.
23:52The weather in London is currently sunny
23:55with a temperature of 12. Then, if I try
23:57something completely different like
23:58this, something it doesn't recognize,
24:00probably it's going to tell me unknown
24:02or it's going to do something else. I
24:05couldn't find your location, so I can't
24:06tell the weather, but hey, if you tell
24:08me your city, I'll fetch the weather.
24:10and zero is the default value for
24:12temperature. And to show you that this
24:14actually works with a follow-up, if I
24:16provide here again a valid user ID like
24:19this for Vienna here, I can also follow
24:22up with the same config to keep track of
24:24this conversation. So I can just copy
24:25this here, paste it down here and I can
24:29say and is this usual question mark. So
24:33when I run this now and of course maybe
24:35before running this I should also print
24:37the result. So just copy this from up
24:39here. print structured response summary
24:42and now it should keep track of the
24:44information. So we have this inmemory
24:46saver. So we have still the information
24:48that uh the weather in Vienna is what it
24:50is and then yes the weather in Vienna
24:52being partly cloudy with mild
24:53temperatures around 15° C and so on is
24:57usual for this time of the year.
24:59However, you will notice that if I take
25:00this and I change this before I do that.
25:03So if I say now the thread ID is two,
25:06it's no longer going to be related to
25:08that thread. So, it doesn't know what
25:09I'm talking about. So, here I get the
25:11information about Vienna. And now I
25:13would need to know the specific weather
25:14conditions or location you're referring
25:16to in order to determine if it's usual
25:18or not. So, since we're in a different
25:20threat, it doesn't know what we talked
25:22about up until this point. If I may for
25:24a second, I would like to plug myself in
25:26as the sponsor of my own video. If you
25:28go to my website, neural9.com, you will
25:30find a tab services and a tab tutoring.
25:33Here you can hire me for all sorts of
25:35stuff like data science, machine
25:36learning, web development. If you need
25:38help with something in a project here,
25:40you can book me for one-on-one tutoring.
25:41If you want me to teach you personally
25:43something that you don't understand, if
25:45you like my teaching style on both pages
25:47at the bottom, you can contact me via
25:49mail and also via LinkedIn. Just wanted
25:52to let you know about this. Next, I want
25:53to show you how we can work with
25:54multimodal input. So, how can we pass to
25:57a model not just text, but for example,
25:59image data. For this, I'm going to say
26:01here model is equal to in a chat model.
26:03I'm going to use again GPT-4.1-
26:08mini. And we're now going to create a
26:11message in the dictionary format. Again,
26:13I'm going to show you a different way in
26:14a second as well. The role for this
26:17message is going to be user. And the
26:19content field now is going to have
26:20multiple values. So, we're going to say
26:22content is pointing to a list. And this
26:25list will contain multiple pieces of
26:27content. For example, the first one will
26:29be of type text and we're going to say
26:32that the actual text content. So again
26:34text here as a key not as a value will
26:37be describe the contents of this image.
26:42Now we can copy that and we can say type
26:45image and now we have two ways to
26:47provide image content. One is by using
26:50URL. So this basically points to an
26:52image somewhere on the web or we can
26:54also pass base 64 encoded image bytes.
26:58Now we're going to do both but I'm going
26:59to start with a URL and for that I have
27:02here a link from my website. So just
27:04neural9.com and the logo on my website.
27:07I'm going to pass this here as image
27:08content. And what I'm going to do then
27:11is I'm going to say model.invoke.
27:14I'm going to pass a list of messages and
27:16just my one message in here. That's
27:18going to be the response. And then I can
27:20just say print response.content.
27:23So if I run this, this will take a look
27:25at my image and tell me that this is a
27:29logo that reads neural 9 and orange text
27:31on a black background. The text is
27:33stylized with a number one. I think this
27:35is just a mistake in the font. And it
27:38explains what my logo looks like
27:39essentially. So this is what we want.
27:41And we can do the same thing with an
27:43image from disk. So if I open the
27:46sidebar here, you can see I have the
27:48logo.png. png. We can also load this and
27:51encode it with b 64. So in this case
27:54here just b 64 as the field as the key
27:58here and I'm also going to say from b 64
28:02import b 64 encode. So the idea is we
28:06load the bytes we encode them with b 64
28:08and then we decode it into a string. So
28:11we're going to say here b64 encode. What
28:13are we encoding? We're opening a file
28:16from disk called logo.png
28:19in reading bytes mode. Then we're
28:21reading the content of that file. We're
28:23encoding it with B 64 and then we're
28:25decoding it into a string. So we can
28:27actually pass it here to the model. Now
28:28what we also need to pass here if we use
28:30B 64 is a mime type. So let's actually
28:34format it like this. And the mime type
28:36is going to be image / PNG. And actually
28:41I think we need to use underscore not
28:43dash. So now when I run this you can see
28:44the image shows a logo with a text
28:46neural 9 written in orange and basically
28:49the same thing as before. Now we can
28:50also do it with the message classes but
28:52we have to do it basically in the same
28:54way. So I can say langchain do messages
28:57import human message and the only thing
28:59I would change is I would get rid of the
29:01role but I would still keep the content
29:03as it is. So I would say here the
29:05message is equal to human message and
29:08then I would say content is equal to the
29:10list. So that is the small difference
29:13here. So context would be equal to the
29:16list of these two things and then we
29:18would close that with an ordinary
29:21bracket but basically the rest stays the
29:23same. So when I run this we should get
29:24the same response. It's just a different
29:26way to write it. Now for the next
29:28example we're going to build a simple
29:29rack use case. So retrieval augmented
29:31generation. Basically we're going to use
29:34a vector store and an embedding model to
29:36find the most similar pieces of content.
29:38In our case, simple messages or simple
29:41statements, let's say. And for this,
29:43we're going to use lang chain in the old
29:45school way. So, we're actually going to
29:46say up here from langchain openai import
29:51openai embeddings. This is for the
29:52embeddings model. And then we're also
29:54going to use a vector store. So, a
29:56vector database in my case face. And for
29:58this, we're going to say from langchain
30:01community
30:02import. And we're going to import uh
30:05face. But actually not from community
30:06directly but from community.
30:09Vector stores and since I'm not getting
30:11autocomp completion I assume I have to
30:13install this separately. So I'm going to
30:15say here now uv at and then
30:18langchain-ash
30:20community. So this installs now
30:22additional packages langchain community
30:24langchain classic langchain textlitter.
30:26So if I now go back into the code
30:29hopefully if I type dot something here
30:32there you go. We can see all the modules
30:34here. So, langin community vector stores
30:36import face. Now, I think actually for
30:38face, we also need to install the face
30:40package. So, let me leave this and let's
30:42say UV add face. And I'm just going to
30:45go save here with the CPU. So, I don't
30:46have to care too much about GPU stuff.
30:49This is just face CPU. And now, if I go
30:52back into the code, we should be able to
30:53use them. So, the basic idea is I'm
30:55going to have a list of statements.
30:57These statements will be stuff like I
30:59love apples or I like oranges or I like
31:01pears and then something about computers
31:04also related to Apple but semantically
31:06different because Apple is a company.
31:07Apple is also a fruit. So we're going to
31:10see if the embeddings can distinguish
31:11the concepts and we're going to retrieve
31:14the most similar statements from the
31:16vector store. So let us start by saying
31:18embeddings is going to be equal to
31:20openAI embeddings and we're going to
31:21provide the embedding model text-
31:24embedding-large
31:27three. I think this was the identifier
31:28and we're going to say of course that
31:30this is the model. So model is equal to
31:32text embedding three large. Yeah. So
31:35actually three large not large three.
31:37Then I'm going to copy paste some very
31:39very simple statements. Nothing too
31:41fancy. Apple makes very good computers.
31:44Whether that's true or not, I'm just
31:45stating it here. I believe Apple is
31:48innovative. I love apples. I'm a fan of
31:50MacBooks. I enjoy oranges. I like Lenovo
31:53ThinkPads. I think pairs taste very
31:55good. So, theoretically, this message
31:58here, I like Lenovo ThinkPads, should be
32:00closer to I'm a fan of MacBooks. And
32:03even to stuff like Apple makes very good
32:05computers. uh it should be closer and
32:08more related than I love apples because
32:10that has nothing to do with apple only
32:12syntactically only in terms of like the
32:14name it has to do something with apple
32:16but it's a different concept. So we're
32:18going to create a vector store from
32:20these texts. I'm going to say vector
32:23store is going to be equal to face dot
32:26from texts and we're going to pass the
32:29list of texts here. Now in order for
32:31this to actually work we need to pass an
32:33embedding model as well. So embedding is
32:35going to be equal to embeddings. And
32:37this is going to basically take all
32:38these texts, embed them into vector
32:40space and then store them in the vector
32:42store. So to see how this works, we can
32:44say print vector store dot similarity
32:47search and then I can add something new
32:49like apples
32:51are my favorite
32:54food. K equals 7 to get all of them uh
32:58just in a specific ranking. And then I'm
33:00going to do the same thing with Linux is
33:04a great operating system. Of course,
33:07Linux is just a kernel, but you get the
33:09idea. So, I'm going to comment this out
33:11and run the first one alone to see what
33:14are the most similar statements in the
33:17vector store. So, you can see here the
33:18most similar was I love apples. Then the
33:20second one was I think pears taste very
33:22good. So, even though the word apple
33:24occurred, we or actually there you go. I
33:27enjoy oranges was before that then I
33:29think pears taste very good and then
33:31only it gives us Apple makes very good
33:33computers. I know this is not the best
33:34view. Uh but we have these individual
33:36documents the most similar the second
33:38most similar the third most similar and
33:41you can see that everything related to
33:42fruits was ranked as more important and
33:44more relevant than apple. Even though
33:47that apple here and apple here is the
33:50same string it recognizes that the
33:52concepts are different. So let's see
33:54what we get for Linux. We get I like
33:56Lenovo ThinkPads. I'm a fan of MacBooks.
34:00And then we get I love apples. For
34:03whatever reason, the company Apple is
34:05ranked lower. Whatever. So that is how
34:08you can use the vector store as a
34:09separate component. So just interact
34:11with embeddings and vector stores. But
34:12now what we want to do is we want to
34:14make this part of agents with lang. So
34:17we want to take this capability of using
34:19this rack feature of doing a similarity
34:22search. We want to take this and build
34:24it into our agent. For this, I want to
34:26use a slightly different example. So,
34:28the code is the same. We still have the
34:30same structure here of embeddings,
34:32texts, and then from text similarity
34:33search, but we have different content.
34:35We have I love apples. I enjoy oranges.
34:37I think pears taste very good. I hate
34:39bananas. I dislike raspberries. I
34:41despise mangoes. I love Linux. I hate
34:43Windows. And then we have here again the
34:46vector store. And here we look for what
34:48fruits does the person like and what
34:49fruits does the person hate. So if I run
34:52this, you can see that these two lookups
34:55give us the information that we want. I
34:57enjoy oranges. I love apples. I think
34:59pears taste very good. Then I despise
35:01mangoes. I hate bananas. I dislike
35:04raspberries. So we have the information
35:05by using the similarity search. What we
35:07can do now with this vector store is we
35:09can turn it into a retriever for our
35:12agent. So I can say down here retriever
35:14is equal to vector store as retriever.
35:18And we're just going to pass the search
35:20keyword arguments. So search_quarks
35:23is going to be equal to a dictionary
35:25where K is going to be equal to three.
35:28So this basically tells us just give me
35:30the top three answers. We're hard coding
35:32this to keep it simple here. But
35:34basically this now is a retriever that
35:36we can use in our agent. But in order to
35:38actually use this retriever in our
35:40agent, we need to turn it into a tool.
35:42So what we're going to do is another old
35:44school import up here from
35:46langchain_core.
35:48tools. We're going to import the create
35:51retriever tool. So the idea is we pass a
35:53retriever and we turn it into a tool
35:56that the agent can use. So quite simple,
35:58the retriever
36:00tool is going to be equal to create
36:01retriever tool. We pass the retriever
36:04and then we pass the name which is going
36:06to be our fruit search I guess or
36:09actually let's call it knowledgebased
36:10search. So KB search and then the
36:12description is search the small
36:16product/fruit
36:18database for information. Let me call
36:22this knowledge base and then we can
36:24close that. And now the cool thing is
36:26this is now just another tool. We can
36:28just add it to the list of tools when we
36:29create an agent. So for this we need to
36:32import again from langchain. This is now
36:34the modern way of doing things from
36:36langchain.agents agents import create
36:39agent and now I can just say as we did
36:43before agent is equal to create agent
36:45the model is going to be GPT4.1 mini
36:47again then we can say tools is equal to
36:50retriever tool then I'm going to copy
36:52paste the system prompt nothing too
36:54fancy here just explaining again your
36:56helpful assistant and if there's any
36:58questions about Macs apples laptops
37:00whatever use the tool that you have
37:02first the retriever tool basically
37:04retrieve the context answer in a concise
37:06way and the hint that maybe you have to
37:08use this tool multiple times because if
37:11I'm asking a question that needs to
37:12combine information, you might have to
37:14do it multiple times, which the agent
37:16can do since it's not just a prompt.
37:18It's an agent that can use tools, then
37:20think, then use tools again, and so on.
37:22So then we do what we already did
37:24before. We pass a prompt. I'm just going
37:26to say here, agent invoke, what three
37:29fruits does the person like and what
37:31three fruits does the person dislike?
37:33And then we get the answer. So, I'm
37:34asking two things in a single prompt.
37:37And up here, we use two different
37:39similarity searches for that. Let's see
37:41if the agent can handle that. I'm going
37:42to run this. And it says here, the
37:44person likes oranges, apples, and pears.
37:46The person dislikes mangoes,
37:48raspberries, and bananas. You can see
37:49also the retrieval um or actually cannot
37:52see it up here. You can see the
37:54retrieval that was done one. And then a
37:58second call to the tool was done down
38:00here. And this gave us the information
38:02for this final response. Now we get to a
38:05very powerful concept in lang chain. The
38:06middleware. Middleware basically meaning
38:08it sits between request and response.
38:11And middleware allows us to do a lot of
38:13different things to enhance the
38:14capabilities of our agents. For example,
38:16we can choose different system prompts
38:18depending on the context. We can choose
38:20different models based on certain
38:21criteria. We can summarize stuff. We can
38:24have rate limits. All sorts of things
38:26can be done in between this window of
38:28getting a request and sending a
38:30response. We can do a lot of stuff
38:31behind the scenes. And instead of
38:32talking about this too long in a
38:34theoretical way, what I want to do here
38:35is I just want to show you a couple of
38:36examples of custom middleware of already
38:39existing middleware of different use
38:41cases and then you can just go and
38:43explore it yourself. So I want to get
38:44started with a simple example right
38:46away. I want to show you how we can swap
38:47the system prompt based on the level of
38:49expertise that the user has. So for this
38:51we're going to use context again and I'm
38:53going to start by importing from
38:55langchain.agents.m
38:57middleware the following things. model
39:00request, model response and in this case
39:04now the decorator called dynamic prompt.
39:06So this is middleware that is already
39:08implemented. So we don't have to build
39:10it and define it ourselves. But what we
39:13can do now is we can say that if the
39:16user has a certain role like he's an
39:18expert or he's a beginner or maybe even
39:20he's a child, we're going to say adjust
39:23your answer, adjust your style based on
39:25that. So we're going to create a data
39:26class again called context. And actually
39:29we need to again import here from data
39:32classes import data class. And this
39:35context this time will not contain the
39:36ID but the user role. So I'm going to
39:38say here data class user ro is going to
39:42be a string. And now we're going to
39:43write a function that takes in the
39:45context from the model request and
39:46returns a system prompt dynamically
39:49based on the user role. So I'm going to
39:50use here the decorator dynamic prompt.
39:53And we're going to call this function
39:54user ro prompt. For example, it will get
39:57a request as input which is a model
40:00request and then we return the string.
40:04So that is the structure and now we can
40:06take the context from the request. So I
40:08can say here the user role is going to
40:10be request runtime context do user role
40:16and we can do the same thing as before
40:17with the user ID just a match case
40:19statement. So match user role and then
40:22we can have different cases. For
40:24example, the user can be an expert. And
40:26in this case, we will return a different
40:28system prompt. Now, what I want to do
40:29here is I want to define a base prompt.
40:32So, we're going to have just a base
40:33prompt. You are a helpful and very
40:37concise assistant. And now, if it's an
40:39expert, we're going to say here, give me
40:42the base prompt. And then just add to
40:44it, provide detail
40:48technical responses. On the other hand,
40:51if we have a beginner, we're going to
40:53say keep your explanations
40:57simple and basic. And finally, to see
41:00the biggest difference, we're going to
41:01say case child. And it's going to
41:03basically be explain everything
41:07as if you were literally
41:11talking to a 5year-old. And then the
41:14default case would just be okay. If it's
41:16none of that, then we're just going to
41:18return the base prompt. Cool. So now
41:20basically we do the same thing as
41:21before, but this time we don't pass it
41:23as context. We don't pass it as a tool.
41:25We pass it as middleware. So I say here
41:27create agent. The model is going to be
41:29the same as before. GBT4.1
41:32mini. And now I'm going to say here
41:33middleware is equal to and that's going
41:36to be a list and it's going to contain
41:38the user role prompt function. But of
41:41course I still have to pass the context
41:42otherwise it of course doesn't work. So
41:45context schema still has to be this data
41:48class. So then I can say response is
41:50equal to agent.invoke.
41:52I'm going to have to provide the
41:53dictionary again here with messages role
41:57is user and content is explain PCA. And
42:01here now I pass the context. The context
42:03is going to be that the user role let's
42:06say in the beginning is going to be
42:08beginner. So user role is equal to
42:12beginner. Print response. And we can see
42:14here content PCA principal component
42:16analysis is the technique used to reduce
42:18the number of variables in data while
42:20keeping the most important information.
42:21It transforms the original data into new
42:23variables and so on. So quite simple but
42:26still not something a 5-year-old would
42:28understand. So let's go and see what the
42:30export explanation would be. And there
42:32we can see principal component analysis
42:34is a dimensionality reduction technique
42:36used to transform a large set of
42:37correlated variables into a smaller set
42:39of uncorrelated variables called
42:41principal components. Yeah. Then we get
42:43here projection and component selection.
42:45Still not I mean I can decomposition.
42:47Okay, it's a little bit more technical.
42:49But if I go now to the lowest level here
42:51which is the child then I should get
42:53something really really simple. So here
42:55okay imagine you have a big box of
42:57crayons with lots of colors. Sometimes
43:00you want to choose just a few crayons
43:02that can still help you color many
43:04pictures nicely. PCA is like a magic
43:06helper. Yeah, you get the idea. So this
43:08is something we can do with middleware
43:10here. In between the request and
43:11response, we can have this dynamic
43:13prompt choice based on the user role,
43:15based on the context. We can say uh that
43:17we want to do different things. We want
43:19to instruct the model to do different
43:20things. Now, another thing that we can
43:22do is we can not only select a prompt
43:24dynamically, we can also select a model
43:26dynamically. For this here, I'm also
43:27going to import from langchain.hat
43:30models our trusted init chat model
43:33function. I'm going to get rid of
43:34dynamic prompt and I'm going to import
43:38wrap model call. This basically means it
43:40happens around the model call whenever
43:42we want to call the model that is part
43:45of the agent. We're going to run our
43:47middleware in that moment or I should
43:49say in between the model request and the
43:51model response. So the idea here would
43:53be I want to have a basic model that
43:55would be in it chat model and then I can
43:57say for example model is equal to GPT4
44:01mini and then I would also have an
44:02advanced model which could be 4.1 mini.
44:05The idea now is I define a function
44:07called dynamic model selection. This
44:10function takes in a request returns a
44:12response. So I take in a model request
44:16and the output of that function is a
44:18model response. In addition to that we
44:20also get a handler as parameter here and
44:24I have to annotate this now with wrap
44:26model call. So this actually happens
44:29when the model is called. In case you're
44:31interested in that, you can also go to
44:32the langchain documentation. And there
44:33you can see how this basically works.
44:35Request comes in. Then we have before
44:37agent, before model, wrap tool call,
44:39wrap model call, after model, after
44:42agent, result. And we're now doing this
44:44here, wrapping the model call with our
44:46middleware. So what kind of logic you
44:48want to apply here is up to you. In my
44:49case, I'm going to keep it simple. If I
44:51have more than three messages, I'm going
44:52to use the stronger model. Otherwise,
44:54I'm going to use the basic model. Not
44:56the most intelligent choice here, but
44:57we're going to do it like this. So
44:59message count is going to be equal to
45:02length of request state and then
45:05messages and then just if the message
45:08count is greater than three the model is
45:12going to be equal to basic model
45:14otherwise we're going to say it's going
45:17to be the advanced model and then we say
45:19request domodel is equal to model and we
45:22return a handler for that request. So
45:26basically we're just taking the request
45:27and setting the model to something else
45:29depending on some criteria. This could
45:31be something more intelligent and then
45:33we just handle the request with this
45:36change. So that's super easy to
45:37integrate. Again agent is going to be
45:39equal to create agent. Model is going to
45:42be basic model by default. And then the
45:44middleware is going to be our dynamic
45:46model selection. Now to keep things a
45:48little bit more beautiful here, I'm
45:50going to import again from langchain
45:53messages the system message, the human
45:56message, and the AI message. And then
45:58down here, I'm going to say response is
46:00equal to agent invoke. I'm going to
46:02invoke here on the following dictionary.
46:05Messages is going to point to a list of
46:08messages. I'm going to start with a
46:10system message. You are a helpful
46:12assistant. Human message. What is
46:15oneplus 1? Keeping it very basic here
46:18and essentially we can take a look at
46:20two things. Uh on the one hand you can
46:22get the idea if you look at the quality
46:24of the output. I mean oneplus 1 is
46:26something that every model should be
46:27able to do. But you can say here again
46:29print response messages
46:32-1.content
46:34to just get the answer but then you can
46:37also take a look at the actual model
46:39that was used to produce this response.
46:40So instead of saying content you can say
46:42response metadata and you can target the
46:45field called model name. And if we run
46:47this now we can see that we use the
46:51wrong model. Why is that? Because we're
46:54we have to swap this. Of course if we
46:56have more messages we're using the
46:57advanced model otherwise the basic
46:59model. But in theory it works. So if I
47:02run this now I should see that we're
47:03using 40 mini. And if I add more
47:06messages. So, for example, let's just
47:08add the same message a couple of times
47:10here. This is going to trigger the
47:12change and now we're using 4.1 mini.
47:14Now, in general, I want to show you how
47:15you can define your own custom
47:17middleware by using these hooks. So,
47:18we're going to do that with a class
47:20here. For this, I'm going to create a
47:21class called hooks. Let's call it hooks
47:24demo. It's going to inherit from a class
47:26that we need to import called agent
47:28middleware. So, we're now defining our
47:30own agent middleware as a class here.
47:33And we can just override these methods
47:35that are representing the hooks. So in
47:37order to do that, let me get rid of all
47:39of this. We're also going to need
47:40something called the agent state. And
47:42basically the constructor is going to be
47:44the init method taking self as a
47:47parameter here calling the constructor
47:50of the parent class. So super init. And
47:52what we're going to do is we're going to
47:54set a start time equal to 0.0. So this
47:56is not going to be an actually useful
47:58example. I'm just going to show you how
47:59we can trigger when the different parts
48:01are triggered. And now we can just
48:02overwrite before agent. For example,
48:05before agent is obviously before we get
48:07to the agent, before the request comes
48:09to the agent, we're going to be running
48:11this. It takes self and an agent state.
48:14So state, which is agent state, and also
48:16the runtime. And in my case, what I want
48:18to do here now is I want to say self dot
48:21start time is going to be equal to and
48:23now I need to import from core python
48:25here time. I'm just going to say time.
48:27So we're importing core python package
48:29time. self.start time is equal to time.
48:32And then just so we know it happened.
48:34I'm going to say here before agent
48:36triggered. Now I'm going to copy paste
48:38it here. For the other ones, it's just
48:40before model, after model, and after
48:42agent. Here we're going to do something
48:43else. So I didn't copy it. But basically
48:46here we're just printing before model,
48:47after model. Whatever you want to do
48:49with that. It's basically also you have
48:51wrap and so wrap tool call, wrap model
48:53call. We saw that already. We're just
48:55going to keep doing it with these four.
48:57And what I'm going to do here is I'm now
48:59going to calculate the time difference.
49:01And I'm going to print the result. So
49:02I'm going to say here print
49:05after agent and then colon time. So the
49:09current time minus self start time. And
49:12now we can go and say agent is equal to
49:14create agent GPT4.1 mini. And then we
49:19can say middleware is equal to and now
49:21we're going to pass hooks demo as an
49:24instance here. So we're creating
49:25instance of this class. This is another
49:27way to use middleware. And then I can
49:29just do the usual stuff from before just
49:31agent invoke some question. And now when
49:34I run this we get a problem of course
49:36because I cannot just call the module
49:38time. Of course I need to say time or
49:40time perf counter whatever you want to
49:42use here but that shouldn't be too much
49:44of an issue. So now we get before agent
49:46triggered before model. I forgot to add
49:50trigger here. But then we should get at
49:52some point after model and then we
49:53should get after agent and the
49:55difference here in time. And since PCA
49:57is probably quite complicated uh for a
49:59model to put into words. It takes some
50:02time but you can see 14 seconds is what
50:04we measured here. This is essentially
50:05what you can do with these hooks in um
50:08lang chain when it comes to middleware.
50:10You can customize what happens at what
50:12point in the workflow in the cycle. So
50:14now, last but not least, I would just
50:15like to show you a couple of examples of
50:17already existing middleware. We're not
50:19going to go too deep into this. I'm
50:20going to just copy paste one of these
50:22examples here into this uh file, which
50:26is going to be summarizing conversation.
50:28We're not going to actually use this.
50:29I'm just showing you how to use this.
50:31Basically, you go from langchain agents
50:33middleware summarization middleware. One
50:36of the many choices that you have. What
50:38this basically does is you have a couple
50:39of parameters here. You can specify the
50:42model for the summarization, which is
50:44not necessarily the same as the model
50:45that the agent uses. The idea is that
50:48after a certain number of tokens, you
50:51summarize the conversation. You keep the
50:53last 20 messages after 4,000 tokens and
50:57GPT40 mini in this case summarizes the
51:00important key points of the conversation
51:02up until this point. And then you just
51:03continue the conversation. That is one
51:05middleware that we can use. And then on
51:07the middleware page in the
51:08documentation, you can also find some
51:10examples. For example, here human in the
51:12loop middleware. This basically means on
51:14certain events on certain tool users, we
51:17get an interrupt and then the user has
51:19to manually say okay continue, edit,
51:21approve, reject, whatever. This is of
51:23course important in key steps. If you
51:25have to do some payment, if you have to
51:26send an email or something, it makes
51:28sense to have this human in the loop
51:29middleware. Then we also have model call
51:31limit. So basically saying we cannot
51:34call the model too often per run or per
51:36threat. Same thing exists for tool
51:38calls. Basically just a limiter for how
51:40often you can call a tool in a run or in
51:42a limit. Same thing also exists for tool
51:45calls. Basically how often can you call
51:47a tool in a specific thread or run. Then
51:49also some interesting stuff here like
51:51model fallback. If something doesn't
51:52work, if you cannot use a model, fall
51:54back to a different one. This can be
51:56quite useful here. Also PII detection.
51:58Basically personally identifiable
52:00information. You can make it redact
52:02certain key pieces of information for
52:05compliance reasons for example and much
52:07more other stuff like retrying tools or
52:10having a to-do list. So basically a
52:12planning middleware. You can take a look
52:14at these examples but basically
52:15middleware allows you to extend the
52:18capabilities of your model or of your
52:19agent I should say because you can tell
52:21it what to do in certain scenarios make
52:23it more dynamic make it more reactive so
52:26to say which is very useful. So that's
52:28it for this video today. I hope you
52:30enjoyed it and hope you learned
52:31something. If so, let me know by hitting
52:32a like button and leaving a comment in
52:34the comment section down below. Also, if
52:36you're interested on my website, you
52:37will find a tutoring tab and also a
52:39services tab. There you can hire me
52:41basically for one-on-one tutoring, for
52:43one-on-one teaching you something or
52:45also for services, machine learning,
52:47backend development, consulting,
52:49freelancing, whatever. If you're
52:50interested in that, check it out. You
52:52can contact me via mail or LinkedIn. And
52:54besides that, don't forget to subscribe
52:56to this channel and hit the notification
52:57bell to not miss a single future video
52:59for free. Other than that, thank you
53:00much for watching. See you in the next
53:02video and bye.

About this transcript

This page contains the full transcript of LangChain Full Crash Course - AI Agents in Python by NeuralNine, generated from the public captions YouTube serves with the video. The transcript has 10,535 words across 1,463 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.