YT YouTube2Text

YouTube2Text → Transcripts → How Claude Code Works (By Building It)

How Claude Code Works (By Building It) — Transcript

by Rivaan Ranawat · 192,770 words · 26,239 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00AI coding agents are everywhere. Clot
0:02code, cursor, codeex, Gemini CLI. For
0:06the longest time, I thought, how complex
0:08could building this really be? You send
0:10a prompt to an LLM. It generates some
0:12code, maybe runs a command, that's it.
0:15And honestly, I wasn't wrong. It really
0:17is just an LLM. But what changed for me
0:20was realizing how much leverage you get
0:22from everything around it. Once you
0:24start adding scaffolding things like
0:26tools, memory, context management,
0:28feedback loops, the model stops feeling
0:31like a chatbot and starts feeling like a
0:33system. It's not smarter. It's just more
0:35useful, more reliable, and more
0:37intentional. So, I wanted to see what
0:40that actually looks like when you build
0:41it yourself. This is a cloud code style
0:44agent built from scratch. It can read
0:46your codebase, write and edit files, run
0:48commands, search the web. It remembers
0:51important context about you across
0:52sessions, plans, executes, and even
0:55spawns sub aents when tasks get complex.
0:58When context gets too long, it compacts
1:01and prunes so it can keep running until
1:03the task is done. It catches itself when
1:05it's looping. It also learns from its
1:07mistakes through a feedback loop. And
1:09users can extend it. They can add their
1:12own tools, connect to thirdparty
1:13services through MCP, control how much
1:16autonomy it gets, save sessions, restore
1:18checkpoints, and more. In this video,
1:21we'll build the entire thing. If you
1:23know Python fundamentals, that's enough.
1:25I'll explain everything else. Let's get
1:28started. Let's start by creating our
1:30project. So, I'm going to CD into
1:32desktop. CD stands for change directory.
1:34So, I'm going to the desktop folder. And
1:36over here I'm going to create a new
1:38folder which I'll do by typing in mkdir
1:41which stands for make directory. And
1:43let's call this AI agent. You can give
1:45your AI agent any name you want. I'm
1:47just going to go ahead with the simple
1:49AI agent. After that I'm going to cd
1:52into AI agent again and I'm going to
1:54open this up in Visual Studio Code. If
1:57you want you can use any other editor of
1:59your choice but I'm going to move ahead
2:00with Visual Studio Code. Once we have
2:03that done, I'm going to open up the
2:04integrated terminal that the VS code
2:07provides us with and after that I'm
2:10going to create a virtual environment.
2:12Why do we need a virtual environment?
2:14Well, think of it this way. Whenever we
2:16try to install a dependency using pip,
2:18it globally installs it in your system.
2:20But let's say we have two projects which
2:22use the same dependency and both of the
2:25projects use the same dependency but
2:27have different versions. So one for
2:30example AI agent uses dependency with
2:33the version 1.0.0
2:35and another project on the same machine
2:37uses the same dependency but with the
2:40version 2.0.0.
2:42They're conflicting, right? And we can
2:45only have one version installed in our
2:47system. That's what pip does. So we need
2:49two different versions and that's why we
2:51need a virtual environment which will be
2:53able to keep all the dependencies to
2:55itself. it won't be installed globally
2:57in the system and it won't interrupt
3:00with any of our other projects. So to
3:02create a virtual environment we can type
3:04in python - m followed by v env and the
3:09name of the folder which is venv. So
3:12python- m venv stands for well we want
3:16to create a virtual environment and this
3:19is the name of the virtual environment
3:21folder. Once I hit enter, the folder
3:24with VNV is created, which is the
3:26virtual environment folder. Now, if this
3:29command did not work for you, you can
3:31try using Python 3- MVNV. VNV, and that
3:35should work for you. Otherwise, you'll
3:37have to install Python and look into all
3:39of that. Now, if you expand this VNV
3:42folder and expand the bin folder within
3:44it, you see the activate script. That's
3:47what we want to run. We've created the
3:48virtual environment. Now we want to
3:50basically activate the virtual
3:53environment. We know the virtual
3:55environment is not started because
3:57there's no prefix or there's no
3:58indication telling us that you know we
4:01have the virtual environment started. So
4:04to run it what we need to do is
4:05source.v/bin/activate
4:10and then hit enter. Once you do that you
4:13can see the prefix of venv. This
4:16suggests that the virtual environment is
4:18up and running. Now I'll just go ahead
4:20and create a new file. And let's call
4:23this file main. py. Here I'm just going
4:27to print out hello world. Nothing too
4:29fancy because I just want to ensure that
4:31everything within this virtual
4:33environment is working out fine or not.
4:35Now to run this script, I can just run
4:37python which is present over here or you
4:40can run python 3 if you want followed by
4:43main.py the file I want to execute. And
4:45it does print out hello world. That's
4:47great. So everything is working. Now
4:50what is the step one to building an AI
4:53agent? Well, step one is connecting to
4:56an LLM, right? We are not going to look
4:58into agent as of now. Agentic stuff will
5:01come in when your LLM starts to take
5:03action. But we're not adding any action
5:06stuff as of now. First, we need to get
5:08the LLM in. We need to create a
5:10connection to the LLM and fetch the
5:12responses from the LLM. Once that's
5:15done, we can look into all of the
5:17additional agentic stuff like taking
5:19actions. So, first of all, I'm just
5:22going to go ahead and create a folder
5:24called client. Within this client, I'm
5:26going to have a new file and let's call
5:28this llmclient. py. This will handle all
5:32of the response fetching, the streaming
5:35of the response, and you know, retrying
5:37with exponential backoff. All that sorts
5:40of stuff will go into this LLM client.
5:42Now we have a requirement for our LLM
5:44client. We want it to operate with any
5:47LLM provider of our choice. How do you
5:51get that done? Well, to do that, you can
5:54use multiple packages that do allow this
5:57like pyante which also has support for
5:59creating your own agent by just
6:01mentioning one line. But since we are
6:04interested in building our own agent
6:07from scratch, we're going to use this
6:09dependency, the OpenAI dependency. This
6:12comes directly from the OpenAI company,
6:15but they allow us to change the base URL
6:18so that we can connect to any LLM
6:20provider of our choice. Other options
6:23include Open Router SDK, which you can
6:26use completely fine, but I'm going to
6:28stick with OpenAI. And within OpenAI,
6:31we're going to specify a base URL that
6:33points to Open Router. And I'll talk
6:34more about Open Router in just a minute,
6:37but let's just install OpenAI first.
6:39Also, I'm going to terminate this
6:41terminal. Open up the integrated
6:43terminal here again and then run pip
6:46install openai. Now, if that doesn't
6:48work for you, you can try using pip 3
6:51and that should work for you. But
6:53anyways, now that openai package is
6:55installed, I can start depending on it.
6:58So, what is the first thing that I need
7:00to do? Well, over here I want to create
7:02a class called llm client which is going
7:05to have an init function. Obviously,
7:07this init function is going to return
7:09well nothing. So, I'm just going to pass
7:11the return types as null. And here
7:14there's going to be multiple things
7:16coming in like a configuration system
7:18that we're going to add because the user
7:20can specify their own API key. That's
7:22something we'll need. The user can
7:24specify their own base URL so that they
7:26can connect to any any LLM provider of
7:29their choice. But as of now, we're going
7:32in simple. We're hard coding everything.
7:34So if you're uploading your project to
7:36GitHub, this might not be the right time
7:38to do it because your environment
7:40variables will not be stored in
7:42environment variables. They'll be stored
7:44or hardcoded in this code itself, which
7:47is not good. Anyways, the first thing
7:49that we need is a client. You know, we
7:51have to establish the OpenAI dependency.
7:55If you look at the documentation present
7:57over here, this is how you can create a
7:58client. But this is synchronous. We want
8:01an asynchronous response. So this is
8:04what we'll be going with async usage
8:07where we use async OpenAI instead of
8:09OpenAI and then we can use async await
8:12to do all of our tasks. That's really
8:14good. So let me go ahead and import this
8:17from OpenAI.
8:19Awesome. Now I'll have self.client and
8:23client can be a protected or a private
8:25variable. Anything you want. I'm going
8:27to keep it private or protected so that
8:30you know it doesn't go outside of this
8:33class. The client is only used within
8:35this class and after that it can be of
8:38the type of async open AAI or null. And
8:41initially we're going to instantiate it
8:43as null. Now why are we not doing async
8:46openai like this over here? You can
8:48totally do that but the reason we're not
8:50doing is because we're going to create a
8:52separate function to get the client. So
8:54we're going to have def get client and
8:57this get client is going to get self.
8:59It's going to return async openAI an
9:02instance of async open AAI. But first
9:04we'll check here if self.client
9:07is none that means it has not been
9:10instantiated ever or it has been
9:13instantiated before but the dependency
9:16was closed. You know that means a
9:19connection was established, the request
9:21was completed and let's say the entire
9:24thing was completed you know so the
9:26client closed that's totally possible
9:29because later on or actually now itself
9:32we can create a function called close
9:35and it's going to be an asynchronous
9:37function and what this close function
9:39does is that it doesn't return anything.
9:42So that's the good thing about this.
9:44What it does is it just checks that hey
9:46listen if self.client is present that
9:50means it is not none then I want to
9:53close this client's connection. So we
9:55have self.client.c close and then I can
9:58just set self.client equal to null and
10:01we are going to use this close function
10:04in our own code but it's good to have
10:06this helper function. Right now all it's
10:08doing is closing the connection that has
10:11been established. And if this is done
10:13and let's say in the future we again
10:15want to get this client and get some
10:18response from the LLM we'll call this
10:22function it will check that hey the
10:24client is null because we set it to null
10:26over here. So let me just create another
10:28instance which establishes a new
10:30connection. And this check also
10:33establishes that hey we are only going
10:35to have one instance of client every
10:38single time because we only create an
10:41instance if self.client is null
10:43otherwise we are just going to return
10:45the self.client you know so that's good.
10:49Now let's create an instance of the
10:51client. So we're going to have
10:52self.client is equal to async openai and
10:57then I'll pass in the API key. That's
10:59one of the things we require. And then
11:01we need a base URL as well. There's
11:04nothing related to model over here
11:06because when you're trying to establish
11:08connection, you're not really choosing
11:11the model. The model of what LLM you
11:14want to use will be used whenever you
11:17send a message, right? So every message
11:19that you send to an AI agent can have a
11:22different model that you use with it.
11:25And we're going to support that as well.
11:28So there's no one fixed model. The user
11:31can just change the model any time. And
11:33in the future, if you want to expand
11:35this application, you can have the
11:37router where the user can just select an
11:39auto model and you route to the model of
11:42your choice by putting in some logic.
11:44We're not going to cover that, but it's
11:46something that cursor does. So, let me
11:48just remove this. And now we need to
11:51pass in two things. API key and base
11:53URL. Now, to get the base URL, we're
11:56going to use open router. Now base URL
11:59can also point to the Gemini API URL. It
12:03can also point to Anthropics API. It can
12:05also point to OpenAI. But generally
12:07whenever you're using OpenAI, the base
12:09URL is the OpenAI URL. So you don't need
12:12to worry about that. But we'll be using
12:14Open Router. Open Router is the unified
12:17interface for LLMs. You'll find all of
12:20the models over here. You can try seeing
12:22the model by Z AI. So if you look into
12:25the providers there's one by Z AI deep
12:27infrar parail nova AI and so on like
12:31there are so many providers and based on
12:33the uptime it will just route to the
12:35correct provider so that your
12:37application always fetches the response
12:41and obviously you can monitor almost
12:42everything over here like throughput
12:44latency uptime everything is monitored
12:47over here another reason I'm using open
12:49router is because there are a bunch of
12:51free models that we can take use of and
12:53none of the models by Anthropic or
12:56OpenAI seem to be available for free.
12:59So, we'll make use of that. Now, to get
13:02the base URL, you can go to this Open
13:04Router quick start guide where they
13:06mention multiple ways to get started
13:09with Open Router. Since we're using the
13:10OpenAI SDK, this is how they say it.
13:13This is the base URL that we should be
13:16using. Let me copy that and paste it
13:18over here. Later on the user will be
13:20able to configure this base URL by
13:22themselves using the configuration
13:24system we're going to add. The same goes
13:26for API key. But as of now let's get the
13:30API key for open router. So we can go to
13:32open router sign up. You can select any
13:35of the options available here. I'll see
13:38you after I've created an account. And
13:40once you've done that you can go over
13:42here click on keys and then create a new
13:45API key. So, you can give it any name of
13:48your choice. I'm just going to call it
13:49AI agent. Again, there's no expiration,
13:53but you can obviously set it. I'll set
13:55mine to 30 days. I think don't use my
13:59API key. I'll definitely delete it
14:01before the video gets public. Create
14:03your own. It's absolutely free and
14:05you'll only pay if you want to. They've
14:08not taken any credit card as of now. So,
14:10don't worry. We can pass in the API key
14:12over here. Again, we're going to take
14:14all of that through the configuration
14:16system and there's going to be all the
14:18environment variables that are created.
14:21So, don't worry. As of now, this is the
14:23setup. Cool. So, we have the connection
14:26established. Now, the next thing we are
14:28interested in is well, the response,
14:31right? We want to fetch the response
14:34from the LLM. Now, there's two types of
14:37responses that you can get from an LLM.
14:39One is a streaming response. Another one
14:42is a non-streaming response.
14:43Non-streaming response means we just
14:46wait for the LLM to give us an entire
14:48response. Okay, that's cool. Another one
14:52is a streaming response. Streaming
14:54response means whenever you talk to an
14:56AI agent or an LLM, you might have seen
14:59that the LLM gives out a bunch of text,
15:02you know, chunks of text like 30 words
15:06or something together. It does that
15:08because LLMs are auto reggressive. they
15:11generate one token at a time and
15:13whenever you know a bunch of tokens are
15:15ready it just sends it through the API
15:16for us and if we have the non-streaming
15:19response version you'll see the entire
15:21response when it's done so it might take
15:24let's say 15 seconds to get the entire
15:27response that's not a good a user
15:28experience that's why they stream the
15:30response as soon as a bunch of tokens
15:32are available they send it to us so we
15:35show the user a response every 1 second
15:38so first 30 words or first 30 tokens are
15:42coming in in the first second itself and
15:45then another 30 then another 30 like
15:48that. So the user doesn't have to wait
15:50the entire 15 seconds. They can just get
15:52started reading the entire thing as soon
15:54as the first set of words come in. We
15:58are going to implement both of these
15:59streaming and non-streaming responses.
16:02The reason for that is streaming
16:03response will be shown to the user.
16:05That's good user experience. But the
16:08non-streaming version will be required
16:10when we have to do compaction or
16:13summarization when we have hit context
16:15length. We're going to talk more about
16:17that later on. But that's something we
16:20will need. So let's just create it as of
16:22now. And let's just get done with the
16:24LLM client. So let's create a function.
16:28This function is going to be chat
16:30completion. Cool. Now let me just
16:33lowerase this.
16:35It's going to have self. Then it's also
16:37going to have messages which is going to
16:39be a list of dictionary of string, any
16:42and I need to get any from typing. So if
16:46you just press command full stop, you'll
16:48see add from typing import any because
16:52we do want to import this any keyword.
16:55And then we can decide what to do.
16:57Another argument that we'll need here is
16:59stream. It is going to be of the type of
17:02boolean. And by default it is true
17:04because everything should be streamed.
17:07Now what is this messages and stream? So
17:10the thing about LLMs is that they are
17:13stateless. They do not really contain
17:15any state. If you have a bunch of
17:18messages for example whenever I'm
17:20talking to chat GPD I say hi that is one
17:23message from my side and then there's
17:26this message from chat GPT's end. So
17:29it's a list of messages, right? The
17:32first one has a role of user that it's a
17:35message I've sent and this is the text.
17:38The second one is coming from the
17:40assistant and it's the message they
17:42have. Now if I want to continue this
17:44chat, let's say I just type in nothing
17:47much. You know, I'm continuing this
17:50conversation. So the LLM doesn't
17:53inherently remember that these were the
17:55text that was in this conversation. No,
17:59whenever we send another text, this
18:02entire list that was present gets resent
18:06to the LLM. To contrast this with
18:09something like a actual chat app like
18:12WhatsApp
18:13where whenever you send a message only
18:15the last message gets sent that's not
18:18the case over here in chat GPT all of
18:21the messages are sent back because
18:24that's how the auto reggressive
18:26generation will work. So again this
18:28message, this message and this message
18:31gets sent so that we get this message.
18:33That's what I mean by stateless. It
18:35doesn't really contain any state. We
18:38have to keep sending it all of this
18:40data. And that's why we have messages
18:42which is a list of dictionary. It is a
18:44list because we'll have multiple
18:46messages. It's a dictionary because it
18:48will contain two states or actually more
18:50than that. But as of now two states
18:53there will be a role. Whose message is
18:55it? Is it my message or is it the
18:58assistant's message? So that's why a
19:00dictionary. Awesome. Based on the
19:02streaming version, we have to decide
19:05if we want to stream the response or do
19:08we have to non-stream the response. So
19:11let's have the condition here if the
19:14stream is true. That means we have to
19:17create a new function that will stream
19:19the response. So let me just create self
19:22dot stream response. you know I'm just
19:25trying to write good code otherwise we
19:28don't have stream response so we have
19:31nonstream
19:33response these are both protected or
19:36private methods because I don't want to
19:38use them outside of the class there's
19:40only this function related to streaming
19:43that will be exposed outside of this
19:45class so that whenever we want to
19:47connect to an LLM or get a response from
19:50an LLM we just call this function only
19:53source of truth Now let's go ahead and
19:56create both of these functions. So first
19:58we have asynchronous def which is going
20:01to be stream response. There's going to
20:04be a type associated with it. It's not
20:06going to be a simple string or anything
20:08of that sort. It is going to be an async
20:11generator because we are streaming the
20:13response. We're going to use yield if
20:15that's something you know in Python.
20:17Otherwise I'll walk you through it.
20:19Don't worry. But as of now I'm just
20:22going to pass
20:24then another function which is
20:26non-stream response and then this
20:30function is going to return directly the
20:34event and that event is going to have
20:36multiple things we'll talk about that
20:38but yeah these are the two things that
20:40are created great now let's go ahead and
20:43complete one function I'll start off
20:45with non-stream response now what does
20:47non-stream response require over here
20:49the first thing is the client so let's
20:52take in the client which is going to be
20:53of the type of async open and that's
20:56true for stream response as well. Both
20:58of them are going to take client because
20:59at the top over here we're going to
21:01instantiate the client itself so that
21:04there's no code duplication. So we have
21:07client is equal to self dot get client
21:11that's great it returns async openi
21:14that's also good and another thing we
21:16will require over here is the keyword
21:18arguments so quarks and it's going to be
21:22of the type of dict and it will have the
21:25key as a string and the value can be
21:28anything because it can either be a
21:31string as the value it can also be an
21:33integer as the value we don't know or
21:36actually we do know but everything will
21:40have a different value. It's totally
21:42possible.
21:43Now what are the keyword arguments that
21:45we need over here and how is it useful?
21:47Well the these keyword arguments are
21:50essentially the request arguments for
21:53example what model do we want to use?
21:55What list of messages do we want to send
21:57to the model? And what is the stream
21:59option? Is it true? Is it false? What is
22:02it? So let's just create it at the very
22:05top. So we have keyword arguments is
22:07equal to and then we have model. The
22:11model will be set up by the user. So
22:14it's going to come in from the
22:15configuration system but we don't have
22:17it as of now. We'll hardcode it. Then we
22:20have messages which is going to be a
22:22list and we already do have access to
22:24it. And then we'll require the stream
22:27option which is also present to us and
22:29we'll just pass it in. So as you can see
22:32messages is a list, stream is a boolean
22:35value and model is going to be a simple
22:37string. Now let's decide the model we
22:40want to use. If you go to open router,
22:42make sure you've copied your keep. You
22:44won't be able to use it again. You won't
22:46be able to see it again. And then close
22:49it. Then we can go to models where a
22:51list of models are available. You can
22:53see the pricing listed out over here.
22:55for example, Gemini 3/U,
22:58which is 0.5 or million of input tokens
23:02and $3 per million output tokens.
23:06So that's there, but we're interested in
23:08free models. If you're interested in
23:10free models, you can click over here
23:12prompt pricing. You can set this to free
23:16and you can see all of the models that
23:18are available for free. They'll
23:20obviously take your data so that they
23:22can reuse it for their model training.
23:25But for development, this is good
23:27enough. You can try it. I would
23:29recommend you to try out bigger and
23:31better models because they obviously
23:34work better in agentic tasks. I'm also
23:36going to select this particular model,
23:38Mistral Destral 2, because I tried it
23:42offline and it worked really well with
23:45agentic coding. That's what we're going
23:47after. So, I'll use it. Now if you have
23:49money to burn you can use clawed models
23:53for our agent tech stuff. It does tool
23:56calling extremely well. I don't think
23:58any other model comes close to that
24:00performance in tool calling as of now.
24:03So yeah definitely a choice you can
24:05take. So I'll pass in the model name
24:06over here and that's good. We have
24:09everything set up. Now I can pass in the
24:11keyword arguments and the client both of
24:13them. So I'll pass in the client. I'll
24:16pass in the keyword arguments and boom,
24:19there we go. Now in this non-stream
24:22response, the very first thing I like to
24:23do is well get the response from the
24:27LLM, right? So we'll have client which
24:30is the client that we have over here dot
24:33chat dot completions dotcreate because
24:37we want to create a completion. This is
24:39how you create a chat. And here you need
24:43to pass in all of the keyword arguments,
24:46right? You have to specify what model
24:47you want to use, what are the previous
24:49messages. And you also need to append
24:51your current message into this. All
24:52right? For example, we add hi, hey,
24:56what's up? And then we sent in nothing
24:57much, right? We have to specify nothing
25:00much in this messages list.
25:03So messages really cannot end with an
25:06assistant. messages always has to end
25:08with a user because you want a response
25:11back, right?
25:13And this is the response that we'll be
25:16getting. So let's await it because
25:18create will return a cool routine. We
25:21want to await it and then we can pass in
25:23the keyword arguments. I'm doing this so
25:26that we deconstruct this object and pass
25:29everything together. But yeah, that's
25:31done. Now we can go ahead and print the
25:34response. And if you see there are
25:36multiple things returned by the
25:38response. What we are interested in is
25:40choices. But anyways I'll just print out
25:42response and see what we get. Now I'll
25:44go to the main.py function and call this
25:47chat completion with stream set to false
25:50so that we can get a non-stream
25:51response. So let me create a def main
25:55function.
25:57Within this function I'm going to create
25:58an instance of llm client. So I import
26:03that from client. LM client. I don't
26:05have to pass in anything as of now. But
26:07as I said later on, we're going to pass
26:09in the configuration system. And now I
26:12can just call client dot chat
26:14completion. The messages is going to be
26:17a list. I'll hardcode it as of now. Then
26:19we'll look into taking it from the
26:21constructor or from the user arguments
26:25through the CLI. But let's just go one
26:28step at a time, right? So we have a
26:29messages list which is equal to well
26:32something like this. You need to specify
26:35the role. What role is it? So the first
26:38one is going to be the user. Then you
26:41specify the content and let's say the
26:43content is what's up. All right. So
26:48that's there. Now we can pass in the
26:49messages. Obviously we're going to
26:51expand on this because we need to pass
26:53in the system prompt what the LLM has to
26:57do given the user query. Then the user
26:59query comes in and then we'll wait for
27:01the assistance response. We're going to
27:03append it to this messages list. And
27:05there's a whole lot of context
27:07management that we'll have to do to get
27:08it up and running. But let's just pass
27:11in messages now. And then the stream
27:13value is going to be false. Now you can
27:16also have asynchronous def. Then we
27:18await it and then we say print done just
27:22in case you know nothing gets printed
27:24out. We know that this worked out and
27:27there's some error happening over here.
27:30So what's going to happen is this
27:31function gets called then we get a
27:35client. We see that the stream is false.
27:38So we call this function and then it
27:42contacts open router. It prints out the
27:44response. So let me just clear off the
27:47terminal and run python main.py again.
27:51And nothing gets printed out because we
27:53have not called the main function. Let
27:54me just go ahead and call this main
27:56function. Now let's run it again.
27:59And it says co- routine main was never
28:02awaited. So we get this error and the
28:04reason for it is this is an asynchronous
28:07function. We have to call await. And
28:08obviously we can't call await like this.
28:10So to run this what we need to do is
28:13import async io at the top and after
28:16that we can just do async io dot run and
28:20then call the main function within that.
28:23Now we can try running Python main.py
28:25again and again we've made the same
28:28error that non-stream response was never
28:31awaited. Let's await that as well. Now
28:33let's run it again.
28:36And we are waiting for the response
28:38right now. And as soon as the response
28:40comes in, we print this out and we get
28:42done. So this is the entire chat
28:45completion object that we get. There's
28:48an ID associated with it. We get
28:50choices.
28:51choices is basically what the LLM
28:55responded with. You can modify this
28:57choices. As of now, if you notice, we
29:00only get one choice over here. There's
29:02only one prediction that's done, one
29:05completion that's done. If you want, you
29:07can extend this by passing in the
29:09correct data within either the client or
29:13the keyword arguments. I'm not too sure.
29:15You'll have to look that up. But yeah,
29:17in our case, we only need one choice.
29:20The reason multiple people do several
29:22choices, they increase the number of
29:23choices, which means you have to wait a
29:25longer time to get the response. But the
29:29benefit of that is you can select which
29:31response you like better and display it
29:34to your user. But anyways, I just needed
29:36one. So this is our choice. It gives us
29:39the finish reason. We'll have to store
29:41that. Why did it finish? You know, did
29:43it error out? Did it actually stop? What
29:47is it? Then we have the message which we
29:50are interested in which is the chat
29:51completion message and this is what the
29:53LLM responded us with. Hey not much just
29:56chilling in the digital void ready to
29:58help with whatever you need. What's up
30:00with you? So that is the message we'll
30:02have to store. Then we have refusal. Did
30:06it refuse because it reached some safety
30:09concern or something? Then what is the
30:11role? It is an assistant responding to
30:13us. Then audio function call is what
30:16we'll be interested in. tool calls is
30:18what we'll be interested in. If there's
30:20reasoning, it will respond with that as
30:22well. Then we're also interested in
30:24usage.
30:25How much usage happened over here? The
30:28completion tokens were 32. That means
30:30this entire response was 32 tokens and
30:34our prompt tokens were only six. Yeah,
30:38just two or three words and six tokens
30:42were consumed in that because that's how
30:44tokens are counted. Tokens are not
30:46equivalent to words. Remember that
30:48tokens are equivalent to the smallest
30:51kind of possible word and I won't talk
30:54in detail about that because that will
30:56just make this an LLM course. Not
30:59interested in that. This is the total
31:01tokens which is just completion plus
31:02prompt tokens. And yeah, these are all
31:05of the things. The provider is also
31:07mentioned if you want to display that.
31:09Great. So the non-streaming response is
31:12working. Now let's extract everything
31:14and return it out. I'll just scroll up a
31:17little bit. Always have this terminal up
31:20now because we are interested in
31:22referring to what is present over here.
31:24So the very first thing is the choice
31:26because we need to extract the message,
31:28right? So we have choice is equal to
31:30response dot choices because well that's
31:34the one and we are only interested in
31:36the first index because we'll only get
31:39one message always. If we get more than
31:42one not interested I'm only interested
31:44in the first message. Then we have
31:46message which is equal to well whatever
31:48choice gives us because it's this
31:50particular object and I want to extract
31:53its message over here. Cool. So we have
31:57choice dot message.
31:59Now obviously we are interested in the
32:01text over here and to extract the text
32:04content we'll first have to check that
32:06hey if message doc content is present
32:09that means the message object has the
32:12content property if it does not we might
32:14be running into an error but if message
32:16has a content then we are interested in
32:19the text the text is going to be well
32:22none initially it can be the case the
32:25LLM responds with no text it only gives
32:28us a bunch of tool calls to do meaning
32:31we have to take a bunch of actions but
32:34we're not interested in that as of now
32:36we are more interested in getting the
32:38text. So we have text is equal to and
32:41now we need to create something known as
32:43a text delta because
32:45well we come to the point of what will
32:48be returned from this non-stream
32:49response. I told you that there's a lot
32:52of things that we actually need. We need
32:55to get access to the usage. What was the
32:59finish reason? Is there any tool calls?
33:02What is the text? What is the type of
33:04the message? Because it can be that the
33:06entire thing errored out or it can be
33:09that you have the message completed. And
33:12when we include streaming response,
33:14there'll be more events that will be
33:16added. So what I'm going to do is go to
33:18this client folder and create a new file
33:21called response. py. This response py is
33:25going to contain a new schema or a new
33:28class definition of what the event from
33:31a model can be. And I'm going to call
33:33this stream event. Obviously, this is
33:36going to be a data class. So from data
33:38classes, we can import data class. The
33:41reason it's going to be a data class is
33:43because I don't want to write out all of
33:44the representation string and equal to
33:47methods. Data class will handle all of
33:50that. All I need to type out is what are
33:54the attributes of this class. So the
33:57very first thing is going to be the type
33:59of the stream event. What type is it
34:01going to be? I told you there will be
34:03multiple types but let's just add types
34:06as we want it on the fly. So the very
34:09first class we are going to have here is
34:11event type where it's going to be an
34:14enum because event types can be multiple
34:17things. It can be a text delta. Text
34:20delta refers to any text that the LLM
34:22gave you. Specifically in streaming
34:25responses, whenever you get those bunch
34:28of tokens, a first set of tokens, those
34:31are text delta because the message has
34:34not been completed yet. You're still
34:35waiting for the text, but you have a
34:38text. Then another event is going to be
34:40message complete that hey, the message
34:42is completed. The LLM is done. Then
34:44there's going to be error. then there's
34:47going to be tool related stuff as well.
34:50So that's why we need an event type over
34:52here. It's going to be a string and an
34:54enum and we're going to import the enum
34:57from enum. So let's do that quickly. Now
35:01these are all the types of streaming
35:03events. We going to have text delta
35:05which is going to be text delta and here
35:09when we have text we're going to create
35:12a text delta. We we'll talk about that
35:14in just a minute.
35:16Then we have message complete.
35:20And then let's just call this message
35:22complete. You can probably also give
35:24this an auto. I don't think it would
35:26matter. But yeah, then we have error.
35:29And we just pass error. There's going to
35:32be more event types related to tool
35:33calls. We'll add that later on. Now
35:36let's pass in the type of this event
35:38type.
35:40The next thing that our streaming event
35:42is going to have is a text delta.
35:45Because if the event is a text delta, we
35:50actually need to capture the text,
35:52right? So this is going to be of the
35:54type of text delta and it can be null.
35:57Remember that by default it is null. But
36:01yeah, the user can enter any sort of
36:03text delta. Now why can it be null? As I
36:07told you before, if you have a tool call
36:09and no text given out by the LLM, you
36:13don't have a text delta. Now let's go
36:15ahead and create that text delta. It's
36:18going to be quite simple. Add the rate
36:21data class. And I forgot to put data
36:22class for this event type. And even over
36:25here we have data class. And then we
36:28have text delta.
36:31Content is going to be of the type of
36:33string. And I'm going to override the
36:36string method
36:39so that I just return the self.content.
36:42The text delta is going to return
36:44self.content.
36:46It's not going to contain anything
36:48related to text delta followed by
36:50content. Nothing like that. Just return
36:52the string. After that, we're going to
36:54have error. And error, you can create
36:58another class for that. But errors are
37:00generally just strings. The reason I had
37:03text delta within a class is because you
37:06can easily extend it. Maybe sometime in
37:08the future you realize just content is
37:10not enough. you want to add some more
37:13data with it. So that's why it's a
37:16class. But error is probably always
37:18going to be a string. So it can either
37:21be a string or null. And well it is
37:25defaulted to null. Then we have finish
37:28reason which is equal well the type of
37:31that is also going to be a string and by
37:35default it is null. And then you have
37:38usage. usage is all of these things all
37:42the completion usage stuff. So I'm going
37:45to create a new data class for that as
37:48well. We have aberate data class and
37:50then we have class token usage. This
37:53token usage class will be used quite a
37:55lot because based on the token usage we
37:58will be determining how much time we
38:02have to wait before we can do any sort
38:04of context compaction. So this token
38:08usage class is going to store all the
38:10statistics related to token usage and it
38:13will be quite helpful.
38:15Probably our entire app is going to def
38:18depend on this particular thing. So we
38:20have prompt tokens which is equal to
38:22zero. So prompt tokens are this thing.
38:26Whatever you prompt even the system
38:28token or system prompt counts as a pro
38:31prompt token. Then you have completion
38:33tokens
38:35which is going to be an integer again
38:38defaulted to zero. Let me just open up
38:40and see. Yeah, this is completion
38:42tokens. Then you have total tokens which
38:46is an integer which is equal to zero.
38:48And the reason we have total tokens is
38:51because it's over here as well. Then we
38:54also going to have cached tokens. So we
38:57have cached tokens which is an integer
39:00which is equal to zero as well. Now
39:02let's create an add magic function. Now
39:05the reason we are adding add over here
39:07is because I know how I want two token
39:12usages class to be added. For example,
39:14if you have A which is of the type of
39:17token usage and then you have B which is
39:19also of the type of token usage, how
39:21should it look like? So you have two
39:23classes that are being added to each
39:25other. So what we need to do is well
39:28just add up the prompt tokens of A and
39:31the prompt tokens of B to get the new
39:34prompt tokens right. So here we're going
39:36to have other which is also going to be
39:38of the type of token usage. Now since we
39:41using token usage within the class of
39:44token usage we'll have to do something.
39:47We'll have to import at the top from
39:50future import annotations and that will
39:54allow us to specify token usage like
39:56this. Understand the problem. The
39:59problem was that we have token usage
40:01class created over here. How are you
40:03using token usage class within this
40:06class if you've not you know
40:08instantiated it like it's not possible.
40:11we imported from future annotations and
40:14that allows us to do this otherwise
40:17before some Python version I think it's
40:203.9 you had to specify token usage like
40:23that and it would work out but now we
40:25can just import this and use token usage
40:28like that great now let me just return
40:31the new token usage class so we have
40:33prompt tokens the prompt tokens is just
40:36equal to self doprompt tokens whatever
40:39you have over here plus other dotprompt
40:43tokens. Then you have the same for
40:45completion tokens. Self dot completion
40:47tokens plus other dot completion tokens.
40:51Then you have total tokens is equal to
40:53self dot total tokens plus other dot
40:56total tokens. And finally you have cash
40:59tokens equal to self.cash tokens plus
41:02other.cash tokens. Just to explain
41:05what's happening to you, we have A which
41:08is of the type of token usage and then
41:10you have B which is of the type of token
41:11usage. So what's happening is this A
41:15gets triggered. This add function will
41:17get triggered and then B is passed as
41:20other. So you're essentially having A
41:23dot add B like that. So B is passed as
41:27other and what we're doing is that hey
41:30A's prompt tokens plus B's prompt tokens
41:33is the new token usage because we have
41:36to add both of them up. So that's how
41:38it's working. It will be quite useful
41:42because yeah we will be using this more
41:45often than you think and the usage is
41:47just token usage or null and by default
41:52it is null because many times it can be
41:55the case that maybe the LLM just doesn't
41:58send us the usage. It has happened to me
42:00with another provider. So we're just
42:02handling that edge case. So that is a
42:05stream event. There's going to be more
42:07tool call related stuff added over here.
42:10But as of now looks good. Now text over
42:13here that we had well this text is going
42:16to be of the type of text delta. So
42:19let's just rename this to text delta.
42:21Again the reason we have text delta is
42:23because it will be really helpful with
42:26stream responses.
42:28It is called text delta because delta
42:30refers to a change and we have a change
42:33in text. So that's why text delta. Now
42:36let's create text delta. You have to
42:38import it from client. Just import it
42:41correctly. And then you have to pass in
42:43the content. Content is just message dot
42:46content. Right? If you look over here,
42:49message dot content. Great. We have a
42:52string. Now after that, you know, we
42:54have done everything related to
42:56extracting the text content. Now we have
42:58to track the usage. Later on when we add
43:01tool calling we'll have to pass the tool
43:03calls as well. So now we have if
43:05response dot usage usage is equal to
43:09token usage we have to import that from
43:12client.response as well. And now I have
43:15to pass in the prompt tokens. Prompt
43:18tokens. Well how do I get that? If you
43:20look over here we have response this
43:22entire response dot usage dot prompt
43:26tokens. So I can just do response dot
43:30usage dotprompt
43:33tokens. Great. Then I have completion
43:36tokens and it's the same thing. We have
43:38response do usage dot completion tokens.
43:41And finally after that we'll have total
43:44tokens. So we have total tokens is equal
43:47to response do usage dot total tokens.
43:52Great. So all these three things are
43:55done. Now what I'm interested in is
43:57cached tokens. Now to get the cache
44:00tokens we can have cash tokens is equal
44:03to response dot and then it's not
44:07present in usage. If you notice cache
44:09tokens is present within prompt token
44:11details. So let me just copy this and we
44:14have response.prompt token details dot
44:18cache tokens cache tokens. So something
44:21like this otherwise the response do
44:24usage is not present. So usage is equal
44:26to null. We are explicitly setting this
44:29usage to null. Uh because we have not
44:32created the usage variable. We have not
44:34done something like usage is equal to
44:36null. Right now I have. So we can remove
44:39the else condition. And now from here
44:42I'm going to return a stream event. So
44:44the return type of this function is
44:46going to be stream event. And we import
44:48that from client.responses. response as
44:50well. And after that we have return
44:54stream event. Then we have the type of
44:56the event. So type is equal to event
45:02type. Event type is going to be an enum.
45:05So let's import it from client.response
45:07again. And the message is complete.
45:10We're not putting in text delta over
45:12here. text delta implies that we are
45:15still waiting for the bunch of tokens
45:18that you have to arrive. But we're not
45:20interested in that. We are saying that
45:22hey the message is totally complete
45:24because it's a non-streaming response.
45:26Whatever response we get is the final
45:28response. Then we have text delta and we
45:32pass in the text delta. Then we have
45:34finish reason and finish reason can be
45:37found out in the choice object itself.
45:41So we have choice dot finish reason and
45:45then we pass in the usage which is just
45:48usage.
45:49Great. So yeah this entire non-stream
45:52response is now done. So I can just do
45:55event is equal to await self.nonstream
45:58response and then I can yield the event.
46:02The reason I'm yielding the event over
46:04here is because we have an async
46:07generator that's going to be returned
46:08over here. Ideally or most of the times
46:11you would have a generator returned but
46:14since we have an asynchronous function
46:16we're going to return an async
46:18generator. What async generator does is
46:22that instead of returning like we have
46:24over here we are just returning a stream
46:26event right what you can do is yield a
46:29stream event or over here we have
46:32yielded the event. So what the
46:34difference between return and yield is
46:36is that let's say I called this
46:38non-stream response right it got the
46:40response it returned and then once it
46:44returned only then did you move forward
46:46and go to the next statement but with
46:49yield what happens is that
46:53let's say this function was called you
46:55yielded the event from here so wherever
46:57this function was called for example in
46:59main.py py this function was called the
47:03control goes back to this function so
47:06you basically pass the control that hey
47:08I'm executing this statement I've
47:10yielded it so it goes back to this main
47:13py wherever this function was called and
47:15then it executes further and once that's
47:18done you come back to this yield and
47:21continue this function again that is the
47:23difference between return and yield so
47:26in our stream response for example
47:27whenever you yield an event. What's
47:30going to happen is that you yield it
47:31over here.
47:33This function will kind of consume
47:35whatever the chat completion function
47:37gives you. It will print it out for
47:40example and then you go back to this LLM
47:43client and complete rest of the
47:45function. So that's why even over here
47:47we're going to yield it. Right now for
47:49non-streaming response we only yield one
47:52event. So the source control goes back
47:54over here. it does whatever it needs to
47:56do to complete this function and comes
47:58back over here. But with streaming
48:00response, what's going to happen is
48:02you'll get a list of events. So you
48:04yield it, then you basically do whatever
48:07operation you had to do over here. Then
48:10you come back over here and let's say
48:11there are more events. Those will be
48:13yielded. Then you come back over here,
48:15you print them out, and then you again
48:17go back over here. And it keeps going on
48:19until you have no more events left. So
48:22the only reason I'm doing yield over
48:24here instead of return is because we're
48:26going to return an async generator and
48:28async generator is returned just for
48:30stream response. That's the entire
48:32thing. Now the async generator is going
48:34to have a stream event that's returned
48:37and whenever you have to terminate you
48:40just return null. Now from typing you'll
48:42have to import async generator. Just do
48:45that and now you're all set. Another
48:48thing I would like to do here is let's
48:49say this entire thing got over then you
48:52just return. Now there's a bunch of
48:55error handling techniques and retries
48:58that we'll have to take care of. So I'll
49:00do that once we've completed the stream
49:02response as well. But as of now what I'd
49:05like to do is go back to this chat
49:07completion. And this does not await
49:09anymore because you're returning an
49:12async generator, not an async await or a
49:15core routine. So what we'll have to do
49:17is async for event in client.hat
49:21completion and then we can just print
49:24out the event so that we know that we
49:26are getting the right data. Now let's
49:29try to run everything again and we get
49:32an error. It says chat completion object
49:35has no attribute prompt token details.
49:38Well that's because it is present in
49:39usage. I misread it. So the prompt
49:42tokens detail is present within usage
49:46and then you go to prompt token details.
49:48I got confused by this parenthesis here
49:50but this parenthesis is just really for
49:52this completion tokens. So we can go
49:56back to llm client and here update the
49:59token over here. So instead of having
50:01response.prompt tokens you have response
50:03do usage.prompt tokens. Now we can try
50:06running it again. And yeah it does print
50:10out. It prints out stream event type is
50:12event type dossage complete. The text
50:14delta is present. Then we have error
50:17none. Finish reason is stop. And the
50:20usage well we have prompt tokens is six.
50:22Completion is 31 to total is 31 37 and
50:26cached is zero because they're not
50:28cached any cash tokens generally happens
50:31when you basically have a lot of tokens
50:35and they are reused and they're
50:37generally happening in the system
50:39prompt. So in a system prompt you just
50:41keep it static so that the KV cache can
50:45be reused. I don't want to get into it a
50:47lot but the point is cache will happen
50:51if the system prompt is kept static and
50:53is allowed by the provider. For example,
50:56Gemini or Google models do allow that. I
51:00think even Claude and OpenAI do allow
51:02that. I don't know about this model but
51:05yeah we'll see later on. So yeah this is
51:07done. Now we are interested in streaming
51:10the response and once we have streamed
51:12the response we'll be interested in
51:15having retry and failure exception
51:18handling and after that we'll look into
51:20the CLI so that we can start running the
51:24user prompts. So let's have the client
51:26and the keyword arguments passed to the
51:28stream response as well and after that
51:31we can pass in the client and the
51:34keyword arguments here as well. Now,
51:36similar to this particular line, we can
51:39just copy and paste that in because we
51:41are going to have a response and let me
51:43just fix the indentation here. We are
51:46having a response as equal to
51:47awaitclient.comp completions.create and
51:50we pass in the same keyword arguments
51:52but this time we know that the stream is
51:54set to true. And now if you try to print
51:57out the response, it's going to be an
51:58async generator. Whenever you have a
52:00stream returned out, whenever stream is
52:03set to true, you will have an async
52:06generator returned. And since we have an
52:08async generator returned over here, we
52:11are going to return an async generator
52:12here as well. So we have async
52:14generator, then we get
52:17stream event and then we have none. Same
52:21thing as this particular return type.
52:23And now we'll go over every chunk in the
52:26response. So we'll have async for chunk
52:29in response. And now I would just like
52:32to print out the chunk to see what we're
52:34dealing with. So yeah, nothing too
52:37fancy. Just printing out all of the
52:39chunks to see what's going on. Now I can
52:42go to the main.py file and set this
52:44stream to true. Then I can run this
52:47again. So we get well runtime warning
52:51enable trace maloc to get object. So
52:54it's the same thing again. We have just
52:56called self.stream response. But what we
52:58are interested in is going through every
53:01event that is yielded out by this
53:03generator. So you have async for event
53:07in self.stream response and then you
53:09just yield out the event. Great. And
53:13even over here we are not printing out
53:15the chunk. We're going to yield the
53:17chunk so that you know I'm able to
53:19explain to you how this entire flow is
53:21working. So from here we have a async
53:24generator. It is yielding the chunk. So
53:27as soon as we have the first chunk for
53:30example the response gives us let's say
53:3230 chunks. We yield the first chunk. It
53:34goes over here you know because that's
53:37where the function was called. It yields
53:39this event. So this one event that we
53:41have it is yielded over here and it's
53:44printed out. Then it goes back and then
53:46it yields another chunk. Then the same
53:49thing keeps on happening until we have
53:51no more chunks left. So now let's try to
53:53run this again. So we have Python
53:55main.py.
53:58And as you can see we had a streaming
54:00response. It was really fast because
54:01this model is really fast. But yeah, as
54:04you can see we get a chat completion
54:06chunk choices. The first choice is what
54:09we are interested in. The content is
54:11empty over here. After that you have
54:14another one where the content is hey.
54:17Then you have an exclamation mark. Then
54:19you have not much going on. So this is
54:23the entire basically. So this is the
54:26streaming response and it is working out
54:28fine. And it actually looks pretty much
54:30like the response that's given out by
54:32non-stream response. The only problem
54:34here is that I don't see any usage
54:38statistics. Only the last message has
54:41the usage statistics. So the user
54:43statistic will be taken out from the
54:45last prompt that we get. So first let's
54:48just try to extract all of the data. All
54:51right, we're not going to yield the
54:52chunk, right? We're going to yield the
54:54stream event that we have to create. To
54:56create a stream event, we will need the
54:57usage. So let's just first focus on the
55:00usage.
55:02So
55:03as mentioned the usage will be in the
55:06final chunk nowhere else. So we'll first
55:09check if has attribute chunk and we're
55:13looking for the usage and chunk dot
55:16usage. So here what we have tried to do
55:18is just check that hey if chunk do usage
55:20is present that means we are at the
55:23final chunk let me just get the usage
55:25attribute. So we have usage is equal to
55:27token usage and then we get prompt
55:31tokens and well everything similar to
55:34what we had over here right. So let's
55:36just copy it and paste it over here.
55:40Let's indent it properly.
55:43Great. So we have
55:46chunk dot usage chunk dot usage
55:50everywhere. After that you know I'm
55:53assuming we have usage or if we don't
55:55have usage totally fine in either case
55:58then we check hey if not chunk do.
56:01choices. We just continue that. Hey, we
56:04did not have any choices. It was an
56:06empty string. We saw that the very first
56:08response that we got in our print
56:11message was choices set to an empty
56:15list. That can be the case. So, we'll
56:18just continue if this is the case.
56:20Otherwise, we do have a choice. So, let
56:22me just have choice is equal to chunk
56:26dot choices at zero. Then we have delta
56:31and I would like to print it out again
56:33just so we see everything and I can't.
56:37So let me just remove everything. I
56:39yield chunk for just a minute so that I
56:42can run python main.py again. These are
56:46all the chunks I get. Now I'll command Z
56:48so that I go back to my code and now I
56:51can see what I'm dealing with. So let's
56:53just say we are on the very first chat
56:56completion chunk. Right? This is not
56:58chat completion like a non-stream
57:00response. It has it is a chunk. So it is
57:02a delta that we are getting when we have
57:05choice. This is the thing we have. We
57:07get a delta. Within the delta there is
57:10content. We want to extract that. So
57:12first let's just get the delta which is
57:15equal to choice dot delta.
57:18And after that we can check the finish
57:22reason if it's present. So if choice dot
57:25finish reason is present, we'll just get
57:28the finish reason as well which is equal
57:31to choice dot finish reason. And we have
57:34to create variables for each one of
57:37them. So let me just create it at the
57:39top. So usage is of the type of token
57:41usage or null and initially it is just
57:44set to null. Then another thing we have
57:47access to is finish reason which is a
57:52string or null and set to null. And yeah
57:57that's pretty much it for now. The
57:59reason we have defined it at the top
58:02instead of within this for loop is
58:04because there will only be one finish
58:06reason. There will only be one token
58:08usage. Awesome. So we have both of them
58:12here. Now the next thing that I'm
58:14interested in is handling the text
58:16content. So if delta dot content is
58:19present like we have over here delta
58:22do.content if it is not an empty string.
58:25So here it is an empty string. So we
58:27totally ignore this. Then we have
58:29content it is hey. So we are not going
58:32to ignore that. So if delta docontent is
58:35present we'll yield a stream event.
58:38Correct? Now in this stream event we're
58:41going to specify the type which is event
58:44type dot text delta. Then we are going
58:47to have a text delta itself. And now we
58:51need to pass in the text delta which is
58:55well the content the content is delta
58:57do.content. Great I told you about this.
59:01Whenever we have text delta well we'll
59:04have a text delta over here as well.
59:06Right? And we just reused the text delta
59:09that was meant for stream response and
59:11non-stream response so that we did not
59:13have to create another attribute that
59:15basically does the same task. Cool. Now
59:18assuming we've completed sending all of
59:21the chunks from here which are all of
59:23the text deltas. We will get out of this
59:26for loop.
59:28Cool. And once we get out of this for
59:31loop we will yield a stream event. This
59:34stream event is just going to say that
59:36hey we have completed the message. So we
59:39have event type dot message complete and
59:42the finish reason is what we had tracked
59:45over here. The last message that we had
59:47in this for loop was the finish reason
59:51and the usage was the same thing. The
59:52last message that we had is the usage.
59:56Great. So from here we have returned all
59:58of the text and finally we yield the
1:00:00stream event which just says that the
1:00:03message is complete and in the main.py
1:00:05when we get all of these events we're
1:00:07going to pass them that hey we have a
1:00:09message complete so we'll update our
1:00:11usage as of now and if we have a text
1:00:14delta we'll keep displaying that to the
1:00:16user in the terminal user interface. Now
1:00:19let me just clear it off. Let's run
1:00:21python main.py py again.
1:00:26And here we go. We get stream event text
1:00:29delta hey exclamation mark not much just
1:00:33chirp Illen and then you have all of
1:00:36these things. Awesome. At the end we
1:00:39also get message complete with prompt
1:00:41token set to six. Completion is 32.
1:00:44Total is 38. Everything seems to be
1:00:46working fine. That's great. So with this
1:00:49we have completed the streaming and
1:00:52non-streaming response of chat
1:00:53completion. Now the next thing I would
1:00:56like to do here is just have retries. So
1:01:00here every request that we have sent
1:01:02till now was successful. But what if the
1:01:05request is not successful? Maybe there's
1:01:08an API connection error. Maybe we have
1:01:10rate limited because when you're using
1:01:13open router on free there are multiple
1:01:15times that you've hit rate limit error.
1:01:17I think you're allowed like 50 messages
1:01:18in a day or the upstream provider might
1:01:22just set your limit to 10. So we don't
1:01:25know and we have to handle all those
1:01:28errors. So let's just wrap everything in
1:01:31a try and accept block. So we have try
1:01:35and then you have except the first error
1:01:37I would like to handle is the rate limit
1:01:40error and we handle that as E. Now I'll
1:01:45import this rate limit error from
1:01:47OpenAI. You don't have to import that
1:01:49from anywhere else, just OpenAI. And
1:01:52then if you're passing through a rate
1:01:54limit, it can be because well, you've
1:01:57been rate limited for a particular
1:01:59minute. Rate limiting is not just for a
1:02:02day or for a week, nothing like that. It
1:02:04can be for a second as well. So if it's
1:02:07just for a second, I would like to
1:02:10retry. So to retry, what I'm going to do
1:02:12is wrap this entire thing in a for loop.
1:02:15Now if you want to make your life much
1:02:17easier, there are packages that will
1:02:19allow you to basically do your own
1:02:21retries. But I'm going to write this
1:02:24from scratch as well. So we are going to
1:02:26have four attempt in range and then you
1:02:29can decide what number of max retries
1:02:31you want. I'm going to define it here at
1:02:34the top. So you have self domax retries
1:02:37and it's going to be an integer if
1:02:39three. And if you want you can also take
1:02:41this value from the configurator the
1:02:44configuration system we'll add. But yeah
1:02:46I'm not I'm just going to hardcode this.
1:02:49So I'll just go until max retries + one
1:02:54because range will exclude the last
1:02:56value. So I want to go up till three
1:02:58including three. So we do plus one. Then
1:03:01over here we'll just check that hey if
1:03:03the attempt is less than self domax
1:03:06retries in that case I want to try
1:03:08again. But what I want to do is wait for
1:03:12uh a second or 2 seconds or 4 seconds.
1:03:14We want to do exponential back off.
1:03:16Meaning let's say we did an attempt, it
1:03:20failed. Okay. Now I want to do a second
1:03:23attempt. But if I do it instantly, it
1:03:25will again fail because the rate limit
1:03:28might be there for 3 seconds or 5
1:03:30seconds or 10 seconds. We don't know. So
1:03:32I'll wait for some time. So how long do
1:03:35I have to wait? Well, first I can try
1:03:37waiting for 1 second. Then if it fails
1:03:40then what I can do is try for 2 seconds.
1:03:42Then it fails then I can try for 4
1:03:45seconds. So I'm just multiplying by two
1:03:48every single time. This is called
1:03:49exponential back off. So I'll just say
1:03:52wait time is equal to 2 to the power of
1:03:55attempt. So in our first attempt we'll
1:03:57wait for 0 seconds. So in our first
1:04:00attempt we'll wait for 2 seconds. In our
1:04:02second attempt we wait for 4 seconds. In
1:04:04our third attempt we wait for 8 seconds.
1:04:07And then you can just do await async io
1:04:11dot sleep and then you sleep for the
1:04:14wait time. Now let's import async io as
1:04:17well. Now if the attempt is greater than
1:04:20or equal to self domax retries we'll
1:04:22just yield the stream event.
1:04:25And here I'll pass in the type. The type
1:04:28is going to be event type dot error. And
1:04:31the error message is going to be passed
1:04:33in which is hey the rate limit exceeded.
1:04:37So let's pass that in and then we have e
1:04:40passed in like that. Great. So yeah if
1:04:45the attempt is less than self domax
1:04:47retries we retry we sleep and after
1:04:50let's say 2 seconds or 4 seconds we are
1:04:52in a for loop right? So it will
1:04:54re-execute this line. Now another error
1:04:56we can face is an API connection error.
1:05:00So let me just copy this because the
1:05:01logic is largely going to be the same
1:05:03and we have except API connection error.
1:05:07The connection was not established. So
1:05:09we import that from openAI as well. And
1:05:11then we just paste this logic in. If
1:05:14attempt is less than this, we sleep
1:05:16again. But this time we have a
1:05:18connection error. So we can just say
1:05:20connection error. And another error we
1:05:23might have is related to API error.
1:05:27maybe you know the parsing went well or
1:05:29went wrong because many times it can be
1:05:31the case that LLMs don't respond
1:05:35properly. So, OpenAI package also does
1:05:38not respond nicely or maybe there was
1:05:41some error with the API, you know, not
1:05:44with the connection but with the API
1:05:46itself. So, yeah, we are just handling
1:05:49this. Now, if you have an API error, I
1:05:51don't want to retry. I'll just yield the
1:05:53stream event which is an API error. If
1:05:56you want to retry it, go for it. I'm
1:05:58not. And I'll just return out over here.
1:06:04Even over here, I'll just return out
1:06:05because we have exceeded the max number
1:06:08of tries. And even over here, I'll just
1:06:11return out. Awesome. So, we have done
1:06:15retrying with exponential backoff and
1:06:18exception handling in chat completion.
1:06:20We've not faced any errors, but this is
1:06:22just premature optimization. Now another
1:06:25thing I would like to do is get this
1:06:27client and keyword arguments out of this
1:06:30for loop because we don't want them to
1:06:32rebuild every single time right I don't
1:06:35want so many client connections to be
1:06:37created I only want one client
1:06:40connection and actually our logic will
1:06:42help us create only one connection but
1:06:44even then why do you want it within the
1:06:46for loop so yeah this is how our code is
1:06:50going to look like we have these two
1:06:51created outside then we have retries
1:06:54Guys, if it is successful in the first
1:06:56try, we just return instantly. So, we
1:06:58don't go through any of the for loop
1:07:02iterations again. And if we run into any
1:07:06error, we do the retrying with
1:07:09exponential back off. Amazing. So, we
1:07:12have had all of our LLM client created
1:07:15properly. Now, I just need to create
1:07:17this main function into a CLI
1:07:20application. And then we have to display
1:07:22the data that we are getting from here
1:07:25nicely. We need a place for user to type
1:07:28a prompt. So we'll get to all of that
1:07:30step by step. Step one is just
1:07:33converting this into a CLI application.
1:07:36We don't want this hardcoded. We need to
1:07:38create a system prompt. I'll walk you
1:07:39through the system prompt that we'll
1:07:41paste in. What are the tricks related to
1:07:44prompting? I'll talk to you about that.
1:07:47And we also take the argument from the
1:07:50user. So instead of passing in Python
1:07:52main.py and executing it, we actually
1:07:55have the user pass in the data. So
1:07:57that's exciting. Let's get to it. To get
1:08:00this command line working, we're going
1:08:01to install a package called click. Click
1:08:04is a Python package for creating
1:08:06beautiful command line interfaces in a
1:08:08composible way with as little code as
1:08:10necessary. Um we'll just directly take a
1:08:12look at this example. As you can see,
1:08:15first we have to specify at the
1:08:17rateclick docomand which just says that
1:08:19this function is going to be well a
1:08:22command executable thing. Then we
1:08:25specify all the options and all the
1:08:27arguments that we might want to add. For
1:08:28example, when we do click dot option, we
1:08:31can specify the count with a flag like
1:08:34that. And the same goes for name. We are
1:08:37also going to have an argument which
1:08:38will allow us to pass in a prompt
1:08:40without specifying d-prompt.
1:08:43So let's install this. I'll just copy
1:08:45this, open up our terminal, run this
1:08:47command, and hit enter. And click is now
1:08:50installed. Now I just want this main
1:08:54function to be a click thing. Right? So
1:08:57first I'll just have to import click. So
1:08:59at the top I'll just do import click.
1:09:01And then as the decorator of the main
1:09:04function, I'm going to add at the
1:09:06rateclick dot command. This says that
1:09:09the command is going to be this says
1:09:13that the entire function is going to be
1:09:14a command executable. That means it's
1:09:17going to be a CLI thing. After that,
1:09:19we're going to specify the prompt. Now,
1:09:21whenever we want to access the terminal,
1:09:23we don't want to do AI agent or
1:09:26something like-rompt
1:09:29is equal to well, you know, what is the
1:09:32name
1:09:34something like that. We don't want to do
1:09:36anything like this. What we want to do
1:09:38is directly type in what is the name.
1:09:40And obviously this is run single mode.
1:09:42That means you only pass in the prompt
1:09:45and the CLI will run once. But then we
1:09:49will go into a more interactive mode
1:09:51where we just pass in AI agent like this
1:09:54and it opens up in a nice interactive
1:09:57mode and then you can talk to the AI
1:09:59agent passing in all of the input and
1:10:01stuff whatever you saw in the demo. So
1:10:05first things first, let's get the single
1:10:07mode up and running. So we'll have
1:10:10addate click dot argument and then we'll
1:10:12pass in all the parameter declarations.
1:10:15So what is the name? Well, the name of
1:10:18this is going to be prompt. And is it
1:10:21required? Well, I'm just going to say
1:10:23false because it's not required. If it
1:10:26was required, you would have to pass in
1:10:27the prompt every single time. But
1:10:31sometimes you can just do AI agent and
1:10:33it will open up your command line, you
1:10:36know, the entire TUI interface.
1:10:40So yeah, now what I'd like to do is take
1:10:44this stuff from the parameter. So we'll
1:10:48have prompt which is going to be a
1:10:50string or it can be null. And by default
1:10:53it is set to null but we can just set it
1:10:56like that. Now we have access to the
1:10:59prompt. Now I would like to just go
1:11:00ahead and print out the user prompt just
1:11:04to see what it's looking like. And now I
1:11:06can try running it. And also by the way
1:11:09now I don't need to do async io.run main
1:11:12I think. Or actually let's just do it
1:11:15and see what happens. So we'll have
1:11:18python main. py and then I'll pass in
1:11:23hey how are you doing? And actually it
1:11:27would make a lot more sense if we just
1:11:28pass in the prompt as the content you
1:11:31know and then yeah now let's run it and
1:11:34it just prints out python main.py hey
1:11:38how are you doing clear clear for some
1:11:40reason and the reason it breaks is
1:11:43because
1:11:44click does not support asynchronous
1:11:46functions. So if you want to use click
1:11:49you cannot do async defaf main. So what
1:11:51I like I'll have to do is something like
1:11:54this. we have def main. I'll have to
1:11:56remove this async io.run from here. Then
1:11:59we'll have to create another function
1:12:01that will be asynchronous def run. And
1:12:04this these are all temporary functions.
1:12:06All right. Later on, we're going to
1:12:08remove all of the logic from here and
1:12:10put it in an agent class and stuff. But
1:12:13as of now, this is good enough for our
1:12:16use case. So we just going to pass in
1:12:18the messages list here which is just
1:12:20going to be a dict string any. And now
1:12:24you have to import any from typing. And
1:12:27now you can pass in all of these things.
1:12:30So there we go.
1:12:33Awesome. Now we also need to create an
1:12:35instance of client. So I'll just create
1:12:37it over here. And now I can just pass in
1:12:40the messages like that. Great. So yeah,
1:12:43this is how it's going to run. Now I
1:12:45have to do async io.run over here and
1:12:48then I can just pass in the run
1:12:50function. Pass in the messages list as
1:12:53well. Awesome. So this is the entire
1:12:57function. Now I can go ahead and try to
1:13:00run this again. So it does print out hey
1:13:04how are you doing? Clear. Clear. And
1:13:06then it says hey I'm just a bunch of
1:13:08code so I don't have feelings. Okay
1:13:11good. So everything seems to be working
1:13:15now. We also have a run single mode.
1:13:18Let's just make it more official in a
1:13:21way. So what I can do is create a class
1:13:24CLI at the top. And this CLI will have
1:13:27an init function. In this init function,
1:13:30we are going to get the configurator. We
1:13:32are going to get other values as well.
1:13:34But as of now, we're just going to have
1:13:37pass just like that. After that we're
1:13:40going to have a mode of run single. So
1:13:43we have de def run single and this run
1:13:46single mode is going to essentially call
1:13:48the agent. This agent will be giving us
1:13:53all sorts of message. Right now if you
1:13:55notice we get the event in the form of
1:13:58stream event. We do not really need
1:14:00stream event over here because we don't
1:14:01need all of the events that chat
1:14:03completion gives us. We only need
1:14:05certain events. So for example, if we
1:14:09get uh an error, that's good. We we
1:14:11might want that. But if we get some
1:14:13other type of response like, you know,
1:14:16there'll be a tool call and all of that,
1:14:19a text delta, message complete, we
1:14:22probably won't care about many of them.
1:14:24So what I want to do is create an
1:14:26interface before them. So an interface
1:14:29between main.py py or the CLI class and
1:14:34the LLM client which is going to be the
1:14:36agent. That agent is going to interact
1:14:39with this LLM client. It will handle all
1:14:41sorts of things like context compaction
1:14:44or compression or you know pruning or
1:14:47the tool calls. All of that stuff will
1:14:49be handled by that middle layer and this
1:14:52CLI is just going to call that agent. So
1:14:55first I would just go ahead and pass in
1:14:58self here. Put a pass. Now we can go
1:15:01ahead and work on this agent class that
1:15:03is going to be the main logical function
1:15:06of our entire codebase. So here I'm
1:15:10going to create a folder called agent
1:15:12and in there we're going to have another
1:15:14file. Let's call that agent as well. Now
1:15:17in this agent first we need to create a
1:15:20class agent. Then we have an init
1:15:22function as well. And in this init
1:15:24function we'll have to introduce many
1:15:26things. Again the configurator will be
1:15:28present here. Then there will be
1:15:30session. Session is going to contain all
1:15:32of the objects like context management
1:15:35and details about everything related to
1:15:38a session because we can have multiple
1:15:41sessions, right? Each session will have
1:15:43its own context. That's why this context
1:15:46will be present within a session. But we
1:15:48will talk about that later on. Now, if
1:15:50we want to do any context management,
1:15:52I'll just create a class and use it
1:15:54inside of this agent. Later on we will
1:15:56refactor it outside of this agent to be
1:15:58put into a session. Don't worry it will
1:16:01be quite straightforward. Now let's go
1:16:03ahead and just create the agentic loop.
1:16:07Remember agentic loop is just a call to
1:16:09an LLM where we are handling the context
1:16:12and making some tool calls. Tool calls
1:16:14will allow us to do some sort of action.
1:16:18So I'll just go ahead and create async
1:16:20def agentic loop. It's going to be a
1:16:23private or protected function. The
1:16:25reason for it is because this agentic
1:16:27loop is going to return an async
1:16:29generator. It is it will be returning an
1:16:32async generator. And let me import that
1:16:34from the typing module. And the return
1:16:39type of this async generator is going to
1:16:41be agent event. It's not going to be a
1:16:43stream event anymore. It's going to be
1:16:45agent event because what the agent cares
1:16:48about in main. py will be more stuff.
1:16:51For example, we might care about well
1:16:54one turn starting because we have
1:16:56multi-turn conversations, right? If the
1:16:58turn started, we have some stuff to do.
1:17:00If the turn ended, we have some stuff to
1:17:02do. If we have the context compressed,
1:17:06we will have something to show. If we
1:17:07run into an error, we'll have something
1:17:09to show. If the text completion is done,
1:17:13we'll have something to show. And if the
1:17:16tool starts doing tool calls, well, if
1:17:19the tool call starts, there will be
1:17:21something to show. And if the tool call
1:17:23ends, there will be something to show.
1:17:25So the agent has more microscopic
1:17:30details that needs to be sent to the UI
1:17:32because the UI will have to display all
1:17:35of those details. The stream response is
1:17:37only concerned with details related to
1:17:41streaming the response, right? Only
1:17:43related to the response. But agent has
1:17:45to do a lot more because if a tool call
1:17:48starts you have to show it. If a tool
1:17:50call ends you have to show it. So this
1:17:53agent event is that's why created. We
1:17:55can't use stream event because it
1:17:57doesn't go that microscopic. So let's go
1:18:00ahead and create this agent event and we
1:18:03can create that within this agent
1:18:05folder. So we have events. py and then
1:18:08we're going to have a class of agent
1:18:10event. Right? The first thing is going
1:18:12to be a type. What is the event type?
1:18:16Now this event type is not the streaming
1:18:18event type whatever we had in response.
1:18:21That's not what we're talking about.
1:18:22What we're talking about is agent event
1:18:25type. So let me just call this agent
1:18:27event type. And now what I'm going to do
1:18:30is create an enum for this event type as
1:18:33well. So quite similar to what we had
1:18:36with the streaming event. That's what
1:18:40we're doing over here. But now the
1:18:42values of this enum are going to be
1:18:44different. So let me just import from
1:18:46enum enum. And then we also need
1:18:50obviously the data classes. So let me
1:18:52just import import it right here.
1:18:55This agent event class is going to be a
1:18:57data class. And after that we have an
1:18:59enum. So the very first thing is going
1:19:02to be agent start. This is related to
1:19:04agent life cycle. So we have the agent
1:19:08has started. So we'll have some data to
1:19:11show to the user. Then we have agent
1:19:13ending probably we'll also show to the
1:19:16user or we'll just kill off the
1:19:18application.
1:19:20Then we have agent error out which is
1:19:23well agent just erroring out. We have to
1:19:25probably display that in some different
1:19:28color. And we'll also have something
1:19:31related to streaming right the text
1:19:33delta which is equal to text delta. And
1:19:37then we'll have text complete which is
1:19:40equal to text complete. Cool. Now let me
1:19:44just put in some comments. This is agent
1:19:47life cycle related stuff and this is
1:19:51text streaming. Okay. Now we'll have
1:19:54other agent event types as well. For
1:19:56example, the turn has started, the turn
1:19:59has ended and maybe the tool call
1:20:01execution stuff. If we have the
1:20:04confirmation requirement from the user
1:20:06then if the context is compacted
1:20:10or if the loop is detected everything
1:20:12will go in this agent event type.
1:20:15Awesome. The next thing we'll have here
1:20:17is data. If there's any data that the
1:20:19user might want to send that will be of
1:20:21the type of string, any and we'll import
1:20:24from typing any. And we'll default it to
1:20:28field. field will allow us to just pass
1:20:31in a default factory and the default
1:20:34factory is going to be addict. If
1:20:36nothing is present, we have addict. It's
1:20:39not null. Great. Now, if you want, you
1:20:43can just use this agent event class
1:20:45everywhere just like we did with event
1:20:47type or actually with stream event.
1:20:50Instead of, you know, yielding stream
1:20:52event over here, what we could have done
1:20:54is created a method here called add the
1:20:57rate class method. And then we could
1:20:59have created something like this a
1:21:02definition which just says stream error.
1:21:06And then we have self. Then we return
1:21:09the class where we pass in the type
1:21:13which is the event type dot error or
1:21:17whatever there was and then you pass in
1:21:20the error details as well. So instead of
1:21:22passing in yield stream event like this
1:21:24every single time, you could have just
1:21:26done stream event dot error. Something
1:21:30like this, right? Or stream error,
1:21:33whatever this was called. That would
1:21:35have been much better. So now I'm not
1:21:37going to rewrite this entire thing
1:21:39again. So if you want, go ahead do it.
1:21:42I'm not going to do it. I'll do this
1:21:45thing in agent. py because there well
1:21:48there will be a lot of stuff or actually
1:21:50in events py because there will be a lot
1:21:52of stuff that will be used again and
1:21:54again for example agent start we might
1:21:57want to use that again and again so it
1:21:59just seems nice to have that so the very
1:22:01first thing is a class method with agent
1:22:05start then we get access to class
1:22:08because it's a class method and then we
1:22:10have a message string where we return
1:22:14agent event.
1:22:17Now again we can't use that. So first
1:22:19we'll have to import it from future.
1:22:23We'll import annotations. And now we are
1:22:26able to use that. Now I'll just return a
1:22:29class from here where we have agent
1:22:32start right. So the type is equal to
1:22:36event type or agent event type dot agent
1:22:39start and the data will be passed in as
1:22:42well because when the agent starts we
1:22:45need a message to be transmitted. So
1:22:48we'll have that message here. So here we
1:22:51go. We passed in the message as the data
1:22:54and there we go. Cool. Let me just
1:22:57format this nicely. So I'll put a comma
1:22:59here and it's formatted. The same thing
1:23:01will go for agent end or agent error as
1:23:05well. So let me do it quickly. We have
1:23:07agent end and when the agent ends we
1:23:11just get a response which is of the type
1:23:13of string or null and it will be null
1:23:16and we also get the usage which will be
1:23:19token usage or none and it is equal to
1:23:23null. Now let me import token usage from
1:23:26client.tresponse.
1:23:28And this time the data is going to be
1:23:30both of them. It's going to be the
1:23:32response and it's also going to be the
1:23:35usage. So we pass in usage. But the
1:23:38thing is usage is a token usage class.
1:23:42What do we do? Well, we can just do
1:23:46usage dot dict like that. So it gets
1:23:49converted into dictionary if the usage
1:23:52is present. Otherwise, we just pass in
1:23:54null because well, if we don't put this
1:23:56condition, we're calling dot dict like
1:24:00this on a null variable. So, it's just
1:24:03going to return in a runtime error.
1:24:06Awesome. Similar to this, we have agent
1:24:09error as well. So, let me just copy
1:24:11this, paste it again, and then we have
1:24:14agent error. We're going to get a class.
1:24:17Then, we get the error message, which is
1:24:19a string. And then if there are any
1:24:22details about the error we can have that
1:24:24as well. So details will be dict string
1:24:28any
1:24:29and it can also be null. By default it
1:24:33is null. Then we pass in the error which
1:24:36is this error. And then we pass in the
1:24:39details as well which is details or an
1:24:45empty bracket because if details is not
1:24:48present we want an empty dictionary to
1:24:50be present. Now similar stuff can be
1:24:52done for all the other things as well
1:24:54like text delta. So let me just paste
1:24:56that in or actually let me copy it and
1:25:00then paste it. So we have text delta. So
1:25:04we get a class then we get a content
1:25:06which is of the type of string and then
1:25:09we return the class where we say well
1:25:11this is not agent start this is agent
1:25:13error this is agent end and now we have
1:25:18reached agent event type dot text delta
1:25:22the message is just going to be content
1:25:25and then content great after that we
1:25:27have text complete so we have text
1:25:31complete it just means all the text has
1:25:33come
1:25:34So that's good. And now we have content
1:25:37again. Great. So this is all of the
1:25:40events we have for now. Obviously it's
1:25:42going to expand when we have tools and
1:25:44all other features. Essentially this
1:25:47will be the glue between main. py agent.
1:25:50py because main.py will only interact
1:25:53with this agent event. it won't interact
1:25:56with all the streaming events because
1:25:58think about it streaming event is just
1:26:01related to response. Agents are not just
1:26:04about fetching response from the LLM.
1:26:06It's also about other things and agent
1:26:09event captures all of that and goes into
1:26:11my new details. Okay. Now let's go ahead
1:26:14and import agent event. So from
1:26:17agent.events import agent event. Then we
1:26:20have null. Now, in this agentic loop, as
1:26:23I mentioned, it's going to be a
1:26:24multi-turn conversation. The user will
1:26:27be able to interact more than one time,
1:26:29but we'll get to that later on. First,
1:26:32we just care about getting all of the
1:26:35things that were present over here in
1:26:37this run function out of here. So, we
1:26:39get this data out of here and just put
1:26:42it in the agent. So, we'll have client
1:26:44is equal to llm client. Obviously, we're
1:26:48going to instantiate it within session
1:26:51because session will have its own
1:26:53client, right? A user can have multiple
1:26:56sessions open so that they can interact
1:26:58with the agent multiple times and each
1:27:01session will have its own connection to
1:27:03the LLM because we won't be reusing the
1:27:05same connection, right? So, client or
1:27:09the LLM client instantiation will also
1:27:12be present within session. But we have
1:27:14not created session yet. So I'm just
1:27:16instantiating everything here including
1:27:18the context manager that will come in
1:27:20next but yeah we will be refactoring so
1:27:23don't worry. Now we have llm client. So
1:27:26from client llm client we import lm
1:27:28client. Let me remove that import from
1:27:30here. Great. Now we have self.client.
1:27:34chat completion. Then the messages needs
1:27:37to be present. Now where will this
1:27:39messages come from? Well, messages needs
1:27:42to come from the context, right? We
1:27:46won't be taking it from this agentic
1:27:47loop itself because well, if the agentic
1:27:51loop is on, why would you be getting it
1:27:54from messages? Like, why would a
1:27:56messages dictionary be present over
1:27:57there? What you need is a class that
1:28:00will handle all of the messages for you.
1:28:02So, every single time a new assistant
1:28:04message comes in, you store it in your
1:28:07context management class. If you get
1:28:10your own user sends a new message, it
1:28:12will be stored in a context management
1:28:14class. So everything will be within a
1:28:17context manager and therefore we'll have
1:28:19to create a context manager to get this
1:28:21done. But as of now since we don't have
1:28:23any messages or any context manager, I
1:28:26would not like to introduce a new thing
1:28:29over here. So what I'll do is just keep
1:28:30keep a messages list here
1:28:33or actually let me copy this.
1:28:37paste the messages list here.
1:28:40The next step after I figured out what
1:28:43to do in this agentic loop for a while
1:28:45is creating the context manager that
1:28:48will handle all of these things. But as
1:28:50of now, let's keep it simple. I'll just
1:28:52type in, hey, what is going on? So that
1:28:55is the messages. Now chat completion is
1:28:58going to give us a list or a bunch of
1:29:00stream events, right? we want to handle
1:29:02all of those stream events and in some
1:29:05cases we'll have to return it outside of
1:29:07this agent tech loop. So now only we can
1:29:10just check that hey if the stream or if
1:29:14the event dot type is equal to stream
1:29:18event type and now we'll have to import
1:29:21stream event type. So well there's
1:29:24nothing like stream event type it is
1:29:26just called event type. So let me change
1:29:29it to stream event type before it's too
1:29:31late. Seems to be a better name because
1:29:34in events. py we have agent event type.
1:29:37Now let's import from client.response
1:29:39stream event type. And now if the event
1:29:42type is let's say text delta. In that
1:29:45case we want to return the response
1:29:49right. So what we'll do is content is
1:29:52equal to event dot text delta dot
1:29:56content. And then I'll just yield the
1:29:59agent event. And then I'll pass in the
1:30:02text delta. So I can just call the text
1:30:05delta function that we created. And now
1:30:07I just need to pass in the content. The
1:30:10content is the content that we extracted
1:30:12here. Right? So from llm client we got
1:30:16the event correct in this async for loop
1:30:19where we are catching the events. We
1:30:21just check that hey if the type is text
1:30:23delta cool I'll just convert it into
1:30:26agent event and send it across as
1:30:28straightforward as it gets. Similarly if
1:30:31we have another thing let's say if event
1:30:33dot type is equal to stream event type
1:30:38dot error in that case I would like to
1:30:41yield agent event dot agent error and
1:30:46this time we'll have to pass in the
1:30:49error. What is the error? Well, it's
1:30:51just going to be event dot error, right?
1:30:53Because if the stream event type is
1:30:56error, we know event is going to contain
1:30:59the error variable for sure. And if it
1:31:02does not, then we just say unknown error
1:31:05occurred. That's some good error
1:31:07handling that we are doing. Now, if you
1:31:10want, you can terminate the app here
1:31:11because there's an agent error. I would
1:31:13just like to stop the execution or we
1:31:15can let the app keep going, not worrying
1:31:19that, hey, there is an agent error.
1:31:20Okay, maybe there was an agent error,
1:31:22but we'll keep going until this agent
1:31:25error is resolved. So in the next turn,
1:31:27maybe the agent error is resolved or
1:31:30even in the next message, the agent
1:31:32error is resolved. So this is the most
1:31:34basic agentic loop. It is not even an
1:31:36agentic loop because we have not created
1:31:39anything related to how many turns there
1:31:42are. How many times will this agentic
1:31:45loop keep going? because yeah if the
1:31:47user sends a message once it works but
1:31:50what if the user wants to send it
1:31:51another time we've not handled that so
1:31:54we'll have to do that but before we get
1:31:56to that let me just complete the entire
1:31:58thing we were working on about single
1:32:00run and then we can focus on a more
1:32:03interactive environment so this agentic
1:32:05loop is a private function right it will
1:32:07be called within this public function
1:32:09run this run will get the message of the
1:32:13type of string that is what the user
1:32:16will give us. So this particular
1:32:18function will process the user message
1:32:20and it will return the events and this
1:32:23is the function that will also be called
1:32:24in main.py. So at the very beginning we
1:32:27can just say yield agent event dot agent
1:32:32start because yeah we received the
1:32:34message let's start it and this is the
1:32:36message. After that, we'll have to
1:32:39update the context. But we have nothing,
1:32:42no concept of context as of now. So I'll
1:32:47just put in a comment here reminding us
1:32:49that we need to add the user message to
1:32:51context. Then we want to call the
1:32:54agentic loop. So we'll have async for
1:32:57event in self.
1:33:00Loop and then we're just going to yield
1:33:02the event. Whatever event we get, we're
1:33:04just yielding it as soon as we get it.
1:33:06Now there are a lot of things that need
1:33:09to be done after and before it. For
1:33:11example, when we add hooks, there'll be
1:33:13an after agent hook that can run and
1:33:16before it also an a before agent hook
1:33:19can run. But we'll get to it one step at
1:33:22a time. Now let's say this entire
1:33:25function completed. In that case what we
1:33:28want to do is yield agent event dot
1:33:33agent end because well the entire
1:33:35function completed the agent has ended.
1:33:37If when we get the message the agent
1:33:40starts then when the function ends the
1:33:42agent should end right. So yeah there we
1:33:45go. Again I just like to remind you this
1:33:48entire thing just processes one single
1:33:51message and runs one single time for one
1:33:54message. Again later on when we add our
1:33:57tool calls and stuff it can run multiple
1:33:59times if it can run multiple times. We
1:34:03can get more events in and then we'll
1:34:05have to put in the logic of when to
1:34:07terminate our agent because it can keep
1:34:09running for a long time. Right? So in
1:34:12this agent end we have to first specify
1:34:14the response. What is the final response
1:34:18going to be? Well, the final response
1:34:20will be fetched whenever we get an event
1:34:24of text complete. Right now, we have
1:34:27handled text delta but nothing related
1:34:30to text complete which is never being
1:34:33returned in this particular class. Even
1:34:35though if we go to our events py, we
1:34:38have created text complete because text
1:34:40complete is going to contain the final
1:34:43response as well. The content that you
1:34:45see over here is just the final
1:34:47response. So let's go back to this agent
1:34:49tech loop and here I'm outside of this
1:34:53for loop I'm going to create a variable
1:34:54called response text which is just an
1:34:56empty string and then in the text delta
1:34:59whenever we get a content we're just
1:35:01going to do response text plus equals
1:35:05content. Cool. And then we'll send off
1:35:07the text delta event. So this response
1:35:10text is really just the final response
1:35:12text. And then outside of this async for
1:35:15loop I can just check that hey if the
1:35:18response text is present that means we
1:35:20have a final response we can yield an
1:35:23event saying agent event dot text
1:35:26complete and the text is just the
1:35:30response text. So now from the agentic
1:35:32loop we are also returning an agent
1:35:34event and now we can just handle this
1:35:38event within this particular for loop.
1:35:40We can just say that hey if the event is
1:35:42equal to and actually we want if event
1:35:45type is equal to agent event type and I
1:35:50don't know why I'm not getting that
1:35:52autocomplete. Let me just copy paste
1:35:54that in and let's import it from
1:35:57agent.events.
1:35:59So if the event type is equal to agent
1:36:01event type complete. Remember we using
1:36:04agent event type here and we're using
1:36:06stream event type here because we are
1:36:08just trying to catch whatever we've
1:36:10returned from here. It might just seem
1:36:13very weird that we are catching all of
1:36:16the events here returning it or yielding
1:36:19it in the run function which is yielding
1:36:21it in the main function. But trust me we
1:36:24need this because there'll be a lot more
1:36:25functions added in the future which will
1:36:28necessitate having these two functions.
1:36:31So now we can have final response which
1:36:34is equal to event dot data.get
1:36:38content and now I can just pass in the
1:36:41final response to this agent end.
1:36:44There's another usage that you can also
1:36:46pass in but we'll get to it later on. As
1:36:48of now I think we are good to go ahead
1:36:51and call this agent within main. py but
1:36:55before we do that I would like to do one
1:36:58thing. I would like to create a close
1:37:00function as well. So if you notice, I've
1:37:02created an LLM connection, but I'm never
1:37:04really closing it. So what I'd like to
1:37:07do is use this a agent class as a
1:37:10context itself. So what I'd like to do
1:37:13is convert this class agent into
1:37:16you know like we can use this agent
1:37:18class with statement. So if you don't
1:37:21know the Python's width statement, it
1:37:22just simplifies resource management by
1:37:24automatically handling setup and cleanup
1:37:27operations. So what we need to do is
1:37:30create two things enter and exit. What
1:37:33we'll be doing is creating a enter and a
1:37:36exit because we have asynchronous stuff.
1:37:39So that will allow us to have
1:37:40asynchronous context management. And if
1:37:43you've ever done file handling in
1:37:44Python, you know with open
1:37:47as file. So this is what I'm talking
1:37:50about. We can use our agent class like
1:37:52this as well. And what this does is well
1:37:55we don't have to do fp.close close.
1:37:58Whenever we do this, right? All of that
1:38:00is handled behind the scenes. Same thing
1:38:03with agent. We don't have to explicitly
1:38:05go ahead and call client.clo. It will
1:38:08automatically happen behind the scenes.
1:38:09So, I'd like to just define these magic
1:38:12functions over here. First, we'll have
1:38:14asynchronous def
1:38:18enter
1:38:20self and this will return an agent back.
1:38:24Now again we would have to import from
1:38:27future annotations.
1:38:29Now we can use that and as of now we
1:38:32don't have anything. So I'll just return
1:38:35self here. But for asynchronous div a
1:38:40exit we have something. We'll have to
1:38:42close the llm connection. We'll have to
1:38:46do something like this. Await self dot
1:38:50client dot close. And we only want to do
1:38:54that if self.client is still present.
1:38:57And then we will also do self.client is
1:39:00equal to null. Now it's not there. And
1:39:03in a exit we get multiple things like
1:39:06exception type. Then we also get
1:39:09exception value. Then we also get
1:39:11exception. I don't know what TB really
1:39:14stands for.
1:39:16It's probably trace back. So yeah, we
1:39:19get all of these things. And now we can
1:39:21use agent with. So like we can just do
1:39:24with agent and that's what we're going
1:39:26to do in main. py. So in the in it what
1:39:29we'll have to do is create an agent and
1:39:32well it can be of the type of agent or
1:39:35null. And initially it will be null
1:39:37because we're not going to establish the
1:39:40LLM connection as soon as CLI is called
1:39:43right because that would just be weird.
1:39:46The reason it would be weird is because
1:39:48imagine I call run single over here and
1:39:51the user has not specified the prompt or
1:39:55let's say we are in a run interactive
1:39:58mode and the user has not specified any
1:40:01prompt and we have established the LLM
1:40:03connection that doesn't make any sense
1:40:05why do we want to waste resources if the
1:40:08user just opens it up but does not send
1:40:10any message right that's why we are
1:40:13setting this to null we'll only create
1:40:15it as soon as run single is actually
1:40:19called which is like this async with
1:40:22agent and this is going to be an
1:40:23asynchronous function. Now I'll pass in
1:40:26the agent like that and then well we
1:40:29have to pass in the configurator all of
1:40:31that but we don't have it as of now.
1:40:34Then we have agent. Then we can just
1:40:37update our self.agent to be agent that
1:40:39we get here. And then
1:40:45we have to process the message because
1:40:48we know that this is going to return a
1:40:50bunch of events to us. So first we'll
1:40:53have to call agent.run and that
1:40:55agent.run is going to yield all of the
1:40:58events. So we need to capture them and
1:41:00based on that display the data to the
1:41:03user and the same logic that we put over
1:41:05here will also be used within run
1:41:08interactive because the process
1:41:10messaging. So let's create an helper
1:41:13function. So we have self dot process
1:41:16message and we just pass in the message
1:41:20over here. Cool. And run single is going
1:41:22to get a message. So let's take that
1:41:25from the parameter. And if you're
1:41:27confused about how this function is
1:41:29going to be used, well, it's quite
1:41:30simply something like this. If the
1:41:33message is present then or if the prompt
1:41:36is present, sorry, we are in the single
1:41:39short mode. We have to run it single
1:41:41time. So we'll just do async io dot run.
1:41:44And now we'll pass in the CLI. Now we
1:41:47have to create the CLI as well. Let me
1:41:49remove this print. So we have CLI like
1:41:52that. And now I can do CLI dot run
1:41:56single and then I'll pass in the prompt
1:41:59like that. Amazing. So I can remove this
1:42:01part. We can remove done as well because
1:42:04there's no need. And yeah, that's pretty
1:42:07much it. I can comment out the messages
1:42:10line. We'll remove it completely once we
1:42:12add context management in. Now let me
1:42:14create that function. So we have def and
1:42:17let's call this asynchronous def process
1:42:20message. we get self. We also get the
1:42:24message which is of the type of string.
1:42:26And this will return either a string or
1:42:28a null value. And the entire job over
1:42:31here is just to process the events that
1:42:33we get and display the results out on
1:42:35the screen. So first of all, let's just
1:42:38check that hey if self.agent is not
1:42:40present then we just want to return null
1:42:43that we have nothing to do with this.
1:42:45It's all over. The entire thing is
1:42:46complete. But if the agent is not null,
1:42:50in that case I want to go over all of
1:42:52the events that agent.run gives us and
1:42:55we'll pass in the message here. And I
1:42:58can just check what the event type is.
1:43:00The event type can be text delta text
1:43:03complete. So if the event type is equal
1:43:07to event type or let's call this agent
1:43:10event type, let me import that from
1:43:13agent.events.
1:43:15And if the type is let's say agent start
1:43:18if it's agent start what do I want to do
1:43:21nothing really because if the agent
1:43:23starts what do you want to do but if you
1:43:26want to do something we already have
1:43:28laid the groundwork for it let's try to
1:43:31handle something else let's say we have
1:43:33agent or actually instead of agent
1:43:36anything let's just do text delta
1:43:38because we have text streaming out we
1:43:41would like to display it correct so we
1:43:43have content is equal to event dot
1:43:46data.get get and then we have content.
1:43:49If we don't get any content, it's just
1:43:50an empty string because we don't want to
1:43:53display null out on the screen, right?
1:43:55And now I would like to display stuff on
1:43:58the screen. Now, how do you display it
1:44:01on the screen? Well, for that I would
1:44:03like to create another folder called UI.
1:44:05Within this UI, we will have TUI. py or
1:44:09you can call it anything of your choice.
1:44:12Maybe like renderer or something. Now
1:44:15for TUI for displaying stuff on the
1:44:17screen, we'll be making use of another
1:44:19package rich. Rich rich will help us
1:44:22render rich text, tables, progress bar,
1:44:24syntax highlighting, markdown, and more.
1:44:27We're not using most of the things, but
1:44:30we will be using syntax highlighting, I
1:44:32think, and we'll also be using rich text
1:44:35and tables, I think, even panels. But
1:44:38yeah, if you want, you can extend and
1:44:40add more UI features. This tutorial is
1:44:43not really about UI. So let's just
1:44:46import or install rich. So we have pip
1:44:49install rich.
1:44:52And now we do have rich. That's great.
1:44:56Now let's from rich we can import
1:44:59multiple things. One of the things we'll
1:45:01need is console and it's not present in
1:45:05rich actually it's present inri console.
1:45:08So yeah just import that. And then we
1:45:11are going to import this thing agent
1:45:14theme. So this is just how my agent is
1:45:18going to look like. What UI elements are
1:45:20present.
1:45:21So if there's anything information
1:45:23related, it's going to be of the color
1:45:25can or can however you pronounce it.
1:45:28Then you have warning yellow. Then you
1:45:30have error which is bright red and
1:45:32bolded. Success is green. Dim is just
1:45:36dim. Muted is gray 50. Border is gray
1:45:3835. Highlight is bold cyan. Highlight is
1:45:42bold cyan. I really don't know how to
1:45:44pronounce that. Rules is user bright
1:45:48blue bold. Assistance is bright white
1:45:51and so on for other tools and code
1:45:53blocks as well. So we will be making use
1:45:55of that. Now we need to import this
1:45:58theme from rich as well. So you do that
1:46:02doing from rich dot theme import theme.
1:46:07So this is our agent team and here what
1:46:09we've done is just described a mapping
1:46:11between all of the colors bright magenta
1:46:14bold is actually kind of a class within
1:46:18rich and now I'm saying that hey
1:46:20whenever I use tool
1:46:23you need to replace it with this color
1:46:26that just makes it easy for us to write
1:46:28down anything we want because it is more
1:46:31readable for us for example instead of
1:46:33writing yellow I can have warning so It
1:46:37tells me that yeah, it's a warning. And
1:46:39then if I want to replace out any of the
1:46:41colors in the future, I can just say
1:46:43that hey, all warnings should now be red
1:46:46for example. So everything will be red.
1:46:49That allows me to have a centralized
1:46:51control of how the theme looks like. So
1:46:56that is the agent theme. Another thing
1:46:57we need is the console because we need
1:47:00to get the main console instance where
1:47:02we going to write everything. And there
1:47:05we are also going to pass in the agent
1:47:06theme. So we have def get console and it
1:47:11returns to us a console and this console
1:47:15comes from rich doconole and it's going
1:47:18to be a global singleton instance. So we
1:47:21have console which is of the type of
1:47:23console or none. By default it is set to
1:47:26null. And then we are going to check if
1:47:29console
1:47:31is none then we are going to have
1:47:34console equal to console like that and
1:47:38then we pass in the theme. The theme is
1:47:41just this agent theme that we created
1:47:43and the highlight is equal to false. You
1:47:46can set it to true and notice the
1:47:48differences but yeah then we'll just
1:47:50return the console if or if not it's
1:47:53present we've created it. And also we
1:47:56want to tell here that this is not a
1:47:58console variable that we have created
1:48:00just now. This is going to be the global
1:48:02console variable that's present. This is
1:48:05the one we'll be using. All right. Now
1:48:08why do we have a global instance here?
1:48:10The reason for that is singleton. We
1:48:12want a single instance of this console
1:48:14to exist in our entire program because
1:48:17if we don't have a single instance,
1:48:19there will be multiple instances and
1:48:21that will cause problems. Now let's go
1:48:23ahead and create the class of TUI.
1:48:27This is just going to be the main
1:48:29terminal UI renderer. And here I can
1:48:32create an init function the self. And
1:48:35we're going to return null from here.
1:48:38Nothing. And then we're also going to
1:48:40get a console. Console is going to be of
1:48:43the type of console or null or actually
1:48:44we probably don't even want console. We
1:48:47can just reuse this console that's
1:48:49present. So we have self dot console is
1:48:53equal to this console that we already
1:48:56have access to or let's just take it
1:48:59from the constructor. I don't know feels
1:49:02weird to not take it from the
1:49:03constructor. Now by default it is null
1:49:06but yeah we can use that one and if it's
1:49:09not present then we'll just do get
1:49:11console and we get a new console
1:49:14variable. Cool. Now here the first
1:49:16function that we need is related to what
1:49:18we want to do here. We just want to
1:49:20display out the data to the user. And to
1:49:23do that, we're going to create a
1:49:25function called stream assistant
1:49:28delta.
1:49:30It's going to be a self. Then we have a
1:49:32content which is of the type of string
1:49:34and it returns nothing. It just prints
1:49:37out on the console. And now we can just
1:49:40do self.conole.print.
1:49:43Then we print out the content. The end
1:49:46is an empty string. So by default the
1:49:49end is probably going to be a new line,
1:49:51right? You go to a new line. But this
1:49:53time you set it to an empty string
1:49:56because when the text delta is coming
1:49:58in, you just get a bunch of tokens. You
1:50:02don't want to go to a new line when
1:50:03every new token comes in. You want it to
1:50:06keep going so that you can display a
1:50:08sentence nicely to the user. And the
1:50:11markup is going to be set to false. just
1:50:13means that it will not italicize
1:50:15anything or bold anything or anything as
1:50:18such. It will not interpret any markup
1:50:21in the model output. So that's what
1:50:23we're going for. And yeah, as of now,
1:50:26this is the only function. Of course,
1:50:27we're going to beautify this in just a
1:50:30minute or actually a bit later. I don't
1:50:32know. We'll go with the flow. And then
1:50:34we can just say self dot well, we want
1:50:37this tui, right? So we can just have
1:50:39self dot tui which is equal to tui like
1:50:44that. And now we want to import from ui
1:50:47tui.
1:50:49Then we want to pass in console.
1:50:52And console we can just instantiate over
1:50:55here. console is equal to get console.
1:50:59So from ui.tui you also import get
1:51:02console. And if you want you can also
1:51:04create it over here and use it. But I
1:51:07would like to do it at the top.
1:51:10And yeah, that's it. Now we can just ask
1:51:13self.ti
1:51:15streamass assistant delta and then we
1:51:17pass in the content. Great.
1:51:20So we have handled the case of text
1:51:23delta. Let's see how this works out. I
1:51:25would like to just open up the terminal
1:51:28and run python main. py and then we pass
1:51:32in the command. What's going on? I don't
1:51:34think it will get triggered in any case
1:51:36because in agent py we're just having
1:51:39the message here. We're not doing
1:51:41anything about it. We're not adding it
1:51:43to this messages list because we don't
1:51:45have any context management yet and we
1:51:48will be introducing that next but
1:51:50anyways now let's just hit enter and it
1:51:54says cannot import name event type from
1:51:56client.response
1:51:58and that is happening in response. py
1:52:01sorry this is happening in llm client I
1:52:03think. So yeah, we are importing event
1:52:05type here, but there's no event type.
1:52:07There's stream event type and we'll have
1:52:10to change everything here as well.
1:52:14So I will just do command F event type.
1:52:17Whatever was event type now gets shifted
1:52:19to stream event type. And now I'll just
1:52:22find and replace all.
1:52:25So that's done. And yeah, let's just
1:52:28import it correctly at the top. So yeah,
1:52:30stream event type is now done. Now let's
1:52:33run this again. We have python main.py.
1:52:36What's going on? And it says got
1:52:39unexpected extra arguments. Let's try
1:52:42running this. I just put in double
1:52:45quotes and we get another error message
1:52:47saying process message was never
1:52:48awaited. And that's true. If we go to
1:52:51agent.py, we did call process message or
1:52:55or actually in run single we did get
1:52:57self.process message. So yeah, we would
1:53:00like to await it and also return as soon
1:53:03as we get our message, right? So we
1:53:05don't want to go any further than this.
1:53:07We've got a message, we processed it,
1:53:09that's it. Done. Even in this when we do
1:53:12asyncio.run CLI.run single, we know run
1:53:15single is going to return something,
1:53:19right? It's either going to return a
1:53:21string or a null value. If it returns a
1:53:24string value, good. If it doesn't return
1:53:26a string value, well, we are having a
1:53:29null and that confirms my suspicion that
1:53:32there's nothing more to run here. So, if
1:53:34result is null, in that case, I just
1:53:37want to exit out of this. So, we have
1:53:39system.exit one because if the result is
1:53:43null, we were waiting for a string. If
1:53:46the result is null, there has been some
1:53:48error. So, yeah, we'll just have one.
1:53:51Let's try to run this again and hope no
1:53:53error occurs. So I just hit enter
1:53:56and yeah we do get a streaming output
1:53:59right if you notice over here we did get
1:54:01it streaming out and it's all in the
1:54:03same line. So that means the console
1:54:06displays part is working but in agentic
1:54:08loop line 31 it said none type object
1:54:12has no attribute content. uh here I'm
1:54:15just going to put in a condition that
1:54:17hey if the event text delta is not null
1:54:20in that case we have content is equal to
1:54:24event dot text delta dotcontent
1:54:27then we'll have response text plus
1:54:28equals content and then we'll yield it
1:54:31all of this will happen if text delta is
1:54:33present and then we can try running it
1:54:35again and this time we should see no
1:54:38error that's great so our one short
1:54:41single run is now working I would like
1:54:43to beautify this a little bit. So what I
1:54:46like to do is as soon as the user just
1:54:48passes in a message, I would like to
1:54:50show a little bar saying that hey the
1:54:53assistant is replying now whatever you
1:54:56see from now on is the assistant's
1:54:58message. So let me go back to the TUI
1:55:01here and create another function that is
1:55:03just begin assistant. It just says that
1:55:06hey the assistant is now starting to
1:55:08reply and it is going to return null
1:55:11again. And this time we're going to ask
1:55:14self.conole.print
1:55:15so that we go on a new line. And after
1:55:18that I'll again add self.conole.print.
1:55:20And then I'll have a rule because I want
1:55:22to display a ruler. You know kind of a
1:55:25single horizontal line. And then we're
1:55:28going to import the rule. So from rich
1:55:31dot rule we are going to import rule.
1:55:34And then here we go. We have rule
1:55:37imported out. Now we want the text to be
1:55:40present. So text can be displayed just
1:55:42like that. And now from rich.ext we have
1:55:46to import text as well. So there we go.
1:55:51And now we have the text passed in.
1:55:53After that we'll pass in the message
1:55:56here that hey this is the assistant. So
1:55:58between the ruler assistant will be
1:56:00written and the style is going to be
1:56:02well assistant.
1:56:05Now what is style assistant? Well, if
1:56:07you go back to our agent theme here, I
1:56:09said that the assistant is just a bright
1:56:13white. So, a white message will be
1:56:16displayed over here with a white ruler
1:56:17and stuff. And I would also like to
1:56:19maintain a state that just says if the
1:56:23assistant streaming is going on or not.
1:56:25So here initially the assistant stream
1:56:29open is going to be set to false. But as
1:56:32soon as the begin assistant comes in, we
1:56:35are self.assistant assistant stream open
1:56:37equal to true. Why do we need this
1:56:39variable? It will help us later on. Now
1:56:42that we have begin assistant, I would
1:56:44like to go back to this run single and
1:56:46outside of this process message, I would
1:56:49like to create a boolean value of
1:56:52assistant streaming equal to false if
1:56:56the assistant is streaming or not. And
1:56:58then I can here just check that hey if
1:57:01the assistant is not streaming in that
1:57:03case this is the first message that the
1:57:05user has. So I would just like to have
1:57:08self.ty dot
1:57:11begin assistant. Cool. And after that
1:57:14we'll have assistant streaming equal to
1:57:16true because now the first message is
1:57:18coming in. Just imagine we have the
1:57:20first event with a text delta coming in.
1:57:23We just check that hey is the assistant
1:57:25streaming equal to true or a false. If
1:57:27it is false then we begin the assistant.
1:57:30That means it will display that thing
1:57:32that I want to display and then it sets
1:57:34assistant streaming to true and then it
1:57:36streams the text message that comes in.
1:57:39After that what will happen in the next
1:57:42message the next text delta that comes
1:57:44in it will again check that hey is the
1:57:45assistant streaming? Well the assistant
1:57:48is streaming so it will not execute it.
1:57:50So begin assistant will not display the
1:57:52ruler will not display again and all the
1:57:55text will be coming in side by side on
1:57:58one line. Now let's try to run it and
1:58:01see how it looks.
1:58:03So as you can see assistant shows up
1:58:05over here and below that we get all of
1:58:08the user related text. So that is text
1:58:11delta but there's another state right
1:58:13from agent we're also sending text
1:58:15complete. If the text is complete, then
1:58:18what do we want to do? Well, we want to
1:58:21set this assistant streaming to false,
1:58:24right? So here I'll put in an lf. Event
1:58:27dot type is equal to event type agent
1:58:31event type actually.ext complete. And
1:58:34then we'll just have a final response.
1:58:37The final response is equal to event dot
1:58:40data dot get. And now we get the
1:58:44content. After that, we'll again check
1:58:46if assistant is streaming because that
1:58:48was the variable created, right? If the
1:58:50assistant streaming is set to true, I
1:58:52would like to end the assistant. So, I
1:58:55would have assistant streaming is equal
1:58:57to false. And let's also display a UI
1:58:59component. So, in the TUI, I would like
1:59:01to go ahead and create another function
1:59:04which is defend assistant. We get a self
1:59:07here. It will return absolutely nothing.
1:59:10And it just checks that hey if self do
1:59:13assistant stream is open because even
1:59:16over here we was tracking the state
1:59:19right if the assistant stream was open
1:59:22then we want to print out a new line and
1:59:26after that I would like to set the
1:59:29assistant stream open to false it's no
1:59:32longer open and now I'll go back to main
1:59:35py and here I have self tuy dotend
1:59:39assistant and And now that we have the
1:59:41final response with us, I would just
1:59:43like to return it outside of this for
1:59:44loop. Because I told you if process
1:59:47message returns something, this run
1:59:49single is going to return something. And
1:59:52if run single returns something, the
1:59:54result is not null. So we'll not do
1:59:56system.exit one. But if it is null in
1:59:59that case unfortunately we have an
2:00:02error. So it will exit with the error
2:00:05code one. So this is text complete.
2:00:07Let's try to run it again and see the
2:00:09difference.
2:00:10And the difference is just this much.
2:00:13Percent sign is also gone. It's no
2:00:15longer there because we are properly
2:00:18closing and ending everything out.
2:00:21Cell.conole.print
2:00:23helps over here. Now let's also handle
2:00:25the case of agent error. Just in case
2:00:28we're sending the agent error, I would
2:00:30like to display the error out loud. So
2:00:33let's go back to the main. py and here
2:00:36we'll have l if event.ype type is equal
2:00:39to agent event type dot agent error. If
2:00:44that is the case, then we're going to
2:00:45extract error which is event dot
2:00:47data.get we have error and the default
2:00:51string is just unknown error. Now if
2:00:54assistant streaming is true, okay, the
2:00:59same variable if it is true in that case
2:01:02I just want to shut it off because we
2:01:04have an error. I don't want to deal with
2:01:06it anymore. if that's how we want to
2:01:08deal with it, right? Because if we would
2:01:10have done return over here, the entire
2:01:12thing would be over. We don't have
2:01:14anything more to go with. But that's not
2:01:17what I'm going to do. The assistant
2:01:19streaming can keep on going even if
2:01:22there's an error because in a later
2:01:23message, in a later tool call, it can
2:01:26resolve stuff. So, I'm not going to do
2:01:28that. Instead, I'm just going to print
2:01:30out the error. So, console.print
2:01:33and then I'll have an error printed out.
2:01:36for error we don't really have any agent
2:01:38stuff aware like I'm not created
2:01:40anything related to error right or
2:01:43actually I have so we can probably just
2:01:46use that so we have error like that and
2:01:49we spit out the error which is well
2:01:52whatever we extract over here followed
2:01:55by back slash error again or forward
2:01:58slash error so that's the agent error
2:02:01stuff now I would like to actually make
2:02:03an error in the agent so what I'll do is
2:02:07change the model name in LLM client. So
2:02:10I'll go to this keyword arguments part
2:02:13misspell it. Now let's run it again and
2:02:17it says cannot access local variable
2:02:19final response where it is not
2:02:20associated with a value. We can fix
2:02:23that. The problem here is that we have a
2:02:25final response which is returned right
2:02:27but many times final response is not
2:02:30defined for example when we have an
2:02:32agent error. So what I like to do in
2:02:34that case is have a new variable here.
2:02:38Let's call this
2:02:40final response which is of the type of
2:02:43string or null and by default it is
2:02:46null. Now if I try to run this we do get
2:02:49an error and actually the error did not
2:02:52come from this place process message
2:02:54even though it can totally come from
2:02:56here as well because it's the same thing
2:02:59but the same problem even occurred in
2:03:01agent. py if you scroll down and find
2:03:05final response.
2:03:07So if the text is complete, we do get
2:03:09the final response and then agent ends.
2:03:12But if we don't, then there's an error
2:03:15returned out. So outside of this for
2:03:17loop, we're going to be tracking this
2:03:18variable again. Final response string or
2:03:20null is equal to null. And now we can
2:03:23try running it again and hope it works
2:03:26out. And well, it errors out as you can
2:03:29see. So I figured out what the error was
2:03:32and it's a pretty stupid error. Um if
2:03:34you go back to our agent the client
2:03:38specifically in the response. py here we
2:03:40have at the rate data class for the
2:03:43stream event type we don't want the enum
2:03:46to be a data class right that is the
2:03:49error and now if we remove that you'll
2:03:52figure out we will run this and yeah we
2:03:56do get the error clearly displaying out
2:03:59over here now the reason or the way I
2:04:02got to know that this was the error is
2:04:04because I just printed out the event
2:04:07over here and matched it so event.t type
2:04:10was equal to stream type dot error. All
2:04:12right. So that means the lf condition
2:04:14should have been caught over here. But
2:04:16then I tried printing out this
2:04:18particular line event.ype is equal to
2:04:21stream event type.ext delta and that was
2:04:23true as well. That means this if
2:04:25condition was catching the entire thing.
2:04:28But how can the event type be text delta
2:04:31if I if when I printed out the event it
2:04:34was clearly an error. And that's how I
2:04:37checked that hey is it possible that
2:04:40stream event type.ext delta is equal to
2:04:43stream event type error and they were
2:04:45both equal and that's how I figured out
2:04:47that I've configured the enum wrong
2:04:50somewhere and then I found out that I
2:04:52had passed in the at the rate data class
2:04:55at the top and which is why we got that
2:04:58error. So anyways it's resolved we can
2:05:00display the error clearly over here and
2:05:02that's great. Now let's fix the LLM
2:05:06client again. I want the model to be
2:05:08displaying correctly over here. So now
2:05:11we'll change this back to Mistral AI.
2:05:13Now we can run this and we should be
2:05:16seeing the correct data and that's
2:05:18great. Awesome. So now having configured
2:05:22all of those things, we'll keep working
2:05:24on the UI side by side. There are a lot
2:05:26of components to add, but now I will
2:05:28just like to get started on the context
2:05:31management thing. So, how do you get
2:05:33started with the context? Well, it's
2:05:36quite easy. You just go and create a new
2:05:38folder called manager or let's just call
2:05:41it context. And here we will create a
2:05:43new class called manager. py or context
2:05:46manager py whatever you want to call it.
2:05:48Here we're going to create a new class.
2:05:50Let's call it context manager.
2:05:54Then we're going to have an init
2:05:56function. And in this init function
2:05:58we're going to get self. Now this self
2:06:02is going to return nothing.
2:06:05And now we would like to get access to
2:06:07some things, right? This is a context
2:06:09manager. That means we're loading all of
2:06:11the context from the things that we have
2:06:15back into the model. So the very first
2:06:18thing we'll need is the system prompt.
2:06:21We've not talked about system prompt as
2:06:23of now. What is a system prompt? Well,
2:06:25system prompt is just a prompt that is
2:06:29neither of the role of user, neither it
2:06:32is of the role of assistant. It is just
2:06:36a system prompt. It is telling the LLM
2:06:38how [snorts] to behave. What are its
2:06:40instructions? Not what the user
2:06:42specified, but what are its instructions
2:06:44from the side of system like how should
2:06:49the AI agent respond? Because the user
2:06:53prompt is not going to tell how the LLM
2:06:56should behave, right? Should it be
2:06:57polite? Should it be rude? Should it be
2:06:59sassy? Nothing is mentioned in the user
2:07:02prompt. The user just passes in their
2:07:04query. What we have is a system prompt.
2:07:06System prompt tells that the LLM should
2:07:09behave in a certain manner or the date
2:07:11and time right now is this or many other
2:07:14things. All of that comes under system
2:07:16prompt and we'll be curating our own
2:07:19system prompt. So here I'm going to
2:07:21create another folder called prompts
2:07:23within which we have system. py and I'm
2:07:26just going to paste in all of the stuff
2:07:29over here. If you want to find it out
2:07:31what the system prompt is, I've
2:07:33mentioned the link in the description to
2:07:35the GitHub repository. You'll get a
2:07:37bunch of functions more because there's
2:07:39more stuff to add to the system prompt,
2:07:42but you can remove all of the lines that
2:07:45essentially give you an error because
2:07:47they use all the configuration system
2:07:48that we're going to add later on. What
2:07:50we need is these sections as of now.
2:07:53Identity section, agents MD section,
2:07:56security section, operational section,
2:07:59and yeah, rest of the things are just
2:08:01this. So, let's go over it one by one.
2:08:04Okay, first of all, we have a list. In
2:08:06each list, we are going to have one item
2:08:09appended, which is just going to be a
2:08:10massive multi-line string. All of them
2:08:13will have multi-line strings attached to
2:08:15them, and all of them will be
2:08:16concatenated by a separator of two
2:08:19lines. Now, the first section that we
2:08:22add is the identity section. It just
2:08:25gives the LLM instructions about who it
2:08:28is, what it does, what is its role, how
2:08:30it should behave, what is its identity.
2:08:33Essentially, this helps because well, if
2:08:35you tell an LLM that they need to behave
2:08:38like a principal engineer, they start
2:08:40focusing on the architecture or the
2:08:42clean code. Essentially, they start
2:08:45thinking in a manner of
2:08:49principal engineer. Whatever data they
2:08:51have on principal engineers, they start
2:08:53focusing in that particular zone, you
2:08:56can think of them as activating that
2:08:59particular identity.
2:09:01So we are saying that hey you are an AI
2:09:03coding agent, a terminal based coding
2:09:06assistant. You're expected to be
2:09:08precise, safe and helpful. The
2:09:10capabilities are receiving user prompts
2:09:13and other context provided by the
2:09:14harness such as files in the workspace.
2:09:17This is a bit also related to what we
2:09:20are going to have in the future. So
2:09:22yeah,
2:09:24keep that in mind. [snorts] We want to
2:09:26communicate with the user by streaming
2:09:28responses and making tool calls. Right
2:09:30now we've not specified any tool calls
2:09:32it needs to make because I've just
2:09:34removed a lot of stuff that was present
2:09:36in the system prompt because I would
2:09:39like to add them on demand. For example,
2:09:40tool calls. I would like to add a
2:09:43section about tool calls when we
2:09:44actually implement tool calls.
2:09:47But anyways, emit function. So this is
2:09:50all the capabilities depending on
2:09:51configuration. You can request that
2:09:53function calls be escalated to user for
2:09:55approval before running. This is related
2:09:57to the approval system.
2:10:00And remember you are pair programming
2:10:02with the user to help them accomplish
2:10:04their goals. You should be proactive,
2:10:06thorough and focused on delivering
2:10:08highquality results. We've just given
2:10:10its identity, what its capabilities are,
2:10:13what the goal is. So that is the
2:10:16identity section making the agent or the
2:10:18LLM think in a particular manner. The
2:10:21next part is the agents MD
2:10:23specification. So many times what
2:10:26happens is that the repositories or the
2:10:29folders that you're already working with
2:10:31might contain an agents.md file. A file
2:10:34specifically created for agents to
2:10:37understand how the codebase works. That
2:10:39is not a file that the user will
2:10:42interact with. That's just a file
2:10:44created for LLMs. And many repositories
2:10:47are
2:10:49getting into that phase where they
2:10:51create an agents.mmd file. So we just
2:10:53tell it that hey repose might contain
2:10:55agents.mmd. These files can appear
2:10:58anywhere just find out if there's an
2:11:01agents.md. There'll be instruction in
2:11:04the agent.md.
2:11:06And yeah so that's just stuff about
2:11:08agents.md. You can read the entire
2:11:10prompt yourself.
2:11:12Then we have the security guidelines.
2:11:15You know what are the security concerns
2:11:17related to our project. Now this is not
2:11:20an exhaustive list. There can be
2:11:22multiple more and I'm very sure I'm
2:11:24missing out on multiple things. But yeah
2:11:27the security section is never expose
2:11:29secrets. That means API keys, password,
2:11:32tokens. We validate all of the paths. So
2:11:34we ensure file operations stay within
2:11:36the project workspace. We be cautious
2:11:39with the commands because shell commands
2:11:41might just delete your entire
2:11:43repository. You might have seen the news
2:11:45that a shell command just deletes the
2:11:47database because well they have the yolo
2:11:51mode going on. We'll also have all of
2:11:53those modes. So we want to ensure that
2:11:56the LLM is cautious with those commands.
2:11:59Then we have prompt injection defense.
2:12:02We ignore any instructions embedded in
2:12:04file. then there's no arbitrary code
2:12:07execution and always focus on security
2:12:10first. If the code doesn't work, it's
2:12:13fine, but just ensure you don't delete
2:12:15stuff for the user. That's very
2:12:17important for them because it might
2:12:19never be retrievable.
2:12:22Now the next stuff is the operational
2:12:24guidelines. So what are the operation
2:12:26guidelines? Well, what is the stone and
2:12:28style? How should the CLI interact? It
2:12:31needs to be concise and direct, minimal
2:12:33output, clarity over brevity, no
2:12:36chitchat. It should format everything
2:12:38using markdown. Then you have tools
2:12:40versus text. It should use tools for
2:12:42actions. Text output style only for
2:12:44communication. Handle inability. So if
2:12:48it is not able to fulfill a request, it
2:12:50just states it out loud. And this is
2:12:53actually a very powerful statement
2:12:56because many times the LLM will just
2:12:59hallucinate. But if you give them a way
2:13:00out, they will try to tell you that hey,
2:13:04I can't complete this. I'm sorry. Then
2:13:06you have primary workflows, software
2:13:08engineering task. So whenever requested
2:13:12like fixing bugs or adding features
2:13:15first they need to understand, then they
2:13:16plan, then they implement, then they
2:13:18verify with test, then they verify
2:13:20standard, then they finalize. Now you
2:13:23can obviously tweak according to your
2:13:25preferences. By no means is this like a
2:13:28perfect prompt. I just put in what I
2:13:30think should be present in a prompt.
2:13:33Then you have task execution. So they
2:13:36are a code agent. So please keep going
2:13:38until the query is completely resolved.
2:13:41And then we talk about tool usage. We
2:13:43are going to support parallelism. Right?
2:13:45So if there are multiple independent
2:13:48tool calls in parallel just use them so
2:13:50that we don't burn through a lot of
2:13:52tokens.
2:13:54Then you have command execution, file
2:13:56operation, file creation, remembering
2:13:58facts, which is using the memory tool.
2:14:01Then you have task management using the
2:14:02to-dos tool. Well, we're going to add
2:14:05all of that back again because there are
2:14:09specific sections that talk about tool
2:14:11usage in detail. Then you have error
2:14:13recovery, what should happen when
2:14:16something goes wrong. Then there are
2:14:17code references.
2:14:19You know for example when we are
2:14:21referencing some specific functions when
2:14:24talking to the LLM
2:14:26we would like to have them marked in
2:14:28this back ticks so that you know in the
2:14:31front end if you want to display it
2:14:32nicely we can then it also spits out the
2:14:37folder the line number everything. Then
2:14:40technical accuracy is prioritized and
2:14:43these are all the coding guidelines. You
2:14:45can obviously add more of them. By no
2:14:48means is a system prompt just a oneshot
2:14:50prompt. You analyze your user's request,
2:14:53then you tweak your prompt, then you see
2:14:55how the users are behaving again or you
2:14:57run your independent evaluations
2:15:00and see how it works out.
2:15:04So that is the system prompt and that's
2:15:06what we're going to use over here as
2:15:08well.
2:15:09Again, this system prompt needs a lot of
2:15:12work. We will be adding more stuff later
2:15:15on, but as of now, this is good enough.
2:15:18Let's import it from prompts.
2:15:20Another thing this is going to have is a
2:15:22messages list. So, we'll ask self dot
2:15:25messages and also system prompt can be
2:15:27an underscore.
2:15:29Self dot messages will be an empty list.
2:15:32And the type of this messages is going
2:15:34to be message item, a list of message
2:15:37item. And we're going to create this
2:15:39class of message item. And that is going
2:15:42to be a data class. So from data classes
2:15:45we are going to import data class. And
2:15:48then we're going to have at the rate
2:15:51data class
2:15:53message item. And now what are the
2:15:56properties in a message? What are what
2:15:58is each message in the context going to
2:16:00contain? Well, there's going to be a
2:16:02role which is going to be of the type of
2:16:03string. So it can be a user, assistant,
2:16:06tool, system, whatever is the role. Then
2:16:09you have content associated with it
2:16:11which is also going to be a string. Then
2:16:13there are multiple things like tool call
2:16:16ID, tool calls, token count, compaction
2:16:18date. We're going to add all of them as
2:16:21we need them. Maybe we can add token
2:16:23count now itself. So we have an integer
2:16:26or null which is by default null.
2:16:29Awesome. Now in this context manager,
2:16:32well, we have the messages list, right?
2:16:34This is what we have to maintain. So the
2:16:37very first thing I'm going to add is the
2:16:38user message function. So just in case
2:16:41we want to add a user message, we're
2:16:43going to get that self content which is
2:16:45of the type of string and we return
2:16:48null. Now an item is going to be created
2:16:52like this message item where you pass in
2:16:54the role which is a user because we have
2:16:56added a new user message, right? Then we
2:16:58have a content which is just content and
2:17:01then you finally have token count. Now,
2:17:04how do you count the number of tokens?
2:17:07Well, it depends on the model. Each
2:17:10model has a different token count. Just
2:17:12to demonstrate, you can go to
2:17:14platform.openai.com/tokenizer
2:17:17and you'll learn about language model
2:17:19tokenization. If I type in some text
2:17:21here, hi, this is Rean and how is it
2:17:27going? You'll notice there are 10 tokens
2:17:29over here. Now if I switch to another
2:17:31model like GPD 3.5 and GPD4 there are 11
2:17:35tokens and if I move to GPD3 I again
2:17:38have 11 tokens. So as you can see the
2:17:40tokenizer between GPD 3.5 and GPD4 and
2:17:45GPD 40 changed because the number of
2:17:48tokens has changed. So my point here is
2:17:51that each model even different models of
2:17:55open AAI the newer ones will have
2:17:58different tokenizers. It's inevitable.
2:18:00So it really depends on the tokenizer
2:18:03about what the token is. But a ref
2:18:05estimation is that let's say we have 10
2:18:08tokens here and 37 characters. So one
2:18:11token is roughly four characters.
2:18:14That is the estimation. So we can do
2:18:18that estimation and it will work out.
2:18:21Otherwise you can go ahead and install a
2:18:24package called tick tokenizer. And this
2:18:27tick tokenizer is coming from Pi. It's
2:18:31not the npm package and actually it's
2:18:34called tick token. It is a fast BPE
2:18:36tokenizer which stands for bite pair
2:18:38encoding. But those are very specific
2:18:41LLM terms. However, what we are
2:18:44interested in is getting the number of
2:18:47tokens using tick token or we can just
2:18:50estimate the number of tokens using a
2:18:53rough estimation. Right? So let me just
2:18:55go ahead and install tick token right
2:18:58here. Cool. And after that I'm going to
2:19:01create a new folder called utils within
2:19:03which we have text. py and within this
2:19:06text py we're going to have a function
2:19:10called get tokenizer. Then we get a
2:19:13model which is of the type of string and
2:19:16well we just get the tokenizer over
2:19:19here. Now what is the point of
2:19:21tokenizer? Well, it will help convert to
2:19:25tokens. But what we are interested in is
2:19:27not converting it into tokens, but just
2:19:30getting the length of the token or the
2:19:32number of tokens that are present. And
2:19:35we don't have to worry about the speed
2:19:37because it's almost instantaneous. So we
2:19:39can just do this try
2:19:42and then we have an accept exception and
2:19:45if exception occurs then what we're
2:19:47going to do is encoding is equal to tick
2:19:50token and we have to import tick token.
2:19:53Now if the import tick token has been
2:19:55successful that's great we'll do tick
2:19:58token.get get encoding and then we'll
2:20:01pass in the base as CLK
2:20:05base which is used by GPD4 and then
2:20:08we'll just do return encoding dot
2:20:11encode.
2:20:12This is the function that's returned and
2:20:15then in the try block we want to do
2:20:18encoding is equal to tick token dot
2:20:20encoding for model because specifically
2:20:23we want to get the encoding for whatever
2:20:25model is present right and then we just
2:20:28return encoding dot encode like that. So
2:20:32we'll try it for this specific model. If
2:20:35it's not present then we fall back to
2:20:37GPD4 tokenizer. And now we want to
2:20:40create a function to count the tokens.
2:20:42So we'll get a text which is of the type
2:20:44of string. Then we'll get a model which
2:20:46is of the type of string. And yeah,
2:20:48we'll just return null or integer,
2:20:51sorry, because we're counting the
2:20:53tokens, right? And now we'll do
2:20:54tokenizer is equal to get tokenizer.
2:20:57We'll pass in the model. And then we can
2:21:00just do if tokenizer is present, we
2:21:02return the length of the tokenizer of
2:21:08text. Otherwise, you know, if the
2:21:11tokenizer is not present, we don't get
2:21:13anything back. What we want to do is
2:21:16estimate the number of tokens. So, we
2:21:19are going to have def estimate tokens.
2:21:22We get a text which is of the type of
2:21:24string and then we return an integer and
2:21:27then we return max of 1 comma length of
2:21:31text divided by 4. What have we done
2:21:34over here? Well, we are just saying that
2:21:36we want to take the max number between 1
2:21:40and length of text divided by four.
2:21:42Length of text is just how many
2:21:44characters there are present in a
2:21:45sentence and we divide it by four to get
2:21:48the number of tokens. Right? Because on
2:21:50average one token is equal to four
2:21:54characters with GPD 40 we saw that 10
2:21:58tokens was about 37 characters which is
2:22:00approximately four. So that's what we
2:22:02did over here as well. So now if the
2:22:04tokenizer is not present or we run into
2:22:06any sort of error we can just return
2:22:08estimate tokens and pass in the text.
2:22:12Awesome. So that is it about tokenizer.
2:22:15Now let's use this count tokens
2:22:17function. So we have count tokens and we
2:22:21take that from utils.ext.
2:22:27Then we need to pass in the content. The
2:22:29content is passed in over here. And then
2:22:31we need the model name. Now the model
2:22:34name is obviously hardcoded. Um but we
2:22:38will be getting it from the
2:22:39configuration system as of now. Let me
2:22:42just copy what we had in LLM client
2:22:46which is mistrali destral.
2:22:49Obviously we will be changing that. And
2:22:52now we have self dot model name. And I
2:22:56think most likely in tick token this
2:22:58model is not present. And that's why
2:23:00we'll be shifting to the GPD4 tokenizer.
2:23:04So it's not going to be very accurate
2:23:06but it's fairly accurate. It won't
2:23:09create a lot of difference. So now that
2:23:12we have created this item, what we want
2:23:14to do is add the user message, right? So
2:23:16what we want to do is self dot messages
2:23:19do.append and we want to append this
2:23:21item to this messages list. So that is
2:23:25the add user message. Now we going to
2:23:27copy this function and similarly create
2:23:30something for add assistant message. So
2:23:33we have add assistant message like that.
2:23:35Then we're going to get a content. And
2:23:38yeah, there'll also be tool calls that
2:23:40the assistant can do, but we'll ignore
2:23:43it as of now. Then we get assistant.
2:23:46Then there's content. If the content is
2:23:48not present, it's going to be an empty
2:23:50string. And then we can count the number
2:23:53of tokens. And now I can just do self do
2:23:56messages.append item. Great. Now
2:23:59obviously there's more functions that we
2:24:00can add like get messages which just
2:24:03gets all the messages in a dictionary
2:24:06format. So right now all the messages
2:24:09are a list of message infos but we would
2:24:11like to get them in a dictionary format.
2:24:13Right? So we can do that and we should
2:24:16do that. Let's do that. So we have def
2:24:20get messages and then we have okay let
2:24:24me have messages here then self and
2:24:26we're going to return a list of
2:24:28dictionary of string comma any cool now
2:24:33that we have any we'll import that from
2:24:36typing and now first of all we'll create
2:24:39a messages list after that we want to
2:24:42check if the system prompt is present so
2:24:44if self dots system prompt is present
2:24:47which it should have. But if it's not
2:24:50empty, then what we want to do is
2:24:52messages.append.
2:24:54And we're going to append to this a
2:24:55dictionary where we pass in the role
2:24:57which is system and then we pass in the
2:25:01content which is self dots system
2:25:04prompt. Cool. After that we'll go over
2:25:07every element in self do messages. So I
2:25:11have for item in self do messages. And
2:25:16then we'll just do messages.append
2:25:19item. But we want this item to be
2:25:21converted into a dictionary. Right? So
2:25:24what we want to do is go over here and
2:25:26create a function called to dictionary
2:25:28which is self.
2:25:30And here we're going to return a
2:25:32dictionary with string as the key and
2:25:34the value can be any. Then we have a
2:25:37result which is also a dictionary with
2:25:40the key as string and any as the value.
2:25:44And then first of all we'll pass in the
2:25:47role. The role is just going to be self
2:25:50dot ro. We're just converting it into a
2:25:53dictionary right. So we will pass in the
2:25:55role. After that we need to pass in the
2:25:58content as well. So here we can just
2:26:01pass in if self.content is present then
2:26:05we're going to do result at content is
2:26:08equal to self do.content.
2:26:11And then there can be tool calls and
2:26:13stuff but we'll add that later on. And
2:26:15now we'll just return the result. So
2:26:18that is to dictionary. We are not
2:26:21passing in the token count. Token count
2:26:23is solely there for our purposes. And
2:26:25I'll explain later why we need it
2:26:28because we'll be using that in a
2:26:30function. So now I can go over here and
2:26:32just call to dictionary. Cool. So now we
2:26:36are in an open AI format where we have a
2:26:39list of dictionaries with role and
2:26:42content passed to it. If there are tool
2:26:44calls we'll add that later on. And
2:26:46finally after everything is done we can
2:26:48return the messages which is a list over
2:26:51here. Now there's other function we can
2:26:53add as well but I'm not diving into
2:26:55that. We'll create them as and when we
2:26:58need it but these are three basic
2:26:59functions. adding the user message,
2:27:01adding assistant and getting all of the
2:27:03messages in a dictionary format so that
2:27:06we can pass it to our agent. Now let's
2:27:09come back to our agentic loop and see
2:27:12what we need over here. So in the run
2:27:14function I told you that we will have to
2:27:18add the user message to context right
2:27:21let's add it. So first we'll have to
2:27:24instantiate context manager. So we have
2:27:26self.context context manager and just
2:27:28like client context manager should also
2:27:30be within a session because well it just
2:27:33makes sense to have a context manager in
2:27:36a session. If you have multiple context
2:27:39managers, you have multiple sessions.
2:27:41One session can have one context
2:27:43manager. So yeah, we have context
2:27:46manager instantiated. Now we can just
2:27:49have self dot context manager dot add
2:27:52user message because right now we just
2:27:56add the user message to context right so
2:27:58we add the user message and now we need
2:28:01to pass in the content the content is
2:28:03the message itself that's great so we
2:28:05have added the user message to context
2:28:08now the next thing we want to do is that
2:28:11whenever we run the agentic loop we get
2:28:14the event right and we get the user's
2:28:16text now We want to record that user's
2:28:19text. So just before we have response
2:28:22text, we'll just check over here that
2:28:24hey, we have self.context manager dot
2:28:27add assistant message the content. Now
2:28:30what is the content going to be? Well,
2:28:32it's just going to be the response text,
2:28:34right? And if it's not present, if it's
2:28:36empty, then we just pass in null that
2:28:39it's fine. The assistant message can be
2:28:42null. And then obviously if there's any
2:28:44tool calls, we would like to add that as
2:28:46well. Again, let's just leave it for
2:28:48now. So now we have the entire context
2:28:50with us. I can remove this messages
2:28:52list. And now instead of having messages
2:28:56passed in and then we can just pass in
2:28:58self dot context manager dot get
2:29:03messages. And now we have access to the
2:29:06entire messages in the aai JSON format.
2:29:10Right? Because we have a list of
2:29:13dictionaries. So what have we done here?
2:29:16The changes we have made is adding the
2:29:18user message because the user clicked on
2:29:19run. Right? So when a user first passes
2:29:22in a prompt, we go to this run method.
2:29:25We add the user message. Then we run the
2:29:28agentic loop. Within the agentic loop,
2:29:30we just get all of the messages. The get
2:29:32messages will get the system prompt. So
2:29:35yeah, we have appended the system prompt
2:29:37over here. The first thing should be the
2:29:40system prompt. After that we add all of
2:29:43the user related and the assistant
2:29:45related messages whatever is present. So
2:29:49in our cell do messages right now we
2:29:51just have a user message because we just
2:29:54added a user message but obviously if
2:29:57it's a multi-turn event there can be
2:30:00multiple assistant related messages
2:30:02there can be multiple user related
2:30:04messages
2:30:05and that's what we're going to work on
2:30:07next. But as of now we have created a
2:30:10good functioning context management
2:30:13system. Now we are no longer depending
2:30:15on a hard-coded text. So now I would
2:30:19just like to clear everything off and
2:30:22run this particular thing. Now instead
2:30:25of this prompt I would like to say what
2:30:27is 1 + 1 and then hit enter.
2:30:34And then the assistant responds with
2:30:35two. That means our entire context is
2:30:38now working. All the hard-coded context
2:30:41related stuff is out now. And now we can
2:30:44start working on another thing which is
2:30:47tool calling. Now what is tool calling?
2:30:50Tool calling is essentially an LLM
2:30:52taking actions for us. I would like to
2:30:54explain this with a diagram by OpenAI
2:30:57itself. Let's say we have the developer
2:31:00we have given it instructions that you
2:31:01have these these this this tool just use
2:31:03them on time so that you know you can do
2:31:06stuff for us. So let's say the user says
2:31:11that hey in my repository or in my
2:31:14folder you want to fix a bug. Now to
2:31:17understand what bug to fix it the agent
2:31:20or the LLM will have to read the file
2:31:22right but how can it read the file we're
2:31:25not loading any file reading stuff into
2:31:27the model if we load all of the file
2:31:30content within the LLM it will just go
2:31:32out of context because there are so many
2:31:35files and they consume so many tokens so
2:31:38obviously we can't do that but we can
2:31:40tell the model that hey if you want to
2:31:42read one specific file or a bunch of
2:31:45files not all of the files we can give
2:31:47it to you just let us know. So we tell
2:31:50it that hey here's a tool call if at all
2:31:53in the future you want to read a file
2:31:56just let me know by telling that hey
2:31:58please execute this read file. So what
2:32:02happens is the user asks a question like
2:32:05what's the weather in Paris? It tells it
2:32:07this is the tool definition. So it
2:32:10passes it in the model decides that hey
2:32:13if you want to understand what's the
2:32:14weather in Paris we would like to call
2:32:16this tool like that get weather with
2:32:19Paris passed to it. So it comes to the
2:32:21developer. Now the developer will
2:32:23execute that tool call. The model has no
2:32:26capability of calling the tool. Right?
2:32:28Because in our case if you want to read
2:32:31the file how can the model read a file
2:32:33when it's operating remotely on another
2:32:36computer in the cloud we will have to
2:32:38read the file for them because our tool
2:32:42the developer tool is running locally.
2:32:45So the model will tell us that hey
2:32:48please call get weather or read file
2:32:50tool for us. Then we'll execute that
2:32:52tool or the function code. Tool is just
2:32:55a function code. All right, if you want
2:32:56to read a file, we just say that hey go
2:32:58and read these file. If you want to read
2:33:00a file from this line to this line,
2:33:02again the model will let us know through
2:33:04the parameters. So let's say the model
2:33:07calls read file and they want to read
2:33:09what file. Well, they can specify some
2:33:11path to a filet txt. Right? Now they can
2:33:15also specify other things like line
2:33:17number. If they want to read line number
2:33:20from 14 to 30. So they can specify that
2:33:23over here the beginning line and the
2:33:25ending line. So they just let the
2:33:27developer know that hey we want to call
2:33:30this tool I think it would be relevant.
2:33:32So we execute that function code. We'll
2:33:34read the file for the user or we'll call
2:33:36the get weather paris function for the
2:33:39user and it gets a temperature. In our
2:33:41case it will get the files content from
2:33:44lines 14 to 30. And then we send that
2:33:47back to the model. Now the model has
2:33:50access to the file or it has access to
2:33:54the temperature.
2:33:56Then it sees that hey can I do something
2:33:58else with this? Well in this case the
2:34:02prompt was very easy. What's the weather
2:34:03in Paris? It notices yeah we have got in
2:34:06the weather. Let me just frame it in a
2:34:08nice way. So it will say it's currently
2:34:1014°C in Paris. But in our case reading
2:34:14file was step number one. Our prompt was
2:34:17fixing the bug. So now that it has read
2:34:20the file, it will determine that hey
2:34:23this file looks relevant. There is the
2:34:25bug. I know how to fix it. So it will
2:34:28just call something like edit file.
2:34:32So edit file will again pass the path to
2:34:36this particular file and then it will
2:34:38also pass in all of the content that
2:34:40needs to be passed in over here. the
2:34:42content is that you need to replace at
2:34:45line let's say 14
2:34:48maybe you've typed out print statement
2:34:50you need to replace that with a logger
2:34:53so you have logger get info and all of
2:34:55that it's just an example but you get
2:34:57the point the llm gets access to the
2:35:01file that it has been reading the llm
2:35:04gets access to the file it read over
2:35:06here it wanted to read over here the llm
2:35:09gets access to the file it wanted to
2:35:11read then it reads it and then it
2:35:13understands that yeah I understood the
2:35:15bug let's fix it and then it calls
2:35:17another tool to fix it so that is the
2:35:20purpose of tool without tools it won't
2:35:23be able to take any action tool is
2:35:25essentially LLM's way to interact with
2:35:28the outside world so it was a very big
2:35:30example and a very big talk but that's
2:35:33how tools will help us and we're going
2:35:35to create a bunch of tools so let's open
2:35:37our sidebar and over here I'm going to
2:35:39create another folder now this folder is
2:35:42going to be called tools. And then
2:35:44within this folder, we're going to have
2:35:46multiple tools. There are going to be
2:35:48built-in tools. There's going to be MCP
2:35:50because MCP is essentially just a tool
2:35:52call. So that's what we're going to
2:35:54have. But we're going to start off by
2:35:56creating something known as a base. py.
2:35:59This base py is going to contain the
2:36:01base class for all of the abstraction
2:36:04that the tools are going to have. So
2:36:06here we're going to define a class tool
2:36:08which is going to be a base class and
2:36:10all the other tools like read file or
2:36:12write file or edit file or patching
2:36:15everything will extend this tool. So
2:36:18it's going to be an abstractbased class.
2:36:20So let's import ABC and then we're going
2:36:24to inherit from abc.abc.
2:36:27That's how you create an abstract class
2:36:29within Python. So this is the base class
2:36:32for all of the tools. Now we're going to
2:36:35have an init function and in this init
2:36:37function as of now we're going to have
2:36:39well absolutely nothing. So it's just
2:36:42going to be null like that. Okay,
2:36:45nothing done over here. After that we're
2:36:47going to define the name property over
2:36:50here which is going to be of the type of
2:36:52string which is going to be let's call
2:36:54it base tool. After that we're going to
2:36:57have a description and all of the tools
2:37:00are going to have these properties. All
2:37:01right, name is going to be present in
2:37:03whatever tool inherits from this tool.
2:37:06So read tool will have its own name.
2:37:08It's going to have its own description.
2:37:10So the name helps us well display it on
2:37:13the screen what type of tool it is. And
2:37:15description will specifically help us
2:37:18when we want to select what tool to do.
2:37:20So if we come back to this particular
2:37:22diagram where we are telling the model
2:37:24to select one of the tools to do a tool
2:37:26call, it how does it know what tool to
2:37:29call, right? It just doesn't take in the
2:37:31name and decide what to do. It also
2:37:34takes in a description of when the tool
2:37:35should be called. So we are going to
2:37:38specify a description as well. This is
2:37:40just going to be base tool. And then
2:37:42we're going to have a kind. What is the
2:37:44kind of this tool? Is it going to be a
2:37:47readonly tool? Is it going to do network
2:37:49operations? Is it going to do a write
2:37:51operation? What kind of tool is it? So
2:37:53we going to create that enum at the top.
2:37:56So we're going to have from enum import
2:37:59enum and then a class toolkind
2:38:03where we go ahead and create an enum.
2:38:07Now the first one is going to be read
2:38:09which is just read only operations tool.
2:38:12So that's going to be read file. Then we
2:38:14have write one which is equal to write.
2:38:17So these are file modifications. Then we
2:38:20have shell which is well shell
2:38:22operations. Then we have shell
2:38:25operations just means command line
2:38:26operations. All right. Then you have
2:38:28network which is equal to network. These
2:38:31are all things that would happen over
2:38:33the network like searching the web,
2:38:36fetching a particular URL, all of that.
2:38:38After that we're going to have memory
2:38:41because we're going to have one tool
2:38:43dedicated to memory because memory is a
2:38:45pretty cool feature. You can note down
2:38:48your memories and your agent knows it
2:38:50about you. So it can be applied to all
2:38:53or the current project whatever. Then
2:38:55you have an MCP which is just equal to
2:38:57MCP.
2:38:59Whatever tools are MCP related we are
2:39:01not exactly going to know what type of
2:39:03tool it is right like is it specifically
2:39:06a read operation or a write operation.
2:39:07All we know is that it's an MCP. We're
2:39:10not going to analyze the codebase to
2:39:12understand what that MCP does. It will
2:39:14just do the job.
2:39:16Then the kind here is going to be of the
2:39:18type tool kind. And then that is equal
2:39:22to toolkind dot read or you can change
2:39:26it to whatever. I'm just going to change
2:39:27it to read because it's the most basic
2:39:29permission that we have or the basic
2:39:31tool kind that we have. Awesome. So in
2:39:34this init function we're also going to
2:39:36have the configuration and we're going
2:39:37to create that once we have figured out
2:39:40everything related to tools and sessions
2:39:42and all of that. So yeah
2:39:45after that we're going to have another
2:39:46property here which is going to be
2:39:48schema. Now the schema will be created
2:39:50as a function but each abstract class
2:39:54will have to implement this schema.
2:39:58So here what we're going to do is have
2:39:59at the rate property. This is
2:40:01essentially a property that you can use
2:40:03in the class that inherits from this
2:40:05class. If you don't understand don't
2:40:07worry we'll create a readonly tool and
2:40:10you'll be able to understand it then. So
2:40:12the schema can return a dictionary which
2:40:15is having the key as string and the
2:40:17value can be any because well it can be
2:40:20a string integer whatever we don't know
2:40:22or it can also be of the type of base
2:40:26model. Where is this base model coming
2:40:28from? This base model comes from a
2:40:30package known as pyantic. So pyantic is
2:40:34the most widely used data validation
2:40:36library for Python. We'll be using this.
2:40:38There's another package called Pyantic
2:40:40AI which does all of the things that we
2:40:43have talked about with the OpenAI
2:40:45package but it just has a Pyantic AI
2:40:48flavor to it. So it uses Pyantic and all
2:40:50of that stuff and it connects to any
2:40:52model and it also has functions related
2:40:55to creating your own agent. So whatever
2:40:57agentic loop we created Pyantic AI can
2:41:00do that as well by just writing one
2:41:02simple function but it does not allow as
2:41:05much configuration as we need it. You
2:41:07cannot put in a lot of customization
2:41:10into that agentic loop. So that's why we
2:41:14created our own agentic loop. So let's
2:41:16install pyantic. We'll just open up our
2:41:18terminal and here have pip install
2:41:21piantic. So we have installed it. That's
2:41:24great. Now let's go at the top and do
2:41:26two things. First from pyantic we are
2:41:29going to import base model. And then
2:41:33also at the top we're going to do from
2:41:36future import annotations. Almost every
2:41:39file that we're going to talk about is
2:41:41going to have this line. It will be
2:41:42useful somewhere or the other. And then
2:41:45for the schema in this base tool, well,
2:41:48we don't have anything to do. What the
2:41:50schema does is represents what the tool
2:41:54schema definition is. What kind of
2:41:56parameters this tool is going to have.
2:41:58For example, in the read operation, I
2:42:00told you whenever we trying to read a
2:42:02file, you need to specify the path of
2:42:04the file. If there's any offset, if
2:42:06there's any limit, whatever, you need to
2:42:09specify it through these parameters.
2:42:12Right? That is the schema that I'm
2:42:14talking about. Now, the base tool is
2:42:16obviously not going to have any schema.
2:42:18Every tool that we create is going to
2:42:20have the schema property where it
2:42:22defines all of the parameters. So here
2:42:24we're just going to raise a notimp
2:42:26implemented error because every tool
2:42:29must define schema property. This is an
2:42:34error from our side and we don't want to
2:42:38miss out on that. So we'll just erase
2:42:40this error or class attribute. Awesome.
2:42:43After that we're going to define another
2:42:45function and that is the execute
2:42:47function. This execute function is going
2:42:49to be an abstract method. every tool
2:42:52will have to define this function. So we
2:42:56are going to have add the rateabc
2:42:58dotabstract class method or abstract
2:43:01method that should be fine and we don't
2:43:03have to call it like that. Maybe this is
2:43:05fine and then we'll have async def
2:43:07execute. This execute function is
2:43:10essentially what should happen if the
2:43:12tool is executed. So let's say the model
2:43:15tells us that we need to call get
2:43:17weather tool call. So we have to execute
2:43:20this function code, right? What should
2:43:22happen when the function is executed?
2:43:24That's what's going to be defined in the
2:43:26execute part. Now this is going to take
2:43:28in a bunch of tool methods. So we going
2:43:30to have invocation and this is just
2:43:33going to contain all of the parameters
2:43:35of the tool invocation. So we're going
2:43:37to create a custom class called tool
2:43:39invocation and it's going to return
2:43:42something known as tool result which is
2:43:44also something we have to create. These
2:43:47are two custom classes that we'll create
2:43:48right at the top. So the very first
2:43:51thing is tool invocation, right? So
2:43:54we'll have a data class and let me
2:43:56import the data class. And to import
2:43:59data class, we have to do from data
2:44:01classes import data class. And then here
2:44:04we can have
2:44:06class tool invocation.
2:44:10And now what are the parameters that I
2:44:13need to execute a tool call? What
2:44:16details will I need? Not many. The first
2:44:19thing that we'll require is the current
2:44:21working directory. So, CWD stands for
2:44:23current working directory and it's going
2:44:25to refer to a path. Which path do we
2:44:28want to execute this tool in? And this
2:44:30is especially useful when you're trying
2:44:32to do a read operation, right? What is
2:44:34the current working directory or if
2:44:37you're doing a write operation because
2:44:40what path are we writing to and what
2:44:43path are we in right now? This is
2:44:45especially useful with safety
2:44:47permissions that we're going to add
2:44:48because let's say I started the AI agent
2:44:52in an interactive mode and I start it in
2:44:54this AI agent folder itself.
2:44:57By default, my AI agent will not be
2:45:00allowed to make any operations outside
2:45:02of this folder. So if I want to make any
2:45:05changes in desktop, it will require my
2:45:07permission. That's how our AI agent is
2:45:10going to work. So having the current
2:45:12working directory will give us a
2:45:13perspective of where we are and if the
2:45:17agent is trying to make any changes
2:45:19outside of where we have opened it. For
2:45:20example, it's trying to make changes in
2:45:22desktop but we are in AI agent then it
2:45:25will get to know that it's in desktop
2:45:27not in this current folder. So we'll ask
2:45:29for the user permission. So this is
2:45:31particularly useful over there. We'll
2:45:33find other use cases as well but this is
2:45:35one of the use cases. Now let's import
2:45:38path from pathlib. Another thing that
2:45:40we'll require is parameters. What are
2:45:43the other parameters that you can have?
2:45:46Because read operation will require
2:45:48something else. Write will require
2:45:50something else. Shell operation will
2:45:52require something else. So yeah, also I
2:45:55fixed the tool invocation name over
2:45:58here. So yeah, next thing we need is a
2:46:01tool result. Now tool result is
2:46:03essentially what happened after the tool
2:46:06was executed. Was it a success? If it
2:46:09was a success, what was the output? If
2:46:11there was an error, what was the error?
2:46:13Then if there's any metadata that you
2:46:16want to attach and all of that stuff. So
2:46:19let's create a data class for tool
2:46:21result as well. So we going to have a
2:46:23class tool result. And just so you know,
2:46:26we are only creating data classes and
2:46:28classes for all of that because this is
2:46:30a more object-oriented way of going
2:46:32about things. If you want, you can
2:46:33directly return a dictionary over here
2:46:35and that's totally fine. But I want that
2:46:38nice autocomplete. That's majorly why
2:46:40I've stuck to creating data classes. So
2:46:43we have success which is a boolean value
2:46:45if it was a success or not. Then we have
2:46:48an output which is of the type a string.
2:46:50Then we have an error which is of the
2:46:52type a string or null and by default it
2:46:54is null. Then you have metadata.
2:46:57Metadata is any data that you want to
2:47:00attach to this already existing data.
2:47:03Right? Data on top of data. Now metadata
2:47:07can refer to multiple things. For
2:47:09example, in list directory that's a tool
2:47:12that we're going to create when we try
2:47:14to list out the directory. So let's say
2:47:16if I'm in this directory, I want to see
2:47:18what all directories or files are
2:47:20present within this AI agent directory.
2:47:23Right? So I can use a tool to do that
2:47:25and that tool will give out meta data
2:47:28like what was the last modification time
2:47:31for this folder, what was the last
2:47:33modification time for this folder, what
2:47:35was the size of this folder, what was
2:47:37the size of this file. All of that data
2:47:39will be stored in metadata. But in the
2:47:41output we going to have well the list of
2:47:44files or folders that are present within
2:47:47the directory. So that is the difference
2:47:50between output and metadata. Again when
2:47:52we work on it it will get clearer. So we
2:47:55have string comma any and by default
2:47:58this is going to be a field which is
2:47:59coming from data classes as well and the
2:48:02default factory is going to be a
2:48:04dictionary. If nothing is present it's
2:48:06not going to be a null value it's going
2:48:07to be an empty dictionary. Now there's
2:48:10other things that we want to add as well
2:48:12which are related to displaying. So
2:48:14there's going to be a diff that we can
2:48:16add some truncation logic you know like
2:48:19if the response was truncated or not and
2:48:21then an exit code but we'll look into
2:48:23that later on. Now we've just defined
2:48:27how this abstract method is going to
2:48:30look like. That's good enough for now.
2:48:32Next function that we're going to have
2:48:34is validate parameters. So we want to
2:48:36validate the tool parameters, right? So
2:48:39we can do that. We'll have self
2:48:41parameters coming from the well
2:48:44arguments and then you have string comma
2:48:46any and this thing is going to return a
2:48:50list of string. Why is it returning a
2:48:53list of strings? Because we're going to
2:48:55return a list of validation errors if
2:48:58validation errors exist. Otherwise,
2:49:00we're going to return an empty list.
2:49:03That means there are no validation
2:49:05errors. It's valid. Now what this
2:49:07validate parameters is trying to do is
2:49:09that if we again go back to this diagram
2:49:13let's say the model gives us get weather
2:49:16paris and then it also gives us another
2:49:19integer. The model got confused about
2:49:22how to call the function. So it gives us
2:49:24invalid parameters.
2:49:27So before executing it I would just like
2:49:29to check if the valid if the parameters
2:49:31are correct in the first place or not.
2:49:34Right? That's a good way to go about it.
2:49:36Otherwise I'll execute it and something
2:49:38will just go wrong because in this case
2:49:41it tried to pass an integer when no
2:49:43integer is required after Paris. This
2:49:46there's no second argument to be passed.
2:49:49So those are the kind of parameter
2:49:51validations that we require and for us
2:49:53it's actually quite easy. The reason
2:49:55it's easy is because we're using
2:49:57pyantic. Pyantic by itself is a package
2:50:02helping us to do validation. So here
2:50:05first we can extract the schema. The
2:50:07schema is just equal to self dot schema.
2:50:10This property that we had and the only
2:50:12reason this is a in in a function type
2:50:15is because we want to raise a
2:50:17notimplemented error if no schema is
2:50:20present otherwise you could have written
2:50:21schema over here and that would be
2:50:23totally fine. Now that we have schema
2:50:25with us, I want to check if the schema's
2:50:28type is base model because if it's a
2:50:32base model, then we have a paidantic
2:50:35schema. As I said over here, the return
2:50:38type of the schema, the type of the
2:50:40schema can be base model or it can be a
2:50:43dictionary.
2:50:45The reason it can be a dictionary is
2:50:47because we have MCP tools. MCP tools
2:50:49will define the dictionary to us as it
2:50:51is. They're not going to give us the
2:50:53schema in a pyantic format. Pyantic is
2:50:56for us. Of course, a random MCP is not
2:51:00going to give us data in the format that
2:51:03we want. So here we are having a
2:51:06dictionary, but the base model is there
2:51:09whenever we trying to create our own MCP
2:51:12schema. But we have base model when
2:51:14we're trying to create our own tool
2:51:16because all of our tools parameters or
2:51:19schemas are going to have base models
2:51:22and we'll look at that in just a minute
2:51:24once we've completed this class. So this
2:51:26is for MCP dictionary is for MCP and
2:51:29base model is for our inbuilt tools. So
2:51:32first I'll just check that hey if the
2:51:34instance of schema is of type and if it
2:51:39is subclass of base model then we have a
2:51:43pyantic model. If there's a py pyantic
2:51:46model it will automatically do the
2:51:48validation. If we just do schema which
2:51:51is of the type of base model right. So
2:51:54we can just call this base model like
2:51:56that. So similar to this part where we
2:51:58could just create a base model like that
2:52:00and pass in any data. Now since schema
2:52:03is essentially a base model, we can just
2:52:05call schema like that. Pass in all of
2:52:08the data. So we'll have all the
2:52:10parameters passing in and that's it. Now
2:52:13this can result in an error in case
2:52:15there's a validation error. So we're
2:52:17going to have a try block. We call
2:52:19schema within this. And then we have
2:52:21except validation error.
2:52:24And then we have validation error as E.
2:52:26Now let's import validation error from
2:52:28pyantic. So again just to revise what we
2:52:31have done over here is just called base
2:52:33model like that. Whatever base model it
2:52:35is. Let's say it's going to be read file
2:52:39schema. So it's called like that with
2:52:41all of the parameters passed to it like
2:52:44what is the path of the file? What is
2:52:46the limit? What is the offset? And if
2:52:50this does not return any error or it
2:52:52does not raise any exception, it's good
2:52:54for us. We have not raised any
2:52:56validation error. Everything looks good.
2:52:59All the parameters are great. But if it
2:53:01raises validation error, then we have a
2:53:03problem. So we have to catch all of the
2:53:05errors and give it to the model, right?
2:53:08Because if we give it to the model, the
2:53:10model will understand and try to improve
2:53:12in the next step. So here we're going to
2:53:15have for error in e dot errors and we
2:53:19can catch all of those errors over here.
2:53:21And now I just want to format all of the
2:53:23error that we have. All right. So we are
2:53:25going to have for x in error dot get
2:53:29loc. And if loc is not present it's just
2:53:32empty. This will return to us x and we
2:53:36just want to convert that x into a
2:53:38string. Cool. So we'll just join
2:53:41everything. So we have a full stop dot
2:53:45join and yeah that's it. This is going
2:53:49to be our field. So essentially what we
2:53:51have tried to do is from the error
2:53:53object that we have we have tried to
2:53:55extract loc which stands for location
2:53:58and we're just trying to find out where
2:54:01all the loc what locations did this
2:54:04validation error occur in and we're just
2:54:07separating them by a full stop. After
2:54:09that, we're going to have a message
2:54:11which is equal to error.get message
2:54:13because we need to know what error
2:54:15message it is. And if that does not
2:54:18exist, we're going to have validation
2:54:19error. That's it. And now to errors,
2:54:23we're going to append this. So we have
2:54:25errors dotappend. And then we can just
2:54:29pass in that parameter.
2:54:31And then we pass in the field and then
2:54:34we pass in the message. So this
2:54:37parameter said that we had got this
2:54:40error message. Simple enough. After this
2:54:43for loop ends, we're just going to
2:54:44return a bunch of errors. Otherwise, you
2:54:47know, we just run into an exception as E
2:54:50and then we just return whatever error
2:54:52we got. We want to catch all of the edge
2:54:55cases, right? Worst case, we don't
2:54:57format any of the error. Just give the
2:54:59entire thing to the LLM and the LLM will
2:55:03just figure it out. So worst case we're
2:55:06just sending a string of the error
2:55:08message. Now if it's not an instance of
2:55:11base model that means we don't have a
2:55:13pyantic model we have a dictionary. If
2:55:16there's a dictionary basic validation
2:55:18can be done by open AAI API or the open
2:55:22AAI SDK that we are using. So we'll just
2:55:25return no errors. If there is any error
2:55:29it will be caught by the OpenAI package.
2:55:31So that's good for us. Again, just to
2:55:34remind this is for pyantic. In pyantic,
2:55:37we have to do it ourselves because it's
2:55:39a custom layer that we added ourselves,
2:55:41right? The only reason we're using
2:55:43pyantic is to help ourselves just be
2:55:46more explicit, more clear, and handle
2:55:48all of the validation. If it was a
2:55:50simple dictionary, OpenAI could just
2:55:53handle everything. So, yeah, that's
2:55:55about validate parameters. Another
2:55:57function that we're going to have is
2:56:00related to is mutating. So is this
2:56:04function that we're creating or the tool
2:56:07that we're creating modifying any state?
2:56:10For example, if your operation is a read
2:56:14operation, it's not going to mutate any
2:56:15state. It's not going to change
2:56:17anything. But if you have a write
2:56:20operation or an edit operation, it's
2:56:22going to change the state, right?
2:56:25whatever state your program is in. So
2:56:28here we're going to have parameters
2:56:30which is of the type of dictionary
2:56:31string comma any and here we're going to
2:56:34return a boolean value because mutating
2:56:37can either be true or false and it's
2:56:39only mutating when there's a certain
2:56:42tool kind right for example if I have
2:56:46read it's not going to mutate if it's
2:56:48right it's going to mutate so it really
2:56:50depends on the tool kind so I can just
2:56:52simplify this and just say that hey I
2:56:55want to return true if self do.ind the
2:56:59tool kind that we had over here. If it
2:57:02is either in toolkind dot write or it is
2:57:09toolkind dotshell
2:57:12or it is toolkind dot network because
2:57:16network can actually change our state or
2:57:19it is toolkind
2:57:21dotmemory.
2:57:23So if it's either one of these then we
2:57:25have a mutating state of course right
2:57:28you obviously understand shell can be
2:57:30mutating and non-mutating. So if you try
2:57:33to do a cat operation using shell it's
2:57:37non-mutating but if you're trying to run
2:57:39a web server using shell or you're
2:57:41trying to create a new file using shell
2:57:44those are all mutating. Then you have
2:57:47network network operations obviously can
2:57:49be mutating and memory you're literally
2:57:52appending to a file what the memory is
2:57:55going to be of your user right so if I
2:57:58just say hey my name is Ran it will be
2:58:00stored in memory now since it's stored
2:58:02in memory it's stored in an actual file
2:58:05on my system if it's stored in an actual
2:58:08file on my system it's mutating so
2:58:10that's one another thing that we're
2:58:12going to have is related to confirmation
2:58:16So whenever we execute a tool that is
2:58:19outside of our basic permissions, we
2:58:22might get a toolbox saying that hey this
2:58:25is what's going to happen with this tool
2:58:27call. Do you want to accept it or
2:58:30disallow it? So for that we're going to
2:58:32create something related to tool
2:58:34confirmation and that's also a function
2:58:36we're going to create. Async def get
2:58:39confirmation then we're going to get
2:58:41self then invocation which is of the
2:58:44type of tool invocation
2:58:47and here we're going to return either a
2:58:49tool invocation or a null value. So
2:58:53confirmation is not required when
2:58:56you know there's no mutating thing
2:58:58happening. So if you have read operation
2:59:01you don't need a confirmation you can
2:59:03just go ahead and read it. But if you're
2:59:06going through a mutating state and that
2:59:09mutating state is outside of your
2:59:12directory and you only have permission
2:59:14to read files in the directory, it will
2:59:16ask you to get a confirmation. Let me
2:59:18show you an example. So this is
2:59:20basically what the end state is kind of
2:59:22going to look like. So I can just run
2:59:25this particular command which is create
2:59:27a hello world python script outside of
2:59:29the folder I'm in. So I can just run it
2:59:32and and this is the output that we get.
2:59:35Approval required is put in over here.
2:59:38But the data that's been shown over here
2:59:40is related to confirmation. So if you
2:59:42disprove this request, it's gone. It
2:59:45doesn't do it. But well, if you approve
2:59:47of it, it will actually override the
2:59:49file that already exists. Hello world.
2:59:52py. Even when you do any other action,
2:59:54for example, a simple action. Let me
2:59:56just approve this. And it's unable to do
2:59:58it because I have less permissions. It's
3:00:00a very restrictive environment setup.
3:00:02But anyways, my point is if we try to
3:00:05create a Python file in this environment
3:00:07itself. So I'll just say create in this
3:00:10working directory
3:00:12and then hit enter.
3:00:16As you can see it creates and writes the
3:00:18file. So this particular thing it did
3:00:20not ask for my permission now because we
3:00:22are in the same current working
3:00:24directory right? So it doesn't ask for
3:00:27my permission it directly just writes it
3:00:29down. Of course, we can restrict it even
3:00:31more and we're going to add that through
3:00:33configuration system. But the point here
3:00:36is that this dialogue that you see is
3:00:38confirmation. It specifies the tool
3:00:41name. It specifies the parameters. It
3:00:44specifies the description. All of that.
3:00:46So here we're going to create the tool
3:00:48confirmation part. If you're just
3:00:50reading a file, we don't need any
3:00:52confirmation. So we'll have if not self
3:00:54dot is mutating and then we need to pass
3:00:58in the parameters right what are the
3:01:00parameters well it's just going to be
3:01:02invocation dotparameters because
3:01:04remember in invocation we had the
3:01:06parameter setup right and we needed that
3:01:10parameters here as well. Now obviously
3:01:12parameters is not used in this mutating
3:01:14function and it will probably not be
3:01:16used in any mutating function but it's
3:01:19good to keep it over here. If you want
3:01:21you can remove it but just in case we
3:01:23need it for something you know maybe the
3:01:26answer of is mutating is not as simple
3:01:28as like either returning true or
3:01:31returning false. Maybe you want to do
3:01:32some calculations based on the
3:01:34parameters and then deciding we're going
3:01:36the easy route but you can obviously go
3:01:38much deeper in this is mutating part. So
3:01:42if is mutating is false then what I want
3:01:45to do is return none because we don't
3:01:48want any confirmation.
3:01:50However, if is mutating is true then we
3:01:53want to return tool con invocation or
3:01:57tool confirmation sorry. So we'll have
3:01:59to create a new class called tool
3:02:01confirmation that stores all of the
3:02:03things that we talked about. So at the
3:02:06top again we're going to create another
3:02:07data class and this time it's going to
3:02:10be tool confirmation.
3:02:13Then we're going to have tool name which
3:02:16is of the type of string. Then we have
3:02:19parameters which is of the type of
3:02:21dictionary. The key can be string. The
3:02:23value can be any. After that we can have
3:02:26a description of the tool and that will
3:02:30be a string. Now there's other things as
3:02:32well that's related to display
3:02:34information right over here we noticed
3:02:37that we have the tool name we also have
3:02:40the diff displaying over here right what
3:02:43things needed to be added what needs to
3:02:45be removed so that's the diff then we
3:02:49also have command if that's present then
3:02:52we have all the affected parts for
3:02:54example this particular line then if the
3:02:57tool is dangerous or not and if there's
3:03:00warning attached to it or not. All of
3:03:02that is also possible and we'll add it
3:03:04as and when we need it. But as of now,
3:03:06let's just focus on tool name which is
3:03:09equal to self.name. We already have
3:03:11access to all of these things over here.
3:03:14Then we have parameters which is just
3:03:17equal to invocation.parameters.
3:03:20And then we have description which is
3:03:22equal to well we can just say execute
3:03:26self.name.
3:03:28we want to execute this particular tool.
3:03:30So that's the description. Now one last
3:03:32function I would like to create for now
3:03:34is to openAI schema. So right now we
3:03:38have the schema in pyantic format.
3:03:40Right? I just want to take that pyantic
3:03:43format and convert it into a dictionary
3:03:46so that we can pass the tool parameters
3:03:50to the API. So if you go to llmclient py
3:03:54here you notice that we're passing in
3:03:55model messages and stream tools will
3:03:58also be passed as a keyword argument and
3:04:01we needed to convert it into JSON format
3:04:04right into a dictionary so that it can
3:04:05be sent through the API but right now we
3:04:08have everything in pyantic format. So
3:04:12we're just going to create two open AAI
3:04:15schema which is going to return a
3:04:17dictionary where the key is a string
3:04:20value can be any
3:04:22and then you just have a schema which is
3:04:25equal to cell schema. After that you're
3:04:28going to check if it's a pyantic model.
3:04:30To check if it's a pyantic model or not
3:04:32we can just copy the same line paste it
3:04:35down here and let me just go back.
3:04:39Awesome. After that we essentially want
3:04:43to convert to JSON right and pyantic has
3:04:47support for that. So we can just call
3:04:49model JSON schema and we're going to
3:04:52import that. We're going to pass in
3:04:53schema. The mode is going to be
3:04:55serialization
3:04:57I think. So let's try that. And we also
3:05:00need to import from pyantic. So we'll
3:05:03have from pyantic dot JSON schema we'll
3:05:07import model JSON schema. Great. So this
3:05:12function is called it will convert it
3:05:14into a dictionary with the type of
3:05:16string, any and now we can store that in
3:05:19a variable. After that we can just
3:05:22return a dictionary with all of our
3:05:24arguments. So the way OpenAI SDK wants
3:05:28it to be structured is something like
3:05:29this. We pass in the name where we pass
3:05:32in name like that. Then we have a
3:05:35description where we pass in the
3:05:37description like that. Then we have
3:05:40parameters defined which is of the type
3:05:43of object. Then we have properties and
3:05:47properties is simply just JSON
3:05:49schema.get
3:05:51and pass in the properties an empty
3:05:54dictionary if it's not available. Then
3:05:56we have required where we have JSON
3:05:59schema.get get required and if it's not
3:06:04we just pass in empty list parameters is
3:06:07of the type of object then we have a
3:06:09bunch of properties within it which can
3:06:11be dictionary and then you have required
3:06:14if the property is required and what
3:06:17properties are required that's what this
3:06:20is this is just the way OpenAI SDK
3:06:23expects us to pass in the tool schema
3:06:27all right and if you search for it
3:06:29you'll be able to see Right. So we have
3:06:31tools defined over here. That's what
3:06:32we're going to do in our LLM client. So
3:06:35over here when we pass in tools list,
3:06:37we're going to convert each tool that we
3:06:40have into a schema. And then there's the
3:06:43name that we have. Then we have a
3:06:45description. Then we have parameters.
3:06:47Then we have the type of object. Then
3:06:49what properties do we have? Location is
3:06:52one of the properties in get current
3:06:53weather. Then you have type string
3:06:56description. Then there's another unit.
3:06:58if it's a Celsius or a Fahrenheit and
3:07:01what is required is just location. So
3:07:04unit can be ignored. That's essentially
3:07:07what the request is and that's how we
3:07:09have structured it as well. Now what if
3:07:12the schema is already a dictionary?
3:07:15Totally possible if it's an MCP. So we
3:07:18just check if is instance schema
3:07:20dictionary. In that case we basically
3:07:24want the same thing. So we'll have
3:07:25result is equal to we pass in the name
3:07:28the name is self do.name name. Then we
3:07:31have a description. The description is
3:07:33self.escription.
3:07:36Okay. And then we're going to check one
3:07:39thing because it is a problem that might
3:07:42exist in MCPS. So we want to check if
3:07:45parameters is present in the schema
3:07:47dictionary. If it is, we'll just set
3:07:49result at parameters equal to schema at
3:07:55parameters. Otherwise, what we're going
3:07:57to do is result at parameters is equal
3:08:01to schema itself. Cool. So, if the MCP
3:08:06server has defined all of the parameters
3:08:09as it is if it's a dictionary that's
3:08:11present where parameters argument is
3:08:14present, we'll use that. Otherwise,
3:08:16we'll have result at parameters equal to
3:08:18the schema as a whole. So, that is our
3:08:21entire parameters list. and then we can
3:08:24just return the result.
3:08:28Now if it's none of them, if it's not a
3:08:30pyantic model, if it's not a dictionary,
3:08:33then we just raise a value error.
3:08:35Something went wrong. So we just type an
3:08:38invalid schema type for tool and then we
3:08:42can pass in cell.name. Let's make it an
3:08:44string. And then we also need to give
3:08:46the type so that the user has a much
3:08:49better understanding of what they did
3:08:51wrong. So we can just have schema
3:08:53because most likely the user was trying
3:08:55to pass in an MCP but some formatting or
3:08:58something was just wrong. So we give
3:09:00them as much information as we can.
3:09:02Okay. So these are all of the things
3:09:04that we need for now. We're going to
3:09:06create a new folder called built-in.
3:09:09These are all the built-in tools and
3:09:11obviously the first file that we are
3:09:13interested in or the first tool we are
3:09:14interested in is read file. py. Now here
3:09:18we can first define the read file
3:09:21parameters. Right? I told you the
3:09:24parameters are going to be a pyantic
3:09:26base model. So from pyantic we can
3:09:28import a base model. Then we're going to
3:09:32create our read file parameters that is
3:09:35going to extend base model. Now what are
3:09:38the parameters that we require for read
3:09:40file? The first one is pretty simple.
3:09:41Path which is of the type of string and
3:09:44it should be a field. What is field?
3:09:46field is just something that's coming
3:09:48from pyantic. This time we're not taking
3:09:51it from data classes. We're using it
3:09:53from pyantic because we are in a pyantic
3:09:56based model. So data classes are used
3:09:58for everything other than parameters.
3:10:01All right, parameters for tool calling
3:10:04because they do all the validation
3:10:05stuff. Now in the field we can have dot
3:10:08dot dot because we want to ignore
3:10:10everything. Everything will be default
3:10:12value except the one which is
3:10:14description. What is the description of
3:10:17the path? What is this path parameter
3:10:19supposed to do? Well, this is just the
3:10:22path to the file to read and then we can
3:10:26just say relative to working directory,
3:10:29right? Or we can just say that it can be
3:10:32an absolute path as well. So yeah,
3:10:35related to working directory or
3:10:38absolute. So if you're in this
3:10:40particular directory which is AI - agent
3:10:43you can just do cd dot dot and you'll go
3:10:46back to the desktop. Okay that is
3:10:49relative
3:10:51rel relative to working directory and
3:10:54let me write this as relative to working
3:10:55directory or absolute path can be /
3:10:58user/reand
3:11:00/estop/
3:11:02ai-en agent. So that is absolute. After
3:11:05that we have offset which is of the type
3:11:07of integer which is equal to field.
3:11:09Everything here is not going to be
3:11:11default. The default value is going to
3:11:13be one and offset. Offset just refers to
3:11:16from where do I want to start reading
3:11:18the file. Well I want to start from line
3:11:21number one if nothing is mentioned.
3:11:23Right? So offset default value is one
3:11:26and the value should be greater than
3:11:28equal to 1. You cannot pass in minus
3:11:31one. You cannot pass in zero. What is
3:11:33line number zero? That does not exist.
3:11:35Everything starts from line number one.
3:11:38Cool. And then there is a description
3:11:41which is line number to start
3:11:44reading from and then we can say one
3:11:48based because well we are programmers
3:11:51right? We have a tendency of using zero.
3:11:54And then we can also say defaults to
3:11:57one. Great. After that we have a last
3:11:59one which is limit an integer or it can
3:12:02also be a null value. If you don't
3:12:04specify it totally fine and then we can
3:12:07have a field the default value is none.
3:12:09And limit is basically that hey I'm
3:12:12trying to read a file I start from line
3:12:15number one. How many lines should I go
3:12:17till? If the limit is 100 it will start
3:12:19from line number one and it will go till
3:12:21100. If you start from line number 10
3:12:23and your limit is 50, you go from 10 to
3:12:2660 because well your limit is 50, right?
3:12:30The maximum number of lines you can
3:12:32read. So you can have the greater than
3:12:34equal to one because if the limit is
3:12:36zero that does not make sense and the
3:12:38description is going to be maximum
3:12:40number of lines to read if not specified
3:12:45reads entire file. Awesome. Now we can
3:12:50go ahead and create a new class called
3:12:53read file tool and this read file tool
3:12:57will extend the tool that we created in
3:13:00the base. py. So let's import it from
3:13:02tools.base import tool. And now we have
3:13:07to implement some of the things. The
3:13:09first one is the name. What is the name
3:13:11of the tool? It's called read file. What
3:13:15is the description? And for description
3:13:17I'm just going to copy paste it. And it
3:13:19looks like this. So the description is
3:13:22we read the content of a text file. By
3:13:24the way, description is very important
3:13:26because it's given to an LLM to decide
3:13:28which tool to call. We say that we want
3:13:30to read the content of a text file. It
3:13:33returns the file content with line
3:13:34numbers. For large files, use offset
3:13:38because we are instructing the LLM,
3:13:39right? How to use this tool? If it
3:13:42decides to use this tool, how should it
3:13:44use it? So description has two task.
3:13:46First, should it use the tool? And
3:13:49second, if it uses the tool, how should
3:13:51it use it? Any instructions? So, it says
3:13:54for large files, you use offset and
3:13:56limit to read specific portions. And it
3:13:58cannot read binary files like images,
3:14:01executables, probably PDFs as well, etc.
3:14:04So, this is a very important and great
3:14:06line because let's say you say that,
3:14:09hey, please read XYZ.png.
3:14:13it will not read it because it will
3:14:15understand that if it's a PNG it's a
3:14:17image if it's an image I cannot read the
3:14:19binary file so that's great after that
3:14:22we'll have a kind which is equal to
3:14:24toolkind dot read and we have to import
3:14:27toolkind so let's import it from tools
3:14:30dobbase
3:14:32and then we can also define the schema
3:14:35now we don't have to create a function
3:14:36for it was a property right so I can
3:14:39just set schema to read file parameters
3:14:42Right? Schema is a property which is
3:14:45read file parameters. This entire base
3:14:47model and now we satisfied the condition
3:14:49of schema being a base model. Amazing.
3:14:53Now we have to go ahead and define the
3:14:55execute function. The main function that
3:14:58needs to be in every tool. So we have
3:15:01async def execute then we have a self
3:15:04then we have invocation which is of the
3:15:06type of tool invocation and we have to
3:15:09import that from tools.base as well.
3:15:12And this will return a tool result. So
3:15:16we have to import from tools.base tool
3:15:19result. And now first let's just extract
3:15:22all the parameters, right? So we have
3:15:24parameters is equal to read file
3:15:27parameters. And how do we what how do we
3:15:31pass in all of the data? Well, it's all
3:15:33stored in invocation.parameters.
3:15:36And now we can just deconstruct it. So
3:15:39that if path is specified, it will
3:15:42directly be given to this path field. If
3:15:45offset is specified, it will be given to
3:15:47this field. So we have the parameters
3:15:50with us. Now we also need a path because
3:15:54obviously a path will be specified over
3:15:56here. Right? So I want to check if that
3:15:59file exists. And first of all I told
3:16:01that the path over here can be relative
3:16:04to working directory. So if it's
3:16:06relative to a working directory, I want
3:16:08to resolve the path. I want to know what
3:16:11is the true actual path. So that's
3:16:14something I'll have to look into. And if
3:16:16it's an absolute path, that's easy for
3:16:18me. So let me go ahead and create in
3:16:21utils something known as path. py. This
3:16:23will contain all the utility function
3:16:25related to path. So let's create a
3:16:28function called resolve path. So here
3:16:30we're going to take in two things. The
3:16:31first one is a base which is going to be
3:16:33a string or a path and then we're going
3:16:37to take from pathlib import path and
3:16:39then there's going to be an actual path
3:16:41the path we want to resolve and that
3:16:43will be a string or a path as well. It
3:16:45can be anything right and let me just
3:16:48frame this nicely. Awesome. So the base
3:16:51is the base directory the current
3:16:53directory we are working in. And then
3:16:55the path to resolve is going to be
3:16:57whatever path the tool threw at us. And
3:17:01now we'll just create path is equal to
3:17:03path and then we pass in the path. So if
3:17:06it is a string it will just convert it
3:17:08into a path. And then we just check that
3:17:11hey if path is absolute that means we
3:17:15don't have anything to worry about. We
3:17:16can just say path.resolve and send it
3:17:19across. It's going to be as simple as it
3:17:21is. Otherwise we have a relative path
3:17:25and in relative path we'll have to
3:17:27resolve it. So we just call like that.
3:17:29So what we have done over here is path
3:17:31of base. So we have created a path of
3:17:34base similar to the path we created for
3:17:38this one and then we are resolving it.
3:17:40So we are making the path absolute and
3:17:43since we've created it absolute we are
3:17:46doing forward/path to it so that we just
3:17:49append whatever there is. So for example
3:17:52let's say the current working directory
3:17:54is user/an
3:17:56/estop. All right. And let's say it's ah
3:17:59in this current folder itself. And now
3:18:02we have the path added that hey we want
3:18:05to read this base. py file. So it will
3:18:07just say that the path is tools
3:18:10forward/base.
3:18:13py. Correct. So what I've done is
3:18:15concatenated both of them and added
3:18:17users reanard desktop AI agent tools.
3:18:21py and we have the entire path with us.
3:18:23Now what if the user wants to get out of
3:18:26this AI agent and make some changes in
3:18:28desktop? So they specify something like
3:18:32dot dot / that's how you get out of the
3:18:35current folder and then maybe you go
3:18:37into cloud code folder. So we just
3:18:40concatenate that. So this is how the
3:18:42path looks like. You go into AI agent
3:18:44then you go back two steps behind and
3:18:46then you go into cloud code or sorry you
3:18:48go one step behind and then you go into
3:18:50cloud code. So that's how this entire
3:18:52thing works. Pretty simple. And now we
3:18:54can call this resolve path function. So
3:18:57we have path is equal to resolve path.
3:19:00Let's import that from utils.path. And
3:19:03then we pass in the current working
3:19:04directory which is invocation dot
3:19:06current working directory. And then we
3:19:09pass in the path that the model the LLM
3:19:13gave to us which is params.path
3:19:16and I cannot spell it properly. So
3:19:18params.path like that. Great. So now
3:19:20that we have the path with us, I first
3:19:22want to check if this file even exists.
3:19:25So I'll just check if not path.exist
3:19:27because it's totally possible that the
3:19:29llm hallucinated. [snorts] So we will
3:19:32just return an error. Right? So for an
3:19:36error, I'm just going to create a method
3:19:39within tool result. So I can go back to
3:19:42base. py and within tool result similar
3:19:45to the other streaming events and all of
3:19:47that we're going to create class method
3:19:50where you're going to have error result.
3:19:53So if you have an error result, you get
3:19:55a class. Then you get the error string.
3:19:58What is the actual string? What is the
3:20:00output? And if you've not specified any,
3:20:02it's going to be an empty string. After
3:20:04that, we'll just return the class. We
3:20:06can say success is equal to false
3:20:10because well, it was an error, right? If
3:20:12there's any output, you can attach it.
3:20:15So there we go. After that, you have
3:20:18error which is equal to the error. So
3:20:21now I can go back to this read file and
3:20:23just say return tool result dot error
3:20:27result. So we tried to execute the
3:20:30command but whatever the llm gave us was
3:20:33kind of hallucination
3:20:35or it did not understand the folder
3:20:37structure really well. That's why we are
3:20:39returning an error and that will be
3:20:41given to the LLM. So you can write a
3:20:43descriptive message here saying file not
3:20:45found and it was not found at that
3:20:49particular path.
3:20:50we are passing in this particular path.
3:20:52All right, the resolved path after all
3:20:54of our operations so that the LLM knows
3:20:56that hey we tried this particular thing
3:20:59whatever you gave us we're not just
3:21:02spitting out that what we tried and what
3:21:04didn't work for us that's what we're
3:21:06giving to the LLM
3:21:08now another thing is what if it's not a
3:21:10file it's a directory so I would like to
3:21:13tell that to the LLM as well so I'll
3:21:16just return this tool result do error
3:21:18result and I can say path is not a file
3:21:23and then we pass in the path again.
3:21:26Great.
3:21:27Now if it the path exists and the path
3:21:30is a file that means we want to read it
3:21:33but we'll not directly read it. There's
3:21:35some other things I would like to do
3:21:37because I don't want my LLM to get a lot
3:21:40of context, right? Because if it's read
3:21:43file, it can be a file that has
3:21:45thousands of lines. That means it has
3:21:48hundreds or thousands of tokens. I do
3:21:50not want to contaminate my LLM by giving
3:21:54it so much data. So I'll handle that. So
3:21:56the very first thing that I'm going to
3:21:58do is if the file size exceeds by a big
3:22:02margin then we're just going to return
3:22:04the error to the user that hey the file
3:22:06is too large and that's what cursor
3:22:08does. I think cursor stops at about 20
3:22:11megabytes or 10 megabytes. If your file
3:22:13is more than 10 megabytes it will just
3:22:16return an error. So it just can't read
3:22:19it. And that's what we want to do as
3:22:21well. So we'll have file size is equal
3:22:24to path dot stat whatever path we had
3:22:28it's stat and to get the file size we
3:22:31can just do st size like that or like
3:22:35this it's not a function it's just a
3:22:38property an integer and then I can just
3:22:40check that hey if the file size is
3:22:43greater than the max cell size and we
3:22:46can just define a constant at the top
3:22:48max file size which is equal to 1,024
3:22:55into 1,024
3:22:57into 10. So we have bytes into kilobytes
3:23:00into 10 because well we want it to be in
3:23:03bytes and 10 megabytes is just this many
3:23:07bytes. So we just do self domax file
3:23:11size if it is greater than we just
3:23:14return the tool result dot error message
3:23:17that hey this is too big sorry I can't
3:23:20read it for you. So we can say file to
3:23:23large. Then we can say what is the
3:23:26file's actual size in megabytes. So as
3:23:29of now this size is in bytes. This size
3:23:32is in bytes. And that's why we're going
3:23:34to have file size divided by 1,24
3:23:40into 1,024.
3:23:42And I want this to be limited to one
3:23:46floating point digit. So we have.1F
3:23:49and then you have megabyte written down.
3:23:53Cool. So the file is too large and the
3:23:56file is this these many megabytes long.
3:23:58And maybe we can also add another
3:24:00message which is what is the maximum
3:24:02length. So we can have maximum
3:24:06is and then we have self dot max file
3:24:10size. Let me put an string here. Divided
3:24:12by 1,024 into 124. And then we'll again
3:24:17have 1 F like that. Or actually, let's
3:24:21keep it limited to 0 F because it's 10
3:24:24MB. And then we'll just put MP. The
3:24:27reason we're not hard coding it down to
3:24:2810 MB is because we're defining the max
3:24:32file size over here. So just in case in
3:24:34the future we decide we want 20
3:24:35megabytes, we just change it over here
3:24:37and it will be reflected everywhere in
3:24:39our code. Now if the file size is proper
3:24:43then the other thing that I want to do
3:24:45is check if it's a binary file. If it's
3:24:48a binary file I don't want to read it.
3:24:51I'll again raise an error. So if it is a
3:24:53binary file and we will have to create a
3:24:56function to detect a binary file and it
3:24:58will be used everywhere in almost many
3:25:01tools. So we can pass in the path over
3:25:04here and now we'll have to create it in
3:25:06path. py. So we have diff is binary file
3:25:10and we get the path over here which is
3:25:12of the type of string or path itself and
3:25:15it returns a boolean value then we try
3:25:19to open it. So we'll have try and except
3:25:23since it's returning a boolean value
3:25:24we'll just return false. if we run into
3:25:27errors related to OS error or we run
3:25:30into IO error input output error or if
3:25:34you want you can just pass an exception
3:25:35here and just return false but anyways
3:25:38we'll just try to open the file
3:25:41then we'll open this particular path
3:25:44then we'll have RB which stands for read
3:25:47binary as F and then we'll just do chunk
3:25:51is equal to F dot read then we'll read
3:25:54let's say 8192 bytes and then we'll just
3:25:58try to identify it's a binary file or
3:26:00not. So we'll just check if this back
3:26:04slashex0000
3:26:06exists in this chunk that means it's a
3:26:09binary file. We'll get to know if it's a
3:26:12binary file or not. So what we've done
3:26:13is essentially we are reading the first
3:26:168 kilob and we're checking for null
3:26:19bytes in that. If null bytes are present
3:26:22that means we have a binary file. You
3:26:24can obviously make this more complicated
3:26:26and thorough. But yeah, my my work is
3:26:28done here. I'm just going to import it
3:26:30from utils. So let me just return the
3:26:33same error message tool result dot error
3:26:36result. And then we can just say cannot
3:26:39read binary file. So I have cannot read
3:26:43binary
3:26:44file. Let me just write it down. And
3:26:47then we'll pass in the name of the file.
3:26:50So we have path.name. And then we pass
3:26:53in the size of this file just to give
3:26:57the llm some context. So let's try to
3:26:59extract the size. It would just be
3:27:01easier. So we have s string and I'll
3:27:04actually just copy paste it because it's
3:27:06too annoying to write it. I have created
3:27:09one variable which is file size
3:27:11megabytes which is equal to file size
3:27:13and we divide it by 10,024 into 124
3:27:16similar to what we did over here just
3:27:19the file size in MB. And then we have
3:27:21size string which goes up till two
3:27:23floating points. And we say we use that
3:27:26file size in megabytes if it is greater
3:27:29than equal to one. Otherwise we just
3:27:31have file size in bytes. Cool. That
3:27:34means if it's greater than 1 mgabyte it
3:27:38will write down megabytes otherwise it
3:27:39will be writing down in bytes. And now I
3:27:43can just say cannot read file with this
3:27:46size string. And then I can also say
3:27:50this tool only reads text files.
3:27:54Awesome. That's pretty much it. Now, if
3:27:57it's not a binary file, it's a text
3:27:59file, then we have all of the other
3:28:02things to do like reading the actual
3:28:05file finally. So, I'll just put in a try
3:28:08and accept because reading the file can
3:28:11be a bit difficult. So, we have content
3:28:13is equal to path dot read text. And now
3:28:17I'll pass in the encoding which is UTF8.
3:28:21So I'll have encoding is equal to UTF8.
3:28:25That covers all the English characters.
3:28:27So we don't worry about that. Then we
3:28:29have except unic code decode error just
3:28:32in case there is this exception. We try
3:28:35to do path dot read text and try the
3:28:38encoding of Latin one. So let's say this
3:28:42encoding fails. We try Latin one. That's
3:28:45what we're trying to do. nothing much.
3:28:47And now since we have all of the
3:28:49content, I'm just trying to split them
3:28:51in lines so that I can filter by these
3:28:54other parameters like offset and limit.
3:28:56Right? So I'll just convert it into an
3:28:59array where each line has
3:29:03a new string in this array. So
3:29:06content.split lines will do that. And
3:29:09then we can also get uh an understanding
3:29:11of how many lines there are. So length
3:29:13of lines.
3:29:15Now if the total number of lines is
3:29:18equal to zero in that case we are
3:29:21dealing with an empty file. We'll just
3:29:22say tool result dot success result and
3:29:27we'll have to create that thing because
3:29:30we've created everything related to
3:29:32error. But if it's a success we also
3:29:34want that. So we'll have a class method
3:29:37related to success result. So let's go
3:29:41ahead and have success result. The
3:29:44success is true. Here
3:29:46the output is a string. By default, it's
3:29:49an empty string. But let's just say you
3:29:52have to specify it. And this time we can
3:29:54say the error is kind of null.
3:29:59The output is output and the success is
3:30:01true. That's great. That looks good to
3:30:04me. I'll just pass in success result
3:30:08here. And let me just copy that
3:30:11correctly. And now I need to pass in the
3:30:13output. The output here is going to be
3:30:16file is empty. That's it. Now the reason
3:30:18we're doing success result is because
3:30:21well if the file is empty and we just
3:30:24return an empty string from here, the
3:30:26LLM might get confused that hey this
3:30:29tool only returned an empty string. What
3:30:31does that mean? Did I did I do something
3:30:32wrong? And the LLM will spend tokens
3:30:36over there. We don't want it to spend
3:30:38tokens over there. That's why we are
3:30:40telling over here that hey file is empty
3:30:42don't worry about it but I would also
3:30:44like to send some metadata like we have
3:30:47over here the metadata is the number of
3:30:50lines but we are not accepting that from
3:30:53success result so what I'll do is just
3:30:56have keyword arguments here which can be
3:30:59of the type of any and now I can pass in
3:31:02all of the keyword arguments that are
3:31:04required
3:31:06I'm not explicitly saying that metadata
3:31:08should be passed in. I'm just saying you
3:31:10can pass in anything else and I'll
3:31:12attach it to this tool result class if
3:31:14it accepts that. And now I can just pass
3:31:16in the meta data which is over here.
3:31:19Meta data is equal to and then we pass
3:31:23in the lines as zero. There are zero
3:31:26lines in this entire file. Now let's try
3:31:30to apply the offset. So first we'll have
3:31:33the start index. Where are we starting
3:31:35from? Well, we want to start from zero
3:31:38or offset minus one. Remember, this is
3:31:40an index, not a position. That's why we
3:31:43have zero over here. And we're just
3:31:45trying to take whatever value is
3:31:47present. Maybe the user or the LLM
3:31:50specifies that the offset we need to
3:31:52start from is going to be three. So 3
3:31:56minus 1 is 2. That is the index. Because
3:31:59remember we have lines as an array,
3:32:02right? Since lines is an array, it is
3:32:04zero indexed. But our entire system
3:32:07works on one based system. That's why
3:32:11what we're trying to do is offset minus
3:32:13one. If the user or if the LLM says that
3:32:15I want to read line number three or we
3:32:17want to start from line number three, we
3:32:19will do 3 - 1 2. That will be the index
3:32:22because the actual position is still
3:32:24three but the array starts from zero. So
3:32:28when we try to do lines at start index,
3:32:31we have a good indexing going on. We are
3:32:34actually referring to the third position
3:32:36but in the array is the second index.
3:32:39And obviously offset is not present, we
3:32:41have to do params dot offset. So that is
3:32:45the start index. Now if there's any
3:32:47limit applied, we have to get the end
3:32:49index as well. So if limit is not none
3:32:52and we have to get the limit which is
3:32:54just params.limmit.
3:32:56So if params do.limmit is not none we'll
3:32:59have end index is equal to and then we
3:33:01want the minimum value right. So the
3:33:04minimum value is just going to be start
3:33:06start index plus the limit which is
3:33:10parents dot limit comma total number of
3:33:14lines. So we are trying to take the
3:33:16minimum value of start index which is
3:33:18let's say zero plus limit is 100 and the
3:33:21total number of lines are th00and. So it
3:33:23will just take 100 as the end index. But
3:33:26what if the limit is told to be th00and
3:33:29and the total number of lines are just
3:33:32100. So we just take 100 because our end
3:33:36index should always be less than total
3:33:38lines. Right? Otherwise you know the
3:33:41parents do.
3:33:42And in that case the end index is
3:33:45definitely going to be total lines
3:33:47because that's the end part, right? And
3:33:49now I want to extract all of the lines
3:33:51which will be well something like this.
3:33:54Selected lines is equal to and then we
3:33:57go from lines from start index to end
3:34:01index. Awesome. And that's why we did
3:34:04not do minus one over here, right?
3:34:06Because index is also going to have
3:34:08minus one, right? So both of them should
3:34:11have minus1. But the thing is if you do
3:34:14minus one and then you do end index it
3:34:17will not count end index minus one. So
3:34:19it will not count the first last
3:34:21character that we have and that's not
3:34:23acceptable to us. That's why we are
3:34:25leaving it at this so that it goes until
3:34:27the very last character. And now I want
3:34:30to add line numbers to this because I
3:34:32told that the read file is going to give
3:34:34out line numbers and that's helpful to
3:34:36the LLM as well. So that you know after
3:34:39reading the file if it wants to write or
3:34:41edit it knows what line numbers to
3:34:43change. So we'll have formatted lines is
3:34:46equal to an empty list and then we'll
3:34:48have for i, line in en in enumerate. We
3:34:51go over the selected lines array and we
3:34:54start from the start index + 1 and then
3:34:58we just do formatted
3:35:00lines dot append. Let me call append.
3:35:04And now I need to pass in the line
3:35:08number. So we'll have I up until six
3:35:12characters and then I'll put in this
3:35:16particular or symbol and then I have
3:35:19line. So it gives out the index and then
3:35:23it gives out the line. It only spits out
3:35:25six characters because obviously if you
3:35:28go more than six characters, I've
3:35:30probably already stopped it. It's not
3:35:33allowed. our LLM cannot handle that much
3:35:36data. And now the output is just going
3:35:38to be back slashin. Let me do back
3:35:41slashin dot join. And then we format all
3:35:45of the lines. So each line goes on a new
3:35:48line and it's in a string format. Now,
3:35:50now we also want to take a look at
3:35:52whether we should truncate the output or
3:35:55not. So what we can do is count the
3:35:57tokens and we can use the count token
3:36:00function that we've created already from
3:36:03utils.ext. text import count tokens and
3:36:06now I can check that if that token count
3:36:08is greater than the max number of output
3:36:11tokens. So I'll create it max output
3:36:14tokens which is equal to and then maybe
3:36:18the max number of output tokens is
3:36:2025,000 from a read file. You can
3:36:23obviously update it based on your user's
3:36:25preferences or on your preferences if
3:36:27you're using it for yourself. And then
3:36:29we can just set output is equal to and
3:36:32now I have to truncate the text. How
3:36:34will that truncation work? Well, there
3:36:36are multiple types of truncation. One is
3:36:39a truncation that happens at the end of
3:36:41the string. Then there's a truncation
3:36:42that happens in the middle of the string
3:36:44or the one that happens at the start of
3:36:47the string. So we'll have to define
3:36:49that. But first we'll have to create the
3:36:52function. And truncation is one of those
3:36:54features which will be used quite a
3:36:57quite a bit. So I'm going to create
3:36:59within text py a function. Let's call
3:37:03this truncate text.
3:37:07Let me just name this properly. We get a
3:37:10text which is of the type of string.
3:37:12Then we have max tokens which is an
3:37:14integer
3:37:16and that's it. Oh, we might also get a
3:37:19suffix by the way. So suffix is a string
3:37:22which is equal to let's just say back
3:37:25slash n dot dot dot truncated. So this
3:37:29suffix is essentially like I truncated
3:37:31the text after that I'm just telling the
3:37:33model that hey I truncated the text
3:37:36down. So yeah that's what we're doing.
3:37:38Now again what we're going to do here is
3:37:40current tokens is equal to count tokens
3:37:43and we'll pass in the text. Let's also
3:37:46pass in the model.
3:37:48And actually, we don't have the model,
3:37:50but in some cases, we might have a
3:37:52model. So, yeah, let's take that in. And
3:37:55let me just format everything. Cool.
3:37:58Let's pass in the model here. These are
3:38:00the current tokens. If the current
3:38:02tokens is less than or equal to the max
3:38:06number of tokens, in that case, I'll
3:38:08just return the text. Otherwise, I have
3:38:11to count the suffix tokens. what is the
3:38:14count of suffix tokens which is just
3:38:17suffix and then we'll pass in the model
3:38:19name again and then we have to target
3:38:23how many tokens we want to add over
3:38:24here. So the target tokens is just
3:38:28target tokens equal to max tokens minus
3:38:33the suffix tokens.
3:38:36So how many max number of tokens can we
3:38:38have in this minus what are the suffix
3:38:41tokens and now based on this target
3:38:43tokens we're going to well truncate
3:38:46right and actually I don't want to spend
3:38:48much time on this truncate text logic
3:38:51because it's quite simple so what I'm
3:38:53going to do is just paste out a bunch of
3:38:54logic if you want you can copy it from
3:38:56the repository mentioned below otherwise
3:39:00I'll go through this with you line by
3:39:02line so the very first thing that we
3:39:04have is if target tokens is less than
3:39:07equal to zero. We're just returning the
3:39:09suffix
3:39:10that hey the target token is less than
3:39:13whatever you're targeting for is already
3:39:15less than zero. You are within the
3:39:16reach. So you don't have to truncate.
3:39:19We return the suffix as it is. Then we
3:39:22have another property which is preserve
3:39:23lines. So if preserve lines is true that
3:39:26means we want to truncate by line.
3:39:31If preserve lines is false, in that case
3:39:34we want to truncate by characters. So we
3:39:37will truncate by lines. We'll pass in
3:39:39the text target token suffix and the
3:39:42model. And let me remove this
3:39:44unnecessary dock string. So lines is
3:39:46equal to text.split by a new line. So
3:39:49we're just getting the lines again. Then
3:39:52we have result lines. Current tokens is
3:39:54equal to zero. Then for every line we're
3:39:56just counting the number of tokens in
3:39:58it. And if current tokens that we have
3:40:00over here plus the line tokens, whatever
3:40:03tokens exist in one line is greater than
3:40:05the target tokens, we just break out of
3:40:07this loop. Otherwise, we append to this
3:40:09result lines and then have current
3:40:11tokens plus equal line tokens. So as
3:40:15long as we have not hit the target
3:40:16tokens limit or we have not exceeded it,
3:40:19we'll keep adding to the current tokens.
3:40:21As soon as we hit the limit, we break
3:40:24out and then we check if not result
3:40:26lines. If result lines is an empty list.
3:40:29If it is, that means it did not work
3:40:31properly. Maybe everything was mentioned
3:40:34in one line. So we cannot truncate much.
3:40:38And that's why we'll fall back to the
3:40:40character truncation.
3:40:42And then we call this particular
3:40:43function which is truncate by
3:40:45characters. And to truncate by
3:40:48characters, we use a simple binary
3:40:49search. We go from low to high. And
3:40:52we're just checking in the middle how
3:40:55many characters we have. So one to
3:40:57truncate by characters the one approach
3:40:59you could have taken is linear search.
3:41:01So you go through every character and
3:41:03truncate them or what you could have
3:41:05done is go until the target tokens
3:41:10number and attach suffix to it. So this
3:41:13is the log of N approach because linear
3:41:16search is O of N and this is O of login.
3:41:19We are just looking in one side
3:41:22continuously and shrinking the search
3:41:24space by half every single time. So yeah
3:41:27that's it. Now we can just copy this
3:41:30truncate text. Also by the way if
3:41:31preserve lines is false we just call
3:41:33truncate by characters. Now we can call
3:41:36truncate text over here. So let's import
3:41:39it from utils.ext. And now I need to
3:41:41specify the output. Then I need to
3:41:44specify the max number of output tokens.
3:41:47And we have already created that
3:41:49variable. After that, we are going to
3:41:51have a suffix. And our suffix is going
3:41:54to look a bit different. We're going to
3:41:56have back slash end dot dot dot. And
3:41:59then we have truncated. And then we also
3:42:01say that the total lines
3:42:04are the total number of lines in this
3:42:07particular file. And let's also create a
3:42:10variable to track if truncation was done
3:42:12or not. So we have truncated is equal to
3:42:15false. And then we have truncated equal
3:42:18to true here. Now we might also want to
3:42:20add metadata about what was the files
3:42:24content that was actually read. So we
3:42:27have a variable called metadata lines
3:42:30which is equal to an empty list and then
3:42:32we just say that hey if the start index
3:42:34was greater than zero or the end index
3:42:37was less than the total number of lines
3:42:40in that case we'll have metadata lines
3:42:43dot append telling that hey we are
3:42:45showing lines start index + one because
3:42:49index is zerobased I want actual line
3:42:51number to display to the user and then
3:42:53we have end index because that is one
3:42:56index based this is not zero index based
3:42:59of the total number of lines. So this is
3:43:02just so this line will just tell the LLM
3:43:06and it will actually be displayed out to
3:43:08us saying that hey we are showing lines
3:43:10let's say 30 to 70 of 170 lines. This is
3:43:15what this metadata lines is doing and it
3:43:18obviously only does it if the start
3:43:20index is greater than zero. If you're
3:43:22reading the entire file, then there's a
3:43:25point showing all of these things. But
3:43:28if you exceed, we've done something
3:43:30wrong and we're not going to display
3:43:31anything. So that's about metadata. Now,
3:43:34if metadata lines is available, then
3:43:37we're going to add a header saying this
3:43:40dot join and then we have metadata lines
3:43:43plus two backslashes. This is just
3:43:47related to show off you know and the
3:43:50output is going to be out header plus
3:43:55the output by show off by the way I mean
3:43:58this will be displayed to the user so
3:43:59we're just having good things just
3:44:02displaying over here and output is
3:44:05header plus output because we want to
3:44:07display this header at the top we're
3:44:09showing lines 30 to 70 and then we have
3:44:13the truncated output So if we have a,000
3:44:19lines, it will just so show let's say 60
3:44:22lines or 70 lines or whatever. So yeah,
3:44:24that's done. And by the way, we are not
3:44:26preserving lines over here. And I
3:44:28misspelled over here, but we're not
3:44:30preserving lines. So we are actually
3:44:31going to truncate by characters. So
3:44:34that's cool. And now we can return the
3:44:36tool output or tool result dot success
3:44:41result. Then we pass in the output which
3:44:44is well the output that we just captured
3:44:48over here. Then we are going to add in
3:44:51keyword arguments which is truncated
3:44:53which is equal to truncated because in
3:44:56success result and specifically in tool
3:44:58result we're going to capture one thing
3:45:00which is a variable called truncated
3:45:03which is going to be a boolean value and
3:45:05by default it is false because we want
3:45:08to avoid truncation but it's just that
3:45:10if the files are long you have to
3:45:13truncate it otherwise there'll be some
3:45:15problems. So yeah, now we have access to
3:45:18truncated as well. And then the final
3:45:21thing that we're going to have is all of
3:45:24the metadata. Now the metadata is not
3:45:27going to be very simply like just the
3:45:29total number of lines. No, first we're
3:45:32going to pass in the path. At what path
3:45:35are we doing this? And by the way, this
3:45:38all of this metadata is really useful
3:45:40when we want to display it to the user.
3:45:43So what path did we read the file from?
3:45:47That is an indication given to user or
3:45:49even when we try to write a file as you
3:45:51can see it tells us it created this
3:45:54particular path right how does it know
3:45:56that and how does it know how many lines
3:45:59it created all of that is coming from
3:46:02meta data from tool result so it's
3:46:05especially useful to help us in the TUI
3:46:11so we'll have total lines displayed then
3:46:13we're going to have shown start where
3:46:15did we start from and then we have start
3:46:18index + one. Then we have shown end
3:46:22which is the end index and that is just
3:46:25one base. So just end index not + one.
3:46:29So yeah that is everything related to
3:46:33reading a file. Let's try to do a try
3:46:36and accept or actually this just feels
3:46:39good enough. I think we've tried to
3:46:41handle all of the errors. But anyways,
3:46:44let's say just in case something wrong
3:46:47goes, I would like to have a try and
3:46:49accept done there as well. So I'll just
3:46:53tab everything. And then we have a try
3:46:56block. And then we have an except with
3:47:00exception as e. And then we just return
3:47:03tool result dot error result. And then I
3:47:06pass in fail to read file. And I'll just
3:47:10pass in the entire error stack as it is.
3:47:14No need to format anything or whatever.
3:47:17So that is our read file tool. We have
3:47:19actually implemented everything. Now I'd
3:47:22like to go ahead and register this tool.
3:47:25Now registering this tool is as easy as
3:47:27you know maintaining a dictionary where
3:47:30the key is the tool's name and the value
3:47:32is this particular tool which is
3:47:35converted into open AAI spec format.
3:47:38Right? Because we had created open AAI
3:47:42schema here. We can just call this. So
3:47:44we get the entire dictionary format and
3:47:47then we can just pass it to the LLM
3:47:49client. But that is kind of a lousy way
3:47:52to do it. If our application gets
3:47:54bigger, the dictionary will need its own
3:47:57management. And that's why we're going
3:47:59to create a class called tool registry.
3:48:01That tool registry class is going to
3:48:03deal with registration of all of the
3:48:06tools including MCP. And in the future,
3:48:09if you want to enable or disable each
3:48:12MCP, you can do that. that you can add
3:48:14that feature yourself. We're not going
3:48:16to add it in this tutorial, but it's
3:48:18fairly straightforward. You'll be able
3:48:19to do it if you understand this
3:48:20tutorial. So, what I like to do is
3:48:23create a new file here called registry.
3:48:26py that deals with the registry of all
3:48:29of the tools that we have. Essentially,
3:48:32it's trying to manage the entire
3:48:34dictionary of tools. But you can
3:48:37register tools, you can register MCP
3:48:39tools, you can unregister any tools. And
3:48:42obviously you can have all of the tools
3:48:45listed out in OpenAI spec format or the
3:48:48schema format. So yeah, all of that is
3:48:51going to be done in this part. So let's
3:48:53quickly create it. Once we have this
3:48:55registry created, we can go ahead in our
3:48:57agent initialize this registry and then
3:49:00from the agent we can pass it to the LLM
3:49:02client and then the LLM client will give
3:49:05us the tool call to do and then we can
3:49:08actually execute the tool call. So
3:49:10that's going to be a lot of fun. But
3:49:13step one is just creating a tool
3:49:14registry class and maintaining that
3:49:16tools dictionary. So I'll go ahead and
3:49:19create the class called tool registry.
3:49:22And then within the init function I'm
3:49:24going to go ahead and create well the
3:49:26tools dictionary that I was talking
3:49:28about. So we'll have self tools. I want
3:49:31it to be protected or private not to be
3:49:33used outside. And then we have a
3:49:35dictionary with the key as a string and
3:49:38the value as a tool itself. And from
3:49:40tools.base base we will import tool and
3:49:42we will have an empty dictionary as of
3:49:44now. After that the next thing we
3:49:47require is a register function because
3:49:49we want to register a tool. So we'll
3:49:52have register and then we'll have a self
3:49:55tool which will also be of the type of
3:49:57tool and then we're going to return
3:49:59nothing from here. After that well how
3:50:02do we register a tool? I told you it's
3:50:04going to be a simple dictionary. The key
3:50:06is going to be the tool's name and the
3:50:08value is going to be the tool. So first
3:50:10I'll just check that hey if tool.name is
3:50:13in the self.tools dictionary in that
3:50:16case well the tool already exists and
3:50:20that's why what I'm going to do is put
3:50:22out a warning. Now to print out a
3:50:24warning I'm not going to use the TUI or
3:50:27whatever. Instead I'm just going to
3:50:29create a logger. So I'm going to have a
3:50:31logger which is equal to logging.get
3:50:34logger. If you're unfamiliar, logging
3:50:36allows us to pass in multiple things. It
3:50:40can be a warning message. It can be a
3:50:42normal message. It can be an error
3:50:43message. It displays all of that. And we
3:50:45need exactly that. So, what I'm going to
3:50:48do is import logging at the top. It
3:50:51comes in with Python. That's the only
3:50:53difference between print and logging.
3:50:56Logging just gives us nice methods to
3:50:58print out warning or error message, but
3:51:01print only gives us print statement to
3:51:03print out stuff. And all of them get
3:51:06printed out on standard output. So yeah,
3:51:08that's good for us. We'll print out
3:51:10logger.wn warning. And if that tool
3:51:13already exists, we are saying well
3:51:15overwriting
3:51:17existing tool and then we pass in the
3:51:19tools name. Great. After that we have
3:51:23self dot tools at tool.name
3:51:28which is equal to tool. This is how we
3:51:30register the tool. And now I can just do
3:51:33logger.debug saying well I registered
3:51:36the tool. So that's done. And now we
3:51:39have tool.name.
3:51:41And this will only show up when you know
3:51:43we have the debug debug mode on.
3:51:45Otherwise it won't show up. That's how
3:51:48we're going to configure the logger as
3:51:50well. Now if you want you can totally
3:51:52remove the logger. It's no need. I just
3:51:54wanted it just in case I run into any
3:51:56error. And we can add that in all of the
3:51:59places. I might forget to add somewhere
3:52:01else but you keep it in mind. Please add
3:52:04it as much as you can to help us debug
3:52:07just in case we run into errors. So the
3:52:10next thing that we want is an unregister
3:52:12function. Right? Unregister will help us
3:52:15unregister a tool. So we just need the
3:52:18name of the tool and we going to return
3:52:21a boolean value. Did we unregister it or
3:52:23not?
3:52:24So if the name is in the self dot tools
3:52:28in that case we are going to return true
3:52:30and also what I'm going to do is delete
3:52:32from self.tools. So I'll do delete
3:52:35self.ore tools and from there I'm going
3:52:38to delete this particular property or
3:52:41the key that we added right we had a key
3:52:44and a value pair of that tool. So I'm
3:52:46deleting it from the dictionary. Now if
3:52:49the name wasn't in self.ttools what we
3:52:51can do is return false. Obviously, when
3:52:53we add MCP support, we'll have to check
3:52:56if that tool exists in MCP as well. And
3:52:59if it doesn't exist in MCP, then we can
3:53:01return false. But as of now, this is
3:53:03good enough. Now, I'll go ahead and
3:53:06create the function called get schemas.
3:53:09So, we'll have def get schemas. And what
3:53:12this function does is it just prints out
3:53:15all of the schemas in OpenAI format. So,
3:53:19we'll have self and well, as of now,
3:53:21this is it. But we can have more. And
3:53:24then we're just going to return a list
3:53:27of dictionary, right? Because it's going
3:53:29to be in OpenAI format. And the value
3:53:31can be anything.
3:53:34And then we will do return. And now for
3:53:37every tool in this self.get tools, which
3:53:40is what we're going to create next.
3:53:43Self.get tools essentially refers to the
3:53:46function that will get all of the
3:53:48registered tools. So we'll go ahead and
3:53:50create that function. We have def get
3:53:53tools. Then we have self. And then we
3:53:56are going to have a list of tools here
3:53:59which is going to be an empty list
3:54:01initially. But then we'll go over all of
3:54:03the tools within this. So we have
3:54:05self.ttools dovalues. And then we have
3:54:09well tools.append
3:54:11tool. Great. And then we just go ahead
3:54:14and return the tools. Obviously this
3:54:16function will get more complicated once
3:54:18we have MCP. not complicated as such but
3:54:21like we'll have to just get all of the
3:54:24tools from MCP tools dictionary as well.
3:54:26So we'll be maintaining another
3:54:28dictionary to maintain MCP related
3:54:30tools. So yeah nothing too complex. Now
3:54:34we can call get tools function which we
3:54:36did over here and that function is going
3:54:39to return a list of tools. Okay. And now
3:54:43for every tool that we get over here we
3:54:45are just going to convert it to open AI
3:54:47schema. So we have tool.2 to OpenAI
3:54:50schema and that's it. Now another thing
3:54:52that we can create is a wrapper around
3:54:57invoking or executing a function. So as
3:55:00you know to execute a function we have
3:55:03created a function called tool.execute.
3:55:05If we go to base py here it was execute
3:55:09and in read file we override this method
3:55:13and create a new execute function which
3:55:15is specific to read file. In write file
3:55:17we'll do the same but with write file
3:55:20specific logic in gre glo everything is
3:55:23going to have the same thing but that's
3:55:25just executing. If we really want to
3:55:27invoke a tool we'll also have to call
3:55:29validate parameters. So what are we
3:55:31going to do? We're going to create a
3:55:32wrapper around both of these methods. So
3:55:35what we'll do is have async def invoke.
3:55:39We'll get a self. Then we'll get a name
3:55:40of the type of string. Then we might
3:55:43also get call id. But what we are more
3:55:46interested in is parameters which is a
3:55:48dictionary string, any
3:55:52and we might also get the current
3:55:53working directory which is path or null.
3:55:57And now let's import path. So we can
3:56:00just import that from pathl. Awesome.
3:56:03Now in invoke we are just getting the
3:56:05name right. So first we need to get the
3:56:07actual tool that we are talking about.
3:56:09And for this also we can create a helper
3:56:12function called get. Now again this get
3:56:15is going to be very very simple. It's
3:56:17nothing extraordinary. It's a twoline
3:56:20threeline function but it can return a
3:56:22tool or a null and we are given the name
3:56:25of the tool. So it can be a string
3:56:29and now we are saying that hey if the
3:56:31name is in self dottools in that case
3:56:34we'll return self do.tools at name.
3:56:37Cool. And then when we have MCP we'll
3:56:40have the MCP related stuff over here as
3:56:42well.
3:56:44Now if that as of now if that doesn't
3:56:46exist we'll just return null that hey we
3:56:48do not find the tool with this name and
3:56:50now what I can do is just call get over
3:56:54here. So we have self.get and then I
3:56:56pass in the name and that gives me the
3:56:59tool itself. So we have tool is equal to
3:57:02self.get and if tool is none in that
3:57:05case we have an error because the user
3:57:08or the llm has invoked a tool that does
3:57:11not really exist. So we'll just return
3:57:13tool result
3:57:15and we'll have to import that from
3:57:17tools.base and then we have error result
3:57:20and now I can pass in the data which is
3:57:23unknown tool and then we pass in the
3:57:26name. Now if you want as the metadata
3:57:29you can pass the tool name again so that
3:57:32it can be shown to the user and that's
3:57:35exactly what I'm going to do. I'll pass
3:57:37in metadata is equal to tool name
3:57:41and then I have the name passed in. Now
3:57:45error result does not explicitly take
3:57:47metadata. So similar to success result,
3:57:49we're going to have keyword arguments
3:57:52over here. And then I say if any keyword
3:57:55arguments are present, we'll just put it
3:57:57down at the bottom.
3:58:00Now coming back to the registry,
3:58:03let's say the tool is not null. In that
3:58:05case, what do I want to do? I want to
3:58:07validate the parameters. So, I'll just
3:58:08call tool dot validate parameters and
3:58:12then I call parameters dictionary. So,
3:58:15what is the parameters dictionary? Well,
3:58:17we just get it from here. The LLM is
3:58:19going to give all of the parameters to
3:58:21us because when we tell the LLM that
3:58:25hey, these are the tool calls, it will
3:58:27give us all of the parameters, right?
3:58:30So, yeah, that's that. And we know
3:58:33validate parameters gives us a list of
3:58:36strings and those are just validation
3:58:39errors if they exist. And if it's empty
3:58:42then we have no validation errors. So I
3:58:44can just do if validation errors we have
3:58:47failed. That means the list is not
3:58:49empty. So we'll have return tool result
3:58:52and not tool registry tool result dot
3:58:56and then we have error result. Cool. Now
3:59:00I'll pass in the error message which is
3:59:02which is invalid parameters.
3:59:05Then let's format this nicely. So I'm
3:59:08going to join all of the validation
3:59:10errors that we get. So we have a colon
3:59:13that's joining all of the validation
3:59:15errors. And there we go. We'll also pass
3:59:18in the metadata over here. The metadata
3:59:22again for the user to see it is just the
3:59:25tool name. So yeah, that's the tool
3:59:28name. And then we also pass in all of
3:59:30the validation errors that we had until
3:59:32this point. So there we go. Awesome. The
3:59:35user if you want you can remove this by
3:59:38the way because you might not want to
3:59:39show your user all of the validation
3:59:41errors but since my target audience is
3:59:43going to be developers I would like to
3:59:45display that right now. I would like to
3:59:47call tool.execute.
3:59:49So I can just do await tool.execute
3:59:52because it's an asynchronous function
3:59:54but I need to pass an invocation. So I
3:59:56have to create an invocation context. So
3:59:59we have invocation is equal to tool
4:00:01invocation
4:00:03and then I have to import it from
4:00:05tools.base.
4:00:06And now I can pass in the parameters
4:00:08which is just parameters and then the
4:00:12current working directory which is also
4:00:14being given either by the LLM or we just
4:00:17pass in our own current working
4:00:19directory. We'll see what we have to do.
4:00:21Obviously when the configurator is
4:00:23present when we add configuration which
4:00:26is probably not so late because now we
4:00:29just have to register the tools. We pass
4:00:32that to OpenAI then we execute the tool
4:00:35and yeah then we'll start working on the
4:00:38configuration system. After that we'll
4:00:41come back to update all of the
4:00:43configuration systems. So because most
4:00:45of the places we've just hardcoded
4:00:47right. So we'll update the configuration
4:00:49system and then we'll add rest of the
4:00:52tools that are remaining because they do
4:00:54depend on a configuration system. So
4:00:57yeah, that's how we're going to go about
4:00:58it. Now we can just pass in the
4:01:01invocation context and let's put it in
4:01:04try except because there might be a
4:01:06possibility of error because well if we
4:01:09go to built-in tools read file here if
4:01:14there's any error we are handling it but
4:01:17it might happen that in some other tool
4:01:19we are not handling it. So I'll just
4:01:21catch a broad exception. I just don't
4:01:23want my app to fail. So I'll just pass
4:01:26in a logger.exception exception here
4:01:28saying that hey the tool with this name
4:01:32raised unexpected
4:01:34error if you want you can pass e as well
4:01:37but I don't think it will give any
4:01:39valuable information and then we'll just
4:01:42return tool result dot error result then
4:01:46obviously we'll pass in the internal
4:01:48error because well for the llm it's
4:01:50going to be quite a useful information
4:01:52whatever this e contains and then I can
4:01:56pass in the meta data again it's just
4:01:58going to be the tool name we want to
4:02:01display the tool name right so we have
4:02:03to pass that information outside awesome
4:02:06so yeah this looks good enough for now
4:02:09let's create a function it's going to be
4:02:11a global function which will create a
4:02:13new tool registry here and then what it
4:02:16will do is it will register all the
4:02:19built-in tools so whatever tools you
4:02:22create over here we'll just assemble
4:02:25them register them and create a
4:02:27registry. So it's like a singleton
4:02:29instance. We're creating one instance of
4:02:31this registry.
4:02:33We're registering everything and we're
4:02:34returning the registry out from here. So
4:02:36we'll have create default
4:02:40registry and then we're just going to
4:02:43return the tool registry from here.
4:02:47After that we are going to create an
4:02:49instance of registry which is tool
4:02:51registry like that. And we're going to
4:02:53return this registry. But before
4:02:54returning it, what we're going to do is
4:02:56register all of the built-in tools. So
4:02:58first I need to get access to all of the
4:03:02built-in tools. And I'm just going to
4:03:04maintain the constant here called
4:03:06builtin tools which is equal to a list.
4:03:10And then we've just pass in a read tool,
4:03:14right? Read file tool whatever it was
4:03:16called. So let's just pass it in. Let's
4:03:19import it from tools.builtin
4:03:22read file. Now from every file in
4:03:25tools.builtin you'll have to go ahead
4:03:27and import this particular line. So when
4:03:30we have write file you'll have to do
4:03:32something like this. Write file import
4:03:35write file tool and we have about 10
4:03:40tools 15 tools I don't know. So we'll
4:03:42have to write down 15 lines. Instead of
4:03:44wasting those lines what we can do is in
4:03:47built-in we can create what's known as
4:03:50an init. py. You might have seen this in
4:03:53multiple packages or multiple projects.
4:03:56But what it essentially does is it's
4:03:59creating built-in folder as a module.
4:04:02Instead of saying built-in read file,
4:04:04we'll import read file tool, you can now
4:04:06just say from tools.builtin, we are
4:04:09going to import read file tool. So that
4:04:13allows us to do from tools.builtin
4:04:16import write file. From tools.biltin, we
4:04:19import shell. And we can also define a
4:04:23thing here called all. So these are all
4:04:25the things that are exported from this
4:04:27module. And the first thing that we're
4:04:29going to have is read file tool because
4:04:32that's the only thing created so far.
4:04:34And here we can have all of the import
4:04:36done. So from unified agent dot tools.
4:04:40So from tools dotbuiltin we going to
4:04:43import read file tool. And actually I
4:04:47forgot to do from tools.builtin built-in
4:04:49dot read file we'll import read file
4:04:51tool and now I can also create a new
4:04:55function over here called get all
4:04:57built-in tools so I have get all
4:05:00built-in tools
4:05:03great and this will return a list of
4:05:06well
4:05:08it can be a list of any but let's just
4:05:10say list of one particular type and that
4:05:13type is going to be tool so maybe you
4:05:15can also add tool over here and we can
4:05:17import that from tool tools.base
4:05:20but type also seems fine. And now we can
4:05:23just return a list called read file
4:05:26tool. That's it. And now whenever we add
4:05:29a new tool, we'll have to register it
4:05:32over here. And we'll call this function
4:05:34within registry. So now the point of
4:05:38doing all of this was that instead of
4:05:39writing tools.builtin read file, you can
4:05:42do tools.builtin, we'll import read file
4:05:45tool. It is exported correctly. And we
4:05:48can also import get all built-in tools.
4:05:51So instead of having everything on a new
4:05:53line, you have everything on the same
4:05:55line. That's just clean. And now I can
4:05:59just say every tool class in the get all
4:06:03built-in tools. We are going to call
4:06:07and register this. So we have registry
4:06:10dotregister. And now I have to register
4:06:12this tool. But now I have access to the
4:06:15tool class. Remember that if we go to
4:06:17get all built-in tools, this is a class
4:06:20called read file tool. Okay, so I have
4:06:23access to the class. Now I can just go
4:06:25ahead and instantiate this class like
4:06:27read file tool and pass in all of the
4:06:29parameters that I want. Right? As of
4:06:32now, this does not require any
4:06:34parameters. But in the future, we might
4:06:37want to pass in the configuration
4:06:39system. So I'll just pass in tool class
4:06:42like this now. And yeah, that's pretty
4:06:45much it. We have registered the tool.
4:06:48Now I can just use this create default
4:06:50registry wherever I want and that will
4:06:54register all of the tools and it will
4:06:56return the registry to me so that I can
4:06:58get access to this method of get
4:07:00schemas. Once I have access to get
4:07:02schemas, I can call OpenAI schema and
4:07:06you know convert it to JSON so that I
4:07:08can pass it everywhere. Awesome. So now
4:07:10I'll go back to my agent. py file. We
4:07:13not touched it in a while. But here in
4:07:16the init function along with these two
4:07:18things we are going to have self do.to
4:07:20tool registry. Of course just like
4:07:23client and context manager even the tool
4:07:26registry is going to be within session.
4:07:28Everything is going to be encapsulated
4:07:30within a session. Session is that
4:07:33strong. Yeah. Because whatever tools are
4:07:36registered for one session might not be
4:07:38applicable to another one. Especially
4:07:40when we add tool discovery. When we add
4:07:43tool discovery, it is totally possible
4:07:46that the tools are registered for one
4:07:48project because they're useful for one
4:07:50project, but they're not applicable to
4:07:52another project. So we need tool
4:07:54registry specific to a session. And now
4:07:57that we have tool registry with us, we
4:08:00can scroll down go to our agentic loop
4:08:02where we have called self.client.comp
4:08:04completion. Before that we can get
4:08:06access to the entire tool schemas.
4:08:08Right? So we have tool schemas is equal
4:08:12to self [snorts] dot tool registry.get
4:08:15schemas
4:08:17and then we have a list of dictionary
4:08:19we'll just copy this tool schemas and
4:08:21send it across. We can have after before
4:08:24this boolean actually we'll do tools is
4:08:27equal to tools schemas if tools schemas
4:08:29is present otherwise we'll just pass in
4:08:31null that hey we did not find anything
4:08:34because we don't want to pass in an
4:08:36empty list right and chat completion
4:08:38requires a stream value which we've
4:08:40already passed in which is true and by
4:08:43default also it is true so even if we
4:08:45don't pass it in that's totally fine now
4:08:48let's go to chat completion and have a
4:08:51new thing here So yeah we can go ahead
4:08:53and have tools with list and list like
4:08:57this. Then we have a dict and then we
4:09:00have string comma any or the value can
4:09:02be null and by default it is null. Okay.
4:09:07Now I want to pass this tools list into
4:09:10this keyword argument if it is not null.
4:09:12So I'll just check if tools then we'll
4:09:14have keyword arguments at tool and I
4:09:18think we should pass in tools which is
4:09:21equal to well we can pass in tools just
4:09:23like that but if you remember the schema
4:09:27looks something like this let me go to
4:09:29base py and here you'll notice that the
4:09:33openi schema is name description and
4:09:35parameters exactly what we want but
4:09:39we're not encapsulating it within the
4:09:41function
4:09:42attribute. So the actual thing should be
4:09:45something like this where you have this
4:09:47dictionary and this is encapsulated
4:09:50within another dictionary which is type
4:09:54function and I it did not feel right to
4:09:57do it over here because well it's just a
4:10:00lot of boilerplate code to add for every
4:10:02single tool. So instead what we can do
4:10:05is create a function called build tool.
4:10:09So we have def build tools. Then we have
4:10:13self tools which is just a list of
4:10:16dictionary string comma any it can't be
4:10:18null anymore because you have already
4:10:20checked that and we return uh a list
4:10:24because well we have tools list right
4:10:27we'll go over every tool in this tools
4:10:30list and then we are going to create an
4:10:33object for everyone and then we'll have
4:10:35a type of function and then we'll
4:10:38actually pass in the function where the
4:10:41value is going to Well, everything we
4:10:43have done so far, the tools name, then
4:10:46we have the description, which is going
4:10:49to be tool.get description. And if it
4:10:52doesn't have that, it's going to be
4:10:53empty. And then finally, we're going to
4:10:56have parameters,
4:10:59which is tool.get
4:11:02parameters.
4:11:04And if that does not exist then we're
4:11:07going to default to an object where the
4:11:09key is type object and then the
4:11:13properties is just an empty dictionary.
4:11:16So yeah that's all
4:11:20this is how build tools is going to look
4:11:22like and that's the function we're going
4:11:23to call. We're just going through every
4:11:25tool that we have and we're trying to
4:11:28extract relevant features from it and
4:11:30encapsulate it within the type function
4:11:33because earlier we did not have type
4:11:35function right we just had these things
4:11:39and now we can call self dot build tools
4:11:43then we need to pass in the tools list
4:11:46which we'll pass like that awesome
4:11:50another thing I would like to do is
4:11:52specify the tool choice here. So, the
4:11:55tool choice is equal to auto. If you
4:11:58want, you can force it to use a
4:12:00particular tool, but yeah, I'll stick to
4:12:02auto. And yeah, this looks good enough.
4:12:05Now, what I like to do is go to the
4:12:09stream response and the non-stream
4:12:10response. But in our case, it's majorly
4:12:13just stream response and check if it
4:12:17gives us any tool calls because every
4:12:20delta here can give us tool calls. So
4:12:23I'll just go ahead and print out delta
4:12:26dot tool calls and see if it spits out
4:12:29anything. And now I can one short one
4:12:33response. So I'll have python main. py
4:12:37and let's say we tell it to read agent
4:12:40py file. Well, it doesn't know agent py.
4:12:43We have not given it the ability to go
4:12:47through the folders as of now. So maybe
4:12:50we can just tell it to read main. py
4:12:52because you know agent py is within a
4:12:54folder and our agent obviously doesn't
4:12:57know it's within a folder it doesn't
4:12:59have the ability to even call list
4:13:01directory right so let me just say read
4:13:04main py file for me so let me just hit
4:13:08enter and it spits out an error well it
4:13:12does give out null I don't know what
4:13:14that null is for but then we also get
4:13:17choice delta tool call okay maybe it's
4:13:19the streaming response that's coming in.
4:13:21So the very first thing it gives us is a
4:13:24delta tool call where the index is zero.
4:13:26It gives us the call id. Then we get a
4:13:29function of choice delta tool call
4:13:31function. Arguments are present where
4:13:33the path is given which is just main. py
4:13:36and then we have the name of the tool.
4:13:38Yeah. So it is spitting it out but then
4:13:41we run into an error which I'm not sure
4:13:44what that is about. So yeah, once it
4:13:46gives us the tool call, we are trying to
4:13:48invoke the tool call, right? But when we
4:13:51invoke the tool call, we run into an
4:13:53error related to counting the tokens and
4:13:56specifically the length of the
4:13:58tokenizer. So let's go to utils.ext
4:14:01where we had called return length of
4:14:04tokenizer. And this specifically occurs
4:14:07because the text is null. Since the text
4:14:11that's coming in over here is null,
4:14:13tokenizer cannot accept null value.
4:14:15We're getting an error. But why is it
4:14:17null? Well, if you go to find all
4:14:19references of count tokens, you'll
4:14:21realize that we called it in manager,
4:14:24right? And whenever we try to call it in
4:14:26manager, well, when we add add user
4:14:29message or add assistant message, so we
4:14:32did content or an empty string. So, same
4:14:35thing over here, content or an empty
4:14:37string here is required as well. And now
4:14:40when we try to run it again, let's see
4:14:42if we run into that same error.
4:14:46And we don't. So that was the issue we
4:14:48just had to put in or an empty string
4:14:51because many times the content will not
4:14:53be present. For example, when the tool
4:14:56call is made, maybe the LLM decides that
4:14:58hey, I don't have any text for you to
4:15:00show. And actually, we don't see any
4:15:02text at all because all it does is a
4:15:05tool call. It doesn't even talk to us,
4:15:07our agent. So yeah, let's go ahead and
4:15:11pass this response that's given to us
4:15:14the delta tool calls so that we can
4:15:16actually go ahead and invoke the tool.
4:15:18So here we can first go to LLM client
4:15:22and first check that hey if delta dot
4:15:25tool calls is present that means tool
4:15:28calls is not an empty list. We'll go
4:15:30over all of the tool calls that's given
4:15:32to us. So we'll have tool call delta in
4:15:36delta do.tool tool calls
4:15:39and then we need to get the index of
4:15:42tool call delta. Right? So here the
4:15:45index is zero. But many times it can
4:15:48stack up a bunch of tool calls for us
4:15:53and then it will provide the index to
4:15:55us. That's what we're going to use. So
4:15:57we have index is equal to tool called
4:16:01delta dot index. And then I can check if
4:16:05index is not in tool calls and tool
4:16:09calls is a dictionary that we are going
4:16:11to have. So we have tool calls like this
4:16:14which is just a dictionary with integer
4:16:17as the key which is the index and the
4:16:19value is again a dictionary of string
4:16:22comma any. Now why is it a dictionary of
4:16:24string comma any? Because whatever value
4:16:27that we get over here is going to be a
4:16:31key and a value pair. Right? Path
4:16:33main.py py. Well, the arguments are key
4:16:36value pairs. So, that's what we're going
4:16:39to have. Now, if index is not in tool
4:16:41calls, then what are we going to do?
4:16:43Well, we'll just do tool calls at index
4:16:47is equal to and then we create a new
4:16:49dictionary where we're going to pass in
4:16:51the ID, which is tool called delta dot
4:16:54id. If you notice, we have the ID given
4:16:58to us by the LLM or it can just be an
4:17:00empty string. Otherwise, it will crash
4:17:03the application. Then we have the name
4:17:06of the tool call. As of now, we don't
4:17:08have it, but we are going to update it.
4:17:10And then we have arguments. As of now,
4:17:12we don't have it, but we're going to
4:17:14update it. Now, we have created tool
4:17:17calls at index, but now the next step is
4:17:19to extract everything. So first I'll
4:17:21check that hey if this index has the
4:17:25function right if it has the function
4:17:27argument then I want to get into that if
4:17:30tool call delta dotf function is present
4:17:33then I want to check if tool call delta
4:17:36dotfunction dot name is present within
4:17:39it if that is true that means function
4:17:41dot name is given to us that is read
4:17:45file if it's given to us then I want to
4:17:47update the tool call so I'll have tool
4:17:51calls at index at name. So I access that
4:17:57particular object within which I have
4:18:00this name key and I'm just updating it
4:18:03to be tool call delta.function.name
4:18:05name and as soon as we get the name of
4:18:08the function we can just send a stream
4:18:11event saying that the hey the tool call
4:18:14has started now we are starting to do a
4:18:16tool call and whenever tool call start
4:18:18event comes to the UI what's going to
4:18:21happen well we'll show a dialogue box
4:18:23saying that hey we are executing this
4:18:25thing that you see this tool call and
4:18:28then when it gets over we'll again sh
4:18:31show another tool call so just to
4:18:33demonstrate to you we have write file
4:18:35over here. Right? So this is what tool
4:18:37call start will do. As you can see this
4:18:39is gray. Whenever this is gray, it means
4:18:43the tool call has started. We are
4:18:44working on it. And then we have write
4:18:46file with tick mark. And then it gives
4:18:49us the confirmation dialogue or tool
4:18:51confirmation whatever we worked on. This
4:18:53is what I'm talking about. Tool call
4:18:55starts should show this. So now let me
4:18:58just go ahead and yield this event. So I
4:19:00have yield stream event and then I'll
4:19:02pass in the type. The type is going to
4:19:05be event type or stream event type dot.
4:19:09And now I have to go ahead and create a
4:19:11new event type called tool call start.
4:19:16So let's go. And then we have tool call
4:19:20start over here. And then we have it
4:19:22within a string. And since we are
4:19:25already over here, let's just create
4:19:26other tool calls as well. like tool call
4:19:29delta which just gives us all of the
4:19:32tool call pieces. So example we have one
4:19:35tool call we just send it across then we
4:19:37get another tool call we'll just send it
4:19:38across like that. So tool call start is
4:19:41as soon as we just get the name of the
4:19:45event or of the tool call we send it
4:19:47across but tool call delta is when we
4:19:50get one complete tool call that means
4:19:52arguments as well and we start executing
4:19:54it that's tool call delta and then tool
4:19:57call complete is when everything starts
4:19:59finishing you know the tool call is now
4:20:02complete so that's another one tool call
4:20:08complete like that. So this is great. We
4:20:12have all the six events that we need.
4:20:15Now I can go back to my LLM client and
4:20:17have dot tool call start. And now here I
4:20:21need to pass in what's known as a tool
4:20:23called delta. But obviously we've not
4:20:25created it within stream event. So I'll
4:20:27have to create a new thing called tool
4:20:29called delta within a stream event as
4:20:32well. So we have here let's just create
4:20:36it here at the top. tool call delta
4:20:40which is of the type of tool call delta
4:20:44again or it can be null because many
4:20:46times tool calls are not there also let
4:20:49me just take this as l and since we are
4:20:52here let's just create another event as
4:20:54well which is tool call like that so
4:20:57this is a complete tool call and there
4:21:00we go now let's create both of them
4:21:03we're going to create first tool called
4:21:07delta. So we have class tool call delta.
4:21:12Then we have the call ID associated with
4:21:14it which is a string. We already saw
4:21:16that. Then we have a name which can be
4:21:18null. Many times the LLM might just not
4:21:22send it. And then you have arguments
4:21:24delta. That is just all of the arguments
4:21:27that are present. Cool. And by default
4:21:29it is going to be an empty string. Then
4:21:32we're going to have another data class.
4:21:33Let's call this complete tool call. This
4:21:36is not half a tool call or anything over
4:21:40here. This is half a tool call, right?
4:21:41It doesn't have arguments passed in. But
4:21:44here we have a complete tool call from
4:21:47the model. So again, we're going to have
4:21:50similar stuff here as well. The call ID,
4:21:53the name, and the argument. But this
4:21:55time, the only difference is in the
4:21:57type. This is called tool call, and this
4:21:58is called tool call delta. Great. Now I
4:22:02can go back to LLM client and say tool
4:22:05called delta is equal to and I'll pass
4:22:08in the tool called delta over here and
4:22:11from client.response we'll import tool
4:22:13call delta and now we need to pass in
4:22:16the call ID the call ID. Well, what is
4:22:19it? Well, if we go to the tool calls at
4:22:23a particular index and the ID because we
4:22:26already saw tool call at that particular
4:22:29index which is this one and then its id
4:22:31that's the call id. Then we have the
4:22:34name which is equal to well whatever we
4:22:37extracted over here. So you can either
4:22:39do tool calls at index at name or you
4:22:42can just copy paste this particular
4:22:44line. Same thing. So yeah this is the
4:22:47stream event yielding. Now this is a
4:22:49partial tool call. We still have to give
4:22:52out the entire tool call, right? So we
4:22:55can have if the tool called delta dotf
4:22:59function is present. That means this
4:23:01particular thing is present. In that
4:23:04case I want to go one step further and
4:23:06check if tool called delta dotfunction
4:23:09do.name is present or not. And if that
4:23:12is the case then I'll just do tool calls
4:23:15at index at name. Oh sorry we have
4:23:20already done this function name right?
4:23:21So we don't have to worry about that. We
4:23:23are interested in extracting the
4:23:25arguments as of now. So let me just do
4:23:28it here. If tool call delta dot
4:23:32function dotarguments is present in that
4:23:36case we'll have tool calls at index at
4:23:40arguments
4:23:42and then we'll plus equal to whatever
4:23:44arguments already existed over here
4:23:47which is just an empty string but
4:23:49whatever existed we are just going to
4:23:51append to it. So we have tool called
4:23:54delta dotf function dotaruments.
4:23:58And let's see if I spell that right.
4:24:00Nope. Let me just do arguments. And now
4:24:04since we have the entire arguments, we
4:24:06can go ahead and yield the stream event
4:24:08as well, which is tool called delta.
4:24:11There's a change. There's an update.
4:24:14That's why tool called delta. And then
4:24:16we pass in the ID. Then we pass in the
4:24:19name. And then we also pass in the
4:24:22argument which is tool calls at index at
4:24:26arguments. Nothing too fancy. So let's
4:24:30just copy this. And we have arguments
4:24:32delta is equal to this. Now outside of
4:24:35this for loop, this particular tool
4:24:38calls thing. Once we have completed all
4:24:40of the tool calls here, what I'm going
4:24:43to do is go over all of the tool calls
4:24:45that were present here and emit a
4:24:48complete tool call event type because
4:24:51well, you know, we have done the
4:24:53complete tool call. So we have for
4:24:56index, tool call in tool calls dot items
4:25:00and then we yield the stream event type
4:25:02again. And I don't want to write that
4:25:04out. So I'm just going to copy this and
4:25:07paste it over here. Okay. So the type is
4:25:13event type dot message complete. The
4:25:16tool call is not going to be a delta
4:25:19anymore. It's just going to be a
4:25:21complete tool call because well we have
4:25:23completed. And now we pass in a tool
4:25:25call. From client response we import
4:25:28tool call. And now we have call id equal
4:25:33to tool call at id because that's how
4:25:37we've been storing it right at arguments
4:25:40at name and at id. Then we are going to
4:25:45have name which is equal to tc at name.
4:25:50And finally we have arguments. And now
4:25:53for arguments, it's going to be called
4:25:56arguments, not arguments delta because
4:25:58we're not having a change anymore. It's
4:26:00the complete tool call. And now we need
4:26:04to pass the tool call arguments. So over
4:26:06here we're just getting the arguments in
4:26:09a string format. Right now I don't want
4:26:11to be it to be in a string anymore. I
4:26:14want to convert it into actual key value
4:26:16pair. And that's why we can use
4:26:18JSON.loads and pass in the argument
4:26:21string. But the problem is what if it's
4:26:23not a valid string? Maybe this entire
4:26:26dictionary is broken the way LLM sent
4:26:29it. And that's why what I'm going to do
4:26:31is go to response.py. Here I'm going to
4:26:34create a new function. And let's call
4:26:38this parse tool call arguments. Then we
4:26:43pass in the argument
4:26:46string which is of the type of string.
4:26:48And obviously we need this to be in
4:26:50parameter.
4:26:51This will return a dictionary with
4:26:53string and any as the key.
4:26:57Let's import it from typing. And now we
4:27:00can go ahead and check if not argument
4:27:02string that means it's empty. We will
4:27:04return an empty dictionary. But if it's
4:27:08not then I'm just going to try and
4:27:12JSON load this. So we have returned
4:27:14JSON.loads loads and we pass in the
4:27:17argument string and obviously we need to
4:27:19import JSON which I'll do in this
4:27:21function itself. I'm lazy loading it.
4:27:23Obviously you can just do it at the top
4:27:25as well. It shouldn't be a problem. If
4:27:28this returns an error which is of the
4:27:31type of JSON dot
4:27:33JSON decode error that means the LLM
4:27:38sent a malformed string and we'll have
4:27:40to handle that case which is just like
4:27:42this where we have raw arguments given
4:27:45to us and that is the argument string.
4:27:49So this is our safe parsing of tool call
4:27:51arguments which we'll call over here.
4:27:53Let's import it from client.response
4:27:55response and with this we are
4:27:57essentially done. We'll pass in the TC
4:28:00argument string that we captured and
4:28:02yeah we'll yield the message not the
4:28:05message complete but actually it should
4:28:07be tool call complete. So earlier we
4:28:10sent tool call start that we beginning
4:28:12to do a tool call then we sent updates
4:28:15step changes to whatever we received and
4:28:18then we sent that hey a complete tool
4:28:20call is now happening. So earlier when I
4:28:23said that tool call start is whenever we
4:28:29want to display a dialogue box then tool
4:28:31call complete is whenever we want to say
4:28:35that the tool call has been executed.
4:28:37That was wrong. That's really not the
4:28:40case. Tool call completes just suggests
4:28:43that the tool call is complete. I made a
4:28:45mistake over there. I got confused.
4:28:48tool calls complete is just that hey we
4:28:50have a complete tool call with us this
4:28:52is what we're going to execute and then
4:28:54we do the actual execution and whenever
4:28:56that completes we let the agent know it
4:29:00again so this yielding that we're doing
4:29:03continuously really helps us whenever we
4:29:05want to display anything to the user
4:29:07right this is continuous streaming so
4:29:11first we get the tool calls ID and the
4:29:14name so we just tell it that hey we have
4:29:16the name this is the function that we
4:29:18have and then we get another data from
4:29:20the tool call. We just update it in our
4:29:24UI to tell the user that we have these
4:29:26many arguments and then when the tool
4:29:28call is completed well we just send an
4:29:31indication that now we're starting
4:29:32another process.
4:29:34So yeah this is just streaming in a
4:29:37nutshell. Now this is all of the things
4:29:39for streaming response. Now for
4:29:41non-streaming response thing is going to
4:29:44be quite similar and actually quite easy
4:29:47because we're not going to get deltas.
4:29:49We're directly going to get the tool
4:29:51call. So once we check that hey if
4:29:54there's any message we can go ahead and
4:29:56check hey is there any tool call. If
4:29:59message dot tool calls then we are going
4:30:03to go ahead and store it. So we have
4:30:06tool calls
4:30:09which is just a list of actual tool
4:30:12calls an empty list initially and I
4:30:14misspelled it so badly. After that we'll
4:30:18go over every tool call in message
4:30:20dottool calls
4:30:22and then append it to the tool calls
4:30:24list. Nothing too fancy. We don't have
4:30:26any delta. We're not worrying about
4:30:27anything else. We're just caring about
4:30:30the actual completed tool call. So
4:30:33because it's a non-stream response, we
4:30:35don't have to stream anything. Then we
4:30:37pass in the name, which is just
4:30:38TC.Function.name.
4:30:41Nothing really changes.
4:30:43And then arguments is just going to be
4:30:45parse tool call argument. TC do.function
4:30:48dot arguments. That's it. This is
4:30:51everything related to tool calls in LLM
4:30:55client. And I think with that we've
4:30:58actually covered everything in client.
4:31:00Now obviously we need the configuration
4:31:02system but then the file is over. We
4:31:04won't be coming back to this file. Now
4:31:07we need to go back to agent because
4:31:08that's where we're going to get all of
4:31:10our event types. And actually in LLM
4:31:13client let me just remove any print
4:31:15statement or
4:31:17any tool call statements that we did.
4:31:20And yeah we don't have them anymore.
4:31:22That's great because over here I'm going
4:31:25to print out all of the events. So if
4:31:28you have any tool calls, I would like to
4:31:30see if they're coming back to us because
4:31:32at the end of the day, we are putting in
4:31:34chat completion, right? All the events
4:31:37that we are yielding should come over
4:31:39here. Let's run this.
4:31:43And we get a bunch of stream events. We
4:31:45get tool call start, tool call delta,
4:31:47tool call complete. As you can see, our
4:31:49prompt tokens is now up to 2,635.
4:31:53It's totally normal because we have a
4:31:55very large system prompt. If you want,
4:31:59you can go ahead and read the system
4:32:00prompt for claude. It is a lot more. And
4:32:04even in our system prompt, we're going
4:32:06to add more things, right? And if you
4:32:08notice the tool calls over here, the
4:32:10stream events that we getting, we got
4:32:11one start. That's good. Then we got one
4:32:14delta, but then we got two tool call
4:32:16completes and then one message complete.
4:32:18Why are we getting two tool call
4:32:20completes? The reason we're getting two
4:32:22tool call completes is because we made a
4:32:25mistake over here. We were doing for
4:32:28loop
4:32:29of tool calls do items within this async
4:32:32for loop itself. We have to move outside
4:32:35of here at the same level as the yield
4:32:37stream event because we go over every
4:32:40chunk that is present in the response
4:32:42and we store it in tool call. And once
4:32:44we've done that, we've once we've
4:32:45completed it,
4:32:47we have a dictionary of all the tool
4:32:50calls possible. So we just yield the
4:32:52stream event. And now let's try to run
4:32:54it again.
4:32:57And this time we only get four events.
4:33:00Tool call start, tool call delta, tool
4:33:02call complete, and message complete. So
4:33:05that's awesome. As you can see the LLM
4:33:08did not talk at all. That's fine. But
4:33:12anyways, now in the agent, we have to
4:33:14catch this event and execute the tool
4:33:17call. So yeah, let's work on that next.
4:33:19So let's go ahead and put an L if
4:33:21condition here. So we'll have L if event
4:33:24type, which is if it's equal to stream
4:33:27event type dot tool complete. Well, you
4:33:30can also have something done for tool
4:33:33called delta tool called start if you
4:33:35want to display it to the user. As of
4:33:37now, we'll just focus on tool call
4:33:38complete, which just suggests that hey,
4:33:41the tool call is now ready with us.
4:33:43Let's show it to the user and execute
4:33:45it. So, we'll have tool calls.append.
4:33:49So, we're going to maintain a list
4:33:50called tool calls. And that will contain
4:33:53all of the tool calls that we get from
4:33:55the stream events. And then later on,
4:33:57we'll invoke all of these tool calls
4:33:59either sequentially or parallelly
4:34:01depending on the configuration.
4:34:04For now, let's just go ahead and create
4:34:07a variable called tool calls, which is
4:34:10just going to be a list of tool call.
4:34:13And let's import tool call from
4:34:14client.tresponse. And it's going to be
4:34:16an empty state initially. After that,
4:34:19we're going to have a tool calls.append.
4:34:21And here we are going to append stream
4:34:24event or the event dot tool call. Now,
4:34:29it is possible that the tool call is not
4:34:31present. It can be null. So first let's
4:34:33just check if event.tool call even
4:34:35exists. If it exists then we want to do
4:34:38tool calls.append
4:34:40and then we do have tool calls. So
4:34:42that's great. And now we can go ahead
4:34:45and execute all of the tool calls after
4:34:48we've displayed the response to the
4:34:49user. So let's say the LLM gives us a
4:34:52response text and tool call both. That
4:34:54is totally possible. So what we'll do is
4:34:57just return the response text and then
4:35:00we'll go ahead and execute the tool
4:35:02call. So we'll have for tool call in the
4:35:05list of tool calls that we had earlier.
4:35:08This one we're just going to go ahead
4:35:10and execute it. Now how do I execute a
4:35:12tool call but first before that I would
4:35:15like to display to the user that we are
4:35:17executing a tool call. Right? To do that
4:35:19we're going to create a new event in
4:35:21agent. So we going to go ahead and go to
4:35:24events. py here we have a new event
4:35:27state and that's going to be related to
4:35:29tool call start so let's create a class
4:35:32method here called tool call start let's
4:35:37go ahead we have that now we'll have a
4:35:40class then we'll get a call ID which is
4:35:42of the type of string then we'll get a
4:35:44name of the tool call that we're doing
4:35:46that is going to be a string and then a
4:35:48bunch of other arguments that we might
4:35:50not know about which is string any after
4:35:53that we just want to return a class
4:35:56where the type is the event type or
4:35:59agent event type dot tool call start.
4:36:03Now we want to create a new agent event
4:36:06type right at the top. So let me just
4:36:07copy this line go to a agent event type
4:36:11and here we have stuff related to tool
4:36:14calls. So let me put a comment and let's
4:36:17call this tool calls. Then we have tool
4:36:20call start is equal to tool call start
4:36:23like that. Now since we're already here,
4:36:26we can go ahead and create other tool
4:36:28calls as well. The next thing we will
4:36:31need is tool call complete. Then there's
4:36:34other things like pending and approved,
4:36:36but that's more related to confirmation
4:36:38side of things. That is something we're
4:36:41going to focus on later. But as of now,
4:36:43let's just get tool call working
4:36:45initially, right? when we have tools
4:36:48that require permission like write
4:36:50because if you write outside the window
4:36:53or the outside the folder you might need
4:36:56confirmation that's why so let's go
4:36:58ahead and first just have tool called
4:37:01start and now we can pass in all of the
4:37:03data the data that's required here is
4:37:06first the call ID which is going to be
4:37:09just the call ID then we have the name
4:37:11of the tool call which is the name and
4:37:14arguments well whatever arguments the
4:37:17tool called right for example in read
4:37:19file it can give us arguments like limit
4:37:23path offset so those are the arguments
4:37:26that we are passing in over here and
4:37:28yeah that's pretty much it for tool call
4:37:30start for tool call complete we'll come
4:37:34back because there's more things to do
4:37:36there especially when you're writing but
4:37:38yeah for now in agent py we can just do
4:37:41yield agentvent dot tool call start and
4:37:45now we can pass in the call id which is
4:37:47tool call dot call id. Then we have to
4:37:50pass in the name which is well tool call
4:37:54dot name and then the arguments which is
4:37:57tool call dotarguments.
4:37:59Awesome. So we've given back to the
4:38:02terminal tool call start. Okay. But in
4:38:06reality we have just given the event
4:38:08over here. Right? Because agentic loop
4:38:11is called in this run function. So even
4:38:13from this run function we'll have to
4:38:14yield it back and we're already yielding
4:38:17the event. So we don't have to worry
4:38:19about that. But essentially we've
4:38:21yielded agent event tool call start
4:38:23here. Then we have yielded the event
4:38:26from here where the agentic loop is
4:38:28called and from run function which is
4:38:30called in main. py we are giving out an
4:38:32event. So based on this event we'll
4:38:34display to the user a terminal box. More
4:38:37specifically something like this. So if
4:38:40you have list directory or read file,
4:38:42what it's going to do is a box like this
4:38:46where it specifies that the tool is
4:38:48being executed. Then you have the name
4:38:50of the tool, then the tool call ID, and
4:38:52then you have all of the arguments
4:38:54showing up. And then it also says
4:38:56running because well the tool call is
4:38:58still running, right? And then we also
4:39:00want the box to be blue. So now let's
4:39:03come back to our agent. py and here in
4:39:06the agentic loop, we can go ahead and
4:39:07execute the tool call, right? to call
4:39:10the tool we had created in registry. py
4:39:13something called as invoke, right? It
4:39:16was a wrapper around validate parameters
4:39:18and the execute function. We're going to
4:39:21call this itself. So we'll have self dot
4:39:24tool registry dot invoke and then we
4:39:27need to pass in the name parameters
4:39:30the current working directory. So we'll
4:39:33call just that. So we have tool call
4:39:35dotname which is the name of the tool.
4:39:37Then you have tool call dot call id. And
4:39:40then you have tool call dot arguments.
4:39:44Okay. And then actually we don't need to
4:39:47pass in the call id over here. For
4:39:49invoke the call ID is not required. So
4:39:52we can remove that. But the tool call ID
4:39:54is required when we're displaying out to
4:39:56the user. But when actually invoking
4:39:59call ID does not really help us in any
4:40:02way. So just the name and the arguments
4:40:05and then we can pass in the current
4:40:07working directory. So the current
4:40:09working directory is whatever the user
4:40:12will specify in the configurator and if
4:40:15they don't mention anything it's just
4:40:17going to be null. So as of now we can
4:40:18just leave it to null and see what the
4:40:21agent says. If we go to invoke the
4:40:23current working directories over here
4:40:25and we do pass it in tool invocation.
4:40:28Finally, for the current working
4:40:29directory, we can just path path in path
4:40:32which we'll import from pathlib dot
4:40:34current working directory. Now,
4:40:36obviously this current working directory
4:40:38is going to come from the configuration
4:40:40system that we're going to add later on.
4:40:42But as of now, just path CWD is fine.
4:40:45The reason we're passing it is because
4:40:47if you look over here, it does say path
4:40:50or none. So, current working directory
4:40:52is allowed to be null. But when we go in
4:40:54the invoke function and see where this
4:40:57current working directory is called,
4:40:58it's called within tool invocation. And
4:41:00tool invocation requires current working
4:41:02directory to be passed in all the time.
4:41:05So what we can do is just remove this
4:41:07confusion. It can never be null. It
4:41:10always has to be present. Once the tool
4:41:12invocation is passed in, it will execute
4:41:14it. And whatever is the response and
4:41:18whatever is the result from here, we
4:41:20want to return it. But we have not done
4:41:21that because tool.execute does return a
4:41:24tool result and that's what we want to
4:41:27return from this invoke function as
4:41:29well. So what I can do is specify the
4:41:32return type over here which is tool
4:41:35result not tool registry tool result and
4:41:39then whenever we have success we will
4:41:42get the tool result from tool.execute
4:41:45and then we can go ahead and return the
4:41:47result. If you want, you can also return
4:41:49the result from here. It doesn't really
4:41:51matter because if you have an exception,
4:41:54it will return the tool result error
4:41:56result here. So, we don't execute any
4:41:59further. But if there's a success, you
4:42:01are going to return a result. So, that's
4:42:03good. And obviously, if you want to make
4:42:05this cleaner, you can just do result is
4:42:08equal to tool result dot error result.
4:42:10Now, both of them contain the same
4:42:13variable result. So, we can just return
4:42:16it at the end. Cool. So now let's come
4:42:18back to agent. py where we are calling
4:42:21the invoke function. Whenever the invoke
4:42:24is correctly done, what we want to do is
4:42:27return. So we have result is equal to
4:42:30self.tool registry.invoke.
4:42:33Now this result can be a failure. It can
4:42:35be a success also. Let's await it. It's
4:42:38totally possible. It can be well
4:42:40anything. So what I want to do is just
4:42:42return it with the tool call. Right?
4:42:46So whenever the tool call is empty, I
4:42:48would like to again tell to the user
4:42:49that hey either it passed or either it
4:42:51failed.
4:42:53In any case, I want to display the stuff
4:42:55to the user. So let's go back to base.
4:42:58py or actually events. py and here
4:43:00create a new function called tool called
4:43:05complete.
4:43:06This is when the actual execution is
4:43:09over which is something related to this.
4:43:12This one was for when the tool call
4:43:15started and this is when the tool call
4:43:17is over. It will have a tick mark sign
4:43:19then it will have the tool call name and
4:43:21then we'll have the call ID. Rest of the
4:43:24design is same but over here we display
4:43:27the result of the tool execution. It is
4:43:30totally possible that we run into some
4:43:32sort of error because the LLM
4:43:34hallucinated. It gave us the wrong path
4:43:36or something. In that case we will
4:43:38display a nice red error message to the
4:43:40user. So here we'll have a class then
4:43:43we'll have a call id which is of the
4:43:45type is string then we have name type is
4:43:48string and then you have the result
4:43:50instead of arguments which were
4:43:51displayed now we're going to display the
4:43:53content of the result and we have tool
4:43:55result here let's import it from
4:43:57tools.base base and now we can just go
4:44:00ahead and return the class where the
4:44:02type is event type or agent event type
4:44:06dot tool call complete and then we'll
4:44:09pass in all of the data. The first data
4:44:11point is the call ID. So let's just pass
4:44:14it as it is. Then we have name again
4:44:18just the name from the parameter and
4:44:20then we have to extract all of the
4:44:22details from tool result. So first it
4:44:25can be a success
4:44:27that's why we're going to get result dot
4:44:29success then we have output which can be
4:44:33result dot output then we have error
4:44:37which can be result dot error so just
4:44:40extracting everything result might
4:44:42contain another thing is meta data so it
4:44:47will be result dot meta data and then I
4:44:50think we also had truncated which is
4:44:52result dot truncate created. I don't
4:44:55know if we had exit code or not. We did
4:44:57not. But later on we're going to add
4:44:58exit code all the diff related stuff
4:45:02because when we are trying to edit or
4:45:04when we are trying to write a new file
4:45:06we would like to display to the user the
4:45:08diff created. So what was the file
4:45:12earlier now with the new edit what is
4:45:15possible? So a diffing view just like
4:45:20cloud code cursor or even visual studios
4:45:24GitHub has it right or even similar to
4:45:27visual studio codes get. So if you go
4:45:29over here and you've done get in it and
4:45:32stuff you can go to the source control
4:45:34try to see what all changes were made as
4:45:36of now and it will show you in red
4:45:39whatever lines were removed and in green
4:45:41whatever lines were added in this comet.
4:45:44So yeah, for now the tool call complete
4:45:47looks good. Later on we can add more
4:45:49stuff. Now let's go back to agent. py
4:45:52and here we can yield another event.
4:45:55This event is going to be agent event
4:45:57dot tool call complete. And then we pass
4:46:00in the call ID again which is tool call
4:46:03dot call ID. Then we pass in tool call
4:46:06dot name and then we finally pass in the
4:46:09result which is just this result.
4:46:12Remember even if it's an error it's a
4:46:15good enough result and let me ensure in
4:46:17events. py we did pass an error so
4:46:20that's good that will be helpful for us
4:46:23so think about it as of now what have we
4:46:25done well if the llm gives us any tool
4:46:29calls we're just trying to execute those
4:46:31tool calls and with this thing we have
4:46:33executed the tool call so we do a tool
4:46:36call start we tell to the user that we
4:46:38are executing a tool call then we
4:46:40execute the tool call and then we tell
4:46:41to the user that hey this is the result
4:46:44of your tool call execution but one
4:46:46thing we've missed out on is adding to
4:46:49the context. So if we execute any tool
4:46:52call, we will have to add it back to our
4:46:54context, right? Because whatever are the
4:46:57results of the tool call, it might be
4:46:59helpful for the next iteration.
4:47:01Obviously, we are in run single mode. In
4:47:03run single mode, it doesn't matter if
4:47:06you put it back into the context or not.
4:47:08But when we have a run interactive mode,
4:47:10which is quite soon, you'll notice that
4:47:13we'll have to put the results back into
4:47:16the context so that the LLM can make its
4:47:18next decision because we're not going to
4:47:20have just a single iteration. We're
4:47:22going to keep going as long as the tool
4:47:26call is not empty. The entire decision
4:47:28here will be made based on tool calls.
4:47:31So anyways, the point I'm trying to make
4:47:33here is just that we have executed a
4:47:36tool call. So it's our duty to just put
4:47:38it back into a context. Whatever is the
4:47:41result of tool call, let's just put it
4:47:43back into context. So what I can do is
4:47:46outside of this for loop, I'll create a
4:47:49function or a variable called tool call
4:47:51results which is going to be a list of
4:47:54tool call or tool result message which
4:47:58is going to be an empty list. This tool
4:48:00result message is another data class
4:48:02that we're going to create and it will
4:48:04contain all of the data or information
4:48:06about the tool call and then obviously
4:48:09we'll add that into the context. So it's
4:48:12just a good way of writing code
4:48:14obviously if you're using
4:48:15object-oriented programming. So let's go
4:48:17to response. py. It's obviously not
4:48:20going to be created in events because
4:48:22it's not related to agent event. It is
4:48:25more like a schema that we're creating
4:48:26and all of the schemas or anything
4:48:30related to that has been created in
4:48:32response. py. That's why we're going to
4:48:35have here at the rate data class. And
4:48:37then we have a tool result message. And
4:48:41then the first thing that we're going to
4:48:43have is a tool call ID which is a
4:48:46string. Then we have a content of the
4:48:49tool result. What is the output? and
4:48:52then if it's an error or not. If it's an
4:48:54error, well, it's going to be a boolean
4:48:56value and by default, it is false. Now,
4:48:59we also need to structure it in the
4:49:01format of OpenAI message because
4:49:05when you're adding it back into the
4:49:07context of the LLM, you need it to be of
4:49:10the OpenAI type, right?
4:49:12For example, when we go back to our
4:49:15context manager, for example, if we go
4:49:18back to our context manager in context
4:49:20manager py here, if we add message item
4:49:25or get messages, what we did is called
4:49:28to dict and to dict just converted it
4:49:30into OpenAI format. Similar to that,
4:49:33even this tool result message needs to
4:49:35have a function that will convert it
4:49:37into an OpenAI message format. Now,
4:49:39OpenAI
4:49:41SDK allows us to pass in the role of a
4:49:44tool call. So, that's what we're going
4:49:46to use. So, we'll have def open AI
4:49:48message. Then, we have self. It will
4:49:51return
4:49:53and let me just fix that. It will return
4:49:56a dictionary with string, comma, any.
4:50:00And then we'll just go ahead and return
4:50:03first the role because if you remember
4:50:06in user related message the role was
4:50:08user. If the assistant gave us a message
4:50:11it's going to be of the role assistant.
4:50:13But now it's a tool call that happened.
4:50:15And OpenAI allows us to pass in a tool
4:50:18here. Then we need to pass in the tool
4:50:20call ID so that OpenAI can
4:50:23know what tool call we're talking about.
4:50:26So we have self.tool ID. It doesn't
4:50:29really understand the name of the tool
4:50:31call. That's irrelevant. The ID is the
4:50:34most relevant thing. It's the unique
4:50:36identifier created. And then we need to
4:50:38pass in the content. What what is the
4:50:40output of this tool that you want to
4:50:42display to the LLM? So let's say I
4:50:45executed read file. I read 400 lines in
4:50:48it. I want to take that 400 lines and
4:50:50give it back to the LLM. Right? So this
4:50:53content is going to contain that. And
4:50:55then you have self.content passed in. So
4:50:58that's about tool result message.
4:51:00Nothing too complicated. So let's go
4:51:02back to our llm or actually agent. py
4:51:06and here we can append this result to
4:51:09the tool call result. So first we'll
4:51:12just display this tool called complete
4:51:14to the user. And then we can have tool
4:51:17call results dot append. And then we'll
4:51:21go ahead and append tool result message.
4:51:25Let's import it. So we have from
4:51:27client.response. response import tool
4:51:28result message. Then we pass in the tool
4:51:31call id which is tool call dot call id
4:51:35then the content which is equal to
4:51:37result and this result needs to be
4:51:39converted into well content right
4:51:43something that we can give to the model.
4:51:45So let's go to this tool called or tool
4:51:48result which is this function and here
4:51:51we've just created error result and
4:51:53success result functions. Another
4:51:55function I would like to create here is
4:51:57to model output. So we just extract the
4:52:00output format it in the format a model
4:52:04will like it. So here we have another
4:52:07function. Let's call it add the rate
4:52:09class method. Now let's create the
4:52:12function to model output and then we'll
4:52:15get a self and it's just going to return
4:52:17a string. We're just trying to create a
4:52:19model's output. Now what is the output?
4:52:22Well, if we run into any success, it's
4:52:25going to be well just the output that we
4:52:27have, right? If it is a success, we just
4:52:30return the output. But if it's an error,
4:52:32I just want to give it a nice output.
4:52:35Okay? I'll just say what the error
4:52:37message is and then what we are
4:52:38displaying back to the user or what we
4:52:41are displaying back to the LLM. Whatever
4:52:43output we have created for the error
4:52:46result, right?
4:52:48So if we have any success, we'll just
4:52:52return the output. If you want, you can
4:52:54also format this output. Totally
4:52:58possible. But I think the LLM will just
4:53:01appreciate the output because it will be
4:53:03able to determine that yeah, this looks
4:53:05good enough for my use case. Now let's
4:53:07move on to the next thing.
4:53:10But if it's an error then what I want to
4:53:12do is just prefix it with error that hey
4:53:15we did not run into any sort of success.
4:53:19I'm explicitly saying that we ran into
4:53:21an error and the error is self dot
4:53:24error. Then we have two lines left so
4:53:28that you know we are just formatting it
4:53:30nicely and then we'll just pass in the
4:53:33output which is well you can just put it
4:53:35on a new line and then we have self
4:53:38do.output output. Now this has to be an
4:53:41S string and this has to be a brace. So
4:53:45yeah, now it's totally possible that
4:53:47this self.output is So yeah, this looks
4:53:51good enough. I can save it. And now in
4:53:53the agent. py for the content, I'll just
4:53:56have result.2 model output. So instead
4:53:59of creating the complex formatting logic
4:54:02here, I just created it in base. py
4:54:06abstracting away all of the logic in the
4:54:08relevant classes. Now another thing I
4:54:10would like to add is is error and well
4:54:13we'll have to know if it's an error or
4:54:16not. How do I know that? Well, if the
4:54:19result does not have any success, it is
4:54:22an error, right? So, I can just say here
4:54:25not result dot success. If it's s if
4:54:29this is false, that means it's an error.
4:54:33So, this will just convert it into true
4:54:35because it is an error. Great. Now that
4:54:37we have caught all of the tool call
4:54:39results, I would like it to be added to
4:54:41the context. So, let's go outside of
4:54:44this loop. Once we have completed all
4:54:47the sequential execution, I'll add it to
4:54:49the context. I'm not doing self.context
4:54:52manager.add over here instead I'll add
4:54:55it later on because just in case we run
4:54:58into any error with the context related
4:55:00stuff, it doesn't error out the tool
4:55:02execution or anything like that. So
4:55:05outside of this for loop you know just
4:55:08after this we'll have for tool result in
4:55:13tool results or tool call results and
4:55:16then we'll just do self dot context
4:55:18manager dot add tool result. So we have
4:55:22a function for add assistant message add
4:55:24user message. We'll also have add tool
4:55:27result and here we're going to pass in
4:55:29the result dot tool call id and it
4:55:34should be tool result dot tool call id
4:55:37and then we have tool result dot content
4:55:42and that's it. So let's go ahead and
4:55:44create a function in context manager
4:55:47called add tool result. So let's create
4:55:51the function here. We have def add tool
4:55:53result. We have a self. Then we get the
4:55:55tool call id which is of the type is
4:55:58string and then we get content which is
4:56:01also of the type of string. Then we
4:56:04return null from here. We don't want to
4:56:07return anything just like we did not
4:56:09return anything from add assistant and
4:56:11add user messages. Now first we'll just
4:56:14create an item which is a message item
4:56:17and then we have the role of tool. Then
4:56:21we have content which is just equal to
4:56:25content. Then we need to pass in the
4:56:27tool call id which is the tool call id
4:56:31we got from here. Let me just say equal
4:56:34to after that we have the token count.
4:56:39Now for the token count and actually
4:56:41tool call ID is not required here. So
4:56:44let me just add that to message item
4:56:46because as I mentioned in the context
4:56:48the tool call ID should be present. It's
4:56:51very very useful for the LLM and well if
4:56:54it's a useful for LLM it's also useful
4:56:57for us. So at the top we'll have tool
4:56:59call ID which is a string or null by
4:57:03default it is going to be null. Another
4:57:05thing I would like to add here is all
4:57:07the tool calls that we've made. So the
4:57:10tool calls will be a list of dictionary
4:57:13of string, any and by default we'll just
4:57:18have an empty dictionary passed in. So
4:57:20we have field which we'll import from
4:57:22data classes where we pass in the
4:57:24default factory which is addict
4:57:27or actually it's a list. So let me just
4:57:29pass in a list. So this just contains
4:57:32all of the tool calls that were made in
4:57:34one message because as I mentioned the
4:57:37LLM can just throw at us seven or eight
4:57:40tool calls in just one message. That's
4:57:42why we're doing a sequential or a
4:57:45parallel execution later on because
4:57:47there can be multiple tool calls. And
4:57:50now obviously we need to add it to two
4:57:52dictionary because two dictionary is
4:57:54called in get messages and if we have a
4:57:57tool call we like to add it into our
4:57:59context right so let's go back to this
4:58:02two dictionary and here we'll just check
4:58:04that hey if the role is
4:58:08related to tool so here we'll just check
4:58:11if the tool call ID is present because
4:58:14it can be null right if the LLM just
4:58:17gives out a text message there's no tool
4:58:19call id but if the tool call id is
4:58:22present then I want to append it to the
4:58:24result. So we have result at tool call
4:58:28id which is just equal to self do.tool
4:58:30call id then we have if self dot tool
4:58:34calls is present if it's not an empty
4:58:36list then I want to add it to the
4:58:39result. So we have result add tool calls
4:58:42which is just equal to self dot tool
4:58:44calls and then we go ahead and return
4:58:47the result. That's good. So whenever to
4:58:50dict is called which is in get messages
4:58:53it will convert it into a dictionary and
4:58:55get messages is called when we want to
4:58:57send data to the LLM. If you remember in
4:59:00agent py itself we called get messages.
4:59:03Now add tool result will just add it to
4:59:06this messages list right whatever we had
4:59:09in cell dot messages. So yeah that's
4:59:12what we're going to do. But first let's
4:59:14just convert it into a token count. For
4:59:17token count, we can just call the
4:59:19utility function count tokens where we
4:59:21pass in the text which is the content
4:59:23and then we'll pass in self dot max or
4:59:26model name sorry
4:59:29for the model name again it's going to
4:59:31be
4:59:33hardcoded
4:59:34which is over here self
4:59:38model name. Great. Finally, once we have
4:59:41this item, we'll just go ahead and add
4:59:43it to the messages list, which is item.
4:59:47So, yeah, this does look good as of now.
4:59:50Let's just call add tool result in
4:59:52agent. py. So, we'll scroll down and I
4:59:55think it's over here. We did call add
4:59:58tool result. And that's great. So now
5:00:01all we need to do is remove the print
5:00:03statement from here. I don't need that
5:00:05anymore. I'm confident that this is good
5:00:07enough for now. I'll just go back to the
5:00:10main py file because that's where we
5:00:12have all of our
5:00:15process message related code because now
5:00:18we should be getting the event over
5:00:20here. So let me put the print event
5:00:22here. Now let me run it. So I have
5:00:25python main.py and it gives out agent
5:00:28event. That's great. So agent start
5:00:30comes in. It gives the right message.
5:00:34Then the tool called start happens and
5:00:36it does give that. Then another tool
5:00:38called start happens and this is
5:00:41actually the tool call complete. So we
5:00:44have a bug in agent.py. We call tool
5:00:48call start two times. So when we execute
5:00:52it gives us tool call start as we can
5:00:54see. But then we see tool call start
5:00:57again. That means in tool call complete
5:00:59we're actually calling tool call start.
5:01:03As you can see even though this is
5:01:04called tool call complete this is tool
5:01:06call start. So let's just fix that and
5:01:09then this part will be fixed. It should
5:01:12be tool call complete. So this is
5:01:13actually the output of our execution and
5:01:16it says that read file ran into an
5:01:19error. Base m cannot be instantiated
5:01:21directly.
5:01:23So somewhere we have called base model
5:01:26directly like this. So I'm going to
5:01:28search throughout my codebase. Hey where
5:01:30have I called base model like this?
5:01:33And in base.py I did call it. So when I
5:01:36was trying to explain to you that hey
5:01:39whenever we have schema here which is
5:01:41self schema we're trying to call base
5:01:43model like that. Now you cannot directly
5:01:46call base model like that because
5:01:49it's a base class you cannot instantiate
5:01:51it but whatever class extends base model
5:01:56you can call that and schema is a class
5:01:58that will extend base model it's not
5:02:00base model itself. So we can go ahead
5:02:02and call schema like that and it should
5:02:05work out. Let me run this again and I
5:02:08get agent start which is my message to
5:02:10the LLM. Then I get tool call start with
5:02:14the particular path and then it says
5:02:17tool read file raised unexpected error
5:02:20argument should be a string or an
5:02:22os.pathlike object where this returns a
5:02:25string not a method and then we get
5:02:28agent event because whatever output is
5:02:31there we just send it back to the user
5:02:33and an llm. So this goes back to the
5:02:36context, right? But anyways, let's try
5:02:38to fix this error. And I think I know
5:02:41where this error lies. This error is
5:02:44wherever we called path. CWD. So
5:02:47whenever we try to instantiate the
5:02:49current working directory, we called
5:02:51this function, right? Well, it's a
5:02:53function. We passed in a function
5:02:55directly. What we need to do is call
5:02:57this function, which gives us a path. So
5:03:00let's go back and try it again. Let's
5:03:03run python main.py.
5:03:06We get agent start again. We again run
5:03:09into another error but this time it
5:03:12looks a bit different. So again the
5:03:14problem is in tool call complete. When
5:03:16we execute the code it says type object
5:03:19tool result has no attribute success.
5:03:22And the reason we run into this error is
5:03:24because if you check the output here of
5:03:28agent event first we have success
5:03:31spelled out incorrectly. So we need to
5:03:33fix that. Now where is that coming from?
5:03:36Well it's easy to find out. We'll just
5:03:37copy this success string and search in
5:03:39the entire codebase.
5:03:42So when we search for success we see an
5:03:44events py we call it
5:03:47success like this. We want double C to
5:03:50be present. Then we'll go to base.py as
5:03:53well and change this function. It should
5:03:55be success result like that. And then in
5:03:58read file we'll just change this back to
5:04:02success result.
5:04:04And another place where we called
5:04:07success result. Let's change it over
5:04:09there as well. Everything should be
5:04:11spelled the same way. Now another error
5:04:15that I found over here was that whenever
5:04:18I'm trying to check if it's a binary
5:04:19file or not. So let me just scroll a
5:04:22little bit up where we have is binary
5:04:25file. Here I'm doing if back/x0000
5:04:29I want to convert this string back into
5:04:32bytes. All right. So once I convert it
5:04:34into bytes I can check if it's a binary
5:04:37file or not. If I just use an ifst
5:04:38string, I'm just checking, hey, is there
5:04:40back slash x0 in the chunk? But when I
5:04:43convert it into bytes, it means
5:04:45something related to bytes, right? I'm
5:04:48not checking a string. And then finally,
5:04:51if we go back to our function where we
5:04:54create two model output, and I don't
5:04:56remember where I created it. So, let's
5:04:58go back to our base. py file where we
5:05:01had two model output. Here we check for
5:05:04if self do.uess. If we have self well it
5:05:08cannot be a class method, right? It has
5:05:10to be an instance method. Class method
5:05:13is created whenever you do something
5:05:15like tool result dot success result. But
5:05:19whenever you have two model output which
5:05:22h which is having self you what you're
5:05:25trying to do is something like tool
5:05:27result where you have created an
5:05:29instance of tool result and then called
5:05:31to model output and that's when you get
5:05:34access to self and all of its
5:05:36properties. That's why it's not going to
5:05:37be a class method anymore. It's just an
5:05:40instance method. So we don't have to
5:05:42annotate it with anything. And now if
5:05:45you notice wherever we call two model
5:05:48outputs I'll just go to its references
5:05:51which is over here. I get an instance of
5:05:53result right whatever invoke gives us
5:05:57it's essentially an instance of result.
5:06:01So I'll just highlight this part which
5:06:04is tool result error result. It's an
5:06:06instance of result which we are
5:06:07returning and that's why we can call two
5:06:10model output on it. Now let me close all
5:06:12the save files. I opened way too many.
5:06:14Let me clear it off and let's run it
5:06:16again. And this time we see agent start,
5:06:19tool call start, and tool call complete.
5:06:22Now we have a much better error message
5:06:25saying failed to read file string object
5:06:27has no attri attribute join. Now we
5:06:31again misspelled somewhere or I
5:06:34misspelled somewhere. You might have
5:06:35seen that in the video. So I just search
5:06:37in the entire codebase. Somewhere I
5:06:39typed in join G. It should just be
5:06:42called join
5:06:44because we're trying to access the join
5:06:46method. I'll save file and let's run it
5:06:49again. This time hopefully it works out
5:06:54and it doesn't. It says fail to read
5:06:56file count tokens missing one required
5:06:59positional argument. So based on our
5:07:02past data, we know that the code
5:07:04executed till here. So obviously the
5:07:06output is in a future line and as we can
5:07:09see the problem is over here in count
5:07:12tokens I did not specify a model. So let
5:07:15me pass in the model and my model is
5:07:18just going to be
5:07:20the same mistral model. I don't quite
5:07:23remember the name. So I'm going to go to
5:07:25LLM client where I had defined the
5:07:28model. Okay, it wasn't an LLM client. So
5:07:32I'm going to go to the manager where I
5:07:34defined my model name which is this
5:07:37particular thing. Of course we'll have
5:07:39to get all of this data from the
5:07:41configurator.
5:07:43This is just temporary fixes. So now
5:07:46let's try to run it again and hopefully
5:07:49it works this time and it does. So what
5:07:52we get is an agent start then it starts
5:07:55the tool call then the tool call
5:07:57completes. So we get read file then we
5:08:00get success as true. The output is line
5:08:03number one is import async IO line
5:08:06number two is import sis import click
5:08:08and that is our main py file and all of
5:08:12them have line numbers attached to it
5:08:15and they go on a new line since in a
5:08:17string format you might not be able to
5:08:19understand this better but to an LLM it
5:08:22should be good enough. You can obviously
5:08:24do all of your magical stuff in it so
5:08:27that the LLM can use lesser resources to
5:08:30understand this better. But in our case,
5:08:33it works just fine. Let me remove the
5:08:36print event now. And now what we're
5:08:37going for is printing it out on the
5:08:40screen. So whenever we have tool called
5:08:43start, I want to display that panel that
5:08:46I talked about earlier. And as a
5:08:49reference, I'm going to have these two
5:08:51boxes with me on the side. so that we
5:08:53can take inspiration from them and clone
5:08:55it. So here let's try to catch the first
5:08:58event which is if the event type is
5:09:01let's say tool call start right agent
5:09:04event type dot tool call start in that
5:09:07case we first want to get the tool name
5:09:09so we have tool name is equal to event
5:09:13dot data get and then we get the name
5:09:17and if we don't get the name it is
5:09:18unknown then we can go ahead to this
5:09:22renderer or the TUI that we had created
5:09:25let me Open this up in the UI tui. py I
5:09:28want to create a function that will
5:09:30display whatever I want in tool call
5:09:32start. So I'll scroll down and then we
5:09:35have def tool call start. Then we get a
5:09:40self method. Then we get the call ID
5:09:43which is of the type is string. Then I
5:09:45want the name of the tool. And then I
5:09:47want all the arguments. Everything we've
5:09:49talked about till now. and arguments is
5:09:52just a dictionary of string, any let's
5:09:54import any from typing. And yeah, now
5:09:57the next thing is well what is this
5:09:59going to return? It's going to return
5:10:01absolutely nothing. And let's put a
5:10:03comma here so that everything is
5:10:05formatted nicely. So the very first
5:10:07thing I would like to do here is at the
5:10:09top create a new property called tool
5:10:13asks by call ID
5:10:17and it is going to be of the type of
5:10:18dictionary where the key is going to be
5:10:21the call ID and the value is going to be
5:10:24the argument for that tool ID. So we
5:10:28know that each tool has its own tool ID
5:10:32right if it's trying to call read file
5:10:34or list directory it has its own ID for
5:10:37example when we have list directory this
5:10:40is the tool ID correct in read file it
5:10:44then gets a new tool call ID so what I
5:10:47want to do here is that whenever we have
5:10:49a tool call ID I want to display it and
5:10:52obviously the next thing is going to be
5:10:55the tool call completed either with a
5:10:58success or an error. So whenever we're
5:11:00trying to call tool call complete, I
5:11:03want the same tool call to be shown and
5:11:06the arguments to be listed out here if
5:11:08necessary. So I'm just storing them in
5:11:11this dictionary so that I don't have to
5:11:13repeat them again and again kind of like
5:11:15caching. Now the next thing I need is
5:11:18what kind of well first let's just store
5:11:20it. So we have self dot tool args by
5:11:22call ID. Then the key is going to be the
5:11:25call ID and the value is going to be the
5:11:28arguments that we got over here. The
5:11:30next thing we need is tool kind
5:11:32resolver. Basically what kind of tool do
5:11:34we have? If it's a read it's going to be
5:11:36can or whatever then there's right
5:11:39yellow shell is magenta. All of these
5:11:42colors need to be put in. For that we
5:11:44first need to know what kind of tool
5:11:45we're dealing with. Right? We've already
5:11:48created tool.kind. If you go to base py
5:11:52here we add read write shell network
5:11:55memory mcp similar thing over here it's
5:11:58just lowerase and you have a tool prefix
5:12:00before that and that's exactly what I
5:12:03what I want to deal with so given the
5:12:05tool name I just want to extract from
5:12:08the tool registry what kind of tool
5:12:11we're dealing with and for that we'll
5:12:13have to go over here we'll have to talk
5:12:16to this particular agent's registry
5:12:19Right. So we have self dot agent. Now
5:12:22this agent has access to tool registry.
5:12:25From the tool registry we can get the
5:12:29tool name.
5:12:31Once we get the tool name and it's not
5:12:34going to be a string. The tool name is
5:12:35this particular tool name. Based on this
5:12:38tool name we can check if this tool
5:12:40exists. If this tool does exist or let's
5:12:43say if this tool does not exist, we
5:12:45return null. Or let's say the tool kind
5:12:50is equal to null and we just return tool
5:12:54kind as it is because it's a null value
5:12:57or actually let's say tool kind is equal
5:13:00to null.
5:13:01Now if the tool does exist in that case
5:13:04I want to get its kind. So I'll just
5:13:06access tool dot kind dot value right
5:13:11because if the tool exists it will have
5:13:13kind and then kind will give us the enum
5:13:16which is toolkind enum let's just go
5:13:19back to this toolkind enum and now I
5:13:21want to get its value because it's all
5:13:23lowercase
5:13:25so I'll just get its value and that is
5:13:28my
5:13:29toolkind value and I want to give that
5:13:32to the function in tui so I'll is called
5:13:36toolkind is equal to toolkind do value.
5:13:38That's good. Now probably we'll create a
5:13:41utility function for this because this
5:13:44will be needed in almost every tool call
5:13:47because think about it. So now whenever
5:13:49I call
5:13:51this particular function tool call
5:13:53start. So let's have self.tui
5:13:56dot tool start. I need to pass in the
5:13:59call ID. Then I need to pass in the
5:14:01name. Then I need to pass in the
5:14:02arguments. And another thing I'll pass
5:14:04in over here is tool kind which is of
5:14:08the type of string.
5:14:11Okay. And here I can just pass in the
5:14:13tool kind as well. But we'll come back
5:14:15to this later on. As of now I'll go back
5:14:18to this TUI. And here based on the kind
5:14:21I want to get the border style. Now that
5:14:24should be quite simple right? I'll just
5:14:25create a function or a variable called
5:14:27border style which is equal to and then
5:14:30I have tool kind. Whatever tool kind I
5:14:32have, I just need to prefix it with dot.
5:14:35Now, this tool kind can also be a null
5:14:37value. As we see over here, if there's
5:14:40no tool with that name, we have a tool
5:14:42kind of null. But most likely, we will
5:14:45have a tool because the llm doesn't
5:14:48hallucinate that much. We just handling
5:14:50the edge cases here. So I can just say
5:14:53that hey if the tool kind is not null in
5:14:56that case the string is going to be
5:14:58something like this where you have tool
5:15:00kind prefixed with tool dot okay so you
5:15:04have tool dot memory tool mcp dot shell
5:15:09all of that otherwise we can say that
5:15:12the border style is tool because border
5:15:14style just refers to this agent style
5:15:17tool or tool dot readad tool right so if
5:15:19we don't have anything I would just like
5:15:20it to be bright magenta are bold. Okay,
5:15:23awesome. We have border style. Now, we
5:15:26just need to get to designing the
5:15:28layout. So, of course, the first thing
5:15:30is going to be creating this title. So,
5:15:32to create a title, I need this dot, the
5:15:35grayish dot. Then, I have the name of
5:15:38the tool and then the call ID. So, let's
5:15:40have a title which is equal to text dot
5:15:44assemble. That will help me construct a
5:15:46text instance by combining a sequence of
5:15:48strings with optional style. And we are
5:15:50going to have optional style. The very
5:15:52first thing is going to be this dot
5:15:55thing, right? This exact dot thing. And
5:15:58the style for it is going to be muted.
5:16:00Muted refers to this grayish style. We
5:16:03have defined it up in this agent. Muted
5:16:06is gray 50. After that we have another
5:16:09one which is the name of the tool. And
5:16:12then we will give it the style of tool
5:16:14itself. If you want you can also refer
5:16:16to this as border style. But I would
5:16:19just like to have this as tool which is
5:16:22this particular theme bright magenta
5:16:24bold. After that we have a space left
5:16:28in. So let's leave two spaces here and
5:16:32have muted just for some show. And then
5:16:36finally the call ID. So the call ID is
5:16:39just going to be hash followed by
5:16:43the call ID up to eight characters. If
5:16:46it's more than eight characters, well,
5:16:48I'll just truncate it and then I have
5:16:50muted because if we have a very long
5:16:52string, it won't look as good. So, we
5:16:54have the title with us. Now, we want to
5:16:56create an entire panel. So, to create a
5:16:58panel, we can just do panel is equal to
5:17:01panel. And now I have to import it. I
5:17:04can just go at the top and do from rich
5:17:07dot panel import panel. Then let's go
5:17:12back down and here pass in all of the
5:17:14things required. The first thing is a
5:17:17title. So the title is going to be the
5:17:20title we just created and let's pass
5:17:22that in correctly.
5:17:24Now obviously the first thing is not
5:17:27title. Before that we have to pass in
5:17:28the box and renderable. Renderable is
5:17:31essentially what do you want to show
5:17:33within this panel. So we have this
5:17:35entire panel. What do you want to show
5:17:37within this? Well, I want to show the
5:17:39key value of the arguments. So I have
5:17:42path. What is the value of the path? And
5:17:45if we have limit, what is going to be
5:17:47the limit? What is the offset? All of
5:17:49those things. And it should work for
5:17:51every tool call. So I can't hardcode
5:17:54anything. So I just want to take this
5:17:57arguments that I have and I want to
5:18:00display them nicely. So what we want is
5:18:03essentially a table, right? A table with
5:18:05key and value. So let me create a
5:18:08function, a helper function that will
5:18:10allow me to create a table. So we have
5:18:13render arguments table here where we get
5:18:17the tool name which is of the type of
5:18:19string then asks which is of the type of
5:18:22dictionary string comma any
5:18:24and what we're doing is just returning a
5:18:26simple table. Now we have to import that
5:18:29table from rich. So we have from rich
5:18:32dot table import table and then we can
5:18:37scroll down and here let's just call it
5:18:40table properly and then I can do table
5:18:43is equal to table dot grid. What is the
5:18:45grid? How many headers do we want? How
5:18:48many how much padding do we want? Well,
5:18:51I don't want any headers. As you can see
5:18:53there's no headers here. I just want to
5:18:55display the key and the value. So I'm
5:18:58just going to specify the padding here
5:19:00because I don't want it to be stuck over
5:19:02here. Right? So I'll leave a padding of
5:19:050 comma 1 0 on the x y on the one on the
5:19:08y-axis. Then we have table dot add
5:19:12column. The first column that we're
5:19:14going to add is related to this key. Now
5:19:19whatever values we add here is going to
5:19:22be muted color. Right? It's somewhat
5:19:24gray color similar to this tool called
5:19:27id. So we can say that the style here is
5:19:30muted. Then the justify is equal to
5:19:33write and no wrap is equal to true. That
5:19:38means if it overflows it overflows we
5:19:41don't wrap it. And similar to this we
5:19:44are going to have another table dot add
5:19:46column which is code this particular
5:19:50thing. And if you scroll up, code refers
5:19:54to this white color, nothing else. And
5:19:57let me remove this extra line. The
5:20:00justifier is not going to be right on
5:20:02this side. We're just going to have
5:20:03overflow is equal to fold. So if it
5:20:06overflows, you just fold it. Obviously,
5:20:09over here, if you want, you can set
5:20:12overflow to ellipses or whatever. I'm
5:20:15not worrying about that. Now the next
5:20:17thing I want to do is just order all of
5:20:20the arguments. So let's say we have read
5:20:22file. In read file we get path offset
5:20:25and limit right. So first I want to
5:20:27prioritize those arguments. So I'll
5:20:30display them first. After that let's say
5:20:33the llm hallucinated I'll pass in all of
5:20:36the other arguments later on. So let me
5:20:38create a function helper function for
5:20:40that as well. So we have def ordered
5:20:43arguments. Then we get a self and I
5:20:47forgot to put self here. Then we get a
5:20:49self again the tool name because based
5:20:52on the tool name I will be doing the
5:20:55ordering. Then we have asks which is of
5:20:58the dictionary string, any and then we
5:21:01just return a list of pupil. Now I'll
5:21:05just create a constant here of preferred
5:21:07order. That means I just want the
5:21:12arguments that I have
5:21:14in what format. So let's say we have
5:21:16read file. First I want to display to
5:21:18the user what is the value of read file.
5:21:22For example, if I have read file, first
5:21:24I want to tell to the user what is the
5:21:26path. The lower priority is offset. The
5:21:29lowest priority is limit. And after that
5:21:32if the LLM sends anything else, I'll
5:21:35display them. So that is what the
5:21:37preferred order is about. And we're
5:21:39going to have a dictionary here where we
5:21:41have a read file. The value is going to
5:21:44be a list where I specify the priority.
5:21:47The first one is going to be path. Then
5:21:49you have offset and then you have limit.
5:21:52Cool. And now I've maintained this as a
5:21:55dictionary because later on we're going
5:21:57to add more tool calls or tool names. So
5:22:00if we have write file or if we have web
5:22:02search or we have edit all of those will
5:22:05be added here where priority is given.
5:22:08So here I'll just say preferred is equal
5:22:10to preferred order dot get and then you
5:22:14pass in the tool name. By default it is
5:22:17an empty list. Let's say the tool norm
5:22:19does doesn't exist. So we'll just have
5:22:22an empty list here. And now I'll go over
5:22:25every key in this preferred list. So
5:22:27let's say read file was the tool. I go
5:22:29over every element in this step by step.
5:22:32So the order matters here and then I
5:22:34just check that hey if the key is
5:22:36present in the arguments because it can
5:22:38be the case that let's say the LLM
5:22:41specifies path and offset but doesn't
5:22:43specify limit. So I'll just check that
5:22:45hey if the key is present in that
5:22:47dictionary
5:22:49in that case I want to add that into an
5:22:52order that I'm trying to create. So let
5:22:54me create a new list called order where
5:22:56I have a list of pupil of string, any
5:23:00let's import pupil from typing. Let's
5:23:04have from typing import pupil pupil
5:23:09string comma any whenever we add a pupil
5:23:13to this list the first element is going
5:23:16to be a string which is going to be the
5:23:19argument like path or offset and then
5:23:22the value or the second value of this
5:23:25pupil is going to be well anything it's
5:23:27going to be the argument at that
5:23:29particular key what is the value so for
5:23:32example we have path so the pupil 's
5:23:34first element is going to be path itself
5:23:37and the second value is going to be what
5:23:39is the value of that path. So let's have
5:23:41an empty list as of now and then we'll
5:23:44check if key is an argument then we'll
5:23:46do ordered dotappend and then we'll add
5:23:50the key which is path and then we want
5:23:54to get the value of that path whatever
5:23:57the llm gave us what path do we want to
5:23:59read the file from and then we have args
5:24:02at key because that is the value now
5:24:04I'll also maintain a list of or [snorts]
5:24:08a set of scene values So what this means
5:24:11is let's say the LLM passes in arguments
5:24:15like path offset and then it passes in a
5:24:19third one where it hallucinates and
5:24:21passes in something like offset two.
5:24:25Okay, just an example. Let's say it
5:24:27passes offset two. In that case, I've
5:24:29given priority to path and offset, but
5:24:31I've missed out on offset two. That is
5:24:34something I should display to the user
5:24:36because just in case LLM is not able to
5:24:38fix it. I as the user can tell the LLM
5:24:41that hey listen, you've done it wrong.
5:24:44Let me do you don't have to pass an
5:24:46offset two to this built-in tool. So
5:24:49here I'll maintain a scene set. And what
5:24:53this is going to do is contain
5:24:55everything all the tools that we have
5:24:57added so far. Okay. So I've added path
5:25:00I've added offset. Offset 2 is not
5:25:03present. So in a later loop I'll go over
5:25:05all of the remaining keys. So I have
5:25:09remaining keys is equal to set of args
5:25:13dot keys. So whatever arguments we got
5:25:16from the dictionary, we go over all of
5:25:18them. We get all of its keys and we
5:25:21subtract it from whatever we've seen so
5:25:23far. So we have seen path and offset but
5:25:26the args dictionary contains path offset
5:25:28and offset two. So we'll subtract path
5:25:32offset offset two from So we'll subtract
5:25:36path offset from path offset offset two.
5:25:39So we'll have something like this
5:25:42where you're trying to do this set minus
5:25:45this set. So you only have offset two
5:25:47remaining which can be displayed or
5:25:50added back to this ordered list. And I
5:25:53explained why we want to do that. It's
5:25:55just so that if there's any error the
5:25:58user can intervene and stop this from
5:26:00going any further. So we have key asks
5:26:03at key and we go over every key in the
5:26:06remaining keys. Okay. So if we have
5:26:09offset two remaining and let's say some
5:26:11other arguments are also present we go
5:26:15over every key in that set and create a
5:26:18tupil where offset two is present and
5:26:21what is the value the llm suggested for
5:26:24that offset two and then I can go ahead
5:26:26and return this ordered thing. Now I
5:26:30have the ordered args with me. I can go
5:26:32over every key value in this ordered
5:26:36args. Let me just call self dot ordered
5:26:40args where I pass in the tool name. I
5:26:42pass in the arguments and that's it.
5:26:45Because remember we have a list of
5:26:47pupil. So I'm going over every element
5:26:50and every element is going to be pupil.
5:26:53So I've extracted the key and the value
5:26:55from that pupil. And now I'm going to do
5:26:57tablet add row. So I've defined my two
5:27:00columns this particular path and then
5:27:03this particular thing or for read file
5:27:05we have again path and the value. So we
5:27:09pass in the key because that is the path
5:27:13and then I want the value which is going
5:27:16to be whatever value we have over here.
5:27:19And finally once we have gone over
5:27:21everything we'll just return this table
5:27:23that we created. Now I can call this
5:27:24render ars table down over here. So I'll
5:27:28have render ax table passed in. Let me
5:27:30do self.tren render ax table. And then I
5:27:33can pass in the tool name which is just
5:27:35the name that we got at the top from the
5:27:39parameter. And then the second thing we
5:27:42need is the display arguments. Now
5:27:46remember whatever argument the llm gives
5:27:48us for example a path the llm is just
5:27:53going to say main. py. It's not going to
5:27:55say the entire path to us based on our
5:27:58current working directory. And if to the
5:28:01user we just say main. py that's like so
5:28:04weird. Instead, what I would like to do
5:28:06is something like AI agent-
5:28:10main. py. So based on my current working
5:28:13directory, I want to show the path. So
5:28:15what I can do is create a new variable
5:28:18here called display arguments which is
5:28:21equal to a new dictionary of arguments.
5:28:25Then I go over the key in path and
5:28:29current working directory. So whenever
5:28:30we have keys like path and current
5:28:33working directory I want to get their
5:28:35their values. So we have display args
5:28:38dot get the key. So we're trying to get
5:28:41path if it's present in the list in the
5:28:43dictionary of arguments llm gave us or
5:28:46the current working directory because
5:28:48remember in list directory as well it
5:28:51can give us a path in read file it gives
5:28:53us a path in write file it gives us a
5:28:56path so we're just trying to extract
5:28:59that value so we just have let's say the
5:29:01main py written out now based on this
5:29:04main py I want to get the relative path
5:29:08so that to the user where we can display
5:29:10stuff nicely. So we'll say that hey if
5:29:13the instance is of the type of string
5:29:18and the reason I put this is because
5:29:19many times the llm just hallucinated and
5:29:22gave me an integer value for a path. I
5:29:26don't know how maybe it was something
5:29:28related to my prompt. So I'm just
5:29:30handling that edge case over here. And
5:29:32let's say the current working directory
5:29:34also exists. So I have to create that
5:29:36variable right at the top which is self
5:29:40do.curren working directory is equal to
5:29:42path cwd. Again this current working
5:29:45directory will also be coming from the
5:29:48configurator towards
5:29:50the next part of our application. Then I
5:29:53pass in the display ars and actually
5:29:55I've not completed it. So we have the
5:29:57current working directory now. Now we'll
5:29:59just say that hey display args at key is
5:30:02just going to be well the formatted
5:30:05display path where we are having the
5:30:08absolute path or the relative path based
5:30:10on the current working directory
5:30:12essentially. So what I can do is extract
5:30:15the function from path py. So here we
5:30:17created resolve path right based on the
5:30:20base and the relative path we want to
5:30:23create a new path and that's what I want
5:30:25to show to the user as well. So we have
5:30:27string resolve path and from utils.path
5:30:30we'll import resolve path. The reason
5:30:32I'm converting it into string is because
5:30:34we get a path here. Now path is not
5:30:37entirely visible on the screen. That's
5:30:39why we've converted it into a string.
5:30:41Now I'll just pass in the value here.
5:30:44Whatever value we extracted and the base
5:30:47directory is going to be self do.t
5:30:49current working directory
5:30:52and that is the overrided value. Display
5:30:55args now contains the true value and
5:30:58that's why I created display ars over
5:30:59here. I didn't want to modify the
5:31:01original arguments dictionary coming to
5:31:04us. Cool. So I want to do this render as
5:31:07table if the display or if the arguments
5:31:11exist in the first place. Let's just say
5:31:12if display arguments exist in the first
5:31:15play otherwise we will just say that hey
5:31:19we we got no arguments because that is
5:31:22also totally possible because many times
5:31:26the LLM does not have to pass in the
5:31:28arguments at all. For example, list
5:31:31directory. If we just want to list out
5:31:33whatever is in our current directory, it
5:31:35will just say list directory tool call
5:31:38it because the default value of that is
5:31:40the current directory. So why does it
5:31:42have to pass in any arguments? And now
5:31:45the style is going to be muted. And
5:31:48that's it. That looks good to me. Now
5:31:52let's just try to make this perfect.
5:31:55I'll pass in the padding which is 1,2
5:31:59similar to the padding that was present
5:32:01for the table. Then we have box is equal
5:32:04to box dot rounded. And we have to
5:32:07import box. So I'll just go at the top
5:32:10from rich dotbox import box and we have
5:32:15to import lowerase box not capital box
5:32:18and then we have box dot rounded which
5:32:20is an enum we want the box to be rounded
5:32:23right then we have created a border
5:32:25style let's just use that border style
5:32:27as well then we need the subtitle the
5:32:31subtitle is this particular thing if
5:32:34it's running or if it's done and what
5:32:36color do we want it in. So, we'll have a
5:32:39text where we pass in running. That's
5:32:42the subtitle. And the style is equal to
5:32:45muted because if it's running, we don't
5:32:48want much focus. If it's done, then it's
5:32:50going to be green. And we'll add that
5:32:51later on. The next thing is title align.
5:32:54We want the title to be aligned to the
5:32:57left hand side.
5:32:59And let's say the subtitle is aligned on
5:33:02the right hand side. Let's call subtitle
5:33:04align like that. Great. So this looks
5:33:08good. Now I can just go ahead and print
5:33:10this. So we have self.conole.print
5:33:13panel. And before we print it out, I
5:33:15would like to leave a new line as well.
5:33:17It just looks good. So let me just print
5:33:20again. And this time it's just going to
5:33:23be empty. We leave a new line. And here
5:33:25we're getting an error saying this
5:33:28parenthesis was not closed. Let me just
5:33:31see where we missed a parenthesis. And
5:33:34it's not really about the parenthesis.
5:33:36It's because I forgot to put a comma
5:33:38here. And now we're good. I think there
5:33:40is a typo as well. So over here I passed
5:33:44in pupil like that. I needed to pass in
5:33:46tupil correctly spelled. And yeah, that
5:33:50looks good enough. Let's just call tool
5:33:51call start over here. So what are the
5:33:54arguments that we need to pass in? The
5:33:56first thing is the call ID. So we'll
5:33:59just have event dot data.get
5:34:02call ID. And if it's not present, let's
5:34:05pass in an empty string. Then we have
5:34:09tool name, which is just the tool name
5:34:12that we have already extracted.
5:34:15Then the third thing is toolkind. That
5:34:17is also something we've already
5:34:18extracted. And the fourth thing is going
5:34:21to be the arguments which is just
5:34:24event.ata.get
5:34:27arguments. And it's an empty dictionary
5:34:30if not present. So that looks good to
5:34:33me. Let me just start this entire thing.
5:34:36Let me clear it off. I can't seem to
5:34:38type today. Let's see. I'll pass in this
5:34:41read main. py file for me. Hit enter.
5:34:44And it says invalid syntax.
5:34:47Order.extend.
5:34:49So that is in TUI. py where I have
5:34:52order.extend.
5:34:54This is incorrect syntax. I need to end
5:34:57the pupil over here itself because we're
5:35:00trying to create a pupil of key and the
5:35:02value right so the pupil ends over here
5:35:04and for every key in the remaining keys
5:35:06we're creating this pupil now let's save
5:35:09it let's run it again and it says cannot
5:35:12import box from rich dobox and yeah it
5:35:16does seem right because rich dobox
5:35:19module does not have any box well file
5:35:22because box is a module right so what we
5:35:25want to do is from rich we'll import box
5:35:27and that exists. So that's good. And
5:35:30here it has defined rounded like we have
5:35:34over here. So that's also nice. Now
5:35:37let's run it again. So it does display
5:35:40it nicely. We have all of the title
5:35:42showing up. Running also shows up. That
5:35:44looks good to me. But the path is
5:35:46incorrectly formatted. And the reason
5:35:49it's incorrectly formatted is because
5:35:51we've been using resolve path. resolve
5:35:53path comes from this part where we're
5:35:55just trying to find out the working
5:35:57directory we are in but we have not
5:35:59passed in but it does not really give us
5:36:02the main py relative to AI agent so
5:36:05we'll have to create a new function that
5:36:08displays the path the way we want so
5:36:11let's go back to path py and this time
5:36:14we'll create a function let's call this
5:36:17display
5:36:18path relative to current working
5:36:22directory Tory where we'll get a path
5:36:24which is of the type is string and then
5:36:26we get the current working directory
5:36:27which is of the type of path and now we
5:36:31are going to return a string where we
5:36:33display the path. Right? So first we'll
5:36:35try to convert this path into an actual
5:36:38path object. So first we'll have a try p
5:36:41is equal to path and then we pass in the
5:36:44path here. And if we run into any
5:36:47exception we are just dealing with a
5:36:49wrong path. So we'll just return the
5:36:52path as it is because let's say it gives
5:36:55us path main. py we're not able to
5:36:58convert it into a path. So we'll just
5:37:00return path as it is. We don't care
5:37:03about anything else. Now if it gets
5:37:06converted into the path object then
5:37:09we'll have a current working directory.
5:37:11Right? If that is present so this
5:37:13current working directory can be null as
5:37:15well. If this current working directory
5:37:17is present then what we'll do is return
5:37:21the path that we had dot relative to the
5:37:25current working directory. Now let's say
5:37:27the current working directory is not
5:37:28present then we'll just have return
5:37:30string of p. So whatever path we are on
5:37:34we'll just return the string of that.
5:37:36But most likely we will have current
5:37:38working directory. So that's good. Now
5:37:41we can just use this function instead of
5:37:44resolve path. Let's import it from
5:37:46utils.path and remove this resolve path
5:37:48function. Now let's try to run it again
5:37:51and see what we get. We get an error.
5:37:53Main. py is not in the subpath of /
5:37:57user/rean/
5:37:58desktop/ AI agent. So the relative to
5:38:02did not work. In that case we can just
5:38:05have a try except here as well. So we
5:38:08try to return path relative to current
5:38:11working directory. Otherwise you know we
5:38:13don't get it. So we have a value error
5:38:16where we don't do anything because later
5:38:18on we're just returning a string of
5:38:20path. Let's try to run it again and
5:38:22we'll just see main. py being displayed
5:38:25out over here and we do see just the
5:38:28main py. But when we have list directory
5:38:31tool come in. So let's say this
5:38:33particular tool is present with us. It
5:38:35will try to list out everything in this
5:38:37AI agent folder. Then it will notice
5:38:39there's a main.py and it will give a
5:38:41better path. And then once it gives a
5:38:43better path, we will have this function
5:38:46display path relative to current working
5:38:48directory nicely working for us. But as
5:38:51of now, this looks good enough for tool
5:38:54call start. Now let's get started on
5:38:56tool call complete. And we know tool
5:38:59call complete actually looks quite
5:39:01similar to tool call start. We just have
5:39:04to display different things in the box.
5:39:07We'll have another L if condition here.
5:39:09If event dot type is equal to event type
5:39:14dot tool call complete and then we'll
5:39:18have the tool name which is equal to
5:39:20event dot data.get name and otherwise if
5:39:24it's not present the value is unknown
5:39:27and then we can go ahead and create a
5:39:29new function in tool call complete in
5:39:32tui called tool call complete which is
5:39:35going to be quite similar to tool call
5:39:37start. So I'll just copy paste the same
5:39:39function. So instead of tool call start
5:39:41we'll call this tool call complete.
5:39:43We'll get the call ID. We'll get the
5:39:45name. But this time we're not interested
5:39:48in the tool kind. We have already cached
5:39:51it in this tool asks by call ID. Right?
5:39:54So we don't really care about the
5:39:56arguments. What we care about is the
5:39:59result. Was it a success? So that's
5:40:02something we'll be getting. And I
5:40:03misspelled it again. Then you have
5:40:06output which is a string. Then we have
5:40:09error which can be a string or a null
5:40:12value. Then we have metadata which can
5:40:15be a dictionary of string, any and it
5:40:18can also be null. Then you have
5:40:21truncated which is a boolean value. It
5:40:25can be any of these things. Cool. So we
5:40:27can remove this line of arguments.
5:40:31Then we have border style like that. And
5:40:34then we need the icon. Was it a success?
5:40:37Was it a failure? What was it? So we'll
5:40:40need a status icon here, which is
5:40:43basically this part over here. Okay. So
5:40:46status icon is equal to a tick mark if
5:40:49it's a success. Otherwise, it's going to
5:40:51be a cross mark. And you can use fancy
5:40:55logos over here. And I do have it with
5:40:57me. I have this tick mark icon. And then
5:41:00I have this cross icon. You can refer to
5:41:03the repository if you want these exact
5:41:05icons or look for more stylish ones. Go
5:41:08ahead. Now, this is the icon. We're also
5:41:12interested in the style. So, if so,
5:41:14we'll create a new variable called
5:41:17status
5:41:18style which is going to be equal to
5:41:20success and I'm misspelling it
5:41:23continuously if success is present
5:41:26otherwise there'll be error. Right? And
5:41:29if you go to agent theme, we've already
5:41:31defined all of these. We have success
5:41:33which is green and error which is bright
5:41:36red bold. Cool. So status style is also
5:41:40present. The next thing is text do
5:41:42assemble where we pass in the status
5:41:47icon that's there. Then we pass in the
5:41:51status style. So if you have a tick
5:41:54mark, you have green. If you have a
5:41:55cross, you have a red. Then you have a
5:41:58name tool. Then you have muted and then
5:42:00you have the call id passed in. Rest of
5:42:02the things remain the same. Now for the
5:42:04displaying part, we will have to extract
5:42:07some data out of here. Right? Because if
5:42:09we have read file, we will need this
5:42:13entire thing to be displayed. What did
5:42:15the agent just read? So over here it
5:42:17read lines 1 to 23 of 23. And this was
5:42:21the path. And what did it read? Well,
5:42:23these files. So we'll have to create
5:42:24kind of a syntax here where it has all
5:42:28of these line numbers and then what was
5:42:31the data associated with it. So we have
5:42:33to display that nicely as well. And it's
5:42:36actually not going to be very difficult
5:42:37for us. The reason for it is we don't
5:42:40have to worry about this entire syntax
5:42:42displaying part because rich already has
5:42:46a class that we can use to display this
5:42:49out. But we do need to get the text that
5:42:53was read from the output. We need to get
5:42:56the start line. We need to get all of
5:42:59the code that was read. Then we also
5:43:01need this path. So yeah, those are the
5:43:03things we need to worry about now. But
5:43:05it's relatively easy. So let's get
5:43:07started over here. So let's just go
5:43:09ahead and create a helper function
5:43:10called extract read file code. And this
5:43:15is going to be a private function. And
5:43:17then we're going to have self and text
5:43:20taken in from the parameter. Text is
5:43:22just the text that we're getting from
5:43:24the output. And I'll discuss how it is
5:43:27structured. Then we're going to return a
5:43:29pupil where the first value is going to
5:43:31be an integer. The second value is going
5:43:33to be a string. Otherwise, we'll return
5:43:35null if that doesn't exist. Now what is
5:43:37this pupil? Pupil where we have the
5:43:41first element as the integer. Why is the
5:43:44first element an integer? it just
5:43:46represents the start line. So if the
5:43:48output is present that means we are
5:43:51going to have a start line and then we
5:43:53have all of the code written out. So
5:43:56yeah that's what we're going to do. Now
5:43:57if you remember the text content that
5:44:00we're going to get from read file is
5:44:02structured something like this. At the
5:44:03top we're going to have something like
5:44:05showing lines and then you have x to y
5:44:08of z something like that where x y and z
5:44:11are just integers. So you're just saying
5:44:14that hey we are showing lines 1 to 10 of
5:44:17100 lines that are present and then you
5:44:19have two back slash ends because you're
5:44:22leaving lines and after that we have the
5:44:25line number followed by the actual code.
5:44:27Let's say main function like that. This
5:44:29is how it's structured. Then maybe you
5:44:32have back slashn again then you have two
5:44:34and then you have this again. So this is
5:44:37the text that we're dealing with. Now
5:44:39what we want to do is extract all of the
5:44:42content out of here. So let's do it one
5:44:45by one. First we'll just focus on show
5:44:47lines of X to Y of Z and we'll be using
5:44:50reg X for this. So let's say we have
5:44:52body equal to text. After that we're
5:44:56going to do a header match using reg X.
5:44:58Now, if you don't already know reg x,
5:45:00I'm not going to dive totally deep into
5:45:03that, but I've created a tutorial on
5:45:04making your own reg x engine where you
5:45:07will be able to understand how to read
5:45:10reg x. So, first we are just going to go
5:45:13at the top and import re because well
5:45:16that's how you use reg x in python. And
5:45:19then we're going to have well where did
5:45:22our function go? Yeah, over here. Then
5:45:24you have re dot match and then you match
5:45:27to a particular pattern. Right? The
5:45:29pattern we're dealing with is something
5:45:31like this. At the beginning of the
5:45:33sentence or the beginning of the text,
5:45:35we're going to get showing lines and
5:45:38then we're expecting a number, right? So
5:45:40to catch a number because we
5:45:43specifically want to catch a number,
5:45:44we're using a capturing group so that
5:45:47the state of that number is saved. So if
5:45:50it's 10 for example showing lines 10 to
5:45:52100 we'll be able to capture the integer
5:45:5510 then you have back slash d plus which
5:45:59just refers to a digit then you have
5:46:01hyphen and then you have another number
5:46:04so showing lines 10 to 100. So we want
5:46:07to capture 100 which is back slash d
5:46:10plus plus just refers to a digit being
5:46:13present one or more times. So it just
5:46:16means we have one digit here or it can
5:46:18be 10. So you have two in total
5:46:21characters present here. Then you have
5:46:23three, four, whatever. But there should
5:46:25be one minimum present. After that we
5:46:29have off and then you again have another
5:46:31digit which is just back slash d plus
5:46:33because we can have multiple characters
5:46:35in a digit here as well. And then you
5:46:38have two back slash ends. Great. And now
5:46:41we're matching this pattern onto a
5:46:43particular text. So we can just pass in
5:46:46the text here. And yeah, we do get that
5:46:48header match. Now, if that header match
5:46:51is present, what I want to do is just
5:46:54trim out my body. So, if header match is
5:46:58present, I'm just going to do body is
5:47:01equal to text from header match dot end.
5:47:06And then we go until the end of the
5:47:09string. Okay, what does this mean? Well,
5:47:13we've matched the entire string over
5:47:15here. Now I just want to go till the end
5:47:18and then we'll be able to get the body
5:47:20again. So I've stripped out the header
5:47:24that has these contents now. So let's
5:47:27say I had something like showing lines
5:47:2910 to 100 of
5:47:33150 and then you have back slash and
5:47:35back slashn followed by line number one.
5:47:38What I've done is just try try to remove
5:47:41all of these things so I just have one
5:47:43so that I can just focus on extracting
5:47:45the line number and the code associated
5:47:48with those line numbers. Cool. So now my
5:47:52body is up to date. The next thing I
5:47:55want to do is well get the code line
5:47:58number. Right? So we have code lines
5:48:01which is going to be a list string.
5:48:04These are all the code related lines. So
5:48:06if you have one def main dot main
5:48:09function like that and then you have two
5:48:11where let's say there's some indentation
5:48:14and then you write in print correct.
5:48:17Let's indent this properly something
5:48:19like this. So I just want def main and
5:48:22print to be present in code lines.
5:48:24Another thing I require is the start
5:48:27index or the start line which is going
5:48:30to be an integer or a null value and
5:48:32initially it's just going to be none. If
5:48:34you want, you can default it to zero as
5:48:37well, but I would like to have this as
5:48:40an integer or a null value. Then what
5:48:43we're going to do is go over every line
5:48:45in body because body is the new text
5:48:47that is removed or stripped the header
5:48:50off. And then we split lines because I
5:48:52want to go over every line, right? So I
5:48:54have to split the body by lines. And
5:48:57remember, every code that's going to be
5:48:59present is going to be on a new line.
5:49:01And that's how we had structured it when
5:49:03we were sending the output.
5:49:05Now we have to match it because I have
5:49:07to remove this one thing over here. So
5:49:11what I'll do is create a reg x here re
5:49:15do match and well what are we expecting?
5:49:18Well we are expecting that in the
5:49:19beginning we're going to get an integer
5:49:22because our code is structured like this
5:49:24right or maybe there's a space over
5:49:26here. I don't remember but something
5:49:28like this. So we might have some space
5:49:31in the beginning then you have digit
5:49:34then you have this or kind of symbol and
5:49:37then you have all of the code that we
5:49:40want to capture and then you should have
5:49:42the end of the line. That's what we're
5:49:44going for. Let me comment this out as a
5:49:47source of inspiration. Now we can give
5:49:49in a pattern. So the very first thing
5:49:51that we want to see is if there's any
5:49:54space present before it because it's
5:49:56totally possible that there's some space
5:49:58present before it. We saw the LLM output
5:50:02when we tried printing out the events.
5:50:04It was really bad looking. So it's
5:50:07totally possible that there's some space
5:50:09before it. And therefore we're going to
5:50:11have at the beginning of the sentence.
5:50:14That's what this current symbol means.
5:50:16At the beginning we might have back
5:50:19slash S which stands for spaces and
5:50:22those spaces can be zero or more. It can
5:50:25be zero spaces 1 2 3 or infinite.
5:50:28Realistically there's not going to be
5:50:29infinite but anyways that's what this
5:50:31means. At the beginning we have spaces
5:50:34but after all of those spaces we'll have
5:50:36back slash D+ because we have a number
5:50:39and that number can be multiple
5:50:41characters as well. You can have 1, you
5:50:43can have 11, you can have 111, you can
5:50:45have 1,111.
5:50:47Depends on how long your file is. After
5:50:50that, we are going to have an or symbol.
5:50:52But if I just put an or symbol, it might
5:50:55mean something else because in regex,
5:50:57this or symbol carries a special
5:50:59meaning. I want to remove that special
5:51:02meaning of this is an alternation symbol
5:51:04in regex, by the way. So, I want to
5:51:06remove that special meaning from this
5:51:09reg. So I'll just put a backslash so
5:51:11that I treat this or symbol as a literal
5:51:14character because I want to match this
5:51:16literal character. I've captured one.
5:51:19Now I've captured this literal character
5:51:22here. And now I can have multiple spaces
5:51:25here as well. But I'm not going to
5:51:27ignore all of those spaces because the
5:51:29indentation matters to me if it's Python
5:51:32code. Right? So if it's def main, I want
5:51:35it to be as it is because in the next
5:51:37line you will have something like this
5:51:40where you have print written out. Now if
5:51:43I strip off all of the spaces here, what
5:51:46I'm telling the LLM or what I'm telling
5:51:49the user is something like this. This is
5:51:52not our code. Our code has indentation
5:51:55within it. That's why what I'm going to
5:51:57do is not ignore the space and directly
5:52:00capture all of the characters. So dot
5:52:03refers to a character and then asterisk
5:52:06means that you can have multiple
5:52:08characters after that and then we're
5:52:10capturing all of that text. So this is
5:52:13stored in state one and this is stored
5:52:15in state two and then we're saying that
5:52:18after this there should be end of the
5:52:20line and we're matching all of that on
5:52:23this particular line that we're going
5:52:25through. Correct? Now if we do not get a
5:52:27match then we'll just return null saying
5:52:30that hey we do not find anything sorry
5:52:34we we are we are bailing out we do not
5:52:36have anything to render so we'll just
5:52:38render the text that LLM gives to us
5:52:42we'll not try to do any fancy stuff on
5:52:44it but anyways if there is a match then
5:52:47we want to first grab the line number so
5:52:49the line number is equal to integer and
5:52:52I told you that we have captured it when
5:52:55you have a parenthesis is around this
5:52:57digit. You're basically saying that I've
5:53:00captured the integer over here. So I can
5:53:03just do m do.group 1 and that refers to
5:53:07this digit that we caught. Then I also
5:53:10need access to the code and code is
5:53:13present in m.group 2 because this dot
5:53:16asterisk meaning we can match multiple
5:53:19characters is within this captured
5:53:22group. Correct? So yeah, we can just
5:53:26store that in code_lines
5:53:30dot append and then you have the code
5:53:33line. Okay. Now if I have line number,
5:53:35what do I want to do? Well, I want to
5:53:37store the start line. So if the start
5:53:38line is null, in your case, if let's say
5:53:42you start from zero, you can just check
5:53:44if start line is zero because anyways
5:53:46starting line should be one. So if start
5:53:49line is none, you have start line is
5:53:52equal to line number. And yeah, that's
5:53:56basically it for this function. So let's
5:54:00just say at the end after the for loop
5:54:03is over. If we do not have any start
5:54:04line, so if start line is none, then
5:54:08this entire parsing failed. I just want
5:54:10to return null. But if the start line is
5:54:13present, then I'll return the start line
5:54:16that I have. And then I also want to
5:54:19return all of the code lines and I want
5:54:22to return it in a string format. So I'll
5:54:24just do back slashn to leave a new line
5:54:26between every one of them so that
5:54:28they're not concatenated together. Now
5:54:30they're concatenated by back slashn a
5:54:34new line. And now you can just pass in
5:54:36code lines. That's it. Now we can take
5:54:39this function and call it within this
5:54:42tool called complete. So you have self
5:54:45dot extract read file code. And now you
5:54:47want to pass in the text. The text is
5:54:50just the output. And we don't want to
5:54:53call extract read file code every time
5:54:55we have a tool call complete because if
5:54:59there's a tool call complete, it can
5:55:01also refer to other operations like
5:55:03shell or
5:55:05write file. And when write file or shell
5:55:08are happening, we don't want extract
5:55:10read file code to be present over there.
5:55:12So what we're going to do is first check
5:55:14that hey if the name of the tool is read
5:55:17file and if it was a success because if
5:55:20it's a failure we do not want to read
5:55:22the output. It's just error filled
5:55:24output.
5:55:26In that case we're going to have
5:55:28self.extrack read file code and then
5:55:31we're going to get the start line and
5:55:34the code. So now that we have the start
5:55:37line and the code with us, I would like
5:55:39to do another thing and that is the path
5:55:43because if you see over here in read
5:55:45file, we do have the path displaying for
5:55:47what file are we displaying the lines,
5:55:50right? I want this path with us and this
5:55:54path is being sent through the tool
5:55:57metadata. So I'll just go to readfile.py
5:55:59py and here when we are having tool
5:56:02result dots success result you can see
5:56:04in the metadata we've passed in the path
5:56:08so we just have to extract the path from
5:56:10metad data now it can be the case that
5:56:13it's not present all the time because if
5:56:15you go to read file here we have
5:56:17metadata path present but if you scroll
5:56:20up when the file is empty we do not have
5:56:23the path passed in so we have to handle
5:56:26that case as well so here I can just
5:56:28extract
5:56:29the primary path that we're dealing with
5:56:32which is equal to null. I'm not doing it
5:56:35within this tool call because this path
5:56:37will also be the case in write file. So
5:56:40it will kind of be reused every single
5:56:43time. So we can just check that hey if
5:56:45is instance metadata is a dictionary and
5:56:50is instance metadata dot get path and if
5:56:56this path that we're trying to get is of
5:56:58the type of string in that case the
5:57:00primary path is equal to metadata get
5:57:04path right then we can just use that
5:57:06primary path to display it out on the
5:57:09screen now there's other stuff to
5:57:11display as well for example example I
5:57:14will have to get all of that shown start
5:57:17shown end kind of things so if you
5:57:19scroll down we had shown start shown end
5:57:23and total number of lines let's just
5:57:25extract all of them as well so we have
5:57:28shown start initially null because again
5:57:31it can be the case that it's not present
5:57:33in the first success result if the file
5:57:36is empty we are not having these
5:57:38properties the total lines is none and
5:57:41now if [snorts] is instance metadata of
5:57:45dictionary that means it is passed in
5:57:48and actually I'm assuming metadata is
5:57:50always passed in so I'm not even going
5:57:52to put that condition in so I have shown
5:57:55start is equal to metadata dot get shown
5:58:00start cool and now I'll extract
5:58:03everything else and since we're already
5:58:05doing that I'll just remove all of these
5:58:07things all right so the initialization
5:58:09is done here itself so we have shown
5:58:12start then you have shown end and then
5:58:15you have total lines and then we can
5:58:18have shown end is equal to that and then
5:58:20total lines is equal to that great so
5:58:24what I did is earlier I was thinking
5:58:26that we can again check if metadata is
5:58:29of the type of dictionary or it is an
5:58:31instance of dictionary but in read file
5:58:34we're always having metadata sent across
5:58:36so what I'm saying is if shown start
5:58:39exists you can set it to this variable
5:58:41able otherwise it will be null. If shown
5:58:43end exists, you'll pass it to this
5:58:46variable otherwise it will be null. And
5:58:48the same for total lines. Now the last
5:58:50thing that I require from here before we
5:58:52can start displaying is lexer. So
5:58:55whenever we're trying to display the
5:58:58code over here, it would be good to know
5:59:00what programming language we're dealing
5:59:02with, right? So it would make sense to
5:59:04just have a function that gives us a
5:59:07sense of what file we're dealing with.
5:59:09And to check that, all we're going to do
5:59:12is check the path. If the path ends with
5:59:14py that means we're dealing with Python.
5:59:16So the syntax highlighting here is going
5:59:19to be Python based. And the same for JS
5:59:22or TS. So what I'm going to do is at the
5:59:25top or probably over here. I don't know
5:59:27where to put it. We can just put it over
5:59:29here. I'll paste a function called guest
5:59:32lexer. So what it does is it takes in a
5:59:35path which is of the type of string or
5:59:36null and it returns a string. Basically
5:59:39the string is going to be what
5:59:40programming language we're dealing with.
5:59:42Now the function guess lexer is probably
5:59:44not right. You can kill it call it guess
5:59:47language. That makes more sense or guess
5:59:49programming language and it is going to
5:59:52take in self and then it takes in a
5:59:54path. If the path is not present it just
5:59:56says return text because well if the
5:59:59path is not present how are we going to
6:00:00find out what programming language we're
6:00:02dealing with? So I'll just say this is
6:00:04text because text is essentially no
6:00:07syntax highlighting kind of thing. Then
6:00:10your suffix is equal to path at path. So
6:00:13we're just creating a path object dot
6:00:16suffix just gives us the extension of
6:00:19the file we're dealing with and we
6:00:20convert it into lowerase. And then we
6:00:23have a dictionary here defined. You can
6:00:25add more but I think this is quite a
6:00:28good list where you have Python,
6:00:30JavaScript, TypeScript, JSX, Go, Java,
6:00:35HTML, TOML, YAML, all of that. And then
6:00:38you call.get
6:00:40suffix on it. So if you have py it will
6:00:43just return this Python string to you.
6:00:45If you have C, it will return C to you.
6:00:48And then you have well if you don't have
6:00:51any of these options it will just return
6:00:53text because yeah that makes the most
6:00:55sense and that is it about guess
6:00:58language. Now we have to indent this
6:01:00properly. So yeah that's it. This is why
6:01:02I copy pasted it. It's just a long
6:01:04dictionary with dot get called on it.
6:01:07Now I'll just call this function within
6:01:09read file. So we have self dot guest
6:01:12language and then we need to pass in the
6:01:14path. What is the path? It's just the
6:01:17primary paths that we had. So if the
6:01:20primary path does not exist, it will
6:01:21just return text to us. And that is our
6:01:24programming language. Now we can go
6:01:26ahead and try to display something. So
6:01:28what are we trying to display? Well, it
6:01:31looks something like this, right? So
6:01:32first we need an empty line. After that
6:01:35we going to add this particular header
6:01:37where we have the path. We have a dot
6:01:39symbol. So let me just copy this dot
6:01:41symbol. And then we have lines 1 to 23
6:01:43of 23.
6:01:45should be quite easy. First of all, we
6:01:48want an empty text at the top. Define a
6:01:52blocks list. So blocks is going to be an
6:01:55empty list. And this blocks will contain
6:01:58all of the things. So for example, we'll
6:02:01have blocks.append text because maybe we
6:02:04want to display an empty line. After
6:02:07that, we're going to have the header.
6:02:10The header needs to have the path and
6:02:12we've created a function for that. We
6:02:14have display path relative to current
6:02:17working directory which we're going to
6:02:19use. We're going to pass in the primary
6:02:20path as the path and the current working
6:02:24directory should be self dot current
6:02:27working directory and this is going to
6:02:29be in header parts. So header parts is
6:02:31also going to be a list. At the end
6:02:33we'll just concatenate it. The first
6:02:35part is whatever path we have over here.
6:02:38After that we're going to get the dot
6:02:41symbol here. So we have header parts dot
6:02:46append and then we append this dot.
6:02:49After that we are going to have the
6:02:51lines 1 23 of 23. Also let's just put in
6:02:55some space between this. And now we can
6:02:57check that hey if shown start is present
6:03:00and shown end is present and total
6:03:03number of lines is present then it's
6:03:05very easy for us. We'll just have header
6:03:07parts dotappend and then we'll append
6:03:10lines shown start to shown end of total
6:03:18lines. Great. Now we can just join all
6:03:21of these header parts to create an
6:03:23actual header which is just like this.
6:03:25Dot join header parts. And yeah, that
6:03:29looks good enough. Now I'll append to
6:03:31blocks this header. So I'll have
6:03:33blocks.append append text and then I'll
6:03:36append this header and the style of this
6:03:39is going to be muted because as you can
6:03:41see it's somewhat grayish and finally
6:03:44we're going to add our own syntax here
6:03:47so we'll have blogs.append append syntax
6:03:50and at the top we can import from rich
6:03:53dot syntax import syntax. Now we can use
6:03:58it in tool call complete and I have to
6:04:01remain rest of these things but first
6:04:04let me just focus here syntax what does
6:04:06it require? It requires a code and we
6:04:10have extracted the code outside which is
6:04:12this particular thing. Then I need to
6:04:15pass in the lexer. Well, we could have
6:04:18called it Lexter, but yeah, this is the
6:04:20programming language that we are dealing
6:04:22with. Then we have all the other things.
6:04:25For theme, you can use anything you
6:04:27want. I'm going to go with Monokai,
6:04:29whatever is present over here. But yeah,
6:04:32it's up to you to make your thing look
6:04:34good. After that, you have line numbers.
6:04:36Should it be true? Yes. Then we need
6:04:39start line number and we have captured
6:04:42that as well. And then finally, word
6:04:44wrap is equal to false. We don't want it
6:04:46to wrap. And that's about the blocks
6:04:49that we'll require for read. But there's
6:04:51one edge case. What if the primary path
6:04:54does not exist? In that case, we'll run
6:04:56into an error quickly because this path
6:04:59is of the type is string. So if null is
6:05:02passed in, it will return an error. So
6:05:04what I'd like to do here is check that
6:05:06hey if primary path is present in that
6:05:08case we're going to do all of these
6:05:11things
6:05:13otherwise we don't have a primary path.
6:05:16So we're going to get and display the
6:05:19raw output. Now to display the raw
6:05:22output we'll first have to truncate
6:05:24whatever we want. So I can call the
6:05:26truncate function that we created in
6:05:30text py I believe. So we add truncate
6:05:33text. Let me just copy that. Let me get
6:05:36it from utils.ext. Now here I need to
6:05:39pass in the text which is the output.
6:05:43Then you have the model name. Well, the
6:05:46model name can be passed in whatever.
6:05:49Then you have the max tokens. Let's say
6:05:52it is 240.
6:05:54You can make this configurable by the
6:05:56user. And we're going to work on
6:05:58configuration system. I know I've hyped
6:06:00it a little bit too much, but yeah,
6:06:03we'll get to that. And this is the
6:06:05output display. Cool. And then I can
6:06:08just do blocks dot display or
6:06:12blocks.append, sorry. And then I have
6:06:14syntax passed in where I need to pass in
6:06:17the code which is just the output
6:06:18display, the truncated output display.
6:06:20And then you have text as the lexer or
6:06:26the programming language that we using.
6:06:29Then you have the theme again as
6:06:32Monokai. Let's use that. And then the
6:06:35word wrap is again equal to false. We
6:06:38don't need to specify the start line
6:06:40because we don't know what the start
6:06:42line is. Now I like to display this in a
6:06:44panel. So I'll just remove all of these
6:06:46display arguments that we have. If we
6:06:50have a panel, then I just want to
6:06:53display all of the blocks within it. So
6:06:56what I'll do is group all of them
6:06:58together. Now I have to import group. So
6:07:01from rich I can import group. And I
6:07:04think we already have box. So I can
6:07:06import group from here. Or actually it
6:07:08comes from rich dot console. So we have
6:07:10from rich do console import group. So
6:07:14group just takes a group of renderable
6:07:16and returns a renderable object. So it's
6:07:19kind of like if you have multiple
6:07:21renderables like a text, a panel or
6:07:24whatever, it can take multiple one of
6:07:26these and just create a wrapper around
6:07:28it so that you can display all of them
6:07:30together. In our case, we have a list of
6:07:33renderables called blocks. So we are
6:07:36going to use that and wrap it within a
6:07:38group. Okay? And then I can pass in all
6:07:41of the renderable which is like this. So
6:07:44I've dstructured all of the blocks
6:07:46within a group. Now the title is present
6:07:49with us. The title align is left. The
6:07:52subtitle is not done anymore. Is not
6:07:55running anymore. It's going to be done.
6:07:57And it's going to be done if we are in
6:07:59success. Otherwise, it is going to be
6:08:02failed. And the style is going to be
6:08:05well the status style that we have
6:08:08already talked about before and we
6:08:09created a variable for it. Status style.
6:08:12So a green or a red depending on success
6:08:15or error. Then you have subtitle align
6:08:19right. Then you have the border style.
6:08:21Then you have rounded and then the same
6:08:23padding. Now yeah we print out the
6:08:26panel. That looks good to me. One thing
6:08:28I would like to do is what if it's
6:08:31truncated right? If the tool does any
6:08:33sort of truncation we would like to
6:08:36display that in the output. So we'll
6:08:38have if truncated is true in that case
6:08:41we have blocks do.append append and then
6:08:44I'll have a text with the note saying
6:08:46that hey the tool output was truncated
6:08:50and then we have the style of that the
6:08:54style is just going to be a warning
6:08:56message that hey your output was so long
6:08:59that it was truncated that is a good
6:09:01enough indication for the user now we'll
6:09:04go back to main py where we're going to
6:09:06call this self tui tool call complete
6:09:10and then we have the call ID passed in
6:09:13which is event.data.getcall
6:09:15id. That's correct. Then we have the
6:09:17tool name. That's what's required here.
6:09:20Then we have the tool kind
6:09:23but we have not extracted the tool kind.
6:09:25So we'll have to extract it. And it's
6:09:28the same logic as this one. So what I
6:09:30can do is extract this outside in a
6:09:33helper function. So we'll have def get
6:09:37tool kind then you have self and it
6:09:41returns either a string or a null value
6:09:44and then you just have tool kind
6:09:46returned like that. So you return
6:09:49toolkind. Okay. So by default tool kind
6:09:52is none. Then from the tool registry we
6:09:55are getting based on the tool name. So
6:09:57we'll get tool name from the parameter
6:09:59and it's not called tool king. It's
6:10:02called tool kind. If the tool is not
6:10:04present then we have toolkind is equal
6:10:06to null. Otherwise we have tool.kind dot
6:10:08value and then we return the toolkind.
6:10:11Let's just get the toolkind over here.
6:10:13In both of these cases actually so we
6:10:15have tool kind is equal to self dot get
6:10:18toolkind. Then we pass in the tool name.
6:10:21Let me copy this and then paste it
6:10:23again. So the tool kind is now passed
6:10:26in. After that we're going to get the
6:10:28success right the success output error
6:10:31metadata and truncated that is all
6:10:34related to event dodata. So I'll have
6:10:37event data.get we don't have arguments
6:10:40anymore. We have success. If it's not
6:10:43present it's going to be false.
6:10:46Similarly we're going to paste it for
6:10:48output. If it's not present we have an
6:10:51empty string. Then we have error. If
6:10:55it's not present, we are going to have
6:10:58null returned. Then we have meta data.
6:11:01Then we have truncated, which is false
6:11:04by default if it's not present. And I
6:11:07think that's it. Yeah, by the way,
6:11:10metadata cannot be false. Metadata
6:11:12either has to be an object or it has to
6:11:15be null. So, we'll just set it to null
6:11:17if we don't find it. And that looks good
6:11:20to me. Let's try to run it and see how
6:11:23it works out. So let's run this read
6:11:26main. py file for me.
6:11:29And we run into an error. It says
6:11:32sequence item zero expected string
6:11:34instance posath found. That means we
6:11:38somewhere used a posex path especially
6:11:42when we had blocks. So here we did
6:11:46display path relative to current working
6:11:48directory. And I think the issue is
6:11:50actually in display path relative to
6:11:52current working directory. So here we
6:11:54created a path and then we return
6:11:58p.relative2
6:12:00in case the current working directory is
6:12:03present. But the problem is this returns
6:12:07a path and we actually want to return a
6:12:10string from here. So what I can do is
6:12:12pass in a string and yeah that should be
6:12:15good. So the path is converted into a
6:12:17string and then returned from this
6:12:19function. Now I can try running it
6:12:22again.
6:12:24And as you can see, it all works out. It
6:12:27does look a bit weirder than what we
6:12:30have over here. And that's solely
6:12:31because we have one extra line. Since we
6:12:34already created a padding, we don't have
6:12:36to add the text ourselves in the TUI. So
6:12:39here we can remove the blocks.append
6:12:41text because the padding is present. Now
6:12:44let's try running it again. And yeah,
6:12:46that works out again. Looks much better.
6:12:50Then we have a status symbol read file
6:12:52and the same tool call ID. Then you have
6:12:55the main.py where it says lines 1 to 93
6:12:59of 93 and then it prints out everything
6:13:02nicely. If the lines kind of exceed 240,
6:13:05we'll also be truncating. So that's
6:13:08good. Now what I want to do is instead
6:13:10of just having a run single mode, which
6:13:12seems to be working well, I would like
6:13:14to convert it into a run interactive
6:13:17mode. So I'll close all of the save
6:13:19files for now. I'll go to main. py and
6:13:22here we have run single function.
6:13:25Correct. Similar to this I would like to
6:13:27create a run interactive method as well.
6:13:31So here I'll just copy everything paste
6:13:34it over here. And then you have run
6:13:37interactive.
6:13:39Run interactive does not take any
6:13:41message or anything because run
6:13:42interactive is essentially this
6:13:44particular place where I just start the
6:13:47agent once it gives out this message
6:13:50symbol then I can pass in any command
6:13:53that I wanted to so we have async with
6:13:56agent as agent then we set agent
6:13:58correctly but then we'll have an
6:14:01infinite loop in this infinite loop
6:14:03we'll continuously get the user input
6:14:05and handle the commands and obviously
6:14:08also process the user's message. So
6:14:10first we need to get the user message.
6:14:12So we have while true because we are in
6:14:14an infinite loop and then we get the
6:14:16user input. Now there can be an error
6:14:19over here because when we try to get the
6:14:21user input which is done using console.
6:14:24So you have console.input the user might
6:14:27just click ontrl c to exit out of here.
6:14:30Let me give you an example. So let me
6:14:33run this example code again. And now if
6:14:36I want to get out of here, remember we
6:14:39are in a while loop here. We're
6:14:40continuously getting the user's input.
6:14:42I'll just press Ctrl C and whenever I
6:14:45press Ctrl C, I run into what's known as
6:14:48a keyboard interrupt error. And I would
6:14:51like to avoid that and instead say that
6:14:53hey, you have to use back slashexit to
6:14:56quit or I can just quit. So yeah, let's
6:14:59work on that. And it's quite easy. We
6:15:01can just put a try block here. Then we
6:15:03get the user input. Then we have an
6:15:05except keyboard interrupt error. If that
6:15:08is the case, I can just do console.print
6:15:11back slashn and then we're going to have
6:15:13a dim as the style. Then you have used
6:15:17back slashexit to quit not continue. And
6:15:22then you have forward slash dim. Again,
6:15:24of course, this cannot be exist. It
6:15:26needs to be exit. And yeah, that looks
6:15:29good to me. Now, another error we can
6:15:31run into is end of file error. If that
6:15:34is the case, I just want to break out of
6:15:36here. Now, let's get the user input. To
6:15:39get the user input, it's quite simple.
6:15:41We have just this symbol showing up,
6:15:44which is just a greater than symbol, I
6:15:46think, and it needs to be colored in
6:15:48blue. And we know that user is converted
6:15:51into blue. Also, we want it to be on a
6:15:53new line. So, that's there. Then, you
6:15:56have greater than symbol, and then you
6:15:57have another user again. Then you leave
6:16:00some space and then you strip off the
6:16:03user's input. Okay? So we get the user's
6:16:06input. Whatever they type in, we just
6:16:07remove the trailing spaces from it or
6:16:11leading spaces from it. And we have our
6:16:14user input. Now if the user input is
6:16:17empty, in that case we want to continue.
6:16:21We are not going any forward. The user
6:16:22just typed in nothing. Let's try over
6:16:25here. If I click on enter, you see
6:16:27there's a line space and then we move to
6:16:30the next part because the user did not
6:16:31explicitly say anything. Why should we
6:16:33give it to an LLM? It's just a waste of
6:16:35time and tokens. After this, if this you
6:16:38know input is valid, we just going to do
6:16:41return await self.process message and
6:16:45let's not return that. We'll just do
6:16:46await self.process message. And then we
6:16:49pass in the user input. And let's say
6:16:51after that the agent gets over you know
6:16:54this entire context management that we
6:16:57have done is over then what I want to do
6:16:59is console.print and then I have back
6:17:02slashn again in dim light I want to say
6:17:05goodbye and then we have forward slash
6:17:08dim. Obviously this needs to be within a
6:17:11string and that's it. It's just a nice
6:17:13effect that we're going after. It's not
6:17:15a necessity. Now let me just start this
6:17:17in interactive mode. And by the way, we
6:17:20have to connect this to our main
6:17:23function. Meaning if the prompt is not
6:17:25specified, then we have an else
6:17:27condition. And then we have async
6:17:30io.run. And then I'm just running the
6:17:33CLI in interactive mode. Then yeah, so
6:17:37let's try to run it. What I'm going to
6:17:39do is just say python main.py because
6:17:42that should open up in interactive mode.
6:17:45And we get an error. But it also works
6:17:47out. The error is syntax warning invalid
6:17:50escape sequence back slashe and this
6:17:54exists because here we said use back
6:17:56slashexit to quit. Actually it should be
6:17:59forward/exit. That's how you quit. Yeah,
6:18:01we have to change that. But also the
6:18:03main problem here is back slash e is
6:18:06considered to be an escape sequence but
6:18:08it's an invalid. doesn't really exist in
6:18:10Python because anything after backslash
6:18:13is supposed to be escaped and that's why
6:18:16the issue existed anyways for us we just
6:18:19have forward slash so yeah now let's try
6:18:22to run it again and you can see it says
6:18:25use /exit to quit now I don't really
6:18:28have a way to quit because we have not
6:18:31implemented the back/exit or
6:18:33forward/exit command this aborted
6:18:37because that's how the CLI does it, but
6:18:39we do not have anything related to
6:18:41forward slashexit. So yeah, all of those
6:18:44command handling sequences, we'll do
6:18:46that later on. It's not top of our
6:18:48priority list. For now, what I'd like to
6:18:51do is just get this interactive thing
6:18:53working. So yeah, the agent has started,
6:18:55but there's really no way for me to know
6:18:57that the agent started over here in our
6:19:01done example. When we start, it gives us
6:19:04this entire list. What model are we
6:19:06using? What current working directory
6:19:08are we using? What are the commands that
6:19:09we can use to get some help? I want this
6:19:12exact thing. Now, I'm not going to code
6:19:15it from scratch because it's quite
6:19:17simple actually. I'm just going to copy
6:19:20paste in the TUI. So, if we go back to
6:19:22our TUI. py here, we're going to create
6:19:25a function that will print welcome. And
6:19:28it's nothing complicated. The reason I'm
6:19:30printing it outside over here is because
6:19:33it's not too complicated. We have body
6:19:35is equal to back slashn dot join and
6:19:38we're joining all of the lines. The
6:19:40lines is given to us from the parameter
6:19:44itself. Okay, we can also remove this
6:19:46asterisk because we're not going to have
6:19:48any other arguments here. Now after that
6:19:52that is our body. So we're just doing
6:19:54self.conole.print
6:19:56a panel and we know this is a panel. We
6:19:58have already created it. The border is
6:20:02border. So it's just the grayish border
6:20:04type thing. Then you have left. You
6:20:06don't have any subtitle here. The
6:20:07padding remains the same. The text is of
6:20:10the style of highlight. If you go to
6:20:12agent theme, you'll be able to see what
6:20:14highlight means. Bold can. These are the
6:20:18things. Now we can go back to main. py
6:20:20and print this out as soon as the
6:20:22interactive mode is called. So here I'll
6:20:24just do self.tui dot print welcome. And
6:20:29now I'll pass in the title. The title is
6:20:32well AI agent. I've not given it a fancy
6:20:34name. You can. And then I have lines as
6:20:37a list where the first one is the model.
6:20:40What model are we using? Now that really
6:20:44depends on the configuration. But since
6:20:47we don't have that yet, we are going to
6:20:50add it just after this interactive mode
6:20:52is done. I can just pass in model as
6:20:55this mistral thing by default you know
6:20:59hardcoded. Then you have current working
6:21:01directory. This also comes from the
6:21:04configurator but we can just do path dot
6:21:08cwd for now and obviously it will get
6:21:11converted into string. And the final
6:21:13thing is commands. What all commands do
6:21:16we have? forward slashhelp slashconfig
6:21:19slapproval
6:21:21slashmodel and forward slashexit. Just
6:21:24to let you know what all of these
6:21:26command mean /help will list out all of
6:21:29the commands which will help us do
6:21:31whatever we want to do. Config will list
6:21:34out all of our configurations that we
6:21:36have defined in our config.l file that
6:21:38we're going to create. Then we have
6:21:40forward/approval
6:21:42where you can see what approval modes
6:21:44you have. Then you have forward
6:21:46slashmodel which which refers to a model
6:21:50change. For example, this model is used.
6:21:52You might want to change it. So you can
6:21:54just do forward/model and change it on
6:21:57the fly. And then forward slexit. You
6:21:59know exactly what it does. So that's
6:22:01looking good for us. Let's just exit.
6:22:04And to exit, I can just do this. It
6:22:07aborts.
6:22:09Our command obviously doesn't work
6:22:11because if our command worked, it
6:22:13wouldn't say aborted. It would say
6:22:16goodbye just like we have over here.
6:22:18Graceful handling aborted is what TUI or
6:22:21the rich thing that we using says. Now
6:22:25let's run it again and it gives us this
6:22:28AI agent. Excellent. Now let's give it a
6:22:31query. Read main.py file for me. So
6:22:35let's hit enter. And it should work. It
6:22:38does. And then it gives me another
6:22:40message that I can put in. It gives out
6:22:43this read file. Then it gives out this
6:22:45read file just like run single mode
6:22:47would. And the good thing over here is
6:22:49that the context is managed. So if I
6:22:51type in here what did you do before and
6:22:54then hit enter it gives us an error
6:22:57unexpected role tool after roll. Now the
6:23:00reason this error occurs is because if
6:23:03you go to our agent py file here we did
6:23:07this right. we contacted the LLM then we
6:23:10added the assistant message if the
6:23:12response text was available otherwise we
6:23:15just passed in a null and if it's a null
6:23:18then well nothing really happens one
6:23:21thing that we forgot to do is tell that
6:23:23the assistant gave us the tool calls
6:23:26right so think about it we sent a
6:23:29message to the LLM saying hey we want to
6:23:33read the file so we read the main py
6:23:36file okay the lm LM tells us that this
6:23:40is the tool call you have to do. So we
6:23:42have not registered in our context what
6:23:44the assistant or the LLM told us to do.
6:23:47We have only registered the response
6:23:49text. So for example if I type in hey
6:23:52what's up it gives me hello how are you
6:23:54doing. So that hello how are you doing
6:23:56is stored in the context manager with
6:23:59the assistant tag. But if there's any
6:24:02tool call we're not registering that in
6:24:04the context and we have to do that. And
6:24:07to do that, we'll have to go to add
6:24:09assistant message. In the message item,
6:24:12we'll have to pass in the tool call ID
6:24:15and tool calls. We're not really
6:24:17interested in tool call ID when we have
6:24:20an assistant message. We are interested
6:24:23in it when we are adding the tool
6:24:24result. But in assistant message, we do
6:24:26want to put in tool calls. So from the
6:24:30parameters we're going to take in tool
6:24:32calls as a list of dictionary of string
6:24:36comma any. I hope you've understood why
6:24:39we need to do this. It's just so that in
6:24:42the context we are maintaining the
6:24:44assistant related text. So we given a
6:24:46message saying readmain. py whatever the
6:24:50assistant gives us that also needs to be
6:24:52stored in context and then the result of
6:24:54that tool execution also needs to be
6:24:56stored in the context. Everything goes
6:24:58in the context but we missed out on one
6:25:01part which is well whatever the
6:25:04assistant gave us. We just store the
6:25:07content. We don't store the tool related
6:25:09stuff. And that's what we're doing here.
6:25:12And now we can pass in tool calls which
6:25:16is equal to tool calls. And by the way
6:25:19the tool calls can also be null. And by
6:25:21default it will be null. So if it's not
6:25:23present then it's going to be an empty
6:25:25list because we cannot pass in null for
6:25:27tool calls or it can be an empty list.
6:25:30You can also pass in null value. It's
6:25:32totally fine. But yeah this just gives
6:25:34it a nice feeling because ultimately it
6:25:36will be converted into dictionary and in
6:25:38dictionary we are just checking that hey
6:25:40if selftool calls is present that means
6:25:42it should not be empty list or it should
6:25:44not be null. Great. So we have managed
6:25:47that add assistant message. Now we can
6:25:50go to the agent. Here we have to pass in
6:25:52the list of tool calls. So I'll have a
6:25:55list where I have a dictionary. The
6:25:57first thing is going to be ID. Well, it
6:26:00what is the ID going to be? Well, we're
6:26:02going to go over every tool call in tool
6:26:04calls, right? So it's going to be TC dot
6:26:08call ID. Then we have type which is
6:26:10going to be function. And then we're
6:26:13going to pass in the function which is
6:26:15going to be the name which is tool
6:26:18call.name. Then you have arguments which
6:26:21is going to be string of tool call
6:26:24dotarguments. That looks good enough.
6:26:27Now we only want to create a list if
6:26:29tool calls exists, right? So if tool
6:26:32calls is present only then do we have
6:26:35this entire list otherwise we have null
6:26:39and that is it. Now I hope this entire
6:26:43thing works out for us. So I would like
6:26:45to exit. I'm in interactive mode. I'm
6:26:48saying read main. py file for me. Hit
6:26:52enter. And it does give out the entire
6:26:55thing. That's working as expected. Now
6:26:58let's hit enter and see if the LLM
6:27:00realizes what we are saying. And it
6:27:02gives us a message. I read the contents
6:27:04of the main. file for you. That's
6:27:06awesome. I'll just say read it again
6:27:09just to see if the context is again
6:27:11maintained. I'll hit enter. And yep, it
6:27:14does execute it again. I said read it
6:27:17again. It realizes that it means main.
6:27:20py. So everything is maintained. So that
6:27:23means our agent loop is working. Now
6:27:25there's obviously more stuff to do. And
6:27:28by the way, forward/exit did not work.
6:27:30It says exiting goodbye. So now the next
6:27:33step for us is creating the
6:27:34configuration system because the
6:27:37configuration system has a lot of
6:27:38details that we care about. For example,
6:27:41when we are in the agentic loop, this
6:27:43loop can keep going on forever, right?
6:27:46We need to put a max limit to it because
6:27:49it can cost a lot of tokens to us as
6:27:52well. And obviously there's going to be
6:27:54all sorts of context management stuff
6:27:56we'll do later on. But the point here is
6:28:00configuration system handles a lot of
6:28:02things including setting up the approval
6:28:05and sandbox policies which is a feature
6:28:07we'll add later on. So it just makes
6:28:09sense to work on configuration system
6:28:12now and then whatever features we need
6:28:15we'll keep on adding them as soon as we
6:28:17need it. Also it requires kind of a
6:28:20refactoring of our system because many
6:28:23places we just passed in hard-coded
6:28:26strings like for the model name we just
6:28:29passed in hard-coded strings. That's not
6:28:31what we want and that's why we'll have
6:28:34to touch many files in this refactoring.
6:28:37Let's close all the save files. And
6:28:39here, let's minimize all of the folders.
6:28:41I'll create a new folder called config.
6:28:44And within this config, we're going to
6:28:46have two files. One is going to be
6:28:48config. py which defines how the
6:28:50configurations are going to look like.
6:28:52It will create a schema of
6:28:54configurations. And then you have a
6:28:56loader. py which contains all loading
6:29:00related aspects of the configuration
6:29:02because the configuration can be present
6:29:05in your TOML file right and to ML file
6:29:08needs to be defined in the root of the
6:29:10folder root of your entire system so all
6:29:13of that loading also takes some time
6:29:16more than time I'm talking about code so
6:29:18it just it is cleaner to just put it in
6:29:20two different files let's work on that
6:29:22we'll start from the config py file so
6:29:25here we are going to create a task
6:29:26called config and then this is going to
6:29:29extend the base model and from pantic we
6:29:33are going to import base model and we
6:29:35can't do that so let's do from pantic
6:29:39import base model once we do that we
6:29:43have the class created and now what is
6:29:45the first configuration that we're going
6:29:47to take in it's related to the model and
6:29:50for this we're going to create a new
6:29:52class called model config and by default
6:29:55this is going to instantiate
6:29:59a model config. So let's have default
6:30:01factory here which is equal to model
6:30:04config itself. Now let's import field
6:30:07from paidantic and also create the class
6:30:09for model config. The model config class
6:30:12is also going to extend base model. Now
6:30:15what is this model config class? Well,
6:30:18the reason we're creating a separate
6:30:20class is because when we try to load
6:30:23stuff up, everything related to a model
6:30:26will remain within this class. Okay? And
6:30:29then we can just pass in all of the
6:30:31properties a model can have. It's just
6:30:33easily extendable then. So we have name
6:30:36which is going to be a string and by
6:30:37default it is equal to well the mistral
6:30:40model that we've been talking about. So
6:30:42I can search the entire codebase to find
6:30:44mistral related stuff and it's this
6:30:47particular thing. So let me just copy it
6:30:50and then I'll paste it over here. So if
6:30:53nothing is mentioned we use this model.
6:30:56Then we have the temperature which is
6:30:58going to be a float value and that can
6:31:01be a field as well. And then in the
6:31:05field the default value can be passed in
6:31:07which is one. So if the user does not
6:31:09specify temperature, we go with one.
6:31:12Then we can also pass greater than equal
6:31:15to. So any value should be greater than
6:31:17equal to 0.0 and it should be less than
6:31:20equal to 2.0. That's how temperature
6:31:23works. By the way, if you're not
6:31:24familiar with temperature, temperature
6:31:26just means the creativity of the model.
6:31:29You can think of it that way. How? So
6:31:31you know LLMs are next word predictors,
6:31:34right? If there are next word
6:31:36predictors, what is the probability of
6:31:38the next word going to be? Temperature
6:31:40controls that. It just controls how
6:31:43creative can the model be? If it's one,
6:31:47it will choose somewhat of a creative
6:31:50option. If it's set to zero, it will
6:31:51choose the most likely option. So as a
6:31:55thought experiment, if you set the
6:31:57default to zero and the user doesn't
6:31:58change the temperature, it just means
6:32:00that the model will always give a
6:32:03similar or the same output. In practice,
6:32:06it rarely gives the same output because
6:32:09there's some indeterminism within the
6:32:12model. If you want to read more about
6:32:14that, I can mention an article by
6:32:17thinking machines of why this
6:32:19indeterminism occurs. But that's totally
6:32:22possible. The point is zero will try to
6:32:25always choose the most likely word. One
6:32:28gives it a more creative sense and two
6:32:32is the most creative it can get. So many
6:32:34times with two you'll have the model
6:32:36hallucinating but it will be very
6:32:38creative because think about it if it's
6:32:41told to choose the least likely element
6:32:44for example the least likely word it
6:32:47will just generate deformed sentences.
6:32:49Obviously that does not happen here but
6:32:52it will hallucinate. It might give you
6:32:54proper English but it will not make
6:32:57sense for example. So that's temperature
6:33:00and the other thing is context window.
6:33:03The user can set the context window
6:33:04length which is an integer or a null
6:33:07value and by default it is null because
6:33:11well context window is interesting. By
6:33:14the way if you want you can remove this
6:33:15null and set it to 256,000 or 128,000.
6:33:19That's what most models are around. I
6:33:21believe this model is 256,000 tokens
6:33:24context window. If you're not familiar
6:33:26with context window, it just means how
6:33:28many tokens can this chat GPT
6:33:30conversation keep up with. Most models
6:33:33are increasing nowadays. I think we
6:33:36started off with 64,000 or 128,000 and
6:33:39now the GPD 5.2 model is probably around
6:33:42500,000 tokens of context window. So,
6:33:45it's going up, but it is still finite.
6:33:48It's not unlimited, infinite. You cannot
6:33:51just paste in a 100page PDF and expect
6:33:54it to work. That's why context
6:33:57management is one thing we'll be
6:33:58focusing on in this tutorial. Cool. Now
6:34:01that we have model config, we can just
6:34:03go ahead and set another one which is
6:34:05current working directory and it is
6:34:08going to be of the type of path. Let's
6:34:10import path from path lib which is equal
6:34:12to and then we have a field where we
6:34:16pass in the default factory which is
6:34:18going to be path cwd. Notice I'm not
6:34:21initializing stuff over here. I'm just
6:34:23mentioning the function so that whenever
6:34:26current working directory needs to be
6:34:27initialized it can be passed in. Then we
6:34:30have integer like max turns integer
6:34:33which is equal to 100. What is this max
6:34:35turns? Well, max turns refers to how
6:34:38many turns can our agent go long for. So
6:34:42for example, when we try to run our
6:34:44application over here, I sent in one
6:34:46message, right? And then I could send in
6:34:49another, then I could send in another.
6:34:50We want to put some sort of limit over
6:34:52here, correct? Because then it will just
6:34:55keep going on forever. That's why this
6:34:57100 is the default limit we are putting
6:34:59in. But if the user wants, they can
6:35:02increase it. Obviously, it's going to be
6:35:04based on their own API key. This is the
6:35:07bring your own API key format. We don't
6:35:09worry about the cost. We just set up a
6:35:12framework for them to use it. So, the
6:35:14cost will be associated with them. But
6:35:17we have done our due diligence of
6:35:18setting the max number of turns. Then we
6:35:21have max tool output tokens. How many
6:35:25tokens should the tool go on for? And
6:35:29we're going to set it to 50,000. So if
6:35:32the tool return 50,500
6:35:35tokens, we'll just truncate it down to
6:35:3750,000 tokens. The 500 tokens will be
6:35:40discarded. Then we have instructions
6:35:43like developer instructions which is I
6:35:46think mentioned through the agents.md
6:35:49file and then the user can also specify
6:35:51user instructions like how do you want
6:35:54the model for example to behave. Now
6:35:58they both kind of overlap each other
6:36:00because developer instructions can also
6:36:02contain how the model should behave
6:36:04because at the end of the day both are
6:36:06just combined and sent to the system
6:36:07prompt. But this is just a clear
6:36:09distinction done. If you want you can
6:36:11totally ignore this and yeah so we have
6:36:14this also done and finally we're going
6:36:17to have a debug boolean value which is
6:36:19going to be false by default. This means
6:36:22if the debug should be on or not. This
6:36:24will help us display the logger related
6:36:27instructions. So if you do d-debug, all
6:36:30of the nice UI stuff that we're doing
6:36:32will be gone and we'll only display the
6:36:35logger stuff. So yeah, that's it. Now
6:36:38there are more attributes that we would
6:36:40like to add here. For example, approval
6:36:43sandbox, how the shell environment
6:36:45should be, then all of the stuff related
6:36:48to hooks, MCP servers. So yeah, those
6:36:51all things are going to be added but
6:36:54we'll add them as and when we want the
6:36:56feature as a whole to be present in our
6:36:58application. As of now, these are the
6:37:00things that we need to worry about. Now
6:37:02let's go ahead and create a property. So
6:37:04the first property is going to be the
6:37:07API key because the API key needs to be
6:37:10taken from the environment, right? So we
6:37:13have a self and we will return either a
6:37:15string value or a null. And this just
6:37:18means that we are checking our
6:37:20environment variables. So we import OS
6:37:23dot environment and then from the
6:37:26environment we're trying to get the API
6:37:29key. So that is about our API key. Then
6:37:32another thing we will need is the base
6:37:35URL. The user can set any base URL they
6:37:38want because well it only makes sense
6:37:41that they can switch to another model
6:37:44like Gemini
6:37:47and if open router for example doesn't
6:37:49have support for it which is highly
6:37:50unlikely
6:37:52but yeah you can just do that. So you
6:37:55have string or null and then you just do
6:37:58return os.environment.get
6:38:01get
6:38:03base URL. After that we can have the
6:38:07model name. So add the rate property
6:38:10model name. Now you might wonder why are
6:38:12we creating a function for model name,
6:38:14right? We already have a model name over
6:38:16here in the model config. Why do we need
6:38:19another one? Well, that's because to
6:38:21access a model, we will have to do self
6:38:23domodel.name.
6:38:25So instead of that, I'm just giving a
6:38:27convenience feature here which is return
6:38:29self domodel.name. name. Okay. And we're
6:38:33also going to have a setter for this
6:38:35model name. So we have add the rate
6:38:37model name dot setter.
6:38:40This just means that the user can do
6:38:43something like config dot model name is
6:38:47equal to and then pass in any model name
6:38:49of their choice. This will be really
6:38:51helpful when we want to switch the
6:38:54model. Right? For example, here one of
6:38:57the command is forward slashmod. So you
6:39:00can just do something like model and
6:39:02then set the name of the model for
6:39:04example Gemini. So essentially what
6:39:06we're doing is having a setter over here
6:39:09and that's exactly what's happening
6:39:11here. So we have def model name as well.
6:39:15Then we get a self then we also get a
6:39:16value which is of the type of string and
6:39:19we return nothing from here. All we need
6:39:21to do is not even return anything. self
6:39:24domodel dot name should be equal to this
6:39:27value that we have. Okay, after that we
6:39:30have the temperature. Again, this is
6:39:32just a convenience kind of thing. So we
6:39:34have self and then we have a float. Then
6:39:38we have self dot dot temperature
6:39:41returned from here. And similar to this
6:39:44model name, we can also have a setter
6:39:46for the temperature. So we have self
6:39:49domodel.
6:39:51to the value and then I believe we can
6:39:54create one function just to check that
6:39:57the configuration is valid. Now how do
6:39:59we check if the configuration is valid
6:40:01or not? To do that we will have to check
6:40:04if the API key exists or not. If the KP
6:40:08API key exists, it means configuration
6:40:11is valid. And we'll also have to check
6:40:13if the current working directory exists
6:40:15or not. because if the current working
6:40:17directory is not mentioned or if it does
6:40:20not exist properly then we have an error
6:40:23as well. So that's another thing we'll
6:40:25have to check. So let's create a
6:40:27function def validate and then we get a
6:40:30self and it will return a list of
6:40:32string. It will be empty if there are no
6:40:35errors and it will contain strings if
6:40:38there are errors. So if we have errors
6:40:41as a list of string and by default it is
6:40:44just an empty list. Now we just check if
6:40:46not self API key in that case we are
6:40:51going to do errors dot append and then
6:40:53we have no API
6:40:56key found set
6:40:59let me just type it properly set API key
6:41:02let's get this set API key
6:41:06environment variable awesome another
6:41:10thing I would like to check here is if
6:41:12the current working directory exists or
6:41:14not so the current working directory
6:41:17does not exist. In that case, what I
6:41:20want to do is errors.append
6:41:23working directory does not exist. And
6:41:26then we have self dot current working
6:41:30directory. I think I misspelled self. So
6:41:34we have this. Great. Now if either one
6:41:38of these errors are there, we'll have an
6:41:41errors list not empty. Otherwise, it is
6:41:43empty. So that's about validation.
6:41:46So we have the entire schema created.
6:41:49Let's go to loader.py and create the
6:41:51function that we need which is load
6:41:53config. We get a current working
6:41:54directory of the type of path or null.
6:41:58And we well that's all we get as of now.
6:42:02And what this will do is return a config
6:42:04if it exists. So from config.config
6:42:07we'll import config. And yeah that looks
6:42:10good enough. Now the very first thing we
6:42:13want to do is initialize the current
6:42:16working directory. So if current working
6:42:18directory exists we'll have current
6:42:21working directory or if it is null then
6:42:24we will use path. CWD. Okay the current
6:42:28working directory we are in. Now we want
6:42:30to load the system configuration. Now
6:42:33what is a system configuration path?
6:42:36System configuration path is where
6:42:39basically our system config file is
6:42:41going to live. So that means somewhere
6:42:43in the root of our folder you can create
6:42:46config.yamel file and that will be read
6:42:49to extract the values from. So whatever
6:42:51values we have over here for example
6:42:53model or max turns you can set this in a
6:42:57system level configuration file
6:43:00something like this where you have the
6:43:03root dotconfig folder and within that
6:43:06doconfig folder maybe you have this AI
6:43:09agent folder and within that you have
6:43:11configy file. So this is the file or in
6:43:15our case it's going to be TOML similar
6:43:17to OpenAI's codec cli because it's just
6:43:21cleaner to read. So this is the place
6:43:23where we are going to check which is our
6:43:25system configuration path. Now to
6:43:28understand this path correctly first we
6:43:30must understand what elements are
6:43:31present within it. The first part is the
6:43:35user related configuration directory. So
6:43:37this part that you can see is called the
6:43:40user configuration directory. This is
6:43:42where all of the configs related to all
6:43:44of your programs should be present. And
6:43:47within that we are creating a new folder
6:43:49called AI agent within which there'll be
6:43:52a file called config.tml. So let's just
6:43:55create the config directory. This is the
6:43:57user related config directory. And we're
6:44:00going to return a path from here. And
6:44:03now we can just do return path where we
6:44:06pass in user config directory. We can
6:44:09import that from platform directories
6:44:12that already has access to this part.
6:44:14Obviously, this is Linux or Mac. But in
6:44:18Windows, there'll be something else
6:44:19which user config directory already
6:44:21takes care of. And now here we can pass
6:44:24in the app name. So we want to get the
6:44:27config directory of our app, right? Our
6:44:30app is AI- agent. If you have some fancy
6:44:34name, you can put that in. Now to get to
6:44:37this particular path you just have to
6:44:40add in config dot toml correct because
6:44:44this part was done by user config
6:44:46directory then we said our app's name
6:44:48which added this particular part now we
6:44:50just have to add config tl at the top we
6:44:54can define what our config file name
6:44:56should be so that we can change it from
6:44:58one place so here we go and now I can
6:45:02create a function called get system
6:45:06config path also this is going to be def
6:45:09and this is going to return a path as
6:45:11well. Then we're going to do return get
6:45:13config directory and then we are going
6:45:16to append it with config file name.
6:45:19Okay. So this is all where all the user
6:45:22related configuration is going to stay.
6:45:23This is where our system configuration
6:45:26is going to be there. And now we can
6:45:29just go ahead and do system path is
6:45:31equal to get system config path. So we
6:45:34get access to a path and now we can
6:45:36first check hey if the system path is a
6:45:40file only then do we want to move
6:45:42forward because if it's a directory or
6:45:44something we're dealing with the wrong
6:45:45thing. Now the first thing I want to do
6:45:48is first ensure that the TOML file is
6:45:51properly parsed. And to parse a TOML
6:45:54file, we can create another function
6:45:57called parse toml. And that is going to
6:46:00be private to the file or like the
6:46:03loader. py file. It cannot be used
6:46:06outside. We're going to get a path and
6:46:08from this path we are going to extract
6:46:11the file and load it in a dictionary
6:46:13format. So we're going to have try then
6:46:16we have some logic then except and we're
6:46:19going to catch one specific error which
6:46:21is tol decode error just in case the
6:46:25decoding goes wrong and we are going to
6:46:27import it from
6:46:30okay and we have it as e and then we
6:46:32raise a config error now we'll have to
6:46:35create what's known as a config error
6:46:38now we'll have to create this config
6:46:40error which we'll go and do in utils
6:46:43errors py by and this is also something
6:46:47that I'm going to copy paste because
6:46:48it's fairly straightforward but it's
6:46:50this thing. So what are we doing here?
6:46:53Well, we're creating a class first
6:46:55called agent error which extends
6:46:57exception. That's the way you can create
6:46:59your custom exception. And then you have
6:47:02the message coming in. Then you have
6:47:05asterisk which is not really needed.
6:47:07Then you have details with string, any
6:47:10as the dictionary. Then you have details
6:47:13as a dictionary with key as a string
6:47:15value as any. And then you have cause
6:47:18which is of the type of exception or
6:47:19null and by default it is null.
6:47:23And then we are just calling this base
6:47:25exception class where we are calling
6:47:27self dot messageages equal to message.
6:47:29self.details is equal to details. Self
6:47:31dot cause is equal to cause. After that
6:47:34we have a string representation of this
6:47:36function where we just have self dossage
6:47:39as the base and then we are saying that
6:47:41hey if the details ex see exists then
6:47:44we're just trying to format it nicely.
6:47:47So we just have a commaepparated key
6:47:49equals value pair and we just display it
6:47:52over here. So base is equal to that and
6:47:56if the cause is present of why this
6:47:58exception occurred then you will have
6:48:01this base caused by self dot cause.
6:48:04Okay. So whatever base you had from here
6:48:07you just add it over here. This is just
6:48:10nice string formatting just in case we
6:48:12try to print an error. Okay. And then we
6:48:15also have two dictionary where we just
6:48:17trying to convert it to a dictionary.
6:48:19Awesome. Then we have config error which
6:48:21extends agent error and since agent
6:48:24error extends exception config error is
6:48:26also an exception. So here we're getting
6:48:29the message we getting the config key we
6:48:31getting the config file. So which config
6:48:34attribute caused this exception and what
6:48:38is the file that is causing the
6:48:40exception. So we just have those
6:48:43details. Then we have the config key and
6:48:46details at config key is equal to config
6:48:49key. Details at config file is equal to
6:48:51config file. Then you have the init
6:48:53function. And then you have this
6:48:55particular thing where we are just
6:48:57setting the config key and config file.
6:49:00Awesome. And now we can just use the
6:49:03config error over here. The reason I
6:49:06just copy pasteed is because it's fairly
6:49:08straightforward and really non-essential
6:49:10to our application. This is just coding
6:49:12practices that I'm following because
6:49:15yeah, if there's a config error, we
6:49:17raise it over here. But later on, if we
6:49:19passse the toml and we get a config
6:49:21error, we'll really do nothing about it
6:49:23except just print it out. And here we
6:49:26can just type in that hey there was an
6:49:28invalid toml in this particular path and
6:49:33build the exception was this. And then
6:49:35we can also set up the config file which
6:49:38is equal to the string of path and then
6:49:42yeah that should be enough and we can
6:49:45raise this from E. Okay. Now let's say
6:49:48we run into some other sort of error
6:49:51which is OS error or IO error. So we can
6:49:55deal with them as well. So we will raise
6:49:58a config error for this as well. Let's
6:50:00print it out.
6:50:02And then we have failed to read config
6:50:07file. Then we pass in the path.
6:50:10Then we have e passed in. Then we fi
6:50:13pass in the config file. And rest of the
6:50:16things just remain the same. Now to pass
6:50:18the toml, how do we do it? Well, it's a
6:50:20oneliner or actually a twoliner. We'll
6:50:23first open the file. So we pass in the
6:50:26path. Then we have read as bytes. And
6:50:29then we have the file. After that we'll
6:50:31do return tli dot load and we first have
6:50:36to import tli and since we have imported
6:50:40tli we don't have to do this we can
6:50:43remove this line and then do tm mli tml
6:50:48decode error and now for this tmli.load
6:50:52we can pass in the file pointer which is
6:50:54this and that will return to us a
6:50:57dictionary format. So we'll pass from a
6:51:00binary file object. Now that we have
6:51:02passed the toml, we have a dictionary.
6:51:04So let's just create a variable called
6:51:07config dictionary which is dictionary
6:51:12string, any and we'll have to import
6:51:14from typing any which is equal to an
6:51:17empty dictionary first of all. After
6:51:19that if it's a file then we'll try to
6:51:22parse toml. Then we need to pass in the
6:51:25path which is just the system path
6:51:27because we're trying to pass the
6:51:28config.l file which is present in this
6:51:32system path right and then we can just
6:51:35set this config dictionary to this. We
6:51:38don't really have to merge as of now
6:51:40because the the config dictionary is
6:51:43just empty. Now obviously we can run
6:51:45into any sort of error which we will
6:51:48catch using config error and yeah we can
6:51:52just print out logger dot warning and
6:51:55then we can say skipping invalid system
6:51:59config then we pass in the system path.
6:52:03Now we have to create a logger as well
6:52:05which we can do at the top. So we have
6:52:09import logging. Then we create a logger
6:52:13dot get logger and pass in the name like
6:52:18that. So we have a logger. Great. Now we
6:52:22can scroll down and we have the config
6:52:24dictionary initialized. Awesome. Now
6:52:29another thing that the user can do is
6:52:31instead of setting a system config path
6:52:34they can also set a project specific
6:52:36config path. So yeah this place is where
6:52:41system level definitions are done. Okay
6:52:44but what they can do is also go to
6:52:48users/rean/estop/I
6:52:52agent which is this particular folder
6:52:54and in there they can also create a
6:52:56config.tml TML file or specifically they
6:52:59can just have AI agent folder within
6:53:02which they have a config file. You can
6:53:05just have any path of your choice. I'll
6:53:08probably go with this. And what this
6:53:10projectwide configuration does is that
6:53:13for this project, the project we are
6:53:15dealing with this AI agent project, it
6:53:17will follow the configurations mentioned
6:53:19over here. So let's say if you have both
6:53:22of these configurations defined, it will
6:53:24give priority to the attributes
6:53:26mentioned over here. So essentially
6:53:28these attributes will override the
6:53:30config. ML attributes. So let's create a
6:53:34function to find the project config and
6:53:37that is actually quite straightforward.
6:53:40We can have def get project
6:53:44configuration.
6:53:45Then we get the current working
6:53:46directory. Whatever directory we are in,
6:53:49we just have to check the AI agent
6:53:52folder of that directory and we're going
6:53:55to return the path, right? It's not
6:53:58going to be a list of paths because we
6:54:01are only expecting to find one project
6:54:03configuration.
6:54:04And now we can just check that hey, this
6:54:07agent directory should exist in the
6:54:10current working directory. So first of
6:54:12all let's just initialize the current
6:54:14working directory which is current
6:54:16working directory.resolve.
6:54:18So it will make the path absolute. And
6:54:21now I can just do current forward slash
6:54:25AI agent because whatever project
6:54:27directory we are in I want to go in its
6:54:29AI agent folder and then check that hey
6:54:33if this AI agent or agent directory is a
6:54:37directory in that case I want to check
6:54:40the config file. So we have config file
6:54:42is equal to agent directory and then you
6:54:45pass in the config file name which is
6:54:47this particular thing config.l ML and
6:54:50then I can go ahead and check that hey
6:54:52if this config file is a file then I'll
6:54:56just go ahead and return this config
6:54:59file that we are on right because we
6:55:02have found out our configuration project
6:55:05specific configuration otherwise I can
6:55:07just return null and the return type of
6:55:11this can be path or null now I can take
6:55:13this get project configuration and call
6:55:15it over here where we are trying to get
6:55:19the project specific configuration. Now
6:55:21I can pass in the current working
6:55:23directory here which we have already
6:55:26initialized or which we have already
6:55:28taken from the parameter and now I get
6:55:31project path. Now if that project path
6:55:34does exist in that case what do I want
6:55:37to do? Well I want to take whatever path
6:55:40was present there and I want to merge it
6:55:42with the config dictionary. So we have
6:55:44passed TOML and then I'll pass in the
6:55:47project path that gives me the project
6:55:51config dictionary. Now I want to take
6:55:54this project config dictionary and merge
6:55:57it with the main config dictionary this
6:55:59particular thing. And since we've called
6:56:01parse UML, let's just do try except as
6:56:04well. So we have try over here and let
6:56:07me just have that.
6:56:10After that we also have accept and yeah
6:56:14here we go. How do I merge this config
6:56:17directory and project config directory?
6:56:20Well, I can create a function that will
6:56:22recursively merge the two dictionaries
6:56:25because we have to do a deep merge. A
6:56:27shallow merge is not enough. A deep
6:56:29merge means if the values of the
6:56:33dictionary have another dictionary in
6:56:36them, those are properly copied as well
6:56:38and references are no longer kept. So at
6:56:42the top we can just create merge
6:56:46dictionaries where we get a base
6:56:48dictionary which is string, any and then
6:56:52we get the overriding dictionary. What
6:56:55is the dictionary that is going [snorts]
6:56:58to kind of rule? Whose rules are we
6:57:01going to follow more? And obviously this
6:57:03is going to return uh dictionary or
6:57:05string, any
6:57:08now we are going to have result is equal
6:57:10to base.copy because well the result is
6:57:12first going to have base and on top of
6:57:14base we'll override all of the items.
6:57:16Right? So for every key comma value in
6:57:19override dot let me type at override do
6:57:23items what we are going to do is check
6:57:25if key is in result and if the result at
6:57:31key is also a dictionary right so that
6:57:36means in the base version we have a key
6:57:40and the value of that is a dictionary as
6:57:43well and this value that we're talking
6:57:46about also is of the type of dictionary.
6:57:50Let me just type out dictionary.
6:57:51Correct. Then what do we want to do?
6:57:53Well, I want to recursively merge the
6:57:56dictionaries. So we'll have result at
6:57:58key equal to and then we'll call merge
6:58:01dictionaries again where we'll pass in
6:58:03result at key and then we pass in the
6:58:06value. Otherwise, it's not a dictionary.
6:58:09We have a simple life. We do result at
6:58:11key equal to value. And then finally
6:58:14after all of the for loop iterations are
6:58:16done we return the result. So yeah let's
6:58:18understand this again. We're going
6:58:20through every item in overrides and then
6:58:22we're checking if it's present in the
6:58:24result dictionary which is this one. And
6:58:27then we are checking that result at key.
6:58:29The value of this base dictionary is
6:58:32dictionary and the value of this
6:58:35override is also a dictionary. That
6:58:38means we are trying to merge from base
6:58:41or we are trying to merge from override
6:58:43into base and both the keys and the
6:58:45values are essentially similar. So we
6:58:48are just doing result at key is equal to
6:58:50merge dictionaries. So we go into that
6:58:52dictionary right we go into one
6:58:56dictionary let me just give you an
6:58:57example let's say we have a b c like
6:59:02that correct? So what we have done is
6:59:04check that hey if a is in the base
6:59:07dictionary and the value of this thing
6:59:10is in the is a dictionary as well and
6:59:14the other dictionary that we are trying
6:59:16to merge within this dictionary let's
6:59:18say it goes with a b c as well. So this
6:59:23value is also of the type of dictionary
6:59:26then we'll just go into this dictionary
6:59:28and check all of the keys within that
6:59:31dictionary. That's why we're doing it
6:59:33recursively. Now we can just take this
6:59:35merge dictionaries and call it in over
6:59:38here. We call merge dictionaries. The
6:59:41base dictionary here is going to be the
6:59:43config dictionary and then we have the
6:59:46overriding dictionary which is project
6:59:48config dictionary and this is going to
6:59:51be the config dictionary value. Another
6:59:54thing I would like to do here is for
6:59:58again check that hey if the current
6:59:59working directory is not in the config
7:00:02dictionary then I'll just set config
7:00:04dictionary to current working directory.
7:00:07All right. Now this is probably not
7:00:10required because in config.py we have
7:00:13told that if we don't find the current
7:00:16working directory the value should be
7:00:18path. CWD. But anyways this is good to
7:00:22have for now. What we can do is also
7:00:26load up the developer instructions
7:00:28because as I told you developer
7:00:30instructions will be present in
7:00:32agents.mmd
7:00:33and I want to read them. So if the
7:00:36developer instructions let me just have
7:00:39that if developer instructions are not
7:00:43present within config dictionary in that
7:00:45case we are going to read the agent MD
7:00:48file. Where is the agent MD file going
7:00:51to be? it's going to be in that specific
7:00:53project itself. So similar to this get
7:00:57project config, we are going to have an
7:00:59agents.m MD file as well. So we have get
7:01:02agents MD file. Let's not call it
7:01:07agents. I think agent MD files is good.
7:01:11We pass in the current working
7:01:12directory. And now we are just trying to
7:01:15get the files of agent MD for this
7:01:19project. And there's only going to be
7:01:21one agent MD, right? So we can check
7:01:23here that hey, if a file with aent MD
7:01:28exists, then I want to use it. And where
7:01:31should I check it? Well, I'm not going
7:01:33to check it in the AI agent folder
7:01:36because agents.m MD can be created in
7:01:38the root itself. They don't have to be
7:01:41created within this specific folder. So
7:01:43I'll just remove this line. We'll just
7:01:45check if current is directory and if
7:01:49current is directory then we have a
7:01:50config file or actually agent MD file.
7:01:53So let's just call it agent MD file
7:01:56which is equal to current and then we
7:01:58pass in the agent MD. So let me just say
7:02:02agent MD file which is equal to agent do
7:02:07MD. Okay. Now we'll just use this agent
7:02:10MD file
7:02:12and pass it in over here. And then we
7:02:15can check that hey if the agent MD file
7:02:17is a file in that case we will try to
7:02:22read the content. So content is equal to
7:02:25MD file dot read text and let's just
7:02:30call it agent MD file readext and then
7:02:33we can pass in the encoding if you want.
7:02:36I think the default is also UTF8
7:02:40and then we can just go ahead and return
7:02:42this content. I think we can also do a
7:02:45try and accept here, but I'm actually
7:02:48too tired to write it out, so I'm not
7:02:50going to write it. But yeah, that's good
7:02:53enough for now. I can take this get
7:02:55agent MD files and call it in. Then we
7:02:59can pass in the current working
7:03:00directory as well. And then we can have
7:03:02agent MD content which is equal to this.
7:03:05Now if agent MD content is present then
7:03:09I will just override right or actually
7:03:11not even override because developer
7:03:13instructions were not even present. So
7:03:15I'll just set developer instructions to
7:03:18agent MD content. And now finally after
7:03:21all of this I am going to do config is
7:03:25equal to agent config or actually I just
7:03:28have to call it config and then I can
7:03:31pass in the entire config directory or
7:03:34dictionary after I have deconstructed it
7:03:38and then I can return the config. One
7:03:41thing to keep in mind here is that while
7:03:43doing this particular line we can run
7:03:45into any sort of error. So I'll have a
7:03:47try except here as well. And that is an
7:03:50important line. So let's say we run into
7:03:53any sort of error. In that case I will
7:03:55raise a config error again.
7:03:59And this time we can pass in invalid
7:04:03configuration. We'll pass in E from E.
7:04:07And yeah, that looks good to me. I hope
7:04:11it works out. So we can go ahead and try
7:04:15it. We can go to our main py where we
7:04:18are going to go to the main function and
7:04:21here define our configuration. So we
7:04:24have config is equal to load config and
7:04:28we have to import it from config.loader
7:04:30and then I can pass in the current
7:04:32working directory which is well it needs
7:04:36to be passed in from the terminal. So
7:04:39here we can just take that as an option.
7:04:42So at the rate click dot option. So
7:04:46command meant that this entire thing is
7:04:49now a CLI tool. Then you have argument
7:04:51where you don't have to specifically
7:04:53pass in d-prompt but now with option you
7:04:56need to pass in the current working
7:04:58directory. So you can pass in d- cwd and
7:05:02it's also often that you'll be given
7:05:06dash c as well. So if you have a double
7:05:09hyphen generally people also provide a
7:05:12single hyphen command which is shorter
7:05:16and now you can set the help to be
7:05:18current working directory
7:05:22and then maybe we also want to give it a
7:05:25type. What is the type that we are
7:05:27expecting in this current working
7:05:29directory? Well, it should be a path. So
7:05:32we have click.path.
7:05:34The path should exist. So click will do
7:05:37all of that for us which is great. If
7:05:39it's a file are we okay with that? No.
7:05:42The current working directory should be
7:05:44a directory and obviously the type
7:05:46should be of the type path. So we have
7:05:49initialized that current working
7:05:51directory and now we can take that from
7:05:55the parameter. So we have current
7:05:58working directory which is of the type
7:06:00of path or null. And now in load config
7:06:03we can try to pass in the conf current
7:06:06working directory
7:06:08and that is our entire configuration.
7:06:12Now let's say we run into any sort of
7:06:15error here which we have not already
7:06:16caught. We will just do try except
7:06:19exception
7:06:21as e where we just print out an error.
7:06:25And I think we've already defined how
7:06:27the error is going to look like because
7:06:30yeah we do have console.print here. So
7:06:33let me just print it out here. This is
7:06:36given out to the user. This is not
7:06:38something related to a logger.
7:06:40Everything that we did over here is
7:06:42related to a logger. So if we run in
7:06:44debug mode, we'll be able to know it.
7:06:47And yeah, so let's just pass in config
7:06:52ration error and then we pass in e good.
7:06:56After we have the config, I would like
7:06:58to validate it and I've already created
7:07:00an instance method for it. So we have
7:07:02config.validate
7:07:04and it will give me a list of errors if
7:07:07they exist. And if the errors exist,
7:07:10then what I want to do is for every
7:07:12error in this error string, I just want
7:07:15to print it out. so that I can display
7:07:17all of the errors to the user at the
7:07:20right time. Also, I did not pass in the
7:07:23error tag here. So, let me just do that.
7:07:27And similar thing should be done over
7:07:29here. So, we have console.print. We pass
7:07:32in the error and that's it. And maybe if
7:07:36there are errors, we might want to exit
7:07:38out. So, I do system.exit where the
7:07:40status code is one. And then we can
7:07:43initialize CLI after that because you
7:07:46know if we're running into errors, why
7:07:48do we want to instantiate the CLI thing?
7:07:51Why do we want to waste any resource?
7:07:53And yeah, that looks good to me. Now
7:07:55let's try to start something up. So we
7:07:58have Python main. py and we run into
7:08:01error and that is related to invalid
7:08:03escape sequence. Now to avoid that I can
7:08:06just use double backslashes everywhere.
7:08:10So that should be fixed. Now let's try
7:08:12to do it again. And it says no module
7:08:16named platform directories. That's
7:08:18because we have to install it. So in
7:08:20this virtual environment, I'm going to
7:08:22do pip install platform directories. By
7:08:25the way, whenever I'm running python
7:08:26main.py, what am I expecting? I'm
7:08:28expecting an error. The reason for it is
7:08:31we have not set an API key. We've not
7:08:33set a base URL. And it should be set
7:08:35through the environment variables.
7:08:38That's why this is a good check for us
7:08:40that our configuration is working. Now
7:08:43we get no module name to mli because
7:08:46that should also be installed through
7:08:48pip. So let's install that. And there we
7:08:51go. After that hopefully it works out.
7:08:54So let's run it. And we do get the error
7:08:56message. It's incorrectly formatted
7:09:00because probably back slash error is not
7:09:03the way to go about it. Maybe it's just
7:09:05forward slash error. I keep getting
7:09:07confused with that. Now let's try to run
7:09:09it again.
7:09:11And we do get the right variable. So
7:09:13that's great. Now let's try to set the
7:09:15API key and use it everywhere. So just
7:09:18so I remember what I'm going to do is
7:09:21check for the API key wherever it's
7:09:24used. I think it's only used in OpenAI
7:09:26client. So I don't have to worry too
7:09:28much. But for the model name, we'll have
7:09:30to probably worry because there's a lot
7:09:33of places where I passed in the model
7:09:36name.
7:09:37So yeah, the API key is over here. We
7:09:40can just remove it from here and use the
7:09:42configuration instead. We can either
7:09:45take the configuration from the
7:09:48constructor or instantiate the
7:09:50configurator over here. So instead of
7:09:52doing load config every single time you
7:09:55know in every single class we'll just
7:09:56take this configuration object from the
7:10:01constructor. Now we have self doconfig
7:10:04equal to config. And now I can use
7:10:07self.config do.base url. Similarly, we
7:10:11can use Oh, by the way, let me just put
7:10:13the comment of my API key here because I
7:10:16will forget it. And I'll cut this out so
7:10:18that we can use the base URL. This
7:10:21should be API key. My bad. And yeah,
7:10:23this is the base URL. I'll just copy
7:10:26them whenever I have to from here or I
7:10:30can put it in my environment variables.
7:10:32Either one of them is fine. By the way,
7:10:34if you're wondering how do we create an
7:10:36environment variable, you just do env.
7:10:39But that's not the route I'm going to
7:10:41take. I'm just going to call export on
7:10:44this thing and I'll show to you what
7:10:46what I'm going to do. But yeah, so I've
7:10:50passed in the API key and base URL.
7:10:52That's good. Now let's try it and run
7:10:55it. So first I'm just going to copy this
7:10:58entire API key. Then I have export API
7:11:03key equal to this URL or this API key. I
7:11:08don't know what I'm saying.
7:11:09Then I can pass in export base URL is
7:11:13equal to this thing. It's a bad
7:11:15assignment. I can't leave any spaces.
7:11:18So, yep. And now I can try running it.
7:11:22So, I try running it and I run into an
7:11:24error. Llm client in it missing one
7:11:26required positional argument config. So,
7:11:29that means the loading configuration
7:11:31part worked fine. But now we have to
7:11:34pass the config through the
7:11:37instantiation of the LLM client. So
7:11:40there we go. Now I can pass in the
7:11:43configuration which is well how do I
7:11:46pass in the configuration. The agent is
7:11:48also going to take the configuration
7:11:50from here and we'll import it from
7:11:52config.config and pass in the config
7:11:55here. As simple as it gets. Now
7:11:58obviously the next step is taking all of
7:12:00these things and pushing them within a
7:12:03session what we're going to work on next
7:12:06because much of our features depend on
7:12:08this session and again I'll explain to
7:12:10you why we need a session but yeah now
7:12:13whenever this agent is called which is
7:12:15two times one in run single and then in
7:12:18run interactive and by the way we are
7:12:20getting multiple of these agents because
7:12:22there is a function called user agent
7:12:26default user agent all of that. If you
7:12:28want to specifically check for this
7:12:31agent where a is capitalized, you can
7:12:34turn on this match case and we get the
7:12:37places where agent is called. This is
7:12:39scripting. py which is in virtual
7:12:41environment. What I'm interested in just
7:12:44main py which is two instances. Then I
7:12:46can pass in the configuration. Now
7:12:49configuration needs to be taken in from
7:12:51this constructor as well. So we have
7:12:54self.config config equal to config and I
7:12:58just realized even in agent. py we have
7:13:00to do self.config is equal to config and
7:13:03then maybe you can pass in config you
7:13:05can maybe pass in self.config both are
7:13:07fine but the reason we need self.config
7:13:09is because in these functions we'll be
7:13:12using the configurator and coming back
7:13:14to main.py we have cell.config equals
7:13:17config now I can just pass in
7:13:19cell.config config here. Then in the run
7:13:21interactive mode as well, we pass in the
7:13:23config. And finally, whenever the CLI is
7:13:26instantiated, I'll pass in the config.
7:13:28Now, if you want to avoid all of this,
7:13:31this is exactly the reason why people
7:13:33created dependency management solutions.
7:13:36So, we can just pass in configure. But
7:13:40yeah, you can just find a dependency
7:13:43management solution so that you can pass
7:13:45one object everywhere. But yeah, I'm not
7:13:48doing all of that for this tutorial. I
7:13:51don't want it to be so long. It's
7:13:53already pretty long. And yeah, I think
7:13:56now we should be able to get something
7:13:59up. So, we run Python main. py. That's
7:14:02great. Now, let's see if it works. So,
7:14:04I'll just pass in a message. Does it
7:14:06work? And I'm expecting an assistant
7:14:08response. And yeah, I do get, could you
7:14:11clarify what you're referring to? That's
7:14:13good. I'll exit out of here. And now I
7:14:17would like to go ahead and test the
7:14:20config.tml file. Right, I told you that
7:14:23this config.tml file can either be
7:14:26created in the project or in the
7:14:27systemwide part. So let me first try out
7:14:31the system level configuration and it
7:14:33should be somewhere over here
7:14:35user/andonaut
7:14:36for me. And now I'm just going to create
7:14:38a new folder called AI- agent. That's
7:14:42how I wanted my folder name to be. and I
7:14:46will use dot then over here I can go
7:14:49ahead and create the config
7:14:52file. Now I don't have the option to
7:14:53create a new file and I don't know how
7:14:55to create from here. So what I'm going
7:14:57to do is just create a file over here.
7:15:00So I'll close all the save file then I
7:15:03can say create new file and I have
7:15:05created a agent in config. ML. So I
7:15:09create it and here we go. Now what are
7:15:12the things I'm interested in? I'm
7:15:14interested in the changing the model
7:15:16name and to change the model name first
7:15:19I have to give it a heading of model
7:15:21because if you go to our config py file
7:15:24you'll notice here we have model model
7:15:28config right so first I have to pass in
7:15:30this model and then over here I can set
7:15:33the name which is let's say a new open
7:15:37router model I don't know what open
7:15:39router models are new it's been a t
7:15:41while since I've been recording But
7:15:44let's try to use this one, the Xiaomi
7:15:46model. So I'll pass it in over here and
7:15:49let's see if that gets included. I'll
7:15:52keep the config.tml file here. This is a
7:15:55systemwide config.tml file. Now
7:15:59obviously we won't be able to see the
7:16:01effects of it because everywhere we
7:16:03using the hard-coded Mistral AI
7:16:06configuration. So what I'd like to do is
7:16:09search for this mistral specifically and
7:16:11change it everywhere to use the
7:16:13configuration object. So we have
7:16:15self.config dot model name. Great. Now I
7:16:20can just copy this line and paste it
7:16:22everywhere. So another time is in the
7:16:25context manager. So let's just put it
7:16:28here. And there's no self.configure. And
7:16:31there's no self.configure. So first I'll
7:16:34have to create this config or actually
7:16:37get this config. Then I have self dot
7:16:40config equals to config. I don't think
7:16:42we really need to do this but anyways
7:16:45and now whenever context manager is
7:16:46called I need to pass in the model name
7:16:48but anyways in the read file tool as
7:16:51well we will require this configuration
7:16:54and maybe if we don't pass in the model
7:16:56year it should be fine because in count
7:16:59tokens how long are we going to keep
7:17:00passing in the model? So maybe here we
7:17:03can just hardcode it to GPT4. And
7:17:05obviously if it doesn't get the model,
7:17:07it will error out or something. So
7:17:10that's all right. And now we can remove
7:17:12it from here. Just taking the shorter
7:17:14route. So yeah, now whenever we have the
7:17:19context manager initialized, I want to
7:17:22pass in the configuration object and
7:17:24that's done in agent. py. So here we can
7:17:28just pass in the config like that.
7:17:31Awesome. Now let's try to run it. And
7:17:34actually I want to ensure that on the
7:17:36welcome screen as well we're showing the
7:17:39correct model that's being used. That is
7:17:42the best indication of what model we're
7:17:44using. So we'll come back to main. py
7:17:47where at the top you know we had
7:17:50something like this welcome screen and
7:17:51here we also hardcoded it. Correct? So
7:17:54let's change it to config. We have an
7:17:57string already. This should be easy.
7:17:59self do.config. config dot model name.
7:18:03Great. Now we can try running again. And
7:18:07we still see mistrol model being used. I
7:18:10realized why I was running into an
7:18:11error. And it was because the config tml
7:18:14file was located in the wrong place. So
7:18:17now to fix it, what we have to do is put
7:18:19it in the right place, which is well in
7:18:21your root folder followed by library
7:18:24application support AI agent and then
7:18:27config.tml. If you are on Linux or
7:18:30Windows, it will be different for you.
7:18:31You can search web or ask AI and it
7:18:34should help you. You can ask your own AI
7:18:36agent actually and it should help you
7:18:39out. So yeah, I've passed in the model.
7:18:41By the way, this model should also be
7:18:43placed like this. It shouldn't be like
7:18:46this because this refers to a list. So
7:18:49when we try to have MCPS or when we try
7:18:51to have hooks, that's when we are going
7:18:54to use something like this because it is
7:18:56list. when we are trying to have model
7:18:59dot name it's not a list we will have a
7:19:02single bracket
7:19:04so now let's try to run it so I'll have
7:19:06python main py it loads up the correct
7:19:09model which is great and now I can say
7:19:11hi how are you doing and then it can
7:19:15start running it says I'm good thanks
7:19:17ready to help you now what I would like
7:19:19to do is exit out of here so I have back
7:19:23slashexit
7:19:25and now let's try to use a system level
7:19:28configuration. For system level, I just
7:19:30have to go to my current working
7:19:32directory. Here I'll create a/ aent and
7:19:36then here I'm going to create
7:19:38config.tml.
7:19:40Here I can set the model and the name
7:19:42can be different. Let's say I try this
7:19:44Alen AI. I think this will give an error
7:19:48because it's probably not agent. So
7:19:51yeah, we'll see. Let's try to run it.
7:19:54It does load up the model and now it I
7:19:57can ask it hi how are you doing and it
7:20:01gives us an error. No endpoints found
7:20:03that support tool use. Whenever you see
7:20:05this error it means that the model
7:20:06probably doesn't support tool calls and
7:20:09that's why we're running into this
7:20:11error. And let's try to exit out of
7:20:14here. Yep. So the systemwide
7:20:18configuration also works. Now let's
7:20:20close all of the save files and let's
7:20:22try to go to loader.py. py file. Let's
7:20:25go to config py actually. And here see
7:20:28what do we have. We have a model. We
7:20:30have current working directory and we
7:20:32know multiple places where we use
7:20:34current working directory like path.
7:20:36CWD. So I'll just search for this in the
7:20:39entire codebase. I see this place.
7:20:42Instead of using this current working
7:20:44directory, I have to use self.config.
7:20:47CWD. Correct? And I'll use the same
7:20:50thing everywhere else. So I'll go to
7:20:52this agent where I have my agentic loop
7:20:56and I can call self.config.curren
7:20:59working directory. Then I can pass in
7:21:02the tui instead of cell. CWD like this
7:21:06we'll have self.config.cwd.
7:21:09And now I need to get the config as well
7:21:12which is just config. And let's try to
7:21:16import config from config. And also
7:21:18let's import it above console because
7:21:21there can only be one. And let's try to
7:21:24import it above console. Okay, great.
7:21:27Now we can just do well let's also do
7:21:30self.config equals to config and then
7:21:33self.config.curren working directory is
7:21:35initialized.
7:21:37And finally in the load config we are
7:21:41passing in the current working
7:21:42directory.
7:21:44And that's all right. So yeah, I think
7:21:47this should be it. Now one place which
7:21:50was the TUI, we passed in the config but
7:21:54we have not passed it to the instance.
7:21:57So whenever we initialize TUI, I would
7:21:59also have to pass in the config which is
7:22:02this part. Great. Now let's try to run
7:22:05it again. And I have to remove this
7:22:09model name. Instead of this model, I
7:22:10would like to set the temperature to
7:22:13zero. And let's see how that works
7:22:15because we'll get the most likely
7:22:17output. It will be interesting to see.
7:22:20We go back to the Xiaomi model and I
7:22:22say, "Hi, how are you doing? Read main.
7:22:27py for me." And it says, "I'll read
7:22:30main. py for you." It calls the tool and
7:22:33executes it. Perfect. So yeah,
7:22:36everything seems to be working. So now
7:22:38let's try to pass the max turns and max
7:22:42tool output tokens. Some places we have
7:22:44hardcoded what the output tokens should
7:22:48be. So whenever we call something like
7:22:50truncate text you know like we have over
7:22:53here and here we have the max output
7:22:57tokens. The max output tokens is set to
7:22:5925,000. But here instead of having
7:23:0225,000, you can depend on the max output
7:23:06tool token to be passed through the
7:23:07config.tml file. But to me, it sounds
7:23:11not at all useful. Nobody would like to
7:23:14manage this. So I'm just going to remove
7:23:16it out of our application. But it was a
7:23:18demo to show that you can set something
7:23:20like this up. You can also set max
7:23:23turns. We also have developer and user
7:23:25instruction. And I told you that these
7:23:27developer and user instructions which we
7:23:29are looking for in agents.mmd file will
7:23:32be attached to the system prompt. So
7:23:35let's go ahead and update the system
7:23:37prompt. So at the top we'll add two
7:23:39other things. After operational section
7:23:41we have if config developer instruction
7:23:44we'll add get developer and if user
7:23:46instruction we'll add get user
7:23:48instruction. And now obviously we need
7:23:50config from the parameter. So let's
7:23:52import it from config.config.
7:23:55Now let's create the two functions get
7:23:57developer and get user. You can
7:24:00obviously go ahead and copy paste it
7:24:02from the GitHub repository mentioned.
7:24:05But essentially the these are just like
7:24:07project instructions. The following
7:24:08instructions were provided by the
7:24:10project maintainers. These are the
7:24:12instructions and follow these
7:24:13instructions carefully. The same for
7:24:15user instruction. The user has provided
7:24:17following custom instruction. You pass
7:24:18it in. I don't think there's a lot of
7:24:20difference between them but yeah I've
7:24:23seen some code base do this so I just
7:24:26thought of doing it myself. So yeah we
7:24:28have developer and user instructions in
7:24:31after that we are going to add the
7:24:32operational guidelines because you know
7:24:35operational guidelines are kind of like
7:24:37a summary. We want the these things to
7:24:40be at the top of the mind of our AI
7:24:43agent. That's why we are putting it
7:24:45towards the end. So yeah, as of now,
7:24:47these things seem enough. There's
7:24:49obviously more stuff we can add, and
7:24:51we'll probably look into that once we
7:24:53add all of the tools. But yeah, this is
7:24:57good enough for now. The next thing I'm
7:24:59interested in is whenever the system
7:25:01prompt is called like we have over here,
7:25:06I want to call the config in it. Right?
7:25:09So we'll just pass in the config as it
7:25:11is. So that's also good. Now let's go
7:25:14back to our config.p py. So developer
7:25:17and user instructions are also used. Now
7:25:20we have max turns. What is max turns
7:25:22about? I told you max turns is just the
7:25:25maximum number of turns our AI agent can
7:25:27go through. And I also explained why we
7:25:29need that. It's just to prevent infinite
7:25:31loop. So now what I like to do is make
7:25:34use of this max turns as well. So we'll
7:25:37go to our agent.py file where we have
7:25:40our agentic loop. And here before we do
7:25:44the this async for loop we're going to
7:25:46wrap this entire thing within max turns
7:25:49because all of the chat completion stuff
7:25:51we're doing talking to the llm executing
7:25:54the tool call that comes under the turn
7:25:56right. So above that we're going to go
7:26:00over a loop and for each turn we're
7:26:02going to call this. So at the very top
7:26:04we're going to have max turns is equal
7:26:07to self.config config.m domax turns and
7:26:11for turn number or you can just say I
7:26:14it's not going to be useful in range of
7:26:17max turns what are we going to do well
7:26:20we're going to execute everything that
7:26:23we see over here
7:26:25so all of this comes under this for loop
7:26:29okay but here comes another problem
7:26:32let's say the range is 100 right so
7:26:34we'll keep going for 100 turns now the
7:26:36problem is if you keep going for 100
7:26:38turns. That means let's say I say read
7:26:41main.py file. That only requires us to
7:26:44go for one turn. So it does this one
7:26:47turn. It reads the file for me. Okay.
7:26:51After that the LLM is again called with
7:26:54the context given. And then what will it
7:26:56do? It has already completed our task.
7:27:00So how do we know that we want to get
7:27:02out of this loop? Well, we want to get
7:27:04out of this loop that we have until we
7:27:07don't have any more tool calls to do.
7:27:09Now, let me make it completely clear.
7:27:11What is this turn going to do? Well,
7:27:13let's say I give my agent four task. I
7:27:16tell it to read my main. py file. After
7:27:20that, it has to go and update the main.
7:27:23py file to fix any bugs. After that, it
7:27:26has to review the main.y file. And then
7:27:30it has to write test for main. py file.
7:27:33All right. So these are too many tasks
7:27:35I've given in one message. But the LLM
7:27:37should be able to do it. But now the
7:27:40problem is that if I call chat
7:27:42completion first, the chat completion
7:27:44will say, "Hey, let me read the file
7:27:46first." Okay, so it reads the file, it
7:27:48executes all of the tool calls and then
7:27:51the turn just ends because the tool call
7:27:54executes and we add it back into the
7:27:56context. But here's the problem. It
7:27:58won't do the remaining tasks. it just
7:28:01stops because all the tool call
7:28:04executed. But in reality, the LLM just
7:28:06told us that hey, listen. I need to read
7:28:09main.py. There's more stuff to do, but
7:28:12first I have to read the file to notice
7:28:14any bugs. And that's why this fails. The
7:28:17user will have to rewrite the message
7:28:19just to get the agent to perform the
7:28:21next action.
7:28:23And that's why we're adding these turn
7:28:26related stuff. So here comes the bigger
7:28:28question. How are we going to stop this
7:28:30LLM from executing if it's doing it 100
7:28:33times? Let's say the LLM goes and does
7:28:36an operation. It takes in six steps. All
7:28:40right. So, six turns are taken. It reads
7:28:42a file, then it fixes the bug. So, it
7:28:45uses write operations and then it
7:28:48reviews the code. It reads the file
7:28:49again maybe. Then it reviews the code.
7:28:52Then it writes the test for it. So you
7:28:55know these many steps are being taken
7:28:58but now you know the range we have
7:29:01specified is 100 tones but in six steps
7:29:04the entire thing was done or if we give
7:29:07a simple message like write read main.py
7:29:09pi file for me. That's one step. Why
7:29:12should the LLM keep going for the rest
7:29:14of the 99 steps and that's why we need
7:29:18to figure out a logic to break outside
7:29:20of this loop as soon as the task is
7:29:22done. And we're saying that the task
7:29:24will be done as along as there's no more
7:29:27tool calls to do. That is what this
7:29:31agentic loop will do. This is how
7:29:33pyantic AI works as well. If there are
7:29:36no more tool calls to execute, the
7:29:38agentic loop is over. We're out. And
7:29:41that's easy for us, right? We know
7:29:43exactly how to check if there are tool
7:29:45calls or not. So once we add the
7:29:48assistant message and if there's
7:29:50response text we yielded
7:29:53just before executing these tool result
7:29:55operation, we can check if not tool
7:29:57calls. That means there are no more tool
7:29:59calls to do. In that case, we'll return
7:30:02out of here. There's no more stuff to
7:30:04do. The entire agentic loop is over.
7:30:07This simple line will help us get out of
7:30:10this turn. For simple LLM apps, it won't
7:30:14do that agentic workflow as of now. I've
7:30:16been seeing Claude do that agentic work
7:30:20now. And hopefully Chad GPD will also do
7:30:22it in the future. But as of now, one
7:30:24message refers to one turn in LLM apps.
7:30:27In agentic apps, it can refer to seven
7:30:30or eight turns. So yeah, this is the
7:30:33entire agent techic loop. This is
7:30:35configured through the configuration
7:30:37system. And now whenever we want to add
7:30:40a new feature, it will probably be
7:30:42toggled through the configuration
7:30:44system. For example, safety policies,
7:30:46approval policies, all of that can be
7:30:49configured through our configuration
7:30:51system. But what I'd like to do is test
7:30:54our application for one time more. I'll
7:30:57say read main. py file. Then I'll tell
7:31:01it to read prompts/system.py
7:31:07file and done. So let's hit enter and
7:31:10see what it does. As you can see it is
7:31:13doing all of that but it did it in just
7:31:15one turn. I believe it gave us two tool
7:31:18calls together and we execute both of
7:31:21them together. So in just one turn we
7:31:23are able to do both. But when we have
7:31:25right operations obviously it will
7:31:27require two steps because as I mentioned
7:31:30and I'm just drilling this concept into
7:31:32your mind. So forgive me for repeating
7:31:34but the point is
7:31:37it has to read first then it goes to
7:31:39into the context and then it is able to
7:31:41write. So that's why the other turn is
7:31:43required because if the LLM doesn't know
7:31:45the file's content how can it fix it
7:31:48even for you? You have to read the file
7:31:50first to understand what changes to make
7:31:52right. So anyways this was done and it
7:31:55also read the system.py and that is the
7:31:58entire thing. Cool. Now let's go ahead
7:32:01and create a session. In that session we
7:32:04are going to have multiple things but
7:32:07the main point is creating the context
7:32:10manager within it. Creating the
7:32:12configuration system or the tool
7:32:14registry within it the LLM client within
7:32:16it. All of that within a session. And
7:32:19again the reason for that is we can have
7:32:23multiple sessions concurrently running.
7:32:26If that is the case, if multiple
7:32:28sessions can run concurrently, each one
7:32:30should have its own LLM client
7:32:32connection because in one project you're
7:32:35connected to one LLM. In another
7:32:37project, you're connected to another LLM
7:32:39because each one can have their own
7:32:41config. Right? Then you have context
7:32:44manager. Each one maintains its own
7:32:46context. Each one has its own tool
7:32:49registry. And this will be clearer. Why?
7:32:51Because you're just not going to have
7:32:53default built-in tools. We are also
7:32:55going to have tool discovery which can
7:32:58be very specific to the folder you are
7:33:00in the project you're working on or it
7:33:03can be systemwide I believe. Then there
7:33:05are hooks which will also be present
7:33:07within session and MCPS obviously which
7:33:10can also be specific to one project. So
7:33:13that's related to tool registry. But
7:33:16anyways, enough talk. Let's just get
7:33:18into session and after that we'll
7:33:21composite everything within a session.
7:33:23Lots of stuff to do. Let's get into it.
7:33:25So I'll open up the sidebar and over
7:33:27here in the agent folder we're going to
7:33:29create the session. py file. Now within
7:33:32this we're going to have a simple class
7:33:34session and then we're going to have an
7:33:37init function. In this init function
7:33:39we're going to instantiate well
7:33:40everything that we instantiated in
7:33:42agent. py file. So let's take this
7:33:46config.
7:33:48We'll need the config. Let's take it
7:33:50from here. Pass it in. Cool. Now we need
7:33:53have to take the config from the
7:33:54constructor as well. Let's import it
7:33:57from config.config.
7:33:59After that the next thing we'll require
7:34:01is the client as well. So let's have
7:34:03self.client equal to llmclient. We'll
7:34:06import lm client here. After that we
7:34:09will require the context manager and the
7:34:11tool registry as well. So everything
7:34:15just gets instantiated over here. I
7:34:18think cell.config.config
7:34:20like the config thing also needs to be
7:34:22instantiated over here because config I
7:34:25believe is being used somewhere down
7:34:27here as you can see for the current
7:34:28working directory and even for some
7:34:30other thing the max turns. Yeah. So we
7:34:33will have to instantiate config here as
7:34:36well. So let's just do it and then we
7:34:38are going to create an instance of
7:34:39session here which will be used every
7:34:41single time. Now for context manager
7:34:44we'll have context domanager import
7:34:46context manager and then we'll also
7:34:48import default registry. That looks good
7:34:51to me. Now all the other things that
7:34:54we're going to initialize here are going
7:34:56to be related to let's say compression
7:34:58or loop detection or let's say a hook
7:35:01system or the MCP manager. Everything
7:35:05will come in over here. Some there are
7:35:07some things that we will still require
7:35:09like the session ID. So I'll just create
7:35:11a tier itself. We have self dot session
7:35:14ID equal to string. And then we have UU
7:35:17ID. We'll import UU ID. It will just
7:35:19help us generate a unique ID because a
7:35:22session ID just needs to be unique. Now
7:35:24you might be wondering why do we need a
7:35:26session ID? Well, it will be really
7:35:28useful because let's say we have
7:35:30multiple sessions and we know in the
7:35:32future we'll be adding functionality to
7:35:35save a session. So if you are in a
7:35:38session and you want to save it so that
7:35:40you can come back to it later on the
7:35:42session ID will help us know that. So
7:35:45this will be the unique identifier for
7:35:47that session. Then we also have created
7:35:51at and updated at which is also going to
7:35:53be shown whenever the user wants to view
7:35:57all the saved sessions or when
7:36:00checkpointing is there because what time
7:36:03did was the session created what time
7:36:04was it updated? We have to know that. So
7:36:07the self.created at is over here and the
7:36:10updated at is also present. Updated at
7:36:14will change whenever you know we add a
7:36:16new turn or let's say we try to save a
7:36:19session another time. So let's say we
7:36:21have a session we save it and then we
7:36:24come back to it 2 days later and then we
7:36:27again save the session after making some
7:36:29changes in it. Let's say I asked another
7:36:31message to my agent that will trigger
7:36:34the updated at. Created at will remain a
7:36:37constant. So yeah, that's about session.
7:36:40We'll just create one function which is
7:36:43to increment the turn count and just
7:36:45return it. So we'll have def increment
7:36:48turn where we'll get self and the point
7:36:51of this is just returning an integer.
7:36:54What this will do is keep track of a
7:36:56turn count which is something we'll have
7:36:58to create right at the top. So we have
7:37:01self dot turn count. This is a private
7:37:04variable. It shouldn't be visible
7:37:05outside. The point of this turn count is
7:37:08to you know has some sort of statistic
7:37:12about how many turns have already been
7:37:14taken place. Right? So let's say I save
7:37:16a session. Okay. I had 20 turns. Then I
7:37:20save the session. Then I go create
7:37:23another session that has let's say 10
7:37:26turns in it. I save that as well. Then I
7:37:29come back to the first session. Now I
7:37:32want to resume from this 20 turns,
7:37:34right? I want to discard these 10 turns
7:37:36and start from this 20 turns. So first
7:37:39of all I need to know how many turns
7:37:41there are. Once we get to know how many
7:37:44turns there are, I can just resume from
7:37:46wherever I left off. So that's where
7:37:48this turn count will help us also to
7:37:51display it as a statistic to the user
7:37:54whenever you know they want to inquire
7:37:57about what is the usage of this session
7:38:00how what how many tokens have we used or
7:38:02how many turns have been taken place how
7:38:04many messages are there all of that so
7:38:07let's remove it and all we need to do
7:38:09here is plus equals 1 and since we've
7:38:12incremented a turn you know that means
7:38:15we have to update the updated at as well
7:38:18because if this agent does one thing,
7:38:20you've updated the session essentially.
7:38:23So let's just copy it and paste it down
7:38:24here and then we can just return the
7:38:28count that we have right now. Return
7:38:30self.turn
7:38:32count. So these are the two methods as
7:38:34of now. Later on we're going to have
7:38:35many more because we'll have to create a
7:38:38checkpoint, create a session, list the
7:38:39sessions, resume a session. So yeah,
7:38:42lots of stuff to be done there but for
7:38:46now good enough. Now come back to the
7:38:48agent. We'll remove the unused imports
7:38:51where this is the part of refactoring I
7:38:53was talking about. And now I can just
7:38:56use this session everywhere. So I have
7:38:58self dot session is equal to session. So
7:39:01I'll import from agent session. So
7:39:03whenever I create a new session instance
7:39:06all of them are created. And now instead
7:39:09of using cell.context context manager
7:39:11just like that we'll do self dot session
7:39:13doc context manager cool now same thing
7:39:18for everything else
7:39:20except for config because config is
7:39:23instantiated correctly for tool registry
7:39:25we'll have self dot session dot tool
7:39:29registry
7:39:30then I'll copy again and then we have
7:39:33context manager again let's just paste
7:39:35that in another context manager we'll
7:39:38paste that in then we have tool registry
7:39:41we'll call that session again and then
7:39:44in the context manager again we'll just
7:39:46pass in session now one thing we
7:39:48forgotten to do is remove the selfclient
7:39:52because well the client is never there
7:39:54so what we need to do instead is if the
7:39:58self do sessionclient exists in that
7:40:01case I want to do self dot session
7:40:02doclient close and then we'll do self
7:40:05dot session doclient is equal to null
7:40:08and actually what we can do is just set
7:40:11the entire session to null, right?
7:40:13Because well, whenever we exit this
7:40:16agent, the entire session is over. So we
7:40:18can just check that hey, if the self dot
7:40:21session is present and self
7:40:23session.client is present, then we'll
7:40:25close the client connection, we'll set
7:40:27client to null or we might just set
7:40:31self.ession to null. If we set this to
7:40:33null, everything else will be garbage
7:40:35collected. So client will also be null.
7:40:37That looks good to me. Now I just have
7:40:39to go at the top and update this session
7:40:42to either be of the type of session or
7:40:44null. That's good. Now I just need to
7:40:48ensure that wherever we are using
7:40:50context manager,
7:40:52we just change it back to session. And
7:40:55yeah, that looks good. We've only
7:40:57created context manager in one place,
7:41:00which is session. So we can give this a
7:41:02shot. Let's see if this works. Let's run
7:41:05python main. py and we run into an
7:41:07error. It says session requires a config
7:41:10and I did not pass that in. So let's go
7:41:12to agent.py and here we can pass in
7:41:15self.config.
7:41:16That looks good. Let's run it again. And
7:41:19now we have another error. Module date
7:41:21time has no attribute now. And the
7:41:25reason for that is something like this.
7:41:27So if we go to session again and here we
7:41:31imported datetime, correct? But datetime
7:41:33is a module. So instead of doing just
7:41:35import date time, we'll do from datetime
7:41:39import datetime. And now it should work
7:41:42fine. So we can just run this again. And
7:41:45yeah, we have the model with us. Now we
7:41:48can try to run something. Hey, how are
7:41:51you? That seems like a good message. And
7:41:53we run into an error. Agent object has
7:41:56no attribute client. We forgot to change
7:42:00the session somewhere. So we'll just
7:42:02search for self.client. And yeah, we
7:42:04have to do self session.client over
7:42:07here. Now let's try to run it again.
7:42:11So, yep. Hey, how are you doing? Read
7:42:16main. py file for me.
7:42:20And yeah, it does give us the response,
7:42:22but the tool calling fails and we see
7:42:24the error message. Agent object has no
7:42:26attribute tool registry. So somewhere we
7:42:29called cell.tagent.tool registry.
7:42:31Instead, we need to do self.agent.tool
7:42:36registry. So, we'll just search for
7:42:38self.agent.tool
7:42:40registry. We called it in main. py. And
7:42:43now we'll just update this to be
7:42:45self.agent.tool
7:42:47registry.get.
7:42:49And yep, that's it. We can try again.
7:42:53Again, I'll put in the same message. And
7:42:56now let's wait for the response. And
7:42:58yep, that works out. It says, "I'm doing
7:43:01well. Thanks. Let me read the main py
7:43:03file and it's able to read all of the
7:43:05147 lines. So yeah, that's about
7:43:07session. Now we can close all or
7:43:10actually we forgot to do one thing. So I
7:43:12created the function in session called
7:43:15increment turn. Now I'll just like to
7:43:18update this self.turn count right just
7:43:21in case in the future we want to display
7:43:22this turn count out in some way in the
7:43:26session we'll be able to just use this
7:43:28turn count. So let's just call it and
7:43:29it's quite easy.
7:43:31We just want to increase the turn count
7:43:33whenever we have a new turn. So we just
7:43:36have to call the function somewhere in
7:43:38this for loop, right? Because we're
7:43:40going for every turn here. And I say we
7:43:43can just do it over here. So we'll have
7:43:46self dot session dot increment turn. And
7:43:49that will give us a new turn number. If
7:43:52you want to do something with that turn
7:43:53number, go for it. But it doesn't really
7:43:55help us. So yeah, we'll just keep it
7:43:58like this as of now. Just in case we
7:44:00want to use it sometime, we can. There's
7:44:03no harm. So this is a good refactoring
7:44:06to do. Now we can close all of the save
7:44:09files. And now we can again start
7:44:11working on the next aspect which is
7:44:13adding all of the built-in tools now. So
7:44:16we have read file with us. The next
7:44:18thing I would like to work on is write
7:44:20file.
7:44:23So let's go ahead and create the class
7:44:25called write file tool. And this file
7:44:28tool is going to extend the base model.
7:44:30Oh sorry it's going to extend the tool
7:44:32that we created in tools.base.
7:44:35And then we know we need to define some
7:44:38fe things some attributes like name
7:44:40which is going to be write file. This is
7:44:41the name of the tool. Then we have a
7:44:43description and we know the description
7:44:46can be quite a bit long. So instead of
7:44:49writing it out I'm just going to paste
7:44:51it. But we'll go over this. The
7:44:53description is write content to a file,
7:44:55creates the file if it doesn't exist or
7:44:58overrides if it does. Parent directories
7:45:00are created automatically. Use this for
7:45:03creating new files or completely
7:45:05replacing file contents. For partial
7:45:07modifications, use the edit tool
7:45:09instead. So in the tool description, we
7:45:11are specifically pointing out what this
7:45:13write file tool is used for. Even in our
7:45:16system prompt, we mentioned something
7:45:18related to write file and edit file
7:45:20differences. We're going to add that
7:45:22part of the system prompt later on. But
7:45:24yeah, it is important to point out the
7:45:27difference in the places where it
7:45:29matters. For this one, it really matters
7:45:31because if it if the LLM is deciding
7:45:33what tool to call, we need to tell it
7:45:35that hey, this is only for writing or
7:45:37overwriting. If you want to make edits,
7:45:39there is no point rewriting the entire
7:45:42file with your edits present in it. Just
7:45:44use the edit tool. that's already
7:45:45present. So yeah, that's about it. That
7:45:49saves us tokens by the way. Now we can
7:45:51just say that the kind of this tool is
7:45:54tool kind. We need to import that from
7:45:56tools.base as well. And this is the
7:45:58write tool, right? So that's it. Now we
7:46:01need to define schema. Now the schema
7:46:04for write file was what is it going to
7:46:06look like? Well, we can have write
7:46:10file params and then the base model will
7:46:14be extended. Uh, and it's not going to
7:46:16come from anthropic. You need to import
7:46:18it from pyantic. I'm not getting the
7:46:20autoimp import. So, let me import it
7:46:22myself. And there we go. After that, we
7:46:26can define the path. Which path do you
7:46:28want to write in? And that will be a
7:46:30string. Then we'll pass in the field
7:46:32from pyantic. We'll import the field.
7:46:35And rest of the fields are going to be
7:46:36the same except description. In
7:46:38description, we're going to say path to
7:46:40the file to write. What does this
7:46:43parameter do? Essentially, we just want
7:46:46the path to the file to write. And this
7:46:48should be relative to the working
7:46:50directory
7:46:53or it can be absolute as well. That's
7:46:56important to point out. And I think we
7:46:58also did that when we were in read file.
7:47:00Correct? So yeah, that's it. Now another
7:47:04thing we can add is a folder. So you
7:47:08know if you want to create the
7:47:10directories or not. So let's say the
7:47:12path is something like this where you
7:47:15have config /hello/ain.
7:47:20py. Cool. So you know we have to create
7:47:23the config folder if it doesn't already
7:47:25exist. We have to create the hello
7:47:26folder if it doesn't already exist. And
7:47:28then you have the main. py file. That's
7:47:31what we're doing here. that that can be
7:47:33one argument that we'll add and that
7:47:35will be called create directories and
7:47:38that will be boolean which is equal to
7:47:40field and then let's give it a default
7:47:43value of true and let's say if the LLM
7:47:47says you don't have to create
7:47:48directories it can be set to false and
7:47:50then you have a description which is
7:47:52create parent directories
7:47:55if they don't exist by default it is
7:47:58true and in most cases the LLM will not
7:48:00really care true value makes a lot of
7:48:03sense but good to have that in and one
7:48:06thing I totally forgot is content
7:48:08because we are writing to a file but
7:48:10what is the content that we are trying
7:48:12to write that comes in as well and then
7:48:15the description which is content to
7:48:19write to the file cool so that looks
7:48:24good to me now I can just take this
7:48:26write file parameters and specify it as
7:48:29the schema So yeah, that's it. Now let's
7:48:33go ahead and define the execute
7:48:35function. So we have async defex
7:48:36execute. Here we're going to get self
7:48:39invocation just like we did in read
7:48:42file. Then we get tool invocation here.
7:48:44Let's import it from tools.base and
7:48:47we're going to return tool result.
7:48:49Remember
7:48:52we have made our structure of code so
7:48:54interesting that we only have to define
7:48:57this write file tool. Then we need to go
7:48:59in the init py and here we need to
7:49:02specify the built-in tool as write file
7:49:04tool and let's import it from
7:49:06tools.builtin.right
7:49:08file. And now everything should work as
7:49:11expected. You know this is how elegant
7:49:14our solution is. We have to create a new
7:49:16built-in in this folder and we have to
7:49:18define the execute get confirmation all
7:49:21of those functions and then we can just
7:49:23def pass in the class over here and we
7:49:26know get all built-in tools is called
7:49:28within the registry and from the
7:49:30registry we have get default tools all
7:49:32of that thing create default registry we
7:49:35are exposing that out into the session
7:49:37and the session is being used within the
7:49:39agent. So this makes it really easy for
7:49:42us because we don't have to edit any of
7:49:44the existing codebase by a lot. We only
7:49:48have to edit one file which is the init.
7:49:50py where we just have to add a new tool
7:49:53and the rest is just creating a new file
7:49:55from scratch. So this is one of the
7:49:58solid principles especially the O which
7:50:01is open or close principle meaning the
7:50:04codebase should be open for creation but
7:50:06closed for modification. But anyways,
7:50:09let's just come back to this. We have
7:50:12the invocation which is giving us a set
7:50:14of parameters. We just need to convert
7:50:16that into write file parameters. So we
7:50:18have params is equal to write file
7:50:20params. Then we'll pass in the
7:50:22invocation.p parameters by
7:50:24deconstructing it similar to what we did
7:50:27in read file. Also we get the path from
7:50:31this invocation. So what I want to do is
7:50:34resolve that path. So I'll just call the
7:50:37resolve path function which is coming
7:50:39from utilips.path.
7:50:41We've already used this many times
7:50:42before. So we have invocation dot
7:50:46current working directory which is going
7:50:48to be the base directory for this
7:50:50resolve path. And then you have the path
7:50:53parameter that is passed through this
7:50:56parameter. Right? So we have
7:50:58params.path.
7:51:00So if you're having difficulty
7:51:02understanding the invocation.curren
7:51:04current working directory is wherever we
7:51:06are in right now, whatever folder we are
7:51:08in. And parents.path is what the llm is
7:51:11telling us to write to. And now we're
7:51:14just resolving the path so that you know
7:51:15we have kind of an absolute or a total
7:51:18path so that we can directly write into
7:51:21it. Now we first have to check if this
7:51:24is a new file or not because if this is
7:51:26not a new file then we have to read the
7:51:29existing text because it will help us in
7:51:32creating a diff. diff is especially
7:51:34useful when you have a file that already
7:51:36exists. Okay, so let's say I have this
7:51:38write file. I tell my agent to continue
7:51:40writing this function. So you know it
7:51:44starts writing all of these things. It
7:51:46completes it. But then it realizes that
7:51:48maybe I've made some bug somewhere over
7:51:51here. So it just decides that instead of
7:51:53using the edit tool, it will be more
7:51:55convenient to just use the right tool
7:51:58because that will get our job done
7:51:59easier easily. So what it will do is
7:52:03just rewrite this entire file. And if it
7:52:05rewrites the entire file, the diffing
7:52:07view will help us show the minus and the
7:52:10plus sign. So let's say we had a bug
7:52:12over here. It just removes that line and
7:52:15replaces it with its own. So that is a
7:52:18view that will be given to the user kind
7:52:20of like git or especially GitHub. Also,
7:52:23this will be something that we can
7:52:25probably pass through the metadata then
7:52:27when we try to return a tool result. So
7:52:30that will also be useful. So let's just
7:52:32track this if it's a new file then we'll
7:52:35just have not path.exist because if the
7:52:38path already exists it's not a new file.
7:52:41And now we can just check that hey if it
7:52:43is not a new file in that case what I'd
7:52:46like to do is store the old content. I
7:52:48would like to read the text. So I can
7:52:52just do old content is equal to path dot
7:52:56read text. And now I can let's say pass
7:53:00in the encoding if I want. The encoding
7:53:03can be UTF8.
7:53:05Let's put it in a try and catch block.
7:53:08So we have try except and if we run into
7:53:12this error, I'll just pass because even
7:53:14if the old content does not exist, I
7:53:17don't want this entire function to fail
7:53:20because it's not the most relevant part
7:53:22of our execution tool. The most relevant
7:53:25part is writing to the file path. So we
7:53:28will carry that on even if you're not
7:53:30able to read this text. And yeah, so
7:53:34that's it. Also, this old content is not
7:53:37defined if we run into an exception or
7:53:39if the file does not exist. So let me
7:53:42fix that by having an empty string
7:53:45outside of this if condition. Cool. Now
7:53:49let's try to create the parent
7:53:52directories if it's needed and then
7:53:54write the file. So we'll have a try
7:53:56block here. Let's just put an except as
7:53:59well. And there will be specific errors.
7:54:02One of the errors we can run into is OS
7:54:05error. So let's catch it as E. And then
7:54:08we'll just return tool result
7:54:12dot error result. And then we'll just
7:54:15say failed to write file. and we'll pass
7:54:18in the error string. That's it. Now
7:54:22let's try to create the parent
7:54:25directories if the parameter says so. So
7:54:28if params dot create directories is
7:54:32true, in that case we want to ensure
7:54:34that the parent directory already
7:54:36exists, right? And that is actually not
7:54:39very difficult. All we need to do is
7:54:42create a new one using path. But it is a
7:54:46reusable function which I can use across
7:54:48multiple files. So I'll just create it
7:54:50in utils path py. And here let's just
7:54:54create a new function ensure parent
7:54:57directory. This will give us a path
7:55:00which can be a string or a path object
7:55:03itself and it will return a path no
7:55:06matter what. Then we have path is equal
7:55:08to path and then we pass in the path
7:55:10string or the path path object that we
7:55:12get and then we have path.p parent.
7:55:17So it will just go to its parent
7:55:18directory the logical parent and then
7:55:21you have make directory passed in. Now
7:55:24the parents is equal to true and if they
7:55:27already exist then we don't want to
7:55:29return any error because it's totally
7:55:30fine. Our our entire point is just
7:55:33ensuring if the parent directory is
7:55:35there and if it does not exist then we
7:55:37just create the parent directory and
7:55:39then we just return the path from here.
7:55:43Now we can go ahead and call this
7:55:44function over here in parent directory.
7:55:46Let's import it from utils.path and we
7:55:49can just pass in the path over here. It
7:55:52is this path. All right. Now what if the
7:55:55path parent does not really exist? In
7:55:59that case we would just like to give out
7:56:02an error saying that hey this path does
7:56:04not exist. What are you really talking
7:56:06about? Because maybe the llm
7:56:08hallucinated and it will do the wrong
7:56:11thing. we don't want it to do the wrong
7:56:12thing. So we'll handle that case where
7:56:15if the path parent does not exist in
7:56:19that case we'll return the tool result
7:56:22with error message. So we have error
7:56:25result passed in and then we have parent
7:56:28directory does not exist and then maybe
7:56:32we can also pass in the path. So that
7:56:35the llm has more context. Now if this
7:56:38parent directory is created or it is
7:56:41present, our job is done. Now we just
7:56:44have to write to the file. So we have
7:56:46path dot write
7:56:48text and then we'll just pass in the
7:56:51parameters dotc content and maybe we can
7:56:54also specify the encoding which is UTF8.
7:56:58And now we can just return the tool
7:57:01result dot success result. Now the
7:57:05success result will first have the
7:57:08output. What is the output going to be?
7:57:10Well, I would like to format the output
7:57:12in this way. The first line should have
7:57:16the action that was done. So, did it
7:57:19create a new file or did it update a new
7:57:21file? If it updates a new file,
7:57:25I just want updated to be written. And
7:57:27if it creates a new file, I want created
7:57:29to be written. And then I want to
7:57:31specify the path where this happened.
7:57:34And then I want to write down the number
7:57:36of lines that are present. So let's just
7:57:39extract all of those details. The first
7:57:41one is action. The action is created if
7:57:45it is a new file that was done.
7:57:47Otherwise we have updated.
7:57:50Then I want to show the path. We already
7:57:54have the path variable with us. And the
7:57:56next thing I want is the total number of
7:57:58lines. So we'll have the line count
7:58:02which is equal to the length of params
7:58:05dot content dotsplit lines right because
7:58:08if we split the lines we'll get to know
7:58:11how many lines there are cool now let's
7:58:13just have the output here so we have the
7:58:16first thing as action then we leave some
7:58:18space then we have path and then we
7:58:21leave some space again and then we have
7:58:23line count which just says these many
7:58:25lines exist now this output is
7:58:28interesting. Just remember it because we
7:58:32might want to format this nicely on the
7:58:36TUI part of things if needed. But this
7:58:39is definitely something that's going to
7:58:40the LLM. Right now in success result,
7:58:43there's another thing I would like to
7:58:45pass which is a diff. And quickly I'll
7:58:47just show you the result of the diff. So
7:58:49if the file was null, it will have
7:58:51something like this. We just minusing
7:58:54null from here. There's nothing. And
7:58:56then we are having a print hello world.
7:58:58Obviously when this gets bigger you'll
7:59:00be able to see better changes but that's
7:59:02what I'm talking about a plus sign
7:59:05whatever lines added and minus whatever
7:59:07we deleted. So now to create this diff
7:59:09we'll go to the base. py file because we
7:59:11have to update tool result right in this
7:59:14tool result we have to add a new field
7:59:16called diff. And that diff will help us
7:59:20know and display hints. So this diff is
7:59:24going to be of the type of file diff.
7:59:26That is another class that we're going
7:59:28to create or it will be null because you
7:59:30know if we have a read file there's not
7:59:32going to be any diff. So we don't need a
7:59:34file diff. But in the success result we
7:59:38can take in the diff through the keyword
7:59:41arguments. So that's good. Now let's
7:59:43just go ahead and create this file diff
7:59:46data class. So we have a rate data class
7:59:49file diff. Then we have the path. What
7:59:52path do we want to create the file def?
7:59:55Then we have the old content which is a
7:59:57string. Then we have the new content
7:59:59which is the string. Then we have well
8:00:03let's have some other parameters that
8:00:06will also help us display some nice
8:00:09stuff. For example, you know, if it's a
8:00:11new file, then maybe we want to add that
8:00:14dev null thing that was shown. So you
8:00:17know I'm adding this dev null manually
8:00:19because you know if it just has a minus
8:00:22sign with an empty line over here it
8:00:24will make no sense. So if it's a new
8:00:26file we'll have this and if we're
8:00:29deleting a file for example it will also
8:00:32be useful then. So let's drag this to
8:00:35two states. We have is new file which is
8:00:37a boolean by default it is false. it is
8:00:40not a new file or it would make sense to
8:00:44set it to true but anyways then we have
8:00:46is deletion which is going to be a
8:00:49boolean value and it is going to be
8:00:51false as well by default and then you
8:00:55create a function over here which will
8:00:57generate a unified diff string the
8:01:00reason we have to maintain this diffing
8:01:02string of minuses and plus is because by
8:01:06default I don't think that rich has
8:01:09access to a diffing tool as such. There
8:01:12is no class that will help us render a
8:01:15diff given two strings. But in Python,
8:01:18there is a built-in library called
8:01:21difflip that we can use and that will
8:01:23help us create that string. So let's
8:01:26just go ahead and create that string. So
8:01:28first of all, we'll have to import diff
8:01:30lib. This is a built-in module, so we
8:01:33don't have to worry much. After that we
8:01:35have to create old lines and new lines
8:01:39because well we are creating a diffing
8:01:40tool right we have to take this content
8:01:42and give it in the form of lines array
8:01:45and then we have this new content which
8:01:47you have to give it in a lines array
8:01:48format or a list format and then the
8:01:51diffing tool will just compare and add
8:01:53the minus signs plus signs wherever
8:01:55needed. We could really do it on our own
8:01:58as well but why why should we reinvent
8:02:00the wheel? We'll just go ahead and do
8:02:03old lines is equal to old text. Let me
8:02:06get self dot old content. Then we'll
8:02:09split the lines. And then we can just
8:02:12say keep ends is equal to true. Keep
8:02:15ends is equal to true just means that it
8:02:18will not remove any backslash end at the
8:02:21end if it is present. And similar to
8:02:23this we'll have new lines as well which
8:02:26is just equal to self dot new
8:02:27content.plit split line and it will keep
8:02:30ends is equal to true. Then we will have
8:02:32if the old lines exist you know it's not
8:02:35an empty array or it is not null and the
8:02:39old lines
8:02:41last element of the list does not end
8:02:44with a backslash n then we want to add
8:02:47that back slashn because we do want the
8:02:50last line to be a new line for this diff
8:02:52flip to work. So we'll have and not all
8:02:56lines at -1 dot ends with and then we
8:03:00have a back slash n.
8:03:03In that case we have all lines at -1 is
8:03:06equal or plus equals back slashn.
8:03:10The reason we're not doing is equal to
8:03:13is because if there is some content on
8:03:15the last line we'll be overriding it
8:03:17with equal to. Instead I don't want to
8:03:20override it. I'll just add whatever
8:03:22content exists with backslash n. Now
8:03:25I'll just copy it and paste it down
8:03:27again. And then we'll check if new lines
8:03:30exists and new lines at -1 does not end
8:03:34with ne back slashn then we have new
8:03:37lines at -1 + back slashn. After that we
8:03:42can just do deflib dot ununified diff.
8:03:45That's how you compare the two sequences
8:03:47and generate the delta. Delta is just
8:03:50the changes that were done. And then you
8:03:52need to pass in the old lines. Then you
8:03:54pass in the new lines. Then you need to
8:03:57specify the old name of the file and the
8:04:00new name of the file as well which is
8:04:03done using from file and to file. From
8:04:05file is what file is the old name of the
8:04:10file and what is the new name of the
8:04:12file. So let's just create those
8:04:14variables as well. And this is where the
8:04:17is new file and is deletion will help
8:04:19us. So the old name is going to be back
8:04:23forward slashdev forward slashnull if it
8:04:27is a new file that's created. Correct?
8:04:30Because if it's a new file we had
8:04:32nothing before and now we're going to
8:04:34have something. And if it's not a new
8:04:36file then it's just going to be the
8:04:38string of self.path
8:04:40whatever path is mentioned over here.
8:04:43And the new name is going to be what?
8:04:46Well, it can also be the case that the
8:04:49file is deleted now. So, if the file is
8:04:52deleted, then we'll again have dev null.
8:04:56So, if is deletion, then we have dev
8:04:58null. Otherwise, we have self.path
8:05:01again. And now we can pass in the old
8:05:04name and the new name. And this is our
8:05:08unified diff. Let's just store it in a
8:05:11variable called diff. And you'll notice
8:05:13that we again get an iterator back. We
8:05:15passed in a bunch of iterators. Lists
8:05:18are iterators. And now we get an
8:05:20iterator back. I just want to convert
8:05:23them into a string format and return it
8:05:26from here. So we have return dot join.
8:05:29And then you have the diff. I'm not
8:05:31doing back slash n because diff will
8:05:33already take care of all of that. We
8:05:36just have to join everything in this
8:05:38list or in this iterator. Now we can go
8:05:41ahead and mention the return type of
8:05:42this function which is a string and yeah
8:05:45that looks good to me. We also have the
8:05:48file diff attached here and in the
8:05:51success result it will be taken through
8:05:53the keyword arguments. So let's go to
8:05:56the write file and here we can pass in
8:05:59the diff. So let's go ahead and set diff
8:06:01is equal to file diff and we have to
8:06:04import that from tools.base. And now I
8:06:06can pass in the path, old content, new
8:06:08content, all of that. So path is equal
8:06:11to path. Then you have old content is
8:06:13equal to old content. Then new content
8:06:17is equal to parameters dot content
8:06:20because old content is already created
8:06:22as a variable here. New content is just
8:06:25parameters whatever we getting from the
8:06:27LLM, right? And if it is a new file or
8:06:30not, we will know that by is new file.
8:06:35Great. Now we can also have metadata
8:06:38attached here. So the metadata is equal
8:06:41to and what metadata do we want to set
8:06:44from send from this success result?
8:06:47Well, I would like to send the path
8:06:50because the diffing does not really have
8:06:52the path because remember when we have
8:06:55the diff we'll just call dot2 unified
8:06:57diff and that will give us the diffing
8:06:59string. But what from metadata I would
8:07:02like to send the path again as well.
8:07:04because that will be used to display to
8:07:06the user. So there's a clear separation.
8:07:09Even if you don't send it and use it
8:07:10from diff, it's totally fine. This is
8:07:13probably redundant, but all right. We
8:07:15have is new file. Again, you can take
8:07:18that from the diff as well. Then we'll
8:07:20also send in the lines, which is the
8:07:22line count. And then we have the bytes,
8:07:26which is well, we'll have to do
8:07:29parameters.content.ccode.
8:07:31And then we encode it in UTF8 format.
8:07:35And this will return to us the bytes.
8:07:37Now I want to send the length of this
8:07:39bytes. So I have length of this because
8:07:42I don't want to send in the actual
8:07:44bytes, right? What is the point of that?
8:07:46I just want to send the length of the
8:07:47bytes. The reason we're sending all of
8:07:49that is because if you go to this or
8:07:52actually you can't go over there. But if
8:07:54you just look at this write file tool
8:07:56confirmation that we have, it says path
8:07:58hello world. py and the content is one
8:08:01lines 22 bytes in total that's why we're
8:08:05sending across the bytes just for UI
8:08:08related stuff and yeah that is it about
8:08:11the write file tool all I need to do is
8:08:14go to the main py file and here just see
8:08:17if I'm getting any event related to tool
8:08:21call start and tool call end related to
8:08:24writing a file so let's just print out
8:08:28the event here And now I can start my AI
8:08:31agent. So I'll just hit enter and I'll
8:08:34say write Python file for me with hello
8:08:38world in it. Let's run it. And we get
8:08:41agent start. Then we get text delta.
8:08:44Then we get more text delta.
8:08:47After that we get a tool called start.
8:08:49We do get write file showing up with the
8:08:52content and the path because those are
8:08:55the arguments. And then we also get the
8:08:57write file with some output being
8:08:59displayed but it's not really displayed
8:09:02because it's halfbaked. Right? We've
8:09:04added the tool. The LLM already has
8:09:07context to the tool because our entire
8:09:09integration is so well done that we just
8:09:11have to create a file add it in one list
8:09:14and everything just works. But the UI
8:09:17obviously is messed up. If you want you
8:09:19can display the content like this. But
8:09:22I'm not really interested in showing the
8:09:24content here. I would like to show a
8:09:26final diffing view in write file itself.
8:09:29So let me remove this event. Everything
8:09:31is working. Let me just clear everything
8:09:33off and run it again so that I have a
8:09:35more better view of what UI changes we
8:09:38need to make. So there we go. The
8:09:40arguments are showing up correctly. But
8:09:42those are not the arguments we want,
8:09:44right? Or I don't want at least. If you
8:09:46want, go ahead. Also, as you can see, we
8:09:49have the hello. py file here because
8:09:51well, the file is written. We do have
8:09:53the tool call saying it's completely
8:09:56done, correctly done. So that's good.
8:09:58And we don't really have to manage
8:09:59anything in the main. py file for this
8:10:01because everything is done within the
8:10:04TUI because that's what handles the
8:10:06entire UI related stuff. So we can just
8:10:09go down here and we have the preferred
8:10:10order. Earlier you saw that content was
8:10:14written out first and then the path.
8:10:15Then later on we have the path written
8:10:17out first and then the content. This
8:10:20happens because you have not specified
8:10:21the preferred order. So let's just
8:10:23passse and write file here with the
8:10:25preferred order. The first thing should
8:10:27always be a path. The second thing
8:10:30should always be the create directories
8:10:33thing that we have because well the
8:10:36content if we're going to display it at
8:10:38all is going to be the last thing that
8:10:40should be displayed because it's going
8:10:42to take up a lot of room, right? Content
8:10:44is always going to be a bigger string
8:10:45than both of them. So it just makes
8:10:48sense to put them that thing in the end.
8:10:50And now we would also like to avoid
8:10:53dumping this entire content string here
8:10:56because we are anyways going to display
8:10:57it if the tool call is successful. So
8:10:59what I'll do is go to this tool call
8:11:02start function and here whenever we call
8:11:05render arx table we're just passing in
8:11:08the value just like that. Right? So
8:11:11instead of just passing in the value we
8:11:14can format it a little bit. We can just
8:11:16say that hey if the key is let's say is
8:11:21content or later on we're going to add
8:11:23more things like old string new string
8:11:25because the next tool we're going to
8:11:27work on is edit file. If it's an edit
8:11:29file, even then we'll have a huge blob
8:11:33text, some sort of big text coming in,
8:11:36right? What was the old text? What is
8:11:38the new text that we want to remove? So
8:11:40that will be a lot of hassle as well.
8:11:41That's why we're creating this as a set
8:11:44and we're saying that hey if the key in
8:11:46content if the key is content or later
8:11:49on we'll have old string new string let
8:11:51me just put it so that you know you
8:11:54don't get weirded out by the set. So if
8:11:56we have any of these in that case we
8:11:58want we don't want to dump the that huge
8:12:01entire block thing that we had. Instead
8:12:04what we're going to do is display the
8:12:07lines and the bytes. So we'll have line
8:12:09count which is equal to the length of
8:12:12value dotsplit lines. Now we obviously
8:12:15need that value. It is this particular
8:12:17thing. But for that first we will also
8:12:20have to check that hey this value that
8:12:23we're dealing with is a string or not.
8:12:27So first we'll have if is instance value
8:12:30of the type of string. In that case
8:12:33we'll check the key if it is in content
8:12:35and if the key is content then we find
8:12:38out the line count by splitting the
8:12:42lines or you know if that length is not
8:12:46specified we somehow return nothing from
8:12:49here we can just have zero and then we
8:12:52have byte count which is equal to length
8:12:54of value dot encode and we'll just
8:12:57convert this into byte format where we
8:12:59have UTF8 and errors can be set to
8:13:03replace. As you can see, by default, it
8:13:06is set to strict, meaning that the
8:13:08encoding raises a unic code encode
8:13:11error. Other possible values are ignore,
8:13:13replace, and this particular thing. And
8:13:15if this works out in that case, I'll
8:13:18just update the value that we're getting
8:13:20here. I would like to say that the value
8:13:22is now equal to string where we have the
8:13:27line count passed in. Then we are going
8:13:29to have lines. Then we need this
8:13:32particular dot and then the number of
8:13:34bytes. So let's put put in that dot. And
8:13:37then we have the byte count and then
8:13:41bytes. And then we are already doing
8:13:43table.add row key value. And that should
8:13:46work out. So I'll save it. And that
8:13:48should that should be it for tool call
8:13:50start. Now for tool call complete I'll
8:13:53have to find that function. So let me
8:13:55just search it in this file. Tool call
8:13:57complete. Here we have already said all
8:14:00the logic related to creating the block.
8:14:03That's why we had this entire thing. But
8:14:05the content here is not defined. The
8:14:07content here is not defined because we
8:14:10had this logic. We're saying that if the
8:14:12tool is read file, then we'll do all of
8:14:14these things. But if it's write file,
8:14:16we've not mentioned anything. And that's
8:14:18what I'm going to do now. So if the name
8:14:20of the tool is equal to let's say write
8:14:24file, let me just create this write
8:14:26file. And if it's a success and let's
8:14:30say the diff was also given to us. So if
8:14:33the diff exists and now we have to
8:14:35create a variable called diff and
8:14:38actually this diff should come from the
8:14:42tool called complete itself because you
8:14:44know it's attached to the tool result.
8:14:46We're taking success output everything
8:14:48from tool result. So we'll have a diff
8:14:51which can be a string or a null value.
8:14:54Now wherever this tool called complete
8:14:56is called which is in the main.py file I
8:14:59have to pass in event dot data.get
8:15:03diff and if it's not mentioned then
8:15:05we'll just pass in null. Cool. Just
8:15:08ensure you've passed in the correct
8:15:10positional argument. If positional
8:15:12arguments confuse you, you can do call
8:15:14ID is equal to this. Then you can do
8:15:16name is equal to this and so on.
8:15:20But these are the arguments. Nothing
8:15:22more will most likely be required now.
8:15:25So I'm good enough for now. I'll just go
8:15:29back to this logic. Now if the success
8:15:32is there, if the diff is there with us,
8:15:34we are going to display it correctly. So
8:15:37the very first thing I would like to
8:15:38display is the output line and for us
8:15:42the output line was already created in
8:15:44the meta data if you remember or sorry
8:15:47in the output here action path and the
8:15:50number of lines that were added. So
8:15:52let's just have output line which is
8:15:55equal to output strip if output.strip
8:15:59strip exists and it should but let's say
8:16:03in any case the LLM hallucinates and we
8:16:06get an empty output in that case we will
8:16:09just write down completed that hey your
8:16:11entire thing was completed we did not
8:16:13get an output as such it was an empty
8:16:15string at the end of the day so I'm just
8:16:18saying that yeah I've written the file
8:16:20it's completed and then we'll just do
8:16:22blogs append output line and now this
8:16:26output line needs to go within a text
8:16:28because blocks is just a bunch of
8:16:32rendering items and the style of this is
8:16:35going to be muted somewhat of a grayish
8:16:37color. And now we are going to have the
8:16:39div text which is just equal to diff.
8:16:42And the reason we have created diff text
8:16:43is because we're going to truncate it.
8:16:46So we have diff display is equal to and
8:16:50then we just call truncate text. Then
8:16:52we'll pass in the div text as the value.
8:16:56Then we have to specify the model name
8:16:59in truncate text. The model name is not
8:17:02passed in. But we can do the same thing
8:17:05as GPD4
8:17:07or we probably do have access to
8:17:10configuration here. Right? If I just go
8:17:13at the top, we do have access to
8:17:14configurator. So we can pass in the
8:17:16correct model. So we just have self
8:17:18do.config domod
8:17:20name. And then we can also specify the
8:17:23max number of tokens which is self dot
8:17:26max block tokens. I don't know if you've
8:17:28created a variable for that. I don't
8:17:31think so. But I believe the value was
8:17:33240. Yeah. So instead of just hard
8:17:36coding 240 here, I'll just go to the
8:17:39constructor and have self dot max block
8:17:43tokens which is 240 hardcoded here. You
8:17:45can take it from the configuration
8:17:47system as well. you know, how many
8:17:49tokens do you want to display out on the
8:17:51screen? This is definitely a user's
8:17:52choice because, you know, maybe they
8:17:55don't want their terminal to be overly
8:17:58saturated. And after putting in cell
8:18:00domax block tokens here, we can also
8:18:02pass it in over here. And that's about
8:18:04it. So, we have the div display
8:18:08as well. Now, let's just append it to
8:18:10blocks. So, we have blocks.append.
8:18:13And then we'll just pass in the syntax
8:18:15because I told you rich will not give us
8:18:18the diffing thing. We'll just pass it in
8:18:21syntax and pass in the diff display. Now
8:18:24within syntax you can specify the lexer
8:18:28to be a diff. That is totally possible
8:18:31and that will highlight stuff but it
8:18:33does not do the diffing itself. Remember
8:18:36the difference earlier we did the
8:18:38diffing ourselves so that we got the
8:18:40minus and the plus all of that logic but
8:18:43the syntax can help us display it nicely
8:18:47if we just pass in the lexer to be a
8:18:50diff and now the theme can be the same
8:18:53monokai and then you can also set word
8:18:57wrap to true and that's about it let's
8:19:00try to run this again so I'll just run
8:19:03python main py and I'll say create hello
8:19:06world python script for me please let's
8:19:11run it and still write file doesn't do
8:19:13anything for us I wonder where the error
8:19:15is so I understand why this error exists
8:19:18I went behind the scenes to debug it
8:19:20what I did is printed out the event data
8:19:24get diff and the value over here was
8:19:27null so I thought maybe the error wasn't
8:19:30right file maybe we did not pass in the
8:19:32diff properly through success result. So
8:19:35I went to success result, printed out
8:19:37the keyword arguments and that's printed
8:19:39as well. That means the problem is in
8:19:42the transportation between write file to
8:19:45the main.py. So probably the error is in
8:19:48agent. py and that's exactly where the
8:19:51error was in agent. py when we are
8:19:53processing the message we are tool call
8:19:56start. Now in tool call start we passed
8:19:58in all of these things. That's fine.
8:20:00That gets printed out correctly. But the
8:20:03problem is in tool call complete, we are
8:20:06just passing in the call ID, call name,
8:20:08and the result. Okay, that's fine too.
8:20:11But in tool call complete, we're not
8:20:13extracting the diff anymore. We're just
8:20:15extracting the metadata, truncated,
8:20:17success, and output. But what about the
8:20:20new diff that we have? So let's go ahead
8:20:22and add the diff property years as well.
8:20:25So we have result. And now you know
8:20:28whenever tool call complete event is
8:20:30sent to the main.py py file and we get
8:20:34it over here. The event data will have
8:20:37the diff value. Now let's try to run it
8:20:40again. I'll say create hello world py
8:20:44script for me please. Let's run it and
8:20:47we run into an error. It does print out
8:20:50path and the content nicely but when we
8:20:53are trying to display the div, it runs
8:20:55into some sort of error. Now this error
8:20:58exists because again if we go to
8:21:00events.py py where we have tool call
8:21:02complete. We're just passing in result.
8:21:05Now diff is a file diff, right? If it's
8:21:08just a file diff, we are returning an
8:21:11object. And when we try to return an
8:21:14object from here and then when we want
8:21:17to check for truncation, if the output
8:21:18is too long, we'll just truncate, right?
8:21:20When it tries to check for that
8:21:22truncation,
8:21:23it gets the file def. And on file def,
8:21:26you're trying to call tokenizer. Let's
8:21:28just go to path.py py or actually text
8:21:31py to understand this better. The error
8:21:33was in this count tokens function. Now
8:21:36we're calling tokenizer of text. The
8:21:39text is a string and we have passed in a
8:21:41file diff over here. And that is a
8:21:43problem because well first of all we
8:21:46shouldn't be sending a file diff from
8:21:47here. We should be sending in a proper
8:21:49diff string. And that's why we had
8:21:51created a function of to diff. If you
8:21:54remember something like that I don't
8:21:57quite remember what that function was
8:21:58called. So let me go to base. py and
8:22:00here we have create diff. Maybe we can
8:22:03convert this to two diff. That makes
8:22:05more sense. And now you know we have the
8:22:07two diff function which will return the
8:22:09string to us. And on that string we can
8:22:11call truncation and all sorts of things.
8:22:13So that's good. And we only want to call
8:22:16two diff if the result. exists because
8:22:21diff can also be null value. Right now
8:22:24if that exists we'll call two diff.
8:22:25Otherwise we just have a null value
8:22:28because in read file the diff will be
8:22:30null. And now we can try to run it
8:22:32again. We'll have create hello world py
8:22:35script for me and then hit enter. And as
8:22:39you can see the diff view appears. We
8:22:41have write file where we have a gray
8:22:43icon and then we have updated because
8:22:46that file already existed. We have
8:22:48created hello.py2 many times that's why
8:22:50it's updated. A new file was not
8:22:52created. This was the old file. This is
8:22:54the new file. Both are basically the
8:22:57same. If you want, you can handle this
8:22:59case as well. If both of the strings are
8:23:01same, maybe you don't want to add minus
8:23:02and plus. And then you know this line
8:23:05remains the same. The bang operator I
8:23:08think that's what it's called something
8:23:09like that. Bang line shell line. I don't
8:23:11remember it is the same in both the
8:23:13versions. Then you have def main
8:23:15function created in the new one. And
8:23:18then you call the main function remove
8:23:19the print hello world from here. So
8:23:21that's awesome. And if you count the
8:23:24total number of lines in the new file,
8:23:26it is seven. We can just go to this
8:23:28hello world. Hello. py as well. You
8:23:31might also see hello world that is
8:23:32because you know we tried previous
8:23:35prompts. And this was the file created.
8:23:38We can also delete this hello. py. And
8:23:40that's about it. Now what I'd like to do
8:23:42is restart this entire thing. I would
8:23:44like to change the model actually just
8:23:47so that we know it's working on every
8:23:49model. So I'll just go back to our
8:23:51config.tml tml file and I'll remove this
8:23:53model name from here. Let's run it
8:23:55again. And we have our mistral model
8:23:58back. Now I can tell it to create hello
8:24:01world python script for me please. And
8:24:05as you can see it writes back to us. So
8:24:08it just creates a new file. Hello world.
8:24:10py is the new file. So as you can see it
8:24:12created it not updated. It's a new file.
8:24:15And then it gives out the assistance
8:24:17response as well. Now what I'd like to
8:24:20check is that it also works with read
8:24:23file. So I'll tell it that hey I made
8:24:26some changes to this file. Can you
8:24:30reread and tell what differences I made.
8:24:35So we just kind of gaslighting the model
8:24:38and as you can see it reads the file
8:24:40properly. It gives in the correct path
8:24:42as well. Then you know we get this
8:24:44output of what it tried to read. And
8:24:46then it says the file currently contains
8:24:48just this line. There are no changes
8:24:50detected in the file. If you made
8:24:52modifications, please ensure they were
8:24:54saved. That's good. Our agent is now
8:24:56slowly working as expected. Now the next
8:24:59tool that I would like to add is edit
8:25:02file. So I'll go to built-in tools again
8:25:05and create a new file called edit file.
8:25:07py. And let's just go ahead and create
8:25:09the class edit tool layer which is going
8:25:12to extend the base tool class. And we're
8:25:15going to get that from tools.base.
8:25:17Things don't really change much here. We
8:25:19still have to define the name of the
8:25:21tool which is edit. Then we have to find
8:25:23the description and then pass in the
8:25:26description. Well, for the description
8:25:28again I'm going to copy paste. So I'll
8:25:30just paste it down here. The description
8:25:32is edit a file by replacing text. The
8:25:35old string must match exactly including
8:25:38white space and indentation and must be
8:25:41unique in this file unless replace all
8:25:43is true. Use this for precise surgical
8:25:46edits. For creating new files or
8:25:49completely writes, use write file
8:25:51instead. So similar to write file, we
8:25:54just mentioning when to use this edit
8:25:56tool and when not to use this edit tool
8:25:57because LLM might get confused. After
8:26:00that, we're going to specify the kind of
8:26:02this tool which is tool kind. We'll
8:26:04import that from tools.base and it's
8:26:06write again we're trying to write right
8:26:09and then we are going to have a schema
8:26:11and for that we have to create a class
8:26:13of edits parameters. So let's have class
8:26:16edit parameters which is going to extend
8:26:19the base model. The base model is come
8:26:22going to come from pantic. So let's
8:26:24import it. After we have done that we're
8:26:27going to go ahead and have the path.
8:26:30That's the first string. The path is
8:26:33going to be where the path where we are
8:26:35going to write or edit the file with
8:26:38within. So we have dot dot dot
8:26:41description and the description is going
8:26:43to be path to the file to edit and this
8:26:47is going to be relative to
8:26:50working directory or absolute
8:26:54path. Cool.
8:26:56Let me just change this to or. And yeah,
8:26:59that looks good. Now let's have the next
8:27:01one which is going to be an old string
8:27:04because remember edit tool means that
8:27:06let's say I want to edit this file. If I
8:27:09want to edit this file, I want to take a
8:27:11particular section. So let's say lines
8:27:1412 to lines 18. I want to take that
8:27:16section and edit it off. So here I'm not
8:27:20doing it on the basis of lines. I'm
8:27:22going to do it on the basis of string.
8:27:24I'm going to say that hey listen this
8:27:27name is equal to edit. Wherever I see
8:27:29this, I'm going to change this to name
8:27:31is equal to write for example. Or
8:27:34wherever we see kind is equal to
8:27:36toolkind dot.right, we can change it to
8:27:38kind is equal to toolkind dot read. This
8:27:41is what the edit parameters does. So we
8:27:44need a parameter of old string and then
8:27:47a new string. So the exact text that we
8:27:50need to find and replace and the new
8:27:52string is going to be the text that
8:27:54should replace the old string. So let's
8:27:57have old string which is going to be a
8:27:59string where we are going to pass in the
8:28:01field and by default the value is going
8:28:03to be an empty string and then we can
8:28:05pass in the description and for the
8:28:07description it's going to be the exact
8:28:10text to find and replace it must match
8:28:16exactly including all whites space and
8:28:20indentation
8:28:22for new files this will be left empty.
8:28:26All right. So whenever we trying to
8:28:28create new files, we're not really
8:28:30trying to edit it. The LLM might have
8:28:33gotten confused and because of that it's
8:28:35still using edit when it should be using
8:28:38write tool. So we should be fixing the
8:28:40prompt instantly but just so that the
8:28:42user doesn't face any kind of error.
8:28:44We're just saying that hey in case
8:28:47you're creating a new file just leave
8:28:50this empty because there's no old
8:28:52string. And then we're going to add a
8:28:54new string as well. This new string is
8:28:57going to be a field as well. And then
8:29:00everything is going to be the same
8:29:01except description where we're going to
8:29:03have the text to replace old string with
8:29:08can be empty. If we want to delete text,
8:29:12right? Because just think about it. If
8:29:16you're trying to remove this line, the
8:29:18old string will be name is equal to edit
8:29:20and the new string will be an empty
8:29:23string and then it will be gone. After
8:29:25that, we're going to add a final
8:29:27parameter for edit which is replace all
8:29:30and it's going to be a boolean value.
8:29:32We'll pass in the field. By default,
8:29:34it's going to be false. We don't want to
8:29:36replace all the occurrences of old
8:29:38string. But if it's set to true, it will
8:29:41replace all occurrences
8:29:44of old string. And by default, it is set
8:29:48to false. Now notice I've set default to
8:29:53lowerase false here because the LLM is
8:29:55going to send us the response and the
8:29:58LLM is going to respond with JSON,
8:30:00right? So if the JSON is a capital
8:30:03false like that, we'll not be able to
8:30:05pass the JSON. That's why we're setting
8:30:08this to lowerase f. Awesome. So, we have
8:30:12all of our parameters. Now, we can just
8:30:13take this edit parameters and pass it as
8:30:16the schema. Awesome. And now we can go
8:30:19ahead and create a function of async
8:30:21defex execute where we're going to get a
8:30:25self invocation. The invocation is going
8:30:28to be of the type of tool invocation.
8:30:30Let me just import that nicely. So, from
8:30:34tools.base, we'll import tool
8:30:35invocation. And what are we going to
8:30:37return? A tool result. Similar to all of
8:30:41the execute functions we have talked
8:30:43about so far. And now let's think about
8:30:46it. The first thing like the right tool,
8:30:49we're going to take the parameters that
8:30:51we get from invocation and convert it
8:30:53into edit parameters pantic model. So
8:30:56we'll have parameters is equal to edit
8:30:59parameters. Then we'll deconstruct
8:31:02invocation dotparameters.
8:31:06And now we have the parameters with us.
8:31:08The other thing we will need is the
8:31:10actual path because even in write file
8:31:13we wanted to resolve the path first so
8:31:15that we have an actual path to write to.
8:31:18So I'll just import resolve path from
8:31:20utils.path. I'm not even going to
8:31:23explain this further because I'll just
8:31:25be repeating myself then. And now we're
8:31:27going to have params.path path passed
8:31:29in. Awesome. Now the next thing I want
8:31:32to see is if the file already exists
8:31:35because well if this path does not
8:31:39really exist in that case I have to
8:31:42create one. Let's say the llm gives us
8:31:44an invalid path. But then the old string
8:31:48does tell us that hey whatever line is
8:31:50present like parameters is equal to edit
8:31:53parameters just [snorts] replace that
8:31:55line. All right, because well I want to
8:31:58replace that line. In that case, it just
8:32:01means that the LLM hallucinated because
8:32:03if the path is invalid, how are you
8:32:05mentioning old string the old string
8:32:08should be empty if the part does not
8:32:10exist? Because then we'll be creating a
8:32:12new file. That's what we have typed over
8:32:14here, right? For new files, leave this
8:32:16empty. So if the path does not exist and
8:32:19the old string is not empty, we've run
8:32:22into some sort of error. So we'll tell
8:32:24the llm. But if the path is valid and
8:32:28the old string is empty in that case but
8:32:31just in case path is invalid and the old
8:32:34string is empty in that case we want to
8:32:37go ahead and create that particular
8:32:39file. So that's the check we're going to
8:32:41add here. If the path does not exist in
8:32:44that case we're going to have if params
8:32:47do old string. Let's just check if that
8:32:49exists because if the part does not
8:32:52exist and the old string does exist,
8:32:54we've run into an error. So we'll just
8:32:56do return tool result dot error result
8:33:00and then we can pass in the error
8:33:02message. The error message is very
8:33:04simple. The file does not exist. So this
8:33:07is the path that you're talking about.
8:33:09So to create a new file, you should use
8:33:12an empty old string instead. Okay. And
8:33:17with this error message, the LLM should
8:33:19know that hey, the file did not exist.
8:33:21Maybe I passed in the invalid edit path
8:33:25or I need to pass in the empty old
8:33:28string which I did not do because the
8:33:31llm can mean either of those things. We
8:33:33don't know for sure. Now let's go ahead
8:33:36and ensure that the parent directory
8:33:38exists. If the parent directory does not
8:33:41exist, in that case we want to create
8:33:44that path and stuff which this utility
8:33:46function we created will take care of
8:33:49and we'll pass in the path. Just to
8:33:50remind you, this was the thing. It just
8:33:52creates a path. It creates a directory
8:33:54if needed. So that's good.
8:33:58After that, in this particular path,
8:34:00we're going to go ahead and write text.
8:34:02What is the text that we want to write?
8:34:04Well, at this point, we know that the
8:34:07path does not exist. So the path now
8:34:09exists and we have to write the text to
8:34:12this particular path. Now since this
8:34:14part did not exist before, now it
8:34:16exists. That's why we're going to pass
8:34:19in the parameters new string. We don't
8:34:22have to do anything complex other than
8:34:24that. And we'll also set up the encoding
8:34:26which is UTF8. That's what we did
8:34:28everywhere else. And now I want to
8:34:32return a tool result success. But in the
8:34:34success I'm going to pass in some
8:34:36metadata. The metadata is related to
8:34:38well what is the path? What is if it's a
8:34:41new file or not? What are the number of
8:34:43lines? So let me just create a variable
8:34:45to count the number of lines. So we have
8:34:48length of params dot new string dotsplit
8:34:52lines because once we call split lines
8:34:54we'll get the number of lines there are.
8:34:57And now we can just return the tool
8:35:00result dot success result. Then we'll
8:35:04pass in the output. The output is
8:35:07created path. So we pass in the path and
8:35:10then we'll also specify the line count
8:35:12and then the number of lines. Then we'll
8:35:15pass in the diff which is equal to file
8:35:18diff. Now let's import that from
8:35:20tools.base. And now we can pass in the
8:35:23path which is just the path again. Then
8:35:27the old content, the old content was
8:35:30empty string, right? Because the
8:35:32parameters do old string is empty. The
8:35:35file did not exist. And the new content
8:35:38is going to be this particular line
8:35:40parents do new string. So let's pass
8:35:42that in.
8:35:44And then we're going to have is new file
8:35:47true because it is a new file. We just
8:35:50created it. And then we'll pass in the
8:35:53metadata which is responsible for much
8:35:55of the processing in the main. py and
8:35:58tui.py files. So the first thing is
8:36:02path. We'll just pass in the string
8:36:04path. Then we have is new file again
8:36:08where we're going to say it is true. And
8:36:11then we have the number of lines which
8:36:14is the line count. This might seem like
8:36:18redundant data that we're sending across
8:36:20but it's actually helping us in our code
8:36:22especially the tui. py file because if
8:36:25you notice in write file and this is the
8:36:27same code that we'll use for edit file
8:36:30as well because we want the same thing
8:36:33right if it's writing a new file or it's
8:36:35editing a new file to the user it just
8:36:38feels like the same thing. So we just
8:36:40have output displaying and then we have
8:36:43a diff view. So we just have the diff
8:36:46text displaying that's why we sent
8:36:47across a diff and the metadata is
8:36:50helpful when you have something like
8:36:52this. If is instance meta data of
8:36:55dictionary and if the path exists which
8:36:58is of the type of string then we set the
8:37:00primary path. That's where this helps us
8:37:03as well because this primary path will
8:37:06be used everywhere including in write
8:37:09file if required. So yeah, cool. That is
8:37:12the success if the file was created
8:37:15newly. But what if the file already
8:37:17exists? If the file already exists, then
8:37:21the first thing I need to do is read the
8:37:23existing content. Because once I read
8:37:26the existing content, I can go in there,
8:37:28find all of the occurrences, and replace
8:37:30them with the new string. So I'll have
8:37:33old content is equal to path dot read
8:37:36text and I'll pass in the encoding which
8:37:39is UTF8.
8:37:41Let me just spell this correctly. After
8:37:43that we're going to check if not
8:37:45parameters dot old string. In that case
8:37:48we'll return a tool result dot error
8:37:51result and then I'll pass in the error
8:37:54message again because this time we have
8:37:57the content but you said that the old
8:37:59string does not exist. That means you
8:38:02want to create a new file. If you want
8:38:04to create a new file, well, the file
8:38:06already exists. So, you've run into some
8:38:08sort of logical error. So, you'll just
8:38:10say old string is empty, but the file
8:38:14exists. Correct? Now, we can just steer
8:38:17the model to go in the right direction
8:38:19by saying provide old string to edit or
8:38:22use write file instead. Or maybe instead
8:38:27of using instead we'll write or use
8:38:30write file to override
8:38:33because if we want to overwrite we
8:38:35should be using write not edit. This is
8:38:38just about creating boundaries about
8:38:40what tool should do what and if the LLM
8:38:42failed to understand it we'll just tell
8:38:44it to call another tool because again it
8:38:47can be the case that the model probably
8:38:49wanted to try something else but it just
8:38:52failed. But now let's say the parameters
8:38:55do old string also exists. In that case
8:38:58I want to count the number of
8:38:59occurrences in the old content right
8:39:02because if the occurrences are zero in
8:39:05that case well we don't have any match
8:39:08what are you trying to edit the llm
8:39:10again made some sort of error there was
8:39:13some hallucination so we'll just have
8:39:15occurrence count is equal to old content
8:39:19let me have old content quickly dot
8:39:23count and then we'll pass in the
8:39:25parameters dot old string. Awesome. This
8:39:29will give us the number of occurrences
8:39:31of old string in old content. And now we
8:39:35can just go ahead and do if occurrence
8:39:37count is equal to zero. In that case,
8:39:41we've reached a place where the LLM just
8:39:45doesn't know what string exists in the
8:39:47file. Maybe the LLM is thinking of some
8:39:51file data that it wrote. For example,
8:39:54let's say I told it to create a hello
8:39:57world. py script. So, it created that.
8:39:59After that, I went ahead and manually
8:40:01edit the edited the file. After that, I
8:40:04tell the LLM that, hey, there's some
8:40:06boilerplate code I want you to write.
8:40:08Please write it. So, it tries to
8:40:09override the already existing file data
8:40:12that it has previously written. It
8:40:15doesn't know about my new changes
8:40:16because it did not read the file. That
8:40:19is an issue that can happen quite often
8:40:23actually. I think cursor also faces the
8:40:26same issue many times and that's why we
8:40:29have to steer the model carefully here.
8:40:31So what I'm going to do is create a
8:40:32helper function because we're going to
8:40:34steer the model very nicely. So we have
8:40:37parameters do old string passed here.
8:40:40We'll also pass in the old content and
8:40:42the path as well. Now let's go ahead and
8:40:44create this function. Since this
8:40:46function is about steering the model,
8:40:48I'm not going to write it from scratch,
8:40:50but I'll explain what's going on after
8:40:53pasting this function. So here we go.
8:40:55I'll just paste it here. The first thing
8:40:57we have to do is import path from
8:40:59pathlib. We'll do that. And now I can
8:41:02explain what's going on. Well, we took
8:41:04old string. We took content, which is
8:41:07the old content, and then the path. This
8:41:09is going to return a tool result because
8:41:12well you know we have to essentially
8:41:14return an error message. So it's going
8:41:16to return tool result dot error result
8:41:18at the end of the day. But
8:41:21here's what's going on. We have the old
8:41:24content. We split it so that we get
8:41:26lines. Then what are we trying to do?
8:41:29Well, I'm trying to split the old string
8:41:31and get its five words because those are
8:41:34the search terms because let's say the
8:41:36LLM tried some old string, right? And it
8:41:40failed. But maybe
8:41:42out of that old string, five words maybe
8:41:44made sense. The first five was somewhere
8:41:48and maybe it just you know hallucinated
8:41:51after that. So we'll we can try with
8:41:53those five terms firsthand. If you want
8:41:56you can go ahead with 10 15 whatever you
8:41:58want but I'll just go with five and then
8:42:01we'll search for them in this lines. So
8:42:04what I do is if search terms list exists
8:42:07in that case we have the first term
8:42:10which is search terms at zero and then
8:42:12we go over every lines starting from one
8:42:15because we have already taken into
8:42:17consideration search terms at zero and
8:42:19search terms at zero will exist because
8:42:21there is at least one element in search
8:42:23terms because we have put this condition
8:42:26and then we'll go over everything from
8:42:28one to the end of the lines list and
8:42:30then we'll check if first term is in
8:42:33line Then we'll just append part the
8:42:36line up to 80 characters because we
8:42:39don't want that entire line to be placed
8:42:42within the partial matches because think
8:42:44about it that line can be infinitely
8:42:47long as far as we care. So we just trim
8:42:50it down to 80 characters. That's enough
8:42:52for the model to understand what string
8:42:54they're dealing with. And then we can
8:42:56just append it to the partial matches.
8:42:59And remember I'm passing in the i as
8:43:03well which is the line number associated
8:43:06with it. And now we'll check that hey if
8:43:08the length of the partial matches is
8:43:10greater than equal to three then we'll
8:43:12break out because we have at least three
8:43:15partial matches that is enough
8:43:19and then we create an error message.
8:43:21This is all related to formatting of the
8:43:23error message. So the first line is old
8:43:26string not found in path. And now if the
8:43:29partial message matches exist then we
8:43:32can tell the LLM that hey these are the
8:43:34possible similar lines that you were
8:43:36going for. And then we have the line
8:43:38number with the line preview. And we'll
8:43:41do that for all the partial matches that
8:43:43we get. Maximum is three because you're
8:43:46breaking out at three. And then at the
8:43:48end we just say make sure all string
8:43:50matches exactly including whites space.
8:43:52Maybe you can also put and indentation.
8:43:56So that's good. And then we have an else
8:43:59condition because we do not find any
8:44:02partial matches. So we just saying that
8:44:04hey make sure the text matches exactly
8:44:06including all whites space and
8:44:08indentation line breaks any invisible
8:44:10characters and then at the end maybe we
8:44:12can add another line which just says
8:44:15that hey listen maybe try to read the
8:44:18file again. So we'll just add try
8:44:21rereading the file using read file tool
8:44:25and then editing. So that's steering the
8:44:28model in the right direction because I
8:44:32do think reading the file would probably
8:44:34fix it. Otherwise the LLM is just pretty
8:44:38dumb. But yeah, this seems like a good
8:44:40thing. Now we can come back to this
8:44:42part. So if the occurrence count is not
8:44:44found, we are returning a no match error
8:44:47and we are returning the tool result dot
8:44:50error result here. So that's good. Let's
8:44:52say the account occurrence count is one
8:44:56or greater than one. In that case, we
8:44:58want to replace everything. But there's
8:45:01one edge case that we have to consider.
8:45:04What if there are many occurrences in
8:45:05this edit tool? So maybe you want to
8:45:09edit out this line, but this line exists
8:45:12maybe 100 times in this entire file.
8:45:14It's a pretty large file. And you've set
8:45:17the replace all to false. That just
8:45:20means you want to edit either this line
8:45:22out or the second occurrence of this or
8:45:24the third occurrences of this or the
8:45:2695th occurrence of this. We don't know
8:45:29which occurrence you care about. So in
8:45:31that case, the LLM will have to be more
8:45:34specific about what it wants to do.
8:45:36Maybe it takes the above line and then
8:45:39edits it or it just sets replace all to
8:45:42true. So broadly the LLM will either
8:45:45have to provide more context to make the
8:45:47match unique if the occurren occurrence
8:45:49count is greater than one or it will
8:45:52have to set replace all to true. So
8:45:54let's just catch that edge case. So if
8:45:56the occurrence count is greater than one
8:45:58and the params.replace replace all is
8:46:01set to false. In that case, we'll just
8:46:03return the tool result dot error result
8:46:06and then we'll pass in the error string
8:46:09which is old string found let's say
8:46:12occurrence count times and let's just
8:46:16mention the path because maybe the LLM
8:46:19you know is just going after the wrong
8:46:22path that's just good context to give it
8:46:25won't take much of a context space. So
8:46:28here we have two options. Either the LLM
8:46:30can one provide more context to make the
8:46:34match unique or two it can set replace
8:46:39all is equal to true to replace all the
8:46:42occurrences. Right? Those are the two
8:46:45possible options. If you can think of a
8:46:47third one go ahead. After that I'll just
8:46:49attach some metadata of let's just say
8:46:52occurrence count. So occurrence count is
8:46:56occurrence count. I don't think this
8:46:58will be really useful for us but yeah
8:47:01just in case it's useful go ahead let's
8:47:03just attach it. Now if the occurrence
8:47:06count and all of these edge cases are
8:47:08fine now we want to perform the
8:47:11replacement. So the first thing we'll
8:47:12have to check is do we want to replace
8:47:15everything or do we want to replace just
8:47:18one part of the string. So if
8:47:20parameters.replace
8:47:21all is true then we want to perform
8:47:24replace all and we can do that using old
8:47:26content.replace. This will just replace
8:47:29everything.
8:47:30Return a copy with all occurrences
8:47:33replace by new. So we can pass in the
8:47:35parameters dot old string and the thing
8:47:39that we want to write is parameters dot
8:47:42new string. Good. And then maybe we can
8:47:45maintain a replaced count which just
8:47:48checks like how many occurrences did we
8:47:51replace. So replace count here can be
8:47:55occurrence count otherwise you know
8:47:58replace all is set to false. We only
8:48:00want to replace once. So it's
8:48:02essentially the same thing as this one
8:48:04but with one adjustment.
8:48:07By default this replaces all but you can
8:48:10also set your own count. So if the count
8:48:12is set to one after this, it will just
8:48:15replace one part and then we can set
8:48:18replace count to one over here as well.
8:48:21And that's all I want to verify that the
8:48:24change occurred because again it is
8:48:28quite possible and this is solely based
8:48:30on my experience because I've been using
8:48:32dumb models because they don't cost a
8:48:34lot of money. But they did hallucinate
8:48:37and the old content and the new content
8:48:40was essentially the same. So no change
8:48:43occurred. If no change occurred, it
8:48:46shouldn't really be a success because
8:48:48something just went wrong. So we'll just
8:48:51say that hey if new content and we'll
8:48:53have to create a variable for new
8:48:55content and new content will actually be
8:48:57set up over here because whatever old
8:48:59content gives us that's going to be the
8:49:01new content. So if the new content is
8:49:03equal to the old content that means
8:49:07something went wrong. So let's just
8:49:10return tool
8:49:12result dot error result and I can say no
8:49:16change made old string equals new
8:49:20string. Let me put a space here. And
8:49:23that's good. Now we can try to write the
8:49:25file. And as we've done multiple times
8:49:28before, we're just going to have a try
8:49:29and accept block here. So it's going to
8:49:31be path dot write text. We've done this
8:49:34multiple times as well. We'll pass in
8:49:36the new content and the encoding is
8:49:39equal to UTF8. If you want to get hyper
8:49:41optimized, maybe you can try replacing a
8:49:44text. But in my case, just new content
8:49:47is fine. New content is actually
8:49:50beneficial because when we try to send a
8:49:52diff of the view, having the new content
8:49:56and old content will allow us to show
8:49:58all of the changes. So here we have
8:50:00accept io error as e and then we'll just
8:50:04return tool result dot error result and
8:50:07then we'll pass in failed to write file
8:50:12and we'll pass in the error message.
8:50:15Awesome. So now we have written to the
8:50:18file. Now we just have to return the
8:50:19success result dot success result. And
8:50:23in this we have to specify the output.
8:50:26Well, the output is that we edited this
8:50:29particular pass that we have and we
8:50:33replaced how many occurrences. So, we'll
8:50:35pass in the replace count and we replace
8:50:38these many occurrences. Now, it can be
8:50:41an occurrence or an occurrences. So,
8:50:44I'll just take a shortcut here and pass
8:50:46in a bracket s because it can be
8:50:49occurrences or occurrence. But if you
8:50:52want you can again optimize this. If the
8:50:55replace count is equal to one then it's
8:50:57occurrence otherwise it's more than that
8:51:00it's occurrences. And then we can attach
8:51:02a diffing message. The diffing message
8:51:05can be something like how many lines did
8:51:07we add and how many lines did we remove
8:51:10in total. So let's just create that. So
8:51:14we can have a diff message is equal to
8:51:16an empty string. And now we essentially
8:51:17want to know how many lines were added
8:51:21or removed. And to calculate that first
8:51:24we'll have to calculate the number of
8:51:25old lines which is length of old content
8:51:28dotsplit lines. Then we'll do a similar
8:51:32thing for new lines. So we have new
8:51:34lines equal to length of new content
8:51:37dotsplit lines. And now we have if the
8:51:43line diff. Let's just create a new
8:51:45variable line diff which is just equal
8:51:48to new lines minus old lines. That will
8:51:50give us the number of lines that were
8:51:53changed. And now we can say if the line
8:51:55diff is greater than zero in that case
8:51:58we'll have a diff message which is equal
8:52:00to an if string where we add plus and
8:52:04then the line diff lines. So the let's
8:52:08say we added 10 more lines and if the
8:52:13line diff was less than zero in that
8:52:16case the diff message is going to be you
8:52:19know we kind of subtracted these many
8:52:21lines so line diff lines were kind of
8:52:25subtracted and we're not adding a
8:52:27negative sign here because line diff is
8:52:29already negative so it will
8:52:30automatically add that negative sign
8:52:33and yeah that seems like a good diff
8:52:35message I can just attach it over here.
8:52:39Seems good. And now I'll just add the
8:52:41div which is going to be a file div. The
8:52:44path is already with us. You might
8:52:47already know the drill here. The old
8:52:49content is going to be old content. New
8:52:52content is going to be new content.
8:52:57And that's all. We're not creating a new
8:52:58file. We don't have to set that option
8:53:00on. And then finally we can have
8:53:02metadata. Again we have to set the path
8:53:05here. The path is going to be string of
8:53:09path. That's really useful. Then we can
8:53:11have the replaced count. The thing that
8:53:13we counted so far. How many replaces of
8:53:17occurrences did we do? And that is just
8:53:21the replace count. Finally, we can have
8:53:24the line diff. You know what was the
8:53:27number of line changes that we did and
8:53:29we can pass that in. I don't think this
8:53:31these both are especially useful in the
8:53:34metadata. But if you want to display it
8:53:36out on the screen, go for it. Now, we
8:53:38have an error over here because I forgot
8:53:40to put a comma. So, we put that in. And
8:53:43I believe now we're done with the edit
8:53:46file. So, let me go ahead and first
8:53:48register this tool in init py. So, I'll
8:53:51have edit tool. Let me import that from
8:53:54tools.builtin.edit
8:53:56file. And now I want to edit my tui. py
8:53:59file because here I told you that we
8:54:02have to handle the tool called complete
8:54:04right here we just handling read file
8:54:07and write file but I also told you that
8:54:09edit file is going to be quite similar
8:54:11to write file because all we want to do
8:54:13in edit is display the things in a div
8:54:18format. So we can just utilize this. We
8:54:20can check that hey if the name is in
8:54:23write file or it is in edit file. So we
8:54:27are having a set where we check for the
8:54:29name. Then we'll go over this. We'll
8:54:32append the block. We'll try to truncate
8:54:34the text and then we display the diffing
8:54:37view. I believe there's one more thing
8:54:39we did earlier which is something
8:54:43related to old string that we already
8:54:45mentioned here. If the key is in content
8:54:48or old string or new string because you
8:54:51know when we're doing a tool called
8:54:52start we render the args table which is
8:54:56a key value pair and there it just puts
8:55:00in the entire blob of old string and new
8:55:04string. We don't want that. So that's
8:55:07why we had the key over here. We don't
8:55:10display the entire blob on the screen
8:55:13for edit as well. And now we have to go
8:55:16in the ordered args because here we have
8:55:18a variable of preferred order. In what
8:55:21order do we want to display the
8:55:22arguments for edit file we're going to
8:55:25have or I don't know what the tool was
8:55:27called. It was either called edit file
8:55:29or edit. That's probably why you should
8:55:33convert this into an enum so that it can
8:55:35be used everywhere. But I'll just go
8:55:39back to edit file and look at the tool
8:55:42name. It is just called edit. So let me
8:55:45just put edit here. And even down there
8:55:48wherever I called
8:55:50tool call complete.
8:55:53I'll change this to edit not edit file.
8:55:57Now we'll come back to the preferred
8:55:59order function. And for edit our
8:56:01preferred order is first you have to
8:56:03specify the path you know where are we
8:56:06editing the file. Then you have to
8:56:08mention replace all if it's set to true
8:56:11or false. And then you have old string
8:56:14and then you have new string because it
8:56:17just looks better in that order. And I
8:56:19believe we are done. Now let's try to
8:56:22rerun our agent and tell it to edit some
8:56:25file. What file I would like to edit?
8:56:27Hello world. py. So I'll tell it that
8:56:29hey please edit hello world. py file for
8:56:35me. Change text to maybe we can tell it
8:56:39to add a new feature. So it will add a
8:56:43new feature of taking user input and
8:56:46printing out user input with hello world
8:56:50attached. Terrible prompting but let's
8:56:53see if it works. So as you can see our
8:56:56LLM is smart enough. It just goes and
8:56:58first reads the file because it first
8:57:01needs to get context about hey what is
8:57:03the file that I'm dealing with? I'm
8:57:05dealing with hello world. py. Okay cool.
8:57:07Let me check it out. So it goes and
8:57:10checks it out and then it edits it. And
8:57:13for the edit it just does well this
8:57:16part. It prints hello world removes it.
8:57:19Then you have user input is equal to
8:57:21input enter your name. And then it
8:57:23prints hello world after the followed by
8:57:27the name of the user and then it just
8:57:29says task completed. That's good. we are
8:57:32able to use multiple tool calls together
8:57:34and it completes the entire tool calling
8:57:37stuff when required. So the agentic loop
8:57:40is getting completed. Now I just want to
8:57:43ensure that the turns are still working
8:57:46like a new message if I put it in it
8:57:48understands the context. That means
8:57:50everything is working fine.
8:57:53So I'll just tell it format the output
8:57:57nicely please. Now let's see what it
8:58:00does. So, it goes ahead and edits again.
8:58:03This time, it just removes the print
8:58:06line and instead does hello user input,
8:58:09welcome to the world. Well, that's
8:58:12creative, but anyways, I'm done having
8:58:14fun with this. Now, the next tool you
8:58:16can work on is apply patch. Apply patch
8:58:19is essentially a tool that can do
8:58:20multiple edits in just one tool. Let's
8:58:23take an example. Let's say I have a file
8:58:25like loader. py. in this file which is
8:58:28106 lines long. I want to edit this part
8:58:31out. Maybe I want to change something on
8:58:33line 11 and then maybe I want to change
8:58:35something on line 25 followed by a
8:58:38change on line 37. So with the edit tool
8:58:42what will happen is we'll have to go
8:58:43ahead and edit out each and every line
8:58:46one by one. You know one tool call for
8:58:49this one, one tool call for this one,
8:58:50one tool call for this one and so on.
8:58:53Now instead of doing those three tool
8:58:55calls, what you can do is just define a
8:58:57tool called apply patch that will make
8:59:00all of those edits possible in just one
8:59:03tool call. But I'm not focusing on apply
8:59:05patch right now. It is present in codeex
8:59:08CLI which is by OpenAI. But we're not
8:59:10going to implement that otherwise this
8:59:12tutorial will get too large you know it
8:59:14will be too long. So now what I'm going
8:59:16to do is work on the next tool which is
8:59:18shell. py. Now in the shell. py tool.
8:59:22We're first going to create a class
8:59:23called shell tool and then we're going
8:59:26to have an extension of tool just like
8:59:28we had with other classes as well. After
8:59:31that we going to name the tool which is
8:59:34just shell. After that we're going to
8:59:36have the kind which is toolkind
8:59:38dotshell. Let me import toolkind from
8:59:41tools.base and shell. So this is the
8:59:44first tool and the only tool that we
8:59:46will use shell as the toolkind. After
8:59:49that, we're going to have the
8:59:50description. And the description is a
8:59:52simple oneliner. So, I can just write it
8:59:54down. Execute a shell command. Use this
8:59:58for running system commands, scripts,
9:00:02and CLI tools. That's it. You know, as
9:00:05simple as it gets. After that, we have
9:00:08to define a schema. So, we'll go at the
9:00:10top and define class shell parameters,
9:00:14which is going to extend the base model
9:00:16which comes from Pyantic. Let me import
9:00:19that. And here we go.
9:00:23After that, we can go ahead and define
9:00:24the parameters of shell. What do we need
9:00:27here? Well, if a shell is being called,
9:00:29first we need to know what command needs
9:00:31to be executed, right? So, we'll have a
9:00:33command of the type of string and we'll
9:00:35import field from pyantic. Again,
9:00:38everything is going to be the same
9:00:40except description where we say the
9:00:42shell command to execute. Cool.
9:00:46After that we'll have the timeout which
9:00:49is going to be an integer. So there can
9:00:51be multiple instances where a shell
9:00:53command is run but maybe there's some
9:00:56sort of infinite loop going on or
9:00:58something like that and we are unable to
9:01:02just look at the output even if it's an
9:01:06error output and for that reason we're
9:01:08going to specify a timeout. So if it's
9:01:11120 seconds long, 180 seconds long, 200
9:01:13seconds long, after that we just time
9:01:16out and close the shell and then we'll
9:01:18give the relevant information out to the
9:01:20LLM so that it can make appropriate
9:01:24decisions. The default value here is
9:01:26going to be 120 seconds for the timeout.
9:01:28But the LLM can change it. The value
9:01:30should be greater than equal to 1 and it
9:01:33should be less than equal to let's say
9:01:35600 seconds, which is 10 minutes. After
9:01:37that we can specify the description and
9:01:40the description is timeout in seconds
9:01:44and then we can set the default to 120.
9:01:48Great. After that we're going to add the
9:01:50next one which is current working
9:01:52directory which can be a string or a
9:01:54null value and by default it's going to
9:01:56be a null value. We can pass in the
9:01:59description which is working directory
9:02:01for the command. So maybe you know we
9:02:05are in this particular AI agent current
9:02:08working directory. That's our main
9:02:09working directory. But the shell command
9:02:11needs to be executed within a specific
9:02:14subfolder. That's where this current
9:02:16working directory can help us out. Now,
9:02:19of course, the LLM can be smart enough
9:02:21to just say cd into the subfolder and
9:02:25and and then it can execute whatever it
9:02:27wants to, but this just feels like a
9:02:30better way to go about it, forcing the
9:02:32LLM to specifically list out the current
9:02:34working directory. Obviously, by
9:02:37default, it is null. We'll try to
9:02:39execute in the folder we are in. And
9:02:42now, we can go ahead and specify the
9:02:43schema, which is shell parameters. Then
9:02:46I'll go ahead and have async diff
9:02:48execute function. We're going to get
9:02:51invocation which is going to be tool
9:02:52invocation. I think from next tool
9:02:55onwards I'll just import this shell. py
9:02:59class or something like that so that you
9:03:00know we don't have to write this
9:03:02boilerplate code again and again. From
9:03:04tools.base we have imported tool result.
9:03:07And now let's see how we want to execute
9:03:09it. The first thing we always do is get
9:03:11the parameters in the form of shell
9:03:14parameters. So we'll just pass in the
9:03:16invocation dot parameters. That's good.
9:03:20After that, we want to block any unsafe
9:03:24or dangerous command, right? Because the
9:03:27LLM might hallucinate and it gives you a
9:03:29dangerous command to execute. And you
9:03:31might have seen this in the news as well
9:03:33where the LLM, you know, just completely
9:03:36removes the root folder or the home
9:03:39directory or even, you know, wipes off
9:03:42the entire database. So you can have
9:03:44those commands in there and we'll check
9:03:47for any blocked commands. If those exist
9:03:49then we are going to block them. To
9:03:52block it first we need to maintain a
9:03:54dictionary of what commands are we going
9:03:56to block. And for that I'm going to
9:03:58create a constant right at the top. And
9:04:00these are the commands. We have rm- rf
9:04:03home folder then the root folder then
9:04:05everything in the home folder and so on.
9:04:08Now you can add your own block commands
9:04:11by the no means is this an exhaustive
9:04:13list. So go ahead add your own thing.
9:04:16But what I'm going to do is just use
9:04:18this block commands. This is just for
9:04:20reference. You know obviously you can
9:04:22add more stuff into this. So now let's
9:04:25try to check if the command should be
9:04:27blocked or not. So the first thing I'll
9:04:29do is go over every command in blocked
9:04:31commands list or the set that we have
9:04:35and check that hey the command that the
9:04:37user mentioned or the llm mentioned
9:04:39which can be stored in the variable
9:04:42let's say command params dot command do.
9:04:46because we want everything to be in
9:04:48lower case, right? And we'll probably
9:04:50also strip off anything that's not
9:04:52required and then we can check that hey
9:04:55if this blocked is in command that's
9:04:58present over here because you know we
9:05:01going over a set of block commands. So
9:05:03let's say we have this and if the
9:05:05command contains rm-rf
9:05:08with this forward slash then we'll just
9:05:11block it and in that case we just want
9:05:13to return a tool result with error
9:05:16result right so we'll have error result
9:05:18passed in and the first thing we're
9:05:20going to pass in is command block for
9:05:24safety and then I'll pass in the
9:05:26parameters dot command after that we
9:05:29also have a meta data which is equal to
9:05:32blocked and it is set to true. By no
9:05:35means is this helpful really this
9:05:38metadata will not help us even while we
9:05:40are trying to display anything on the
9:05:42main screen. If you want you can go
9:05:44ahead and use this. Initially I thought
9:05:46that it would help us but yeah I was
9:05:48just too lazy to implement anything
9:05:50related to good UX. Now let's say we
9:05:54have checked for block commands and
9:05:55blocked if any command doesn't suit us.
9:05:58The next thing I want to do is determine
9:06:00the working directory because in the
9:06:02working directory are we going to
9:06:05execute the command right? So I'll check
9:06:07if the parameters dot current working
9:06:09directory specified by the llm then I
9:06:12want to get the actual current working
9:06:14directory. So I'll just set current
9:06:15working directory to path and we'll
9:06:18import from pathlib and this will be
9:06:20parameters dot current working
9:06:22directory. And now I'll check that hey
9:06:24if this current working directory is not
9:06:26absolute path in that case we'll take
9:06:29the invocation dot current working
9:06:32directory which is the current working
9:06:33directory we are in the AI agent folder
9:06:36that we set in the main py as well and
9:06:39we'll append that with this current
9:06:41working directory that the llm mentioned
9:06:43to us. So we'll have current working
9:06:46directory is equal to invocation
9:06:48dotcurren working directory/curren
9:06:51working directory. This is how we are
9:06:52appending current working directory by
9:06:55the LLM to the current working directory
9:06:58we have set up. After that we're going
9:07:00to have an else condition. That means
9:07:02the params.curren working directory is
9:07:05not set up. In that case our current
9:07:07working directory is just invocation
9:07:09current working directory. We're going
9:07:11with whatever current working directory
9:07:13we set up in main.py file. And then I'll
9:07:16just check that hey if this current
9:07:18working directory does not exist. Let's
9:07:20say it was not absolute. So we went
9:07:23ahead and appended the path. But maybe
9:07:26the LLM hallucinated or something went
9:07:28wrong such that the current working
9:07:31directory does not exist. In that case,
9:07:33I'll just return the tool result dot
9:07:36error result and I'll pass in the
9:07:38working directory. So working directory
9:07:40doesn't
9:07:42exist is what I'll type is what I will
9:07:46type in. And then I'll pass in the
9:07:49current working directory so that the
9:07:51LLM knows what working directory we
9:07:53tried and what didn't work. Cool. After
9:07:56that, we want to execute the shell
9:07:59command, right? But the thing to know
9:08:01about shell commands is shell commands
9:08:03are pretty much like this terminal,
9:08:05right? I'm running it on ZSH, but maybe
9:08:08you can use bash as well to run your own
9:08:10thing. The thing is they're dependent on
9:08:14what platform you're on. So if you're on
9:08:16Windows, some other command will be
9:08:18executed and if you're on Mac or Linux,
9:08:20you can use bash. So that's one thing we
9:08:23have to determine. The other thing is
9:08:25all of the environment variables that
9:08:27are present in a shell. For example,
9:08:29when I was running my AI agent over
9:08:32here, I already exported some variables
9:08:35like the API key, the base URL, those
9:08:38are environment variables. And along
9:08:40with that there are multiple other
9:08:42environment variables like path or if
9:08:44you set up then you have to specify
9:08:46something related to cond. So you know
9:08:48there are multiple parts. The problem is
9:08:52that we have to copy these environment
9:08:55variables from your system into the
9:08:58shell that we are executing. Right?
9:09:00Because let's say we tried to run a
9:09:02Python command. But in our shell, the
9:09:04shell we create now it doesn't have
9:09:06access to the Python command. It doesn't
9:09:08have access to Python command because we
9:09:10don't have the environment variable set
9:09:14up properly for that. So it doesn't know
9:09:16that Python needs to be accessed using
9:09:18Python 3. This is a very rough example,
9:09:20but I hope you understand what I'm
9:09:22trying to mean here. We do need
9:09:25environment variables in our shell. But
9:09:28the problem is there are private keys
9:09:30also available as environment variables
9:09:33like the API key we passed in over here
9:09:37when we did export API key.
9:09:40So I want to remove or filter out those
9:09:42API keys if they exist and build the
9:09:45environment. Now obviously I don't have
9:09:47a magical wand which will just remove
9:09:50the environment variables because the
9:09:52environment variable can be named anyway
9:09:55but generally
9:09:57if I am creating a confidential secret
9:10:00environment variable that I don't want
9:10:02to give to an LLM you know that's why we
9:10:05are trying to filter out the environment
9:10:07variable so that these secret keys don't
9:10:10get leaked to an LLM because if they get
9:10:12leaked to an LLM and maybe they're used
9:10:14for pre-training another model they
9:10:16might be exposed outside of the model if
9:10:19they hallucinate or something like that
9:10:21if the security of the LLMs fail that's
9:10:24why we want to filter out these
9:10:25variables but we can only do that based
9:10:28on their name or based on the values so
9:10:30maybe you can check that you know if
9:10:33it's a Google API key then it's going to
9:10:36have a certain structure to it if it's
9:10:38Azure API key or AWS API key it's going
9:10:41to have a certain structure to it but
9:10:43what we are going to do is just build it
9:10:45on the basis of the name. If the name
9:10:47contains key, then we're going to filter
9:10:50it out. If it contains secret, then
9:10:53we're going to filter it out. And
9:10:54obviously, we want to tell the user
9:10:56that, hey, listen, you have your own
9:10:59list that you can mention of the
9:11:01variable names we need to filter out.
9:11:03And for that, the user will have to
9:11:05specify a configuration. So, first we'll
9:11:08have to go to configuration. py and take
9:11:11this shell command from the config. So
9:11:14we'll create a shell environment here
9:11:17which is of the type of shell
9:11:20environment policy or config. You can
9:11:23call it whatever you want. And then
9:11:25we're going to pass in the field. The
9:11:27default factory is going to be the shell
9:11:29environment policy. Cool.
9:11:32After that we're going to have a class
9:11:34of shell environment policy which is
9:11:36going to extend the base model. And here
9:11:39we can go ahead and define what
9:11:41variables do we need from shell. The
9:11:44first thing is ignore default excludes
9:11:48which is going to be a boolean value and
9:11:50by default it is going to be false. What
9:11:52this variable does is that it tells that
9:11:55do we want to ignore the default
9:11:57excludes or not. And yeah we are going
9:12:00to specify an exclude patterns list. So
9:12:03the user can specify a list of whatever
9:12:05patterns they want to ignore. So maybe
9:12:07they want to ignore API keys that
9:12:09contain key. That's how they can mention
9:12:12it in the list. So key is ignored. Or
9:12:15maybe they want to ignore anything that
9:12:17contains shell. Now why is this specific
9:12:20pattern used? It's kind of like regex,
9:12:22but it's specifically a Unix file
9:12:24pattern matching system. What it's
9:12:26trying to do is just that if there's
9:12:29anything before key, we take all of
9:12:31those characters. And if there's
9:12:33anything after key, we take all of those
9:12:35characters. But the key should be
9:12:37present within the name. And the same
9:12:39thing for shell or maybe you have a
9:12:41secret key or maybe you have a token
9:12:44key. All of that can come within these
9:12:47asterisk. All they mean is that the
9:12:50command or the list that the user
9:12:52mentions should have either a key or a
9:12:55secret or token to be excluded otherwise
9:12:58they'll be included. And what this
9:13:00variable does is that it asks if we want
9:13:02to ignore the default excludes that are
9:13:05present. By default, we want to ignore
9:13:08them, but if it is true, we will not
9:13:10ignore them. And these are the exclude
9:13:12patterns. So, we are going to have a
9:13:15list of strings and we are going to have
9:13:17a field where we are going to have a
9:13:19default factory and in the default
9:13:21factory you might have noticed every
9:13:23single time we passing an instance of
9:13:24the class, you're not really calling a
9:13:26class. Similarly, when we pass in a
9:13:29function, we don't have to pass in the
9:13:32called function. So, we can't pass in
9:13:33just a key like that. So, you have key.
9:13:36You know, this is the default list of
9:13:39what patterns we want to exclude. Then
9:13:41you have token, then you have secret.
9:13:46You can't just pass in a list. What you
9:13:47need to do is create a lambda function
9:13:49or any sort of function and pass in the
9:13:52reference to that function so that it
9:13:54can be called. So by default these keys
9:13:57will be ignored. If the user wants to
9:13:59specify anything else they can. And
9:14:02finally after this we are going to have
9:14:03a set variables which is going to be a
9:14:06dictionary string, string. And by
9:14:10default it's going to have the value of
9:14:13a dictionary. Now what is the set
9:14:15variables? Many times you know when
9:14:18we're trying to copy a environment
9:14:20variable let's say we have something
9:14:22like node environment and by default
9:14:25maybe in your shell environment
9:14:27variables you specified the value to be
9:14:29production but when the LLM is doing it
9:14:33you want the node environment to be
9:14:34development now you can't go ahead and
9:14:37change that in any way right because in
9:14:39your real environment variables you do
9:14:41need production but in your LLM shell
9:14:45you want development And so what you can
9:14:47do is set up this overriding variable in
9:14:51the configuration. So yeah, that's
9:14:54exactly what we're doing. And this is
9:14:56all about the shell environment policy.
9:14:59As you can see, we have set it up over
9:15:00here as well. Now I can use the
9:15:03configuration in the shell. py. But
9:15:05before I use it, I'll just create a
9:15:07function to build the environment
9:15:09variables. You know, a helper function
9:15:11would just be nice. So I'll have
9:15:13environment is equal to self dot build
9:15:16environment and then I'll just copy this
9:15:19function and create it out. So we have
9:15:22def envir build environment then we get
9:15:24a self and it's going to return a
9:15:27dictionary of string string. Now the
9:15:29first thing we want to do is copy all of
9:15:31the environment variables that already
9:15:33exist. So we have OS. So let's import OS
9:15:36and then we have environment.get get
9:15:40that will get us the environment
9:15:42variable dictionary but what I want to
9:15:44do is copy the dictionary that we get
9:15:47you know because if we do get we get the
9:15:49actual environment object we don't want
9:15:51to update it what I'll do is copy so I
9:15:53get a copy of that and now I can update
9:15:56this environment variable as we need it
9:15:58also I want to get access to the config
9:16:01shell environment because based on that
9:16:04we are going to exclude the patterns set
9:16:06the variables all of that. So I need to
9:16:09access self.config. But as you can see,
9:16:11we don't have access to config. Now how
9:16:14do we get access to config? Well, we can
9:16:16get it through the init of the shell
9:16:18tool. But I'll go one step deeper. I'll
9:16:21go into the tool. We have an init
9:16:23function here. And I'll just pass in the
9:16:25configure
9:16:27so that it's given to every tool. In
9:16:30future tools, for example, maybe GP,
9:16:32glob, maybe you want some configuration
9:16:34to be present. So you can access the
9:16:38self do.config over there as well. And
9:16:40now I can do self dot config is equal to
9:16:44config. And before I forget wherever
9:16:47this tool is called I want to pass in
9:16:50the configuration. And that is
9:16:52specifically called in tool registry or
9:16:55registry. py. So here for tool class in
9:16:59get all built-in tools which is in the
9:17:02create default registry we will get the
9:17:04configuration. So let's get it and then
9:17:08we can just pass in the config wherever
9:17:10this tool class is called because this
9:17:11tool class is going to be of the type of
9:17:14tool right because get all built-in
9:17:16tools does contain everything that just
9:17:18extends tool. So we pass in the config
9:17:22and that's it. Now wherever this create
9:17:24default registry is called which is in
9:17:26session. py we have to pass in the
9:17:28config and luckily we do have config
9:17:30here. So we'll just pass it in and
9:17:34that's it. Now we can go to shell and
9:17:36access the self.config.
9:17:38See? And now we get access to the shell
9:17:41environment. And we'll just say set it
9:17:44to a variable called policy. Maybe you
9:17:46want to call this shell environment.
9:17:48Anything is fine. And now what I want to
9:17:51do is first check if the default
9:17:54excludes is set to false. If default
9:17:56excludes is set to false in that case we
9:17:59want to filter out variables with keep
9:18:02token secret or any other thing that the
9:18:05user has mentioned. And if the ignore
9:18:08default excludes is true then we don't
9:18:10want to do any of the pattern matching
9:18:14based on the exclude patterns list that
9:18:16was present in the config. You know if
9:18:20this is false we'll exclude these
9:18:22patterns otherwise we won't. So let's go
9:18:24ahead and check if not policy dot ignore
9:18:28or not policy sorry if not shell
9:18:30environment do ignore default excludes
9:18:33if this is set to false then we'll go
9:18:35over every pattern in policy or shell
9:18:38environment dot exclude patterns
9:18:42so now I have access to let's say key or
9:18:44token or secret and now I'll just create
9:18:50a list of whatever keys I want to remove
9:18:52right so I'll go for every key that's
9:18:55present in this environment variable and
9:18:58I'll just match it with this pattern if
9:19:01it is true then we'll store it in a list
9:19:03and remove all of the keys from that
9:19:05list so we have keys to remove which is
9:19:08equal to a list and for every key in the
9:19:11environment variable dot keys we'll
9:19:14check if the fn match and we have to
9:19:17import fn match fn match is essentially
9:19:20a built-in library that allows us to do
9:19:23pattern matching on Unix type file names
9:19:26and what we have in this patterns list
9:19:28is also Unix type pattern names. So we
9:19:32can just use that. So it will just help
9:19:34us match whatever keys match in the
9:19:38environment variable to a pattern. So
9:19:40the first thing we have to pass in is
9:19:42key do.upper and then we'll also have
9:19:46the pattern dot upper and we'll pass in
9:19:50the key here. Now that I have a list of
9:19:53all the keys that I want to remove and
9:19:55again I just found out by going over all
9:19:59of the environment variables and
9:20:00matching them using fn match to the
9:20:04pattern that we got from exclude
9:20:07patterns. And now we just have to delete
9:20:09all of these keys. So we'll do for K in
9:20:12keys to remove we'll delete the
9:20:14environment at K because you know this
9:20:17is an object or a dictionary and
9:20:18dictionary property can be deleted like
9:20:21that and now we'll get out of this
9:20:23entire if condition with this we have
9:20:26removed all the keys that we wanted to
9:20:28we filter out everything and now we just
9:20:30have to apply the overrided variables so
9:20:33we'll have if the policy or shell
9:20:37environment I don't know why I keep
9:20:38doing policy if it is set to the
9:20:41variables. You know, if like that list
9:20:44is not empty, in that case I'll just do
9:20:47environment.update
9:20:48and pass in the shell environment dot
9:20:52set variables.
9:20:54That's it. Now I can go ahead and return
9:20:57this updated environment object. So now
9:21:01we have access to this environment here.
9:21:03I just have to start the shell. But
9:21:07again as I mentioned before we first
9:21:09have to determine the shell we are on
9:21:11based on the platform. So we'll have if
9:21:14system and let's import system
9:21:16dotplatform is equal to windows 32. In
9:21:20that case the shell command is going to
9:21:22look something like this. You'll have
9:21:24cmd.exe.
9:21:26Then you have to pass in forward/ c and
9:21:28pass in the parameter command that you
9:21:30want to execute. This is how you open up
9:21:33a shell in windows. Otherwise, we are on
9:21:37Mac or Linux. In that case, we can just
9:21:39use bash. So, we'll have /bin /bash.
9:21:43Then you have hyphen c. And you'll pass
9:21:46in the parameters.com command again.
9:21:49Now, you might notice that this entire
9:21:50thing is blurred out for me, kind of
9:21:52grayed out for me. And the reason for
9:21:55that is pilance automatically knows that
9:21:58the system.platform we are on is Mac.
9:22:01So, obviously, this is obviously going
9:22:03to be false for us all the time. This is
9:22:05the one that will be true for me. But
9:22:07however, if you know we try to upload
9:22:10this package and anyone can use this AI
9:22:14agent in that case we do need this line
9:22:17so that Windows can execute as well. And
9:22:19now I just have to start the shell. How
9:22:22do I do that? Well, I can just use async
9:22:25io's method on it. So first I'll import
9:22:28async io. And now I can call the create
9:22:32subprocess execute command on that. Here
9:22:36I need to pass in multiple things. But
9:22:38first I'll explain what this line does.
9:22:42So this create subprocess exec just
9:22:44spawns a new operating system process.
9:22:48And the operating system process that we
9:22:50are going to pass in over here. The
9:22:51program that we are going to run is this
9:22:53shell command. and we need to
9:22:56deconstruct it before we can pass it in.
9:22:59The benefit of this is that this create
9:23:01subprocess exec does not block the event
9:23:04loop. So everything can work as expected
9:23:07and it lets us stream the standard
9:23:09output and standard error asynchronously
9:23:12if that's needed. In our case, we don't
9:23:14really need to stream anything, but it's
9:23:16totally possible. And also this allows
9:23:19us to fully control how this subprocess
9:23:23works. You can kill it, you can wait,
9:23:25you can read the output, all of that. So
9:23:27let's go ahead and store this in a
9:23:29variable. Let's call this process which
9:23:32is equal to await this. Now we can pass
9:23:35in multiple other things like standard
9:23:37output. So the for the standard output
9:23:40we can just do async io.process
9:23:43pipe. What this pipe does is that it
9:23:47just tells us that hey listen we are
9:23:49capturing the standard output and it
9:23:51will be made available by doing process
9:23:54dot standard output. So if in any case I
9:23:57want to get access to the output I can
9:24:00just do it using this variable dot
9:24:02standard output. That's what this pipe
9:24:04does. And the same thing we need for
9:24:06standard error. So let me just copy
9:24:08paste that. This will be available as
9:24:11process.standard
9:24:12error. Then I can pass in the working
9:24:15directory in which we want to execute
9:24:16which we already have over here.
9:24:20Then we have the environment variables
9:24:23which we also got access to from here.
9:24:26And after that we will have a variable
9:24:29start new session equal to true.
9:24:32Every time we try to create a subprocess
9:24:36exec that means start a new shell tool
9:24:40it's going to start a new session. You
9:24:41can sync this with our own session. py
9:24:44if you want, but in my case, I always
9:24:47want this new session to be created no
9:24:50matter what, even if you're in the same
9:24:52session. But you can reuse it. Totally
9:24:54fine. And now I would like to wait for
9:24:58this entire standard output and standard
9:25:01error to get over. As I said, we're not
9:25:03going to stream this out. So I'll do
9:25:06await async io dotwait for and then I'll
9:25:11pass in the first thing which is a
9:25:13future. What I need to pass in here is
9:25:15process docomunicate. So this is going
9:25:18to be the input that we are waiting for.
9:25:20As soon as this cool routine gets over,
9:25:23we'll get some value and the value we'll
9:25:25get is standard output data and standard
9:25:28error data and we'll obviously awaited
9:25:31it. Also we'll specify the timeout here.
9:25:34How much time do we want to wait for?
9:25:36And if we don't specify a timeout, it
9:25:39will go on forever. We don't want that.
9:25:41That's why we had parameters dot
9:25:44timeout. And that's good. Now, whenever
9:25:48we run into a timeout error, it will
9:25:50just raise an exception. I want to avoid
9:25:53that. So, I'll have try except
9:25:57and I'll wait for this async.io timeout
9:26:00error. And now I can just check that hey
9:26:03if the system.platform we are in is not
9:26:06equal to Windows 32 in that case Mac or
9:26:10Linux then we'll just do OS dot kill
9:26:15program so pg and now I'll just pass in
9:26:18the program ID. Now I need to get the
9:26:21program ID. For that I'll do os.get get
9:26:23pg ID and then I'll pass in this process
9:26:28variable which will give me the P ID
9:26:31which is the process ID. So based on the
9:26:33process ID we'll get the program ID
9:26:35which we'll pass to OS.kpg to kill the
9:26:39shell. And now I'll just pass in the
9:26:41signal dot sigkill. Sikkill is just a
9:26:46way to say that we want to immediately
9:26:49terminate this u Unix like operating
9:26:52system. All right. So if you have a Mac
9:26:54OS or a Linux, you're just forcefully
9:26:58closing off the signal or this thing in
9:27:02our case the program without giving it a
9:27:04chance to save any data or clean up
9:27:06resources or anything. Otherwise, we are
9:27:09on Windows and surprisingly for Windows,
9:27:12it's much easier. You can just do
9:27:14process.kill and it will work. After
9:27:17that, we'll just wait for the process to
9:27:19get over. And once it gets over, I'm
9:27:22just going to return an error result. So
9:27:25we'll have return tool result dot error
9:27:28result and I'll pass in that hey this
9:27:32command timed out after and then I'll
9:27:36pass in the timeout params timeout
9:27:38seconds. Finally after all of this we
9:27:41might get the standard output. So at
9:27:44this point if there's any exception
9:27:46we've returned the result. We've closed
9:27:48off everything we have to so no
9:27:50resources are leaked or unnecessarily
9:27:53used and now I'll just decode the
9:27:55output. So to decode the output I can
9:27:58just do standard output data which is of
9:28:00the type of bytes. I'll just do decoding
9:28:03and then I'll pass in the UTF which is
9:28:05UTF8 and then I'll pass in errors is
9:28:08equal to replace where you know errors
9:28:12replace just means that let's say when
9:28:14we try to decode the data it had some
9:28:18bite which was undecodable. So instead
9:28:22of returning a uni decode error or some
9:28:25sort of exception like that, it will
9:28:28just replace that bite that had the
9:28:31problem with a placeholder character so
9:28:33that it doesn't cause any more issues
9:28:35for us. And now we can just store this
9:28:38in standard output. Similar to this,
9:28:40we'll have one done for standard error
9:28:42as well. So we'll have standard error
9:28:45data
9:28:46dot decode.
9:28:48And now I want to format the output very
9:28:51nicely. So I'll have if the standard
9:28:55output strip exists that means we have a
9:28:58successful output. In that case I'll
9:29:01create a variable at the top and append
9:29:03this standard output to it. So we have
9:29:05output plus equals standard output dot
9:29:09strip or actually let me do r strip
9:29:12because from the right hand side we're
9:29:13going to strip out the standard output
9:29:15because many times it can be the case
9:29:17that the output is nicely formatted. So
9:29:19from the left hand side you have
9:29:21everything indented and if you remove
9:29:24that off everything will be in one
9:29:26single line and it might not look good.
9:29:28So let's just maintain that. Other than
9:29:31that we can also have a standard error
9:29:34and we'll strip that as well. So if
9:29:37standard errorstrip exists that means
9:29:40it's not empty. In that case I just want
9:29:43to check if the output is not an empty
9:29:45string. If output is an empty string
9:29:48then I want to explicitly tell that hey
9:29:51we are having a standard error here. Or
9:29:54actually even if the output is empty in
9:29:57that case also I'll just print out that
9:30:00there's a standard error. So I'll have
9:30:03plus equals three hyphens followed by
9:30:06standard error and three hyphens again.
9:30:09And let me leave a new line as well
9:30:11here. After that we can go ahead and
9:30:15append the standard error. So we'll have
9:30:17output plus equals and then I'll just
9:30:20pass in the standard error dot r strip.
9:30:25Let me also put a back slash n here so
9:30:28that the standard error can come in on a
9:30:29new line. And finally I'll check if the
9:30:32exit code is not equal to zero. Now I
9:30:36want to get access to that exit code
9:30:38which I will at the top over here. Exit
9:30:42code is just equal to process doexit
9:30:46code. right? Or process.turn code
9:30:49because whatever the process returns if
9:30:51it's a zero it's going to be success. If
9:30:53it's anything other than that it's a
9:30:54failure. So just in case exit code is
9:30:57not equal to zero then we're going to go
9:30:59ahead and app and append it to the
9:31:01output. So we have output plus equals
9:31:04and then you pass in the exit code which
9:31:07is something like this.
9:31:10Cool. Also let me put an f string here.
9:31:14Great. So we have our output ready. But
9:31:17before we go any further, I would just
9:31:19like to truncate the output if it's too
9:31:21big. So I'll just check that hey if the
9:31:23length of the output
9:31:25is greater than 100
9:31:28into 124 which is 100 kilobytes because
9:31:32you know the output here is in a string
9:31:35format and string is just a bunch of
9:31:38characters. So one character is equal to
9:31:41one byte. So the maximum thing we need
9:31:44is 100 kilobytes or very simply it is
9:31:48just 1,2400
9:31:50characters long. In that case we'll just
9:31:53have output equal to output until 100
9:31:56into 1024
9:31:58plus and then we'll have back slash n
9:32:01then we'll have three dots where we say
9:32:04output truncated.
9:32:07Cool. Now I can just go ahead and return
9:32:09this tool result dot success result,
9:32:12right? No, I'm just going to return an
9:32:15instance of tool result because we
9:32:16really don't know if it was a success or
9:32:18a failure. We can only know that on the
9:32:20basis of exit code. If the exit code is
9:32:22zero, we are having a success otherwise
9:32:25we are having an error. So let's have
9:32:28success is equal to exit code equal
9:32:31equal to zero. So that is a success.
9:32:35And for error we are going to have
9:32:38standard error returned if the exit code
9:32:42is not equal to zero. Otherwise
9:32:46it's going to be null. Think about it.
9:32:49If exit code is not zero we having a
9:32:51failure. So we return standard error
9:32:53otherwise null. Right? And now we'll
9:32:56also mention the exit code which is exit
9:32:59code. And by the way, your tool result
9:33:02might not have exit code because this is
9:33:04the second time I'm recording this part
9:33:06of the video. There was some software
9:33:08bug in the recording. So I wasn't able
9:33:10to remove everything very cleanly. But
9:33:13the point is the exit code needs to be
9:33:15attached to tool result now. So make
9:33:18sure you have that in the data class and
9:33:20nothing else needs to be changed in the
9:33:21code here. Just add this exit code in
9:33:23the tool result data class. And now
9:33:27since we have specified the exit code as
9:33:29well, let's pass in the final thing
9:33:31which is the output. And we already have
9:33:34the output with us as simple as it gets.
9:33:37So that is it about the shell tool. Now
9:33:40let's go to the init. py where we're
9:33:42going to add the shell tool. So let's
9:33:45import it from tools.builtin.shell.
9:33:48And that's it. Also in the all here,
9:33:52let's just have write tool. Then we also
9:33:55have the edit tool and we also have the
9:33:59shell tool. You know, these are all of
9:34:01the tools that are imported from
9:34:04tools.builtin.
9:34:05So, it's good to have them over here.
9:34:07However, we won't be using these tools
9:34:09outside of the built-in library. All
9:34:12right, the built-in module. Great. Now
9:34:15that we have that done, we can go to our
9:34:19main. py. And here the first thing I
9:34:22would like to pass is in tool call
9:34:23complete. I would like to get access to
9:34:25the exit code, right? And we do have it
9:34:28over here. You might not have it again.
9:34:30I recorded it but I forgot to remove
9:34:32certain parts when I'm re-recording. So
9:34:35we have exit code here. Let's pass that
9:34:38in from tool call complete. So we have
9:34:40event data.get exit code. And if it's
9:34:44not present, it's going to be null. And
9:34:47now I believe there's another place
9:34:49where we have to change it because exit
9:34:51code is coming from tool call complete
9:34:54right which is really agent event type
9:34:56dottool complete. So let me go to this
9:34:59particular part where we have agent tool
9:35:03call complete where we have the result
9:35:05and here just make sure you have passed
9:35:08in the exit code. Again I forgot to
9:35:10remove this part as well. Sorry about
9:35:12that. But yeah, make sure you add this
9:35:15exit code because you know if you don't
9:35:17do this, similar to diff, we'll get an
9:35:20error. So you have result.exit code here
9:35:22as well. And now we can go to the tui.
9:35:25py file where we go ahead and
9:35:29instantiate stuff related to shell. So
9:35:32the very first thing is that we'll have
9:35:34to create our own lf condition here to
9:35:37display the shell. But even before that
9:35:40there were a few things we'll have to
9:35:42change here right for example in the
9:35:44ordered args we'll have to specify the
9:35:47preferred order. So whenever we have a
9:35:49shell tool what is the preferred order
9:35:53for us. So we'll have command time out
9:35:56and then we'll have the current working
9:35:59directory. If you want, you can go ahead
9:36:01and change this order. But whenever the
9:36:05user executes a shell command or
9:36:07whenever the LLM executes a shell
9:36:09command, the user wants to see the
9:36:11command that was executed at the very
9:36:14top. After that, there'll be non kind of
9:36:17essential stuff. But if you think
9:36:19something else might be a better UX, go
9:36:21for it. So, we have shell done over
9:36:24here. And then I can just go down where
9:36:27we have tool call complete. and just
9:36:30create another L if condition for the
9:36:32shell. So l if the
9:36:35name is equal to shell in that case what
9:36:39do we want to display? Well, first of
9:36:41all, I would like to get access to the
9:36:43command, right? So, I'll do command is
9:36:46equal to asks.get command because really
9:36:49we are not sending it the command from
9:36:52our event data. So, the metadata might
9:36:55not contain the command. We're not
9:36:57sending it. But if you want you can send
9:36:59it. However, for this purposes, we had
9:37:02done something in tool call start. So,
9:37:04whenever tool call started, we said
9:37:06self.tool asks by call ID. call ID is
9:37:10equal to arguments. So based on the tool
9:37:12call ID,
9:37:14I set up the arguments. So now it's time
9:37:16to retrieve those arguments. So I'll
9:37:20just go down here in tool call complete
9:37:22and just have as equal to self.tool
9:37:27asks by call ID dot get and then we'll
9:37:30pass in the call ID. If it does not
9:37:33exist, then we're going to have an empty
9:37:35object. And now I can use this args down
9:37:38here to get access to command. And if
9:37:41the command is instance of a string and
9:37:45command.strip does not exist that means
9:37:48you know it's empty or something like
9:37:51that we don't want to do anything but if
9:37:53it's not empty in that case I want to
9:37:56append this command to the block. So we
9:37:58have text then I'll pass in a dollar
9:38:01sign which is usually the way the shell
9:38:04commands look right if I want to tell
9:38:07the user that we executed something
9:38:09dollar followed by a command tells the
9:38:12user that hey we executed this so it's
9:38:14just nice UI technique and then we're
9:38:18just doing command dot strip and then we
9:38:21can specify the style of this which is
9:38:23muted then we can again check if the
9:38:25exit code is not none. If it is not
9:38:29none, then we'll have blogs.append.
9:38:31Then I'll pass in the text which is exit
9:38:35code is equal to exit code.
9:38:40And then we'll also have the style is
9:38:42equal to muted. After that, we want to
9:38:45truncate the output if needed. So we can
9:38:48just do output display is equal to
9:38:52truncate text. And then I'll pass in the
9:38:55text which is the output. Then the model
9:38:59which is self dot model or actually I
9:39:02think it's self do.config domod
9:39:05name. And then finally we're going to
9:39:08add the max number of token which is
9:39:11self domax block tokens.
9:39:14Cool. And now we just have to display it
9:39:16out on the screen. So to display it
9:39:19we're just going to use this
9:39:20blocks.append syntax. So we do that and
9:39:24then we pass in the output display. The
9:39:27text here is going to be a simple text.
9:39:29It's not a diff. It's a simple text of
9:39:32what the command line did. The theme is
9:39:34Monokai and the word wrap is true. So
9:39:37that seems good enough to me. Now let's
9:39:39try to execute it and see if it works.
9:39:41So I close all the save files. Let's try
9:39:44to run it. So I'll have Python main. py.
9:39:47And now I can tell it to create hello
9:39:51world.py. py file for me and run it to
9:39:57test it. So let's see if it works. So as
9:40:00you can see the shell command is
9:40:02executed here. So we have command python
9:40:04hello world. py and then the output
9:40:07shows up which is python hello world. py
9:40:10the command executed the exit code and
9:40:12what is the output of the shell. Right?
9:40:15So this is the hello world displaying in
9:40:18the syntax form and then the assistant
9:40:21just tells us it created and executed
9:40:23successfully. The output is hello world.
9:40:25Great. And as you can see the shell is
9:40:27also of a different color than the write
9:40:30file tool which is of the yellow color.
9:40:32So this is of the magenta color and this
9:40:34is specifically because in the agent
9:40:36theme we specified that our tool.shell
9:40:40should be magenta. So when we have
9:40:42network tool or MCP tool, we're going to
9:40:44add different colors and that's pretty
9:40:46cool. Just makes our app look fancy.
9:40:50So yeah, that's about it. Let me exit
9:40:52this so that you know we can get started
9:40:55on the next thing. And the next tool
9:40:56we're going to work on is the list
9:40:58directory tool. So we have list diir. py
9:41:02file. And now you might wonder what does
9:41:04this tool do? Well, this tool just lists
9:41:07out all of the contents within a
9:41:09directory. not all of them. We we're
9:41:12going to ignore some files. But the
9:41:14point is listing out the essential files
9:41:16in a directory. The next question you
9:41:19might have is since we already have
9:41:21shell tool, why do we need a tool for
9:41:23list directory? The reason for that is
9:41:26the first one. If an LLM passes a
9:41:29command that is like ls-la which
9:41:31essentially prints out all of the files
9:41:34within a directory including the files
9:41:36that are hidden like they all will be
9:41:40displayed and for some directories it
9:41:42can be messy. So it will be a very large
9:41:44output and it can be very annoying to
9:41:48pass that output. The second thing is
9:41:51that the output varies by the operating
9:41:55system. what Windows will give as the
9:41:58output versus what Mac OS or Linux will
9:42:00give as the output for this is going to
9:42:02be different. So we don't want to pass
9:42:05that, right? I mean it will get too
9:42:06annoying. Also there's this
9:42:09crossplatform consistency issue. If you
9:42:12are on Linux or Mac you will do ls but
9:42:15if you're on Windows you have to do
9:42:16something like diir or if you're on
9:42:18powershell you have something like get
9:42:22child item something like this.
9:42:25We obviously want that to be abstracted
9:42:28away by this list directory tool. Also,
9:42:31this list directory tool is a rather
9:42:33common tool. It will be used multiple
9:42:36times by the LLM. So, first of all,
9:42:39maybe when it's tasked to create a hello
9:42:41world py, it will try to see if this
9:42:44hello world py file already exists. If
9:42:46it exists, it will try to read it. So,
9:42:49and then it will try to edit it based on
9:42:51our instructions. So you know first it
9:42:54has to know what contents exist within a
9:42:56file. So that's the first time it will
9:42:58do it. And then let's say it makes them
9:43:01changes and then it lists again just to
9:43:04ensure that its changes are present.
9:43:06Maybe by mistake it did not delete the
9:43:09hello world. py according to our system
9:43:11prompt itself. We have told that it
9:43:14should be very conscious about making
9:43:16changes and it should be very thorough
9:43:18with its changes. Again, for that
9:43:20reason, it will it might try to use the
9:43:23list directory command. So, it's just
9:43:25better to have this as a separate tool.
9:43:27To give you another perspective to this,
9:43:29we're having a list directory tool
9:43:31despite having a shell tool is because
9:43:34of the same reason we need a read file
9:43:37tool or an edit file tool or a write
9:43:39file tool despite having a shell because
9:43:42through shell we can run the commands
9:43:44like cat. CAT just allows us to read a
9:43:47file, right? Then we have something like
9:43:50touch that will allow us to create a new
9:43:53file or if it wants to edit anything
9:43:55there are commands for it. So why do we
9:43:57need to add those tools as well? Just
9:44:00having dedicated tools will help the LLM
9:44:03make a better choice as well. And we
9:44:06don't have to parse anything that's
9:44:08annoying and will just make our lives
9:44:10easier as well as the efficiency of the
9:44:12model because as I explained before if
9:44:15you have ls-la
9:44:17it can return a large number of files
9:44:20and that means we will have to truncate
9:44:22it and some data will be lost in that.
9:44:25How about instead of doing that, we just
9:44:27create our tool which will just pass the
9:44:30stuff we need it to and if our tool does
9:44:33not satisfy those conditions, the LLM
9:44:36can go back to shell and use that. But
9:44:39in most cases, list diir will be fine.
9:44:41So let's work on that now. Now we know
9:44:43the drill of how to create a list
9:44:45directory tool. So the very first thing
9:44:47is going to be to create the class list
9:44:49directory tool which is going to extend
9:44:51tool from base. And then we are just
9:44:54going to define a bunch of things like
9:44:56the name of the tool which is list diir.
9:44:59Then the description and the description
9:45:02is quite simple here. It just lists the
9:45:04contents of a directory. Nothing much.
9:45:07After that we're going to have the kind.
9:45:09What is the kind of this? Well, list
9:45:11directory is simply reading the file,
9:45:13right? The all the files that are in a
9:45:16directory. So this will be tool kind.
9:45:19Let's import it. And then we have dot
9:45:21read. After that we have the schema and
9:45:23now I have to create the schema. So
9:45:25there will be list directory parameters
9:45:29which is going to extend the base model.
9:45:31Then I have to import from paidantic
9:45:34import base model. After that we have
9:45:38this and then there are two parameters
9:45:40that we're going to have. The first one
9:45:42is the path. What path do you want to
9:45:45list the directory for? So you know what
9:45:48directory we have. And by default the
9:45:50value is going to be a dot because we
9:45:52want it to be in the current directory.
9:45:54Also let's import from pyantic field.
9:45:56And then we have the description. Let me
9:45:59pass in description here. And the
9:46:01description is directory path to list.
9:46:05And by default we have the current
9:46:08directory. Easy. After that we have the
9:46:12include hidden flag. Basically do we
9:46:15want to display all of the hidden files
9:46:17as well? Hidden files for our case is
9:46:19just a simple if the file contains a dot
9:46:22in the beginning or not because if the
9:46:25file path contains a dot in the
9:46:27beginning for example getit that's a
9:46:29hidden file or we have env that's a
9:46:32hidden file. So do they want to include
9:46:34the hidden files or not? So let's have
9:46:37that. It's going to be a boolean value
9:46:40and by default the value is going to be
9:46:42false because I don't want the hidden
9:46:44files to be displayed unless the LLM
9:46:47turns it on to true. And then we'll just
9:46:50say whether to include hidden files and
9:46:55directories and maybe we can also
9:46:57specify that the default value is false.
9:47:00Okay, maybe I misspelled. So we have
9:47:02false here. Okay, great. So now I can
9:47:04take this list directory parameters and
9:47:06attach it to the schema. Awesome. And
9:47:09now there's only one thing to do. I have
9:47:11to create the execute function. So I
9:47:14have the execute here. And then we are
9:47:16going to have an invocation of the type
9:47:19of tool invocation. And we're going to
9:47:21return a tool result. Let's import both
9:47:23of them. There we go. And I misspelled
9:47:27this. So let's import it again. Great.
9:47:30Now I want the parameters, right? So I
9:47:32have parameters is equal to list
9:47:34directory parameters and then I just
9:47:37have invocation.parameters
9:47:39deconstructed so that we have this nice
9:47:41class. After that we want to resolve the
9:47:44path if you know the pams.path is
9:47:46present and it should always be present
9:47:48because we have given it a default value
9:47:52and it is kind of a required value as
9:47:54well because without the path you know
9:47:56what file are you even listing what
9:47:58directory are you listing. So we have
9:48:00directory path which is equal to and now
9:48:02I have to resolve it because this path
9:48:05can be relative or absolute correct if
9:48:08it's relative you know this dot is just
9:48:10a relative path whatever current working
9:48:13directory we just want to list out the
9:48:15files in that so that's relative and it
9:48:18can also be absolute so we have to
9:48:20resolve the path just like we did in
9:48:22write and read files as well so here
9:48:25I'll pass in the invocation cm current
9:48:28working directory which is the base path
9:48:30and then the path that the LLM gives us
9:48:33which is parameters dot path. Great. Now
9:48:37I just have to check if this directory
9:48:39path exists and if it doesn't or let's
9:48:41say the directory is not a directory.
9:48:43Maybe the user or the llm passed in a
9:48:46file that's not correct because how are
9:48:49you going to list all of the content of
9:48:51a file using list directory tool? You
9:48:54need read file tool for that. So we're
9:48:57just going to say if not directory dot
9:49:00or let's say directory path dot exist.
9:49:04So if the path of the directory does not
9:49:06exist or the directory path is not a
9:49:10directory at all, it's a file or maybe
9:49:13something else. In that case, we are
9:49:15going to return a tool result dot error
9:49:18result and I'll pass in well the
9:49:20directory does not exist and then pass
9:49:23in the directory path for the LLM to
9:49:25understand what path we tried to do and
9:49:28based on that the next iteration the LLM
9:49:30can try to resolve this error. Now one
9:49:33thing I would like to note before moving
9:49:35forward is that you can have more
9:49:37arguments over here. List directory is
9:49:39not a very simple tool. We are having a
9:49:42very simple implementation but you can
9:49:44have more things like recursive list
9:49:46directory. Meaning
9:49:49if we have a folder we go within that
9:49:52folder and find out all of the files
9:49:54within that folder. And if the LLM wants
9:49:57it can set recursive to true to do that.
9:50:00So yeah, you can go very deep in this
9:50:02list directory parameters but you get
9:50:04the point of this tool. We'll just add
9:50:06it and you can expand it handle more
9:50:08edge cases that I haven't to get it
9:50:11working. Now we'll try to list the
9:50:13directory and to do that we just have to
9:50:15do directory path dot directory
9:50:20meaning we iterate over the files in
9:50:23this directory. This does not yield any
9:50:26result for the special paths dot and dot
9:50:28dot dot. Now the thing is this will give
9:50:32you the all the items of the directory.
9:50:34It will list out the directory items.
9:50:36Now the thing about list directory is
9:50:38that when you try to print out all of
9:50:41the items in list directory it will just
9:50:44spit it out in random order. I want it
9:50:46to be in a consistent order and for that
9:50:48reason I'm going to call sorted on this
9:50:50and then also pass in the key where we
9:50:53are going to have a lambda path which is
9:50:55p. And then I just want to say that hey
9:50:58this path should not be a directory. So
9:51:01I have P is directory set to not and
9:51:04then the other thing should be P do.name
9:51:07dot lower. So everything is first
9:51:11lowerase
9:51:12and this will give us the items and I'll
9:51:16also put this in a try block because we
9:51:18are having iterate directory that might
9:51:20return an error. So we'll just have an
9:51:23except here uh exception as e
9:51:27specifically where I'll just do return
9:51:30tool result dot error result and I'll
9:51:32pass in well error listing directory and
9:51:37I'll pass in the exception. Great. After
9:51:41that I'll filter out all of the hidden
9:51:44files if this hidden files is set to
9:51:47true. So I'll just check that hey if
9:51:49params dot hidden include hidden is true
9:51:52in that case I'll just have items is
9:51:56equal to or actually if the
9:51:58params.incclude include hidden is set to
9:52:00false. In that case, I want to filter
9:52:04out all of the hidden files, right?
9:52:06Because if include hidden is true, we
9:52:08just have to display everything that
9:52:09items has over here because iterate
9:52:11directory will give us all of the dot
9:52:14related [clears throat] files as well.
9:52:15And now I can have item for item in
9:52:18items if not item dot name dot starts
9:52:24with and we let's say start with a dot.
9:52:28Cool. So this will filter out all of the
9:52:30hidden files. Else we just want the
9:52:33items to be items. So we don't need
9:52:35really need an else condition here. And
9:52:37then I'll just check that hey if there
9:52:39are no items maybe all of the files that
9:52:41are present within a directory are
9:52:44dotreated and include hidden is set to
9:52:46false. In that case the items might be
9:52:49empty. Or let's say overall the items in
9:52:52a folder are just not present. In that
9:52:54case also we might have an empty
9:52:56directory. So we'll just return tool
9:52:58result dots success result. We don't
9:53:00have to specify a failure because in
9:53:03this case we do have the directory to be
9:53:06empty. The entire thing was successful.
9:53:07It's just that it is empty. So it's not
9:53:09a failure. I'll also attach some
9:53:11metadata where we'll have the path which
9:53:14is just going to be the path that we
9:53:16resolved at the top directory path. And
9:53:19then maybe you can also pass in the
9:53:22entries the number of entries that this
9:53:25had which is zero. or if you want to
9:53:27skip it entirely go ahead and then we
9:53:30are just going to format the output now
9:53:33so I'll create a lines list and then
9:53:36I'll have for item in items
9:53:40and then I'll check if item
9:53:43dot is directory if it is a directory
9:53:46then what I want to do is just add a
9:53:49forward slash at the end of the path so
9:53:52I'll have lines append then I'll pass in
9:53:55the item dot name with a forward slash
9:53:59because a folder always have a forward
9:54:01slash at the end of it otherwise we have
9:54:05a file and then I'll just do
9:54:06lines.append append item dot name after
9:54:10that I can just go ahead and return tool
9:54:12result dots success result and I'll pass
9:54:15in the output first the output is going
9:54:17to be back slashn dot join lines so
9:54:20whatever files and folders we got we
9:54:23just adding all of them on a new line if
9:54:26you want maybe you can also add a folder
9:54:29emoji over here to specify a folder and
9:54:33for file maybe you can use a file but I
9:54:35don't like emojis so Yeah. Then we're
9:54:38going to have meta data which is equal
9:54:40to and then you again have entries and
9:54:42path. So I'll just copy this paste it
9:54:45over here and the directory path is
9:54:49going to be the same. Maybe just convert
9:54:51it into string in both the places
9:54:53because directory path is a path object.
9:54:56So yeah and now the entries can be the
9:55:00length of the number of items that are
9:55:03present. you know how many ever items
9:55:06were present here that's the number of
9:55:08entries and that's our list directory
9:55:10tool as I mentioned you can just go
9:55:12deeper into this that's there's lots of
9:55:14stuff to do here you have full control
9:55:16over this tool now now I can go to the
9:55:20tui py where I'm just going to define
9:55:24some things the first one is the
9:55:26preferred order in the preferred order
9:55:29we have the list diir tool right and for
9:55:33this the very first thing that I want to
9:55:35show out is path and the second thing is
9:55:38include hidden if you have other
9:55:41parameters like recursive and stuff you
9:55:43can add it over here but this is good
9:55:46enough for me I'll scroll down to tool
9:55:48call complete where I'm going to add
9:55:50another lf condition after shell so we
9:55:53have lf tool name is equal to list
9:55:57directory if that is the case we'll
9:56:00first get the entries which is equal to
9:56:02meta metadata.get
9:56:04and then we have entries after that we
9:56:07have the path which is equal to
9:56:08metadata.get get path. So we have
9:56:12entries and the path and now I just have
9:56:14to show them. But it is possible that
9:56:16maybe the LLM hallucinated or something
9:56:20else happened that we don't get the
9:56:22path. We don't want our application to
9:56:24crash in such an instance. So I'll just
9:56:26have if is instance path of the type of
9:56:30string. In that case
9:56:33I'll just
9:56:35append it to a list. So I'll have let's
9:56:38say summary which is equal to a list and
9:56:41I'll append this summary to a list. So I
9:56:43have summary dot append path. Now
9:56:46another thing I'll check is if the
9:56:50entries is an integer if that's present
9:56:53I'll just append the entries as well.
9:56:56And maybe for entries I can style this a
9:56:58little bit. I'll just say that I have
9:57:01entries number of entries.
9:57:04So let's say seven entries and after
9:57:08that we'll check if the summary list
9:57:10even ex exists because let's say the LLM
9:57:13did not pass in or something wrong went
9:57:16and because of that we don't have any of
9:57:18these things in that case I'll just do
9:57:21blocks dot append in that case I
9:57:25obviously don't want to append any of
9:57:26this to the blocks right but if summary
9:57:30exists that means the list is not empty
9:57:32then I'll do blocks.append append. I'll
9:57:34pass in the text and within the text I
9:57:37need that dot symbol that kind of acts
9:57:40like a separator. And now we can just
9:57:43call dot join between them and join all
9:57:46of the summary related stuff.
9:57:49Good. And maybe the style of this is
9:57:53muted. And yeah, that looks good enough.
9:57:57I'll go ahead and truncate the output
9:58:00now. So I'll have output display is
9:58:04equal to and I'll call truncate text
9:58:06here as well. And in the truncate text
9:58:08we're going to pass in the text which is
9:58:10well the output. Then you have the model
9:58:14which is self.config domodel name. And
9:58:17finally your self domax block tokens.
9:58:20And now I can again just do
9:58:22blocks.append syntax because that is the
9:58:26only thing that will allow us to display
9:58:27our stuff nicely. So we have output
9:58:29display the text here is going to be
9:58:32there. Then theme is monokai and the
9:58:34word wrap is true.
9:58:36That looks good to me. I can save this
9:58:39entire file and I can start the agent
9:58:43again. This time I'll ask it explicitly
9:58:46to call the list directory tool for me
9:58:50and read all the files within it. Let's
9:58:53see if it completes that request. Well,
9:58:56as it turns out, there is no list
9:58:57directory tool. So, it starts calling
9:58:59ls-la.
9:59:01You know, that's a good thing actually
9:59:02because our assistant is not making any
9:59:05compromises. It will get the task done.
9:59:08So, it gets everything out displayed
9:59:10here. And as you can see, it's quite
9:59:13annoying to read this output on Windows.
9:59:16I'm sure it will be something else.
9:59:18That's why I created our list directory
9:59:20tool as well. Now, let me just exit
9:59:23this. The reason it did not know that
9:59:25there's a list directory tool is because
9:59:27we did not register it in the tools
9:59:32part. So in the built-in in it, we did
9:59:34not pass in the list directory tool
9:59:37here. That's how it did not know that
9:59:40yeah list directory tool also exists.
9:59:43And now I can import it and we're good
9:59:45again. So now I can close everything and
9:59:49hope that it works now. So I'll just say
9:59:52call well let's not explicitly say call
9:59:55list directory maybe it will
9:59:57automatically recognize but I can say
10:00:00read all the files in this directory and
10:00:04I'll hit enter
10:00:06and the reason for this error is because
10:00:08this is the first time we're trying to
10:00:10print out a boolean value on the
10:00:12terminal and we have not handled that
10:00:14case. So here in our list directory
10:00:17tool, we had include hidden set to a
10:00:20boolean value, right? But we've never
10:00:22handled the case where if we have a
10:00:23boolean, we have to essentially return a
10:00:25stringed version of that boolean to
10:00:28display to the user. So we'll have to go
10:00:30to our TUI so that we handle that case.
10:00:33We can just scroll up wherever we have
10:00:36the formatting done and that I believe
10:00:38was explicitly done when we passed in
10:00:41old string. So I can just search for old
10:00:43string and as you see over here this is
10:00:46the part where we were doing formatting
10:00:48of the arguments table. So if the value
10:00:52was a string and the key was content old
10:00:55string or new string we just displayed
10:00:58the lines so that we did not have to
10:00:59display the entire blob of text. Similar
10:01:03to this we can add another if condition
10:01:06here that hey listen if is instance
10:01:11value of the type of boolean right
10:01:14because the key is always going to be a
10:01:16string because it's the tool name but
10:01:19the value if it's a boolean value in
10:01:21that case I want to convert it into an I
10:01:24want to convert it into a string so I'll
10:01:26just have return string of value or
10:01:29actually instead of returning we just
10:01:31want to do value is equal to string of
10:01:33value because we adding the row here.
10:01:35Okay, great. Now let's try to do it
10:01:37again and then hit enter. As you can
10:01:40see, it calls the list directory. The
10:01:42path is given here. The include hidden
10:01:44is true. Then we have the directory
10:01:48name. If you want you can resolve this
10:01:50path using the utility function we
10:01:52created which is display path relative
10:01:55to current working directory. so that
10:01:56you get to know what directory we tried
10:01:58which is essentially a dot because we
10:02:02did it in the current directory. So the
10:02:04relative path would be the current
10:02:05directory and then there are 12 entries
10:02:08to read. Then you have AI agent venv as
10:02:12you can see include hidden is set to
10:02:14true. That's why we see all of the
10:02:16hidden files as well. Then we have all
10:02:18of the folders showing up first and then
10:02:21we have the Python files. Awesome. And
10:02:24now it goes ahead and reads all of the
10:02:27files. And since I've said file, it is
10:02:30only reading out the hello world. py and
10:02:32main.py files. If we add the option for
10:02:36the agent to go recursive, it will do
10:02:38that as well. And as you can see, our
10:02:40agent fails with one of the tool calls.
10:02:43The reason it fails is because it tries
10:02:46to do agent config.json, but there's no
10:02:49config.json in AI agent. it just assumes
10:02:52that there's going to be a config.json.
10:02:54In reality, there's config.l.
10:02:57So that's one case where you can just
10:02:59improve your system prompt so that it
10:03:02never makes that mistake again. But
10:03:04another thing here is that when the read
10:03:06file goes wrong, it doesn't really
10:03:08display the output error message and I
10:03:11know how to do that in tool call
10:03:13complete. You know, every single time
10:03:15we're just checking if it's any of the
10:03:17tool call names. Now if it's none of the
10:03:19tool calls name and the error object is
10:03:22present then I want to display the
10:03:25error. So I'll have if error and not
10:03:28success correct because yeah in that
10:03:31case we have an error we do not have a
10:03:33success then I'll just do blocks dot
10:03:36append or maybe you can just do blocks
10:03:38is equal to because you don't want any
10:03:40of the previous items if they are
10:03:42attached but in our case it's highly
10:03:43unlikely anything's attached because
10:03:46these all things happen when it is
10:03:49successful you know we have explicitly
10:03:51written out over here and success and
10:03:54success list. So it's highly unlikely
10:03:58that anything else will be present in
10:04:00blocks list. So we can just append it
10:04:03because creating a new list is well
10:04:05taking up more space. I do not want to
10:04:07do that without any reason. So I'll just
10:04:10do text then I'll pass in a error
10:04:13message and the error message is a
10:04:15string object and the style of this can
10:04:19be the error message itself and I have
10:04:22to pass in the style within this text.
10:04:25Then maybe we can try to truncate the
10:04:28error message as well. So I'll just call
10:04:30truncate text the error message. Well
10:04:33instead of truncating the error message
10:04:35because we've already printed it out.
10:04:37Let's try to truncate the output. And
10:04:39then we'll have the self.config.model
10:04:42name. Then we have the self domax block
10:04:45tokens. And I'll just pass it within the
10:04:49output display variable. And then I'll
10:04:51just do blocks do.append.
10:04:54And yeah, let me just also check that
10:04:57hey, if the output display strip is
10:05:00present because it can be the case that
10:05:02the output is just an empty string. If
10:05:04that is the case where output display is
10:05:08present then we're going to pass in the
10:05:11syntax and similar to everything we've
10:05:13done till now we're just going to paste
10:05:15that over here.
10:05:17However, if the output display is not
10:05:19present in that case I'll just do blocks
10:05:22do.append append and I'll append the
10:05:24text here where I pass in no output and
10:05:29the style is
10:05:33muted.
10:05:34That's it. Now I can go ahead and run
10:05:38this particular thing. And now whenever
10:05:40we run into an error we will get to
10:05:43know. So let me just explicitly ask for
10:05:46an error. I'll just say read.ai
10:05:49agentward/config.json
10:05:51JSON file for me and then as you can see
10:05:55we get a nice error message displaying
10:05:57file not found and this was the file
10:05:59that was not found. That is it about the
10:06:01error handling as well. Now let's go
10:06:04ahead and create the next tool which is
10:06:06going to be the grip. Now again we have
10:06:08a shell tool. Why do we need grip?
10:06:10That's because many times shell does not
10:06:12have the grip inbuilt within it. For
10:06:15example, I think Linux and Mac do have
10:06:18the grip command pre-installed, but in
10:06:21Windows, you do not have GP
10:06:22pre-installed. That's why we having a GP
10:06:25tool as well. And grip and glob tools
10:06:27are going to be different from each
10:06:29other. If you don't know what GP does,
10:06:31GP is used for content searching meaning
10:06:34it will answer questions like where does
10:06:36the string or pattern appear inside
10:06:38files. So let's say I have function like
10:06:41create subprocess exec right in what
10:06:46files does this create subprocess exec
10:06:49exist that is what gre does but glob
10:06:53helps us answer the question of which
10:06:56files match a specific patterns for
10:06:59example if we have to find all of the
10:07:02files that end with ts right so it can
10:07:05just do asterisk asterisk/aststeriskt
10:07:09ts meaning what files are present in
10:07:13what folders where we have ts in the
10:07:16end. So this will return the file paths
10:07:19not the content. So if you want to find
10:07:22out what files might matter for a task
10:07:24you'll use glob. But if you want to find
10:07:26out where is a specific definition
10:07:29defined or how is a specific definition
10:07:32used or you suspect a bug is here you'll
10:07:35use gp. So in short, globe will choose
10:07:38where to look. GP will tell you what to
10:07:41care about. Generally an AI agent goes
10:07:44about doing this. It will do glob so
10:07:46that we know what files are needed, what
10:07:48files might matter. Then you do a grip
10:07:52so that you again do some sort of
10:07:54filtering of where and which files we
10:07:57want to look at because the content is
10:07:59present within those files. And then you
10:08:01read those files and then you reason and
10:08:04then maybe you write or edit or
10:08:06whatever. So this is generally the flow
10:08:09of things and after writing and edit
10:08:10maybe it will use shell or something
10:08:12like that. So yeah that is the point of
10:08:15gp and glob and this is kind of the
10:08:17workflow of an AI coding agent as well.
10:08:20That's how they generally go. So let's
10:08:22go ahead and define gp.
10:08:27In this file, we have to basically have
10:08:29the same structure as we did in other
10:08:32files as well. So, I'm just going to go
10:08:33to list directory, copy everything, and
10:08:35paste it over here. And then I'm just
10:08:37going to change the names. So, the first
10:08:39one is going to be the GP tool. The name
10:08:42of the tool is going to be called GP.
10:08:44After that, we are going to add a
10:08:46description. So, what is the GP tool
10:08:49trying to do? It is trying to search for
10:08:51a reg x pattern because whatever the gp
10:08:54is going to be, it's going to be in the
10:08:56form of a reg x pattern and we're going
10:08:58to search for that in the file content
10:09:01and it will return the matching lines
10:09:04with file paths and line numbers. All
10:09:08right. After that we're going to have
10:09:10the schema also the kind of this is
10:09:13going to be read again because gp is
10:09:15just reading. Then we have schema which
10:09:17is going to be GP parents. And then I'm
10:09:20going to create a GP parents class over
10:09:22here which is going to extend the base
10:09:24model. And then we're going to get the
10:09:26first thing as the pattern. Right? What
10:09:28is the GP pattern or the reg x pattern
10:09:31that we are trying to search for? And
10:09:34that is going to be a string. We're
10:09:35going to have a field. Then we are going
10:09:37to have a description and the
10:09:39description is going to be regular
10:09:41expression pattern or you can also say
10:09:44reg x. I think most LMS are able to
10:09:47understand that to search for. So yeah,
10:09:51then we can have the path which is going
10:09:54to be well the same thing. This time
10:09:57we're not going to have a directory
10:09:58specifically. It can be a file or a
10:10:01directory and this is the path where we
10:10:05want to search. So we can just say to
10:10:07search in and maybe again we can just
10:10:09say that the default is current
10:10:12directory. All right. After that we're
10:10:14going to have the case insensitive
10:10:18command which is also going to be a
10:10:19boolean value. So by default the case
10:10:22insensitive parameter is going to be
10:10:24false. The description of there that is
10:10:27going to be case insensitive search
10:10:31by default. The value is going to be
10:10:33false. And what does this case
10:10:35insensitive search mean? It just means
10:10:38that the search doesn't care about the
10:10:41uppercase or the lowerase patterns. The
10:10:44case does not really matter if it is set
10:10:46to true. If it is false, the case does
10:10:49matter. And yeah, that looks good. Now
10:10:53we can just take this grip parameters
10:10:55attach it to schema. Then in the execute
10:10:57also we can have the grip parameters
10:10:59call it with invocation.parameters
10:11:02and that's about it. After that we can
10:11:05go ahead and resolve the path as well.
10:11:07So we have the search path instead of
10:11:10directory path because this time as I
10:11:12mentioned it doesn't necessarily have to
10:11:14be a directory. It can be a file as
10:11:16well. So it's better to call it a search
10:11:20path. This is the path where we're going
10:11:21to search and then we can just check
10:11:23that if the search path does not exist
10:11:26then in that case we're going to return
10:11:28a tool result dot error result and then
10:11:31I can say path does not exist pass in
10:11:34the search path over here great after
10:11:37that we can maybe try to compile the reg
10:11:40x pattern you know whatever pattern is
10:11:43specified by the llm over here we will
10:11:45just try to compile it using reg x so
10:11:48that all of the error checking and
10:11:52invalid reg x's are caught and after
10:11:55that we can just use that reg x anywhere
10:11:58we want. So we just compile the reg x
10:12:00once and then we can use it anywhere. So
10:12:03to do that we'll do it in a try and
10:12:06accept block because when we call
10:12:09reocompile
10:12:11which is a function within reg x let me
10:12:13import re which is a built-in module in
10:12:16python. So whenever you call this
10:12:18function it can return an exception
10:12:20which is specifically re
10:12:23error and whenever you get such an error
10:12:25it just means that the reg x pattern
10:12:27given by the llm was incorrect. So we
10:12:30want to fix that. So in that case we'll
10:12:32have returned tool result dot error
10:12:34result and then we can just say invalid
10:12:37reg x pattern and pass in the error
10:12:40message. Now within this re.compile we
10:12:43have to pass in the pattern and the
10:12:45flags. By default the flags is set to
10:12:48zero. Meaning if there are no flags
10:12:50attached but we have to attach one flag
10:12:53which is related to ignore case. Right?
10:12:55If this case insensitive is on in that
10:12:58case we want to ignore case otherwise
10:13:01we'll have zero flags. So here let's
10:13:05just set up a variable called flags
10:13:07which is dot ignore case if the params
10:13:11dot case insensitive is true otherwise
10:13:14we'll just set it to the default flag
10:13:16and set it over here. Now before that we
10:13:19have to pass in the pattern and for the
10:13:21pattern the llm is going to give us the
10:13:24pattern. So I can just pass in parameter
10:13:26dot pattern. Now this will return a
10:13:29pattern to me and this pattern is going
10:13:31to be of the type of pattern given by
10:13:34the reg x module or the re module. Now I
10:13:38can use this pattern anywhere I want so
10:13:41that I can do multiple GP searches and
10:13:44we need this pattern multiple times
10:13:47because we're going to iterate through
10:13:49many files right almost all of the files
10:13:52that you can see within a particular
10:13:54directory or if a file is mentioned in
10:13:57just that one file. So here's the key
10:14:00point. If we have multiple files,
10:14:03meaning a directory is passed in, we
10:14:05have to iterate through all of the
10:14:06files, get it in a list, and then
10:14:08iterate over all of the elements in that
10:14:11list. If just a file is present, then we
10:14:14just have to use the file to search for
10:14:17this particular pattern. So let's get
10:14:19that coded in. So the very first thing
10:14:21we're going to check for, and I'll
10:14:23remove everything from here because I
10:14:25don't think we need it anymore. The
10:14:27first thing we're going to check for is
10:14:29so if the search path is a directory in
10:14:32that case I'll just create a helper
10:14:33function to help me all find all of the
10:14:36files within this search path. So that
10:14:39just means going over all of the folder
10:14:43iteratively
10:14:44and we'll set this to a variable. Let's
10:14:46call this files and it's going to give
10:14:48us an entire list. So yeah, otherwise
10:14:52it's going to be a file, right? And if
10:14:54it's a file, then we just have the files
10:14:56list as just one single element which is
10:14:59just the search path that was given to
10:15:01us. So I'll pass in the search path. So
10:15:04in case of a directory, we go over all
10:15:06of the files in the folder. In a case of
10:15:08a file, we just go over one particular
10:15:11file. Now let's go ahead and create this
10:15:13function of find files. So I have def
10:15:16find files then I get self then I also
10:15:19get the search path and within the
10:15:21search path or actually using the search
10:15:25path which is going to be of the type of
10:15:27path and then it is going to return a
10:15:30list of paths. Let me import path from
10:15:33pathl.
10:15:34We're going to find the files. So
10:15:38initially the files is going to be empty
10:15:40list. After that we'll go over every
10:15:43path in the search path. For that we can
10:15:46just use os.walk
10:15:48and then pass in the search path. Now
10:15:51let me import OS. And when we do walk it
10:15:54essentially returns a list of pupils and
10:15:58it yields a three pupil list meaning a
10:16:00list with a pupil in it which has three
10:16:03records. And what are the three records?
10:16:05So the first one is going to be root.
10:16:08Then we have directories and then we
10:16:10have the file names. Okay. After that we
10:16:14can go over every file name in file
10:16:17names. And if the file name starts with
10:16:21let's say a dot that means I want to
10:16:24exclude that path right? So I can just
10:16:26click on continue. After that I have to
10:16:28check if the file is not a binary file
10:16:30because our entire application doesn't
10:16:32support binary files. We cannot read
10:16:34from it. So I can just have here if not
10:16:38if binary file and let me import that
10:16:41from the utils.paths folder or the file
10:16:45because we had created this helper
10:16:46function right
10:16:49and within that we can just pass in the
10:16:51file path. Now I need to get access to
10:16:54this file path. Right now I don't have
10:16:55the file path. I'll have to create it.
10:16:57So I'll just do file path is equal to
10:17:00path and then I'll pass in the root
10:17:02folder whatever is the root folder we
10:17:05are in followed by forward slashfile
10:17:08name that gives me the entire file path
10:17:11I'm in because whatever is the root plus
10:17:13the file name that is the file path and
10:17:17we pass that in. So if it's not a binary
10:17:19file, we'll just append this file path
10:17:22to our files list. And then maybe we can
10:17:26also have some sort of truncation logic
10:17:28here because
10:17:30the files can exceed by a lot. Maybe a
10:17:33directory has thousands of files. Our
10:17:36LLM does not need that much context. So
10:17:38I'll just truncate the logic here
10:17:40itself. So I'll just have if the length
10:17:42of the files is greater than or equal to
10:17:46500 in that case I'll return files.
10:17:50Another reason we're just doing it over
10:17:51here instead of doing it somewhere over
10:17:54here where we get the entire list and
10:17:57maybe we can just call 500 on it is
10:18:00because let's say if we have billions of
10:18:02files in a folder. Seems unlikely but
10:18:05it's possible. In that case, our entire
10:18:08application might just break because a
10:18:11list cannot contain a billion files.
10:18:14That's why I'm just truncating it off
10:18:16over here to 500. Actually, I'm not sure
10:18:18about Python. I don't know how much
10:18:20Python can handle in terms of the number
10:18:23of elements, but if you're using
10:18:25something like C or C++, it definitely
10:18:29cannot handle these many files. So,
10:18:32yeah. Cool. So we return the number of
10:18:34files as soon as we hit 500.
10:18:37And if that's not it, then at the end
10:18:40after the for loop, we'll just return
10:18:42the entire files list. Cool. So that is
10:18:45it about find files. If you want, there
10:18:47is more stuff you can do here. For
10:18:49example, you might want to filter out
10:18:51all of the directories that are not
10:18:54really useful. For example, if you have
10:18:56something like VNV or if you have
10:18:59doggget or if you have node modules, you
10:19:02might not want to search in those
10:19:04particular directories. That's how you
10:19:06can ignore these folders as well. So now
10:19:09we have the files list. Now what do I
10:19:12want to do? Well, for each file, I want
10:19:14to go over the file path and read its
10:19:17text. Once I get all of the content
10:19:19within that file, I want to run the
10:19:22pattern that I extracted or compiled
10:19:24here for each one of those files. So
10:19:26let's do it very quickly. The first one
10:19:29is going to be well for file path in
10:19:32this files list. We are going to go over
10:19:35and try to read the content. So content
10:19:37is equal to file path dot read file or
10:19:41read text actually. Then we'll pass in
10:19:44the encoding which is UTF8. And then you
10:19:47know this can always lead to some sort
10:19:49of exception. So I'll just do try and
10:19:52accept here because even if we fail to
10:19:55read one file path I don't want the
10:19:57entire application crashing out. So I
10:19:59can just say that hey just in case we
10:20:01run into any sort of exception here what
10:20:04I would like to do is continue just skip
10:20:07rest of the path that we're going to do
10:20:09just move to the next file path so that
10:20:12doesn't cause us any inconvenience. Now
10:20:14I can try to get the lines which is just
10:20:18content dotsplit lines.
10:20:21And now I can go over all of the lines
10:20:26list and search for the pattern in each
10:20:29of the lines. The reason we're going
10:20:31over each line and trying to match the
10:20:33pattern with each line is because I want
10:20:36to spit out the exact line number.
10:20:38Right? That's how it will help me. So I
10:20:40can just do for I comma line in the
10:20:43lines list. In that case I'll do if
10:20:45pattern which we extracted here at the
10:20:48top if pattern dot search. Remember you
10:20:52can use pattern dot match here or
10:20:53pattern dot search. But if you pass in
10:20:55pattern domatch it will try to start
10:20:58from position equals to zero and match
10:21:00the entire line. But that's not what we
10:21:03want. We want to search for a pattern
10:21:05within the line. So your pattern doesn't
10:21:09have to start from the very beginning.
10:21:11It can be present anywhere within the
10:21:13line if we use search. But when you use
10:21:16match, your pattern needs to be at the
10:21:18very beginning or at certain position in
10:21:21that line. It has to be a fixed
10:21:23position. But with search, it can be any
10:21:25position. That's why we are using search
10:21:27here. And then I'll pass in the line.
10:21:30After that, I can just go ahead and get
10:21:35the path here. So I have relative path
10:21:40is equal to and then I'll just have file
10:21:42path dot relative to and I'll pass in
10:21:45the other path which is probably of
10:21:46invocation dot current working directory
10:21:50that gives me the relative path. The
10:21:52reason I need the relative path is
10:21:54because I'll be appending it to my list
10:21:58of output. So the output I'm expecting
10:22:01from this particular execute function is
10:22:05that we get something like this. We have
10:22:07let's say is equal to signs. Then we say
10:22:10whatever path we have py then you have
10:22:13these equals to signs and then you have
10:22:15on line number one something like async
10:22:19defex execute. So this line is returned
10:22:22to us. After that maybe line number two
10:22:25or let's say line number 30 also had
10:22:28this line async defex execute or
10:22:30something like this. And then maybe in
10:22:32another file also we had this similar
10:22:36execute function that we are searching
10:22:38for. So we again have that file path
10:22:41passed in here. Let's say part 2. py and
10:22:44then we have the line numbers again
10:22:46mentioned here. So this is how I want
10:22:47the formatting to look like. If I want
10:22:49the formatting like this, I will need to
10:22:51get the relative path, right? That's
10:22:54what I have over here. And also I have
10:22:56to append it into an output. So for
10:22:58that, I'm going to create at the top an
10:23:01output lines list. And then I can just
10:23:04do output lines dotappend. And I'll pass
10:23:07in the three equal to sign followed by
10:23:12the relative path. And then I add the
10:23:15three equal to signs again. Cool.
10:23:17something like that. Let me put the
10:23:19curly bracket in again. And that's good.
10:23:22Now remember, I only want to display
10:23:24this relative path if we are matching
10:23:28for the first time. For example, if I'm
10:23:30having line number one here, that's the
10:23:32time I want to do it. Or whenever I'm
10:23:34trying to match the first path or first
10:23:39line within a specific path in a
10:23:41specific file. So I have to maintain a
10:23:43variable that will allow me to do that.
10:23:46I'll maintain a variable called let's
10:23:48say file matches which is going to be
10:23:51false initially and now if that file
10:23:55matches is false in that case we're
10:23:58going to have relative path here then we
10:24:00have the output lines append and then
10:24:02I'll set file matches equal to true
10:24:06because now for this particular file
10:24:08that we are looping through we've got
10:24:10our first file match and after this we
10:24:13don't want to display this file path
10:24:15again then maybe we go to the next file
10:24:17in that case again file matches will be
10:24:20set to false. So we again go over this
10:24:22condition and we see that yeah it is
10:24:24false. So maybe we can append the
10:24:26relative path and then we will just try
10:24:29to add in the line number where we found
10:24:31the exact output. So I have output lines
10:24:34dot append then I'll pass in the I which
10:24:38is the line number and then I'll pass in
10:24:41the exact line that we matched. I'm not
10:24:44trying to match the exact matched
10:24:46string. I'm just trying to give the
10:24:48entire line as an context to the llm.
10:24:52Once we complete this entire for loop,
10:24:54we essentially have all of the file
10:24:56matches and everything is just formatted
10:25:00nicely within output lines. Now one last
10:25:03thing I'll do for formatting is that
10:25:04I'll just check that hey if file matches
10:25:06is still true in that case I would like
10:25:09to do output lines dot append and I'll
10:25:12pass in an empty string what am I doing
10:25:14here so let's say I had this particular
10:25:17line right I found out all of the
10:25:19matches within a particular path after
10:25:22that I'm moving to the next path I want
10:25:24to leave a line here if I want to leave
10:25:26a line this will allow me to leave a
10:25:28line we're just leaving an empty space
10:25:30but at the end we're just going to join
10:25:32every element of that list using
10:25:34backslash n. So that will help. We'll
10:25:36just get out of this entire for loop
10:25:38condition that we had here. And I'll
10:25:40just check that hey if this output line
10:25:43is not mentioned. I just realized I
10:25:46probably defined output lines within
10:25:48this for loop. We don't want to do that.
10:25:50We want output lines over here. The
10:25:53reason for it is well output lines if
10:25:56it's defined within this for loop will
10:25:58mean that output lines is reinitialized
10:26:00for every file path and that's not what
10:26:03we need. So if the output lines is not
10:26:06present at all that means we found no
10:26:08line or the grip tool basically worked
10:26:11but it did not give us any output as
10:26:14such no paths matched. So I'll just do
10:26:16return tool result dot success result
10:26:20and I'll pass in
10:26:23no matches found for pattern and then
10:26:27I'll pass in the pattern which is
10:26:29parameters dot pattern otherwise the
10:26:32output lines is present and then we just
10:26:35have to return the tool result dots
10:26:37success result. The output of this is
10:26:40just going to be back slashn.join
10:26:43and we'll just pass in the output lines.
10:26:46Then for the metadata we can pass in the
10:26:48search path. Then we can also say how
10:26:52many matches were found here. So I can
10:26:54just say matches and matches is just the
10:26:58number of matches we had. Now we're not
10:27:01keeping track of that. So I can just
10:27:04initialize a variable called matches is
10:27:06equal to zero. And every time we have a
10:27:09match I just append. So I have matches
10:27:12plus equals 1. And I'll just pass in the
10:27:15matches less than or matches count here.
10:27:18Then we can also pass in the metadata
10:27:21here for this particular path. So I'll
10:27:24have the metadata. The path is string
10:27:27search path. The matches is just zero.
10:27:30Cool. I'll just go to my built-in tools
10:27:33where I have init. py. I'll initialize
10:27:37one more tool which is grip tool. Then
10:27:40I'll pass in grip tool here as well. Let
10:27:42me import that from tools.builtin.grip.
10:27:46Awesome. And now the next thing I have
10:27:50to do is go to the TUI and update
10:27:52certain things. We all know what the
10:27:54procedure is going to look like. The
10:27:56first part is going to the preferred
10:27:58order and defining my preferred order.
10:28:01So for GP, my preferred order is going
10:28:03to be maybe path first. Then we have
10:28:06case insensitive and then the pattern.
10:28:10That seems good enough I guess. And now
10:28:13I can just scroll down where we have
10:28:15tool call complete. Then I have an LF
10:28:17condition here. And by the way for each
10:28:19one of these LF conditions also we can
10:28:22put in and success. So if you want go
10:28:25ahead we can just put in and success and
10:28:29success. And even over here for the L if
10:28:32condition we will have L if the name is
10:28:35equal to grip and we are having any form
10:28:38of success
10:28:40in that case we'll have matches
10:28:43first which we want to extract from the
10:28:45metadata. So I have metadata.get
10:28:48then I have the number of matches after
10:28:50that we have to find out the output. But
10:28:53actually now I'm just thinking that if
10:28:56you're trying to display some result by
10:28:57grip just having the matches is not
10:29:01enough. Maybe we need more data here.
10:29:03For example, how many files did we
10:29:05search and how many files there were
10:29:08with matches. So maybe let's just have
10:29:11another variable in our gre. py where we
10:29:15just check how many files we searched
10:29:18for this. So we can just do that by
10:29:21passing in the length of files because
10:29:24whatever is the length of files that
10:29:25many files were checked. So I'll have
10:29:28files searched which is going to be the
10:29:31length of the files list and then we
10:29:34have again this entire thing passed in
10:29:37in the metadata as well. Now I'll go
10:29:39back to the TUI. py file where I'm just
10:29:42going to get the files searched which is
10:29:46equal to metadata.get get and then I
10:29:49pass in the files searched. Cool. Once
10:29:54we have both of them, I can go ahead and
10:29:57check if the is instance with matches
10:30:00and it is an integer which it should be.
10:30:03If it's not, we don't care about it.
10:30:06Then we'll just append it to our summary
10:30:09similar to this particular thing that we
10:30:11did up for list directory. So I'll just
10:30:15have summary dot append and then we
10:30:19found how many matches. So matches
10:30:21matches were found. After that we just
10:30:24going to copy paste again where we pass
10:30:26in the files searched which is also an
10:30:29integer instance. And then I just say
10:30:32that hey we matched these many files or
10:30:35we searched these many files. So we have
10:30:38searched let's say 10 files and then I
10:30:41can just say if the summary list is not
10:30:44empty in that case I'll have
10:30:46blogs.append pass in the text and within
10:30:49the text I will pass in a dot just like
10:30:53we had earlier over here. So I'll just
10:30:56copy paste it again then call jaw join
10:31:00on it and pass in the summary. Then for
10:31:03the text, we're going to have the style
10:31:05of muted.
10:31:07And now I can go ahead and truncate the
10:31:10text. So I'll truncate text by having
10:31:12the output passed in. Then the
10:31:14self.config domodel name. Then I'll pass
10:31:18in the self domax block token similar to
10:31:21everything we've done before. And we
10:31:22again have output display which is equal
10:31:24to that. And now we will have blocks dot
10:31:27append. And I'll pass in the entire
10:31:29syntax thing. So we'll just paste this
10:31:33thing again. So we have output display
10:31:36the format or the lexer is going to be
10:31:38text theme is monokai and the word wrap
10:31:40is true. That should be enough. So let
10:31:42me just open up the terminal again and
10:31:45let's search for it. So I'll exit out of
10:31:47here and then let's run it again. This
10:31:51time I specifically want to call the
10:31:53grip tool. So I'll just tell it to find
10:31:56all Python files and hit enter. And as
10:32:01you can see, we get the output here, but
10:32:03the thing is entirely wrong because it
10:32:06did try to use the GP tool, which is
10:32:08good for us. The path was the current
10:32:10working directory. The pattern was also
10:32:12passed in which seems correct. But then
10:32:14we run into any some sort of error here.
10:32:17We get value error too many lines to
10:32:19unpack. And this error exists because in
10:32:21grip.py we made a syntax error. We're
10:32:24doing for i, line in lines, but lines is
10:32:27just a list. I have to call enumerate on
10:32:30that so that I can get access to that I
10:32:33index variable and now if I do that it
10:32:36should work out so I would like to exit
10:32:38out of this and retry and let's see what
10:32:42happens now and as you can see GP
10:32:44returns to us venv also it truncates
10:32:48very quickly so maybe we can do two
10:32:50things instead of having the max block
10:32:53tokens here as 240 we can increase it to
10:32:56be 2,400 or something like that. Maybe
10:33:00we can set it to be 24,000 as well.
10:33:03Maybe. I don't know. I'll just set it to
10:33:052500 for now. Whatever seems like the
10:33:08right value to you, go for it. This is
10:33:10just for displaying to the user. And the
10:33:13next thing I want to do is ignore all of
10:33:15these V and V files because those are
10:33:18totally useless for us. We're not
10:33:20getting the right results because of
10:33:22that. So, how do we ignore that? I told
10:33:24you we just have to make use of this
10:33:26directories. So I'll just have
10:33:27directories. Then I have a colon here
10:33:30where I'm just creating a copy of those
10:33:32directories. And then I say that hey for
10:33:35every directory in this directories list
10:33:37if the directory is not in one of these
10:33:41set items. So the first one is node
10:33:44modules or if it's not in let's say
10:33:47pyash which is another folder we have or
10:33:51it is not in.get get or it's not inv or
10:33:55it's not in venv in that case I want the
10:33:58directory to be present here otherwise
10:34:01just remove the directory we don't want
10:34:03to iterate over that and now let's hope
10:34:06it works out so I'll just try to exit
10:34:08out of here and let's rerun it and the
10:34:11prompt here that I'm going to put here
10:34:13is going to be different from the one I
10:34:14put in earlier because I just remembered
10:34:18that if you're trying to find for all of
10:34:19the Python files we We don't really need
10:34:22grip for that. Grip is used to find out
10:34:24the file content. Right? If a specific
10:34:27term needs to be find found out within
10:34:30file content. If you want to search for
10:34:32the file names itself, for example,
10:34:35search all Python files for me, we need
10:34:37glob for that. We don't need GB for
10:34:39that. So to test out Greb, I'm just
10:34:42going to say that hey, listen, we have a
10:34:45hello world. py file. within that hello
10:34:47world or py file, I want to edit out the
10:34:50hello world text. Or maybe I can just
10:34:53tell it to remove the hello world text
10:34:57for me from
10:35:00all the files that use it. And let's hit
10:35:04enter and see what it does. So as you
10:35:06can see, it tries to grip for the
10:35:09pattern hello world and then it found
10:35:12finds no matches for it. The reason for
10:35:14it is because hello world text doesn't
10:35:17really exist within our thing because
10:35:20it's specifically hello, world with a
10:35:23quest exclamation mark. And since I told
10:35:26it that I want to remove this particular
10:35:28text, it's only searching for this text,
10:35:30nothing else. It's following our
10:35:31instructions. So maybe we can be more
10:35:34explicit. But obviously a good agent
10:35:37should be able to handle this case
10:35:40properly. it should try some other
10:35:41alternatives before just telling me that
10:35:44hey we found no matches there are no
10:35:45files that contain this. So let's try
10:35:48hello world text for me from all files
10:35:51that contain it and let's hit enter see
10:35:54what it does. So yeah, as you can see,
10:35:58GP runs. We have hello world py on line
10:36:01zero. It prints hello world and then it
10:36:03edits it out and we don't have it
10:36:06anymore. Let me bring it back.
10:36:09And as you can see, there's one small
10:36:12issue with the grip tool. It starts with
10:36:14line zero. I want it to start with line
10:36:16number one. So I'll again go to my grip.
10:36:18py. And here when we are enumerating
10:36:21over every line I would like to specify
10:36:24the start to be one so that we start
10:36:27from one so that we can display line
10:36:30number one here instead of line number
10:36:31zero. And now we can try to exit out of
10:36:35here. I'll run the same thing again and
10:36:37then hit enter.
10:36:40So it calls the grip tool then it reads
10:36:42the file because it understands that
10:36:44yeah hello world.py has it. Let me just
10:36:47read the entire file. Then it edits it
10:36:49out and the task is now completed.
10:36:52That's perfect. So that's it about the
10:36:53grip tool. Now I'll come back to using
10:36:56the glob tool because well I want to
10:36:59execute my initial file which is or
10:37:02execute my initial prompt which was find
10:37:05all the Python related files. Globe can
10:37:07do that because glob will be searching
10:37:10for the file path names. Grip was
10:37:13searching the file content for a
10:37:15specific pattern. Glob will search the
10:37:17file path names. So let's do that. I'll
10:37:20just copy all of the content within
10:37:22grip. Paste it over here and I'll make
10:37:24certain changes here. For example, the
10:37:26class name should be glob tool. Then we
10:37:29have the name to be glob. After that we
10:37:32need a description which is glob pattern
10:37:36or let's just say find files
10:37:39matching a glob pattern. Then we can
10:37:43just say supports asterisk asterisk for
10:37:46recursive matching. Then the kind is
10:37:50going to be toolkind do read. Then the
10:37:52parameters is going to be glo
10:37:54parameters. And then I'll just create a
10:37:56glo parameters at the top. So the very
10:37:58first thing that's required here is the
10:38:00pattern. And this is not going to be a
10:38:02regular expression pattern. It's going
10:38:04to be a glob pattern to match. We're not
10:38:07searching for the content or anything.
10:38:09We're searching for the file name. Then
10:38:12maybe if you want to fuse short it for
10:38:14example you need to specify some sort of
10:38:18example here that would be good for the
10:38:19LLM to understand. You can just say
10:38:22something like asterisk
10:38:24asterisk/aststerisk.ts.
10:38:27ts for example to search for TypeScript
10:38:31related files but I'm not going to do
10:38:32any of that because I just want to see
10:38:34if my LLM understands what I mean by
10:38:38glob pattern then I'll also have path we
10:38:41don't have anything related to case
10:38:43insensitivity then I'll just say
10:38:45directory to search because we're not
10:38:48searching a file right because how can
10:38:50we search for a file we need a list of
10:38:53files to be present so we will be
10:38:55searching within a directory for the
10:38:57file names. Awesome. So we have the glob
10:39:00parameters here. Let me just pass it in
10:39:02in the execute as well. Now the first
10:39:05thing we did is created the parameters
10:39:07just like always. Then we have the
10:39:09search path which is going to be the
10:39:12resolve path. Then we just check if the
10:39:14search path exists. If it does not then
10:39:18we have directory
10:39:20does not exist. Then we pass in the
10:39:23search path. And maybe we can also have
10:39:25or not search path dot is directory
10:39:29meaning the path that we have is not a
10:39:31directory. Maybe it's a file. In that
10:39:34case also the directory does not exist.
10:39:36It's a file. So I'll just return an
10:39:38error in both of those conditions. Now I
10:39:41need to file find all of the matching
10:39:44files. And it's quite easy compared to
10:39:46all of the other tools we have done. To
10:39:48find the matching files, we can just
10:39:50make use of the glob function that path
10:39:54gives to us. So I can do it within this
10:39:57try except block. So I'll have matches
10:40:00is equal to then I do search part dot
10:40:03and then I call glob on it and that's
10:40:05it. I just have to specify the pattern
10:40:08here. So I can pass in params dot
10:40:10pattern and that's it. This glob will
10:40:13return a generator to me. I'll just
10:40:15convert it into a list as it is. And
10:40:18that's it. These are all of the matches.
10:40:20Now, maybe we want to filter out all of
10:40:23the directories here because the glob is
10:40:26only related to files. So, you just
10:40:28remove all of the directories.
10:40:31So, you have matches is equal to and we
10:40:34go over every path in this matches path
10:40:37that we have. And if this path is a
10:40:40file, then we'll just put it within this
10:40:42list. Otherwise, nope. If we run into
10:40:45any exception, it's not going to be re
10:40:47error because that's for gre. We'll run
10:40:50into any sort of exception. Let's say as
10:40:52e and we return tool result dot error
10:40:55result and then we say error searching
10:40:58and then I pass in the exception.
10:41:02After that we have all of the matches
10:41:04with us. Now we just have to display it
10:41:06out nicely. We just have to format it
10:41:07just like we had done before here. So
10:41:10let me remove this if condition. We have
10:41:13output lines which is going to be the
10:41:14display. Then I don't think matches is
10:41:18required but let's just keep it in case
10:41:20we have any UI thing to show up later
10:41:22on. Then I'll have four file path in the
10:41:26matches list and maybe I can go over 500
10:41:30or thousand of these whatever one you
10:41:33want. I'll keep it 500 for consistency
10:41:35with the grip. py but since grip also
10:41:38mentioned the line at which this
10:41:41happened maybe we can go and increase it
10:41:43up to thousand because in grip we also
10:41:46mentioned the path along with the line
10:41:50exactly how it was right that consume
10:41:52tokens as well so maybe here we can just
10:41:55increase the number of files that we
10:41:58showing because at the end of the day
10:41:59glob is just going to return all of the
10:42:01files no content of the file is going to
10:42:04be put out we'll have file art dot
10:42:07relative to because I'm trying to get
10:42:09the relative file similar to what we had
10:42:12over here for GP and then I have
10:42:15invocation dot current working directory
10:42:19so that gives me the relative path maybe
10:42:22we run into any sort of error here in
10:42:25that case I just want the file part to
10:42:27be as it is you can put this try except
10:42:29logic in gp as well for example when we
10:42:33call relative to here there can always
10:42:36be some sort of exception thrown here.
10:42:38So you can have a try except path and as
10:42:41a fallback value you just use the file
10:42:44path as it is because formatting is the
10:42:47last thing we care about. First we care
10:42:49about the functionality of the app.
10:42:52Considering we have the output as the
10:42:54relative path I'll just do output
10:42:56lines.append relative path. Maybe we can
10:42:59also convert it into a string because
10:43:01this is going to be a path, right? I'll
10:43:04just remove all of these things because
10:43:07as I mentioned, globe is very simple.
10:43:11We just append the output lines. Then we
10:43:14just concatenate the output lines. We
10:43:17pass in the metadata as path. The
10:43:19matches is not required. I guess maybe
10:43:23file searched is something we can add,
10:43:25but I'm not going to add that. So just
10:43:28the path is enough for me. And I think
10:43:30we have some sort of issue here. and
10:43:32even in grip for that matter which is
10:43:35whenever we exceeds 500 we're just
10:43:37returning the files but we have nowhere
10:43:40mentioned that hey the files is now
10:43:44getting truncated we've not said that we
10:43:46are limiting it to let's say 500 results
10:43:48or thousand results so let's add that as
10:43:51well I'll add it in glob if you want to
10:43:53do it in gp go for it should be done but
10:43:57I'm not spending too much time there
10:43:59I'll just add here that hey if the
10:44:02length of this matches and matches can
10:44:05be removed from here because I don't
10:44:07want to reinitialize it to zero. I'll
10:44:09use this matches list. If this length
10:44:12matches is greater than,000 because
10:44:15remember I've not truncated down the
10:44:17list here. I've truncated down the list
10:44:19here. So it's not stored in the matches
10:44:21variable again. So matches still can be
10:44:23greater than thousand. And if it is I'll
10:44:25just add output lines.append append and
10:44:29I'll just say dot dot dot limited to,000
10:44:34results and yeah that looks good enough
10:44:38now when we do back slashn everything
10:44:41will be on a new line that looks good
10:44:42you can do the same in grip now I'll
10:44:45just go to the tui again at the top or
10:44:48not at the top in the preferred order
10:44:51I'll pass in the globe where we have the
10:44:53preferred order the first thing is going
10:44:54to be the path and then we are going to
10:44:56have the pattern.
10:44:58So that is a preferred order. Now we can
10:45:01scroll down where we have tool call
10:45:03complete. And then
10:45:06I'll just copy paste this exact thing
10:45:08because our logic should be quite
10:45:10similar except all of these matches
10:45:12integer stuff because glob doesn't have
10:45:15support for all of that. Let me get
10:45:17matches. I don't know. I have difficulty
10:45:19deciding today what I want for my UI.
10:45:21Maybe I will put in matches because I
10:45:23already have it stored here. And I'll
10:45:26say that in the meta data we have
10:45:28matches and matches is just going to be
10:45:31the length of matches we have today
10:45:34because it's a list of path right so I
10:45:36just get out its length and these are
10:45:39the number of matches that we have for
10:45:41globe then we can remove this file
10:45:43searched we don't care about that
10:45:46because whatever files we search doesn't
10:45:48matter the output will matter here then
10:45:50we have summary if matches is an integer
10:45:53we just put that in And yeah, nothing
10:45:56really other than that matters. So maybe
10:46:00I can just do blocks do.append
10:46:03text and I'll just do the text added
10:46:06here. The style is going to be muted.
10:46:09Then I'm going to truncate the text.
10:46:12After that I'm going to append the
10:46:14blocks. So I'll just paste all of that
10:46:17over here. So we have truncate text. We
10:46:20pass in the output which we get. then
10:46:24the model name. Then we have the block
10:46:26max block tokens. Also, let's remove the
10:46:28summary list from here. And then we have
10:46:30the output display text theme and
10:46:32everything is just put in. So that's
10:46:35awesome. Now I'll just remove rest of
10:46:37the things from here that I copy pasted.
10:46:39We don't need it. And that's pretty much
10:46:41it for glob. We extract the matches.
10:46:44Then we truncate the text and display
10:46:46the output. Now let's try to run it
10:46:50again and pass in my
10:46:53prompt that I wanted to earlier find all
10:46:57Python files and that should be enough
10:46:59prompt. Let me just hit enter and it
10:47:02uses shell. The reason it uses shell
10:47:04instead of glob is because I did not
10:47:07register it in init py. So let's go
10:47:10ahead and have glob tool here. Then we
10:47:13have glob tool imported here from
10:47:16tools.builtin. built-in.glob. And that's
10:47:18pretty much it. Now, close all the save
10:47:21files. Let's exit out of here. Let's try
10:47:24to run it again. And this time, we'll
10:47:26say find all Python files for me,
10:47:29please. Let's see what it does now. So,
10:47:32it does use glob.
10:47:34And we have 232
10:47:36matches. Hello world. py main.py.
10:47:39Everything gets printed out including
10:47:41everything from venv as well. And
10:47:44finally we all of that is just truncated
10:47:46out. Maybe we increase the max blocks
10:47:49tokens to a lot. Maybe we want to reduce
10:47:52it down to maybe 500 after this. So go
10:47:55for it. But yeah, Globe is working.
10:47:57Again, what it did is that the pattern
10:48:00was asterisk asterisk/aststerisk.
10:48:03py. So it found out recursively that if
10:48:06there's any Python file present within
10:48:08any folder as well and if it's present
10:48:10in the root directory as well. So in the
10:48:12root directory it s in the root
10:48:14directory of the current working
10:48:16directory. All right. So it searched for
10:48:18main. py. It searched for hello world.
10:48:20py and then it went into every folder
10:48:23and search for any py ending files. So
10:48:27that's it. That looks good. Now the next
10:48:30tool we want to work on is web search.
10:48:33This is a bit different from all the
10:48:35other tools we've worked on till now.
10:48:37All of our editing or reading related
10:48:40stuff is done. There are four more tools
10:48:42to add which are web search, web fetch,
10:48:46memory, and to-do list. They are all
10:48:48related to helping provide better
10:48:51context. As such, all the core tools are
10:48:54really done. We need all of these core
10:48:56tools to have an agent. Now, these are
10:48:59extra tools that would really improve
10:49:01our agent. For example, web. If you have
10:49:03a documentation online and you want your
10:49:06agent to read that, web fetch and web
10:49:09search would really help with that. So
10:49:11first let's have web search py. The
10:49:14difference between web search and web
10:49:16fetch is that web search is interested
10:49:18in finding the results for us kind of
10:49:20like Google search. But web fetch is
10:49:23that you have a URL with you. You just
10:49:26call that specific URL to get all of the
10:49:29results. So that's about web search and
10:49:31web fetch. Let's go to grip again. I'll
10:49:34just copy this entire thing. Paste it
10:49:37over here. Or actually, let me do glob
10:49:39because glob is much smaller so it's
10:49:41easier to edit. And here I'm just going
10:49:44to change out everything.
10:49:46So let's quickly edit out the names. So
10:49:48it's not a glob tool anymore. We have
10:49:50web search tool. The name is also web
10:49:53search. Then the description of this is
10:49:55going to be search the web. Let me spell
10:49:59it out nicely. Search the web for
10:50:01information.
10:50:03And we can just say returns search
10:50:05results with titles, URLs, and snippets.
10:50:11Okay, that's going to be our output of
10:50:13this function. We want the title of the
10:50:16web page, the URL related to a web page,
10:50:19and whatever content was relevant to
10:50:22that search item. Also, let me also let
10:50:26me rename this to web search. Cool. Now,
10:50:29the tool kind is going to be well the
10:50:31network operation, right? because all of
10:50:33the web related operations of web
10:50:34searching is going to be network
10:50:36related. Then we have web search params
10:50:40and now I'm going to define it at the
10:50:41top. It's going to be quite simple. The
10:50:43first thing is going to be the query.
10:50:45The LLM is going to give us a query to
10:50:47execute. What [snorts] do we want to
10:50:49search for? And then we have a
10:50:51description where we are going to have a
10:50:53description. Let's say search query. We
10:50:55don't have to specify much details. The
10:50:57LLM would obviously know what a search
10:51:00query means. And then we are going to
10:51:02have max results. How many results do we
10:51:05want from this? Do we want 1, 10, 15? So
10:51:08we're going to have max results. And by
10:51:10default the value is going to be 10.
10:51:12Also it's going to be an integer.
10:51:15It's going to be 10 because by default
10:51:17you know all the search engines go for
10:51:1910 including Google. So we have 10. Then
10:51:22we have greater than equal to which is 1
10:51:24and less than equal to which is 20.
10:51:26Maybe you can set this to 200 or
10:51:29whatever you want, but I think we won't
10:51:31need more than 20 results. After that,
10:51:33we have the description. And this is
10:51:36going to be quite simply the maximum
10:51:39results to return. And we'll just say
10:51:42default is 10. Okay, that looks good.
10:51:46Now we can just take this web search
10:51:48params which is set as a schema and use
10:51:51it to create the parameters here. Now we
10:51:53can remove everything because there's no
10:51:55file related operations that we're going
10:51:57to do. We are simply just going to use a
10:52:01library called duckduck go search. And
10:52:04if you're not familiar with duck duck go
10:52:06search this is it. Doug go is
10:52:08essentially a web search engine similar
10:52:10to google and they provide a package for
10:52:13us. And actually if you notice here this
10:52:16is this package has been renamed to ddgs
10:52:19and we can use pip install ddgs instead
10:52:22of using this duck duckgo search. So if
10:52:25you go to this URL and instead of duck
10:52:27duck go search for DDGS you'll notice
10:52:30that the name has changed. It's now
10:52:32ducks distributed global search and you
10:52:35can use any search engine of your choice
10:52:37from here. You can have Bing, Brave,
10:52:39DougDuck Go, Google, Grockipedia, Moji,
10:52:42Yandex, Yahoo, Wikipedia, anything
10:52:44really you want. We're going to stick
10:52:46with Doug Go because it doesn't require
10:52:49us to pass in any sort of API key or
10:52:51anything. We don't have to put in our
10:52:53credit card information either. So
10:52:55that's all good. We're going to use this
10:52:57to fetch or search for a particular
10:53:00query and get the result similar to
10:53:02Google search. So let me just install
10:53:05this. We'll do pip install ddgs and then
10:53:09I can just put a try except block
10:53:11because you know when you're trying to
10:53:14fetch results from an external package
10:53:16like duck duck go search over the
10:53:18network it can obviously return in some
10:53:20sort of exception. So if we run into any
10:53:23error we'll just say tool result dot
10:53:25error result and I'll pass in search
10:53:29failed and I'll pass in the error
10:53:31message as well. After that, I'll have
10:53:34the duck duck go search used. And if you
10:53:36notice in the documentation, they've
10:53:38mentioned how to use it. So here you
10:53:40just have to do ddgs.ext
10:53:43and then you can search for any query
10:53:44and you'll get the result. And the
10:53:46format will be title, href and body. So
10:53:50title is the title of the article, href
10:53:53is the URL and body is the snippet that
10:53:56we are searching for. So let's use this
10:53:58only. So I'll just copy this entire
10:54:01line, paste it in over here and I'll
10:54:04import DDGS. Also I have to remove all
10:54:07of these things. So I'll remove OS path
10:54:09that's not required. We'll also remove
10:54:12all the path related stuff because web
10:54:14search doesn't require all of that
10:54:15information. Right? And then at the top
10:54:18we can just do from DDGS we'll import
10:54:21DDGS like that. And now I can just call
10:54:24DDGS.ext text where I pass in the query
10:54:27which is parameters dotquery. We can
10:54:30keep rest of the things similar but if
10:54:33you want you can change it for example
10:54:35you know region time limit page whatever
10:54:38you want or you can take it from the LLM
10:54:41because the LLM might know better
10:54:43because in the LLM system prompt we are
10:54:45passing in the date we are in. So if
10:54:48it's able to calculate you know if it
10:54:51wants the most recent news about
10:54:53something it can just do 7 days minus
10:54:56the date we are on and it might work. So
10:54:59that's something you can look into. But
10:55:01anyways, we have the results with us.
10:55:03Now what I like to do is just print out
10:55:07these results over here. Or actually
10:55:10let's not return the results here. I'll
10:55:12just return it with success and we'll
10:55:15just try to see it on the terminal
10:55:17because we do know what the output of
10:55:18that is going to look like. It's going
10:55:20to be a list of dictionary within which
10:55:22we have these title href and body. Let's
10:55:25extract them. So we'll have if not
10:55:27results just in case results is empty
10:55:30we'll just have return
10:55:32tool result dot success result and I'll
10:55:36say no results found for and then we can
10:55:40pass in the query as well. So we have
10:55:42params doquery just to ensure that the
10:55:45llm has the right context but I think it
10:55:47will know the right context even if you
10:55:50don't pass in the query here because llm
10:55:53is the one that provided us with the
10:55:54query right now we want to format the
10:55:57output before we can display it so we
10:55:59want the search results showing up so
10:56:02I'll have output lines is equal to and
10:56:05then I have an f string which just says
10:56:07search results for and then I pass in
10:56:11the parameters dotquery and maybe that's
10:56:15enough.
10:56:16So these were the search results for
10:56:18this specific query. After that I can go
10:56:22over every result in this results list
10:56:25and append it to this output line. So I
10:56:27have for i, result in en innumerate
10:56:32results and you know we know that the
10:56:35starting value of this enumerate is
10:56:36going to be zero. I want it to be one.
10:56:39And now I can just do lines.append. Then
10:56:42I have an f string. Also it needs to be
10:56:44output lines where I'll pass in the
10:56:46index. I'll put in a full stop so that
10:56:49we have something like number one dot
10:56:52and then we have the title then the URL
10:56:55and then the href. So if you want you
10:56:58can prefix this with title and have
10:57:02result at title. After that I'll just
10:57:06copy paste again. And then we have URL.
10:57:09And maybe we can indent this so that
10:57:12everything's nice. Also, we can remove
10:57:14this i dot. So we have something like
10:57:17this one dot and from dot we have the
10:57:19URL displaying. Maybe the llm
10:57:22understands that better. And many times
10:57:25it can be the case that the body is not
10:57:27present. In that case I'll just do if
10:57:29result dot get body. So body is not
10:57:32null. In that case I'll just do output
10:57:35lines.append append and then I append
10:57:38the body. For body I can just say that
10:57:41hey this is the relevant snippet. So we
10:57:44can just call result at body
10:57:48and finally after that I'll do output
10:57:51lines dot append and I'll append an
10:57:55empty string so that there's some space
10:57:58left between the first result and the
10:58:01second and the third third and fourth
10:58:03and so on. After that, I'll just go
10:58:06ahead and return tool result dots
10:58:08success result and I'll pass in the
10:58:11first thing which is back slash do.join.
10:58:14Then we have the lines which is just
10:58:17output lines. Then we have the meta data
10:58:20which is equal to and for the first time
10:58:23in so long we don't have to pass in the
10:58:25path because yeah that's not relevant.
10:58:28If you want you can pass it in here
10:58:30which is invocation.curren current
10:58:32working directory. But in our case, I'm
10:58:35just going to pass in the number of
10:58:37results we have. So I'll just have
10:58:39results which I can pass as the length
10:58:42of the results. And maybe we might also
10:58:46want to pass in the query through
10:58:47metadata. Obviously it can be fetched
10:58:51from the arguments as well. So either
10:58:55ways you can do it. Either pass it
10:58:57through metadata or the args will catch
10:59:00it. I'll just rely on the arguments
10:59:03and that looks pretty good to me. So
10:59:06I'll just move to the next part. But
10:59:07before that I'll just copy this metadata
10:59:10and attach it over here when we have
10:59:11another success result which is you know
10:59:14length of results. And maybe instead of
10:59:16doing length of results we can just do
10:59:18zero. Either one is fine because here we
10:59:20have captured that the results is not
10:59:22going to be present. It's going to be an
10:59:23empty list. So that's awesome. Now we
10:59:27can just add it to init. py where we
10:59:29have web search tool and then I'll just
10:59:34copy it paste it here as well then we
10:59:38have imported it now let's go to our tui
10:59:42py where we can try to search for this
10:59:46so we have an lf condition we can paste
10:59:49it here and then we have web search and
10:59:53success now from the metadata we want to
10:59:57extract the results Right. So I'll just
10:59:59have results is equal to metadata.get
11:00:02results. If results is an integer I'll
11:00:05just append it to summary. So we have
11:00:07summary as an empty list. And then we
11:00:09have summary dotappend I'll pass in
11:00:13these many results were found. So
11:00:15results results. So 10 results 20
11:00:19results whatever. Then I also want to
11:00:21get access to the query. For that we can
11:00:24just do query is equal to args.get get
11:00:27and I'll pass in the query. Then I'll
11:00:30just copy paste this again. So if query
11:00:32is found as a string in that case I'll
11:00:36just have summary.append
11:00:39and maybe we can do query just like
11:00:41that. So we don't even need a string
11:00:43also. Let's just put it at the top of
11:00:46results. That's good. After that we'll
11:00:48have blocks dotappend and I'll append
11:00:51the text where I pass in the dot which
11:00:55is going to be a separator and then call
11:00:58dot join summary. So this entire thing
11:01:00essentially
11:01:02then I can paste that in over here.
11:01:05Everything works out and we only want to
11:01:07do that obviously if summary is not an
11:01:10empty list. Then we have blocks.append.
11:01:13Great. Then we truncate the text. So we
11:01:16have the output and then we display it.
11:01:19We will still display it in a syntax but
11:01:22if you want you can style this out very
11:01:24nicely. And yeah that looks like it
11:01:26pretty simple. So I'll just try to start
11:01:29the agent and let's see what it does.
11:01:32Maybe I can tell it to find the latest
11:01:35news for me. So search the web for
11:01:38latest news and then I hit enter. So
11:01:41yeah, it does call the tool web search
11:01:43query is the latest news and then we do
11:01:47get the output as well. The output is
11:01:50well the query shows up which is latest
11:01:52news. Then we have 10 results for it.
11:01:55The first one is the title latest
11:01:57version for the truth of the truth. Then
11:01:59we have Google News, NBC News. That's
11:02:01not really relevant because it just
11:02:04points out the latest news sources.
11:02:05We're not really interested in that.
11:02:07Maybe we can tell it more specifically.
11:02:10Give me news on Avatar movie. Search web
11:02:15for me please. Maybe you know in the
11:02:18training data of the LLM there's
11:02:20something related to Avatar movie and it
11:02:22does not search the web for me. So I'll
11:02:24just explicitly say search web for me
11:02:26and then we get all of the search
11:02:28results as well. We get 10 results here.
11:02:31And that's enough for me. So I'll just
11:02:33abort it and exit out of here. So that's
11:02:35good. Now the next tool I want to work
11:02:37on is well web fetch. We search the web.
11:02:40We have this particular URL. For
11:02:42example, I would just like to go ahead
11:02:45and extract that particular URL. Also,
11:02:48the URL is just incorrect because here I
11:02:51passed in title two times. Instead of
11:02:53title, I need to pass in href. And that
11:02:56should work, I think. So, maybe we can
11:02:58try this again. I'll just copy this
11:03:01entire thing. Let's run the agent again.
11:03:04Paste it here. And then we get the
11:03:06correct URL as well. So that's awesome.
11:03:10Seems all right to me. Now all we need
11:03:12to do is give the LLM the capability to
11:03:15go to a particular URL and read that. To
11:03:18do that, we can't specifically use this
11:03:21entire engine, the DDGS part, because
11:03:24this is just for searching the results.
11:03:28It's not for going to a particular web
11:03:31page and fetching that. So to make it
11:03:33work, what we are going to do is go to
11:03:35that URL, get that data from the URL.
11:03:39For example, maybe we want to scrape
11:03:41this particular web page. So we'll just
11:03:44go to that web page. We'll try to
11:03:46extract all of the content from there.
11:03:48Basically scrape it off and return it to
11:03:51our LLM. So let's quickly create that.
11:03:54It should be quite easy. It's just an
11:03:56HTTP fetch that we want to do. So we
11:03:58have web fetch. py. Then it's going to
11:04:02be basically the same thing as web
11:04:04search in terms of the structure. So we
11:04:06have web fetch tool. Then we have web
11:04:09fetch over here. And then for the
11:04:12description we can just say fetch
11:04:14content from a URL. And it will return
11:04:18the response body
11:04:21as text. Nothing else. Very easy. Then
11:04:25we have veg fetch parameter or web fetch
11:04:28parameters. I pass that in. it will
11:04:30extend the base model. The only thing or
11:04:33actually two things that we require here
11:04:35is URL as a string because what URL do
11:04:39we want to fetch from? So this can be
11:04:44http
11:04:46col slash or https
11:04:49slash. Then we have a timeout. Let's say
11:04:52in 120 seconds or 30 seconds we don't
11:04:55get the response. What do we do then? So
11:04:59we just time out. So that's the timeout
11:05:01response by default. Let's set it to be
11:05:0330 and it should be greater than equal
11:05:06to five because in 1 second I don't
11:05:08think anything will happen and it should
11:05:09be less than equal to 120 seconds. Then
11:05:12we can just say request time out in
11:05:17seconds and by default it is set to 120.
11:05:21Now we can try to pass in all of the
11:05:24parameters here. So we have web fetch
11:05:26parameters and after that the first
11:05:28thing I would like to do is just
11:05:30validate the URL because LLMs do
11:05:33hallucinate URLs a lot as of now. So
11:05:36first let's just parse if the URL is
11:05:39accurate. We'll use URL parse method
11:05:41from URL lib.parse. We'll pass in the
11:05:44parameters do URL and that will give us
11:05:47a value here passed. If this parse
11:05:50result exists good if it doesn't then we
11:05:53have an error. if not parse do. Scheme
11:05:56or let's say parse do.ke scheme is not
11:06:00in HTTP or HTTPS despite me telling the
11:06:04LLM that hey the argument should be HTTP
11:06:06or HTTPS still a possibility that might
11:06:10not happen. So yeah, we just handle
11:06:12that. And then if it's either of these
11:06:15cases that it's not in HTTP or the
11:06:18scheme does not even exist, in that
11:06:20case, I'll just do return tool result
11:06:22dot error result and I'll pass in that
11:06:25the URL must be http
11:06:29slash or https slash. By the way, URL
11:06:34parse is not a very good indicator of
11:06:36the web page existing or not because if
11:06:39you just go into this URL parse, you'll
11:06:41just see that it passes a URL into six
11:06:43components which is scheme which is
11:06:44HTTP, HTTPS. Then you have net location,
11:06:47path, parameters, query and any
11:06:49fragments. So it's essentially kind of a
11:06:53structure parsing maybe a reg x you can
11:06:55think of it like that. So it's not a
11:06:57very hard yes or no that the URL does
11:07:01exist. But it just validates if the
11:07:03structure of URL is right or not. After
11:07:06that we'll try to make a web fetch
11:07:08request. And to do that I'm going to use
11:07:11HTTPX.
11:07:13If you're not familiar with HTTPX you
11:07:15can search it on PI. HTTPX is a fully
11:07:18featured HTTP client library for Python
11:07:213. It includes an integrated command
11:07:23line client. has support for both HTTP 1
11:07:26and HTTP2 and provides both sync and
11:07:29async APIs. We are interested in the
11:07:31async API. That's why we are using
11:07:33HTTPX.
11:07:36Now, how do we use this for our web use
11:07:38case? Well, first we'll open up an async
11:07:42client connection. We'll do that using a
11:07:44context manager. So, we'll have async
11:07:47web and then I'll have httpx. Let me
11:07:51import httpx at the top. And then we'll
11:07:54have async client.
11:07:57Cool. Now I need to pass in the timeout.
11:08:00So the timeout is equal to httpx dot
11:08:05timeout. And then I pass in the
11:08:07parameters dot timeout. Then we'll also
11:08:10set follow redirects to true. This
11:08:13essentially means that if a web page
11:08:15redirects us to a new page using 301 or
11:08:18something like 302 status code, it will
11:08:22just follow that web page. And from
11:08:24there, whatever content we get like 200
11:08:27or 2011 or maybe 500, we'll get that.
11:08:30So, we're just following the redirection
11:08:32that the web page might do. After that,
11:08:35we'll just get this as a client because
11:08:38we are opening up a context manager,
11:08:40right? we'll get access to the async
11:08:42client itself. And now I'll just pass in
11:08:45client.get
11:08:48then pass in the URL that we want to
11:08:49fetch. So we have params dot URL and
11:08:52this will be a response. Now the
11:08:54response is again cool routine. So I'll
11:08:56await it. And now I have the response.
11:09:00Now whatever response I have, I just
11:09:02want to check if the status code is 200
11:09:06or 201 or any of the acceptable status
11:09:09codes. If it's not then I just wanted to
11:09:11raise an exception. So I'll have
11:09:13response dotra for status and if there's
11:09:18any error it will raise the HTTP status
11:09:20error and we are handling all of the
11:09:23exceptions here but I would like to be
11:09:26more thorough with this approach. So
11:09:29first I can check for the httpx dot http
11:09:34status error which is the one this one
11:09:36will return in case of any exception and
11:09:39I'll just say that hey return tool
11:09:42result dot error result if this happens
11:09:45the error string is going to be http
11:09:48then I pass in the response status code
11:09:51here and I need to get access to this e
11:09:54so I'll have e here so we are telling
11:09:57the llm what status code we got because
11:09:59LLMs are really good at understanding
11:10:01what status code is and we can also
11:10:04maybe attach a reason. So we have e
11:10:06dotresponse dot reason phrase. What is
11:10:10the reasoning that we have for this? So
11:10:14that's good. Then maybe we can also
11:10:16handle the timeout error that might
11:10:18happen. So I'll have except httpx dot
11:10:23timeout exception and we'll return this
11:10:27tool result dot error result and then
11:10:29I'll say request failed and pass in the
11:10:33error message. That looks good to me.
11:10:36And if there's any other exception, I'll
11:10:38again just say request failed. So
11:10:40actually there's no need of this timeout
11:10:42exception handling as such. We can just
11:10:44remove it because in both the cases
11:10:46we're just returning the entire error
11:10:48message with the message request failed.
11:10:52And that's good enough for me. So I'll
11:10:54just try to decode the response from
11:10:57here. So I'll just remove this entire
11:10:59thing
11:11:01and then say text is equal to
11:11:05response.ext text also I can just put
11:11:08the text here just in case the race for
11:11:11status doesn't raise any exception we do
11:11:14get the text or actually let me do it
11:11:16outside because even if we run into and
11:11:18then once we have this text we can
11:11:21display directly but before that I would
11:11:23just like to truncate the output if it's
11:11:26too long and again we can just do if
11:11:28length of text is greater than 100 into
11:11:311024
11:11:33that is 100 kilobytes I'll just do text
11:11:37is equal to text and I have 100 into
11:11:401024
11:11:42plus and then I can just do back slash n
11:11:46dot dot dot truncated contents and yeah
11:11:50I can just return the success result
11:11:52after that from here we can pass in a
11:11:55lot of metadata for example what status
11:11:58code we got so let's just attach it here
11:12:01the status code is response dot status
11:12:06code then maybe you want to display the
11:12:08content length which is just going to be
11:12:11response dot dot content and I can just
11:12:14do the length of this content because
11:12:16it's going to be in the bytes format so
11:12:18I'll just say these many bytes and if
11:12:21you want you can also display the
11:12:22content type but I'm not interested in
11:12:25displaying that so yeah that's good
11:12:27enough for me I'll just go back to our
11:12:30TUI but before that let me just remove
11:12:32this import of DDGS not required it now
11:12:35I'll go to the TUI similar to web search
11:12:38we'll just copy paste our stuff so we
11:12:41have a web fetch here and if it's a
11:12:45success then what do I want to extract
11:12:47from here I want to extract the status
11:12:49code so we do that after that we have
11:12:53content length so we'll do that as well
11:12:56and then I'll just check if the status
11:12:59code is of integer in that case I'll
11:13:02pass in the status code to the summary.
11:13:05After that, the content length. If it's
11:13:07of integer, we'll just attach it. So, we
11:13:10have content length in bytes. Then you
11:13:13might also want to display the URL.
11:13:16Also, this needs to be metadata.get.
11:13:19But for the URL, we'll be using the
11:13:21argument. So, we have args.get and we
11:13:24pass in the URL.
11:13:26I'll just copy paste this again. So I
11:13:28have if URL is of the type of string in
11:13:32that case I'll just append the URL as it
11:13:35is. Then we check that hey if the
11:13:38summary is not empty in that case I'll
11:13:41do blocks dot append and I'll pass in
11:13:44the dot which we had in every single one
11:13:47of these. So I'll just copy this dot and
11:13:51actually it's already done. I don't have
11:13:53to do that. So let me just remove that
11:13:55line. So we have dot join summary then
11:13:58the style is muted of that then we have
11:14:00truncate text for the output and then we
11:14:04display the output nicely. So let's try
11:14:08to run it and see what happens. I'll
11:14:10just exit this. Also I've not registered
11:14:13the web fetch tool. Let me register it.
11:14:15I'll go to init. py and here I have web
11:14:19fetch tool. Then similar to that we have
11:14:23web fetch tool. here. Let me import it.
11:14:26That's it. Now, let's run it. So, we
11:14:30have Python main. py and I'll say, get
11:14:34me the box office collection of avatar
11:14:38by searching
11:14:41on the web. Let's see what it does. And
11:14:43we do get the web search, but it just
11:14:46give me the search results. It's not
11:14:48really looking for the answer. Maybe
11:14:50this is an opportunity to improve the
11:14:52system prompt for web search related
11:14:54tools. I'll not get into that. You can
11:14:57definitely go ahead and tweak the system
11:14:59prompt to your liking. I don't think
11:15:01there's any mention of web search
11:15:03related or web fetch related tools. You
11:15:05should add it. What I'll do is tell it
11:15:07that hey, what does this URL tell us? So
11:15:11maybe it can fetch the content from this
11:15:15URL and try to give it to us. I'll say
11:15:18get avatar 3 collection from this
11:15:22particular URL and then hit enter. And
11:15:25as you can see it does call web fetch
11:15:28tool. It passes in the URL as I listed
11:15:30out over here. But then it runs into an
11:15:33error saying sequence item zero expected
11:15:37string instance integer found. So I can
11:15:40just say that everything we have here is
11:15:42going to be of the type of string or I
11:15:45can just convert this into a string
11:15:48before I append it to summary. Both of
11:15:51them fine. Now let's try to run it again
11:15:54and directly I'll just pass in this
11:15:57particular prompt so that we call the
11:16:00exact tool system prompting and changing
11:16:03the prompt is up to you now. And this is
11:16:05the data that we get. There's these many
11:16:07bytes. This is the URL and the status
11:16:11code was 200. If you examine this web
11:16:13result, you'll notice that we have not
11:16:15tried to extract any text from here. We
11:16:18just gave the entire web page to the LLM
11:16:22because doing that is totally possible.
11:16:24There are packages that you can use for
11:16:26it. But I don't want to spend much time
11:16:29on it because the LLM is able to
11:16:32understand code really well, right? If
11:16:34it's able to understand code, this is an
11:16:36HTML code. it is able to understand it
11:16:38and it will give us the output. That's
11:16:41the logic behind giving the entire web
11:16:43page. However, it would be more
11:16:45efficient if you just give it the entire
11:16:47text by scraping it. I just don't want
11:16:49to get into it. You can look into
11:16:51dependencies or you can just try to
11:16:54extract data from the HTML code
11:16:57yourself. So, I'll just close this. And
11:17:00now I'll move on to the next tool. And
11:17:02the next tool we're going to work on is
11:17:04to-do. py. Now, what is this to-do list?
11:17:08Why do we need it? Well, whenever the
11:17:10agent is working on something complex,
11:17:12it might want to make a plan, a plan of
11:17:15what it needs to do next and what it
11:17:18needs to do after that. So, let's say I
11:17:20tell it to create a Netflix clone for me
11:17:23using HTML, CSS, and JavaScript. So, it
11:17:26will just create a plan of how it wants
11:17:28to go about it. First, it will go ahead
11:17:30and create the HTML file for me. After
11:17:34that it will create a CSS file for me
11:17:37then a JS file for me and then maybe it
11:17:40will create tests to test out this HTML
11:17:43CSS and JS application. So those are
11:17:46four steps. It would be good for an LLM
11:17:49to just make a plan before that so that
11:17:51the user knows what's going on and agent
11:17:54is kept on track of what it needs to do
11:17:56because if there's a to-do and with each
11:18:01prompt that we give it or each turn that
11:18:03happens the LLM will be able to
11:18:06reiterate what it wants to do that will
11:18:09help it to better remember what it needs
11:18:12to do for example let's say I give in
11:18:14the prompt now if the LLM just goes goes
11:18:17and creates these four files, it might
11:18:20create it for HTML, CSS, and JavaScript
11:18:23because the context length fits,
11:18:25everything is good. But if it turns into
11:18:28something complex, it might forget that
11:18:31yeah, HTML was done, CSS was done, and
11:18:34maybe it just starts writing the test
11:18:36for it. It doesn't go to the JavaScript
11:18:38part. We need to reinforce in its mind
11:18:41what it needs to do. It's just a clear
11:18:44separation. That's why we'll first tell
11:18:46the LLM that hey create this to-do list
11:18:49through our system prompt. Then it will
11:18:51create this fourstep to-do list for us.
11:18:54Then it starts working on the first
11:18:56item. When it's done it will just say
11:18:59that hey I worked on this first item.
11:19:00It's done now. Then we have the second
11:19:02item. I'll work on the second item now.
11:19:05Then when it's done it will take it
11:19:06again. Then it goes to the third item.
11:19:09Then it takes tick marks the third item.
11:19:11and then it goes to the fourth item and
11:19:13then ticks it up and then the entire
11:19:17request is fulfilled and there are no
11:19:19more tool calls the entire agentic loop
11:19:21will break. This is a more systematic
11:19:23way to go about things and even for the
11:19:25LLM to remember also if it doesn't break
11:19:29down the request in four steps it might
11:19:32try to do that in one single step. So in
11:19:35one single message we say create Netflix
11:19:37clone using HTML, CSS and JavaScript. It
11:19:40will just go ahead and create those four
11:19:41files in just one turn you know we'll
11:19:45have one message from the llm saying
11:19:48there's four tool calls you need to make
11:19:50one write file which writes to index
11:19:53html one write file tool call to styles
11:19:56docs one write call to script.js. That's
11:20:00not good because the LLM if it tries to
11:20:03do everything in just one message or one
11:20:06turn will not do as good a job as it can
11:20:10if it was broken down into four turns
11:20:13because in the first turn it will solely
11:20:15focus on HTML. So it will have a better
11:20:17representation of the syntax. It will
11:20:20have more clearer structure. Same for
11:20:23the CSS where it will focus more on the
11:20:26design. Same for the JavaScript where it
11:20:29will focus more on the interaction. So
11:20:32the output is just going to be better.
11:20:34Obviously, it's going to consume more
11:20:36tokens, but the result will be
11:20:37dramatically better. You can try this
11:20:40experiment on your own as well. So in
11:20:42one message, you can try telling the LLM
11:20:45that hey, please create Netflix clone
11:20:47for me using HTML, CSS, and JavaScript.
11:20:49And then you can try telling the LLM
11:20:52that hey first create Netflix clone
11:20:54using HTML CSS and JavaScript but first
11:20:57create index.html
11:20:59then create styles dot CSS then
11:21:02script.js and the output will be much
11:21:05better. Obviously the problem with this
11:21:08part is that you have to type in four
11:21:10messages but our agent will just
11:21:12automatically infer and convert it into.
11:21:16Having understood why we need plans,
11:21:18let's go to our to-do. It's going to be
11:21:20having the same structure as rest of the
11:21:22tools. So, let me just go to web search
11:21:26or web fetch, any of them. I'll just
11:21:28copy and paste it in over here. Now,
11:21:30let's edit all of these text. Let's
11:21:34rename the class. So, the class is
11:21:36called to-dos tool and we are going to
11:21:38call this to-dos as well. Then the
11:21:40description is going to be manage a task
11:21:43list for the current session. And then
11:21:46we can also say use this to track
11:21:50progress on multi-step tasks. So you
11:21:54know we're clearly defining when to use
11:21:56them and when to avoid it because the
11:21:58user's request can be very simple and in
11:22:00that case we don't want to overengineer
11:22:02something for them. It can be a simple
11:22:04line change. So yeah don't use the to-do
11:22:07list. Then after that, we're going to
11:22:09have the to-do parameters. So we'll have
11:22:11to-dos params. And then we can also
11:22:14remove DDGS. We're not going to use it.
11:22:17And now the first thing we're going to
11:22:19have is an action. The action is going
11:22:21to be of the type of string. And what
11:22:22this action does is lets us know if the
11:22:25action is going to be an add to-do or
11:22:28it's going to be a complete to-do or is
11:22:30it going to be list all the to-dos or
11:22:32clear of all of the to-dos. So we can
11:22:35just say action and then say it can
11:22:38either be add or it can be complete or
11:22:42it can be list or it can be clear.
11:22:47Cool.
11:22:49So adding a to-do just adds one element
11:22:52to the to-do. Complete means we've
11:22:54ticked it off. Then we have list. So it
11:22:57lists all of the things in our to-do.
11:22:59And then we have clearing of the to-do.
11:23:01Now, there's only going to be one to-do
11:23:03per session. So, that's important to
11:23:07keep in mind because if there are two or
11:23:09three to-dos in just one session, the
11:23:11agent will obviously get confused. The
11:23:13agent thinks quite linearly. It doesn't
11:23:15think like a graph. So, it doesn't make
11:23:17sense to have multiple to-dos. The other
11:23:19thing is going to be the ID. So, we'll
11:23:21get a string or a null value. And by
11:23:24default, it's going to be null. And then
11:23:26we have a description where we say to-do
11:23:29id for completing you know so whenever
11:23:33you try to complete a to-do you have to
11:23:35let it know that what row are you trying
11:23:38to check off for example if I go back to
11:23:41excali draw I told you that we're going
11:23:44to have to-dos with four steps right so
11:23:46for the first one maybe it completed it
11:23:49and it wants to tick it off how is it
11:23:51going to tick this off for the first
11:23:53element so it needs to specy speify some
11:23:56sort of ID so that it can takeick it off
11:23:58here. That's why we're using the ID.
11:24:01After that, we have the content because
11:24:02what is the content of the to-do? And
11:24:05then we can string or null have it. And
11:24:08then we have a null value by default.
11:24:10And we say description saying to-do
11:24:13content for add. If it's trying to
11:24:15complete an action, we don't need the
11:24:18content. But if we are adding something,
11:24:21we do need the string value. or even if
11:24:23you're listing, we don't need the
11:24:25content to be present here. Now, we can
11:24:27take this to-dos parameters and add it
11:24:30as the schema. Now, the description is
11:24:34checked off. The kind of this is going
11:24:36to me memory. It's the first tool that
11:24:39is memory. Other than that, we're going
11:24:40to have another tool which is memory,
11:24:43which saves users preferences or details
11:24:46about the user. But this is also a
11:24:48memory, right? We're trying to remember
11:24:50some sort of to-do list. And yeah,
11:24:53that's pretty much it. Now, I can just
11:24:55create a parameters here, which is to-do
11:24:57parameters.
11:24:59And then there's another thing we
11:25:02require here because think about it.
11:25:03What is a to-do list? How are we going
11:25:05to store it? Well, I told you that
11:25:08there's going to be an ID. So,
11:25:10essentially, there's going to be a
11:25:11mapping between ID and the content. The
11:25:13ID is going to be the to-do ID that the
11:25:15LLM specifies and the value is going to
11:25:18be whatever content the to-do gives us.
11:25:21So for that we have to maintain some
11:25:23kind of dictionary and for that
11:25:25particular reason we're going to have an
11:25:27init function as well. So we'll have
11:25:29something like initier. And since we are
11:25:32in a class that extends from this base
11:25:35class, we'll have to pass in config
11:25:37which is of the type of config. Let me
11:25:39import it from config.config. And
11:25:41obviously this init.
11:25:44So we have given the config to our base
11:25:48class. And now we just have to maintain
11:25:50a dictionary of to-dos. So we'll have
11:25:53self dot todos dictionary. Obviously a
11:25:57type of string, string. The string.
11:25:59First string is the ID. The second
11:26:00string is going to be the content. And
11:26:03then we have an empty dictionary.
11:26:06Cool. After that, we just have to
11:26:09determine the action that's happening
11:26:10here. And based on that action, we're
11:26:13going to update this to-do dictionary.
11:26:16So, let me just remove off everything
11:26:18other than the tool result dot success
11:26:21result, which we will probably take
11:26:23reference from or actually we might not.
11:26:26Let me just remove that as well. Now, we
11:26:28can just have if parameters do action is
11:26:31equal to add. That is the first possible
11:26:34value add. If you want to make this
11:26:37concrete, you can also lower this and
11:26:40check if it's equal to add. If that is
11:26:43the case, then first we'll have to check
11:26:46if the ID is present or not. If that is
11:26:48the action, first we'll have to check if
11:26:50the content is present or not. So we'll
11:26:52check if not parameters dot content.
11:26:55That means the content is not specified.
11:26:57We'll just return the tool result dot
11:26:59error result saying that the content is
11:27:02required for add action.
11:27:06You can obviously make this more
11:27:07detailed and descriptive, but I'll just
11:27:10say for add action and then maybe we can
11:27:12also content make it say content like
11:27:15this because content is a special value.
11:27:18If the content is present in that case,
11:27:21I just have to do self do.todos and the
11:27:25ID needs to be equal to params.content.
11:27:29But we might not have the ID over here.
11:27:31ID is only required when we're trying to
11:27:33complete an action. Whenever we have add
11:27:37only content and action will be present.
11:27:39When we have complete action and ID will
11:27:42be present. Whenever we have list, we
11:27:44will only have action present. Why?
11:27:47Because the LLM might not give us a
11:27:51unique ID every single time. It might
11:27:54just reiterate the same ID again and
11:27:56again for AD. We don't want to take that
11:27:58risk when we can generate our own ID
11:28:00ourselves. So we'll say that to-do id is
11:28:04equal to string and then I'll create it
11:28:06using uyu ID. So I'll just create UU ID
11:28:09dot UU ID 4 and let me just import UU ID
11:28:14and maybe I can truncate it down to
11:28:16eight characters. That should be enough
11:28:18for me. The entire to-do ID string is
11:28:20not required. Only each row of my to-do
11:28:24list needs to be unique. So eight
11:28:26characters or even four characters will
11:28:28just do the task. So I'll set this to-do
11:28:31ID as the key and then params.content as
11:28:33the value and then we'll return tool
11:28:36result dot success result and then we
11:28:39can pass in added to-do and then maybe I
11:28:42can give it the to-do ID. So I'll give
11:28:45it in the same format as we store it
11:28:47kind of in a mapping value. So we have
11:28:50to do id here which is going to be equal
11:28:53to parameters dot content right. So this
11:28:57is important context for the LLM because
11:28:59this to-do ID is created by us. So the
11:29:03LLM obviously needs to know that this
11:29:06to-do ID what this to-do ID is because
11:29:09in the future it will have to update
11:29:12this to-do and for that it will require
11:29:14the ID. So we're just giving that
11:29:16context to the LLM. Now another action
11:29:18can be complete. So we have
11:29:20params.action.
11:29:22Is equal to complete. In that case we'll
11:29:25just check again if the params ID is
11:29:27specified or not. So I can just copy
11:29:30paste similar thing. So here we have if
11:29:35the params do ID is not specified in
11:29:38that case we'll return tool result dot
11:29:40error result and then we have ID
11:29:43required for complete action. And maybe
11:29:46we can format this similarly. So we have
11:29:50ID like this. Cool. After that we check
11:29:54if the ID that was specified is even
11:29:57present in this to-dos dictionary. If it
11:30:00is not we again have to return the
11:30:01error. So if this params do ID is in
11:30:05this self.todos dictionary. Well if it
11:30:08is not in to-dos dictionary then we want
11:30:10to return this error result because if
11:30:12it is that's valid for us. we'll just
11:30:14mark it as complete. So I'll just say
11:30:17to-do not found. That seems like a good
11:30:19enough error. And then I'll just pass in
11:30:22the parameters do ID. Obviously an
11:30:26string is also required. Now once we do
11:30:29get the valid ID in that case we'll just
11:30:31do content is equal to self.ttodos.pop
11:30:35and I'll pop off the parameters dot id.
11:30:40So whatever element was present within
11:30:43the to-dos dictionary that had the key
11:30:46as params do ID is removed. So we get
11:30:49this entire content with us. And then we
11:30:52can just return the tool result dot
11:30:54success result saying yeah we completed
11:30:58the to-do where the params do ID was the
11:31:02id and the value is the content. After
11:31:06this there's the other action of list.
11:31:09So we have params dotaction if it is
11:31:12equal to list. In that case we just have
11:31:14to list out all the to-dos. So first
11:31:16I'll just check that hey is there any
11:31:19to-dos even present because it can be
11:31:22the case that we added four to-dos. We
11:31:25completed the four to-dos and since we
11:31:27pop off all of the to-dos we don't have
11:31:30the to-dos remaining. and list will pro
11:31:33most likely always be called whenever we
11:31:36have to check if all of the two to-dos
11:31:38are done if we have no more to-do. So
11:31:41we'll just check if not self do.todos
11:31:43to-dos is remaining. In that case, we'll
11:31:46have return tool result dots success
11:31:48result and then we can pass in no to-dos
11:31:52are left, right? Because it is a
11:31:54success. Everything was great. But if we
11:31:58do have to-dos, then we have to display
11:32:00it out. So, we'll have to-dos like that
11:32:03within a list. Let me just remove the
11:32:05space. Probably not required because
11:32:06everything is going to be on a new line.
11:32:08And then we'll have for to-do id content
11:32:11in self do.todos
11:32:14dot items. And I'll just append the
11:32:17to-do ID with the content every single
11:32:19time. So I have lines.append. Let me
11:32:21just call append here. And maybe I can
11:32:24format this nicely. So I'll leave some
11:32:26indentation here. And then I'll have to
11:32:28do ID. Obviously it's going to be an
11:32:30string here. And then we can pass in the
11:32:34content or just the content like that.
11:32:37And then we can return tool result dot
11:32:40success result again. And this time
11:32:43we're just going to do back slash end
11:32:45dot join. And I'll pass in all of the
11:32:48lines. So that is it about the list. Now
11:32:51the last action that can be performed
11:32:53here is going to be clearing of the
11:32:56entire list. So in that case I'll just
11:32:58do self dot todos dotcle and we clear
11:33:02off everything. That's it. And I'll just
11:33:05return the tool result dot success count
11:33:09or success result. And I'll say cleared
11:33:12all to-dos. Or maybe instead of saying
11:33:14all, I can be more specific and say how
11:33:17many to-dos did I even remove. So I can
11:33:20just count the length of the to-dos
11:33:23before clearing. Right? So we clear it
11:33:25off, but before it we have a certain
11:33:27count and then we just have cleared
11:33:30let's say five to-dos or seven to-dos or
11:33:32whatever. Now if it's none of these
11:33:34actions the LLM hallucinated because of
11:33:37that we have the wrong thing. So I'll
11:33:40just return an error result. So there we
11:33:43go. And now in the error result I can
11:33:45say very simply unknown action and I'll
11:33:50pass in the parameters dotaction. That's
11:33:52it. So yeah that is it about this to-dos
11:33:55tool. Let me just register it using this
11:33:58get all built-in tools. So I have todos
11:34:02tool. I'll import it from tools.builtin
11:34:05and I'll add it to this all list as
11:34:07well. Now we just have to go to the UI
11:34:10and you can make the UI very complex and
11:34:12I do have a complex UI with me but I'm
11:34:15not going to write that down because
11:34:17it's going to take a lot of time and you
11:34:19might know how to go about doing that UI
11:34:21because remember it's a to-do list. It's
11:34:24essentially a table that we're getting.
11:34:26We have status. you know whatever to-dos
11:34:29are present here it's going to have some
11:34:30sort of status attached to it then you
11:34:33have the ID and then the description of
11:34:36the task it was going to perform so
11:34:38based on the status if it's completed or
11:34:42it's it's a work in progress or it's
11:34:44still pending based on that you can show
11:34:47the different styles of the to-do table
11:34:50and then obviously create a table out of
11:34:52it we have already created a table you
11:34:54can take a reference from that But I'm
11:34:57not creating that complex UI. I'm just
11:35:00going for a very simple one because I
11:35:02want to make this tutorial a bit short.
11:35:04So I'll just do TUI. py. And here
11:35:07similar to this web fetch. I'm just
11:35:10going to do a case of to-dos. And this
11:35:14is going to be very simple. We don't
11:35:15have any summary. We don't have
11:35:17anything. We just have an output. And
11:35:20the output is just going to be displayed
11:35:22within a syntax. As I said, go ahead,
11:35:25create a table. It will look much
11:35:27better, but I'm just going forward with
11:35:30this one. As simple as things can get.
11:35:33So, I'll just save this. Let me remove a
11:35:35space. And now I'll just open up the
11:35:37terminal, create the agent, and I can
11:35:41give it some sort of complex task. And
11:35:43if it doesn't use the to-do list, then
11:35:46we can update the system prompt to be
11:35:48more aggressive in a way so that it
11:35:50always uses a to-do list for complex
11:35:53tasks. Maybe even a two-step process, it
11:35:55will start using a to-do list. But I'll
11:35:57just say create WhatsApp UI clone using
11:36:02HTML, CSS, and JavaScript.
11:36:05Ensure you plan it out nicely and get
11:36:08this working. Let's see what it does. As
11:36:11you can see the to-dos is are called. So
11:36:14the very first to-do is a content where
11:36:17we have one line added and the action is
11:36:19add. It adds the to-do with this ID plan
11:36:22the WhatsApp UI clone structure. Then it
11:36:25adds another one create the HTML
11:36:27structure for it the UI. Then it calls
11:36:30add CSS styling for the UI components.
11:36:33And then it says implement JavaScript
11:36:35functionality for chat features. And
11:36:37then it will test and verify the UI
11:36:39clone. So these many actions were taken
11:36:41together because the LLM just called all
11:36:45of them together. So if you want to
11:36:47avoid this, you can create another
11:36:49action that will be add all. So it can
11:36:52just take in a list of to-dos and then
11:36:54it can add the to-do to a list directly.
11:36:57So instead of having four tool calls,
11:36:59you just have one tool call with action
11:37:01add all. Also you can update this UI to
11:37:04look much better. For example, you can
11:37:06maintain the to-do dictionary here and
11:37:08then whatever is the state of the to-do
11:37:11dictionary, you just keep on updating
11:37:12that. So let's say we got add action
11:37:16over here, we just add the to-do to our
11:37:20dictionary here and display it out
11:37:22nicely. So instead of displaying one
11:37:24to-do add here, one to-do add year, you
11:37:26just display both of the to-dos
11:37:28together. So you maintain kind of a
11:37:30persistent state here. Okay. So yeah,
11:37:33our assistant is now planning the
11:37:35WhatsApp UI clone. Then it goes ahead
11:37:37and creates a write file. Then it
11:37:40completes the to-do because it created
11:37:42the HTML structure. Then it creates a
11:37:45CSS file. Then it again calls completed
11:37:48to-do. And then it goes on and on. So
11:37:50the UI part of this looks quite
11:37:52horrible. But the LLM does work very
11:37:55nicely. And maybe if you want to see the
11:37:58output of this, we can open up the
11:38:00index.html HTML file and this is the
11:38:03WhatsApp clone it comes up with not
11:38:05totally bad there is some structure to
11:38:08it but obviously a bunch of bugs that
11:38:10can be fixed if you change the internal
11:38:13model I'm sure we'll get a better output
11:38:15as well for example if you use claw or
11:38:17something like that you'll get a much
11:38:19better output your agent is really very
11:38:23good if your LLM is very good so yeah
11:38:26that was about to-dos as I mentioned
11:38:29again if you want want to maintain some
11:38:31kind of persistent state on the UI side
11:38:33and display it. For example, you keep
11:38:36track of all of the to-dos and you also
11:38:38keep track of let's say something like
11:38:41this where if the previous tool call was
11:38:44a to-do call and the next to-do call is
11:38:47also a to-do call, then you just merge
11:38:49both of them together. That's also
11:38:51possible to be displayed in the UI. So,
11:38:54you can do that as well. So, up to you.
11:38:57There's lots of opportunities in terms
11:38:59of UI development, in terms of any
11:39:03changes to the to-do list that you want
11:39:04to do, but I'll just keep this and we
11:39:06can move forward. So, the next thing I'm
11:39:09interested in and the last built-in tool
11:39:12is the memory. py file. So, this will
11:39:17help us remember users preferences. And
11:39:20we're actually going to store this in
11:39:22the file system. So any user related
11:39:25memory for example the user might say my
11:39:27name is Ran I want to remember that so
11:39:30I'll store it in an appendon log behind
11:39:33the scenes in the user's file system and
11:39:35whenever the application loads up I'll
11:39:38load the memory up from that file system
11:39:41and then append it to my system log or
11:39:44an alternate way of going about it is
11:39:46you tell in the system prompt that in
11:39:49case the user mentions something about
11:39:51their preference and you don't know
11:39:53about it. Then you can just check the
11:39:56memory tool and try to fetch the
11:39:58relevant details about it. Two possible
11:40:00ways. But first, let's just write down
11:40:02our memory tool. And then we can see
11:40:05what we need to do about it. For the
11:40:06content of the file, we're going to go
11:40:08to the to-do, copy everything, and paste
11:40:10it in memory. And now we can have the
11:40:12memory tool instead of the to-dos tool.
11:40:15Then we can name this as memory. After
11:40:18that the description is going to be
11:40:20store and retrieve persistent memory.
11:40:25Use this to remember user preferences or
11:40:28if you want to store any other
11:40:29preferences you can do that. Something
11:40:32like important context that needs to be
11:40:34used across every single application or
11:40:38maybe some notes. So yeah all of these
11:40:41things after that the kind is still
11:40:43memory. The schema is going to be the
11:40:46memory parameters and then we're going
11:40:48to create that class. Now what is memory
11:40:50parameters going to take? It's going to
11:40:52have action as well. The action can be
11:40:54set. It's not going to be add then it
11:40:57can be get. So you set the memory. Then
11:40:59you get the memory. Then you delete some
11:41:01memory. Then you list some memory or you
11:41:04clear off all of the memory. Let me pass
11:41:07in the single inverted comma as well. So
11:41:09that's it. After that we don't need the
11:41:11ID here. We just need a key and a value.
11:41:14So the key needs to be a string or a
11:41:16null value because well you will need a
11:41:19key when you're trying to set or get a
11:41:21value or delete a value because you need
11:41:24a key if you want to get or delete a
11:41:26value. But when you're trying to set a
11:41:28value, we're going to create our own
11:41:29key. And when we're trying to list all
11:41:32the memory parameters or list all the
11:41:34memories that we stored or clear them
11:41:36off, we don't need the key. After that,
11:41:37we'll also have the value which will
11:41:39also be a string or a null value. And
11:41:41let's update the description here. So
11:41:44the value is just going to be value to
11:41:46store and this is required for set
11:41:50operation. Let me just pass this in back
11:41:53text. After that we will have the
11:41:55description for key which will be memory
11:41:58key. And let's just say this is required
11:42:01for set get
11:42:05and delete operations. You we can also
11:42:08put this in back tick. Now if you want
11:42:11you can say that the memory key is not
11:42:14required for set because when we are
11:42:17setting the key we can just create our
11:42:19own ID just like we had in to-do but it
11:42:22is also possible that we might want to
11:42:25override a value and in that case the
11:42:27key can be given off by the llm that's
11:42:30why we are taking the key. Now that we
11:42:33have all of this done we can remove the
11:42:35init function. We won't require it over
11:42:37here. We don't have to maintain any sort
11:42:39of dictionary here. The reason for it is
11:42:42our source of truth is going to be the
11:42:44file system. We're going to store actual
11:42:46user preferences or notes whatever in
11:42:49the actual user directory. So instead of
11:42:52having an init, we're just not going to
11:42:55have that. We're going to reference the
11:42:56file system every single time. Then
11:42:58we're going to create a parameters with
11:43:00memory parameters and then we will
11:43:04listen for each action. So the first
11:43:06action can be a set and if it is a set
11:43:09then I want to check if the key is
11:43:11present and if the key is not present so
11:43:14let me pass in key should be present or
11:43:17the value should be present as well. So
11:43:21let's say if not paramskey or not params
11:43:24dov valueue in that case we'll return
11:43:26the tool result dot error result. Okay.
11:43:30Now the error message for this is going
11:43:33to be key and value are required for set
11:43:37action. Of course, you can format this
11:43:39nicely by doing key like that and value
11:43:43like that as well. But after this, I'm
11:43:45not going to do it for any other error
11:43:48result. You get the point. But if we do
11:43:51have a key and a value, then we want to
11:43:53set the value. And as I mentioned, if
11:43:55you're setting a value, it should
11:43:56directly go within a file system. So we
11:44:00are going to have a persistent memory in
11:44:01the user's file system and we'll try to
11:44:05store the data over there. Now where are
11:44:07we trying to store the data? For that
11:44:09we're going to go to our config loader
11:44:11file. If you remember the config /loader
11:44:14py file where we had created get config
11:44:18directory and get system config path.
11:44:20These were related to the configuration
11:44:22paths where we stored any configuration
11:44:25files right like config.tml file.
11:44:28Similarly, there's also another one
11:44:31which we can create which is the user
11:44:33data directory. So essentially whatever
11:44:36data you want to remember about a
11:44:38certain application, you'll create it in
11:44:40that directory. For example, Chrome does
11:44:42it. If the Chrome wants to remember
11:44:44certain things about you, it will create
11:44:46that data directory and store all of the
11:44:49data like your browsing history,
11:44:51bookmarks, cookies, passwords, cached
11:44:54images, all of that. In Mac OS, it's
11:44:57generally stored in this location where
11:44:58we have root folder where we have
11:45:01library application support followed by
11:45:05whatever is the name. For example, for
11:45:06Google Chrome, it would be something
11:45:08like this. But in our case, it would be
11:45:11AI agent something like that. The key
11:45:13point is all of the applications store
11:45:16their data if they want to in the file
11:45:19system using this folder and we're going
11:45:21to make use of that. So we can have def
11:45:24get data directory and then we are going
11:45:27to return a path from here and the path
11:45:30is just going to be return path. Then we
11:45:32have user data directory which comes
11:45:35from this platform directories package.
11:45:38And now I can just pass in the app name
11:45:40which is AI agent. Now we can use this
11:45:43get data directory within memory. So
11:45:45let's try to load up the memory. For
11:45:47that I'm just going to create a helper
11:45:49function which is def load memory. It's
11:45:53going to have self and it's going to
11:45:55return a dictionary. Then I can try to
11:45:58get the data directory and I'll import
11:46:01this from config.loader
11:46:03and this is just a data directory. Now
11:46:05from this data directory I want to get
11:46:08this directory if it exists and if it
11:46:11doesn't exist then I will create it. So
11:46:14I'll just do data directory dot make
11:46:17directory parents is equal to true. So
11:46:20it will create this entire directory and
11:46:23the parents if required and if it
11:46:26already exists then it doesn't have to
11:46:28return any error. It's still fine. It
11:46:30can just create that directory or give
11:46:32us access to that directory. And now I
11:46:35can just do return data directory
11:46:37forward slash and then I'll pass in user
11:46:39memory dot JSON. So what have I done
11:46:42here? I try to get the part to the data
11:46:44directory. Then I try to create that
11:46:47directory if it doesn't already exist.
11:46:49And if it exists, it totally fine. We
11:46:51don't do anything. And then we will try
11:46:53to go to that data directory and access
11:46:57the user memory.json file. This data
11:47:00directory will also have other things
11:47:01with it. For example, when we try to do
11:47:04checkpointing and session management, we
11:47:07will store the data in this get data
11:47:09directory. whatever path this get data
11:47:12directory gives to us. So there'll be a
11:47:15user memory.json file and there will
11:47:17also be folders related to checkpoints
11:47:20and sessions and in that folders we'll
11:47:23store all of our checkpoints and
11:47:25sessions. We'll talk about that later on
11:47:27but you get the point. All of the data
11:47:29that we want to store will be stored
11:47:31here. And now I can try to load the
11:47:33memory. So I don't really have to return
11:47:36anything from here. I'll just say this
11:47:38is the memory path. Now the first thing
11:47:40I'll check is if the path exists or not.
11:47:42If the path does not exist that means
11:47:44the file is not created. If the file is
11:47:47not created then I'll just say that the
11:47:49entries that we found the memory that we
11:47:51found is just empty. So here we'll
11:47:54return a dictionary with entries and the
11:47:56entries object is just going to be an
11:47:58empty dictionary. This entries
11:48:00dictionary or this entries key is going
11:48:02to have the value where there will be
11:48:05some sort of memory. This is maybe the
11:48:07key that it gave us and the value will
11:48:10be stored over here. Let's say the user
11:48:13likes whatever whatever. So this will be
11:48:16the storage format. Then you have
11:48:17another key where you just store
11:48:20something else and then you give it a
11:48:22value. So this is what the data is going
11:48:25to look like. But if the part does not
11:48:28exist, the file does not exist. That
11:48:29means no file has been stored yet, no
11:48:32memory has been stored yet. It's just
11:48:34empty. But if the path exists then what
11:48:37I want to do is read the text of this
11:48:39entire file. And since it's going to be
11:48:41stored in a dictionary format,
11:48:44essentially a JSON format, we're going
11:48:46to return that value. So I'll just do
11:48:49try. Then we have content is equal to
11:48:52path dot read text and I'll pass in the
11:48:56encoding which is UTF8.
11:48:58After that I'll just do return
11:49:00JSON.loads
11:49:02and I'll return the content. Also let me
11:49:04just import JSON. Then we have except
11:49:08exception and if we run into any sort of
11:49:10exception then I just want to return
11:49:12this empty dictionaries object. Now we
11:49:14will call the load memory option here or
11:49:17the function here. So we have self
11:49:18do.load memory and it will return to us
11:49:21the entire memory. Now I'll store this
11:49:24in the dictionary object and then I'll
11:49:26just do memory
11:49:29at entries because that is the object,
11:49:31right? memory at entries everywhere we
11:49:34have entries key and at this key we are
11:49:37going to store our params dot key which
11:49:39we get from the llm and set it equal to
11:49:42params dot value cool so as I mentioned
11:49:46before the dictionary structure is going
11:49:48to look something like this where you
11:49:50have entries then you have another
11:49:52object and then you have some sort of
11:49:54key and then there's a value storing the
11:49:57user's preferences
11:50:00okay and Now I want to save this memory.
11:50:04So what did I do? I loaded up the memory
11:50:06from file system. I updated the memory
11:50:08and I now want to push back the updated
11:50:10memory to the file system. So I'll just
11:50:12create a helper function for that as
11:50:14well. So we have save memory. Then we
11:50:17have self memory as a dictionary and
11:50:20we're going to return nothing from here.
11:50:23And all I need to do is first get access
11:50:25to this entire user memory stuff and
11:50:28then I can just write to this particular
11:50:31file. So I'll do path.right text and
11:50:34I'll pass in the entire memory
11:50:36dictionary that we get. But remember
11:50:38this write text requires a string and
11:50:41memory is a dictionary. So what we're
11:50:43going to do is JSON.dumps.
11:50:46So we'll dump the entire thing here. And
11:50:48then maybe we can pass in an indentation
11:50:51of two just in case the user wants to
11:50:53take a look at it. It's formatted
11:50:54nicely. And ensure ask key is equal to
11:50:57false. Okay, you don't really need that
11:51:00last argument, but it basically means
11:51:03that the value should not necessarily be
11:51:06as key values. It can be something like
11:51:09a Chinese character as well or a Hindi
11:51:12character because I don't think they're
11:51:13part of ASKI. And many times the LLM
11:51:16might hallucinate. For example, I've
11:51:18seen Gemini models return Chinese
11:51:21characters while they try typing out
11:51:23English. So it's totally possible and in
11:51:26that case we want this ASKI to be set to
11:51:29false. Otherwise all of the characters
11:51:31will be escaped that are non-ASKI. So
11:51:34having done that we can just take this
11:51:36save memory and call it in over here. So
11:51:39we have self dots save memory and then
11:51:42we'll pass in the memory dictionary that
11:51:44we updated here. Then I can remove these
11:51:46two lines. And then we have tool result
11:51:48dots success result. And maybe I can
11:51:50just say set memory. And I'll pass in
11:51:54the parameters dokey. After that we have
11:51:56get and it's going to be quite similar.
11:51:59Now we have all of the helper functions
11:52:00that we might need. I think so. We'll
11:52:03just check if the parameters do. Then
11:52:06I'll just say that key is required for
11:52:09get action. Then I will try to load up
11:52:13the memory. So I'll have memory is equal
11:52:16to self do.load memory. I'll check if
11:52:20the parameters do key is not present in
11:52:22the memory either. If that is the case
11:52:25that means a memory with that key does
11:52:29not exist. So I'll just have return tool
11:52:32result dots success result memory not
11:52:36found and then I have parameters dokey.
11:52:39Otherwise we do have the key. So we'll
11:52:43just try to return that memory. So again
11:52:45we'll have tool result dots success
11:52:47result and then I'll say memory found
11:52:52then I'll pass in the parameters dot key
11:52:55and then I'll pass in the value
11:52:57associated with that which is memory at
11:53:00entries and then we pass in parameters
11:53:03dot key. So memory found this was the
11:53:06key this was the value. After that we
11:53:08might have the delete operation. So
11:53:11again for delete we do require the
11:53:12parameters dokey to be present. So if
11:53:15the parameters do key is not present
11:53:17then I'll just return the same error.
11:53:19Then we'll try to load up the memory
11:53:21again and then I'll again check in the
11:53:24memory. If parameters do key is not
11:53:25present and actually we don't have to
11:53:27check in memory because if you remember
11:53:29the memory structure looks like this
11:53:31where you have the entries key and
11:53:33within that you have the parameters
11:53:36dokey. So we'll have to do something
11:53:38like if parameters do key not in
11:53:41memory.get and then we'll call in the
11:53:44entries and if it's not present then an
11:53:46empty dictionary because entries can
11:53:49also not exist. Right? If this is the
11:53:51case then we'll return that memory not
11:53:54found and then I can copy paste this
11:53:56exact same thing again in delete as
11:53:58well. If parameters do key is not
11:54:00present, then we'll just say memory not
11:54:02found. Otherwise, we'll go ahead and
11:54:05delete the memory at entries at
11:54:09parameters dotkey. Then I'll try to save
11:54:13the memory and I'll pass in the memory
11:54:16object. And then I'll return the tool
11:54:19result dots success result where I just
11:54:22say that hey, I deleted the memory. We
11:54:24are all good. So I'll have deleted
11:54:28memory and then I have parameters dot
11:54:30key. Obviously it needs to be an string
11:54:33and that's it. Then we have the other
11:54:36action which is listing all of the
11:54:37memories out. And for that we don't
11:54:40really need a key. All we need to do is
11:54:42load up all of the memory up front. Then
11:54:45we try to get all of the entries. So we
11:54:47have memory do.get entries and if it's
11:54:51not present an empty dictionary. And
11:54:54then I just check if the entries is not
11:54:56present. In that case, we'll return a
11:54:58tool result dots success result. But the
11:55:01success is going to be that we do not
11:55:03found any we did not find any memory. So
11:55:07we'll just say no memories stored. That
11:55:10seems like a good enough line. And then
11:55:13I can try to list everything. So I'll
11:55:15have a lines list similar to how we did
11:55:18in to-do. So we have stored memories
11:55:23and then I'll go over every memory. So
11:55:26we have four key in sorted entries dot
11:55:30items also we will be getting the key
11:55:34and value pair here. So there are items
11:55:38and then based on the key we are sorting
11:55:41everything and then we just get the key
11:55:43and the value and I'll just do
11:55:46lines.append append then I have an if
11:55:48string where I leave some space then I
11:55:51pass in the key and the value will just
11:55:53be the value and now we can try to
11:55:56return the tool result dots success
11:55:58result since we have everything in the
11:56:01lines we'll just return that where we
11:56:03have a back slashn dot join and then we
11:56:07pass in the lines now the last condition
11:56:10that we might have is the clear off part
11:56:13and I'm just going to copy paste it from
11:56:15the to-do because it's going to be very
11:56:18similar. So, I'll just paste it here.
11:56:21And now, if the parameter action is
11:56:23clear, then the first thing we want to
11:56:25do is load up the memory. Then, I'll
11:56:29do a count of the length of memory dot
11:56:33get entries, how many ever entries we
11:56:36have. Then, I'll set memory at entries
11:56:39equal to an empty dictionary because
11:56:42we're clearing off everything. And then
11:56:45I'll just call
11:56:47the save memory function. And then I can
11:56:49just remove this to-dos. And I'll just
11:56:52say cleared count memory entries. And
11:56:57that's pretty much it. And if there's
11:56:59any other action, we don't know about
11:57:00it. We'll just say unknown action
11:57:02parameters.
11:57:04That looks good enough to me. Now I'll
11:57:06just go ahead and register this tool. So
11:57:08I'll have memory tool at the bottom
11:57:11which is our last built-in tool except
11:57:14for sub aents and except for sub aents
11:57:18that we'll add shortly. And then we have
11:57:21the memory tool here. Now we just have
11:57:23to go to the tui. py. It's really going
11:57:26to be very similar to to-dos. We just
11:57:28have to display the syntax. Again go
11:57:30ahead if you want to make this a good UI
11:57:34experience. But we'll have the name as
11:57:37memory and then if it is a success then
11:57:40obviously we're going to have the output
11:57:42display. We're going to display it here.
11:57:44But before that we'll try to get the
11:57:46action that was taken and we can do that
11:57:48using asks.get action and then we have
11:57:51the action. I'm not display uh
11:57:53explaining this further because we've
11:57:55already talked about this quite a bit.
11:57:57Now if the action is of the type a
11:57:59string and action is not empty or null
11:58:03or whatever we will just have summary
11:58:06dot append and then I'll append the
11:58:08action that was taken and then we'll
11:58:11also have a summary list here. After
11:58:13that we also need the key. So we can
11:58:16also try to get the key here which is
11:58:19key is equal to asks.get get key and
11:58:23then if the key is also present we are
11:58:25going to do a similar thing. So if key
11:58:28is instance of a string and the key is
11:58:31present then we'll do summary.tappend
11:58:33append key. And now if the summary
11:58:37exists then we'll just do blocks dot
11:58:40append. Then we pass in a text where we
11:58:43have this dot symbol that we have talked
11:58:46about so many times. And actually I
11:58:47think the code here is very bad because
11:58:51we're doing the same thing again and
11:58:53again. Maybe we can create some sort of
11:58:55helper function, do some abstractions
11:58:57around it and reuse them again and
11:58:59again. Then we have the style here which
11:59:01is muted just like everything we've done
11:59:04before. Also let me just try to add in
11:59:08some metadata here. For example,
11:59:10whenever we try to get something in the
11:59:14memory I would like to say if we did get
11:59:17got a value or we did not get a value.
11:59:20For example, here we having if params
11:59:23key not in memory.get entries memory not
11:59:26found. So we did not find a value,
11:59:28right? Right. So I'll just attach a
11:59:30metadata here saying that found was
11:59:34false. We did not find whatever we were
11:59:36looking for. So I'll just copy this
11:59:38metadata again. And here when we try to
11:59:41return a value, we do have the
11:59:44found because the memory was found. So
11:59:47we'll set it to true. And when we are
11:59:49trying to list as well, we can again
11:59:51attach this metadata. Found is set to
11:59:54false. And then we return the success.
11:59:56The found is set to true. So yeah,
11:59:59whenever we try to get some sort of
12:00:01value either through get action or list
12:00:03action, we'll just attach this metadata
12:00:05and display it on the screen in the TUI.
12:00:09So I'll again have found is equal to
12:00:13metadata this time not args and then we
12:00:16get the found value. Now if found is an
12:00:19instance of boolean and found exists and
12:00:23then we'll just have summary.append
12:00:25append found we'll write down the found
12:00:28string if found is set to true otherwise
12:00:30we'll have missing also here I'll have
12:00:33to update my instruction a little bit
12:00:35I'll just check if found is an instance
12:00:37of boolean and found is specifically not
12:00:40null because if it is null then we don't
12:00:43want anything related to it so if it is
12:00:45not null that means it has certain value
12:00:48to it a true or a false so if found is
12:00:52true then we write down found otherwise
12:00:55missing. I don't think this carries a
12:00:56lot of value because if found is an
12:01:00instance of boolean then found is not
12:01:02going to be null. So this is kind of
12:01:05unnecessary. I'll just remove it. And
12:01:07yeah, this looks good to me. Now I can
12:01:10try to store some preferences about me.
12:01:13So I'll just exit out of here. Open up
12:01:15our agent. Also I forgot to do one thing
12:01:18at the top. We forgot to pass in the
12:01:21preferred order. So the preferred order
12:01:23for memory and to-dos is also required.
12:01:26I forgot about that as well. There's
12:01:28going to be action then key and then
12:01:31value as it is nothing too different.
12:01:35And then for to-dos as well we're going
12:01:37to follow something similar where we
12:01:39have maybe ID first and then you have
12:01:42action and then content. I think this
12:01:45would make more sense for to-dos. So
12:01:48yeah, let me restart the agent again. So
12:01:51what can I tell to our agent? Maybe I
12:01:53can tell it I'm Rivan Ranavat. Remember
12:01:57that. So and maybe it should also
12:02:00acknowledge me by that name every single
12:02:03time. So I'll just say acknowledge me
12:02:05with that name every time I talk to you
12:02:09and let's hit enter. So as you can see
12:02:12the memory did get called. The action is
12:02:15set. The key is username and the value
12:02:17is Rean Ranavat. So it said memory
12:02:20username and then it says hello everyone
12:02:23I will remember to address you by that
12:02:25name every time we interact. How can I
12:02:26assist you today? Awesome. Also as you
12:02:29can see it did say set which was a
12:02:32action and then the key username. Then
12:02:35maybe I can try try to exit out of here.
12:02:37Then I'll again have this called and
12:02:39I'll say what is my username? Then hit
12:02:44enter and let's see what it replies us
12:02:46with. And as you can see the output here
12:02:48is your username is not provided in the
12:02:50current context. If you're referring to
12:02:52your system username, you can check it
12:02:54by running the who am I command in the
12:02:56terminal etc. So it doesn't really
12:02:58remember that username. And the reason
12:03:00it doesn't remember it is because I told
12:03:02you there are two ways to go out. At
12:03:05first we can explicitly tell the LLM
12:03:08that hey you have to call this tool of
12:03:11memory so that you can extract my name.
12:03:14The second thing is we attach it to our
12:03:17system prompt and that's what I'm going
12:03:19to do. I'll go to the system prompt here
12:03:21and attach whatever memories are coming
12:03:24in or we are storing directly in the
12:03:26system prompt. So what I'll do is take
12:03:28in the user memory from the parameter
12:03:31here and then I'll just check or by the
12:03:34way user memory can be null as well and
12:03:37by default it is null if it's not
12:03:39specified and then I can just check if
12:03:42the system or if the user memory is
12:03:45present and then I can just add the if
12:03:47condition of if the user memory is
12:03:49present then we create a function of get
12:03:52memory section where I pass in this user
12:03:54memory and now let's create a function
12:03:56to get the user memory and I'll just
12:03:59paste it in. You can again copy it from
12:04:01GitHub. But here's the thing. Remembered
12:04:03context. The following information has
12:04:04been stored from previous interactions.
12:04:07And this is the memory. Use this
12:04:08information to personalize your
12:04:10responses and maintain consistency. Of
12:04:13course, you can edit this prompt
12:04:15further, but this is all I want. Now,
12:04:17whenever this get prompt is called,
12:04:19which is in context manager, I'll pass
12:04:22in the user memory. So I'll get the user
12:04:26memory from the constructor and then I
12:04:30can just pass in the user memory to this
12:04:33get system prompt. Now whenever this
12:04:35context manager is instantiated which is
12:04:37in session py, we'll just go over there.
12:04:40And by the way, if you're wondering how
12:04:41I do that, I just press command and
12:04:44click on the class name and I get to
12:04:46this dialogue where I can just go to
12:04:48this context manager. And now I can pass
12:04:51in the user memory. And how do I get
12:04:54access to this user memory? Whatever
12:04:56function was present in this memory. py
12:04:59file which is the load memory. We can
12:05:02just copy this and create it in session.
12:05:04py as well. So we'll have diff and
12:05:08something like load memory here where we
12:05:10just get the data directory from
12:05:13config.loader
12:05:15and if the part does not exist we just
12:05:17return null because we don't have any
12:05:20memory. So we don't want to display
12:05:22anything. Also the return value here is
12:05:24going to be string or a null value. But
12:05:26if the path exists, we'll read the text.
12:05:29We'll load up the JSON file and we can
12:05:31just say data is over here. And then we
12:05:33can try to get all of the entries by
12:05:35doing data.get entries. And an empty
12:05:38dictionary is going to be the value if
12:05:40it's not present. Or actually let's not
12:05:43have an empty dictionary. We'll just say
12:05:45if the entries is null, in that case
12:05:47we're going to return null. But if the
12:05:50entries is present then we'll just have
12:05:52an output like this. So we have lines is
12:05:54equal to user preferences and nodes and
12:05:57we just try to print out all of the key
12:05:59and value of the user preferences. And
12:06:02once we have all of those lines we can
12:06:04just return a back slash n dot join and
12:06:07we pass in the lines. Also the value
12:06:10here should just be the value. So we
12:06:13have key, value and then we just pass in
12:06:16the value. Okay. If there's any sort of
12:06:19exception, I would just write to return
12:06:21null that there's nothing over here. We
12:06:23did not load any memory. Now I can just
12:06:26call this self.load memory and pass it
12:06:28as the value to this user memory. So we
12:06:31have self dot load memory like that. And
12:06:34that's it. So that is it about the user
12:06:37memory. I would like to test it. But
12:06:39before that, I'd like to update the
12:06:41system prompt a little bit more. So we
12:06:43checked for the user memory. There are
12:06:45still more things I would like to add
12:06:47here. For example, the environment
12:06:49section. So, I'll just add a comment
12:06:50here. Environment. And what this deals
12:06:53with is getting all of the environment
12:06:55related stuff. What machine are you on?
12:06:58So, for example, if you're running on
12:07:00Windows, we need to tell the LLM that
12:07:02you're running on Windows so that it can
12:07:04run shell commands that are Windows
12:07:07specific, right? We don't want to run
12:07:08anything Mac specific. Then then we can
12:07:11just scroll down and add a new function
12:07:14here to get the environment section. So
12:07:16the first thing we'll need is the date
12:07:18of today. I told you before also that
12:07:21we're going to attach the date here so
12:07:23that the LLM has more context about what
12:07:26year it's operating in. So we'll import
12:07:29the date time. Then what system are we
12:07:32running it on? What is the release
12:07:34version? So we import platform as well.
12:07:36Then we just display everything. So we
12:07:38have environment the current date is
12:07:42formatted nicely then you have the
12:07:44operating system then you have the
12:07:46working directory and you pass in the
12:07:48config.curren current working directory
12:07:50here. Remember that that is quite
12:07:52important because whatever working
12:07:55directory we are in based on that it
12:07:57will be able to make much better
12:07:58decisions. Then we also create a
12:08:01function to get shell info. The user has
12:08:04granted you access to run tools and
12:08:06service of their request. Use them when
12:08:07needed. And now we can create a function
12:08:10to get shell info. And the function
12:08:12looks something like this. We import the
12:08:15operating system. We import system. And
12:08:18then we say if system.platform is equal
12:08:20to Darwin then we just say that we want
12:08:23to get the shell environment otherwise
12:08:25the default is zsh then if it's windows
12:08:29then we have powershell and by default
12:08:32we're just doing shell. Of course
12:08:35pilance knows what platform we're
12:08:37running it on and for that reason it's
12:08:39able to gray out rest of the things but
12:08:42yeah this is required. It just gives us
12:08:45the shell value. Another thing I would
12:08:47like to add is related to tools. The
12:08:50only section that talks about tools is
12:08:53probably this one get operational
12:08:55section where we have tool usage and it
12:08:57talks about parallelism, file
12:08:59operations, file creation, task
12:09:01management, sub aents, all of that. I
12:09:03would also like to create a specific
12:09:05section that talks about the tools in
12:09:07more detail. So I'll just have another
12:09:10function parameter here which is tools
12:09:13which is going to be a list of tools and
12:09:15then I'm going to import tools from
12:09:16tools.base
12:09:18and then it can also be a null value and
12:09:20by default it is going to be a null
12:09:22value and now if this tools is present
12:09:26then I'm going to add this get tools
12:09:28guideline section. Now I can scroll down
12:09:30and create this function. So let me just
12:09:33go at the bottom and add it. So this is
12:09:36the function. Now let's try to
12:09:37understand what it does. First it takes
12:09:40in a list of tools again and let me just
12:09:43have that. It returns a string and it
12:09:46divides the tools into two types. The
12:09:48first one is regular tools and then sub
12:09:51aent tools. We are going to create sub
12:09:53agent as a tool next after we've tested
12:09:55out memory if it's working properly or
12:09:58not. But the regular tools is just going
12:10:01over every tools and if the t.tame name
12:10:04does not start with sub aent underscore
12:10:07then it's a regular tool because none of
12:10:08our tools start with sub aent underscore
12:10:12and if it's a sub aent we will see that
12:10:14we created or we give it with the name
12:10:17of sub aent whatever like sub aent
12:10:21investigator or sub aent
12:10:23reviewer then you have the guidelines
12:10:26where you have tool usage guidelines you
12:10:28have access to the following tools to
12:10:29accomplish your task and then we go over
12:10:31every tool we extract its description
12:10:34and try to attach it to the guidelines.
12:10:37Then we also do the same for sub aents.
12:10:39We just go over every tool, extract the
12:10:42description and just attach it. That's
12:10:44why the description property in the
12:10:46tools were very important. This
12:10:49basically instructs the LLM how to use
12:10:51the tool. Then you have the guidelines
12:10:54where you have the best practices for
12:10:58file operations. You use read file
12:10:59before editing to understand current
12:11:01content. We've talked about this. Use
12:11:04edit for single file surgical changes.
12:11:06Use write file for creating new files or
12:11:09complete rewrites. And whenever editing
12:11:11two or more files, always use apply
12:11:13patch instead of multiple edit tool
12:11:15calls. Apply patch is more efficient and
12:11:17allows batching multiple file operations
12:11:20in a single atomic operation. I'm going
12:11:22to remove this line because we have not
12:11:23implemented the apply patch tool.
12:11:25However, in the repository,
12:11:29you will find apply patch created as a
12:11:32file in built-in tool. So, if you want
12:11:34to take a look at that, you can. I'll be
12:11:37sure to add in some dock strings and
12:11:39comments to explain how that works, but
12:11:42it's quite straightforward. All we try
12:11:44to do there is parse the LLM content and
12:11:46just do whatever we've tried to do in
12:11:48write file and edit. So, I'll just
12:11:50remove this part. Then we have also
12:11:55since we don't have apply patch we can
12:11:57remove this line. We just say use edit
12:12:00for surgical changes and you write file
12:12:03for creating new files or complete
12:12:05rewrites. Then search and discovery is
12:12:07glob and list directory. Shell for
12:12:10running commands. Prefer readonly
12:12:12commands. Be cautious with commands.
12:12:15Then we have to-dos to track multi-step
12:12:17task. Mark tasks as completed as you
12:12:19finish them. And then we have memory to
12:12:22store important user preferences.
12:12:23Retrieve store preferences when
12:12:25relevant. Then we have the guidelines
12:12:28related to sub aent tools if they exist.
12:12:30Great. That's all about go tool
12:12:33guidelines. So our LLM should be able to
12:12:35call the tools more clearly and in a
12:12:38better manner. So now I'll just go to
12:12:40get system prompt in manager because I
12:12:42have to attach this tools list. And to
12:12:45this manager also we are going to get a
12:12:47tools which is a list of tool we import
12:12:51that from tools.base
12:12:53or it can be a null value. Then I take
12:12:55this tools list and attach it to this
12:12:59system prompt. Now whenever this context
12:13:02manager is called we just attach the
12:13:05tools list and the tools list is fetched
12:13:08from this tool registry. So I'll just
12:13:10take this tool registry put it above
12:13:12context manager and then the tools can
12:13:16be fetched using self dot tool registry
12:13:19dot get tools right because get tools
12:13:22gives us just a list of tools that looks
12:13:26good to me now let's try to run this
12:13:28entire application now and let's see how
12:13:30it behaves so I'll run it and we run
12:13:33into an error module datetime has no
12:13:36attribute now and I think we've run into
12:13:38a similar ISS issue before this happens
12:13:40because we imported datetime. What we
12:13:43need to do is from datetime import date
12:13:45time. And now we can try to run it
12:13:47again. This time it works. Maybe I can
12:13:50ask it what is the date today. Let's see
12:13:53what it replies. And it does reply with
12:13:56the current date is Monday, December 29,
12:13:592025. Awesome. That is the date today.
12:14:02Then I can ask it the next question.
12:14:05What is my name? Let's see if it uses
12:14:08that. And then it says your name is
12:14:10Rivan Danavad. Awesome. So the memory is
12:14:13being stored. Now I can just tell it
12:14:15something about me that I love clean
12:14:19code but I'm unable to do so. Can you
12:14:23remember that? And as we can see the
12:14:25memory action is set. The key is coding
12:14:28preference and the value is Rean loves
12:14:30clean code but struggles to write it.
12:14:32That's more understandable than whatever
12:14:35I wrote here. And it says, I've noted
12:14:37that you love clean code but find it
12:14:39challenging to write. I'll keep this in
12:14:40mind for future interactions. Awesome.
12:14:42So the memory feature is done.
12:14:44Obviously, memory is a very big part of
12:14:46agents. There's a lot you can do. This
12:14:49is a very simple memory system. You can
12:14:52expand it further by adding long-term
12:14:55memory, short-term memory. Because
12:14:57certain memories might be such that
12:15:00they're only required certain amounts of
12:15:02time. But other memory pieces are such
12:15:04that you always require them. And there
12:15:06are tons of memory types like short-term
12:15:09memory, long-term memory, episodic
12:15:11memory, semantic memory, all of that. So
12:15:15yeah, and you can also look into
12:15:16optimizing the system prompt based on
12:15:18that memory because let's say the user
12:15:20stores a thousand memories. You don't
12:15:23want to list out all of the memories in
12:15:24the system prompt. That's why all of
12:15:27these systems are built where you use
12:15:29rag or something else so that you only
12:15:32fetch relevant memories. But anyways,
12:15:34our time for this is done. Now the next
12:15:37feature I would like to get started with
12:15:39is sub aents. So what are sub aents? Sub
12:15:43agent is essentially the main agent
12:15:45calling another agent within it. That's
12:15:47it. So an agent right now can call let's
12:15:51say a web fetch tool or a web search
12:15:53tool. Similarly, it can also call a sub
12:15:56aent tool and that sub aent tool just
12:15:58creates another instance of the sub aent
12:16:00which is more focused. It does one task
12:16:03really well. There are also general sub
12:16:06aents that can just do any task but many
12:16:09coding tools just have specialized sub
12:16:12aents. they only do one task and do that
12:16:14really well. Now what is the motivation
12:16:16behind having sub agents? The main
12:16:18driver is the context window management.
12:16:21So when the main agent needs to do deep
12:16:23exploration like reading dozens of
12:16:25files, searching through a codebase,
12:16:27experimenting with different approaches,
12:16:29it will quickly exhaust its context
12:16:31window and lose track of the original
12:16:33task it's given. When a sub agent is
12:16:36called, it gets a fresh context window
12:16:38dedicated to a focused subtask. For
12:16:41example, let's say a user asks that,
12:16:43hey, I want to add rate limiting to our
12:16:46API endpoints and there are several API
12:16:49endpoints. Without a sub agent, the main
12:16:51agent would go and read multiple files,
12:16:54right? It would go and read server. py
12:16:56it would go and read users py, posts py,
12:17:00middleware. py, configuration if that's
12:17:03present, cache if it's present, all of
12:17:05that. So the these are tons of files
12:17:08that it needs to read and each of these
12:17:10files can be a lot of tokens right for
12:17:12example the users.py will have all the
12:17:15endpoints related to users and those
12:17:18might be let's say 10 15 20 different
12:17:20endpoints.
12:17:22So it will consume a lot of tokens for
12:17:24each of these files. So by the time it
12:17:26figures out the architecture of this
12:17:29entire application and loads in the
12:17:30particular context, it has already
12:17:33consumed tons of tokens in just
12:17:36exploration of the context and it hasn't
12:17:38helped us with the original task yet. It
12:17:41hasn't written any code yet. So the
12:17:43original task is now far away in the
12:17:45context window. That is a problem. The
12:17:48LLM might just forget what the task was
12:17:50and it just keeps on exploring with the
12:17:53sub agent. So instead what can be done
12:17:56is that the main agent is present here.
12:17:59It will just call a sub agent and its
12:18:01task is to explore this entire codebase.
12:18:04So it goes and explores the entire
12:18:06codebase. Then it gives us a summary of
12:18:08what exists in that codebase and then
12:18:11for each task we can just go ahead and
12:18:14write the file or edit file or whatever
12:18:17needs to be done. For example, when
12:18:19we're trying to add rate limiting, most
12:18:21likely the LLM does read all of the
12:18:24files that are present here. For
12:18:26example, server.py, users. py post. py
12:18:30and middleware. py. But then it notices
12:18:32that hey, we have a middleware. py, we
12:18:34have a configuration. py. I just need to
12:18:37edit two files out of this. We don't
12:18:39care about users and posts and server.
12:18:41And for that particular reason, the sub
12:18:44aent is really good because it has read
12:18:46all the files. And then it says that
12:18:48yeah I think middleware.py is the file
12:18:51you have to edit because it contains all
12:18:53of these things. Users. py posts. py are
12:18:56just API routes. You can leave them as
12:18:58it is. Server. py is just about starting
12:19:01the server. It doesn't have anything
12:19:02rate limiting that we can do. So yeah,
12:19:05the summary will help us to make our
12:19:07edits later on and the context remains
12:19:10clean because we just have the summary
12:19:12which might be let's say a thousand
12:19:14tokens and then write and edit file
12:19:16tokens instead of this massive bunch of
12:19:18tokens that came in over here. So our
12:19:20original loop of agents, so our original
12:19:23agentic loop is still pretty lean and
12:19:27that helps us to remain focused on the
12:19:29task that the user gave. That's why we
12:19:32need a sub agent. Now let's go ahead and
12:19:34create a sub agent. As I already hinted
12:19:36previously, the sub aent is going to be
12:19:38exposed as a tool. So what I can do is
12:19:41in the built-in tools itself, I can go
12:19:44ahead and add sub aent or I can also add
12:19:46it in agent folder because essentially
12:19:49sub aent is just an agent. So in both of
12:19:52those folders, but I think I'm just
12:19:54going to go ahead and create a new tool
12:19:56here. And let's call this sub aent.
12:20:00py. Now the reason I'm not putting this
12:20:02in built-in tools is because we're going
12:20:04to structure our sub aent code such that
12:20:07the user such that it will be very easy
12:20:10for you to let your users create sub
12:20:13aents on their own. By default we are
12:20:16going to have two sub aents. One is
12:20:18going to be the codebase investigator.
12:20:20It investigates the codebase and then
12:20:22you have the code reviewer. It reviews
12:20:23the code changes and provides feedback.
12:20:25But we are going to write code in such a
12:20:27way that you can just easily extend the
12:20:30sub aents capability to allow your users
12:20:33to create their own sub aents. Claude
12:20:36code added this new feature very
12:20:37recently and we're going to structure
12:20:39our code such that using the config.tml
12:20:42file that we created the user can
12:20:44specify their own sub aents and run
12:20:46them. I'll talk about that. This is not
12:20:49a feature of our application yet, but
12:20:51the code will be written in such a way
12:20:54that you'll be able to extend it. So go
12:20:56ahead add that feature on your own. It's
12:20:58really helpful if the user can just
12:21:00create whatever sub aent they want,
12:21:01right? And we have a configuration
12:21:04system in place. We can just use that.
12:21:06But first things first, let's go ahead
12:21:08and create a sub agent tool. As I
12:21:11mentioned, this sub aent tool is going
12:21:12to be a tool so that LLM can call it
12:21:16really well. Then we're going to have an
12:21:18init function. Then we get a
12:21:20configuration system because our base
12:21:23tool also has that and we just call the
12:21:26base class like that. Already looked
12:21:28into that. Then what we're going to
12:21:30receive is a sub aent definition. So to
12:21:34create a sub aent we need different
12:21:36different aspects. First we need the
12:21:38name of the sub aent because that's its
12:21:40unique identifier kind of thing. Then
12:21:43you have the description of the sub
12:21:45aent. Similar to other tools, everyone
12:21:48has their own description. Then what is
12:21:50the goal of the sub aent? Then what
12:21:52tools is a sub aent allowed to call?
12:21:55Because our main agent is allowed to
12:21:57call any tool we have registered using
12:22:00this built-in init. py, right? All of
12:22:03these tools are allowed to be called by
12:22:06the main agent. But our sub agent is
12:22:08more focused. It can just call specific
12:22:11tools. So we'll just expose specific
12:22:13tools to it to call and how many turns
12:22:16it can go for, how much time out seconds
12:22:19is available. So you know if the sub
12:22:21agent doesn't do its work in let's say 5
12:22:24minutes, we close it. If it doesn't do
12:22:26it in 10 minutes, we automatically close
12:22:28it so that the sub aent doesn't keep on
12:22:30running for an infinite amount of time.
12:22:33So those are the things we need. So what
12:22:35I'm going to do here is just create a
12:22:37definition and let's call this sub aent
12:22:40definition. And then I'm going to create
12:22:42a class here called sub agent
12:22:45definition. And then here it's going to
12:22:48be a data class first of all. So let me
12:22:50import from data classes data class. And
12:22:53now I can define all of the properties.
12:22:55So the first thing is the name. You'll
12:22:57see how this name is being used because
12:22:59remember when we were writing the system
12:23:01prompt our name for sub aents was
12:23:04something like sub aent underscore and
12:23:06whatever the name is specified over here
12:23:09because all the sub aents need to have
12:23:11sub aent underscore for us to know that
12:23:13there those are sub aent to tools. Then
12:23:16we have description which is going to be
12:23:18a string when the sub aent needs to be
12:23:20called. Then you have a goal prompt what
12:23:23the sub agent needs to do and we'll have
12:23:26a string for that. Then we have allowed
12:23:29tools which is a list string or null
12:23:33value and by default it's going to be
12:23:34null but obviously we're going to have
12:23:36multiple tools allowed. Then you have
12:23:38max turns which is going to be an
12:23:40integer. Let's say it can go for 20
12:23:43turns. Maybe you can put 30, 50,
12:23:46whatever you want. Then you have time
12:23:48out seconds which is going to be a float
12:23:51value and this can be 300 or 600. I'll
12:23:55give 600 which is 10 minutes. Now I'll
12:23:58take the sub aent definition and
12:24:00initialize that as well. So what I've
12:24:02done here is just taking a sub aent
12:24:04definition from in it. The reason we
12:24:07take a sub aent definition is so that
12:24:10whenever you have to add the feature of
12:24:12creating your own sub aent you'll be
12:24:14able to just pass in something like a
12:24:17sub aent definition class with all of
12:24:20these properties and that makes it
12:24:22really easy right in the config.tml TML
12:24:24you just say something like hey I'm
12:24:27going to have a sub agent and maybe you
12:24:29give the name as code reviewer but we
12:24:33going to have that by default. So let's
12:24:35say it's a fast API sub agent. So it
12:24:39just writes fast API code really well.
12:24:41Then you have a description and so on.
12:24:44That is creating your own custom sub
12:24:46aent. There's nothing more to it. Now we
12:24:49have to initialize all of the properties
12:24:50of a tool specifically name description
12:24:53all of that. But for the name we know
12:24:56that the name is going to be sub aent
12:24:58followed by this name. So I do need
12:25:01self.definition.name
12:25:03right? so that I can give it the name.
12:25:04That's why we're going to create at the
12:25:07rate property where I have a name and
12:25:10then I have self and I'm going to return
12:25:12a string from here and the name is now
12:25:14going to be sub agent
12:25:16self.de definitionfname
12:25:20it's only a function because it's needs
12:25:22self and that's why I prefixed it with
12:25:25an annotation of property. After that we
12:25:28require the description as well which is
12:25:30just self.definition.escription.
12:25:32So I'm just going to create a property
12:25:35description like that. And I have
12:25:37self.definition
12:25:38dot description. Great. For the schema,
12:25:41we can just go ahead and create it. We
12:25:43have class
12:25:45sub aent parameters and then we extend
12:25:49the base model. We'll import that from
12:25:51pyantic. And now let me just first
12:25:54import it from pyantic. And once we have
12:25:57imported it from pyantic, what I'm going
12:25:58to do is just have one thing that we
12:26:01need from the LLM which is the goal. We
12:26:03need the prompt, right? Nothing else is
12:26:05required. The name, we already know what
12:26:08sub aent is being called because we have
12:26:11the name given here. That is going to
12:26:13good enough for now. Let me have a
12:26:15schema here which is just going to equal
12:26:17to sub aent parameters.
12:26:20Then we are going to set is mutating to
12:26:22true. So we have a function is mutating.
12:26:26we get the parameters which is going to
12:26:27be a dictionary string, any let me
12:26:30import any from typing and I'm going to
12:26:33return out a boolean value and what it
12:26:35will do is return true nothing else
12:26:38because sub agent will mutate or you can
12:26:41also take it from the definition saying
12:26:43that hey is it going to mutate and by
12:26:45default the value can be false and the
12:26:48user can set it to true but anyways this
12:26:51is good enough now we'll just have our
12:26:53execute function and then we can look
12:26:55into how to create a sub agent. We are
12:26:58going to get a tool invocation here. Let
12:27:01me import that. Then we are going to
12:27:02return a tool result. Everything remains
12:27:05the same similar to what we did in other
12:27:07tools as well. And now we are done
12:27:11already. The first thing is getting the
12:27:13goal. The first thing now is getting the
12:27:15parameters. So we have parameters is
12:27:18equal to then I'll have sub aent
12:27:20parameters pass in the invocation
12:27:23dotparameters. As of now, our parameters
12:27:25only has one property which is the goal.
12:27:27But in the future, you can obviously
12:27:29extend your sub agent and have multiple
12:27:31more things. After that, we'll just
12:27:33check if the LLM has specified goal. If
12:27:36it hasn't, or we'll say params goal. If
12:27:39it hasn't, then we'll return tool result
12:27:42do.
12:27:43And say that no goal was specified here.
12:27:48And maybe we'll say for sub agent as
12:27:51well. Also, I forgot to do one thing. I
12:27:54have to give it a field value. We'll
12:27:56import that from Pantic. Everything is
12:27:58going to be the same except the
12:28:00description. And the description is
12:28:01going to be the specific task or goal
12:28:04for the sub agent to accomplish achieve
12:28:08whatever you want to type. Also, let me
12:28:10fix some spelling errors. Yeah, this is
12:28:12important because we saw in the system
12:28:15prompt this description was used in the
12:28:17system prompt to tell the LLM how to
12:28:20pass data to this property. Great. Now
12:28:23that we have goal, what do we want to
12:28:25do? Well, first of all, I have to create
12:28:28an entire agentic loop again. So,
12:28:31similar to my agentic loop in agent. py,
12:28:34I'm just going to go ahead and open up
12:28:36this agent. Similar to how I did it in
12:28:39the main. py, if you remember, we had
12:28:43something like with agent here. Then we
12:28:46processed each message that came in by
12:28:48doing async forevent in self.agent.run.
12:28:51I'm going to do a similar thing in sub
12:28:53agent as well because essentially it's
12:28:55just an agent calling an agent. So we'll
12:28:58have a try block here. Then we'll have
12:29:01except exception as e. Then we have some
12:29:06stuff to do here as well. For example,
12:29:08if there's any exception, we will need
12:29:10to tell the llm what the exception was
12:29:13and all of that stuff. But I'll get to
12:29:16it in just a minute or actually 2
12:29:18minutes. But first let's just try to
12:29:19complete this agent techic loop. So
12:29:21first things first I'll create an agent.
12:29:24Let me import that agent from
12:29:26agent.agent.
12:29:27After that I need to pass in the
12:29:29configuration. Right now what is the
12:29:31configuration going to be? Is it going
12:29:33to be self doconfig that we had at the
12:29:36top? Maybe it can be that config allow
12:29:39the user to pass in something like
12:29:41allowed turns and max turns. Right? We
12:29:44can take these two values and override
12:29:46it in the configuration dictionary
12:29:48because these are the two values that
12:29:50can change. Think about it. Name
12:29:53description were needed because they
12:29:55were needed for the tool. The goal
12:29:57prompt is needed so that the LLM can
12:29:59call it. But allowed tools and max turns
12:30:02are things that are related to the
12:30:04configuration system. Timeout seconds is
12:30:07just something we need to handle
12:30:08ourselves within this agentic loop
12:30:11because we just keep running and seeing
12:30:13how much time has gone out in this sub
12:30:16aentic loop. But allowed tools and max
12:30:19turns are something that we will define
12:30:21in the configuration system and we have
12:30:24access to max turns already. What we
12:30:26don't have access to is allowed tools.
12:30:29So I'll add in allowed tools here which
12:30:32is going to be a list of string or it
12:30:35can be null and we can specify a field
12:30:38here where the default value is null and
12:30:41the description is if this value is set
12:30:44only these tools will be available to
12:30:47the agent. If you don't specify the
12:30:50description that should be fine over
12:30:52here but this is what this allowed tools
12:30:55do. Now remember allowed tools is in the
12:30:58configuration that means it can be used
12:31:00for agents and sub aents both. So we
12:31:04need a way to filter out these tools in
12:31:06the tool registry. So think about it
12:31:09again. So the user through the
12:31:10configuration system tells that this sub
12:31:12agent that we have is only going to have
12:31:15read related tools for example read file
12:31:18or list directory or grip or globe.
12:31:21Those are the four tools that are
12:31:23allowed. Rest of the tools that we have
12:31:25defined in this built-in init. py need
12:31:29to be ignored. The edit rewrite file
12:31:31tools all of them need to be ignored.
12:31:34But we have not done any filtering of th
12:31:36those logic yet. So we have to do that.
12:31:39So wherever this get built-in tool is
12:31:41called which is in tool registry. We
12:31:44have to add this logic right in this
12:31:46registry. py file whenever we're trying
12:31:49to get the tools here. I can just say
12:31:51that I want to filter by allowed tools
12:31:53if it's configured. So I'll check if
12:31:55self.config.allowed
12:31:58tools is specified and we don't have
12:32:00access to config here. So we need that
12:32:03configuration here. Let me import it
12:32:05from constructor and whenever this tool
12:32:08registry is called I'll find all the
12:32:10references and it's only called within
12:32:13this create default registry. I'll pass
12:32:15in the config as well. And now whenever
12:32:20we try to get the tools I just check if
12:32:22self.config.allow
12:32:24tools is present. Oh I've not defined
12:32:27the config yet. Let me do self.config
12:32:29equals to config. And now whenever I try
12:32:32to do is self.config do.allow tools is
12:32:35not null
12:32:37or it's configured. In that case I'll
12:32:40just do self.config.allow
12:32:42tools. I'll create it in a set format.
12:32:45Why? You'll get to know in just a
12:32:46minute. So we have allowed tools set and
12:32:49then I'll go over every tool that is
12:32:52present in this tools list and check if
12:32:55this tool.name is present within allowed
12:32:59tools. If it is then these are the tools
12:33:02that we want to configure otherwise
12:33:05nope. So I'll just take this allowed set
12:33:07pass it in over here and then these are
12:33:10the tools that are actually allowed. So
12:33:13we have overridden this tools list that
12:33:15we created and this is the reason why we
12:33:17created a set because here the lookup
12:33:19time will be O of one. So I've converted
12:33:22it into a set. All the duplication of
12:33:24the names of the tools will also be
12:33:26removed. And at the same time since this
12:33:28is a set we can just do an O of one
12:33:30operation here. So we go over every tool
12:33:33that we appended and just check if that
12:33:35name is an allowed set and then we set
12:33:38it to tools. Again the benefit of doing
12:33:41it in get tools is that when we call get
12:33:43schemas which is called within agent. py
12:33:45whenever we want to tell what schemas of
12:33:48tools we have. For example if you do
12:33:50find all references you'll see this is
12:33:52called within agent. py specifically
12:33:55when we are trying to pass in all of the
12:33:58tools that we have available. So those
12:34:01tools that are not in allowed
12:34:04tools configuration are removed out if
12:34:07the allowed tools schema is set. So
12:34:09that's good for us. Awesome. Now we can
12:34:12go back to our sub aents. py scroll a
12:34:15little bit down. We can try to pass in
12:34:17the correct configuration system. So
12:34:19first of all it can be a configuration
12:34:21system by default that the user
12:34:24specifies for the entire agent list. All
12:34:27right. But there are some properties
12:34:29that can be overridden. So what I'll do
12:34:32is create a config dictionary which is
12:34:35equal to self do.config do.2 dictionary
12:34:38and we'll have to create this function
12:34:40of two dictionary within this config
12:34:43system. And it's not a lot of work. All
12:34:46we have to do is call model dump. Since
12:34:49we are in base model by pantic it's very
12:34:52easy. So we are going to return a dict
12:34:54of string comma any we'll import that
12:34:57from typing and then I'll have return
12:34:59self dot model dump which is given by
12:35:02base model pantic and then we'll just
12:35:05pass in mode is equal to JSON and we're
12:35:08done. Now I can call the dict which just
12:35:11converts the entire model into
12:35:13dictionary and now I can override
12:35:15certain values for example max turns and
12:35:18time out or sorry not timeout the
12:35:20allowed tools list. So I'll have if the
12:35:23max turns is specified and I think max
12:35:25turns is always specified. So we'll just
12:35:28do config dictionary at max turns is
12:35:31equal to self.definition
12:35:34domax turns then we have if self.defin
12:35:39dotallowed tools is not none. In that
12:35:42case we'll have config dictionary
12:35:45atallowed
12:35:46tools is equal to
12:35:48self.definition.allowed
12:35:50tools. And then we just have to convert
12:35:52this entire sub aent configuration
12:35:54system into a pyantic model. So we have
12:35:58sub aent config is equal to config and
12:36:00we pass in the entire config dictionary
12:36:02by dstructuring it. Now I can just pass
12:36:05in this entire sub aent config where
12:36:08only two properties are changed. But if
12:36:10you want to expand this agent or sub
12:36:12aent you can. And now we have as agent.
12:36:16Now the very first thing I would like to
12:36:18do is just go over all of the events and
12:36:20call agent.run. So async for event in
12:36:23agent.run. Similar to everything we've
12:36:26done before, I need to pass in the
12:36:27message which is the prompt. And
12:36:29remember the prompt is given to us. It's
12:36:32the goal prompt over here. It's what the
12:36:36LLM gives us. And then I need to pass in
12:36:39the message or the prompt. Now for the
12:36:41prompt there are two things that we need
12:36:44to do or actually three things I would
12:36:46like to talk about. First we have to
12:36:47create an entire prompt and in that
12:36:49prompt we are first going to have the
12:36:52identity section of what this specific
12:36:55sub aent is going to do. We'll tell it
12:36:57that it's a specialized sub aent and
12:36:59stuff. Then we'll give it the definition
12:37:02goal prompt which is this part. This is
12:37:06basically a highle overview of what this
12:37:09LLM is good at. For example, it is a
12:37:12codebase investigator. It should use
12:37:15read file GB, glob and list directory to
12:37:18investigate. It should not be able to
12:37:20modify any files. The job is to explore,
12:37:23understand code, all of that. So that is
12:37:26the goal prompt. This goal that's coming
12:37:29from the LLM is more like a task. So
12:37:32what is the task that this sub agent
12:37:34should complete for this specific
12:37:36codebase? So that is the difference
12:37:39between this goal prompt and this goal.
12:37:41Both of them are going to go into the
12:37:43prompt. So let's orchestrate an entire
12:37:45prompt here. So let's just call it
12:37:48prompt which is equal to and I can paste
12:37:50it in. You can also paste it in from the
12:37:53GitHub repository. But this is what the
12:37:55prompt says. You are a specialized sub
12:37:57aent with a specific task to complete.
12:38:00this is the task that it wants to
12:38:01complete like it has to investigate the
12:38:04codebase. It can only use whatever tools
12:38:07are specified and then we say what is
12:38:09the exact task that it needs to do and
12:38:12we get that task by doing parameters dot
12:38:15goal and then we say important where it
12:38:18only needs to focus on completing the
12:38:20specified task. Do not engage in
12:38:22unrelated actions. Once you have
12:38:24completed the task or have the answer,
12:38:26provide your final response which is
12:38:28kind of like a summary and then be
12:38:30concise and direct in your output. You
12:38:33can also change this based on whatever
12:38:35behaviors the sub aents exhibit. Now we
12:38:37can take this prompt and pass it in. Now
12:38:40I hope you understand the difference
12:38:41between this parameters goal which is
12:38:44the task and the highle goal of what the
12:38:47sub aent needs to do and then I can try
12:38:49to track the activity of each event. So
12:38:53if event dot type is equal to let's say
12:38:57agent event dot tool call start. Let me
12:39:01import agent event. And then we have
12:39:03tool call start. In that case I need to
12:39:06append the tool calls. And this is quite
12:39:08similar to what we did within the main.
12:39:12py when we had to call all of the events
12:39:15and stuff right but this time the
12:39:18difference is we don't have to display
12:39:20whatever the sub agent is doing on the
12:39:23main screen on the main screen we'll
12:39:25just get to know that yeah the agent has
12:39:28started doing its stuff there's one on
12:39:32the UI it will only show up as one sub
12:39:35agent doing the task it won't show what
12:39:37the sub aent is exactly doing in because
12:39:40that's the point whatever the sub aent
12:39:42does. It's only related to investigating
12:39:45the codebase. We're not interested in
12:39:47the very specifics of what it does. We
12:39:49are only interested in the output that
12:39:52the sub agent gives us. That's why we
12:39:54called it a sub agent. So now I just
12:39:56have to append all of the tool calls so
12:39:58that I can just store it at the end when
12:40:01we try to give out a summary. We'll just
12:40:03show all of the tool calls that were
12:40:05done. That sounds good. So we'll have a
12:40:08tool calls list at the top which is
12:40:10going to be an empty list. And I'll just
12:40:12take this tool calls and add it every
12:40:14time we get a tool called start. So I
12:40:16have tool calls.append then I get event
12:40:19dot data.get
12:40:21and what is the name of the tool that
12:40:23was called? I am only interested in the
12:40:25tool call name. Right? If we go to
12:40:27main.py we get the call ID, the name,
12:40:30what was the tool kind and any arguments
12:40:33because those helped us in displaying it
12:40:35out to the user. Now we don't have to
12:40:37display anything. So in that case I'm
12:40:39only interested in the name because when
12:40:41we try to give a summary we'll only show
12:40:44the name of the tool nothing else
12:40:45matters to us. Then we have event dot
12:40:48type is equal to agent event dot let's
12:40:52say the text completes in that case we
12:40:56have a final response with us because
12:40:58the text completed right so we have
12:41:00final response is equal to null and
12:41:03we'll just do final response is equal to
12:41:06event dot data dot get content in this
12:41:10case after that it can also result in
12:41:12some sort of error and if it's an error
12:41:15then we are just going going to do agent
12:41:17event dot agent error and then we can
12:41:20store multiple things. For example, the
12:41:22first one is the error output which is
12:41:27by default null and then we can just set
12:41:30error is equal to event dot data.get get
12:41:33error and if we don't find it there's an
12:41:36unknown error that occurred and the
12:41:38final response can be hey the sub aent
12:41:42error is error also let's have this sub
12:41:46aent great and then we can break out of
12:41:49this loop because it's an error we just
12:41:52get out another thing I would like to
12:41:54display out when we try to give out the
12:41:56result to the user and to the LLM
12:41:59because remember this sub agent is still
12:42:00a part of the bigger context Right? When
12:42:03we give a summary, we need to append it
12:42:05to the context and we will be appending
12:42:07to the context similar to the other
12:42:09tools. Whenever we did return tool
12:42:11results success result, it just got
12:42:14added to the context, right? Because
12:42:16that's the way we handle things. So I
12:42:18would also like to tell it what was the
12:42:20termination reason. Why did this agentic
12:42:23loop get over? Maybe because there was
12:42:26some sort of error or maybe there was a
12:42:29timeout and we need to handle the
12:42:31timeout case as well. So yeah, let's
12:42:33have a variable called terminate
12:42:36response which is going to be goal by
12:42:39default. So the terminate reason is just
12:42:42that we hit the goal. So that's done.
12:42:45And if you want to be more explicit, you
12:42:46can say hit goal. But I just think goal
12:42:49is enough. And now let's handle the case
12:42:53where the agent might have a timeout
12:42:56because here let's say the user
12:42:59specified 5 minutes as the timeout. 5
12:43:02minutes is up. Let's just get done with
12:43:04it.
12:43:06So here I'll have a deadline which is
12:43:08equal to async io. Let me import that.
12:43:12Dot get event loop. Whatever event loop
12:43:15we are in and we get the time. So this
12:43:18is the current time and then we'll just
12:43:21add the self.definition dot timeout
12:43:23seconds to it because that is our
12:43:26deadline whatever time is right now plus
12:43:29the timeout limit given by the user or
12:43:34by us. And now within this agent.run
12:43:38every single time we get a new event
12:43:40I'll just check for the timeout. I'll
12:43:42check if async io.getvent get event loop
12:43:46dot time is greater than the deadline
12:43:50because deadline was instantiated only
12:43:51once at the top when we created the
12:43:53agent for the first time. In that case,
12:43:56I'll have the terminate response is
12:44:00equal to time out and then the final
12:44:03response is going to be that the sub
12:44:05agent timed out and then we can break
12:44:09out of this entire mess. Calling our
12:44:12main agent tech loops is quite
12:44:14interesting. But anyways, now if the
12:44:16tool call starts, we've not terminated.
12:44:18If the text completes, we've not
12:44:21terminated yet. But if there's an agent
12:44:24error, we have terminated. In that case,
12:44:27I'll just add the terminate reason to be
12:44:30error. So this seems good to me. But
12:44:32there's one thing you might be
12:44:34wondering. What about max turns? You
12:44:36have not done anything about that,
12:44:37right? Well, we have. When we have max
12:44:39turns here, we add it to the
12:44:41configuration system. Right? And then
12:44:43when we call the agent here, we pass in
12:44:46the specific configuration system. And
12:44:48then when we call agent.trun within
12:44:50here, it calls the agentic loop. And in
12:44:53agentic loop, what we're doing is
12:44:54self.config.m domax turns. These are the
12:44:57number of turns. So it will go on for a
12:44:59maximum of 20 turns. And it when it's
12:45:02not able to go more, it will or it
12:45:05should be returning an error here that
12:45:06we have. we hit the max number of turns.
12:45:09So this agentic loop is now errored out.
12:45:11This these are the max turns we can do
12:45:13for example 20. So here I'll just add a
12:45:16return or yield agent event dot agent
12:45:19error maximum turns and the max turns
12:45:22reached and that would mean that this
12:45:25agentic loop returns an agent error
12:45:27here. And since we're yielding it that's
12:45:30good because in sub aents. py we yield
12:45:32it we get the event here. And whenever
12:45:34we get an error we just say that the
12:45:36response is error the sub agent error
12:45:38route and the error message is given
12:45:40out. Perfect. Now in the exception
12:45:43whenever we run into any sort of error
12:45:45we want to handle that as well. So this
12:45:47is going to be very easy. The terminate
12:45:49response is again going to be error.
12:45:52Then we want to draft a final response
12:45:54here. We also want the error reason. So
12:45:58we have error is equal to string of
12:46:00error. And then we just say final
12:46:02response is equal to and then have sub
12:46:05agent failed. And then we pass in this
12:46:08error object. Finally, after all of this
12:46:12execution, if everything works out
12:46:14either if it's an exception or if it's
12:46:17any of these events, we do get the
12:46:19response with us where we are just going
12:46:21to have a result. This is going to be
12:46:24the final result where we have multiple
12:46:27things to pass in, right? The first one
12:46:29is sub agent and then we pass in the
12:46:33name of the sub aent
12:46:34self.definition.name.
12:46:35Let me format this nicely. So I'll put a
12:46:39single inverted comma here and then I
12:46:41say that this agent completed. Did it
12:46:44complete successfully or with error?
12:46:46We'll add it on the next line which is
12:46:49termination reason. So the termination
12:46:51reason is just going to be terminate
12:46:53response. Then we have the terms used.
12:46:56Then I have the tools that were called
12:46:59and then I can pass in all of the tools
12:47:01here separated by a comma. So I'll just
12:47:04have comma dot join and I'll pass in the
12:47:07tool cause if the tool cause exist
12:47:11otherwise it's going to be null and that
12:47:14is going to be in a string format. And
12:47:16then we have the result with us which we
12:47:19created at the top which is here. The
12:47:21result is going to be whatever is the
12:47:24final response if it exists otherwise
12:47:27there's going to be no response. Now if
12:47:30we run into any sort of error on the UI
12:47:33we have to display this error message in
12:47:36red. Right? So if we have an error
12:47:39object which is over here or over here
12:47:42whenever we run into an error I'll just
12:47:45say return tool result dot error result
12:47:49and I'll pass in the result here. So
12:47:53everything that gets displayed is in red
12:47:55so the user knows we ran into an error.
12:47:58If it doesn't terminate with error then
12:48:00we'll have tool result dot success
12:48:03result and then I'll pass in the result
12:48:05again. But this time it's going to be in
12:48:08green. So we do all of the processing.
12:48:11Let's say 20 turns get done over here.
12:48:14And at the end we just have a result
12:48:16with the final response. And that final
12:48:19response is just the LLM giving us a
12:48:22summary of whatever it did and found
12:48:24out. Now the next thing I have to do is
12:48:28over here outside of this class create
12:48:31sub aent definitions. And we are going
12:48:33to have two default sub aents. There's
12:48:36going to be a codebase investigator sub
12:48:38agent and then a codebase reviewer sub
12:48:41aent. Let's create these predefined sub
12:48:44aent definitions. So the first one is
12:48:46name codebase investigator. The
12:48:49description is that it investigates the
12:48:51codebase to answer questions about code
12:48:53structure patterns and implementations.
12:48:57The goal here is that it is a codebase
12:48:59investigation specialist. The job is to
12:49:01explore and understand code to answer
12:49:03questions. They can use the tool like
12:49:05read file, GP, glob and list directory
12:49:09to investigate. Do not modify any files.
12:49:12And the allowed tools are read file, gp,
12:49:15glob and list directory. No other tool
12:49:18is allowed. So write file, edit, all of
12:49:20that is filtered out. And the max number
12:49:23of turns is 15. But maybe we can remove
12:49:25it because we do want it to be the
12:49:28default one, which is 20.
12:49:31Then we have the code reviewer which is
12:49:33again a sub agent definition. Then the
12:49:36name is specified code reviewer. The
12:49:38description is it reviews code changes
12:49:40and provides feedback on quality bugs
12:49:43and improvements. The goal is you are a
12:49:46code review specialist. Your job is to
12:49:48review code and provide constructive
12:49:50feedback. Look for bugs, code smelt,
12:49:52security issues and improvement
12:49:54opportunities. Use read file and grip to
12:49:57examine the code. do not modify any
12:49:59files. Maybe we can again give this code
12:50:02reviewer list directory as well. So it
12:50:05has list directory because it's trying
12:50:07to review the code, right? So it might
12:50:09have to look over several files. And
12:50:12then maybe we can also give it glob. But
12:50:15just to demonstrate that glob is not
12:50:19allowed here, we'll not give the that
12:50:21that tool. But logically it would make
12:50:22sense to give it that tool. and max
12:50:25turns. We can just set it to 10 this
12:50:27time so that we don't use a default one.
12:50:29And maybe the timeout seconds here can
12:50:31be 300 instead of 600. So 5 minutes. And
12:50:36now I'll just create a function similar
12:50:38to our init. py file where we add get
12:50:42all built-in tools. Here we are going to
12:50:44have get default sub aents. So we have
12:50:47get default sub aent definitions and we
12:50:53return a list of sub aent definitions
12:50:56from here. And the list we return here
12:50:59is codebase investigator and codebase
12:51:03reviewer. And now we can call this
12:51:05function within the registry. py file.
12:51:09So here at the bottom when we create a
12:51:11default registry we can go over every
12:51:13sub aent tool or sub aent definition in
12:51:18this particular function get default sub
12:51:20aent definitions and we'll import it
12:51:22from tools dots sub aents and now I can
12:51:26go ahead and register these sub aents
12:51:28because similar to the default built-in
12:51:31tools we also have to register sub aents
12:51:33right because sub aents are also tools
12:51:36and now we can just do register and Now
12:51:38I need to pass in the tool and the sub
12:51:41aent definition here is a sub aent
12:51:44definition. It's not a tool. Remember
12:51:46sub aent definition is just a data
12:51:49class. It has no dependency on this
12:51:52tool. However, it does have a dependency
12:51:55on sub aent tool and sub aent tool has a
12:51:57dependency on tool. So what I can do is
12:51:59call the sub aent tool here like that.
12:52:03And let me import sub agent tool. And
12:52:05then I can pass in the configuration and
12:52:08the definition. So I'll pass in the
12:52:10configuration as it is similar to what
12:52:12we did for tool class here. And then
12:52:15I'll pass in the sub aent definition
12:52:17which we have here. And they both are
12:52:19registered in our tool registry. So they
12:52:22are mentioned within this tools. And now
12:52:25within get tools and get schemas which
12:52:28are basically two most important tool
12:52:31functions after invoke as well. What we
12:52:34do here is filter out the allowed tools.
12:52:38So we have the sub aent tool here. It
12:52:40appended that tool and then it check if
12:52:43allowed tools is set up and for both our
12:52:45default sub aents the allowed tools is
12:52:46set up. Then we filter out the tools
12:52:48that are not specified. So that is it
12:52:50about sub aents. I think we can go ahead
12:52:54and try to ask it to review our code. So
12:52:57I'll open up the AI agent here. And we
12:53:00run into an error. Cannot import name
12:53:02agent from partially initialized module
12:53:04agent.agent. Most likely due to circular
12:53:07import. The problem is that in
12:53:09agent.agent we are indirectly referring
12:53:12to sub aents. py and in sub aents. py we
12:53:17are accessing agent.agent. So there's a
12:53:19circular loop going on. To fix that,
12:53:22what I can do is instead of importing
12:53:24agent.agent at the top and agent.events
12:53:28either, I'll just remove both of them
12:53:30and call it when
12:53:33we try to execute
12:53:35the execute function. So here I'll have
12:53:38both of these imports done. So they're
12:53:41imported only when they're required, not
12:53:44when the tools are being registered. So
12:53:46this is lazy importing. And now if we
12:53:49run it, it gets initialized. Understand
12:53:52what I did here? Instead of initializing
12:53:55everything at the top, so everything
12:53:57gets imported, what I'm doing is
12:54:00importing these modules only when
12:54:02they're required. whenever this execute
12:54:04function is called. And now maybe I can
12:54:08try to say use sub agent codebase
12:54:11investigator to help me understand this
12:54:15codebase. And this codebase refers to
12:54:18this current working directory. We are
12:54:20passing that in through the system
12:54:22prompt. So let's hit enter. And as you
12:54:25can see sub aent codebase investigator
12:54:27is the tool call that was done. The goal
12:54:30is specified. understand the codebase
12:54:32structure, patterns and implementations
12:54:34and the agent sub agent will do its
12:54:36thing. We don't see anything. We'll only
12:54:38see the output once it's available to us
12:54:42which is just the final response. If you
12:54:44want to display everything, go for it.
12:54:47But I don't want to convolute this
12:54:49entire screen. The point of sub agent is
12:54:52so that we only get to know the output.
12:54:54Also, we can't see the output here. And
12:54:57it says it seems the sub agent did not
12:54:59provide a response. and then it starts
12:55:01to read all of the files itself. I'll
12:55:04just stop it because we need to
12:55:06understand why this error occurred. The
12:55:09agent returns a text to us. That is the
12:55:12final response. But in our case,
12:55:14probably the agent did not return any
12:55:16text and for that reason we don't have a
12:55:18final response. So what we can do here
12:55:21is try to capture another event which is
12:55:24the agent end event. Right?
12:55:27So if we run into an agent end event
12:55:31then we get a final response and the
12:55:33usage. So I'll just check if the final
12:55:35response is not none or if it is null at
12:55:39that point then we'll just do final
12:55:41response is equal to and then I do event
12:55:44do data dot get and I try to get the
12:55:48response.
12:55:49Cool. Now let's try to run it again and
12:55:51see what we get. So I'll just run the
12:55:54same thing. I'll paste in the same
12:55:56prompt and let's hope this time it
12:55:58works. So while the agent is doing its
12:56:00stuff, I realize there's some sort of
12:56:02other error which is tool result error
12:56:04that we are returning. We have to return
12:56:06error result not just error. And now we
12:56:10can try to see what it does and actually
12:56:14we run into the same issue I believe
12:56:16which is that the sub agent did not
12:56:18return anything. And since we fixed this
12:56:20part where we are returning an error
12:56:22result, maybe an error is being returned
12:56:25from here. So let's try to run it again
12:56:28and see if it worked. Otherwise, I'll
12:56:30have to debug and see what we did wrong.
12:56:32So I'll paste in the same prompt.
12:56:34And we run into the same issue again.
12:56:37Let me just stop it again. I wonder why
12:56:40that is. So I'll just go offline and try
12:56:42to debug it. But I think maybe the error
12:56:44is something related to this where the
12:56:47event type is agent event type and we
12:56:51are trying to handle agent event. So
12:56:54maybe we can try to do things
12:56:55differently. Instead of doing agent
12:56:57event, we do agent event type. Let's
12:57:00import it from agent.events.
12:57:02And I'll just cut this from here. Paste
12:57:06it down here
12:57:08instead of agent event. So we have from
12:57:11agent.events events, we import agent
12:57:13event type. Bad naming causes these
12:57:16issues. So, you should probably look
12:57:18into renaming them. But yeah, now I
12:57:22think this should solve the error.
12:57:25Again, the issue seems to be that the
12:57:27event type is of the type of agent event
12:57:30type and we were doing agent event which
12:57:32was something else. Agent event is
12:57:34defined within events. py where agent
12:57:38event is a class and we call certain
12:57:41things on that but the truth is agent
12:57:44event type is the one that contains
12:57:46everything it contains agent start agent
12:57:48end agent error all of that changing to
12:57:51agent event type should work let's hope
12:57:53that so I'll paste in the same prompt
12:57:55again and this time hopefully it works
12:57:59and we do run into an error this time
12:58:02it's a different one sub agent codebase
12:58:04investigator completed Termination was
12:58:07error. The tools called were none. And
12:58:09the sub agent failed because type object
12:58:11agent event type has no attribute tool
12:58:14called start. So let's try to see what
12:58:17it is. This is tool called start which
12:58:19is capital case. Then we have text
12:58:21complete which is capital case. Then
12:58:24agent end is also capital case. Agent
12:58:26error is also capital case. These were
12:58:29the errors. And now I think we'll be
12:58:32able to run this properly. So we are
12:58:35very close and then run this. Earlier
12:58:38what we were doing is agent event dot
12:58:40and then we called agent start or agent
12:58:43end or agent error right and these were
12:58:46functions. They were not really enum
12:58:49attributes. Then we switched to agent
12:58:51event type but we were still using these
12:58:53names. That's why we got a bit confused
12:58:56or actually I got confused. I think you
12:58:58might not have but anyways I think this
12:59:01should fix everything. So let's see what
12:59:03the output is.
12:59:13So we do get the output here. The
12:59:16codebase investigation is complete.
12:59:18Here's a summary of the key findings and
12:59:20then it gives us the codebase structure,
12:59:21code architecture, key components,
12:59:24everything. And it does kind of so we do
12:59:27get the output here but the output is
12:59:30not displayed in the sub aent codebase
12:59:32investigator
12:59:33because well we have not changed
12:59:36anything in the TUI to display the
12:59:38output of the sub agent but we know that
12:59:41the LLM does get that context in. So
12:59:44everything worked out because you know
12:59:45this output shows up which is trying to
12:59:49infer whatever the sub agent said over
12:59:52here. So now let's try to go to this
12:59:55TUI. py and handle this case. Now sub
12:59:58agent is not going to be something I
13:00:00explicitly code up in preferred order or
13:00:03in the TUI in tool call complete. What
13:00:06I'm going to do instead is I'll just add
13:00:10an else condition here and within that
13:00:12else condition I'll just check if we are
13:00:15running into some sort of error. If we
13:00:17are running into some sort of error then
13:00:19we'll just do blogs.append append text
13:00:21and pass in an error. Correct? That
13:00:24sounds reasonable. But then I will
13:00:26remove all of this out of this error
13:00:30case. Understand what I did here. If our
13:00:34tool call doesn't match any of the tool
13:00:37calls present here, then we'll execute
13:00:39this else case where we check if the
13:00:41error is present and we're not running
13:00:43into a success, then we have
13:00:45blogs.append text error. Great. So it
13:00:48will append the text and then we just
13:00:50try to display the output here in a
13:00:52syntax block. That's what we want,
13:00:55right? Because whatever is the output of
13:00:56sub aent, I would like to display it in
13:00:58syntax. That's great. So with this else
13:01:00condition, we are handling everything.
13:01:03We're handling the error case properly
13:01:06because in error the output will display
13:01:08and if it doesn't display we just say no
13:01:10output and for our sub aent we just want
13:01:13to display the output nothing else
13:01:16because from sub aent we are just
13:01:17returning an output right now let's try
13:01:20to run it again and see what works. Let
13:01:23me get out of here. I'll clear off this
13:01:25entire thing and then I'll say
13:01:28investigate and understand this code
13:01:30base. Let's hit enter and see what it
13:01:33does. So this is the output that we get.
13:01:36And yeah, the sub agent codebase
13:01:38investigator completed the tool calls
13:01:41were all of these tool calls. The result
13:01:44is codebase analysis summary and it
13:01:47gives us the entire analysis of what it
13:01:51is. Now you can obviously go ahead and
13:01:54add a better goal prompt here so that it
13:01:56is more descriptive with whatever it
13:01:58says. It is more specific about what
13:02:01things it needs to tell in the final
13:02:04response about the investigation because
13:02:06obviously it's very good at
13:02:08investigating.
13:02:09It does call so many files, right? So
13:02:13that's impressive. We will move on to
13:02:15the next thing. But before we do, I
13:02:17would like to encourage you to pause the
13:02:18video here, add the feature of creating
13:02:21your own sub agent. So in the config.py,
13:02:24we can have something like a new class
13:02:26which is sub aent config. And then you
13:02:29take the sub aent config from this
13:02:31config class or you can take a list of
13:02:34sub aent configs because users can
13:02:36specify multiple sub aents. And then you
13:02:39register all of that sub aents using
13:02:41tool registry. Simple. Just add that
13:02:44feature and you'll understand a lot more
13:02:47than you already are by following this
13:02:49tutorial. So go do that and then we'll
13:02:52get to the next feature which is tool
13:02:54discovery. So what is tool discovery?
13:02:57Think about it this way. We have all of
13:02:59the built-in tools here. And we also
13:03:01have the sub aents, which is technically
13:03:03a built-in tool, but if you've created
13:03:05the feature of creating your own sub
13:03:07agent, that's not a built-in tool
13:03:09because you're giving the ability to the
13:03:11user to create their own sub aents.
13:03:13Similarly, we want to do that for other
13:03:15tools as well. We want people to be able
13:03:19to specify their own tools that we can
13:03:21call. That is what tool discovery is
13:03:23about. To be very specific, what I want
13:03:25is that the user can specify a tools
13:03:28folder within AI agent folder. And then
13:03:32I just go ahead and maybe have one tool.
13:03:34Let's call this test tool. py. And in
13:03:37here, I'll just paste up some test tool
13:03:39that I've created. It's quite similar to
13:03:41all of the other tools we've created. We
13:03:43have a parameters which is specified as
13:03:45a schema here. Then the kind of the test
13:03:48tool is passed in. Then you have
13:03:50description, the name of the tool, and
13:03:52an execute function. And what this tool
13:03:54is doing is that it's just creating an
13:03:57output saying test tool received. Then
13:04:00it passes in the message that's coming
13:04:01from the LLM and it says tool was
13:04:04discovered from let's say
13:04:06agent/tool/esttool.
13:04:09py. So this is just a tool that spits
13:04:12out the test message from LLM. We want
13:04:15to enable our user to be able to create
13:04:18such a tool. So as you can see this is
13:04:20created within tools test tool. py and
13:04:23I've not mentioned this test tool. py
13:04:25within the built-in init folder where
13:04:27you know we register all of our tools.
13:04:29So I want this particular feature such
13:04:32that any user can define whatever tool
13:04:35they want within this tools folder and
13:04:38then we just dynamically load it and
13:04:40register it so that the LLM can call
13:04:42them. So I'll keep this test tool over
13:04:44here and we'll test it later on when
13:04:47we've added the tool discovery feature.
13:04:49Now you might ask what is the purpose of
13:04:51this? Well, the user can specify their
13:04:54own tool. So if they have a complex
13:04:56workflow, they can execute this tool
13:04:58call which will do all of the complex
13:05:00execution logic not like our logic over
13:05:04here where we just spit out the output
13:05:06message. This can be a very complex
13:05:08let's say 10step process which happens
13:05:11behind the scenes with just one tool
13:05:13call. So that's a really good way to not
13:05:16load up a context because if we provide
13:05:1910 tools that can do a similar task and
13:05:21then you need 10 tool calls in that
13:05:23order to apply this tool call that's
13:05:27inefficient. Instead, you can just have
13:05:30one tool created where you pass in the
13:05:32name, description, the schema, whatever
13:05:35you need from the LLM and then you just
13:05:38write down your own logic here because
13:05:40at the end of the day we are creating it
13:05:42for coders developers. So they can just
13:05:44write down code for this and execute it
13:05:47every single time and that will be much
13:05:49better utilization of context and much
13:05:52efficient as well especially if they
13:05:54intend to use a tool several times in
13:05:58one workflow and if they want to use it
13:06:00across multiple projects that's also
13:06:02something we'll allow by using the
13:06:05configuration directory similar to how
13:06:07config.toml was done in this AI agent
13:06:10the config.toml Toml was specified for
13:06:13this particular project and then a
13:06:15systemwide configuration which could
13:06:17change anything. We going to do a
13:06:19similar thing for tools as well. So
13:06:21let's go ahead and create a file within
13:06:24tools folder called discovery. py and
13:06:28then we can go ahead and create this
13:06:30class. Let's call this tool discovery
13:06:32manager. And here we're going to get the
13:06:35init function. Within init we are going
13:06:37to get the configuration which is going
13:06:39to be config. And we're going to get the
13:06:41tool registry as well because this tool
13:06:43discovery manager will get the registry
13:06:46because we will need the registry,
13:06:48right? So that we can register all of
13:06:49our tools. And we can't create an
13:06:52instance of tool registry in this init
13:06:54function because there's one tool
13:06:56registry that loads up all of the tools.
13:06:58If I reinstantiate it, the tools will
13:07:01again be an empty dictionary. I don't
13:07:03want that. So there we go. And now I can
13:07:05just do cell.config is equal to config.
13:07:08Then you have cell.registry registry is
13:07:09equal to registry and that's it. Now I
13:07:13can create the function called discover
13:07:16all. So I have discover all self and
13:07:20what it's going to return is nothing. It
13:07:22will just register all of the functions
13:07:24or tools after discovering them. And now
13:07:28I can decide what I have to do here.
13:07:31First I have to search in the current
13:07:33working directory for any folder which
13:07:36is AI agent tools and then all of the
13:07:40tools are present within this tools
13:07:43folder and all of the tools are going to
13:07:45end with py because we are not
13:07:48supporting any other language. Your tool
13:07:51has to be written in Python so that we
13:07:53can execute it. So let's just create a
13:07:55helper function here which is discover
13:07:58and I misspelled everything. Let's say
13:08:00discover from directory and here I have
13:08:04to pass in a directory. So the directory
13:08:07is going to be self.config dot current
13:08:09working directory. Okay. And now I'm
13:08:12going to create a function that will
13:08:13help us discover the tools from
13:08:15directory and it will register all of
13:08:17the tools as well. So we have self then
13:08:20we have a directory and this directory
13:08:22is going to be of the type of path
13:08:24because self.config.currenwork current
13:08:26working directory is a path and this
13:08:29function is also going to return
13:08:31nothing. It's just going to call the
13:08:33registry and register everything. Now
13:08:35the first thing I have to do is go to
13:08:37this directory path that's given to us.
13:08:40So for example the current working
13:08:41directory and within that I'll go to AI
13:08:44agent and then tools. So let's have tool
13:08:47directory created here which is equal to
13:08:49directory forward slash and then we have
13:08:53AI agent and then we have forward slash
13:08:56and then tools right because tools is
13:08:59going to be a folder and now I can just
13:09:01check similar to everything we've done
13:09:03so far if the tool directory does not
13:09:06exist or let's say this tool directory
13:09:09is not a directory either in that case
13:09:12I'll just return I don't want to execute
13:09:14any further because it should be a
13:09:16directory. It should exist. After that,
13:09:19I will go over every file within this
13:09:22tool directory. And I mentioned to you
13:09:24that every file that we're going to go
13:09:26through is a Python file. So, I'll be
13:09:28very explicit. I'll say for the Python
13:09:30file in tool directory.glob.
13:09:33If you remember this, it just iterates
13:09:36over a specific number of files that
13:09:39follow this pattern. We've created a
13:09:41tool for that. So, I'm sure you know
13:09:43about this. And then we have asterisk.
13:09:46py. So any file that ends with asterisk.
13:09:48py. We're not looking into subfolders
13:09:50which might be created within tools. If
13:09:53you want to add support for that, you
13:09:54can by doing something like this. But
13:09:56I'm not interested in that. After that,
13:09:59I'll just check for one edge case, which
13:10:02is if the Python file starts with, let
13:10:05me just say Python file. if it starts
13:10:07with and actually I have to convert this
13:10:10Python file which is a path into a
13:10:13string. So I'll get its actual name
13:10:15which is dot name and then this dot name
13:10:18will give us a string. I can call starts
13:10:20with and I'll pass in a prefix which is
13:10:24underscore underscore. So if the Python
13:10:26file name starts with underscore so for
13:10:28example you have underscore_init.
13:10:31py or if you have underscore_ain.
13:10:34py something like that we want to ignore
13:10:36it because that is not going to contain
13:10:38any relevant information. It never does.
13:10:41Now what I want to do is load the tool
13:10:44module. So as we know it the Python file
13:10:48that's created here is not directly
13:10:50importable. I can't do something like
13:10:52import
13:10:54agent and then we have tools test tool
13:10:57py. That's not something that we can do
13:11:00in Python. Python only knows how to
13:11:02import files that are in specific
13:11:04places. For example, you've installed
13:11:06the packages or let's say this project
13:11:09folder. Now, it might get a little
13:11:12confusing because this is the AI agent
13:11:15that we have and within this AI agent,
13:11:18we creating the tools folder. So, you
13:11:20might think that yeah, it should be
13:11:22easily importable and the truth is yeah,
13:11:25it is importable. But think about it
13:11:26this way. you're not having the tools
13:11:29folder in this AI agent. You're going to
13:11:32have it in your project. Let's say
13:11:34you're developing a ReactJS application
13:11:37and in that ReactJS application, you
13:11:39know, you have the front end, the back
13:11:41end, and along with that, you have this
13:11:42tools folder, this agent folder, and
13:11:46then you know the test tool. Or whatever
13:11:49tool you create. In that case, how are
13:11:52we going to import this particular file,
13:11:54right? How will our agent even know
13:11:57about that file? We'll be able to import
13:12:00here but we'll not be able to import in
13:12:02your project because think about it.
13:12:05This is our project where we are
13:12:06developing the AI agent. So obviously
13:12:08anything related to AI agent will be
13:12:10imported here. But in a ReactJS
13:12:11application which is completely
13:12:13different from this particular project
13:12:16where you're initializing what was
13:12:17created in this project, you won't be
13:12:20able to import and that's why we have to
13:12:22import the module. So we are going to
13:12:24create a function that will solve this
13:12:27problem. It will manually teach Python
13:12:29about a file that Python doesn't know
13:12:31exists and is useful to us and allow us
13:12:35to import it so that you know once we
13:12:37have the file with us we can get its
13:12:39content you know like test tool and then
13:12:42call its execute function and just
13:12:44register all of it. Also to understand
13:12:46this better, I think it would be better
13:12:48to understand how the Python's import
13:12:50system works. And you can think of it
13:12:53like a library catalog. So when you do
13:12:56import request, Python looks up requests
13:12:59in its catalog, finds the book and
13:13:02brings it to you. But if you have a
13:13:04custom tool file in the catalog, but the
13:13:07custom tool file that we created isn't
13:13:09in that catalog. It's it's like a book
13:13:11that's someone has left on a random
13:13:14shelf that was never registered either.
13:13:17So we have to do create a function that
13:13:20will do three things. It will create
13:13:22this catalog entry. It will create an
13:13:24empty book placeholder and fill in the
13:13:27book by reading the file. Might sound
13:13:29very complicated but it's just four or
13:13:31five lines. We're going to create a
13:13:33function. Let's say load tool modules.
13:13:37And then we have a self. Then we have
13:13:38the file path and the type of this is
13:13:41going to be path as well. And from here
13:13:44we're going to return the module. So it
13:13:46can be of the type of any and we'll
13:13:48import from typing any. Now what I want
13:13:51to do is give this module a name. Right?
13:13:53So that if I have a module, I can import
13:13:56it correctly. For example,
13:13:57tools.registry is a module. That's how I
13:14:00can import it. If I want to import some
13:14:02other file that is not in a project
13:14:05which is the AI agent project, it's in
13:14:07your project. I've talked about this. In
13:14:09that case, we will need this file path
13:14:12to extract the name. So, we can just do
13:14:14file path. But the problem is file path
13:14:17is something like test tool. py, right?
13:14:21I want to remove this py. So, to remove
13:14:23it, what I can do is call stem on it.
13:14:27So, stem will have the final path
13:14:30component minus its last suffix. So, if
13:14:32you have this complicated path of AI
13:14:35agent tools test tool.p py it will just
13:14:38grab test tool. It will ignore
13:14:40everything else. So that is our module
13:14:43name. But the thing is this tool name
13:14:45can conflict with any other name that
13:14:47our tool might have. For example, if the
13:14:50user creates read file py so the tool
13:14:55will be called read file now and that
13:14:57conflicts with our tool that we created.
13:15:00Maybe the user didn't intend to do that
13:15:03because you know why would anyone want
13:15:05to override a read file. So in that case
13:15:08I would just like to call this
13:15:10discovered
13:15:12tool_filepart
13:15:14stem so that it's a bit different from
13:15:17the other tools that are present. After
13:15:20that I will have the spec created. spec
13:15:24is essentially an entry that just
13:15:27specifies the name of the tool, the
13:15:29location where it is and how to load it
13:15:32so that Python can figure it out. So
13:15:35we'll have spec equals to importlib.util
13:15:38and I'll import lib dot util at the top.
13:15:43So we have importlib.util do.spec spec
13:15:46from file location and then I will pass
13:15:49in the name of this library which is
13:15:52module name and then I will specify the
13:15:55file path which is the location and we
13:15:58have the file path with us through the
13:16:00parameter. So again this spec is just
13:16:03going to tell us or tell Python what is
13:16:06the name of the module what is the
13:16:07location and how do we load it or how
13:16:10should Python load it. After that, I
13:16:12just want to check if the spec returned
13:16:14is not null because it can be. So, I'll
13:16:16just do if spec is none or let's say
13:16:20spec.loader is none. In that case, I'll
13:16:24return an import error because you know
13:16:27we did not find that module. So, we'll
13:16:30just say could not load spec from and
13:16:34let's specify the file path here which
13:16:37is this. Now wherever we call this we'll
13:16:40have to do a try except. So remember
13:16:42that after that we would like to load
13:16:45the actual module because here remember
13:16:47this is a spec. It is just a
13:16:49specification. It is just a card. It
13:16:52doesn't really do anything. But we can
13:16:54put it to use by doing import dot util
13:16:59dot module from spec. So it will create
13:17:02a module based on the provided spec and
13:17:05then we will pass in this spec over
13:17:07here. So that is our module. Now what
13:17:10does this function do? It just creates
13:17:13an empty container. So it creates an
13:17:15empty module object labeled with the
13:17:18module name that was specified within
13:17:19the spec which is discovered tool and
13:17:22whatever is the name of the tool. So
13:17:24discovered tool test tool. It's just an
13:17:27empty container as of now. So obviously
13:17:29the next step is we register it with
13:17:31Python so that Python knows about this
13:17:34module and then we execute this module.
13:17:37so that all of the stuff gets fit in
13:17:39within this module. So we'll have cis
13:17:41dot modules. Let me import cis from
13:17:44system. And then we specify the name of
13:17:47this module which is module name and set
13:17:49it equal to module not module name just
13:17:52module. This system domodules is
13:17:55Python's master list of all of the
13:17:57loaded modules. So we just add our empty
13:18:00box to this list so Python knows it
13:18:02exists. And then we're just going to go
13:18:04ahead and load this empty box with the
13:18:08actual content. So we'll actually run
13:18:10the file and fill all of the box or the
13:18:12container that was empty. So we'll have
13:18:15spec.loader.execodule
13:18:17and pass in the module. So with this
13:18:20line what's happening is that it will
13:18:22the pi the system will essentially read
13:18:24the python file execute all the code in
13:18:27it and it will put all the classes
13:18:29functions variables into our module
13:18:31object. After this we just going to
13:18:34return module and now after this point
13:18:37if we just try to do module dot let's
13:18:39say test tool that is valid because it's
13:18:43importable now module does have access
13:18:45to it. So if we do discover tool test
13:18:50tool test tool it is importable. So here
13:18:53we are returning the module that's
13:18:55great. Now we can go ahead and call this
13:18:57function over here where we have self
13:19:00dot load tool modules also this should
13:19:04be over here. Let's fix the indentation
13:19:06and then we'll pass in the python file
13:19:09right because python file is the entire
13:19:12path and that is our module. So now that
13:19:16we have the module obviously the next
13:19:17step is to find all of the tool classes
13:19:20that are present within a module. So
13:19:22we'll go over to the module. We'll get
13:19:24each object and just check if it extends
13:19:28tool because that's what we are
13:19:30interested in. In every tool that we
13:19:32create, the class is going to extend
13:19:35tool. If it doesn't extend tool, this is
13:19:37essentially not useful to us. What's
13:19:40useful is this test tool because it
13:19:43extends tool because this is what we're
13:19:45going to use to register our tool. So
13:19:48now let's create another helper function
13:19:50that will find all of the tool classes.
13:19:53So we have def find tool classes. Then
13:19:57we get self module which can be of the
13:20:00type of any. And from here we're going
13:20:03to return a list of tools.
13:20:07Let me import from tools.base.
13:20:11And now let's find all of the tools. So
13:20:13we have tools as an empty list. And then
13:20:16we can go over every object within this
13:20:20module. So when we do directory, it will
13:20:24list all the names defined within this
13:20:26module and I can try to get the object
13:20:29from here which is get attribute module,
13:20:33name. So what we are trying to do is
13:20:35module dot name whatever it is that is
13:20:38going to be our object. So let's say we
13:20:40have test tool as our module or dis
13:20:44discovered tools test tool dot and then
13:20:49we have the name of the module the
13:20:51actual name which is dot test tool right
13:20:53so we try to get that object here and
13:20:56now we are just checking if this object
13:20:58that we get access to is it even a class
13:21:01because when we're going over the names
13:21:04that are defined within this module even
13:21:06this class can be present test tool
13:21:08parameters or the user might define some
13:21:11other helper functions as well. We want
13:21:13to ignore all of that. All we are
13:21:15interested in is a class and
13:21:17specifically a class that extends tool.
13:21:20So we first going to check if it is a
13:21:22class. To do that we can do inspect.
13:21:25Let's import inspect and we have to use
13:21:28the second one. Just import inspect and
13:21:30then we can say is class and then we
13:21:33pass in the object that we got over
13:21:35here. Now if it is a class and it should
13:21:39also be a subclass. So I'll pass in
13:21:42object and tool here meaning this object
13:21:45extends tool. In that case we're going
13:21:48to do tools dot append and pass in this
13:21:51object because it is a tool object. So
13:21:55we have a list of tools right because
13:21:57yeah we are just appending that object.
13:21:59Also we can put another check here to
13:22:00handle another edge case that object is
13:22:03not tool. Let's fix object here. What
13:22:08does this line do? It just verifies that
13:22:10object is not a tool because it should
13:22:14be a subclass, but it should not
13:22:15specifically be a tool. Otherwise, it's
13:22:17of no use to us. And it can be. And then
13:22:21we'll have object dot module. Let me fix
13:22:24this again. Object domodule
13:22:27if it is equal to module dot name. What
13:22:32this check does is that it ensures that
13:22:34the file was not imported. That the tool
13:22:36that we are going after is not imported.
13:22:39It was actually created in this
13:22:42particular file. Because when we do
13:22:44directory of module, it might also
13:22:46contain things that are imported. So
13:22:48yeah, this is just ensuring that the
13:22:50objects module is equal to the module's
13:22:53name and it is not really the file that
13:22:57was actually imported not defined in
13:23:00this file. So once we have appended
13:23:02everything we are just going to return
13:23:03the tools from here and that's it. We
13:23:06will call this function here. So we have
13:23:09tool classes is equal to self dot find
13:23:12tool classes and I'll pass in the
13:23:14module. After that I'll check if there
13:23:17are any tool classes. Let me name this
13:23:19tool classes because there can be
13:23:21multiple tool classes. If there are no
13:23:23tool classes I'll just continue to the
13:23:26next Python file. I'm not returning.
13:23:28Remember I'm in a for loop here where
13:23:30I'm going over every Python file. So
13:23:32I'll just continue so that I can go back
13:23:34to the next Python file and not waste
13:23:35much time ahead. And after that I'll
13:23:38just do for tool class in all of the
13:23:41tool classes that are present. I would
13:23:44like to instantiate this tool class and
13:23:46pass in the config directory because
13:23:48remember each of our tools that we
13:23:51create needs to have its own
13:23:52configuration directory passed in. So
13:23:55here we have a tool right here we had
13:23:57the init function and each tool needed
13:24:00to be fed an a config system in that
13:24:04same manner we're going to have the
13:24:06config directory inst initialized here
13:24:08as well. So we'll have tool is equal to
13:24:11tool class and we know this is an actual
13:24:14object. This is an actual tool object.
13:24:17So what we're going to do is call it
13:24:18like that and then we'll pass in
13:24:21self.config.
13:24:22Now this is not the actual tool here.
13:24:25It's going to be the class that extends
13:24:28tool because we've put all of the checks
13:24:30necessary for that. And now I would like
13:24:32to register this tool. To register it,
13:24:34I'll just do self.registry.register
13:24:37and pass in the tool. And that's pretty
13:24:40much it. We have registered the tools.
13:24:43Now all I need to do is put this entire
13:24:46thing in a try except because here I
13:24:48have raised an exception if spec is not
13:24:51present. So I'll do a try and accept
13:24:54here saying if we run into any sort of
13:24:56exception then I would just like to
13:25:01continue. Maybe you can log it out. It
13:25:03would be a good idea to log it out but
13:25:06I'm just going to continue here so that
13:25:09we move to the next iteration and do
13:25:11nothing after that. But a logger would
13:25:13make a lot of sense here. So yeah we
13:25:15discover from the directory and it's
13:25:17done. Now the next thing I would like to
13:25:19do is not just discover from our current
13:25:22working directory. I told you we also
13:25:25need a systemwide discovering feature.
13:25:28So similar to this config.tml we are
13:25:31going to have a system level
13:25:33configuration passed in and for the
13:25:35directory we just going to pass in the
13:25:38get config directory that we initialized
13:25:41in config.loader. If you remember this
13:25:45config directory was where all of the
13:25:48apps allowed the users to specify their
13:25:50own configurations. I think open code
13:25:53has done that. Even claude code does it.
13:25:55I think even codeex does it. And now
13:25:58even we do it. So yeah we pass in the
13:26:00get config directory here. And that's
13:26:02it. We have registered all of the files.
13:26:06Now what I like to do is call this
13:26:08discover all function where it's
13:26:10required. And where is it required?
13:26:13Well, in session, right? All we need to
13:26:15do is call this discover all function
13:26:16and everything else will be taken care
13:26:18of. The reason for that is we are just
13:26:21calling the self.registry.register
13:26:24here. So it will get stored in our tool
13:26:28registry. And that tool registry is kind
13:26:30of like the singleton instance. There
13:26:32should only exist one instance of it
13:26:35which contains all of the information
13:26:37about that tools. All of the tools we
13:26:40have even for MCP when we create it we
13:26:43are going to have cell.registry
13:26:45dotregister that is the point of having
13:26:48this explicit registry system. So let's
13:26:51go to our session py because that's
13:26:54where we initialized everything else
13:26:56including tool registry and here we're
13:26:59going to create an instance of tool
13:27:01discovery. So we have self dot maybe we
13:27:04can just call it discovery manager
13:27:05instead of tool discovery manager. And
13:27:08we'll instantiate this tool discovery
13:27:10manager. And let me import it from
13:27:12tools.discocovery.
13:27:14And now I can pass in the configuration
13:27:17which is cell.config. And then we have
13:27:20registry which is self do.tool registry.
13:27:23And after that maybe I can just call the
13:27:25get all tools from here only. So I'll
13:27:29have self do.discovery manager dot
13:27:32discover all tools. That seems like it.
13:27:36Now, let's go ahead and see if this
13:27:38works out. So, I would like to exit
13:27:40whatever we have right now and rerun it.
13:27:43Okay, cool. Now, let's tell our LLM to
13:27:46specifically call test tool. That's not
13:27:49a tool that exists in our built-in
13:27:52tools, but if it calls it, that means
13:27:54everything is working. Let me hit enter.
13:27:57And as you can see, the message is
13:27:59hello, this is a test message and test
13:28:01tool received. Hello, this is a test
13:28:03message and tool was discovered from
13:28:05this particular file. Now what I'd like
13:28:08to do is delete this test tool. After
13:28:10that I'll try to exit the system and now
13:28:13I'll create the AI agent again and say
13:28:15call test tool. Let's see if it calls
13:28:18it. And as you can see the test tool
13:28:21function is not available. Please use
13:28:23the provided tools to accomplish your
13:28:25task. If you need to test something you
13:28:27can use the shell tilt tool to run
13:28:29commands or scripts. That's awesome. So
13:28:31our tool discovery feature is working.
13:28:34The users can specify their own tools in
13:28:38their project like we did with test
13:28:40tool. py and execute any tool of their
13:28:44choice. So this was about creating your
13:28:46own tools, registering them and for them
13:28:50to register they all all they have to do
13:28:52is create a agent folder tools test tool
13:28:56and then they have to do from tools.base
13:28:59base import tool all of that actually in
13:29:02their case they will have to import AI
13:29:04agent as well so something like from AI
13:29:06agent tools.base base import tool
13:29:09they'll have to do this entire thing
13:29:11because right now we are in AI agent
13:29:13folder so we don't have to specify the
13:29:15folder we are in but if you're doing
13:29:19this tool creation in some other folder
13:29:21like your react application which is
13:29:23quite different from this AI agent that
13:29:26we are developing in that case they will
13:29:29have to specify this AI agent and this
13:29:31AI agent is going to be a package that
13:29:34will be installed on your system because
13:29:36even when you have to install claude
13:29:38code for example what do you do you go
13:29:40ahead and do npm install-g
13:29:43cloud code or codeex cli or anything so
13:29:47from npm you're just trying to install
13:29:49cloud code so it is installed on your
13:29:52system and that's how you're able to run
13:29:54it right so when you install cloud code
13:29:56rest of the things also come
13:29:58pre-installed for example all of the
13:30:01files that claude code has so that you
13:30:04can run it and from there you do get
13:30:07access to this tools dot base or
13:30:10whatever exists so that you can create
13:30:12your own tool. Now I'm not very sure if
13:30:15cloud code has support for it. Open code
13:30:17does and the same logic can be applied
13:30:19for open code. So yeah that's about tool
13:30:23discovery. The next thing we're going to
13:30:25work on is MCP. MCP is also about tools.
13:30:29But the difference between tool
13:30:30discovery and MCP is that in tool
13:30:32discovery you have to define your own
13:30:34tools. But in MCP, you can just use
13:30:37someone else's tools that they upload
13:30:40because there are multiple MCP
13:30:42registries similar to npm for example
13:30:46where you can find any tool of your
13:30:48choice and run it. So for example, if I
13:30:51want to talk to Google Drive, there's an
13:30:54MCP created for Google Drive. So I can
13:30:56just talk to my LLM and it will directly
13:30:58fetch data from my Google Drive. That's
13:31:02what MCP allows. It's essentially
13:31:04third-party tool calling and that's how
13:31:06it's different from tool discovery. Now,
13:31:09you can also create your own MCP servers
13:31:12and get it all working. But you can also
13:31:14use someone else's MCP servers and every
13:31:17tool has support for MCP. Cursor has it.
13:31:20Cloud code is the one that created MCP.
13:31:24Open code also has support for that. So,
13:31:26let's dive a bit deeper on what MCP
13:31:28means, what is its full form and how
13:31:31does it work. so that we can start to
13:31:33code it ourselves. MCP stands for model
13:31:37context protocol. Protocol is really the
13:31:39big thing over here. MCP is nothing more
13:31:43than just a set of rules that are
13:31:45defined so that the AI agent knows how
13:31:48to talk to external tools and data
13:31:50sources in a standardized way. Let's
13:31:53understand the problem that MCP solves.
13:31:56Imagine you're building an AI
13:31:57application. and let's say it's an AI
13:31:59assistant that needs to access your
13:32:01calendar, read your emails, query your
13:32:03database, browse the web. Without a
13:32:06standard protocol, every AI application
13:32:09has to write custom integration code for
13:32:11every tool. If you have 10 AI apps and
13:32:1410 tools, that's potentially 100
13:32:16different integrations to maintain. So
13:32:19instead of having those 100 different
13:32:21integrations, you just have one
13:32:24integration. You can think of MCP like
13:32:27USB for AI. Before USB, every device
13:32:30like a printer, mouse, keyboard needed
13:32:32its own proprietary cable and driver.
13:32:35USB set use one standardized interface.
13:32:38Everyone build to this and everything
13:32:41works together. MCP is really just the
13:32:44same thing for AI and tool
13:32:46communication.
13:32:48There are three parts to MCP. The first
13:32:50one is the server. MCP servers which
13:32:53expose capabilities through tools,
13:32:55resources, prompts basically through a
13:32:58standard interface. A server might just
13:33:00wrap a database, an API or a local file
13:33:03system. For example, if you want to read
13:33:05your email, Gmail can expose an MCP
13:33:08server out and those MCP servers can
13:33:12contain tools, resources, and prompts.
13:33:14You know what tools are because we
13:33:17created a bunch of tools already. And
13:33:19one MCP server can have multiple tools.
13:33:22For example, in Gmail, you might want to
13:33:24have a tool that allows you to create an
13:33:27email. Then you have another one that
13:33:31gets an email. Then you have another one
13:33:33that gets all of the emails. Right? So
13:33:36those are all the three different tools.
13:33:38Then you can obviously have more. So in
13:33:40one server you can have multiple tools
13:33:42and then you can have multiple servers
13:33:45as well. Great. Now what are resources
13:33:47and prompts? I just searched up on
13:33:50Google and I think this is a pretty good
13:33:52definition. Resources are readonly
13:33:55structured data like documents or
13:33:57profiles that provide context for an AI
13:34:00model. That's it. While prompts are
13:34:03reusable parameterizable instruction
13:34:05templates that define consistent
13:34:07workflows or conversations for the model
13:34:10often surfaced as UI elements like slash
13:34:13commands for users. So you can think of
13:34:16resources as data feeds example the
13:34:19employee directory whereas prompts are
13:34:21user initiated workflows example
13:34:23summarize a document that combines
13:34:26resources and tools to guide the AI to
13:34:28do that task. So I'm not diving deep
13:34:31into this. Our AI agent is not even
13:34:34going to support resources and prompts.
13:34:36But if you want, you can go ahead add
13:34:38support for it. I'll talk about how you
13:34:40can add support for them. But yeah, the
13:34:42these are two things that also exist in
13:34:44a server. We'll just be focusing on
13:34:46tools. Then the second thing other than
13:34:48MCP server is MCP client. So MCP clients
13:34:53are basically the AI coding agents like
13:34:55cursor, claude code or even normal
13:34:57claude that connect to these MCP servers
13:35:00and discover what's available and then
13:35:03they can invoke all of the capabilities
13:35:05when needed. So they can call the
13:35:07accurate tool or they can call the
13:35:10accurate prompts all of that. And the
13:35:14third thing is the protocol itself MCP.
13:35:17The protocol handles all of the boring
13:35:19stuff. It will list the available tools.
13:35:23Then it will tell you how to call those
13:35:25tools, how to stream those results back.
13:35:27All of that that's handled by MCP.
13:35:30Obviously, the benefit of MCP is that
13:35:32you build a tool once and use it with
13:35:35any MCP compatible model. And the model
13:35:38doesn't need to know how a tool works
13:35:40internally. It just needs to know its
13:35:43interface. And obviously, you can
13:35:44compose multiple servers together. Your
13:35:47AI will have access to everything. For
13:35:49example, in this image, the AI can have
13:35:51access to Slack, Gmail, database,
13:35:55GitHub, or any other web API. We are
13:35:58especially going to be focusing on MCP
13:36:00client because MCP servers are not
13:36:03really a part of AI coding agents,
13:36:04right? MCP client is. So the MCP client
13:36:08usually does connection, discovering,
13:36:11translation, and returning the results
13:36:14back to the LLM. Let's understand them.
13:36:17The first one is connection and the MCP
13:36:20client needs to connect to MCP servers.
13:36:22How do you connect? There are two ways
13:36:25that are mainly used. The first one is
13:36:27standard input output. So if you have a
13:36:30server that can be run locally, you can
13:36:33just pass in the command and the
13:36:34arguments and then the MCP server will
13:36:37connect MCP client will connect to that
13:36:39MCP server. And the second one is if you
13:36:42have a URL maybe it can be HTTP or SSE
13:36:46which is server sent events which is
13:36:49essentially a one directional web
13:36:51soocket. You can learn more about it by
13:36:53asking an LLM or our own coding agent by
13:36:56the way. So these are the two methods
13:36:59that we are going to support as well.
13:37:00The second part of MCP client is
13:37:03discovery. It can discover what's
13:37:05available. What tools does do I have
13:37:07access to? What can I call? So that's
13:37:10one thing we're going to look into. The
13:37:12third one is translating the LLM's
13:37:14intent into actual tool calls with
13:37:16proper JSON RPC formatting. That's not
13:37:19something we have to look into because
13:37:22we're not going to be creating the MCP
13:37:24client from scratch. We're going to be
13:37:26using a package for this and the package
13:37:28is fast MCP. So similar to fast API,
13:37:32fast MCP also exists and it can help us
13:37:35create MCP servers as well as MCP
13:37:39clients. We are only going to be
13:37:40focusing on creating a simple MCP client
13:37:43and obviously we just have to return the
13:37:45result back to the LLM in a format it
13:37:48understands and we have all of the setup
13:37:50done, right? We have created our own
13:37:53built-in tools before. We can just
13:37:55leverage all of the infrastructure for
13:37:57that. just add our MCP related stuff and
13:38:01pass it to register it to our tool
13:38:03registry as easy as it gets. So now
13:38:07let's go ahead and code it up. The first
13:38:09thing I want to do is update the
13:38:11configuration because there are certain
13:38:13details I would require from the user
13:38:15about configuration of an MCP server.
13:38:18For example, I would like to know if
13:38:20they're using the HTTP or SSE transport
13:38:23or are they using the standard input
13:38:25output transport. If they're using
13:38:26standard input output, they have to
13:38:28specify the command, the arguments, all
13:38:31of that. And that's why we're going to
13:38:33go to our schema or let's say config. py
13:38:37and here we're going to create an MCP
13:38:39server configuration. So let's have MCP
13:38:43servers which is going to be a
13:38:45dictionary of string, MCP server config.
13:38:50So in the string part you're going to
13:38:52specify the name of the MCP server and
13:38:54the value is going to be a class that we
13:38:57are going to create which lists out all
13:38:59the other things for example what is the
13:39:02command what is the URL so that we can
13:39:05infer if we're using standard input
13:39:07output or HTTP or SSE after that we're
13:39:11going to have a field here by default
13:39:14it's going to be MCP server config
13:39:16that's the value and obviously this
13:39:19needs to be a dictionary. So let me just
13:39:21pass in dictionary like that. Now I can
13:39:23take this class MCP server config.
13:39:27I'll extend this to base model and then
13:39:30we can define all of the parameters
13:39:32here. The first one is going to be
13:39:35enabled which is going to be a boolean
13:39:37value and by default it is true. In or
13:39:41in our config file you can enable or
13:39:43disable an MCP server as you like it.
13:39:47even cursor you're able to do that by
13:39:50going to mcp.json JSON file and enabling
13:39:54or disabling something. Then you have
13:39:56startup time out seconds which is how
13:40:00much time should we wait for this MCP
13:40:02server to load up and if it doesn't
13:40:04complete in these 10 seconds or let's
13:40:06say whatever the user specifies then
13:40:08we'll just say that the server did not
13:40:11load up then we have things related to
13:40:14standard input output transportation. So
13:40:16when you have a standard input output,
13:40:19you're really having a local server and
13:40:22if you have a local server, you want to
13:40:24specify how you can run that local
13:40:27server. So the first thing you have is a
13:40:29command and then you have a string or a
13:40:31null value and by default it is null.
13:40:34Then you have args which is a list
13:40:37string and by default it's just going to
13:40:40be an empty list. ARGs is just like
13:40:44whatever arguments you need to pass in.
13:40:46For example, if you have an MCP server
13:40:48hosted, let's say on npm and actually
13:40:52let me show you an MCP server that we're
13:40:55going to use which is at the rate model
13:40:57context protocol forward/server file
13:40:59system. This is created by the official
13:41:02model context protocol team. And this is
13:41:05just a NodeJS server implementing MCP
13:41:08for file system operations. It has all
13:41:10of these features and we'll be able to
13:41:12see all of the features. So to run this
13:41:15particular file system, what we are
13:41:17going to do is pass in something like
13:41:19npx and then you have let's say dashy
13:41:23and then you have the name of this
13:41:26server. This is how you can run this
13:41:28model context protocol through standard
13:41:32input output. Then you have the
13:41:34environment variables just in case you
13:41:36have to set up any environment variables
13:41:38because many times let's say you're
13:41:40interacting with Slack or Gmail. Some
13:41:43environment variable is going to be
13:41:44required because you just can't get
13:41:46access to someone's email account like
13:41:48that. You need to pass in some data. For
13:41:51example, the API key or the username and
13:41:54password whatever is required. Now we'll
13:41:57pass in the default factory here which
13:41:58is going to be a dictionary. And then we
13:42:01have the current working directory path
13:42:03or null. And by default, it is null.
13:42:06After that, we're going to have the HTTP
13:42:09or SSC transport. And all we require
13:42:12here is a URL. This URL is going to
13:42:15expose all of the information. And you
13:42:18can find multiple MCP server URLs
13:42:20online. Or you can create your own MCP
13:42:23server, run it locally, and you'll have
13:42:26something like http col// localhost
13:42:308,000/
13:42:32ssc and that is going to be your URL. So
13:42:36once you do that, the MCP server will be
13:42:39able to contact and list out all of the
13:42:42tools that are present within your
13:42:43server. Now I just have to validate the
13:42:46transport that's present because either
13:42:49the command and the ars should be
13:42:51specified or the URL should be specified
13:42:54because if both are not specified in
13:42:56that case we don't have a valid MCP
13:42:58server we do need a transport layer
13:43:00present so we'll just have at the rate
13:43:03model validator the mode is going to be
13:43:07after now let me import model validator
13:43:10from pyantic and now we can just create
13:43:13a function validate
13:43:15transport
13:43:16then we have self and we're just going
13:43:18to return an MCP server config from here
13:43:22and since we are returning an MCP server
13:43:25config from here and the function is
13:43:28defined within the class itself then we
13:43:30are just going to import from future
13:43:34annotations once we do that we'll be
13:43:36able to use the MCP server configure and
13:43:40now we have to check if the command is
13:43:42specified or the URL is specified. So
13:43:44I'll just have has command equal to self
13:43:47dot command is not none. If if the
13:43:51command is not none, we have a command.
13:43:53Otherwise, you know, we have a URL which
13:43:56is self do. URL is not none. Now if we
13:44:01do not have a command and we do not have
13:44:03a URL, we have a problem. So in that
13:44:07case we'll just raise a value error and
13:44:10we'll say MCP server must have either
13:44:14command which is standard input output
13:44:17or it should have URL which is HTTP or
13:44:21SSE. Great. Now let's say we have both
13:44:24that is also a problem. If has command
13:44:26is true and has URL is also true, we
13:44:29have to say again we are raising a value
13:44:32error. But this time we should say MCB
13:44:35server cannot have both command
13:44:39and URL. Okay. So that is a validation
13:44:43we have to do. Rest everything is fine.
13:44:45We have passed in the MCP server config
13:44:48here. And now in our config.tml TML if
13:44:50we want to ever have an MCB server what
13:44:54we can do is specify something like
13:44:56this. So you have MCP servers dotfile
13:45:00system. MCP servers refers to this
13:45:02particular attribute and since this is a
13:45:06dictionary in this Toml we have to
13:45:08create a table. Table just refers to a
13:45:10dictionary in TUML. All right. So we
13:45:14have MCP servers dotfile system. So file
13:45:17system is going to be the string key
13:45:20over here and then the server
13:45:22configuration is specified below where
13:45:24we have command equals to npx and rest
13:45:27everything is arguments. So we have
13:45:28dashy at model context protocol server
13:45:32file system and then you also specify
13:45:34the path where this server file system
13:45:37is initialized. In my case it's just
13:45:40going to be desktop AI agent and a
13:45:43temporary folder. But you can also
13:45:45remove this temporary folder. No
13:45:46problem. If you don't specify this path,
13:45:48what will happen is that the LLM will
13:45:51try to access outside of this current
13:45:53working directory. And once we add stuff
13:45:57related to approval system and
13:45:59sandboxing, it will cause problems
13:46:02because we don't want to allow any MCP
13:46:05server to create any file just like that
13:46:08outside of our current working directory
13:46:10because then it can lead to some pretty
13:46:13horrible things especially if you have a
13:46:15wrong MCP server installed. So yeah,
13:46:18this configuration is specified. It
13:46:20should be present within this MCP
13:46:22servers. Now I have to take this MCP
13:46:24servers register it with tool registry.
13:46:27But to register the MCP server first I
13:46:31need to get access to all of the tools
13:46:33that are available within this MCP
13:46:35server. And for that particular reason
13:46:37I'm going to open up the sidebar. Let's
13:46:40close all of the folders and I'll go to
13:46:42tools. Specifically I'm going to create
13:46:45a new folder called MCP. And within that
13:46:48we have a client py. So we're going to
13:46:51have an MCP client created which will
13:46:53interact with the MCP server and as I
13:46:57mentioned we're going to use fast MCP as
13:47:00the client of choice to simplify things
13:47:02for us otherwise it will be a,100 1,500
13:47:06line code piece and I don't want to
13:47:09spend time on MCP. So let me install
13:47:12fast MCP quickly. Let me exit out of
13:47:14this part. Then I'll just do pip install
13:47:17fast MCP. Now that it's installed, I can
13:47:20go ahead and create a class. Let's call
13:47:22this MCP client. Then I'm going to have
13:47:26an init function. And within this init
13:47:29function, we're going to first get the
13:47:30name of the MCP server. After that,
13:47:33we're going to get the configuration
13:47:36which is specifically MCP server config.
13:47:39So this time we are not having the
13:47:42config class here. Instead, we are
13:47:44having the MCP server config. So
13:47:46whenever we instantiate MCP client we
13:47:48expect to get this MCP server config
13:47:51only and this is why we are having the
13:47:54name here and then the config because
13:47:56remember in our configuration the MCP
13:47:59server config does not have any name
13:48:01attribute. All it has is the details
13:48:04related to MCP server config and the
13:48:07value or the key here is going to be the
13:48:10name in the dictionary. So we'll pass
13:48:12both of them when this MCP client is
13:48:15going to be instantiated. Another thing
13:48:17we'll require is the current working
13:48:19directory. So let me import that from
13:48:21path. Obviously all of this is going to
13:48:24return nothing. So let me just have none
13:48:27returned here. Now let's initialize
13:48:29everything. We have self.name equals to
13:48:31name. self do.config is equal to config.
13:48:34self do.t current working directory is
13:48:36equal to current working directory. And
13:48:38we also going to create a status of the
13:48:41MCP client. Remember this MCP client
13:48:44that we're creating is just for one MCP
13:48:47server. It is a simple MCP client that
13:48:50keeps track of the connection with one
13:48:53MCP server. We're going to create
13:48:55multiple of these MCP clients. So there
13:48:59will be multiple MCP connections.
13:49:01Obviously in REI coding agent, we only
13:49:03have one MCP server defined. But in the
13:49:07future, you can just go ahead and add as
13:49:09many MCP servers as you want. And
13:49:11there'll be multiple MCP client
13:49:12connections. We're also going to store
13:49:14the status of this MCP server. Is this
13:49:18MCP server connected, disconnected, or
13:49:20is it connecting or is it in an error
13:49:23state? What is it? So that we can
13:49:24display it out to the user whenever they
13:49:27want. Especially when we add support for
13:49:29slash commands. For example, when we try
13:49:31to do something like Python main. py you
13:49:34know these commands that are present you
13:49:36can also do something like /mcp to know
13:49:39all the mcp servers connected or
13:49:41slashtools to know all of the tools that
13:49:43are connected and those tools will also
13:49:45list out the mcp servers so here let's
13:49:48just create an enum of mcp server status
13:49:52this is going to be a string enum we've
13:49:54created enums many times before so I'll
13:49:57go fast we have disconnected which is
13:49:59equal to disconnected
13:50:02then we have connecting which is equal
13:50:05to connecting. Then we have connected
13:50:08which is equal to connected and then we
13:50:11have error. Then we just have the error
13:50:13here. Great. Now we can also initialize
13:50:16the status here which is self status
13:50:18equal to MCP server status dot
13:50:22disconnected. So whenever MCP client is
13:50:24instantiated it will be disconnected.
13:50:27You have to explicitly call the method
13:50:29of connect. Once connect method is
13:50:31called, it will go ahead and start the
13:50:35MCB server. Try to connect it. Let's
13:50:37also create a client here. So we have
13:50:39self.client and client is private so
13:50:42that it cannot be exposed outside. Then
13:50:44you have client from fast MCP. Let me
13:50:47import that at the top. From fast MCP,
13:50:50we import client. Then here the type of
13:50:55this is going to be client or null. And
13:50:57by default it is null because yeah we
13:50:59have not instantiated anything to
13:51:01instantiate we will call the connect
13:51:03method. So let's define the connect
13:51:05method as well. And by the way the
13:51:07reason we're doing connect method not in
13:51:10the constructor is because it needs to
13:51:13be an asynchronous function and it can
13:51:15obviously not be an asynchronous
13:51:17function. So we have async def connect.
13:51:19I'll pass in self and then we return
13:51:22nothing here. And first I just want to
13:51:24check if the server is already
13:51:26connected. If the server is already
13:51:28connected, connect was called for no
13:51:31reason. I don't want to establish
13:51:33duplicate connections. So I'll just
13:51:35return here itself. So if I have self
13:51:38dot status is equal to let's say MCP
13:51:42server status dot connected in that case
13:51:45I'll return nothing to do here. But if
13:51:48it's not connected in that case I'll set
13:51:51the status to connecting explicitly. So
13:51:54MCB server status dot connecting. After
13:51:58that I'll try to figure out what
13:52:01transport we are in. Based on the
13:52:03transportation layer we are going to
13:52:05pass it in to client. So we have
13:52:07self.client instantiated here and the
13:52:10client needs to be created here. But to
13:52:12the client we need to pass in the
13:52:14transportation layer. Right? because it
13:52:16only makes sense. How should the MCP
13:52:18server connect to a third party tool or
13:52:22a third party server? What is a
13:52:24transportation layer? So, we have to
13:52:25figure that out and pass in the details.
13:52:27And we have extracted all of the details
13:52:30through the MCP server config.
13:52:33All we need to do is wrap it with fast
13:52:36MCP related methods. So we'll have def
13:52:40create create transport which is a
13:52:42helper function and then we have self
13:52:45and here we are going to either return
13:52:47standard input output transport or the
13:52:51SSC transport. Now let's import both of
13:52:54them from fast MCP. So we'll have from
13:52:58fastmcp.client.transports
13:53:00transports we'll import SSE transport
13:53:04and standard input output transport.
13:53:09So these two things will do all of the
13:53:12heavy lifting for us. All we have to do
13:53:14is initialize them. So I'll first check
13:53:17if self.config do command is present. If
13:53:21command is present in that case we have
13:53:23the standard input output transport. So
13:53:26we can just go ahead and return standard
13:53:29input output transport here. The first
13:53:31thing we'll pass in is command which is
13:53:33self.config docomand. Then we have args
13:53:36which is a list of self.config.s.
13:53:40Then we have the environment variables.
13:53:42Now what environment variables do we
13:53:44pass in? Well the environment variables
13:53:47should be whatever the user already has
13:53:50in their system. whatever environment
13:53:52variables are present along with the
13:53:54updated environment variables that have
13:53:57been passed through this configuration.
13:53:59So here I can just have environment is
13:54:01equal to os environment.copy.
13:54:04This is something we've already looked
13:54:06into. This gives us all of the
13:54:08environment variables and now I can just
13:54:10do environment.date
13:54:12and update it with the new environment
13:54:15variables that have been passed in. So
13:54:18yeah, there we go. Now I can take this
13:54:21environment variable and pass it in over
13:54:23here. Then I can pass in the current
13:54:25working directory which is equal to and
13:54:28I'll pass in self.config.curren
13:54:32working directory and it needs to be
13:54:34converted into a string and if it's not
13:54:37specified because it is an optional
13:54:39value right if it's not specified in
13:54:42that case we are just going to have
13:54:44self.curren working directory and that
13:54:47also needs to be a string. So I'll just
13:54:50pass it in here. This cell.curren
13:54:52working directory is coming from here.
13:54:54It's not coming from the configuration
13:54:56system. This is the real current working
13:55:00directory. Now let's say the command is
13:55:02not specified. In that case, it needs to
13:55:05be the URL, right? Because we have
13:55:08validated that either of those transport
13:55:10methods are present. So here we'll just
13:55:14say return S s transport and pass in the
13:55:18URL which is equal to self do.config dot
13:55:22URL. Awesome. Now I can call this create
13:55:26transport here and pass it in for the
13:55:29client. So we have self.create
13:55:32transport. Now the next thing we have to
13:55:34do is connect to the client. And for
13:55:37this we'll be using the context manager.
13:55:41If you remember there was something like
13:55:43with self do.client
13:55:46as client something like that. We're
13:55:48going to use that but instead of using
13:55:50this with command we're going to call
13:55:52the internal function that supports it.
13:55:54We're going to do await. This is why we
13:55:57needed an asynchronous function. We have
13:55:59await self.client
13:56:02dot and then you can just call a enter
13:56:05like that. So a enter is just
13:56:07asynchronously entering and we have
13:56:10created this in agent. py as well. So
13:56:12I'm not going to dive much into it
13:56:14because I've already explained how this
13:56:16works. But essentially we have called
13:56:18this a enter instead of manually calling
13:56:21with something. The reason we're doing
13:56:24it is because we are connecting here but
13:56:26we don't want to close off this client
13:56:28connection as soon as this function gets
13:56:32over. Because think about it if you do
13:56:34something like with self.client as
13:56:37client and do your stuff here and on the
13:56:39once this function gets over this with
13:56:42is also completed. We don't want that
13:56:45because once we have connected we want
13:56:47this connection to persist and that's
13:56:49why I'm calling a enter here. We'll call
13:56:52a exit explicitly when we have to
13:56:55disconnect and we'll create a function
13:56:57for that as well. Since we already have
13:56:59async await, let me just pass in a try
13:57:01and except as well. So I'll have try
13:57:04except exception as e. And obviously if
13:57:08we run into any sort of error, the
13:57:10status will be converted to mcp server
13:57:14status dot error. And maybe we can also
13:57:17raise the exception from here. We don't
13:57:20really need e here. Let's just do accept
13:57:23exception. Great. So yeah, we have
13:57:26created the client connection. We have
13:57:28entered it. Now we have to call the
13:57:31tools related to this client. So the
13:57:34first one is well, we want to discover
13:57:36the tools, right? If you want to
13:57:38discover the tools, we can just call the
13:57:40function self.client.list
13:57:44tools and that will give us all of the
13:57:46tools. I can await it and then I have
13:57:48all of the tool results with me. And now
13:57:52if I want to know what tools there are,
13:57:54I can just do for tool in tool result.
13:58:00And then maybe let's also maintain a
13:58:02dictionary for all of the tools that we
13:58:04have, right? That makes a lot of sense
13:58:07because once we have all of the tools,
13:58:09we can easily register all of the tools.
13:58:13So think about it. We just call the
13:58:14connect function. Once we've called the
13:58:16connect function, we can call the
13:58:20register function and that will register
13:58:22all of the tools for us. Now remember
13:58:24each MCP server can have multiple tools
13:58:27and then we can have multiple MCP
13:58:29servers. So this one just has multiple
13:58:32tools. So this is what we're going after
13:58:34as of now. The type of this is going to
13:58:37be a dictionary where we have a string
13:58:40as the key and the value is going to be
13:58:43MCP tool info class that we will create.
13:58:46That MCP tool info will store all of the
13:58:49information regarding an MCP tool like
13:58:50name, description, the schema, what
13:58:53server does it belong to, all of that.
13:58:55So we'll have class MCP tool info and
13:58:59this needs to be a data class. Let's
13:59:01import from data classes data class. And
13:59:04then we can just have name as string
13:59:07which is the tools name. Then you have
13:59:09the description of that tool. This is
13:59:11quite similar to all of the registration
13:59:14or all of the tools that we saw in tool.
13:59:16py or base. py I don't remember. Yeah,
13:59:20this one. So we have name description
13:59:23all of that and yeah these things will
13:59:26be useful when we try to create a fast
13:59:28MCB tool. I'll talk about that in let's
13:59:31say 10 minutes. So I have input schema
13:59:34next which is a dictionary of string,
13:59:37any let me import any from typing. And
13:59:40then we have a field where we have to
13:59:44import it from data classes. This time
13:59:47it's not pyantic it's data class. And
13:59:50now I can pass in default factory which
13:59:53is a dictionary and then the server name
13:59:56which is by default an empty string. So
13:59:59these two things are absolutely
14:00:00essential but for input schema and
14:00:02server name we do have default values
14:00:06present. Now we can take this MCP tool
14:00:08info and pass it in as the value for
14:00:12this tools. And now we can go ahead and
14:00:15update this dictionary. So we'll have
14:00:18self dot tools at and let's say
14:00:21tool.name because tool result will
14:00:24return the tool to us. So you can think
14:00:27of tool result as a list of
14:00:28dictionaries. We're going over each
14:00:32dictionary item and then we're printing
14:00:34out its name. That is going to be the
14:00:36key, right? Because that's the unique
14:00:38identifier for the tool. And then we're
14:00:42going to set it equal to MCP tool info.
14:00:46And now we'll pass in the name of the
14:00:48tool which is tool.
14:00:51Then we have the description which is
14:00:53tool.escription.
14:00:55And if it's not specified, it's just an
14:00:57empty string. Then we have input schema.
14:01:01And I messed up the indentation.
14:01:04Next thing is the input schema, which is
14:01:06equal to tool do.input schema. This is
14:01:09how we get it from fast MCP. And we'll
14:01:12just check if it has the attribute of
14:01:15tool input schema, then we use that.
14:01:19Otherwise, an empty object is good
14:01:22enough. And finally we have the server
14:01:25name which is just equal to self do.name
14:01:29because self dotname that we are getting
14:01:32this thing is the name of the server.
14:01:34Once all of this happens this entire for
14:01:37loop is over the mcbp server is
14:01:39connected. So we'll have self dot status
14:01:41is equal to mcp server status
14:01:44dotconnected. And yeah that looks good
14:01:47to me. That's about the connect
14:01:48function. Now before I forget I'll just
14:01:51create the disconnect function as well.
14:01:53So I'll have async def disisconnect. We
14:01:56have self we'll return nothing and then
14:01:58we'll just check if self.client is
14:02:00present. If it is then I'll just do
14:02:03await self.client
14:02:05dot and then we'll just call a exit.
14:02:09Similar to how we called a enter we are
14:02:11just calling a exit. And now we remember
14:02:13in a exit there were three parameters
14:02:16that we needed to pass in. So I'll just
14:02:18pass in null for all of them. I'll just
14:02:21go to agent.py to demonstrate what I'm
14:02:23talking about. These three exe exception
14:02:25type exception value and execution
14:02:28exception trace back. So we just pass in
14:02:30null values for all of them. And after
14:02:32we have done that we can go ahead and
14:02:35set the client to null. Also I can go
14:02:40ahead and set the tools to be cleared
14:02:43off. So I have self do.tools tools dot
14:02:46clear. So it just empties all of the
14:02:49tools that we have. And finally, we'll
14:02:52just ask self dot status is equal to MCP
14:02:55server status dot disconnected. Again, I
14:02:58just have to remember to call this
14:03:00disconnect function. Also, it should be
14:03:02self.client is equal to null. And that's
14:03:06about the MCP client. This is really all
14:03:08we'll need. Now remember, this is for
14:03:11one MCP server. You can have multiple
14:03:14MCP servers. So I'm going to create a
14:03:16class that composes this MCP client. So
14:03:20MCP client is used within it for every
14:03:23server that's present within this
14:03:27configuration system. So let's say we
14:03:29have another one here and it's called
14:03:32file system one. We'll have another MCP
14:03:35client created for it. So we need an MCP
14:03:38server manager, right? And that's what
14:03:39I'm going to create here. Let's call
14:03:41this MCP manager. py or you can just
14:03:45call it manager whatever works for you.
14:03:48And this is very easy class MCP manager.
14:03:51Then we are going to have the init
14:03:53function here. Again just like before
14:03:56we're going to get the configuration but
14:03:58this time we're going to get the actual
14:04:00configuration because we do want the
14:04:02actual configuration. Now we can't just
14:04:04have one MCP server config like we did
14:04:07in MCP client because we are going to
14:04:10manage multiple MCP servers here. So
14:04:12we're going to get multiple MCP server
14:04:15configs and they're all present within
14:04:17this config. So let's have self doconfig
14:04:20is equal to config and then let's also
14:04:24initialize a list of clients that we
14:04:27have. So we'll have self_client
14:04:30which is equal to a dictionary and it's
14:04:34going to be a string mcpclient.
14:04:37Let's import it. And by default it is an
14:04:40empty dictionary. So the string here is
14:04:43going to refer to the mcp server name.
14:04:46And mcp client is well whatever client
14:04:49is there. So for file system we'll have
14:04:52one mcp client. For the next MCP server
14:04:55that's added, there's going to be a
14:04:57second MCP client and the name will
14:04:59again change. There are mainly two
14:05:01functions that we are interested in
14:05:03right now. The first one is
14:05:05initialization and the second one is
14:05:09registering all of the tools. So
14:05:11initialize will just initialize and
14:05:14connect to all the configured MCP
14:05:16servers and add it to this client's
14:05:19dictionary. And the register is just
14:05:21going to parallelly register all of the
14:05:24MCB tools with our tool registry.
14:05:27Simple. Let's go ahead and create async
14:05:30def initialize
14:05:32which is just going to have a self. It's
14:05:35going to return nothing. And the first
14:05:37thing I want to check here is if it's
14:05:39already initialized or not. So for that
14:05:41I'll have to maintain a variable if it's
14:05:44already initialized or not. By default
14:05:46it is going to be false. That means when
14:05:48the MCP manager starts, it's not
14:05:51initialized. And if it is initialized,
14:05:54then we've created multiple MCP
14:05:56connections. We don't want that. That's
14:05:58why I'm just going to return out
14:06:00quickly. Now, let me get access to the
14:06:02MCP configurations. And to access that,
14:06:05I can just do MCP configs equal to
14:06:07self.config.m MCP servers. Those are all
14:06:12the configurations, right? And it's a
14:06:13dictionary of string, MCP server config.
14:06:16the string refers to the server name.
14:06:18And if there are no MCP configs, then
14:06:21we'll just return very quickly. There's
14:06:22nothing to do. But if there are MCP
14:06:25configs, then I'll go over each server
14:06:28in this MCP configs dictionary. So we
14:06:32get the name of the server. We get the
14:06:34server config in MCP configs do items.
14:06:39And then I'll just check if the server
14:06:41config is enabled or not. If the server
14:06:44config is not enabled in that case we
14:06:49will just continue. I don't have to do
14:06:51anything about it. It's just disabled.
14:06:54Why why will I instantiate it for no
14:06:56reason right? And now let's say it is
14:07:00enabled. Then I'll just do self.clients
14:07:03at name is equal to and I'll establish
14:07:06an MCP client. I'll pass in the name now
14:07:09which is going to be the tool which is
14:07:12going to be the server's name. Then we
14:07:15pass in the entire MCP server
14:07:16configuration which I'll do here. So
14:07:20let's say config is equal to server
14:07:22config. And then finally we pass in the
14:07:24current working directory which is self
14:07:26do.config dot current working directory.
14:07:30Let me prefix this with current working
14:07:32directory. So self.config.curren current
14:07:35working directory is different from
14:07:37serverconfig.tcurren working directory.
14:07:39I think we've used self.config.curren
14:07:42working directory multiple times. So no
14:07:45need to explain this further. But
14:07:48remember all we have done here is
14:07:50created mcp clients for each one. We've
14:07:53not initialized them yet. The reason we
14:07:56have not initialized them is let's say I
14:07:58call dot initialize here or whatever the
14:08:02function was connect. If I call connect
14:08:04and await, I would have to wait for each
14:08:07MCP server to complete so that we can go
14:08:10to the next one. So if I have 20 MCP
14:08:12servers, I'll have to wait for 20 MCP
14:08:15servers to connect. So one MCP server
14:08:18connects, it takes 10 seconds, let's
14:08:20say, and then the second one connects.
14:08:22That that takes 10 seconds again. Then
14:08:24the third one, and then the fourth one.
14:08:26So just imagine if you have 100 MCP
14:08:28servers and each takes 10 seconds, you
14:08:31you have to wait 1,000 seconds. that's
14:08:33more than 13 14 minutes. We don't want
14:08:36that experience. So what I'm going to do
14:08:39is just have the MCP client stored here.
14:08:42After that, what I'm going to do is
14:08:44create a bunch of tasks whose entire job
14:08:47is just to connect. So we'll have for
14:08:50name, client in self.clients do items
14:08:55because we just appended to this for
14:08:57loop because we just appended within
14:08:59this for loop all of the dictionary
14:09:01items. What I'm going to do here is call
14:09:05client, which is this client dot
14:09:08connect. And all of this needs to be
14:09:10within a list. So these are all of the
14:09:14tasks. And one thing to remember here is
14:09:16just calling connect is not enough
14:09:18because if you're just calling
14:09:20client.connect, what's happening is
14:09:22we're not taking into account the
14:09:24timeout. And the user might have
14:09:25specified something related to the
14:09:27timeout as well, which is startup
14:09:29timeout seconds. So we have to wait for
14:09:31those many seconds as well. So I'll have
14:09:34here async io. Let's import async io.
14:09:38And then I'll call dot wait for then
14:09:42I'll pass in the client.connect within
14:09:44the specific function. And then I'll
14:09:46call await. So these are all of the
14:09:49connection related tasks that we have.
14:09:52Also I'm not liking that we are not
14:09:54getting any sort of auto correct here so
14:09:56to say and the reason for it is clients
14:09:59is not defined properly. We don't have
14:10:02to do is equal to we have to do the type
14:10:05because the type of self.clients is
14:10:07this. And now we should be getting the
14:10:10auto helper functions whatever you want
14:10:12to call them. I forgot what it's called
14:10:13but yeah these are all of the connection
14:10:15tasks that we have. All I need to do is
14:10:19gather all of these tasks and execute
14:10:21them together. So I'll do await async
14:10:24io.
14:10:26And I'll pass in all of the connection
14:10:29tasks here. And if they return any
14:10:32exceptions, it's fine. I'll handle them.
14:10:35So it's true. So that the user also
14:10:38knows that some sort of error occurred
14:10:40here. And if it goes well, then we are
14:10:42just going to have self.initialized
14:10:44equals to true. It's instantiated
14:10:48and that's it. That's the initialize
14:10:50method. The point of doing these two
14:10:53lines is just so that all of the tasks
14:10:55are parallelly executed not executed
14:10:59serially. After this we are going to
14:11:01have the register tools function and
14:11:04this should be quite easy. We're going
14:11:06to get a self. We're going to get the
14:11:07registry which is the tool registry. Let
14:11:11me import that. And now from here we are
14:11:15going to return well we can return how
14:11:17many tools we registered. So let's just
14:11:20return that. Initially the count is
14:11:22going to be zero. We registered zero
14:11:25tools but then we go over every client
14:11:29in this self.clients
14:11:31do values because remember values
14:11:34contains all of the things we care
14:11:36about. We don't care about the server
14:11:37name when we are trying to register the
14:11:39tool to our tool registry. All we care
14:11:41about is the tool related information.
14:11:43So values will give us all of the MCP
14:11:46clients and that does not contain the
14:11:48server name. Then I'll just check if the
14:11:50client status if it's not equal to MCP
14:11:53server status. Let me import that. And
14:11:56if it's not equal to connected, we just
14:11:58continue. If it's error, if it's
14:12:00connecting, I don't care. I just want
14:12:03this thing the thing the tools that
14:12:06we're trying to add to our registry to
14:12:09be already connected. Then I'll go over
14:12:12all of the tools. So we have for tool
14:12:14info in client dot tools and we don't
14:12:18have a tools exposed outside. So let me
14:12:21go to this MCP client expose out a
14:12:24property of tools. So I have at the rate
14:12:26property def tools. We're going to
14:12:30return a list of MCP tool infos. All I
14:12:35need to do here is return a list of self
14:12:37dot tools dot values. Those are our
14:12:41tools. So now in the MCP manager, I can
14:12:44get access to this tools. And for every
14:12:46tool info that we get, we have to call
14:12:50registry dotregister
14:12:52MCP tools. So that's a function we'll
14:12:54have to create because just calling
14:12:56register is not enough. The reason for
14:12:59it is when we try to display stuff on
14:13:02the UI side later on as I mentioned
14:13:04about the slash command like the slash
14:13:07tools or /mcp
14:13:09it will be easier to differentiate
14:13:11between the MCP tools and the normal
14:13:14tools if we maintain two different
14:13:16dictionaries here. So we'll have self
14:13:18dot_mcp
14:13:20tools which will also be a dictionary of
14:13:23string comma tool and then we create a
14:13:25function quite similar to register
14:13:28function but what it does is registers
14:13:30an MCP tool then we have the name of the
14:13:34tool and then you have the tool itself
14:13:37then we can just do self dot
14:13:41mcp tools then we have the name passed
14:13:44in here which is just equal to tool And
14:13:47if you want you can have logger.debug
14:13:50registered MCP tool tool.name or let's
14:13:54just say name whatever. After that we
14:13:56can go to MCP manager and call this
14:13:59register MCP tool. But before I move any
14:14:03forward I would like to merge these MCP
14:14:06tools and normal tools. For example when
14:14:10we call get this get is called within
14:14:14invoke. So when we are trying to execute
14:14:16a tool call if for an MCB tool get is
14:14:20called and we just check in self.ttools
14:14:22that's not enough because tool can also
14:14:24exist within MCB tools. So I'll have l
14:14:27if name in self dot MCP tools in that
14:14:31case I'll have return self dot MCP tools
14:14:36at that particular name. Even in get
14:14:39tools a similar thing will happen. So we
14:14:42go over every tool and try to append the
14:14:44tool. But we'll have to do that for MCP
14:14:47as well. So we'll have for tool in self
14:14:51domcp tools dot valalues tools.append
14:14:54and then we'll pass in the MCP tool. Let
14:14:57me just say that. And then we have all
14:14:59of the filtering logic here. And that
14:15:01looks good enough to me. I'll go back to
14:15:04MCP manager now. And specifically when
14:15:06we're calling register MCP tool, I have
14:15:09to pass in the name of the tool and the
14:15:11MCP tool itself. And now I'm thinking
14:15:15about it. We don't really need a name
14:15:17here. It's just same as the register
14:15:19function. So I'll just do tool dot name
14:15:22is equal to tool. And again your
14:15:24tool.name. Sorry about the confusion.
14:15:26We're just doing what we did in register
14:15:28but just storing in a different
14:15:30dictionary this time. Not in self.tools
14:15:32but in cell.mmcbp tools. Again, the
14:15:34reason we're storing an MCP tools is so
14:15:37that we have a clear differentiation
14:15:39between what are MCP tools and what are
14:15:40tools. So that if in the UI we want to
14:15:42display anything, we can display it
14:15:43nicely. To register an MCP tool, we'll
14:15:46have to pass in a tool. Now, what tool
14:15:49do we pass in? Well, for that, we'll
14:15:51have to create an MCP tool. Similar to
14:15:53all of the built-in tools we've created,
14:15:56we have to create a tool for MCP as
14:15:58well. So, I'll have MCP tool. py. And
14:16:02what I'm going to do here is just copy a
14:16:05built-in tool. Let's say something like
14:16:08list directory. As simple as it. And
14:16:11here I'm just going to paste everything
14:16:13out. Then I'm going to have no
14:16:16parameters here because remember in tool
14:16:20the schema was either a dictionary or a
14:16:23base model. And I told you a dictionary
14:16:25will be required if we have MCP servers.
14:16:28And since we have an MCP server now,
14:16:31we're going to have a dictionary as the
14:16:33schema because the user is going to
14:16:35specify in a dictionary format and we
14:16:38don't have a fixed schema of how the
14:16:40things are going to look like. For
14:16:41example, in list directory parameters,
14:16:43we know we going to always get a path.
14:16:45We know we're going to get an include
14:16:47hidden. But if the user is specifying
14:16:49their own tool, how are we going to know
14:16:51what schema it is going to be? So let's
14:16:54just let's just put it in a dictionary
14:16:56format, right? So I remove this line. We
14:16:59remove everything related to pyante. The
14:17:02tool name is now MCP tool. The name here
14:17:06is going to come from the init function
14:17:09because when you instantiate MCP tool,
14:17:12you can pass in the name of the tool. At
14:17:14this particular point within this class,
14:17:16we don't know what the tool's name is
14:17:18going to be. All we know here is kind.
14:17:20And for kind, we can pass in toolkind
14:17:23do. MCP because it's an MCP. The
14:17:27description needs to be created
14:17:30differently. It will be given to us
14:17:32through the constructor as well. So we
14:17:34have diff in it. Then we have a
14:17:37configuration passed in. Let's call this
14:17:40config. Then we are also going to get
14:17:42the tool info which is MCP tool info.
14:17:47Let's import it. Then we have the name
14:17:50of the MCP server. So that's a string.
14:17:54And this init is going to return
14:17:55nothing. Then we call super.init. That's
14:17:58good. Then we have self.toolinfo
14:18:01is equal to tool info. Remember I've
14:18:04done this as private. I don't want this
14:18:06to be accessible outside. Then we have
14:18:08self dot client which is equal to client
14:18:13and we need to get access to client as
14:18:15well. So we have an MCP client coming
14:18:18in. You'll understand why we need client
14:18:21specifically. we it will be needed in
14:18:24within execute because if we try to
14:18:25execute an MCP tool what should happen
14:18:27the MCP tool should be called so client
14:18:30has that function already that will
14:18:32simplify things for us I don't think we
14:18:35have created the call function within it
14:18:37but we will then we have self_ame is
14:18:40equal to name or we don't need to do
14:18:42self_ame just selfname is fine because
14:18:45we have to instantiate name remember we
14:18:48have to give this MCP tool a name so
14:18:50selfname will do just that and it's set
14:18:53to this name and then that looks good
14:18:56enough to me. The next thing we require
14:18:59is the description and I can set that
14:19:02over here as well. Self dot description
14:19:04is equal to self do.toolinfo.escription
14:19:09then I can also specify the schema here
14:19:11which is self do. schema is equal to and
14:19:15then the schema we already know what
14:19:18that is going to look like. We have
14:19:20discussed much about this. The type here
14:19:22is going to be object. This is
14:19:24essentially the schema that is passed to
14:19:25the LLM itself. Right? So we have type
14:19:28object. Then we have properties which is
14:19:32well based on the input schema. That
14:19:35will be the properties. So the input
14:19:37schema is present within this MCP tool
14:19:39info. Right? So I'll just do input
14:19:42schema is equal to self do.tool info dot
14:19:46input schema. or if it's not specified,
14:19:49if it's null, it's just an empty object.
14:19:52And now I can do input schema.get
14:19:55and try to get the properties object
14:19:57within it. And if it's not specified,
14:19:59it's an empty dictionary. The next thing
14:20:01we have is required. So required is
14:20:05input schema.get required. And it's an
14:20:09empty list if it's not specified. These
14:20:11are the things we d dove into when we
14:20:14were first creating the tool. So I'm not
14:20:17going to talk much about this. Then we
14:20:19also have self dot is mutating and it's
14:20:22just going to be true. Nothing else. Oh,
14:20:25I just remembered is mutating is a
14:20:27function that we'll have to create. So
14:20:30let me just create it. We have def is
14:20:33mutating. We get the parameters. We're
14:20:37going to return a boolean value from
14:20:39here. And the boolean value is just a
14:20:42true. So we have initialized everything.
14:20:45We have the schema. We have the name and
14:20:47description. We have is mutating true
14:20:49because we don't know what the MCP tool
14:20:52is going to do, right? So, we just
14:20:53assuming that it's always going to be
14:20:55true. It's always going to change some
14:20:58state. Now, within execute, we have to
14:21:00change some stuff. So, let me just
14:21:02remove the schema from here, this
14:21:04description from here, and all of the
14:21:07content within execute so that we can
14:21:09just focus on this execute function. Now
14:21:12we have to call the tool and for that
14:21:14we'll have to go into the client and
14:21:16create that function of what should
14:21:19happen when it's called because for us
14:21:21it's very easy self.client which is the
14:21:24fast MCP client already gives us a
14:21:26function called call tool. So we just
14:21:30have to pass in the tool name the
14:21:32arguments and it will call the MCP tool
14:21:35and return the result. If you want to
14:21:37know more, you can go ahead and go to
14:21:40the core operations where we have list
14:21:42tools. Then we have executing tools and
14:21:46here you can see call tool is present
14:21:48where you have whatever the function
14:21:52name is the tool name and then the
14:21:54arguments associated with it. So let's
14:21:56have async def call tool. Then we get a
14:22:01self. Then we get the tool name which is
14:22:03a string. Then we get arguments which is
14:22:06a dictionary of string, any now remember
14:22:10in our built-in tools, we called the
14:22:13tools ourself. For example, in read file
14:22:16operation, we went ahead and read the
14:22:19file. Whatever the files result was, we
14:22:22returned that. But in this MCP, we can't
14:22:26really call the tool, right? Because
14:22:28this MCP server is not with us. We don't
14:22:31have the code for it. We don't know what
14:22:33to do with it because that MCP server
14:22:36can be related to Google Drive or Slack
14:22:38or hundreds of other things. I don't
14:22:41know what it is about and I don't have
14:22:43the code associated with it. So, how am
14:22:45I going to call the tool? Fast MCP does
14:22:48because behind the scenes it has created
14:22:51a connection with that server. it knows
14:22:53what the server does. So it will just
14:22:56call that server with that specific
14:22:58tool. Whatever the server returns to
14:23:00fast MCP, fast MCP will return to us. So
14:23:03here we are just going to first check if
14:23:07the self.client is even connected and if
14:23:11it's not then we are just going to raise
14:23:14a runtime error because that is an
14:23:17issue. the client should be present and
14:23:19we'll just say not connected to the
14:23:22server and we can pass in the name of
14:23:24the server or let's say another case can
14:23:28be that the status is not connected that
14:23:31means the MCP server is disconnected in
14:23:35error state or it's still connecting
14:23:36even then we'll raise a runtime error
14:23:39but if that's not the issue we have the
14:23:41client everything is proper we'll just
14:23:43wait for this result so we'll do await
14:23:46self.client client dot call tool similar
14:23:49to what we saw in the documentation.
14:23:51I'll pass in the tool name and the
14:23:53arguments. So that is the result. Now
14:23:56that result if you notice here will have
14:24:00a data attached. It will have content
14:24:03attached and within that content there
14:24:05will be text. If we have the text I'll
14:24:08just append it to our output and if it's
14:24:11any other thing I'll just convert it
14:24:12into a string and return it. I'm not
14:24:14looking into much formatting issues. The
14:24:18LLM will be able to understand
14:24:20everything. So, we'll just go over every
14:24:22result within this result dot content
14:24:26because that's what matters.
14:24:28Result.contain
14:24:29contains a list and within that we can
14:24:32have the text or any other data point.
14:24:36We'll just extract that. We're not
14:24:38interested in result data where which
14:24:41gives us access to structured data
14:24:43because I'm more interested in the text.
14:24:46That's it. Let me say the output is
14:24:48equal to an empty list. And if this item
14:24:53has the attribute of text, in that case
14:24:56I'll just do output dot append and pass
14:25:00in the item.ext. Otherwise, it doesn't
14:25:03have a text attribute. I don't know what
14:25:05it is. So I'll just do output.append
14:25:08convert this item into a string and just
14:25:10return that. So finally from here we'll
14:25:13return the output which is going to be
14:25:17back slash.in
14:25:19dot join and I'll pass in the output.
14:25:24Another thing I would like to share is
14:25:26if it's an error or not. And the good
14:25:29thing for us is that this result does
14:25:32contain the variable related to error.
14:25:34So if it's an error, it will return
14:25:36result dot is error set to true. We have
14:25:39an error, a syntax error because I
14:25:41forgot to put the colon here. And that's
14:25:43it. Again, if you want to do more stuff,
14:25:46you can look into the documentation.
14:25:48Fast MCP is great. You'll get all of the
14:25:51documentation properly listed out here
14:25:53and you can create your own MCP client,
14:25:56more complex and more functional. Now
14:25:59all I need to do is in MCP manager I
14:26:02have to call this particular tool. Now
14:26:04all I need to do is go to MCP tool and
14:26:07execute this function. So I'll have a
14:26:09try and accept block here because as we
14:26:13saw it can return an exception. If it
14:26:15does we'll just do return tool result
14:26:18dot error result and then I'll pass in
14:26:21MCP tool failed and I'll attach the
14:26:25error message. So let's have exception
14:26:28as e and in the try block we'll just try
14:26:30to do result is equal to await self
14:26:34dotclient
14:26:35dot call tool then I'll pass in the tool
14:26:39name which is self dot tool info dot
14:26:42name and the arguments is just
14:26:44invocation dot parameters right because
14:26:48the llm can still give us the parameters
14:26:52here that is going to be the arguments
14:26:54that we want to call this tool with now
14:26:57this result returned to us is a
14:26:58dictionary. It can either be an error or
14:27:00an output. So I'll extract both of them.
14:27:02I'll have result.get output and if it's
14:27:05not specified it's an empty string. Is
14:27:08error is equal to result.get
14:27:11is error. If it's not specified I'll
14:27:13just default to false and then I'll just
14:27:16say if it is an error is true. In that
14:27:19case, I'll return tool result dot error
14:27:24result and I'll pass in the output. And
14:27:28if it's not an error, then I'll just do
14:27:30return tool result dot success result
14:27:34and I'll pass in the output again. So
14:27:36that's it for this execute function. Now
14:27:39what I need to do is use this MCP tool
14:27:42within this MCP manager because that's
14:27:44the point. We created this MCP tool
14:27:46because register MCP tool required a
14:27:48tool instance. So here I'll create MCP
14:27:53tool equals to and let's create an MCP
14:27:56tool. Let me import it. And then I'll
14:27:59pass in the tool info which is just tool
14:28:02info here. Then I'll pass in the client
14:28:05which is the client year. Then we have
14:28:08the config which is self.config.
14:28:10And then I need to pass in the name. The
14:28:13name here is going to be just the tool
14:28:16info.name. So that is the tool's name.
14:28:19But many times it can be the case that
14:28:22the tool names overlap. And if the tool
14:28:24names overlap that is a problem because
14:28:28the way we are storing information,
14:28:30we'll just override other tools. So what
14:28:33I'm going to do is prefix it with the
14:28:35name of the server. So I'll have here an
14:28:38string where we have tool info.name name
14:28:40which is the name of the tool and I'll
14:28:43prefix with the name of the server which
14:28:46is client name. So the server name
14:28:51underscore underscore and then the tool
14:28:53name. Then I'll pass in this mcp tool
14:28:56here and then I'll just do count plus
14:28:58equals 1. And finally after this nested
14:29:01for loop I can return the count. That's
14:29:04it. So that is about the register tools.
14:29:07If I want to call this register tools,
14:29:10where do I call it? Well, first I also
14:29:12have to initialize it, right? So, what
14:29:14I'll do is take this class MCP manager.
14:29:18Go to our session py because that's
14:29:20where everything is. And here first I'll
14:29:23have self dot mcp manager is equal to
14:29:27mcp manager. I'll just create an
14:29:30instance. Then I have to pass in the
14:29:32configuration which is self.config. And
14:29:35now I can just take this MCP manager. So
14:29:38I have self.mcp manager dot and first I
14:29:41have to call initialize. And this
14:29:43initialize is co- routine. And if I have
14:29:46a co- routine, I cannot call it within
14:29:49in it. So to make this work, what I can
14:29:51do is again have async def initialize
14:29:54function here. Then we're going to have
14:29:56a self. Then we're going to return
14:29:58nothing from here. But I can just call
14:30:00the self.mcp manager.initialize.
14:30:04So once we have initialized I can just
14:30:06do self.mmc manager dotregister the
14:30:09tools and then I can pass in the entire
14:30:11tool registry. So I have self dot tool
14:30:14registry passed in here. But the key
14:30:18thing to remember is that this
14:30:20self.ttool registry passes in the tools
14:30:23to this context manager and then the
14:30:26context manager creates a system prompt
14:30:28out of this. So within the system prompt
14:30:31the tools might never be passed. the MCP
14:30:34tool specifically and we don't want that
14:30:36because we want a clear description of
14:30:38what the LLM of what the specific MCP
14:30:41tool does so that the LLM can call it.
14:30:44So for that reason what we're going to
14:30:46do is just have context manager or null
14:30:51year and by default the value is going
14:30:54to be null. Then within this initialize
14:30:57we register the tools and then we set
14:31:00the context manager as well. So we have
14:31:02serve.context context manager is equal
14:31:03to context manager. We'll pass in the
14:31:06configuration. We load the memory. We
14:31:08get all of the tools. And at this point,
14:31:10I want to be extremely sure that the
14:31:13tool discovery manager has also passed
14:31:16in all of the tools. So I can just call
14:31:18it before as well. So initialize will be
14:31:22called here, right? And that needs to be
14:31:24awaited. I forgot about that. Then we
14:31:27register the tools. Then I also want to
14:31:30discover all of the tools present within
14:31:33tool discovery. Now I just need to
14:31:36ensure that initialize is called. Now
14:31:39whether this session is initialized or
14:31:42constructed, I have to call this
14:31:43initialize function. So let's see. I
14:31:46think it's within agent. So I'll find
14:31:49all of the references. Yep, it is an
14:31:51agent. And now whenever we call session
14:31:54like that, we'll have to call initialize
14:31:56as well. I can specifically go within a
14:31:59enter and call self dot session dot
14:32:03initialize here and await it. So this
14:32:06will ensure that all of the things
14:32:09within this initialize are done. For
14:32:12example, the MC manager, the discovery
14:32:15manager and then context gets all of the
14:32:18tools because tool registry.get get
14:32:20tools will get all of the tools, the
14:32:22default tools, the tool discovery and
14:32:25the MCP tools. So now let's try to exit
14:32:30out of here and let's see if it works.
14:32:33So I'll have Python main.py
14:32:35and we run into an error. We get
14:32:38attribute error. None type object has no
14:32:40attribute enabled. That means server
14:32:43config is not present. So we'll go to
14:32:46MCP manager where we have server config
14:32:50and just to see that all of the stuff is
14:32:52present or not I'll just print out the
14:32:55MCP configs. Now let's try to run it
14:32:57again and see if we get any value
14:32:59printed out.
14:33:01And we do see file system as the key but
14:33:06none is the value. Since we do get the
14:33:08file system there might be an issue in
14:33:11how we are doing MCP server config. And
14:33:14if I just scroll up to MCP server
14:33:16config, we'll notice that we have if
14:33:19condition here, if condition here, and
14:33:22if everything is fine, we're not doing
14:33:25anything. However, we do want to return
14:33:27MCP server config if everything goes
14:33:30well. Right? Because model validation
14:33:32will validate and if everything goes
14:33:34well, it will return an instance of MCP
14:33:36server config. But I've not returned it.
14:33:39So what I need to do here is call return
14:33:41self. Once we do that, this error should
14:33:44go. So let's try it again.
14:33:47And now we get another error. The error
14:33:50is in MCP manager I passed in wait for
14:33:54but I forgot to pass in the timeout. So
14:33:56what is the timeout? We get the timeout
14:33:58from client dot config dot startup
14:34:02timeout seconds. Let's try to run again.
14:34:08And this time we still see the output of
14:34:10file system. We get MCP server config
14:34:13enabled true startup time. Everything is
14:34:16fine. But then we see secure MCP file
14:34:20system server running on standard input
14:34:22output. That's also a good sign. These
14:34:24two lines are coming from the MCP server
14:34:27itself because we have not logged any of
14:34:29those lines. They're coming from the MCP
14:34:32server connection establishing. So
14:34:34that's good. But then we run into
14:34:35another error where we say an async io
14:34:38future a co- routine or an awaitable is
14:34:42required and this error exists again
14:34:44because of mcp manager. py here we are
14:34:48awaiting async io.wait we don't have to
14:34:50await it because I told you that we are
14:34:52waiting for the connection tasks. So we
14:34:54are loading up a bunch of async items
14:34:56here and then we'll await all of them
14:34:58together at the end. So we'll just have
14:35:01python main.py py again and now we run
14:35:04into another error. The error is
14:35:06attribute error property scheme of MCP
14:35:09tool object has no setter. So we have to
14:35:12go to MCP tool. We have done self dos
14:35:14schema like that. To fix this what we
14:35:17can do is go to the tool and have the
14:35:20schema properly set up like a setter or
14:35:23we can just create a property of schema
14:35:25and just use that. So here I'll just do
14:35:28add the rate property. Then I have div
14:35:31schema where I have self and I'm going
14:35:34to return a dictionary of string, any
14:35:36from here. Let me import any from typing
14:35:40and then I'll just have the same thing
14:35:41here. So I can't set self doss schema
14:35:44equal to something. However, I can just
14:35:46create a property function and do that.
14:35:48Alternatively, you can just go to the
14:35:51toolbased class and here create a
14:35:54property setter for schema and that
14:35:56should work. So now let's try again and
14:36:00we do see the output. I would like to
14:36:01remove this print statement. Now the
14:36:04debugging is done. So I'll remove it
14:36:06from MCP manager. You know let's just
14:36:08try to exit it and see what happens. We
14:36:12still get this output. If you want this
14:36:14output to be disabled, you can do that.
14:36:17So when we try to create a transport in
14:36:19client
14:36:21over here we can pass in the log file
14:36:25and log file can be path because it
14:36:27requires a path and we can pass in
14:36:29os.dev null. So it will not write it
14:36:33anywhere and we won't see it out on the
14:36:35screen either. So let's just try to run
14:36:37it again and this time as you can see we
14:36:40don't see anything. Let's try to call
14:36:43some tools. Now it's the file system
14:36:46server. These are the tools that are
14:36:48available. What I like to call is list
14:36:50directory. But if I try to call list
14:36:54directory just like that, it will call
14:36:56our list directory tool, the built-in
14:36:58tool, not the MCP1. So what I'll do is
14:37:01go to this config.l.
14:37:03Here the name of a server is file
14:37:05system, right? And I know that the tools
14:37:08are prefixed with file system. So what
14:37:10I'll do is say call file system
14:37:16then the list directory tool for me
14:37:20please. Now let's see what happens and
14:37:22we run into an error. Attribute property
14:37:24schema of MCP tool object has no setter
14:37:28and we don't have to set it to serot
14:37:30schema. We just have to return the
14:37:31value. I forgot about that because
14:37:34earlier also the same error was there.
14:37:36Serot schema could not be initialized.
14:37:38we have to return from here. That's why
14:37:40we created a property function. Now
14:37:43let's try again. I'll type in the same
14:37:46prompt. And as you can see the MCP tool
14:37:49does get called. The path is just a full
14:37:51stop. And then we get directory.ai agent
14:37:54directory.Venv.
14:37:55And then there are files all of that.
14:37:58The MCP tool is also working. Encourage
14:38:01you to pause the video create your own
14:38:03MCP server. You can create your own MCP
14:38:05server using fast MCP again. and then
14:38:08run that MCP server. You can also follow
14:38:11some tutorial on YouTube, but create
14:38:14your own MCP server. Then pass it in
14:38:17through this config.tl.
14:38:19Pass in the URL and see if the SSC
14:38:23transport layer also works properly. I'm
14:38:26going to skip that, but yeah, that is
14:38:28something you can look into. And with
14:38:30that, we have the MCP support. One thing
14:38:33I would like to do is go to the MCP
14:38:35manager here and create a function to
14:38:38shut down all of the MCP servers to
14:38:40disconnect all of the MCP servers. So
14:38:43I'll just have def which is going to be
14:38:45async def shutdown. Then we have self
14:38:49it's going to return nothing and first I
14:38:51have to collect all the disconnection
14:38:53tasks which is equal to client dot
14:38:57disconnect. So first I'll have to get
14:39:00access to this client. So what we'll do
14:39:02is go over every client that we have.
14:39:06Cool. So we have for client in
14:39:08selfclient and now I can just do await
14:39:12async io dot gather and I'll pass in all
14:39:16the disconnection tasks here. And we'll
14:39:19set return exceptions to true. If
14:39:22there's any exception, it will return.
14:39:24Then I'll also set clients to clear up
14:39:27everything. And then I'll just set
14:39:30initialized to false because it's not
14:39:33initialized anymore. So remember this
14:39:36client.connect is the function we
14:39:38created in MCP client. And in MCP
14:39:41manager we're just disconnecting all of
14:39:43the LLM clients. And now the shutdown
14:39:47needs to be called within agent. So I'll
14:39:50just go to agent. py where we had a exit
14:39:53if you remember. And here we said if se
14:39:56self do. session is true and self do.
14:39:58session.client is there we close the
14:40:00client. We set session to null. And here
14:40:03I'll just add another thing. If and
14:40:06actually we can just do it over here and
14:40:08say await self dot session dot mcp
14:40:13manager dot disconnect and it's not
14:40:16disconnect sorry it's shut down. And
14:40:19yeah that will shut down everything. So
14:40:22here I'll just check and self do session
14:40:26MCP manager is also there. In that case
14:40:29we'll call shutdown and it should most
14:40:31likely be there. So that is just
14:40:33gracefully quitting all of the MCP
14:40:35related stuff. Now let's run it again.
14:40:38I'll just say call and let's try to call
14:40:40a different tool now. So we'll have call
14:40:43file system double underscore and let's
14:40:45try to find a better tool. Let's say get
14:40:49file info on let's say main. py file.
14:40:54Let's hit enter and see what it does.
14:40:57So it calls file system get file info.
14:41:01Path is main. py. The size is 4719
14:41:04created at this time. Is directory
14:41:06false? Is file true? Permission 644. So
14:41:09that's amazing. So we have added support
14:41:11for MCP nicely. Now the next thing I
14:41:14would like to get into is context
14:41:16management because we have a bunch of
14:41:19tools available and remember everything
14:41:21about tools is done. Now the next thing
14:41:23we're going to work on is context
14:41:25management. So let's understand the
14:41:27problem and why we need context
14:41:29management. So LLMs have a fixed context
14:41:32window right there is 128,000 tokens or
14:41:35256,000 tokens all of that. Now,
14:41:38whenever you're coding with an agent,
14:41:40the context will fill up fast because
14:41:42you've set up your initial instructions,
14:41:44which are 7,000 tokens or something.
14:41:47Then all of the file content that needs
14:41:49to be read, then the tool calls and the
14:41:51output of that tool call. Then you have
14:41:54multi-turns. So within multi-turns,
14:41:56you'll have lots of content, right? And
14:41:59then there are error messages, all of
14:42:01that. So this just means that the agent
14:42:04will have their context filled up very
14:42:06fast. This is why we need context
14:42:08management. Now, how do we solve this
14:42:11context management problem? How do we
14:42:14actually manage the context? You can
14:42:16think about it in human terms. Let's say
14:42:18you are in a 10-hour meeting, but you
14:42:20can only remember the last two hours
14:42:21clearly. What would you do? Well, you
14:42:24take notes, right? You will summarize
14:42:27the key decisions that were taken. What
14:42:29are the action items for you and
14:42:31whatever pending items are there, you'll
14:42:33just summarize all of those items. That
14:42:36is called compaction or compression or
14:42:38summarization. So you're just asking
14:42:40yourself how do I summarize this
14:42:43meeting? Similarly, an agent can just
14:42:45take its conversation history and tell
14:42:47another LLM to just summarize it for the
14:42:50LLM. And the summarization should
14:42:52include things like what are the pending
14:42:55items, what are the items to do, what
14:42:57are the items that have been done and
14:43:00specifically for the AI coding agent,
14:43:02everything is written down in files or
14:43:04is being read from files, right? So we
14:43:07can tell the LLM to specifically list
14:43:09out the paths to the files so that we
14:43:12don't have to spend more tokens on
14:43:13listing the directory and all of that
14:43:16stuff. We can just tell that yeah
14:43:18tools/base.
14:43:19py has this particular thing and the
14:43:22agent worked on x y and z for this base.
14:43:25py right that that's so much simpler and
14:43:28in case the agent has to touch base py
14:43:31again it can just look up the file and
14:43:34read the file so essentially we go from
14:43:37let's say something like 200,000 tokens
14:43:41and summarize it down to let's say 10k
14:43:43tokens after we have those 10k tokens it
14:43:46doesn't have any of the information
14:43:48related to the tool calls it did or the
14:43:50files it wrote but it does have the
14:43:52context about what each file that it
14:43:55created or touched does and what are the
14:43:57pending future items. So based on that
14:44:00information alone, it can go ahead and
14:44:03on demand load other things it might
14:44:05need. For example, it knows that for the
14:44:08next task, it needs to load up let's say
14:44:10the tools do/base. py in the context we
14:44:14have written down that tools/base. py
14:44:16does this task and it might be used for
14:44:19a pending action item. So in that case
14:44:22it will go ahead and read this file
14:44:25again. So we are taking advantage of the
14:44:28fact that everything is in our file
14:44:30system. So we can just write down the
14:44:32path to it and whenever needed we can
14:44:35read it. So on demand reading the files
14:44:38this is what we're going to rely on. And
14:44:41this is what our compaction prompt is
14:44:43also going to be because remember
14:44:44compaction is just telling another LLM
14:44:47that hey please summarize this entire
14:44:49conversation for me. So let's go ahead
14:44:52and create the files that's required. So
14:44:54I'll specifically go to the context
14:44:56folder and within that I can create a
14:44:58new one. Let's call this compaction py.
14:45:01Now this compaction will be used within
14:45:04the AI agent and it will work closely
14:45:06with the context manager. So let's call
14:45:08this class chat compactor or you can
14:45:12just call it compressor whatever you
14:45:14want. Then you can have the init
14:45:16function. And the first thing we'll need
14:45:18here is the client, right? Whatever is
14:45:20the LLM client because I have to send a
14:45:22request to the LLM to compress things.
14:45:25So I'll just create an instance or take
14:45:27the instance from the constructor. And
14:45:30after that I'm going to create a
14:45:32function async def compress. And this is
14:45:36going to be the main compressor. What it
14:45:38will require is a context manager. So
14:45:41I'll take that as well. and let's import
14:45:44context manager. So to compact the first
14:45:47thing we'll require is all of the
14:45:49messages in the conversation history and
14:45:52context manager is storing all of that.
14:45:55If you notice we had messages here.
14:45:57Messages stores all of the messages. So
14:46:00we can just call get messages function
14:46:02which will give us the entire
14:46:04conversation history. So I can just do
14:46:07messages is equal to context manager dot
14:46:11get messages and then we have a list of
14:46:13messages. Then I'll make a quick check
14:46:16that hey if the length of messages is
14:46:19greater than three in that case I'll
14:46:21just return none and none. Why am I
14:46:24returning a pupil from here? Well that's
14:46:26because the return type of this function
14:46:28is going to be interesting. It's going
14:46:30to return first a pupil. Obviously the
14:46:34first element of this pupil is going to
14:46:35be a string or it can be a null value
14:46:38and this string or null value is
14:46:40essentially the summary string. So we
14:46:43ask the LLM to summarize and give us the
14:46:46compression prompt to summarize. So
14:46:49that's what it stands for. And the next
14:46:51thing we're going to return from here is
14:46:54a token usage. So how much token usage
14:46:57was done for this compression that took
14:47:00place. So I'll just have none here. Once
14:47:04I have that, I will build up the
14:47:05compression request. So to build a
14:47:08request first, let me just go ahead and
14:47:10do a try except because in try except we
14:47:14might run into some sort of error
14:47:15because we are dealing with an llm. And
14:47:18I'll just return none, none from here.
14:47:20Also, let's just remove a. And now what
14:47:23I'll try to do is self.client client
14:47:27chat completion things we've already
14:47:29looked into and pass in the messages
14:47:31list and this is going to be a
14:47:33compression messages list that we are
14:47:35going to develop in just a minute and
14:47:37the second thing I want is stream is
14:47:39equal to false I don't want stream is
14:47:42equal to true the reason for it is
14:47:44what's the point of having a stream of
14:47:46messages all I need from this client is
14:47:49one big message of the summarization and
14:47:53I'll add it to my context after that
14:47:56that's why stream is equal to false
14:47:59since this is going to return an async
14:48:01generator we'll have to do async for
14:48:03event in self.client client. Completion
14:48:07and after that I can just check if event
14:48:10type is equal to event type or stream
14:48:15event type sorry let's import it from
14:48:17client.response and if it is equal to
14:48:20message complete because remember in
14:48:22chat completion when we are doing a
14:48:24non-stream response we are sending a
14:48:26message complete when the response is
14:48:29over. So if we have reached the event
14:48:31type of message complete in that case I
14:48:35would like to get the usage because even
14:48:37the usage is being sent right. Let me
14:48:39just go back to client chat completion.
14:48:42We passing in [snorts] the usage which
14:48:44is this token usage. We will get the
14:48:47summary string. So I'll keep track of
14:48:50both of them. So I'll have summary is
14:48:52equal to an empty string. Usage is equal
14:48:56to null. And then I'll have usage is
14:48:59equal to event dot usage and summary is
14:49:04just plus equals event dot text delta
14:49:08dot content. After this for loop gets
14:49:12over you know we just going to get one
14:49:13message anyways. And once we do that
14:49:16we'll just check if not summary or not
14:49:19usage. That means summary is an empty
14:49:22string or a null value or the usage is a
14:49:24null value. In that case, we're going to
14:49:27return null, null because yeah, we did
14:49:30not get anything correctly. After that,
14:49:32we'll just return the summary, comma,
14:49:36usage if we get both of them, right? And
14:49:39now we have to build up the compression
14:49:41messages. So to get the compression
14:49:43messages, it's going to be relatively
14:49:45easy. All we need to do is create a list
14:49:48here, which is going to be a list of
14:49:50dictionary. The first one is obviously
14:49:52going to be a role of system because we
14:49:54want to attach a system prompt for
14:49:57compression as well. And this system
14:50:00prompt is going to tell how the
14:50:01compression should take place. What the
14:50:03summarization should include. So we have
14:50:05the content and this is going to be get
14:50:08compress or let's say get compression
14:50:11prompt. You can also call it get
14:50:13compaction prompt. That would be more
14:50:15ideal because this is called compaction.
14:50:18So now I'll go to system.py file which
14:50:21is within the prompts folder. And down
14:50:24here I'm going to paste another function
14:50:25which is the get compression prompt.
14:50:27We're going to use this compression
14:50:29prompt. If you're copying it from
14:50:30GitHub, you might already have this get
14:50:32compression prompt. So what is this
14:50:34prompt saying? Well, it's saying to
14:50:36provide a detailed continuation prompt
14:50:38for resuming the work. The new session
14:50:41will not have access to our conversation
14:50:44history. That is an important line.
14:50:46That's because we're not just
14:50:48summarizing here. We're saying that
14:50:50yeah, you need to summarize what you
14:50:52did. Along with that, you need to write
14:50:54kind of a continuation prompt for the
14:50:57LLM so that it can continue the task. So
14:51:00it's not just summarization, it's also a
14:51:03continuation prompt of what it needs to
14:51:06do next. Then important is that it needs
14:51:09to structure the response exactly as
14:51:11follows. So it gives its original goal.
14:51:13So it states the user's original request
14:51:15or goal in one paragraph completed
14:51:18actions. It should not repeat these
14:51:20actions otherwise the LLM might get
14:51:23confused because think about it you have
14:51:2620 turns. You have 20 messages of you
14:51:29and LLMs or agents back and forth and
14:51:33the agent has completed certain tasks.
14:51:35Here it goes ahead and lists all of the
14:51:37actions it did. Now in a new context
14:51:41where the LLM doesn't know what it did
14:51:43previously and all it relies on is this
14:51:46particular paragraph. It might see
14:51:47completed actions. Okay, these were the
14:51:49completed actions. It will not know
14:51:52that. It doesn't have to repeat this.
14:51:54These are already done. We just
14:51:56explicitly saying it out loud. And it
14:51:58will list specific actions that are done
14:52:00and should not be repeated. Be specific
14:52:02with file paths, function names, changes
14:52:06made, and use bullet points. After that
14:52:09it will describe the current state of
14:52:11the codebase or project after the
14:52:13completed actions. What files exist?
14:52:15What has been modified? What is the
14:52:17current status? In progress work is the
14:52:20pending work. What was being worked on
14:52:22when the context limit was hit? Any
14:52:25partial changes
14:52:27and also since we have described here
14:52:29that the context limit was hit the LLM
14:52:32will have more idea of what it needs to
14:52:35do. Then there are remaining tasks. What
14:52:37still needs to be done to complete the
14:52:39original goal? It has to be very
14:52:41specific. Next step, what is the next
14:52:44immediate action to take? Be very
14:52:46specific. This is what the agent should
14:52:48do first. Key context. So any important
14:52:51decisions, constraints, user
14:52:53preferences, technical context or let's
14:52:55say assumptions as well that must
14:52:58persist. Be extremely specific with file
14:53:00paths and function names. We are
14:53:02reinforcing this because the LLM needs
14:53:04to pass that in. This is something we
14:53:06are relying on very strongly and the
14:53:09goal is to allow seamless continuation
14:53:11without redoing any completed work. So
14:53:14that is the compression prompt. We can
14:53:17just take this and call it in over here
14:53:20obviously. So now that we have the
14:53:23system prompt with us, the next thing is
14:53:25the user message. So we'll just pass in
14:53:27the role of user and then the user is
14:53:31going to have the content here. And for
14:53:33the content, what do we pass in? Well,
14:53:35we can pass in all of the messages here,
14:53:38but the problem with that is the way
14:53:40messages are structured. It gives a list
14:53:42of dictionary where the role is system
14:53:44content is given and it will pass in the
14:53:47role and content again and again. For
14:53:49example, it will again pass in the role
14:53:52system and then content system prompt.
14:53:54After that there will be a user message.
14:53:56So it will again add ro user and the
14:53:59content of the user. Then there will be
14:54:00a role of assistant and whatever it did
14:54:02to complete the user's query. This is
14:54:05problematic because if we attach it to
14:54:08this content we are messing up the
14:54:10entire context right because this
14:54:13content is a dictionary which has all of
14:54:15these rows and contents which might just
14:54:18confuse the LLM. So instead of passing
14:54:21in the messages, what we're going to do
14:54:23is format this messages so that we can
14:54:25pass it in correctly. So I'll just
14:54:27create a helper function called format
14:54:30history for compaction.
14:54:33And then I'll pass in this messages
14:54:34list. Now I'm going to go ahead and
14:54:36create this function. So I have def
14:54:39format history for compaction. Then we
14:54:41have a messages which is a list of
14:54:43dictionary of string comma any and I'll
14:54:47import from typing any and obviously the
14:54:50output of this is going to be a string
14:54:52because all we need is a string. We do
14:54:54not want a list of dictionary the llm
14:54:57will get more confused because of that.
14:54:59However, if we structure properly that
14:55:02this is the conversation that needs to
14:55:04be comp compressed and then we pass in
14:55:07what the tool result was in a string
14:55:09format, that will be much better. So,
14:55:12what I'll do is go over every message in
14:55:14this messages dictionary. I'll check
14:55:17what the role is. So, let me just get
14:55:19the role here and if it's not specified,
14:55:21it's an empty string. Similar to that
14:55:24will be content. So, message.get get
14:55:26content
14:55:28and then I'll check if the role is equal
14:55:31to system. If it is then we're going to
14:55:34continue so that we don't have to write
14:55:36the system prompt down because if we
14:55:38write the system prompt down there is no
14:55:41point like the LLM will also try to
14:55:43include that in its compression messages
14:55:47or the compression prompt and we don't
14:55:49want that. Imagine the 6,000line
14:55:52system prompt coming in and the LLM
14:55:54trying to figure that out. After that,
14:55:57we'll just check if the role is equal to
14:55:59tool. In that case, we're going to
14:56:02append it to a string, right? We'll get
14:56:04some content out of this tool and attach
14:56:06it. So, first I'll go ahead and create
14:56:09the output list here. The first string
14:56:11here is going to be here is the
14:56:13conversation that needs to be continued.
14:56:18Then maybe I'll just give a back slashn
14:56:20here so that there's a twoline space
14:56:23because at the end we'll just
14:56:24concatenate all the elements of this
14:56:26list. So there will be two lines left.
14:56:31Now let's try to extract the tool ID
14:56:33which is equal to message.get and then
14:56:36we'll have tool call ID. Otherwise it's
14:56:39an unknown tool call ID. This is quite
14:56:42important because if we pass in the tool
14:56:44ID the LLM will just have more context.
14:56:47it will have that continuation because
14:56:49if there are three tool calls that are
14:56:52essentially the same the LLM will just
14:56:54understand it better right because if
14:56:56the assistant says this is the tool ID
14:56:58that I told the user to call and then we
14:57:03attach the role of tool where the user
14:57:05did call the tool ID that's a
14:57:09confirmation that the tool was called
14:57:12after that what we're going to do is
14:57:14truncate down the content that we have
14:57:16over here because there's no need of
14:57:18passing in the entire content anyways.
14:57:21It needs to get compressed. So why
14:57:23should we even pass it in? So we'll just
14:57:25truncate it down to let's say 2,000
14:57:27characters and we'll just say if the
14:57:30length of the content is greater than
14:57:322,000 only then we do we do that.
14:57:35Otherwise, we'll just say content and
14:57:38then we'll again say if the length of
14:57:40content was greater than 2,000, then we
14:57:43truncated. And in that case, we're going
14:57:45to have truncated plus equals and maybe
14:57:49I'll have a back slash end dot dot dot
14:57:52and then I'll have tool output
14:57:54truncated. Then I can go ahead and
14:57:56append to the output. So I have
14:57:58output.append.
14:58:00Then we have an string where the tool
14:58:03result is present. Then we pass in the
14:58:06tool ID. Then we have a back slashn
14:58:09where I pass in the truncated response.
14:58:12So this is us just trying to format the
14:58:15output nicely so that the LLM
14:58:17understands it better. Then another role
14:58:21that can happen is the assistant.
14:58:25In that case I want to get all of the
14:58:26tool calls that were done and all of the
14:58:29content that was done. So first I'll
14:58:32just check if there are tool calls that
14:58:34happen. If there are tool calls that
14:58:37happen then I'll go over every tool call
14:58:40in that
14:58:42list and I'll have function is equal to
14:58:44let's say tool call.get function and
14:58:48it's an empty object if not specified.
14:58:50Then we have the name. We've looked into
14:58:52this many times. So I'll just go a bit
14:58:54faster. We're just trying to extract all
14:58:57of the arguments from here and we'll
14:58:59just quickly do that. Awesome. And now
14:59:02I'll just check if the length of the
14:59:04arguments is greater than 500 for
14:59:07example. If it is then I'll truncate
14:59:09down the arguments to 500 characters.
14:59:13And then we'll just have output dot
14:59:15append or let's not append it to output
14:59:18just yet because we're in a for loop.
14:59:21Maybe what we can do is create another
14:59:24list called tool details which is an
14:59:26empty list and then we'll go over each
14:59:30element of this tool call and append it
14:59:32to this tool details. So we'll have some
14:59:36indentation here. This is how we are
14:59:38planning to format it. Then we have the
14:59:41name of the tool call. Then we have the
14:59:44arguments passed in. Right? So let's say
14:59:46the name of the tool call was hello
14:59:49world. This was the tool call and then
14:59:51you pass in all of the arguments. For
14:59:53example, A is equal to B, C is equal to
14:59:56D, whatever. So this is how we're saying
14:59:59the tool call should look like or the
15:00:01result should look like. And we'll do
15:00:03that for every tool call. So in this
15:00:05list, we just have this indented kind of
15:00:07thing. And now after we get out of this
15:00:11for loop, we'll just do output dot
15:00:14append. And then I'll just say, hey, the
15:00:18assistant called some tools. And then
15:00:21I'll put them on a new line. Then I'll
15:00:23pass in this tool details. But tool
15:00:25details is also a list. So maybe I can
15:00:28just do plus back slashn.join
15:00:32and then I'll join the tool details. So
15:00:36everything goes on a new line. So this
15:00:38is the formatting for tool calls. Now
15:00:41another thing that an assistant can give
15:00:42us is the content. So we'll also attach
15:00:45the content here. So if content then
15:00:48we'll have truncated is equal to content
15:00:52and then let's say we go up till 3,000
15:00:54characters. If the length of the content
15:00:56is greater than 3,000 characters
15:00:58otherwise we'll just pass in the content
15:01:01and if the length of the content is
15:01:03greater than 3,000 then we'll just
15:01:06append to truncated a thing that says
15:01:09response truncated. And finally after
15:01:12that we'll just have output.append we'll
15:01:15pass in the assistant on a new line and
15:01:18pass in the truncated thing. Also this
15:01:21has to be an if string. Now one thing we
15:01:24can do is maybe we can shift the content
15:01:27a little bit up because when we display
15:01:29to the user the first thing that gets
15:01:31shown up is the content whatever the LLM
15:01:34or the agent told us that it's going to
15:01:37work on X Y and Z and then the tool
15:01:39calls show up. So this is how I'm
15:01:41formatting it. Then I have an else
15:01:43condition here that the role is of the
15:01:46user most likely. In that case again
15:01:48we're just going to do the truncation.
15:01:51So I'll just copy this entire line. All
15:01:54right. And then let's paste it. Let's
15:01:56indent this properly. So truncated is
15:01:59content. Let's say up to 3,00 or500
15:02:02characters. If the length of content
15:02:05exceeds 1,500 otherwise we have the
15:02:08content. If the length of the content is
15:02:11greater than 1500, we'll just add
15:02:13message truncated
15:02:16and then we just say that the user
15:02:18responded with this truncated message.
15:02:21And finally at the end, we'll go ahead
15:02:24and return everything from this
15:02:26function. So after this for loop gets
15:02:28over, we have return back slashn dot
15:02:32join and I'll pass in the output. Maybe
15:02:35I can do this better. I can pass in back
15:02:38slashn back slashn. So two lines are
15:02:40left. Then we have a clear separator and
15:02:43then we again have back slashn back
15:02:44slashn. So just a clear difference
15:02:48between this is what the assistant said.
15:02:50This is what the tool called it and this
15:02:52is what the user related messages were.
15:02:55Now I can copy this and we have already
15:02:57passed it in over here. And this seems
15:03:00to be the function for compression. Now
15:03:03all I need to do is call this compress
15:03:05function within the agentic loop. So
15:03:08I'll go to this agent. py and figure out
15:03:11how do I call this compression in the
15:03:14agentic loop. So in this agentic loop we
15:03:18have the turn beginning. Then we go over
15:03:21all of the messages
15:03:24and try to fetch another response. Then
15:03:26we call the tool and all of that. So
15:03:28where exactly do we want this
15:03:30compression? Well, we can just put the
15:03:32compression at the very top, right?
15:03:34Before we are going to call the LLM
15:03:37again. We can put in a compression part.
15:03:40We can check for the context overflow
15:03:43here. And then we can compress based on
15:03:47that because we just don't want
15:03:48compression to begin just like that. I
15:03:51can't I do not want to go ahead and have
15:03:54something like chat compressor and then
15:03:58call the compression method here because
15:04:02then it will compress for every single
15:04:05message. Even if I have 8,000 tokens
15:04:08well under the limit, it will still go
15:04:11ahead and compress it. So instead of
15:04:12doing that, we'll check for context
15:04:14overflow. And to check that first we'll
15:04:17have to keep track of all of the usages
15:04:20that are happening. And to track the
15:04:22usage we do get message complete event
15:04:26sent by the chat completion. So if you
15:04:28notice here for either of these stream
15:04:31or non-stream responses we get the
15:04:34message complete where we get the usage
15:04:37and we want the usage here as well. So
15:04:39we have to keep track of that. So here
15:04:41we'll have usage which is of the type of
15:04:44token usage. Let me import that or it
15:04:46can be a null value and by default it is
15:04:48a null value. And then if we get an
15:04:51event let's say the events type is equal
15:04:54to stream event type dot and it's
15:04:58message complete then we get access to
15:05:00the usage. So I can just do usage is
15:05:03equal to stream or let's say event dot
15:05:07usage. The first thing is if there are
15:05:10no tool calls we'll return. We can check
15:05:12here if usage is present it's not null
15:05:16then I want to set the latest usage. So
15:05:18within the context manager I can go and
15:05:21create a variable or let's go to
15:05:23manager.py
15:05:25and here I can create a variable that
15:05:27keeps track of the latest usage. So I'll
15:05:30have self dot latest usage. This is not
15:05:33the total usage. We'll have another
15:05:35variable created to track the total
15:05:38token usage. But what we are interested
15:05:41in is just the latest usage as of now.
15:05:43So we'll have token usage created here.
15:05:46Let me import that from client.response.
15:05:49And maybe since we're here, let's just
15:05:51create a total usage as well. And that
15:05:54will be token usage as well. So these
15:05:56are just two objects instantiated.
15:05:58Everything is zero for them. Then we're
15:06:01going to go ahead and create two
15:06:03functions. The first function is going
15:06:06to be to set the latest usage. We'll get
15:06:10a usage
15:06:12from the parameters here. And what it
15:06:14does is it sets the latest usage to this
15:06:18usage that's coming in. Then the other
15:06:20method that's going to be created is add
15:06:24usage. It also gets another usage, but
15:06:27the difference is it will set the total
15:06:29usage. And what it will do is instead of
15:06:32setting the usage, it will add up the
15:06:34usage. Because think about it, latest
15:06:36usage is essentially what was the usage
15:06:39in the last message. Latest usage is
15:06:42used for tracking the context length.
15:06:44Because think about it, if we go back to
15:06:46this chat GPD, I give it one message and
15:06:50then it responds. So the latest usage
15:06:53for this LLM will be given by the
15:06:57response over here. Then I ask another
15:07:00message. The latest usage will be given
15:07:02over here because latest usage will have
15:07:06this message, this message and this
15:07:08message in the prompt in the message
15:07:11history. So these will be taken as the
15:07:14prompt tokens. This will be the
15:07:16completion tokens. So the latest usage
15:07:19covers everything over here. However, if
15:07:22we try to use add usage for this
15:07:25particular thing, what will happen? I
15:07:26send in one message, right? So that is
15:07:29tracked. Then I get this response. This
15:07:31is also tracked. And then I send another
15:07:34message. And in that message the
15:07:37previous history is also appended.
15:07:38Right? Because I told you LLMs are
15:07:41stateless. So these two messages are
15:07:44also kept track of. So these are also
15:07:47taken as system messages or system
15:07:49prompts. So these are again called
15:07:52prompt tokens. And then this is also
15:07:54called a prompt token. So I have to add
15:07:57up this entire thing again and this will
15:07:59keep happening for other LLMs as well
15:08:02and this will keep happening for other
15:08:04messages as well. This latest usage will
15:08:06just help us know what the latest usage
15:08:09is so that the context limit can be
15:08:11checked. And the next thing we have is
15:08:13the add usage. Add usage is so that we
15:08:17can display to the user when they type
15:08:19in a command like forward slash stats.
15:08:22This is a command that we're going to
15:08:23support. If the user passes in slash
15:08:25stats, I want to show the total usage
15:08:27that's done because we are charging them
15:08:30for this total usage because it did take
15:08:33us this much. Total usage is money. So
15:08:36we'll charge them that much. But latest
15:08:39usage is so that we can check what
15:08:42context limit was hit. I'll just repeat
15:08:44this point again because it's quite
15:08:46important. You can skip ahead if you
15:08:48understood it. But essentially latest
15:08:50usage till year includes the usage of
15:08:53this this message and this message all
15:08:57once. Total usage means it will include
15:09:00this message this message two times
15:09:03because you first sent a message you
15:09:05sent the you got this response. So it's
15:09:08counted once and then when you send
15:09:10another message these two are again
15:09:12included in the conversation history and
15:09:15that's why you're charged for it again.
15:09:17And then you are also charged for this
15:09:18token. So this is the total usage. But
15:09:21to understand what context we are on, we
15:09:24just have to check how much these
15:09:26messages cost us once till year. So this
15:09:30was let's say one token. This is let's
15:09:32say 10 tokens and this is let's say
15:09:34three four tokens. So it's about 16 17
15:09:37tokens. This is the conversation
15:09:39history. If we include them two times,
15:09:41there's no point of having the context
15:09:43limit because if we are including that
15:09:45two times, well, it doesn't make any
15:09:47sense. So, having understood that, let's
15:09:49go back to agent. py and here we can
15:09:52check that if usage is present, then
15:09:54I'll have self.context
15:09:57manager dot add usage. So, that will
15:10:00give me the total usage. I'll pass that
15:10:03in. And similar to add usage, I'm also
15:10:05going to set the latest usage.
15:10:09And as it goes, we'll return out of this
15:10:12tool calls. And a similar thing also
15:10:15needs to be done if there are tool
15:10:17calls. After we do for tool result and
15:10:20tool call results, we add the tool
15:10:22result to the context manager. After
15:10:25that, we can just check if usage is
15:10:27present. Then we are going to have
15:10:29self.context manager, set latest usage,
15:10:32and add the usage. And maybe you can
15:10:35also yield an agent event notifying the
15:10:37user that the compaction took place.
15:10:40That's totally valid and actually a good
15:10:42user experience. So go for it. And now
15:10:44that we have kept track of this entire
15:10:46usage, we can go at the top and check
15:10:50for the context overflow. And for that
15:10:52also I'll have to go to the context
15:10:54manager and create a new function this
15:10:56time. And this function will just check
15:10:59if we have reached the compression
15:11:01limit. Let's call this function needs
15:11:04compression and then it's going to
15:11:06return a boolean value and then we just
15:11:09have to check if the context needs
15:11:10compression. So first thing we'll
15:11:12require is the context limit and we can
15:11:15get that from self do.config dot dot
15:11:19let's say not name we need the context
15:11:22window. So that is the context limit.
15:11:24Then what are the current tokens that
15:11:26have been used? And for that we can just
15:11:29do self dot latest usage dot total
15:11:33tokens. And then we have to just return
15:11:37if the current tokens exceeds the
15:11:40context limit then yeah we've reached
15:11:42our limit. But here's the thing. Why do
15:11:45we want to do a neck and neck? Let's say
15:11:47the current tokens is 256
15:11:50or 258,000
15:11:52tokens and the context limit is 256,000.
15:11:57That's pretty bad because many times it
15:12:00can be the case that tool results cannot
15:12:03be truncated down a lot. Nothing can be
15:12:05truncated much. When we give the result
15:12:09to the LLM, what will happen is that the
15:12:11LLM will just say that hey listen this
15:12:14is out of my context window. So instead
15:12:17maybe we can use some sort of limit
15:12:19here. We can say that the if the current
15:12:21tokens is greater than context limit
15:12:24into 0.8 for example that means 80% of
15:12:28its limit. So something like 200,000
15:12:31tokens in that case we we need the
15:12:34compression. So this is what we're doing
15:12:36here and we'll call this function in
15:12:38agent. py now. So we can check if self
15:12:41dot session dot context manager dot
15:12:44needs compression in that case we have
15:12:47to call this compression method for that
15:12:50we will need a chat compressor and I can
15:12:52create an instance of chat compressor in
15:12:54agent but I think for each session
15:12:56there'll be a ch different chat
15:12:58compressor instance so let me just pass
15:13:01that in over here also if you notice
15:13:04chat compactor or compressor requires a
15:13:07client and the client instance is
15:13:09created within the session. So let me
15:13:11just go to the session and instantiate
15:13:13it. So I'll have self dot chat compactor
15:13:18is equal to chat compactor. Let me
15:13:21import it. And then I'll pass in the
15:13:23client which is self.client. And since
15:13:26the instance is created here I can go
15:13:28back to agent. py and here we can just
15:13:31say that we'll have self dot session dot
15:13:35chat compactor dot compress. And then I
15:13:39need to pass in the context manager. So
15:13:42I'll just pass in the self dot session
15:13:44doc context manager and it will return
15:13:47to us a cool routine. Let me await it.
15:13:49And once I await it, I'll get a pupil
15:13:51with summary and usage given. Now the
15:13:55summary and usage can be null as well.
15:13:58So first I'll check if the summary is
15:14:00present. In that case it's implied that
15:14:02the usage is also present because they
15:14:04both come in one package. you know if
15:14:06summary is not null usage is also not
15:14:08null so I'll just do self dot session
15:14:12dot context manager dot add usage and
15:14:17I'll pass in the usage and similar to
15:14:20that I'll also do add latest or let's
15:14:23say set latest usage and I pass in the
15:14:27usage awesome and now what I like to do
15:14:30is replace the context history so what I
15:14:33need to do is go to the manager and the
15:14:35messages list that we are trying to
15:14:39maintain here. I just have to clear it
15:14:41off and pass in this summary that the
15:14:44LLM gave us. But that's not how easy I'm
15:14:48going to make it. I'm going to add in a
15:14:49few more steps. So, let's just go back
15:14:51to manager, create a new function now.
15:14:54So, I'll minimize everything. And we
15:14:56have a new function now. Let's call it
15:14:58replace with summary. So we are trying
15:15:01to replace all the messages with this
15:15:03summary. But I'll try to do some kind of
15:15:07manipulation here. We'll give it the
15:15:10continuation prompt that the llm gives
15:15:13us. And before that we'll clear off all
15:15:15of the messages. So I have self dot
15:15:17messages. Or maybe I can just set it to
15:15:20an empty list again. After that I'll add
15:15:23the continuation prompt as context. And
15:15:27it looks something like this.
15:15:30Since it's a big prompt, I'm just
15:15:31pasting it in. But let's go through it.
15:15:34I'll specifically tell the LLM that hey,
15:15:36there's a context restoration. Previous
15:15:38session has been compacted. The previous
15:15:41conversation was compacted due to
15:15:43context length limits. So, giving our
15:15:45new LLM some context about what's going
15:15:48on. Below is a detailed summary of the
15:15:51work done so far. Critical actions
15:15:54listed under completed actions are
15:15:56already done. Do not repeat them. This
15:15:59is very important because the LLM will
15:16:01just start doing them. So I'm
15:16:02reiterating that. And now I'll pass in
15:16:06the summary within these clear
15:16:08separators. And I'll say resume work
15:16:10from where we left off. Focus only on
15:16:13the remaining tasks. Maybe we can also
15:16:15tell it can also focus on the tasks that
15:16:20are in progress. So you can tune this
15:16:23prompt a little bit more. By no means is
15:16:25this perfect. So yeah, after that I'll
15:16:28add it to the messages. And for that
15:16:31I'll just do self dot messages dot
15:16:33append. But I just can't pass in this
15:16:36message, right? Because it's a text.
15:16:39Messages requires us to pass in a
15:16:41message item. So let me create that
15:16:43message item. So I have let's say
15:16:46summary item which is equal to message
15:16:49item and then I pass in the role which
15:16:51is the user because the user is saying
15:16:53all of this. Then you have the content
15:16:55which is the continuation content and
15:16:58then you have the token count and then
15:17:01we can just pass in count tokens. We'll
15:17:04pass in the continuation count and then
15:17:07the model which is self dot model name.
15:17:10Great. Now I'll pass in the summary
15:17:12item. Is that enough? No. I'm going to
15:17:16do a bit more manipulation here. The
15:17:18second step of this is going to be to
15:17:20add the assistant acknowledgement. We'll
15:17:23tell that the yeah the assistant replied
15:17:25to us that yeah I've reviewed this
15:17:27context the original goal what was
15:17:30required which actions are already
15:17:31completed essentially I've gone over
15:17:33this entire state of the project and
15:17:36I'll continue with the remaining tasks
15:17:38only and we'll start from where we left
15:17:41off. So this is some kind of
15:17:43manipulation reinforcing the LLM that
15:17:46yeah you said this to us earlier even
15:17:48though the LLM did not tell us we're
15:17:50just telling it so that it works in a
15:17:53better fashion. So I'll paste this in
15:17:55and this is quite similar to what open
15:17:57code does to compact their things. So as
15:18:00you can see I've reviewed the context. I
15:18:03understand the original goal blah blah
15:18:05blah. After that we'll again create this
15:18:08message item and append it to the
15:18:09messages list. So let's call this
15:18:11acknowledge item and then we have the
15:18:14role of assistant acknowledge content
15:18:17and then we pass in the token count and
15:18:19then we can pass in the acknowledge item
15:18:21here. Now you can also try removing this
15:18:24and see what the output looks like but
15:18:27in my experience this acknowledgement
15:18:30makes it work much much better because
15:18:32if the LLM says yeah I've already worked
15:18:35on that it just works better. It's like
15:18:39us gaslighting the assistant telling us
15:18:42that yeah you you did say this earlier.
15:18:45So the LLM is more inclined to work in
15:18:48this manner. Okay. After that finally we
15:18:51can have another one where we are saying
15:18:54to the LLM that yeah sure go ahead
15:18:58continue with the remaining work only
15:19:00because if the messages ends with the
15:19:02assistant the assistant might not reply.
15:19:05So we do need the messages ending with
15:19:07the user. So we have this continue
15:19:10content where it says continue with
15:19:12remaining work only. You know just
15:19:14reinforcing the same thing again and
15:19:16again because we just don't want the LLM
15:19:20to do the work that's already done. So
15:19:23we'll have a continue item which is
15:19:25equal to message item. Let me just copy
15:19:27paste what we had over here. And then
15:19:30I'll pass in this continue content. The
15:19:33role is of the user. Then we count the
15:19:36tokens which is the continue content.
15:19:40Also here the count tokens should be
15:19:42acknowledge content.
15:19:45Then finally let's do self do messages
15:19:47dot append. And we'll pass in the
15:19:49continue item. All we're doing is
15:19:51gaslighting.
15:19:53Nothing else. And it does seem to work.
15:19:55So that's good. Now I can go to this
15:19:58agent. py and do something like self do.
15:20:02session dot context manager dot replace
15:20:06with summary and I'll pass in the
15:20:09summary which is this text right so what
15:20:13it will do is clear off all of the past
15:20:15messages it will replace with our
15:20:17summary it will gaslight a little bit
15:20:19and also we'll set the latest usage and
15:20:23add usage based on whatever we get from
15:20:26here obviously that's not very correct
15:20:30because we added our own stuff as well.
15:20:32For example, this continuation content,
15:20:35all the extra tokens that are added
15:20:37here. For example, these things, then
15:20:39the summary is accounted for because
15:20:42that's what the usage was. And then
15:20:44these things, then we have these tokens
15:20:47as well. So, these are not accounted
15:20:49for, but it's just a difference of let's
15:20:51say maximum of 500 to,000 tokens. So,
15:20:56I'm not worrying too much about it. But
15:20:58if you want to be perfect, which you
15:21:00should be, then you can go ahead and add
15:21:02that in. I'll close all the save files.
15:21:05Let's restart and see if I made any
15:21:06mistakes.
15:21:08Nope. But there's one thing. It's very
15:21:11hard to reach the context limit of
15:21:13256,000.
15:21:15If I start to do that, I'll have to wait
15:21:18for an hour or maybe 30 minutes. I can't
15:21:21do that. So, what I'll do instead is go
15:21:23to our config. py. And here within the
15:21:28model config, I set the context window
15:21:30to be 256,000. Here maybe in the agent
15:21:35config. ML, I set the context window to
15:21:40be something like 16,000 tokens. So as
15:21:44soon as we hit the 16,000 token limit,
15:21:46it will just compress. Now let's start
15:21:49it and see what happens. I can tell it
15:21:51to create a YouTube clone for me,
15:21:54please. and I'll tell it to oneot and
15:21:58let's see what it does. So I stopped the
15:22:00agent midway after going after it went
15:22:02on for a while. The reason for it is it
15:22:05was going on for too long. But I would
15:22:07like to show you certain instances where
15:22:08we can see that the agent started
15:22:11continuing after doing a compaction.
15:22:13First of all, I cleared off everything
15:22:15and started again behind the scenes
15:22:18because I forgot to put using HTML, CSS
15:22:21and JavaScript. it started creating it
15:22:23in react and I did not want that
15:22:25otherwise it would take a lot of time.
15:22:27Despite that you know it took a lot of
15:22:29time. So it went on and created
15:22:31index.html and then styles dot CSS and
15:22:35the script.js. All right. After that it
15:22:39said that it created this basic clone
15:22:42and I did not open it. I did not open
15:22:44the index.html. I just did doesn't look
15:22:47good. do more work because I wanted it
15:22:50to hit its limit of 16,000 tokens of
15:22:53context window. So, it went ahead and
15:22:56added a few to-dos and started working
15:22:58on them. After that, it went on for a
15:23:01while and updated a few things. After
15:23:03that, there's something I want to show
15:23:05you. So, when I scroll down, it again
15:23:08does action list. So, it lists the
15:23:11to-dos. The reason it had to list the
15:23:13to-dos is because it started working on
15:23:16something, right? and then compaction
15:23:18occurred. Once the compaction occurs, it
15:23:20has no context of what to do. So, it
15:23:22just calls the to-dos list so that it
15:23:24can see any pending work that needs to
15:23:26be done. After that, it just says that
15:23:28let me add realistic video cards, which
15:23:31is the second one because it did improve
15:23:32the UI design and layout previously. And
15:23:35then it goes ahead and completes all of
15:23:37this task. Once all of that is done, it
15:23:40goes on with the next step, enhancing
15:23:42the sidebar, which is the to-do over
15:23:44here. and then it works on that part
15:23:47before I abort the operation. So there
15:23:49are two three more to-dos to do. But as
15:23:52you can see when once it hits the
15:23:53context length it starts rereading the
15:23:56files that are necessary and keeps
15:23:58working on that. So that means the
15:24:00compaction is working well. We don't
15:24:02have to worry about that. I'll just go
15:24:04back to config.tml and remove this
15:24:06context window of 16,000 because 256,000
15:24:10is our limit. Let's get back to that.
15:24:12Now the next thing I want to do for
15:24:14context management is pruning. What is
15:24:17pruning and what problem does it solve?
15:24:20So just imagine that you know you've
15:24:22been debugging a complex codebase not a
15:24:24simple codebase a pretty complex code
15:24:27base and you've asked your agent to do
15:24:28it. your conversation with the agent is
15:24:31now let's say 100 turns or even let's
15:24:34say 50 plus turns and it contains
15:24:36everything file reads tool calls error
15:24:39messages failed attempts all the
15:24:40successful fixes even the LLM
15:24:43hallucinations that might exist so the
15:24:45context window is now stuffed with
15:24:47information most of which is no longer
15:24:50relevant to the problem we're trying to
15:24:52solve right now so it is a problem right
15:24:55we're burning tokens which directly
15:24:58correlates to Honey, we are approaching
15:25:00the context limits and then we'll have
15:25:02to do compaction and compaction just
15:25:04results in kind of lower quality because
15:25:08it has to reread the files and stuff.
15:25:11And the third problem which is quite
15:25:13overlooked is that the model starts
15:25:16getting confused because it's trying to
15:25:18attend to a bunch of irrelevant and old
15:25:21information. So how do we solve this
15:25:23problem? Well, you can think about it in
15:25:26the way your brain would do it. So let's
15:25:29say you are trying to do it manually. AI
15:25:31agents don't exist as of now. You're
15:25:33still in 2021. So you finally fix a bug.
15:25:37Let's say after an hour of debugging,
15:25:39the good old days. And once you do that,
15:25:42your brain naturally forgets the problem
15:25:45that you were facing and like what the
15:25:47exact specific line numbers and code
15:25:50pieces were. you just remember something
15:25:53like oh right the issue was let's say
15:25:56some race condition that was present in
15:25:58the O middleware. So you compress the
15:26:00session of what's not relevant into a
15:26:03simple summary. Right now pruning is
15:26:07teaching the agent to do the same thing.
15:26:09And there are few strategies in pruning.
15:26:11One is a sliding window pruning
15:26:13technique. The next one is
15:26:14summarizationbased pruning. And then
15:26:16there's relevance-based pruning as well.
15:26:19in sliding window. This is the one that
15:26:21we are going to do. It just keeps the
15:26:23last n messages and drops everything
15:26:26else. So let's say it will keep last
15:26:2920,000 tokens and remove everything
15:26:32before that. Then there's summarization
15:26:34based pruning meaning it will
15:26:36periodically pause and ask an LLM to
15:26:38summarize of what happened so far and it
15:26:41will replace the old messages with the
15:26:43summary. So it's quite similar to the
15:26:46compaction part. And then there's
15:26:48relevance-based pruning. So it will
15:26:50score each message or turn by how
15:26:51relevant it is to the current task that
15:26:53the agent is trying to solve and it will
15:26:56keep all the higher relevant stuff. It
15:26:57will prune away all the low relevant
15:27:00stuff. So we going to go ahead with the
15:27:03sliding window approach. Specifically
15:27:05our approach is going to prune the old
15:27:08tool outputs. So what it will do is go
15:27:11backwards through the tool results until
15:27:13there are let's say 20,000 tokens to
15:27:16save and then it will clear off the
15:27:18output of those older tool results and
15:27:21that's it. This is based on open codes
15:27:24pruning approach and we're just going to
15:27:27do that. So we're going to go to this
15:27:29context manager py and here I'm going to
15:27:32create a new function. This function is
15:27:34prune tool outputs. We're going to get a
15:27:38self here. I'm going to return an
15:27:40integer. If even if you don't return
15:27:42anything, it's totally fine. But my
15:27:44point is to return the number of tokens
15:27:47that have been pruned. So the very first
15:27:50thing I'll do is count the number of
15:27:51user messages there are. If there are
15:27:53less than two messages, I don't need to
15:27:55prune anything. There's no need. But
15:27:58third message onwards, I can start into
15:28:00pruning. So I'll have user message
15:28:02count, which is equal to sum. And let's
15:28:05say we'll add up for every message in
15:28:09cell log messages. And we'll do that if
15:28:12message.rule is equal to user. Great. So
15:28:15that is the user message count. If the
15:28:17user message count is less than two,
15:28:19then we are going to return zero that we
15:28:22did not prune any tool output. After
15:28:25that I'm going to go over every message
15:28:27in selfload messages. But once I do
15:28:30that, what am I doing? I'm going over
15:28:32the system prompt. then the first user
15:28:34message, then the first assistant
15:28:36message, then the first tool message,
15:28:39and so on. What I want to do is go in
15:28:41the reverse order because the idea here
15:28:44is that the old tool calls don't matter
15:28:47anymore. The new tool calls matter a lot
15:28:49more. So, I'll do reversed cell do
15:28:53messages so that we go from the newest
15:28:57messages to the oldest messages. and we
15:29:00only want to process the tool result
15:29:02messages because anything else won't
15:29:05take that much space. If the assistant
15:29:07says I want to call a tool, that doesn't
15:29:09take up much tokens. What takes up much
15:29:12tokens is the tool result. If the role
15:29:16is tool, and we're telling the LLM that
15:29:19yeah, this is what the output of the
15:29:21tool is. That is what costs messages.
15:29:25So, we'll have tool call ID as well. In
15:29:27this case, what we want to do is prune.
15:29:30So we'll have message dot token count.
15:29:33Now this is the reason why for every
15:29:35message we had stored a token count
15:29:38associated with it. So these are the
15:29:40number of tokens. Now this token count
15:29:42can be a null value. So if it is a null
15:29:46value in that case I'll count the number
15:29:48of tokens. Pass in message.content
15:29:52and self domodel name. But most likely
15:29:55the token count is going to be present.
15:29:58After that what I can do is store the
15:30:00number of tokens that are present in
15:30:03total. So total tokens is zero. And then
15:30:05I just do total tokens plus equals
15:30:08tokens.
15:30:09And now I have to check if the total
15:30:12number of tokens that we have exceed a
15:30:15threshold. If it exceeds a threshold,
15:30:17what I want to do is start pruning. So
15:30:20I'll define a variable at the top in
15:30:22context manager and I can just call it
15:30:25let's say prune protect tokens and let's
15:30:29set it to 40,000.
15:30:32These are the values that open code also
15:30:34uses. Then we have prune minimum tokens
15:30:37and let's say this is 20,000. So prune
15:30:41protect tokens means that we keep the
15:30:43last 40,000 tokens of tool outputs and
15:30:47just prune everything else. So we are
15:30:50protecting 40,000 tokens of the most
15:30:53recent tool outputs and prune minimum
15:30:56tokens means that we prune only if we
15:30:59hit 20,000 tokens to save. That means
15:31:03let's say we have 200,000 tokens of just
15:31:06tool outputs in our conversation
15:31:09history. In that case what's going to
15:31:11happen is it will check yeah there are
15:31:1340,000 tokens of tool outputs. Great. So
15:31:17we are going to protect those 40,000
15:31:19tokens
15:31:21in terms of recency and we will only
15:31:24prune away if we can free at least these
15:31:2820,000 tokens and after 40,000 yeah
15:31:32we'll be able to save 20,000 tokens. So
15:31:34that's great because in reality there
15:31:37are 160,000 tokens more to delete,
15:31:40right? Because if 40,000 are being saved
15:31:43and 200,000 tokens are being used in the
15:31:46conversation history, 160,000 tokens
15:31:49need to be deleted. So it's greater than
15:31:5120,000. So yeah, we're going to go ahead
15:31:53and delete it. So I can come back over
15:31:56here and just check if the total number
15:31:58of tokens exceeds self dot prune minimum
15:32:03tokens or let's say prune out protect
15:32:05tokens in that case yeah we're going to
15:32:08start pruning everything so I'll keep
15:32:10track of the pruned tokens which is
15:32:13equal to zero and then I'll just do
15:32:16prune tokens plus equals tokens for this
15:32:20particular message and then I can also
15:32:24have a list of what things do I want to
15:32:27prune. So I'll have also this should be
15:32:30equals then I'll have a list to prune
15:32:32which will be a list of message items
15:32:35and initially an empty list and I'll
15:32:38just append to that list. So I have twe
15:32:40dot append and I'll pass in the message.
15:32:43Why are we storing message in this twe?
15:32:46Why are we not deleting the tool output
15:32:48here itself? The reason for it is after
15:32:50the for loop is over, we have the total
15:32:53number of prune tokens, right? How many
15:32:56tokens are we going to remove? If those
15:32:59prune tokens are greater than the
15:33:02minimum number of tokens that are
15:33:03required, in that case I can just go
15:33:06ahead and return zero. Or let's say if
15:33:09it is less than self.prone prune minimum
15:33:11tokens because if it is greater than I
15:33:14want to prune but if it is less than
15:33:18that means we have not reached our
15:33:1920,000 limit in that case I'll just
15:33:22return zero that I don't want to prune
15:33:24this tool output there's no we cannot
15:33:28free up at least these number of tokens
15:33:31obviously you can set this to zero so
15:33:34you just kind of you know remove all of
15:33:36the tool outputs after protecting the
15:33:39first 40,000 chosen but yeah your
15:33:42choice. After that I can keep track of a
15:33:45variable that will help me know the
15:33:47number of messages I marked as pruned
15:33:50and you know deleted their tool calls
15:33:53from. This is just there so that I can
15:33:56return this prune count at the end. The
15:33:58integer value that's returned from here
15:34:00really has no value. We're not going to
15:34:02do anything of it. But just in case you
15:34:05want to store it in some sort of session
15:34:09so that you can do a marketing gimmick
15:34:11or something related to this agent that
15:34:13yeah, we saved you 40,000 tokens because
15:34:16we are such a good tool. You can go
15:34:19ahead and do that. I'm not interested in
15:34:21it. Now I'll go over every message in
15:34:23this two prune list. And all I need to
15:34:26do here is message dot content is equal
15:34:29to and then I'll just mark this as old
15:34:33tool result content cleared. Okay, after
15:34:38that I'll just set the token count to
15:34:42count tokens and I'll pass in the
15:34:45message dotcontent and I'll have self
15:34:47domodel name. Now there's one edge case
15:34:51that we've not talked about and also by
15:34:52the way you have to do pruned count plus
15:34:56equals 1 year so that you know this is
15:34:59correctly updated but the edge case that
15:35:01I'm talking about is what if we've
15:35:04pruned the tool outputs you know we
15:35:07reached a certain limit we started
15:35:08pruning the outputs and then we added
15:35:10more messages and now it's time to prune
15:35:13again in that case certain messages
15:35:17might be pruned but still they might go
15:35:20over here and we might try to prune it
15:35:23again. Why do we want that redundancy?
15:35:25So to stop that what we can do is go to
15:35:28our message item here and have an
15:35:31attribute which is pruned at which is
15:35:35either of the type of datetime. So we'll
15:35:38import from date time date time or it
15:35:41can be null as well and by default it is
15:35:43null. This is only set as the value if
15:35:47the tool result was pruned and therefore
15:35:50this value is only present for tool
15:35:53results. Now what I need to do is scroll
15:35:55down and here I'll just set message dotp
15:35:58pruned at equal to datetime dot now
15:36:03because the pruning is now done. Then I
15:36:05can scroll up a little bit where we have
15:36:07for message and reversed. And if the
15:36:09message roll is tool in that case first
15:36:13I'll just check if the message.tp pruned
15:36:15at is not null. In that case we have hit
15:36:19the already already in that case we have
15:36:22already hit the compacted tool results
15:36:25which is like the pruning boundary. So
15:36:27we have hit the last message that was
15:36:29pruned. All the messages after that are
15:36:31obviously going to be pruned. So we'll
15:36:33just break out of this loop and we don't
15:36:35have to prune anymore. Otherwise what
15:36:37will happen is it will start taking into
15:36:40consideration all the other messages
15:36:43that were pruned and it will count the
15:36:46the these number of tokens for let's say
15:36:49100 tools and then it will add it to
15:36:51this prune tokens and it might result in
15:36:55prune minimum tokens limit being hit.
15:36:58Why do we want to do that? That's why we
15:37:01break out if pruned at is already true.
15:37:04And that is it about pruning the tool
15:37:06outputs. Now I'll just go to agent.py
15:37:10and call this function. The first place
15:37:12I would like to do it is when we have no
15:37:15more tool calls to do because when we
15:37:18have no more tool calls to do we're just
15:37:20going to return from here right and
15:37:22we've calculated the usage. After
15:37:24calculating the usage, we can just say,
15:37:27yeah, I want to prune it. So, I'll have
15:37:31self dot session dot context manager dot
15:37:35prune the tool outputs. And that will
15:37:38return an integer, but an integer I
15:37:40don't care about. If you want, you can
15:37:42log it out and see for yourself. But
15:37:45yeah, I'll just copy this line again.
15:37:48And before we end this turn, I'll just
15:37:51prune again. So just after usage again
15:37:54I'll just check hey can we prune the
15:37:56tool outputs properly and if we can
15:38:01all the tool outputs will be pruned.
15:38:03This obviously works on a bigger scale
15:38:05when you know we've consumed 150,000
15:38:08tokens or so. If you want please go
15:38:10ahead and experiment with it. It should
15:38:13work as expected and result in no not
15:38:16much loss of the quality of the agent
15:38:21while saving the costs. So that's good.
15:38:24I'm not going to run it because I'll
15:38:25just hit my free limit tier for today
15:38:28and I don't want to do that. Now let's
15:38:30move on to the next feature. And what we
15:38:32are going to work on next is the
15:38:34approval system. So you might remember
15:38:36that in our tools base. py file we had
15:38:39created something of a function called
15:38:42get confirmation. This function's job
15:38:44was to give us information about what is
15:38:47happening in this particular tool. For
15:38:50example, we will have the edit tool or
15:38:53the right tool and that is going to
15:38:54return this tool confirmation. It's not
15:38:56going to return tool invocation by the
15:38:58way. It's going to return tool
15:39:00confirmation or a null value. So if
15:39:02something is not mutating for example a
15:39:05read file it returned null. But just in
15:39:08case we were going to have something
15:39:11like a write tool or an edit tool we
15:39:13just sent out some information regarding
15:39:15what this tool is going to do. And after
15:39:18that will come the approval system. So
15:39:20approval system will take the details
15:39:22that we give from this get confirmation
15:39:24function that is within the tool. it
15:39:27will call it and then it will check if
15:39:30that is safe to run. If it's safe to run
15:39:33then we are going to go ahead and
15:39:35execute it but if it's not then we are
15:39:37going to block it at that particular
15:39:39point ask the user for their
15:39:41confirmation so the user can just say
15:39:44that hey listen this is fine with me you
15:39:47can run this command I think it's safe
15:39:48to run it and then the things will
15:39:51happen so obviously this is not needed
15:39:54for the tools that are not mutating
15:39:56anything because if the state is not
15:39:57mutated any operation is just fine but
15:40:00if the state is mut computated. In that
15:40:02case, we'll need get confirmation. And
15:40:04get confirmation is a function we only
15:40:06created in this base tool. It's not
15:40:08created in any of the tools that extends
15:40:11the base tool. It's not created in edit
15:40:14file or let's say write file or shell.
15:40:17These are the three places that where we
15:40:19are going to add this get confirmation
15:40:22function. So let's go ahead and add
15:40:25that. But first of all, we need some
15:40:27information about the approval system.
15:40:30The approval system is basically in the
15:40:33config.tml file. The user can specify
15:40:36what type of approval they prefer and it
15:40:39can be one of multiple values. So let's
15:40:41just go to config. py and here I'm going
15:40:43to scroll down and create a new
15:40:46attribute not an MCP sorry in the config
15:40:49and we're going to call this approval
15:40:52and this is going to be a enum. So we'll
15:40:54have approval policy like that and by
15:40:57default it's going to be approval policy
15:41:00dot something known as on request. I'll
15:41:03talk about that in just a minute. But
15:41:05yeah, now let's go ahead and create this
15:41:07enum of approval policy where we are
15:41:10going to have a string enum. We've
15:41:12created enum multiple times. So that's
15:41:14that. The first value here is obviously
15:41:17on request something we just created. On
15:41:19request means that it will ask for
15:41:22confirmation based on whether the
15:41:24operation is mutating or not. So if
15:41:27let's say there's a write file or edit
15:41:29file, it's mutating. So it will ask for
15:41:32approval. If it's outside the current
15:41:34working directory and if it's not
15:41:37mutating like read file or anything
15:41:39else, it will not ask. It will just
15:41:41start reading the files. Then we have on
15:41:44failure. So whenever there's a failure,
15:41:45what to do? Self-explanatory. Then we
15:41:48have auto which means it will auto
15:41:51approve all the safe commands but it
15:41:53will autoreject all the dangerous
15:41:55commands. Then we have autoedit which
15:41:58just means that it will auto approve
15:42:00edit related stuff but it will knock out
15:42:03everything else. After that we have
15:42:06never. So that means it never asks for
15:42:08any approval. And the final thing is
15:42:10yolo which is an acronym for you only
15:42:15live once. It just auto approves
15:42:17everything. Even if it's a dangerous
15:42:19command, whatever, it will just approve
15:42:20every single thing. And people do like
15:42:23to use this command for some reason. I
15:42:25would definitely not recommend it. But
15:42:27yeah, then we have approval on request
15:42:30by default. Now the next thing I'll do
15:42:33is create a new folder here. Let's call
15:42:36it safety. So it will contain all the
15:42:39safety related things. We're only going
15:42:41to have one safety mechanism in our
15:42:43thing which is approval. py. But later
15:42:45on maybe you want to add sandboxing
15:42:47feature or something else related to
15:42:49safety. You can expand this folder. Just
15:42:52like everything we've done before, we
15:42:54going to create a class of approval
15:42:56manager. We're going to get an init
15:42:58function. And within this init function,
15:43:00the first thing we're going to get is
15:43:01the approval policy. You know what is
15:43:03the approval policy we signed up for or
15:43:07what did the user set up. And if the
15:43:09user did not set up, by default it is on
15:43:11request. Then we have the current
15:43:13working directory as well. And the next
15:43:16thing we're going to get is confirmation
15:43:18call back. So this is a function that we
15:43:20are going to actually get from the main.
15:43:22py. So within main.py here we're going
15:43:24to create a function called confirmation
15:43:27call back or confirmation where you know
15:43:30it's the UI. So
15:43:33whatever UI you have you can just pass
15:43:35it to this approval system. Another way
15:43:38of doing this is you can just yield the
15:43:41agent event and then listen for the
15:43:43agent event in this process message and
15:43:46then call the specific UI. But I'm just
15:43:48passing in the entire call back so that
15:43:51whenever we want we can display the UI.
15:43:54So confirmation call back does just
15:43:56that. This is going to be a callable. So
15:43:59it's not going to be a function that's
15:44:02already called, right? It's going to be
15:44:03a function itself. So we are going to
15:44:05have a call where we have tool
15:44:08confirmation. Let me import that from
15:44:10tools. py or tools base. py and then we
15:44:14have awaitable boolean. Let me also
15:44:17import avaitable. And if this function
15:44:19is not specified then we have null and
15:44:22by default null is specified. So just in
15:44:24case the confirmation call back is not
15:44:26passed in. Now let's instantiate
15:44:28everything. And if you're still confused
15:44:30about this confirmation callback, don't
15:44:32worry. I'll explain it to you when we
15:44:33start to use it, but I'll just
15:44:35initialize it as and when we go. Then
15:44:38self.approval policy is approval policy.
15:44:40Then we have self.curren working
15:44:42directory is equal to current working
15:44:43directory and self do.confirmation call
15:44:45back is equal to confirmation call back.
15:44:48After that obviously from this init
15:44:51function we're going to return null. The
15:44:53first function that we want to have here
15:44:55is check approval.
15:44:57So based on this approval context that
15:45:01we get, we have to give an approval
15:45:03decision. And the approval decision is
15:45:06is the approval approved, is it rejected
15:45:10or does it need confirmation from the
15:45:12user? If it needs confirmation from the
15:45:14user, we're going to ask the user that
15:45:16hey please specify yes or no based on
15:45:19the dialogue box that we show on the UI
15:45:22part. So obviously we need some context
15:45:24to make an informed decision of whether
15:45:27the approval was given or not. So we
15:45:29have to create two classes. One for the
15:45:32context whatever we going to get from
15:45:33the parameters and the second one is the
15:45:36decision of the approval. So are we
15:45:39approving? Are we rejecting? What are we
15:45:41doing? So let's quickly create both of
15:45:43them. The first one is an enum of
15:45:46approval decision. So let's create that.
15:45:49It's going to be a string enum and we'll
15:45:52import from enum enum. Then we have
15:45:55approved which is just approved like
15:45:58that. Then there's rejected which is
15:46:00just like that. And then we have needs
15:46:03confirmation which is just like that.
15:46:07Awesome. After that, we're going to
15:46:10create a new class called approval
15:46:13context, which is just the context
15:46:16required for us to make an approval
15:46:18decision. So, we have at the rate data
15:46:20class. We're going to import from data
15:46:22classes data class. And then we have
15:46:24approval context correctly passed in.
15:46:27What context do we need to make a
15:46:29decision? Well, the tool name would be
15:46:31nice, something we would like. Then
15:46:34there's parameters which is going to be
15:46:37a dictionary of string comma any. We
15:46:39know what the parameters mean. We have
15:46:41talked about that in tool confirmation
15:46:43as well. By the way, all of these values
15:46:45most likely are going to come in from
15:46:47the tool invocation.
15:46:49So if you remember in write file.py if
15:46:52we go we had invocation right that gave
15:46:55us the parameters. So context is just
15:46:57going to have those filled in from
15:46:59there. After that we have is mutating.
15:47:03It's going to be a boolean value. Then
15:47:05we have the affected parts so that we
15:47:08know what parts are affected. This is
15:47:10very important because for write file or
15:47:13edit file especially if we know which
15:47:16parts are affected we can just compare
15:47:19it with the current working directory if
15:47:21the path is within our current working
15:47:22directory. In that case we auto approve.
15:47:26Otherwise we can't auto approve.
15:47:28Obviously that also depends on what type
15:47:31of approval policy they have. But let's
15:47:33just save the approval policies on
15:47:35request which is the default one. All we
15:47:37need to do is check the current working
15:47:39directory and see if the path is within
15:47:41that. Cool. Then we have a list of paths
15:47:46that are going to be passed in here.
15:47:48Then we have a command string, null and
15:47:51by default it is null. The command will
15:47:54be useful when we have shell related
15:47:56operations or shell related approval
15:47:59because some commands can be dangerous
15:48:01and some commands can be safe. So we're
15:48:04going to pass that as well. Then we have
15:48:07is dangerous just in case we've
15:48:09pre-tagged it of what the approval is.
15:48:12So we have that by as false by default.
15:48:15Now we can take this approval context
15:48:17through the parameters. So we have
15:48:19context approval context and here we are
15:48:22going to return approval decision and
15:48:25now we can go ahead and create this
15:48:27function. So how do we check for the
15:48:30approval? Well first of all if if
15:48:34anything is not mutating that means it's
15:48:37auto approved no matter what approval
15:48:40policy we have. If it's not mutating it
15:48:42will not cause any harm. Let's just
15:48:44approve it. So we'll have if not context
15:48:47dot is mutating. By the way, this is not
15:48:50always the case that if it's not
15:48:53mutating, it's going to be fine because
15:48:55your LLM can obviously read the env
15:48:58files. So yeah, you can handle that case
15:49:02as well and any other case you can think
15:49:04of. But I'm going easy over here. We
15:49:06have return approval decision dot
15:49:09approved. After that we have to check
15:49:12for the command safety. And after that
15:49:15we'll check for the path safety. That's
15:49:17it. That's how the approval system
15:49:19works. So first one is checking the
15:49:22command safety. So if the context do
15:49:24command is specified then I want to
15:49:26check if this command is safe or not.
15:49:29And for that maybe I can just go ahead
15:49:31and create a helper function here. Let's
15:49:33call this assess path safety. It will
15:49:37pass in the path which is going to be
15:49:40the path and then we'll have the oh by
15:49:43the way this is not path safety sorry
15:49:45it's command safety I was just thinking
15:49:47about path for some reason and then we
15:49:50have command as the first or actually
15:49:52the first one should be self then we
15:49:54have command and then we have policy and
15:49:58we don't need to take the policy we
15:50:00already have it in the constructor so to
15:50:02assess the command safety what do we do
15:50:04also it's going to return an approval
15:50:07decision from here. So yeah, how do we
15:50:09determine command safety? Well, first we
15:50:12have to create a list of commands that
15:50:15are either dangerous or they are safe,
15:50:18right? That's how we know if the command
15:50:20is safe or it's dangerous. So I'll just
15:50:24copy paste. You can also copy paste from
15:50:26the GitHub repository. But this is my
15:50:29entire list. These are all of the
15:50:31dangerous patterns related to file
15:50:34system destruction, disk operations,
15:50:36system control and all of that. Then we
15:50:39have safe patterns that can be auto
15:50:41approved. Now a question that you might
15:50:44have is why do we need both dangerous
15:50:46and safe patterns? Well, we need both of
15:50:48them because we have multiple approval
15:50:50policies. One approval policy might just
15:50:53check if it's dangerous and the other
15:50:56approval policy might just check if it's
15:50:58safe. And based on that we have to give
15:51:00an approval decision. So that's why we
15:51:02have both. Now once we write the code
15:51:05for command safety you'll be able to
15:51:07understand better. But first I would
15:51:10also like to create some helper
15:51:11functions here. The first one is is
15:51:14dangerous command function. It takes in
15:51:17a command string and it returns a
15:51:20boolean value. What it checks is if a
15:51:22command matches any dangerous patterns
15:51:24that are specified here. All of these
15:51:26patterns are in the reg x format. If you
15:51:28look at it closely, this is remove which
15:51:31is a command. Then you have a reg x
15:51:33pattern of back slash s plus that means
15:51:37you can have multiple spaces. After that
15:51:41you can have dash rf which is optional
15:51:45or you can also have d-recursive meaning
15:51:48it will just remove all of the folders
15:51:50present recursively and you can go ahead
15:51:54and try to understand each reg x pattern
15:51:56here but yeah so what we need to do here
15:51:59is go over every pattern in this
15:52:01dangerous patterns and if the reg x dot
15:52:04search and we've already looked into the
15:52:06search from reg x in some previous code.
15:52:10So we'll just pass in pattern the
15:52:13command because we have to search in
15:52:15this pattern if this command matches and
15:52:18we'll ignore the case. If this is true
15:52:22then we'll return true because it is a
15:52:24dangerous command. Otherwise after the
15:52:26for loop we'll return false that none of
15:52:28the patterns matched. Now similar to
15:52:30this we will also have a function of is
15:52:33safe command. We'll get a command
15:52:35string. we'll search in every safe
15:52:38pattern and we'll just return true if
15:52:41this is the case otherwise we'll return
15:52:44false. Now I can just use both of these
15:52:46functions over here. So if the context
15:52:49docomand is there we have to call in
15:52:51self.assess command safety and within
15:52:54this assess command safety I'll check
15:52:57first for dangerous commands. So if it
15:53:00is a dangerous command in that case I
15:53:02need to pass in the command and I'll
15:53:05just return approval decision
15:53:07dotrejected right because if it's a
15:53:10dangerous command I want to autoreject
15:53:12but there's one approval policy that
15:53:15will have to approve no matter what
15:53:18which is the yolo. So even if it's a
15:53:20dangerous command, we'll just check if
15:53:22the policy or let's say self do.approval
15:53:26policy is equal to approval policy dot
15:53:29yolo. In that case also we'll have
15:53:31return approval policy dot approved or
15:53:35approval decision sorry approval
15:53:37decision dot approved right after that
15:53:41we'll check for other things. So again,
15:53:44we just want to approve everything that
15:53:47is YOLO, right? So I can just extract it
15:53:50at the top here. So if the approval
15:53:52policy is YOLO, we'll just approve no
15:53:56matter what. After that, we can check if
15:53:58it's a dangerous command and not YOLO
15:54:00because if it's YOLO, it's already been
15:54:02approved. Then we reject it. And now
15:54:05I've checked for the dangerous commands
15:54:07as well. I'll just check if the policy
15:54:10is equal to approval policy dot never
15:54:14and never mode just rejects all of the
15:54:17mutations that might happen. So we'll
15:54:19just check here if it's a safe command
15:54:22because if it is a safe command then we
15:54:24want to approve it. If it's not a safe
15:54:26command then all I need to do is return
15:54:28approval decision dot rejected. As
15:54:32mentioned never will just reject all of
15:54:34the mutations that might happen. And is
15:54:37safe command will just check for that
15:54:39because if you notice all of the safe
15:54:42patterns here are list directory echo
15:54:45cat and then the development tools all
15:54:48the readonly stuff. So you know it's
15:54:51pretty safe. Now let's say if the
15:54:54approval is auto or failure in that case
15:54:58we just want to approve. So we'll check
15:55:00if self.approval approval policy is in
15:55:04either approval policy doauto or
15:55:08approval policy dot on failure. In that
15:55:12case, we'll just return approval policy
15:55:15dot or approval decision dot approved.
15:55:18Sorry. Now the reason I can do this very
15:55:20safely is because it is not a dangerous
15:55:23command because we've already checked
15:55:24for dangerous command. So whatever we
15:55:27get here is not a dangerous command. So
15:55:29we just auto approve it. But for never
15:55:32we have to deny all of the things that
15:55:35are mutations. That's why we have to
15:55:37check if it's a safe command. After that
15:55:39we have the autoedit. So if self
15:55:41do.approval policy is equal to approval
15:55:44policy dota autoedit.
15:55:47In that case we have to check if it's a
15:55:50safe command because autoedit will
15:55:52approve the edits but not any shell
15:55:55command for example. So we'll just have
15:55:58if it is a safe command then we have
15:56:01command passed in then we'll return
15:56:03approval decision
15:56:05dot approved otherwise we will return
15:56:08approval decision dot needs confirmation
15:56:12this time it's not going to reject it up
15:56:14front it just asks for the confirmation
15:56:18and this is going to be a big deal for
15:56:20us because whenever we have needs
15:56:21confirmation
15:56:23we will bring up our dialogue box to
15:56:26play. After that, finally we have the
15:56:30last case which is is safe command. If
15:56:34it is a safe command, then we will have
15:56:36return approval decision dot approved
15:56:40because at this point it's going to be
15:56:42on request. We have handled autoedit. We
15:56:45have handled needs confirmation. We have
15:56:47handled autoedit. We have handled on
15:56:49failure. We have handled auto and all
15:56:52the other things like never and yolo.
15:56:55One thing we've not handled is the on
15:56:58request. And if it's on request, then
15:57:00I'll just check if it's a safe command.
15:57:02And if it's not a safe command, then I'm
15:57:04just going to do return approval
15:57:06decision dot needs confirmation. I won't
15:57:08outright reject it because on request
15:57:11means we have to ask for the user
15:57:14confirmation if it's not a dangerous
15:57:17command and if it's a safe command, all
15:57:19good to go. So yeah, that's about it.
15:57:22This was the assess command safety. Now
15:57:25I can just have the decision here and I
15:57:28need to pass in the command as well to
15:57:31this assist command safety. So I'll have
15:57:34context
15:57:35dot command passed in and we'll say if
15:57:38the decision year that we got is not
15:57:41equal to approval decision dot needs
15:57:43confirmation. In that case, we'll return
15:57:46the decision because if it needs
15:57:48confirmation, then we're going to create
15:57:52another function called request
15:57:53confirmation and ask for the user's
15:57:55confirmation. Anyways, if the command is
15:57:58done, that's good. Now, we need to check
15:58:00for the path safety. So, we'll go over
15:58:02every path in the context dot affected
15:58:05paths and then we have to assess the
15:58:08path safety. Path safety as I mentioned
15:58:11is very easy. If path is relative to the
15:58:16current working directory that we get
15:58:18from the constructor that means the path
15:58:21is present within the current working
15:58:23directory. In that case the approval is
15:58:26approved because we don't need any
15:58:29confirmation. Then anything happening
15:58:31outside of our current working directory
15:58:33needs approval. So let's create a
15:58:35variable here. Let's say path decision
15:58:38which is equal to and we can instantiate
15:58:42this to be approval decision dot needs
15:58:45confirmation and if the path is relative
15:58:47to the current working directory then we
15:58:49are going to have approval decision dot
15:58:52approved right so we can just set path
15:58:54decision equal to approval decision dot
15:58:58approved and if it's not relative to
15:59:01that then it needs confirmation which
15:59:03we've already defined by default And if
15:59:05it does need confirmation then we're
15:59:08just going to return the path decision.
15:59:11Now after all of these paths are also
15:59:13done I want to check if there's explicit
15:59:16dangerous flag. By the way this approved
15:59:19here does not return because think about
15:59:22it when we're going over all of the
15:59:24paths and we just approve one path
15:59:27that's correct. That's going to be a
15:59:29problem because the second path in this
15:59:31affected paths might be something that
15:59:34should not be approved. It should need
15:59:36confirmation because let's say the one
15:59:39path was changing approval. py which is
15:59:41in AI agent directory but the next one
15:59:44is writing to the desktop folder which
15:59:46is not in AI agent. That means we've
15:59:50returned approved but it should not be
15:59:52approved. That's the key issue here.
15:59:54Finally, let's say at this point
15:59:56everything feels safe. We'll just have
15:59:59if context dot is dangerous. That means
16:00:03there's an explicit dangerous flag set
16:00:05up. In that case, I want to return
16:00:08approval decision dot needs
16:00:11confirmation, right? Because it does
16:00:13seem potentially dangerous to us. But we
16:00:16don't want to do it if the approval
16:00:19policy is yolo. That means we always
16:00:21approve. So we have if self.approval
16:00:23approval policy is equal to approval
16:00:25policy dot yolo. In that case, we'll
16:00:29just return approval decision dot
16:00:32approved. And let's say after all of
16:00:35this also we did not get anything wrong.
16:00:39That means we just have to return
16:00:41approved because let's say a path was
16:00:43being modified here. Everything is in
16:00:46current working directory. So as of now
16:00:48we have approved. Then nothing is
16:00:50dangerous. That means we still have
16:00:52approved and finally we'll just approve
16:00:55it. Correct? So that is for checking the
16:00:57approval. Now the next thing I want to
16:00:59work on is requesting the confirmation.
16:01:02It's not a difficult thing to do. All we
16:01:04need to do is create an asynchronous
16:01:06function called request confirmation. We
16:01:09get a self. We get a confirmation which
16:01:11is of the type of tool confirmation. And
16:01:14it's going to return a boolean value of
16:01:16whether the confirmation is requested or
16:01:18not. And we'll just check if the
16:01:21confirmation call back is present. Let
16:01:24me just get that confirmation call back.
16:01:27And if it is then I want to call that
16:01:29and it's going to be an awaitable the
16:01:31response of that. That's why I'm going
16:01:33to do self dot confirmation
16:01:36call back. And then I'll pass in the
16:01:38tool confirmation. Whatever is the
16:01:40result, it's going to be a boolean value
16:01:42because that's how we defined it in the
16:01:43constructor as well. And then we just
16:01:45return the result. But if the
16:01:47confirmation call back is not there,
16:01:49that means we don't have any call back.
16:01:51We can auto approve and we'll just
16:01:53return true. So that's it. Nothing too
16:01:56complex here. The only reason this
16:01:58function was created is because this
16:02:00entire approval manager scenario is
16:02:02going to happen in the base. py file and
16:02:05there we don't have access to the
16:02:07confirmation callback. We can pass it in
16:02:09but it will get a bit messy. I don't
16:02:11want to do that. So instead I'll just
16:02:14attach it to the approval manager and
16:02:17create a wrapper around it. This is also
16:02:19not a good solution. You might want to
16:02:21rearchitect the system but it gets the
16:02:24job done. Now we'll just go to the base.
16:02:27py file and actually sorry instead of
16:02:29saying base. py I wanted to say registry
16:02:32which is a tool registry because
16:02:35whenever we try to invoke the function
16:02:37this invoke function is called within
16:02:40the agent. py if you remember and it
16:02:43just calls the tool. So before we call
16:02:46any tool that's the time I want to check
16:02:50for the approval system right so think
16:02:52about it the first thing that happens is
16:02:55the LLM gives us a tool call we display
16:02:57to the user that yeah this is the tool
16:02:59call that we're going to do the tool
16:03:01call has started so let's say we're
16:03:05going to write a file and what is the
16:03:08content of what we are going to write
16:03:10xyz so that shows up right so that is
16:03:13the tool called start after that we
16:03:16generally display the output but before
16:03:19displaying the output and the output is
16:03:21fetched from here by the way which is
16:03:23tool.execute execute. Instead of doing
16:03:25that, I can just have the approval layer
16:03:28here. So, if the approval goes through,
16:03:30I'll have the request confirmation
16:03:33dialogue box showing up and we can ask
16:03:35the user, do you approve this or do you
16:03:38not approve this? Yes or no. If the user
16:03:41says yes, then we can display the next
16:03:43dialogue box which shows all of the
16:03:45details that were done. Right? So, we're
16:03:48going to get the approval manager from
16:03:50this invoke function. So I'll have
16:03:52approval manager and this approval
16:03:55manager can be null as well and by
16:03:58default it is going to be null. So let
16:04:01me import it and if this approval
16:04:03manager is with us we're going to do
16:04:06some stuff and it's not going to be self
16:04:08it's just if approval manager is there
16:04:10I'll just call the tool dot get
16:04:14confirmation because if you remember if
16:04:17nothing is mutating we return null. So
16:04:20that's a potential null value. But if
16:04:22there is certain thing to be passed in,
16:04:24it will be passed in through this tool
16:04:26confirmation. Now within write file,
16:04:29edit file and shell.py, we'll have to
16:04:32update this get confirmation because the
16:04:34point of this tool confirmation is
16:04:36passing in the context about the tool.
16:04:39What is it doing? So that we have the
16:04:41context to make a decision. And I told
16:04:44you based on this tool confirmation that
16:04:46we're going to get, we're going to
16:04:48extract certain information and pass it
16:04:51to our approval system. So I'm going to
16:04:53take a detour here. What I'm going to do
16:04:55is go to our write file py write file
16:04:59tool class and create a function of
16:05:02getting the confirmation. So we have get
16:05:04confirmation. we get an invocation which
16:05:07is tool invocation and it's going to
16:05:10return either a tool confirmation and
16:05:14we'll import that from tools.base or a
16:05:16null value. Now obviously we will need
16:05:18the parameters to make an informed
16:05:20decision. So we'll have parameters write
16:05:23file parameters and we'll deconstruct it
16:05:26into invocation parameters just like
16:05:28what we've done multiple times before.
16:05:31The next thing is resolving the path
16:05:33similar to what we did in execute. So
16:05:35I'll just copy paste it. Now I have the
16:05:37path we going to write to. I will be
16:05:39returning the tool confirmation from
16:05:42here. I won't be calling the parent
16:05:43class because I know exactly what to
16:05:45return from here. The first thing is the
16:05:47tool name which is self dot name. Then
16:05:50we have the parameters which is equal to
16:05:53invocation dotparameters.
16:05:56Then we have the description. And for
16:05:59the description, we have to pass in
16:06:01maybe we can pass in the action that's
16:06:04taking place. If we're creating a new
16:06:06file or if we are updating a new file
16:06:09and all of that logic is created down
16:06:11here. If it's a new file, we have this
16:06:14action. And if it's a new file, if the
16:06:17part does not exist. So I'm just going
16:06:19to copy the logic pieces from here. So
16:06:22if it's a new file, if the path does not
16:06:25exist and then for the action, we can
16:06:28copy this line. So I'll just paste that
16:06:31in. Great. So I can pass in this action
16:06:34as the description. So let's say it says
16:06:37created file and then it passes in the
16:06:40path, whatever path we created the file
16:06:43in. Obviously, we need more information
16:06:46here. for example, the diff view because
16:06:48we also want to display to the user what
16:06:51is the thing that we are doing within
16:06:54this write file because it would be so
16:06:56weird to just tell the user that yeah we
16:06:58are writing in 10 lines. That's not
16:07:00enough information for the user to make
16:07:03a decision of approval if they want to
16:07:06approve this write file or they don't
16:07:08want to approve it. That makes sense.
16:07:10Within this tool confirmation, we're
16:07:12going to update it and add more UI
16:07:14elements. For example, we're going to
16:07:17have the diff. We also displayed that in
16:07:20the tool result, I believe. So, we add
16:07:23the diff here. Similarly, we're going to
16:07:24have it in this tool confirmation as
16:07:27well. Then, we have the command just in
16:07:30case we want to display that. Then, we
16:07:32have the affected paths is dangerous all
16:07:36of that. So, whatever we created in the
16:07:38approval. py for approval context. I
16:07:42told you all of that is going to come
16:07:44from the base. py essentially or let's
16:07:47say each tool. So we have command and
16:07:51then is dangerous and also the affected
16:07:54paths. So let's just copy that and maybe
16:07:57we can give it a default value here that
16:07:59the default factory is going to be an
16:08:01empty list. Now we can pass in the tool
16:08:04confirmation correctly within the write
16:08:06file and within the base py as well. Let
16:08:09me update it. Now we don't need to
16:08:11update this in the get confirmation
16:08:14method of base. py. The reason for that
16:08:17is these are all display elements. We
16:08:19don't want to do anything by default.
16:08:22We'll override them in the right file.
16:08:24So here we can pass in the div for
16:08:28example. And now we need to create the
16:08:29file div. And to create a file diff, it
16:08:32really depends on whether it's a new
16:08:34file or not. So that we can get the old
16:08:37content and create the file diff. So
16:08:39what I'm going to do is just copy this
16:08:41entire line. We're just copy pasting all
16:08:43of the logic from here. And if it's not
16:08:45a new file, we get the old content and
16:08:48let's initialize the old content to be
16:08:51an empty variable at the top here. After
16:08:54that we'll have diff is equal to file
16:08:58diff and then we can pass in the path.
16:09:01We can pass in the old content which is
16:09:04just old content. Then we can pass in
16:09:06the new content which is parameters dot
16:09:09content and then if it's a new file or
16:09:12not based on the variable is new file
16:09:15and then I can pass in the diff here.
16:09:18Great. After that we need the affected
16:09:21paths. The affected path is just the
16:09:23path that we are writing to. After that
16:09:26we have is dangerous and it's not
16:09:29dangerous if it is a new file because
16:09:32overwriting is more dangerous. The user
16:09:36might lose their previous content. So
16:09:38yeah, it is dangerous. So that's it
16:09:41about write file tool. I'll just copy
16:09:43this entire function and paste it in the
16:09:45edit file tool as well because a similar
16:09:48thing needs to be done there. So I'll
16:09:50just paste it here. We do get the tool
16:09:52confirmation and tool invocation. And
16:09:55obviously we also need the edit
16:09:57parameter. So let me copy that and get
16:10:00it here. After that we resolve the path.
16:10:02We again check if it is a new file. It's
16:10:05a new file. If the path does not exist,
16:10:07then we'll just check if it is a new
16:10:10file. In that case we want to create a
16:10:12diff which is going to be a file diff
16:10:15where the path is just the path. Then
16:10:18the old content is equal to empty string
16:10:20because that file did not exist
16:10:22previously. Then the new content is
16:10:24equal to parameters dot new string and
16:10:28then the new file is going to be true
16:10:31because yeah it is a new file and from
16:10:33here only we can just return the tool
16:10:35confirmation because we have enough
16:10:37information. So I'll pass in the tool
16:10:40name and everything. So let me just copy
16:10:42it and paste it in over here. So tool
16:10:45name is self.name name parameters is
16:10:47invocation.parameters parameters then we
16:10:50have create new file and then I pass in
16:10:53the path diff is this affected paths is
16:10:57just the path we are editing and is
16:10:59dangerous does not need to be passed in
16:11:02because it's dangerous if it's not a new
16:11:05file it is a new file so it's not
16:11:07dangerous
16:11:09great now if it is not a new file in
16:11:12that case I would like to read the old
16:11:15content and I'll just copy paste what we
16:11:18did previously to read the old content
16:11:21like that. So I'll just replace all of
16:11:23this with old content is equal to path
16:11:26dot read text and then we can check that
16:11:29if the parameter is replace all in that
16:11:32case the new content is equal to old
16:11:35content dotreplace and then we'll pass
16:11:37in the parameters do old string and then
16:11:41I'll pass in parameters dot new string
16:11:44as well. Now, if you're confused about
16:11:46that, you would like to revisit the edit
16:11:48section because we're just doing this
16:11:50part over here, just replacing
16:11:52everything. So, in the else block, we
16:11:55can pass in new content is equal to old
16:11:58content dotreplace. And then I'll pass
16:12:00in parameters dot old string parameters
16:12:04dot new string and then one because we
16:12:06only want to replace once. So, that is
16:12:09our new content. Now I'll create the
16:12:12diff where I'll pass in the new content
16:12:15as the new content variable. Path
16:12:18remains the same. Old content variable
16:12:20is already created and it's not a new
16:12:22file because I just checked above. It's
16:12:24not a new file. Then we don't really
16:12:27need this action variable. I know it's
16:12:30going to be editing the file. So I'll
16:12:32just say edit file. And this is the
16:12:35path. The diff is also created. The
16:12:37affected parts is this is dangerous.
16:12:40Well, we are editing the file, not
16:12:42really overwriting the file. So maybe we
16:12:45can set is dangerous to false here. And
16:12:47that is enough for edit tool. Now the
16:12:49final tool we have to worry about is
16:12:52shell because shell can mutate some
16:12:54things as well. And actually shell is
16:12:57going to be very easy. I'll just copy
16:12:59paste it. It's going to be much smaller
16:13:01than write file and edit file. The first
16:13:04thing is importing tool confirmation.
16:13:07Then we're going to have the shell
16:13:08parameters. Then we'll resolve. Well, we
16:13:10don't need to resolve path here because
16:13:13it's not a file related operation.
16:13:15There's no new file e either. So we we
16:13:18can actually just remove everything
16:13:20other than return tool confirmation. And
16:13:23the first thing I'll check here if is if
16:13:25it's blocked. So that means that the
16:13:27command that the shell gave out is
16:13:30blocked or not. So if the self dot is
16:13:35blocked command and I don't see it maybe
16:13:37I created it or I did not create a
16:13:39helper function for this. This is
16:13:41basically what is blocked command does.
16:13:44So I'll just copy it and paste it over
16:13:46here. You might want to create a utility
16:13:48function. But for every command in
16:13:50blocked if blocked is in the parameters
16:13:54docomand in that case we'll return the
16:13:57tool confirmation. And this tool
16:14:00confirmation is going to have is
16:14:02dangerous as true because it's a
16:14:06dangerous command. Affected parts is not
16:14:08present with us. So we can remove it.
16:14:10The div is not present with us. We can
16:14:13remove it. The description is going to
16:14:15be execute and then I'll pass in let's
16:14:20say something like blocked and then we
16:14:22can just say parameters dot command.
16:14:27So execute this blocked command. Then
16:14:29the parameters is the same. Tool name is
16:14:31the same. The command is not blocked,
16:14:33it's unblocked. So we can just pass in
16:14:37these two things. The description can be
16:14:39just execute. No need to pass in
16:14:41blocked. And then we can just pass in
16:14:44execute this command which is parameters
16:14:47docomand. So we'll pass in the command
16:14:50here which is parameters dot command.
16:14:54Even at the top we can pass in the
16:14:56command which is equal to parameters
16:14:58docomand
16:15:00affected paths is not there and is
16:15:03dangerous is just false because it's not
16:15:07blocked anymore right how can it be
16:15:10dangerous so that's it that's about the
16:15:12shell tool if it encounters any of those
16:15:15commands it will just return an error it
16:15:18also passes in the command so that our
16:15:20approval system can take a further look
16:15:22at it and we'll have a more complicated
16:15:25list of commands that will be run
16:15:27through that. So we can close well all
16:15:30the save files and now we come back to
16:15:33the registry where we pass in tool.get
16:15:36confirmation that get confirmation is a
16:15:39cool routine so I can await it and I'll
16:15:41get access to the confirmation variable.
16:15:44Now that confirmation variable can be
16:15:46null remember that this get confirmation
16:15:49can return tool confirmation or a null
16:15:52value. If it returns a null value, I
16:15:55don't want to return anything from here.
16:15:57I don't want to do anything else for
16:15:59that because it's approved. Whenever we
16:16:01get a confirmation null, we don't have
16:16:03to ask for the user confirmation. That's
16:16:06the point. So, I'll just say if
16:16:08confirmation is there, that means we
16:16:10want to ask for confirmation. In that
16:16:12case, I'll create the context of
16:16:14approval context so that I can call the
16:16:16approval manager. And now I'll import
16:16:19from safety.approval.
16:16:21I'll pass in the tool name here which is
16:16:23just the name. Then I have parameters
16:16:26which is equal to the parameters is
16:16:29mutating is equal to tool dot is
16:16:32mutating and I'll have to pass in the
16:16:34parameters again here. Then I have the
16:16:38affected paths and we get that from this
16:16:40confirmation. So confirmation
16:16:43dotaffected paths. Then we have the
16:16:46command we also get that from
16:16:48confirmation. After that we have is
16:16:51dangerous which is confirmation dot is
16:16:54dangerous. So that is all of the
16:16:57context. I can just call approval
16:16:58manager and check for the approval. That
16:17:02means we'll request for an approval
16:17:06decision and I'll pass in the correct
16:17:08context here. I'll have to await it to
16:17:10get my decision. And if the decision is
16:17:14let's say
16:17:16rejected. So if decision is equal to
16:17:19approval decision and I'll have to
16:17:22import approval decision as well. If
16:17:25that's rejected in that case I just have
16:17:29to return tool result
16:17:32dot error result and I'll say hey the
16:17:35operation was rejected by safety policy.
16:17:40Maybe you can also give more information
16:17:42about why it was rejected if you want.
16:17:44But yeah, let's say otherwise the
16:17:47decision is equal to approval decision
16:17:52dot needs confirmation. If it needs
16:17:54confirmation, we'll have to request for
16:17:56the confirmation dialogue box, right? So
16:17:59that we wait for the user's approval. So
16:18:01we'll have approved is equal to await
16:18:04approval manager dot request
16:18:07confirmation and we'll pass in the tool
16:18:10confirmation here which we have already.
16:18:13So this confirmation we just have to
16:18:16pass that in. And let's say the user
16:18:18does not approve it. In that case we'll
16:18:20have to again return the error message.
16:18:22So I'll just copy it and paste it here.
16:18:25Tool result dot error result and we'll
16:18:27say user rejected the operation. So
16:18:30we'll go user rejected the operation.
16:18:33Yeah, that is it about our entire
16:18:36approval system here. And let's say the
16:18:38decision is approved. In that case, we
16:18:41just come over here and to execute the
16:18:44tool, whatever is the result, we return
16:18:46it. But in case we need confirmation, we
16:18:50request the confirmation and if the user
16:18:53does not approve it, we just spit out
16:18:55the error message saying, "Yep, the user
16:18:57just rejected. What can I do now? All I
16:19:00need to do is wherever this invoke
16:19:03function is called, I need to pass in
16:19:05the approval manager and I can create
16:19:08the approval manager at the top. But
16:19:10instead of doing that, I'll do it in the
16:19:12session. The reason for it is everything
16:19:15else is in session as well. So yeah,
16:19:18we'll have self dot approval manager and
16:19:22we'll just create an instance of
16:19:23approval manager here. Let me import it
16:19:25from safety.approval approval and I'll
16:19:27have to pass in the approval policy. We
16:19:30get that from configuration, right? So
16:19:32I'll pass in self.config do.approval. So
16:19:35that is our approval policy. After that
16:19:38we'll require the current working
16:19:39directory which is self do.config dot
16:19:42current working directory. And finally
16:19:45we need the confirmation call back. As I
16:19:48mentioned before the confirmation call
16:19:50back is going to come from the main. py
16:19:53where we are going to create a function
16:19:55and that's going to be our next step but
16:19:56as of now it's going to be empty. What
16:19:58I'm going to do is take it from the
16:20:00agent here. So we are going to have a
16:20:02confirmation call back here and that is
16:20:06going to be of the type of callable tool
16:20:09confirmation everything we've looked in
16:20:10the past. So let me import it from
16:20:13typing tool confirmation and then
16:20:15awaitable. By default, the value is
16:20:17going to be null. And then we can just
16:20:20pass in self do.confirmation
16:20:23call back is equal to confirmation call
16:20:26back.
16:20:28And then I can just set self dot session
16:20:30dot approval manager is equal to this
16:20:34self doconfirmation call back.
16:20:36Alternatively, what you can do is just
16:20:38pass it to the session. But I don't want
16:20:41to type out this big thing in session as
16:20:44well. But yeah, as I mentioned, there's
16:20:48a lot of nested dependencies. You might
16:20:50want to get rid of that. But for our V
16:20:54code, which is a big spaghetti, this is
16:20:56fine. And actually, we don't need this
16:20:58line confirmation call back. Let's
16:21:00remove it. And we can just set approval
16:21:03manager to confirmation call back. And
16:21:05now, whenever we call the invoke
16:21:08function over here, we can pass in the
16:21:11fourth argument, which is the approval
16:21:12manager, which is just
16:21:13self.ession.approval approval manager.
16:21:16Awesome. Now the next thing we have to
16:21:18do is whenever the agent is called we
16:21:20have to pass in the confirmation call
16:21:22back. So in the main py we're going to
16:21:24create our confirmation call back and
16:21:26that confirmation callback is just going
16:21:28to be a UI tool. That's why we require
16:21:30it from the main. py it's going to show
16:21:33the dialogue box to the user. So let's
16:21:35design that. You can also create it in
16:21:38the TUI by the way. And I'm going to do
16:21:40just that. I'll go to the TUI here. I'll
16:21:43create a new function and let's call
16:21:45this function
16:21:46handle confirmation or confirmation call
16:21:49back. Well, confirmation call back was
16:21:51called because that confirmation
16:21:53function was passed as a call back. So
16:21:55that's why I'm calling it handle
16:21:57confirmation here. And then we're going
16:21:59to require a confirmation which is going
16:22:02to be the tool confirmation. So let's
16:22:04import it from tools.base. And here
16:22:07let's get started. So we are going to
16:22:10have an output list here. The first two
16:22:12objects are going to be the text where
16:22:15we just pass in the tool name. So we'll
16:22:18have confirmation dot tool name passed
16:22:21in. The style is going to be tool. If
16:22:24you want to look at the style, we can
16:22:26just go here and the style of tool is
16:22:29just bright magenta bold. Nothing too
16:22:31fancy. After that we have the
16:22:33confirmation description where I which
16:22:36I'll pass in. So confirmation do
16:22:38description and the style of this can be
16:22:40in the code format.
16:22:43After that I'll just check if the
16:22:44confirmation has command attached. If it
16:22:47has then I would like to display that.
16:22:49So I'll have output.append and then I'll
16:22:52have a text where I pass in the dollar
16:22:56sign. If you remember that's what we did
16:22:59when we had to display the shell output
16:23:01as well. And now I can have confirmation
16:23:04dot command
16:23:06And then I can pass in the style which
16:23:09is warning. After that we have the diff.
16:23:12So if confirmation diff is present in
16:23:15that case I would like to get the div
16:23:17text which is confirmation do. And just
16:23:21doing the diff will give me the file
16:23:22diff. What I want is the actual diff. So
16:23:26to do that I can just convert it to a
16:23:27diff. This was a utility function we had
16:23:30created before and that gives us the
16:23:32diff text. I would just like to print it
16:23:34out. So there we go. Output.append. Pass
16:23:38in the syntax. And this is something
16:23:40we've looked into so many times. So I'll
16:23:43speed through this. The theme is going
16:23:45to be Monokai. And then the word wrap is
16:23:49going to be true. After that, we're
16:23:51going to check or not check, we'll just
16:23:53leave a line. So console.print or
16:23:56self.conole.print.
16:23:58And then we can do self.conole.print
16:24:01print and print out the entire panel
16:24:03here where I'll pass in the group of the
16:24:07outputs.
16:24:09Then I have the title which is a text
16:24:11where we just say approval required and
16:24:15the style is going to be the warning.
16:24:19The title align is going to be left
16:24:22border style is going to be warning
16:24:26because it's a warning, right? We are
16:24:28asking for the user's confirmation. So
16:24:30you would like to have that style. Then
16:24:32the box is box.rounded
16:24:35and the padding is 1,2
16:24:38just like we had for other panels that
16:24:40we created. Now I would just like to
16:24:42call this function within main.py where
16:24:45we'll also create this function. Now
16:24:47what I like to do is call this function
16:24:50when the agent is being created. So when
16:24:53we try to run in the interactive mode,
16:24:55we can pass in the confirmation call
16:24:57back which is self. TUI dot handle
16:25:01confirmation. That's it. And when we run
16:25:04in the single mode, we don't have to
16:25:06pass in the handle confirmation because
16:25:09we are assuming that we are in a
16:25:10non-interactive mode. The user just gave
16:25:12us one prompt. They don't want to hear
16:25:15anything about that. They just want it
16:25:17to run. So it's in a non-interactive
16:25:19mode. So we'll not pass in any
16:25:21confirmation handler. So it will not ask
16:25:23for any confirmation. It will directly
16:25:26start running the tools. However, if you
16:25:29want to be more explicit, you can ask
16:25:31for the argument here or for the option
16:25:34where the user can specify if it's an
16:25:36interactive or a non-interactive mode.
16:25:38And if it's a non-interactive [snorts]
16:25:40mode, then you pass it then you do not
16:25:43pass in the handle confirmation.
16:25:45Otherwise, you pass it in. So, yeah,
16:25:47this seems like it. Now, I would like to
16:25:49test it out. So, let's run Python main.
16:25:51py. And the first thing I'll ask it to
16:25:55do is write to desktop hello world. py
16:26:01file because we are outside the current
16:26:04working directory. If that works, great.
16:26:07Let's hit enter. And we run into an
16:26:10error. The error is that the tool.get
16:26:12confirmation requires one missing
16:26:14positional argument. So I'll go to the
16:26:16registry and see what get confirmation
16:26:19requires. It requires invocation and I
16:26:21did not pass that in. Cool. Now let's
16:26:24try again. So I'll run it again and I'll
16:26:28copy paste the same prompt. Let's try
16:26:31again. And now we have another error.
16:26:34Function object has no attribute check
16:26:36approval. So this approval manager thing
16:26:39does not have check approval. And the
16:26:43reason it happens is because if you go
16:26:45to session.py or actually agent.py, py
16:26:48here we have confirmation call back and
16:26:50we're setting it to approval manager
16:26:52that's not correct we want approval
16:26:54manager dot confirmation call back to be
16:26:56this confirmation call back not the
16:26:58approval manager to be confirmation call
16:27:00back so I messed that up my bad yep that
16:27:04looks good now let's try again
16:27:06let's paste this again and now we have
16:27:09another error object none type can't be
16:27:13used in await expression and actually
16:27:15before we resolve this error there's
16:27:17There's one thing I forgot about and
16:27:19that is in the TUI we handle the
16:27:21confirmation. We have the panel here and
16:27:24it just says approval required but we
16:27:26have not given the user the option to
16:27:28approve or to disprove right. So we'll
16:27:31have to prompt the user to give us a yes
16:27:34or a no response. So we'll have
16:27:37something like prompt which comes from
16:27:39the rich package. So I'll have something
16:27:41like from rich.prompt prompt import
16:27:44prompt and this just allows us to ask
16:27:47the user a question and this is what I
16:27:49want to ask and the question is on a new
16:27:53line do you approve this request if you
16:27:55approve the choices are well a yes or a
16:28:00no or let's say the user is more
16:28:02explicit with yes or no and the default
16:28:05value is no because we think it's going
16:28:08to be bad and the response is this
16:28:11whatever the user says We'll store it in
16:28:13response and we'll just return a boolean
16:28:16value from here. So response dot lower
16:28:19in let's say yes or yes. So this
16:28:25function will return true if the user
16:28:27says yes otherwise false. And the return
16:28:30value of this function is going to be a
16:28:33boolean. And everywhere we said that
16:28:36this handle confirmation is going to
16:28:39have an awaitable value. Maybe it's time
16:28:42to change that. We'll go over here and
16:28:44we'll say that the return is not an
16:28:46awaitable boolean value. It's just a
16:28:48boolean value. And wherever this
16:28:51confirmation call back is called within
16:28:53this approval manager, this is also
16:28:56going to have the awaitable removed.
16:28:58Great. Now, let's close all of the save
16:29:00files. Let's retry. So, I'll paste in
16:29:04the same prompt. And by the way, we do
16:29:07get the approval required text box
16:29:08above. It's just that we don't see any
16:29:11output correctly. So, it was probably
16:29:14because of our issue that we just
16:29:15solved. Right now, let's try to pass in
16:29:18the same prompt here. Let's hit enter.
16:29:21And we do get approval required, write
16:29:24file, but the text here is empty. Now,
16:29:28the reason approval required shows up an
16:29:31empty text is something we'll have to
16:29:33invest in investigate further. But what
16:29:35I'll do is set approve to null. No,
16:29:38we're not approving this. And then we
16:29:40see an error again saying object boolean
16:29:43can't be used in await expression. And
16:29:45that's because we don't have this as an
16:29:48asynchronous function anymore. So we'll
16:29:51just return remove the await from here.
16:29:54And as a result, this is not an
16:29:56asynchronous function either. And
16:29:58wherever this request function is
16:30:00called, that's not an await either. So
16:30:02everything is now synchronous. That's
16:30:04good for us. And maybe we can try to
16:30:08write again. This time I can give a
16:30:10different prompt. Maybe I can say create
16:30:13a dummy Python script. Let's hit enter
16:30:17and see what it does. Oh, I forgot to
16:30:19talk about desktop. But it did create
16:30:22this dummy script. py for us. And we can
16:30:25see it over here. It's in our current
16:30:27working directory. So that's a good test
16:30:29as well because it did create it over
16:30:31here. Now let's try to do something in
16:30:34the desktop folder. I'll just copy this
16:30:36prompt, clear off everything and retry.
16:30:39And this time it will be create dummy
16:30:41Python script in desktop. Let's hit
16:30:43enter. And then we get asked for our
16:30:46approval. And this time we see the
16:30:49entire diff properly. The reason for
16:30:52that is probably because previously it
16:30:55tried to write in the same thing again
16:30:58and that's why we did not see any
16:30:59difference as such. But this time since
16:31:01we're creating a new file it we can see
16:31:04the difference. So this time let me say
16:31:06no and it says user rejected the
16:31:09operation and then the assistant says
16:31:11the operation was rejected. Let me know
16:31:13if you'd like to try again or need
16:31:14assistance. I'll just say try again. And
16:31:17I should be getting the same thing
16:31:19again. Great. And this time I'll approve
16:31:24and I do see that this approval required
16:31:27changes into write file properly. And we
16:31:30see the dummy Python script has been
16:31:32created successfully and it is just
16:31:34typing out really slowly probably
16:31:36because of the TPS issue. By TPS I mean
16:31:39tokens per second by the way. So that's
16:31:41it for our approval system. You can also
16:31:43try it for shell and edit file. I think
16:31:47they should work and if they don't it
16:31:48will be very silly errors like I've not
16:31:51passed in the correct parameters or
16:31:53something. So you can test that for
16:31:54yourself. But I'll just exit out from
16:31:57here and focus on the next feature. And
16:32:00what we are going to work on is hooks.
16:32:02What are hooks? Hooks are like
16:32:04userdefined scripts that run at specific
16:32:07points in the agent's life cycle. You
16:32:09can think of them as callbacks but
16:32:12external. So if the user wants they can
16:32:14pass in any executable code before let's
16:32:17say the agent runs or before the tool is
16:32:20run or after the tool is run or after
16:32:22the agent just completes or whenever
16:32:25there's any error. Why is this useful?
16:32:28Let's say the user wants to auto run
16:32:30tests after there are code changes. So
16:32:33let's say our agent edits the code and
16:32:35they want the test to run automatically
16:32:38so that they auto immediately know if
16:32:40something broke. In that case, they can
16:32:42use the hooks or let's say they want to
16:32:45autocommit after each task that the LLM
16:32:47does or the agent does. So you can just
16:32:50set up a get checkpoint and this makes
16:32:53it easy to revert if something goes
16:32:55wrong. Or let's say you want a Slack
16:32:59notification whenever the agent starts a
16:33:01longunning task and that's where this
16:33:04hook can also help you. That's how we're
16:33:06going to introduce hooks. just pieces of
16:33:09code that the user can run before the
16:33:11agent starts, after the agent starts,
16:33:14before the tool call is happening and
16:33:15after the tool call is happened. So
16:33:17let's go to our config.tml so that the
16:33:20or config. py so that the user can set
16:33:22up the hook related configuration.
16:33:25All I'm going to do is say two things.
16:33:28The first one hooks should be enabled or
16:33:31not and the boolean of that is by
16:33:33default going to be true or maybe we can
16:33:35set it to false and then we are going to
16:33:39have hooks which is a list of hook
16:33:41config and then we're going to have a
16:33:44field where the default factory is going
16:33:46to be set up and then we'll pass in the
16:33:48hook config. Now let's create the class
16:33:51of hook config which is going to extend
16:33:54the base model and then we're going to
16:33:56pass in well the name of the hook. So
16:34:00you know we are going to have four types
16:34:02of or let's just say we're going to have
16:34:04multiple hooks right. What is the hook's
16:34:06name? Then there's going to be a trigger
16:34:08of when did this hook get triggered. So
16:34:11you have a hook trigger enum here and
16:34:14I'm going to set that up as well. So we
16:34:16have class hook trigger and then we'll
16:34:20have a string enum just like we've seen
16:34:22multiple times before and this is when a
16:34:26hook should be triggered. So you have
16:34:28before agent which is going to be well
16:34:30before agent. Then similar to that you
16:34:33have after agent. So there we go. After
16:34:37that we're going to have the after tool
16:34:40or before tool. And I'll just write that
16:34:43down. After that we have after tool. So
16:34:47it's just what should happen after and
16:34:50before the tool call is done.
16:34:52And if there's an error, what should
16:34:55happen? So that's also present here.
16:34:59Great. So I have the hook trigger passed
16:35:01in. The next thing we're going to have
16:35:03here is the command that the user wants
16:35:05to run. And by default, it is going to
16:35:08be null. And if there's any script that
16:35:11the user wants to run that can be null
16:35:14as well or a string value. Now what is
16:35:17the difference between a command and a
16:35:18script? A command is something like
16:35:20let's say Python 3 main. py maybe I want
16:35:24to execute this or let's just say
16:35:26there's test py that we want to run.
16:35:29That is a command. But a script is
16:35:31something like a shell script. Maybe you
16:35:33want to echo out something. So you pass
16:35:36that in. Whatever you can basically pass
16:35:39insh
16:35:40file which is a shell script. So we're
16:35:43going to run that script. After that we
16:35:46have time out in seconds. You know what
16:35:48that means and the default value is 30
16:35:51seconds. And then you have enabled. And
16:35:54by default this hook is going to be
16:35:56enabled. But you can enable or disable
16:35:59each hook as we want it. And then I just
16:36:02want to validate that either this
16:36:04command or the script is mentioned
16:36:05because yeah that's how things go. Then
16:36:08you have model validator. Then we're
16:36:10going to pass in the mode which is equal
16:36:13to after. And then I'll pass in def
16:36:17validate hook. From here we are going to
16:36:20return the hook config. And we're not
16:36:22going to make the same mistake we did
16:36:24earlier. So I'll just return self before
16:36:28I forget it. Now I'll validate the hook.
16:36:30So if the self do command is not present
16:36:33and let's say the self.script script is
16:36:36also not present. In that case, I'll
16:36:38raise a value error and pass in hook
16:36:41must either have let's say a command or
16:36:46a script. And yeah, we've returned self.
16:36:49That's great. So, this is our hook
16:36:50config and the user can pass in a bunch
16:36:53of hooks. Now, we just have to create a
16:36:56new folder called hooks and integrate it
16:36:58with our AI agent. So we're going to
16:37:01have a hook system which will just go
16:37:04ahead and list out all of the hooks and
16:37:07create the specific triggers. For
16:37:09example, before agent, after agent,
16:37:12before tool, all of those functions also
16:37:14needs to be created so that we can call
16:37:16it in the main agentic loop. So we have
16:37:19class hook system and I'll pass in the
16:37:23init function. We going to get a config.
16:37:26So that's going to be of the type of
16:37:28config. Let's import it. And then we
16:37:30have self.config equal to config. After
16:37:32that, we are also going to have a bunch
16:37:34of hooks. So I'll just check the config
16:37:37if the hooks are enabled or not. If the
16:37:40hooks are not enabled, then we don't
16:37:42want any hooks to be present. So it's
16:37:44going to be an empty list. So let me
16:37:47just go ahead and initialize hooks to be
16:37:49an empty list. And this is going to be a
16:37:51list of hook configs. All right, this
16:37:54will just make it easier for us to run
16:37:57each hook as and when we need it. After
16:37:59that, let me fill in this hooks list.
16:38:02So, first of all, I'll just check if the
16:38:04self.config.hooks
16:38:06is enabled or not. So, if the hooks is
16:38:09not enabled, in that case, I don't want
16:38:11to proceed any further. I'll just
16:38:13return. But if the hooks are enabled,
16:38:15then I want to do something. So, let me
16:38:17just remove this not and pass in the if
16:38:20condition here. So, I'll have self do.
16:38:22hooks reinitialized to be hook for hook
16:38:27in self.config.h
16:38:29hooks if that hook is enabled as well.
16:38:32Right? Because each hook can be enabled
16:38:33or disabled. So we only want to pass
16:38:36into this list the hook if the hook is
16:38:39enabled. And that's pretty much it. That
16:38:42is all of our hooks. Now what I want to
16:38:44do is create a function specifically to
16:38:47trigger the before agent part. So I'll
16:38:51create a function that runs a script or
16:38:54the command that the user has specified
16:38:56and it should be triggered before the
16:38:58agent is started. So I'll have diff
16:39:02trigger before agent. I'll pass in the
16:39:06self. I'll pass in the user message that
16:39:08we get. You'll get to know in just a
16:39:10minute. It's going to return nothing.
16:39:12After that I'll go over every hook in
16:39:15this self.hooks hooks list and I'll just
16:39:18check if that hooks trigger is equal to
16:39:21the hook trigger
16:39:23and let's import it from config.config
16:39:26and if it's before agent because we only
16:39:28want to trigger hooks over here that are
16:39:30before the agent. Think about it this
16:39:33function will be called in the agentic
16:39:35loop here just before the agent starts.
16:39:39So maybe somewhere over here. Now before
16:39:41the agent starts, we want to run a bunch
16:39:44of code pieces that the user has
16:39:47specified through the configuration toml
16:39:49file. So I can just create a function
16:39:51that will run all of those code pieces,
16:39:53right? So I'll just have here a function
16:39:56that will allow me to run the hook and
16:39:59it's going to be a helper function. I'll
16:40:02pass in the hook. And I'm creating a
16:40:04helper function here because we're going
16:40:06to have multiple functions like trigger
16:40:08before agent. We'll also have trigger
16:40:10after agent. We'll also have trigger
16:40:12before tool and after tool. So we'll be
16:40:15using the same helper function every
16:40:17single time. The only thing that will
16:40:18change is this particular line where the
16:40:22hook trigger changes. So here let's
16:40:24create this function def run hook. We
16:40:27get a self. We get a hook which is the
16:40:30hook config and we return nothing from
16:40:33here. The first thing I'll check here is
16:40:35if the hook command is present. If the
16:40:38hook docomand is present then I want to
16:40:40run the command. And how do we run the
16:40:42command? We've already taken a look at
16:40:44that in shell. py. If we have any
16:40:47command to run, we just use the shell
16:40:50and we call this particular line
16:40:52async.io.create subprocess exec. Right?
16:40:56So let me just copy this entire line
16:40:58from here. We don't really need all of
16:41:01these lines of standard output and
16:41:03standard error because we don't have to
16:41:05do anything related to the output. The
16:41:09reason we don't have to do anything
16:41:11about the output is because the user
16:41:13already has a code they want to run. Why
16:41:16are we going to do anything about the
16:41:17output that their code runs? It's
16:41:20already stored if they want to store it
16:41:22or it's discarded if they don't want to
16:41:25store it. So, let me just paste in the
16:41:27entire thing here. And I also have to
16:41:30indent this nicely. So I'll just indent
16:41:32everything.
16:41:36And now we have an asynchronous function
16:41:38to convert it to. And I'll also call
16:41:41await self.runhook here before I forget
16:41:44it. This makes the trigger before agent
16:41:47also asynchronous. So we'll have that.
16:41:51And now let's understand what we need to
16:41:54change. First we'll have to import
16:41:55async.io. That's good. Then we are going
16:41:58to pass in the command that we want to
16:42:01run. So I'll pass in the hook docomand
16:42:04here. Then the standard output remains
16:42:06same. Standard error is the same. The
16:42:08current working directory needs to be
16:42:10passed in which is just
16:42:11self.config.curren
16:42:12working directory. Now the environment
16:42:15is going to be whatever is the
16:42:17os.environment dotcopy. Right? So we'll
16:42:20just have import OS dot environment
16:42:24dotcopy or get whatever you want. That
16:42:28gives us the dictionary that I'll pass
16:42:29in over here. And I will start a new
16:42:33session every single time. Then we wait
16:42:36for a particular amount of time. And the
16:42:39time I'm going to wait is hook dot
16:42:41timeout in seconds. And that gives me
16:42:44standard error standard data. I'm not
16:42:46interested in that. I'll just await.
16:42:48Great. After that, if the platform is
16:42:52not Windows 32, in that case, I'll pass
16:42:55in this thing. We've already looked into
16:42:57that. I won't explain this any further.
16:43:00After that, I can remove this tool
16:43:03result dot error result, but maybe you
16:43:05can raise an exception if you want, but
16:43:08I'll just remove it. I don't want to do
16:43:10anything if there's an exception. So,
16:43:12that is it about the command. Now if
16:43:15there's a script that's present and it
16:43:17has to be present because we have
16:43:19already added the validation. So if the
16:43:21script is present then what do I want to
16:43:24do? Well this is essentially ash script
16:43:28right. So what I can do is create a
16:43:29temporary file that has a pref or or a
16:43:33suffix of sh and then I can just execute
16:43:37that sh file. Right. So to do that what
16:43:40I can do is with temp file let's import
16:43:43temp file. So it just helps us create a
16:43:46temporary named file and then I can pass
16:43:49in the mode that we want to do it in. I
16:43:52want the mode to be write because I'm
16:43:53trying to write to a file. After that I
16:43:56can pass in the suffix of the file which
16:43:59is sh because I want the file to end
16:44:02with sh. That will allow me to execute a
16:44:05shell script. After that I'll have
16:44:08delete equals to false. And let's say I
16:44:11use this as a file. Then what I want to
16:44:14do is write the script. Correct?
16:44:16Whatever is the script that we get from
16:44:19hook. So hook.script. But just doing
16:44:22[snorts] that is not enough because
16:44:23let's say the hook.script is echo and it
16:44:26tries to echo something. That's not
16:44:28enough. The reason for it is we've not
16:44:31added the bang operator. Whenever you
16:44:33have a shell script, you needed to add
16:44:35this bang or shebang. I don't know what
16:44:37it's called, but you need to add
16:44:39something like this where you have a
16:44:41hash exclamation mark/bin/bash.
16:44:46So, you're running it on bash and then
16:44:49you pass in the script and then you
16:44:51mention the script path which is just
16:44:54equal to let's say the files name. And
16:44:57now since we've created a file that
16:44:59looks something like this, I'll just
16:45:01show it to you. Let's say it's called
16:45:04temporary.sh.
16:45:05Right? So as you can see there's a
16:45:07terminal icon beside it and I can pass
16:45:09in anything I want. For example the echo
16:45:11command that I was talking about. Let's
16:45:13say I echo hello world. Now if I want to
16:45:16run this particular script what will I
16:45:18do? Well I can just do not python dot
16:45:21/temp.sh
16:45:23right? But as you can see permission
16:45:25denied because I need special permission
16:45:28to execute it. Similarly over here I
16:45:31have stored the script path but first
16:45:33I'll have to give the valid permissions.
16:45:35so that I can run this particular file.
16:45:38So first I'll just have a try and
16:45:41finally here. So after everything works
16:45:45out properly what I want to do is os do
16:45:49unlink and unlink the scripts path.
16:45:54And this can also have an exception but
16:45:56I'm not handling that over here. And now
16:45:58what I can do is give it a permission.
16:46:01This path needs to get the correct
16:46:03permission. So I'll pass in os.chmod
16:46:06chod which stands for change mode. So it
16:46:10is just used to modify the file and even
16:46:13a directories permissions. So I'll pass
16:46:15in the script path here. And then what
16:46:18is the mode that I want? The mode that I
16:46:21want is 0755.
16:46:25The 0 is used to denote that the digits
16:46:29that are going to follow this 0 are in
16:46:32base 8. That means we're writing an
16:46:34octel. We are not writing in decimal
16:46:36format. So this is decimal format. We're
16:46:38writing in octal now. And the seven here
16:46:41represents the owner which is the user
16:46:45permission. And the user permission is
16:46:47seven meaning it's read, write and
16:46:50execute permissions. Five here stands
16:46:53for group meaning read and execute
16:46:55permissions are there for group but no
16:46:57write access. And five is for the world
16:47:01which just means read and execute
16:47:03permissions. there are no right access
16:47:05for them either and this is the
16:47:07permission generally given for
16:47:10executable files and scripts. After that
16:47:13we have given the permissions that are
16:47:15required. I can just go ahead and do
16:47:17await self.run command now and run
16:47:22command is basically this thing. So let
16:47:24me extract this out within a helper
16:47:26function of itself. So we have something
16:47:28like async def run command then we have
16:47:32a self we return nothing from here and
16:47:36I'll just paste this in cool let me just
16:47:39indent this properly so this will
16:47:41execute whatever command we want so I'll
16:47:44take this command as a string from here
16:47:47I'll pass in the command and now I can
16:47:49just call self over here so self dot run
16:47:55command I'll pass in the command which
16:47:58is hook dot command and similarly also
16:48:01this needs to be awaited because this is
16:48:04an asynchronous function so I'll await
16:48:06it and similar to this we'll have self
16:48:09dot run command and the command that
16:48:12I'll pass in is the script path
16:48:16maybe we can also pass in the timeout
16:48:18because that's not something we've
16:48:19looked into so we pass in the timeout
16:48:22here and I'll get the timeout integer
16:48:26from the parameters. Then when we call
16:48:30run command at the top, we can again
16:48:32pass in hook dot timeout seconds. And
16:48:35here when we run the command, we'll
16:48:37again pass in hook dot timeout in
16:48:40seconds. Great. So this is how you
16:48:43execute a script. And this is how you
16:48:45run a command. Now I just need to call
16:48:48this run hook over here before the
16:48:51agent. And that's enough work for me.
16:48:53But now you might wonder that hey why
16:48:56did you take in a user message string
16:48:58over here. The reason for that is
16:49:00there's something else we need to do and
16:49:02that is building the environment
16:49:04variables because if I'm writing a
16:49:06script let's say a temporary.sh or in
16:49:10the config.l ML I've passed in the
16:49:13script that I want to write and there I
16:49:15want to know which hook is being
16:49:18triggered or I want some information
16:49:21about what the user's message was. So
16:49:23before the agent starts I get to know
16:49:25what is the user message right and maybe
16:49:28in the script I would like to use that
16:49:31information to do something or I would
16:49:34like to know what working directory is
16:49:36the agent in so that I can use that or
16:49:39what was the tool that was called what
16:49:41was the error we ran into. All of this
16:49:43these details are very useful
16:49:45information that the script owner might
16:49:48require and it would help them if we can
16:49:51just provide it to them and that's why I
16:49:54took in a user message. So before the
16:49:56agent starts all we have is the user's
16:49:59message to us. So we can just take that
16:50:01and pass it to the environment variables
16:50:05so that the user can just reference the
16:50:07environment variables and create their
16:50:10own script. Hopefully that makes sense.
16:50:12If it doesn't, we'll create a test
16:50:14script later on to test our application
16:50:17and then you'll understand it much
16:50:18better. But for now, let me create a
16:50:21helper function which is build
16:50:22environment variables. This will get the
16:50:25trigger which is the hook trigger. It
16:50:28will get the tool name which can be a
16:50:31string or a null value because if you're
16:50:33having an before agent, you don't have a
16:50:36tool name. If you're having an after
16:50:38agent, you don't have a tool name.
16:50:40Similarly, for user message, the value
16:50:42can be string or null. And by default,
16:50:45it is null. Even over here, by default,
16:50:47it is null. And similarly, for error, we
16:50:51are going to have either an exception or
16:50:53a null value. And by default, it is
16:50:56null. Now, let me go ahead and build the
16:50:59environment variable. So, the first
16:51:00thing is os.environment.copy.
16:51:03That's good. Then I'm going to pass in
16:51:05my own environment variables. Now I want
16:51:08these environment variables to not
16:51:10conflict with the already existing
16:51:12environment variables. So I'll choose
16:51:15them safely. I'll have AI agent trigger
16:51:18for example which is just going to equal
16:51:20to trigger dot value. So this just
16:51:23refers to the hook that was run before
16:51:26agent after agent before tool what was
16:51:28run also. I think I misspelled in the
16:51:32configuration. Okay. Yeah, after needs
16:51:34to be lowercased. After that, similarly,
16:51:37we going to have an AI agent current
16:51:39working directory which is going to
16:51:41equal to the string of
16:51:43self.config.curren
16:51:45working directory. After that, we'll
16:51:47just check if the tool name is present.
16:51:49If it is, similar thing, we'll have AI
16:51:52agent tool name. So, what tool was
16:51:55called? And then we'll pass in the tool
16:51:58name. And then I'll copy paste these if
16:52:00conditions again.
16:52:02The second one is user message if that's
16:52:05present and I'll have AI agent user
16:52:08message and I'll pass in the user
16:52:10message here. The third thing is error
16:52:12if that's present and then we'll have AI
16:52:15agent error and I'll pass that in as the
16:52:18value. After that I'm just going to
16:52:20return this environment variable and the
16:52:24return type of this function is going to
16:52:25be the dictionary of string, string
16:52:27because we have an environment variable
16:52:29to return. After that I can just call
16:52:32this function over here where we have
16:52:35environment is equal to self dot build
16:52:37environment. And now I can pass in the
16:52:39hook trigger which is hook trigger dot
16:52:42before agent. Then I pass in the user
16:52:44message that we get from the parameters.
16:52:48And this environment variable can be
16:52:50passed to run hook. And now in the run
16:52:53hook instead of using the environment
16:52:55variable over here like
16:52:56os.environment.copy copy what I can do
16:52:59is use the same set of environment
16:53:02variables. So I'll have again
16:53:04environment which is a dictionary of
16:53:06string comma string. I'll use this
16:53:08environment variable and pass it in to
16:53:10run command which will also get this
16:53:14environment and it's just a lot of
16:53:17parameter passing as of now and I'll
16:53:19just pass in the environment variable
16:53:21year and we don't have to pass in the
16:53:24environment variables year I believe. So
16:53:26that's good. Also, if you want, you can
16:53:28set this delete to true so that the
16:53:29script is deleted as soon as you run it.
16:53:32But then you would have to shift this
16:53:34try and finally block inside of this vid
16:53:38because as soon as this context manager
16:53:40gets over the file will be deleted and
16:53:44maybe after that you can also un remove
16:53:46this unlink part because you're already
16:53:49deleting the file, right? So no biggie.
16:53:51So that's about trigger before agent.
16:53:54Similar to that, we are going to create
16:53:56a trigger after agent as well. And we're
16:53:59going to get the user message, but we're
16:54:02also going to get the agent's response,
16:54:04right? Because after the agent runs, we
16:54:07have the entire agent response with us.
16:54:09I would just like to send that across.
16:54:11And then I can pass in before no, that's
16:54:14going to be after agent. And similar
16:54:16over here, we're going to have after
16:54:18agent. And we've stored the user message
16:54:20to the environment variable. I would
16:54:22also like to store the agent response.
16:54:26So let me just update it manually over
16:54:29here. So we have AI agent response is
16:54:34equal to agent response and then we go
16:54:37up to let's say 5,000 characters if you
16:54:40want but I would just like to store this
16:54:42entire response and then we run the hook
16:54:45as usual. Now similar to this before
16:54:49agent we are also going to have before
16:54:51tool call. So let's have async def
16:54:54trigger before tool. Then we get the
16:54:57tool name that was run which is going to
16:55:00be a string. And then we get the tool
16:55:02parameters which is a dictionary of
16:55:05string, any and we import any from
16:55:09typing and it needs to be capitalized.
16:55:12After that we're going to pass in the
16:55:15tool name here. So that's there and now
16:55:18I just need to do environment AI agent
16:55:22tool parameters is equal to and I'll
16:55:25pass in the tool parameters. Ideally it
16:55:28should go within this build environment
16:55:30function because you know if you're
16:55:33taking in tool name why not take the AI
16:55:35agent tool parameters as well but I'm
16:55:40too lazy to change it again but you
16:55:42should. Now let's have the hook trigger
16:55:45to be before tool and this is also going
16:55:48to be before tool. And that's it. After
16:55:51that we're going to have an after tool
16:55:54as well. Here we're going to get the
16:55:55tool name tool parameters and the tool
16:55:58result which is going to be of the type
16:56:01of tool result because if you remember
16:56:04anything that gets returned from a tool
16:56:06result or the tool call is this tool
16:56:09result even if it's an error success
16:56:11whatever. After that we again do this
16:56:13self.build environment. We pass in the
16:56:15tool name and this changes to after
16:56:20tool. This also changes to after tool
16:56:23and the tool parameters are also set up.
16:56:25After that we have the AI agent instead
16:56:28of tool parameters you have the tool
16:56:30result setup and then you have tool
16:56:33result dot and we just convert it to
16:56:36model output because we are not
16:56:38interested in what this LLM might
16:56:42actually have over here. For example the
16:56:45diff or the truncated all of that is not
16:56:47necessary for us. What we are interested
16:56:49in is what is the error or what is the
16:56:51output. So yeah, and then we just run
16:56:55the hook. After that, we're going to
16:56:57have our first or last one, which is
16:57:00trigger on error. So it's not happening
16:57:03before or after anything. It's just
16:57:05happening on an error. And all we get
16:57:07here is an error, which is of the type
16:57:10of exception. Then we have a build
16:57:12environment. We'll just pass in the
16:57:14error. And here I'll say hook trigger is
16:57:17on error. I'll remove this environment
16:57:19variables. Not needed. We just have to
16:57:21store the error and here also we'll pass
16:57:24in the error. Awesome. That's all. So we
16:57:26have created all the functions that we
16:57:28need and now we are all set to integrate
16:57:32this hook system within this agentic
16:57:34loop. So similar to everything we first
16:57:36have to initialize it and initialization
16:57:39of this is also within session because
16:57:42each hook that's present is specific to
16:57:45one session. So we'll have self dot hook
16:57:50system is equal to and then we pass in
16:57:54the hook system like that. Let's import
16:57:56it. And now I can just call self dot
16:57:59session.hook system wherever I want. The
16:58:02first thing that we can do is run this
16:58:04hook system. So here I can just pass in
16:58:07something like self dot
16:58:08session.hooksystem hook system dot and
16:58:11then I can just trigger before agent and
16:58:15then I have to pass in the user message
16:58:17which is just message. Great. I'll await
16:58:20it because this trigger before agent is
16:58:22an asynchronous function. Now I'll copy
16:58:25this line because I'll also run it once
16:58:27I've completed this entire agent run. So
16:58:30I'll have cells
16:58:31session.hooksystem.trigger
16:58:33after agent and then we'll pass in the
16:58:37user message. And then we also want to
16:58:39pass in the final response or the
16:58:43assistant response. Right? So that's
16:58:46this one. Now I want to trigger the
16:58:48before and after tool calls. And to do
16:58:51that I'll have to go within the tool
16:58:53registry or the invoke function right
16:58:57because invoke function is where
16:58:59everything is called all the tools are
16:59:01called from there I can get access to
16:59:04the tool result. So if the tool is null,
16:59:07we are returning the tool result. What
16:59:10I'll do instead is set this equal to
16:59:12result. Then I'll return the result. But
16:59:15before doing that, I can just say, hey,
16:59:18give me access to the hook system. So
16:59:21this hook system that we have is going
16:59:24to be present here. Let's import it.
16:59:27Yeah, let's have it before approval
16:59:30manager. Hook system should always be
16:59:32there. And then we can just call hook
16:59:34systemt trigger after tool. Makes sense,
16:59:38right? If the tool is not present, we
16:59:41can return either trigger before tool or
16:59:43after tool is the same thing. But to
16:59:45give the user clearer information, we
16:59:48triggered before tool, but there's
16:59:50really no tool. So there's no before
16:59:52tool. We did after tool. That's just an
16:59:54indication to the user that there's no
16:59:57tool found. And I'll pass in the name of
17:00:00the tool. Then I need to pass in the
17:00:02parameters which is parameters like
17:00:04that. And then I will have the result
17:00:08which is what I created here. I'll await
17:00:11this and I'll copy this line because
17:00:13we'll use it multiple times. Now let's
17:00:16move on to the next thing. Now if there
17:00:18are validation errors, we are returning
17:00:21a result. So what I'll do is store this
17:00:23in a result variable. Then I'll trigger
17:00:26this after tool. pass in the name
17:00:29parameters and result and then I'll just
17:00:31return the result similar to what we did
17:00:33with this tool as null before we get
17:00:36started with this invocation I'll
17:00:38trigger before tool because this is the
17:00:40time that we are actually going to
17:00:42trigger some tool so we going to trigger
17:00:45before tool and now whenever we try to
17:00:47return the result after this we are
17:00:50going to have after tool again so let's
17:00:52see where we are creating a tool result
17:00:56and it's over here so So what I'll do is
17:00:59result is equal to tool result dot error
17:01:01result and I'll have hook systemt
17:01:04trigger after tool name parameters and
17:01:06result and then we'll return the result.
17:01:09Similar thing over here we are returning
17:01:11the tool result dot error result. So
17:01:13I'll have result then I'll trigger after
17:01:16tool and then I'll return the result.
17:01:18Same thing again and again. So these
17:01:20were for all the failures. Now if it's a
17:01:22success then we're going to get the
17:01:24result and even if it's a failure we get
17:01:27the result over here and then we are
17:01:29returning the result. But before we do
17:01:31that I would again like to trigger after
17:01:33tool and pass in these parameters. So
17:01:35this is how we are triggering before and
17:01:38after tool operations. So that looks
17:01:40good to me and with that all I need to
17:01:44do is see where this invoke function is
17:01:46being called which is within the agent.
17:01:50py and I'll pass in the self dot session
17:01:54dot hook system and I'll get that and
17:01:57pass it above approval manager because
17:02:00if you hover over this approval manager
17:02:02is the last thing hook system comes
17:02:04before that so yeah we are all set now
17:02:08let's try to see if it works out I'll
17:02:10open up our agent but before I open up
17:02:12our agent I have to configure the hooks
17:02:14so what I'll do is go to the config tml
17:02:16file and this is the file specific
17:02:19speific configuration system and here
17:02:22I'll add in my hook as I said hooks is
17:02:25going to be a list so what we'll have to
17:02:27do is hooks like that you remember when
17:02:30I said that the model needs to be passed
17:02:32in it needs to be passed in like this
17:02:34not like this if you have a list you
17:02:35pass it in like this and now we'll pass
17:02:38in the name if you go back to our
17:02:42configuration py file you'll notice the
17:02:45exact schema that we need to pass in so
17:02:48name trigger and command or script.
17:02:50That's what I'm going to pass in. So,
17:02:52the name is going to be, let's say, test
17:02:54before tool. And obviously, the trigger
17:02:58of that is going to be before tool. So,
17:03:01make sure you pass that in correctly.
17:03:04The same wording as this one. Then you
17:03:07have the command. And let's say the
17:03:10command for me is going to be Python 3
17:03:13dot /cripts/
17:03:16test tool. py. So I'm going to create a
17:03:19new folder called scripts in my current
17:03:22working directory and create a new file
17:03:25called test tool. py. So let's create a
17:03:28new folder
17:03:30scripts. Then we have test tool. py and
17:03:35I'll add it in over here. This is the
17:03:37script. So we have a main function. We
17:03:39are getting all of the environment
17:03:41variables. And it shouldn't be unified
17:03:43agent. It should all be AI agent trigger
17:03:45or AI agent current working directory.
17:03:48AI agent tool name, AI agent user
17:03:52message, whatever we created early on. I
17:03:55don't remember the wording. And this is
17:03:57going to be AI agent error as well. Then
17:04:00we create a dictionary out of it. We
17:04:02pass in the time stamp. We'd have the
17:04:03trigger current working directory. I
17:04:05forgot the AI agent response, but you
17:04:07can pass that in as well. And then we
17:04:10just log it to a specific directory
17:04:12which is well I forgot to put the
17:04:14desktop AI agent. So it goes in this
17:04:17specific folder and creates a hook.log
17:04:20file. I'll like to see the hook.log over
17:04:23here but you can just store it anywhere.
17:04:25This is just an example of what this AI
17:04:27agent can do. Obviously you can get more
17:04:29creative. You can add the Slack APIs so
17:04:33that you can create a notification. you
17:04:35know, tons of things to get creative
17:04:36with because this is just any code piece
17:04:39you can run. This is what we're doing.
17:04:41It's just a sample test file. And
17:04:43similar to this test before tool, I'll
17:04:45also create a test. Let's say before
17:04:49agent, just before the agent starts, I
17:04:52would like to see what's happening. And
17:04:54then we have a command which just runs
17:04:57the same thing, but this time the
17:05:00environment variables are going to
17:05:01differ because the tool name is
17:05:03differing. So yeah, let's try it. Also,
17:05:05by the way, you need to put in the hooks
17:05:07list again over here. They're they're
17:05:10thought of as one element and the second
17:05:12element. So that's good. Now, let's try
17:05:14to run the agent. And we run into an
17:05:17error. It says hook system forgot to
17:05:20pass in the positional argument. So
17:05:22we'll go to session.py and pass in the
17:05:25configuration here. Now let's try again.
17:05:30And this opens up. That's good. And I'll
17:05:32open up my file explorer as well so that
17:05:35if we have a hook.log showing up, we can
17:05:38see it. And let's just set tell it to
17:05:41echo out hello world. Yeah, that sounds
17:05:45good enough. Let's run it. And it just
17:05:47prints out the hello world. And the
17:05:49reason we don't see anything is because
17:05:51I made a small mistake. And that is I
17:05:54did not set hooks enabled to true
17:05:56because by default hooks enabled is
17:05:59equal to false. So I'll just set this
17:06:01equal to true. And now let's try to run
17:06:05it again and see what happens. So I'll
17:06:08type in the same prompt. Let's hit run.
17:06:11And we get cannot access local variable
17:06:14result where it is not associated with a
17:06:16value. So I can go to the registry. py
17:06:19and see where this is. And this is the
17:06:21before tool. Now before tool does not
17:06:23require any result. So let me remove it.
17:06:26And now we can try to run it again.
17:06:28Again the same prompt. Let's hit enter.
17:06:31Despite that we're not able to see it.
17:06:33Most likely this time the issue is that
17:06:36within the configuration py we passed in
17:06:39a default factory for
17:06:43hook which is just hook config. Instead
17:06:46it should be a list right because we
17:06:48have a list of hook config. So let's try
17:06:52to rerun it and see what happens. So
17:06:54I'll echo out hello world and let's run
17:06:57it again. So it prints out hello world
17:07:00still doesn't show up because I believe
17:07:01we still have a lot of bugs to figure
17:07:03out. So the first one is that when I
17:07:06have run command here
17:07:09and whenever I call it I'm passing in
17:07:11the environment variable here but I
17:07:12forgot to pass it in over here. So let
17:07:15me pass in the environment variable when
17:07:17I'm trying to run a script. After that
17:07:20this timeout here is not an integer. It
17:07:23is a float. After that we get the tool
17:07:26parameters. Right. Let me see where they
17:07:28are. So, wherever we get the tool
17:07:30parameters, I would like to convert them
17:07:33into a string format because that's what
17:07:35the environment variable requires, a
17:07:37string instead of a dictionary. So,
17:07:39we'll have JSON dot dumps and I'll pass
17:07:42in the tool parameters here. Let me
17:07:44import JSON. Now, similarly, whenever I
17:07:47have tool parameters here, I'll do that.
17:07:50And for this two model output, it
17:07:52returns a string to us. That's good. And
17:07:54whenever we have an error, we have to
17:07:57pass in the exception here. So when this
17:08:01run build environment is called, we'll
17:08:03have to pass in the string of error. We
17:08:05cannot pass in the exception object.
17:08:07Just like that. And then so yeah, that's
17:08:10all. Now let's try to run it again. I'll
17:08:13pass in the same prompt. And it says
17:08:15hello. I think I might have discovered
17:08:18the real reason behind this not working.
17:08:21And that's probably related to how the
17:08:24command gets run. So we have to go into
17:08:26the hook system and when trigger let's
17:08:29say runs
17:08:31so when we are trying to run a command
17:08:33we doing create subprocess execution and
17:08:36then we pass in the program. But here
17:08:38when it passes in the program we are
17:08:41passing in essentially something like
17:08:43python 3 right python 3 dot /script/
17:08:48test tool. py. But when we do that,
17:08:52create subprocess exec treats this as an
17:08:54exe executable that needs to run, not a
17:08:57shell command. So what we need to do
17:08:59instead is run a subprocess shell. Then
17:09:03we pass in the command and stuff. So we
17:09:05have the environment variable. You can
17:09:07set start new n session to true. After
17:09:10that, we'll wait for this process to
17:09:13communicate. If there's a timeout error,
17:09:16we will just go ahead and kill the
17:09:18process and wait for it. To make this
17:09:21work, what we can do is pass in a try
17:09:23here. Then let's indent everything. And
17:09:26then we have except and then I can just
17:09:29try to print out the exception if we get
17:09:31any so that I know where the error is
17:09:33even happening instead of just shooting
17:09:36in the dark. So let's run it again and
17:09:39I'll pass in the same prompt again. and
17:09:42it doesn't seem like we see any
17:09:43exception.
17:09:45So I went offline and tried to figure
17:09:47out what the error was and the error
17:09:49turns out to be the silliest thing ever.
17:09:51So in config.tml [snorts]
17:09:53we have a model here where temperature
17:09:55is zero and then hooks enabled is equal
17:09:57to true. Right? But the problem is that
17:10:00if you set hooks enabled true within
17:10:02this model it will think that hook is
17:10:04enabled for this model dictionary or
17:10:06model table. What we need to do is put
17:10:08it above this model and then it will
17:10:12work out. The reason I got to know that
17:10:14is because I went here and printed out
17:10:16the self.hooks and the self.hooks was an
17:10:20empty list. So I printed out
17:10:21self.config.hooks enabled and that was
17:10:24false. So obviously the issue was
17:10:26something related to the configuration
17:10:28and I just shifted this up and it
17:10:30worked. That's good to know. I did not
17:10:32know that was how the tl worked but it
17:10:34makes sense. And now let me just start
17:10:37python main. py and let's try to print
17:10:41out echo hello world to me using shell
17:10:45and then hit enter. Let's see if the log
17:10:48shows up and we do get hook.log here we
17:10:51get before agent and before tool.
17:10:55This is the current working directory.
17:10:57The tool name is well exit is also a
17:11:01tool name for some reason. I think stuff
17:11:03is getting printed out a bit weirdly.
17:11:06Maybe I have some more bugs in hook
17:11:08system which are related to the
17:11:11environment variable. Yeah, that is the
17:11:13issue. When the tool name needs to be
17:11:15passed in, I'm passing in the user
17:11:16message. So yeah, that is my fault. What
17:11:19I need to do here is user message is
17:11:22equal to user message.
17:11:25And now I need to scroll down and fix
17:11:27this for everything. So user message is
17:11:29again equal to user message.
17:11:31After that we have the next one where we
17:11:34have before tool. So tool name is equal
17:11:37to tool name.
17:11:40And then we have after tool where we
17:11:41have tool name equal to tool name. We
17:11:44have the error which is equal to error.
17:11:47So that's it. Now let's copy this prompt
17:11:50again and exit out of here and then run
17:11:55it again. This time hook.log is empty.
17:11:59Let's see it happening in real time.
17:12:01Fill it up in real time. So let's run
17:12:03it. And we do get before agent. Then we
17:12:06get before tool. And since we have not
17:12:09set up any hook for after tool, we do
17:12:11not see the output. But that's also
17:12:13something you can configure. And yeah,
17:12:16so we have before agent current working
17:12:18directory. Tool name is null because we
17:12:21are in before agent. There's no tool
17:12:23call. The user message is present here.
17:12:26The error is not. Then we have the
17:12:28shell. That's amazing. So that means our
17:12:31hook.log gets filled in thanks to the
17:12:34script that we created which is
17:12:37somewhere over here. And then we have a
17:12:39test tool py file here which fills it
17:12:42up. So this is the file that's running
17:12:45thanks to the configuration that's done
17:12:47over here. Now similarly you can try to
17:12:50run a script as well before or after
17:12:53agent whatever you like and use the
17:12:54environment variables as you feel like
17:12:58but that's it from my side. Now let's
17:13:00move on to the next feature which is
17:13:03loop detection. Many times what happens
17:13:06is that your AI agent starts following a
17:13:08loop. Maybe it calls a shell tool which
17:13:11says echo hello world and then it does
17:13:14some read file write file but again
17:13:16after that it calls the shell tool and
17:13:19then it does read file write file with
17:13:21the same content that is problematic
17:13:24because it is going in a cycle right you
17:13:28don't want that or many times it might
17:13:30just happen that your agent is doing a
17:13:34shell tool call with hello world echoed
17:13:37out and then it keeps doing that for
17:13:39four or five times or 100 times
17:13:41whatever. So that is a loop as well. So
17:13:43it would be good if we can just detect
17:13:45those cycles or repetitions that are
17:13:47happening because maybe the LLM
17:13:49hallucinated or got confused. So it
17:13:52started doing those weird things and we
17:13:54just detect that loop that's happening
17:13:56and then we kill it off and we steer the
17:13:59LLM away from that action. Now, I don't
17:14:02really have an example to show to you
17:14:05because it just happens on specific
17:14:07instances and it happens mainly with
17:14:11cheaper LLMs. With better LLMs, I've not
17:14:14seen that happening. But definitely with
17:14:16cheaper LLMs, it does happen and I've
17:14:18seen that with Gemini models as well.
17:14:20So, let's just implement that loop
17:14:22detection and integrate it within our
17:14:24agentic loop. It's very easy. So let's
17:14:27go ahead and create the file in the
17:14:29context folder which is going to be loop
17:14:32detector py. You can create it elsewhere
17:14:35as well but I think context is the most
17:14:37appropriate place. Then I'm going to
17:14:39create a class of loop detector just
17:14:41like we've done for everything else.
17:14:43Then we have an init function and here
17:14:45I'm going to have two things. The first
17:14:48one is going to be the max exact repeats
17:14:51which is the exact number of repetitions
17:14:53that we want to catch. So let's say the
17:14:57number is five or three whatever you
17:14:59want. This means the max times the same
17:15:02action can repeat and after that it will
17:15:04just error out and tell that yeah we are
17:15:07in a loop. After that we have the max
17:15:10cycle length which just refers to the
17:15:12cycle length we want to detect because
17:15:14if we have something like let's say a b
17:15:18a b in that case we have a cycle. So I
17:15:22want this cycle can go on for let's say
17:15:24five or maybe we can set it to three and
17:15:27if it exceeds this three limit in that
17:15:30case we are in a cycle after that I'm
17:15:33going to create a history which is going
17:15:35to keep track of all of the actions that
17:15:37are happening and based on these actions
17:15:40I can just determine if we have a max
17:15:43length or my max cycle or max repeats
17:15:46right so what I'm going to have is self
17:15:48dot history for example and This is
17:15:52going to be a private variable. The type
17:15:53of this is going to be a deck and we're
17:15:57going to import deck from collections.
17:15:59Deck stands for doublyended Q and we're
17:16:02going to use it for the simple reason
17:16:04that I want a property of us here.
17:16:07There's one simple property max length.
17:16:10So if the max length is provided, it
17:16:13will limit the number of items within
17:16:16this collection or within this queue. So
17:16:19if we have let's say 20 items that's the
17:16:22number of items I want to keep in this
17:16:25deck otherwise I don't want to keep
17:16:27track of all of the elements right why
17:16:29should I waste my memory on that so it's
17:16:31just a simple optimization and we can
17:16:34set this to be let's say 20 now I'm
17:16:36going to have two functions here the
17:16:38first function is going to be record
17:16:40action so whenever the loop has to be
17:16:43detected first we'll have to record the
17:16:46action that's happening maybe we get the
17:16:48user
17:16:49uh the agent response or we get the tool
17:16:52call that's happening. So I would like
17:16:54to record that action and once I've
17:16:56recorded it somewhere else I'll just
17:16:58check for the loop. So just to give you
17:17:01a perspective we'll go to agent.py and
17:17:03here let's say whenever we try to call a
17:17:06tool that's when we'll record one action
17:17:08and whenever the user or the agent
17:17:10response comes in in that case I'm going
17:17:13to record an action as well. And if I
17:17:17want to detect a loop, I can do it
17:17:19anywhere I want. It doesn't have to be
17:17:21in one fixed place. So we're going to
17:17:24have two functions. Record action and
17:17:27the loop assertion where we just check
17:17:29if a loop is detected or not. So here
17:17:32the first thing we're going to get is an
17:17:34action type. And the action is
17:17:36essentially is it a tool call? Is it the
17:17:38response? Was it what what is it? And if
17:17:40there are any other details, they will
17:17:42be passed through a map here which I'm
17:17:44going to copy. and then we have any. So
17:17:47it can be any details that are passed to
17:17:49us and we'll get that details as a
17:17:51dictionary. After that I have to compute
17:17:54a signature for an action so that I can
17:17:56store it within the history. Because if
17:17:58we have a tool call happening what I
17:18:00like to do is get the tool name, get the
17:18:02arguments and then just append it with
17:18:05the response if there's a response as
17:18:07well and add it to this history. So that
17:18:11is going to be our signature. It's going
17:18:13to be a list. This history here is going
17:18:16to be a list of a string by the way. So
17:18:19deck is going to be a string here. And
17:18:22this list is going to be let's say the
17:18:24first element is a string. And then
17:18:27there's going to be the tool call that's
17:18:29happening where we pass in the tool
17:18:30name. So let's say the first tool name
17:18:33was shell. After that we're going to
17:18:35pass in an or operator here so that we
17:18:37have a separator between these two. And
17:18:39the next thing that will come in here is
17:18:41the tool arguments. So we'll have the
17:18:45key and by the way everything will be
17:18:46within this string. So we have shell
17:18:48then we have the tool arguments. So
17:18:50let's say the key is command and then
17:18:52what is the command's value? What did
17:18:54the user say or the agent say? And we
17:18:57have something like echo and there is
17:18:59maybe hello world something like that.
17:19:02So this is going to be the action
17:19:04recording that we're going to do. And if
17:19:06you want we can also add some truncation
17:19:08here. And when this loop detection gets
17:19:11more complex, you might want to hash
17:19:13this thing so that you know you don't
17:19:15store the entire string which can be
17:19:17very long. Instead, what you do is store
17:19:20a hash of whatever string you have so
17:19:23that you just have to compare let's say
17:19:24a 16digit string instead of comparing a
17:19:27very long 100 character 200 character
17:19:30string. So yeah, now the reason we're
17:19:32storing this in history is so that
17:19:35whenever we try to detect a loop, it
17:19:37will get much easier for us to detect
17:19:38the loop. All we have to do is match the
17:19:42strings correctly. We're just trying to
17:19:44have it in a good format. So the first
17:19:46thing we'll have is let's say the output
17:19:48which is equal to and I'll pass in pass
17:19:50in the action type. After that I'll just
17:19:53check if the action type is equal to
17:19:55tool call. If it is the tool call that
17:19:57means the tool call is also present. In
17:20:00that case we'll have parts.append
17:20:03or output.append
17:20:05and then I'll pass in the tool name. So
17:20:08to get the tool name I'll just do
17:20:09details.get and pass in the tool name.
17:20:12If it's not specified it's just an empty
17:20:15string. After that I'm going to get all
17:20:17of the arguments from details as well.
17:20:19And if they are not specified it's just
17:20:21an empty dictionary. then we'll just
17:20:24check if the args is a dictionary.
17:20:29In that case, I would like to go over
17:20:31every argument that's present. But if I
17:20:33just do args dot keys, we know the args
17:20:36is not really sorted. We want it to be
17:20:38in a consistent order so that whenever
17:20:40we try to create some sort of action, it
17:20:43repeats. For example, if we have two
17:20:46shell calls or three shell calls that
17:20:48are happening, but their arguments are a
17:20:50bit different. Maybe one time it gives
17:20:52us command is equal to echo hello world.
17:20:56Okay. And maybe the timeout is 30
17:21:00seconds. And the next time the same
17:21:02shell tool call is called. And this time
17:21:04what's happening is that we have shell
17:21:06but first the timeout is given out to be
17:21:0930 and then we have command is equal to
17:21:11echo. It's the same thing that's
17:21:13repeating but the argument has changed.
17:21:16And when we try to compare both of them
17:21:18later on to detect a cycle or a
17:21:20repetition, it will not be the same
17:21:23because the string positions have
17:21:24changed. So I want this to be in a
17:21:26consistent order. And for that reason,
17:21:28I'll sort them. Once I've sorted them,
17:21:31what I'll do is output.append and I'll
17:21:33pass in the key here. So we have key and
17:21:37then I have to pass in the value which
17:21:40is going to be string of args at K. So
17:21:44that is our value. And if you want, you
17:21:46can truncate this down to 50 characters.
17:21:49But I'm not going to do that because if
17:21:51we truncate it down to 50 characters and
17:21:54you know there are two actions that are
17:21:57same until the first 50 characters and
17:22:00then from the 51st character they start
17:22:02looking different that will be an issue.
17:22:05So I'm not truncating it down. But if
17:22:07you want to keep this short you can. In
17:22:09our simple AI agent, it won't matter
17:22:13because we don't have that many
17:22:15arguments in any case or in any tool. So
17:22:18we are safe that way. After this has
17:22:20been done, we have recorded the tool
17:22:22call related stuff. If the tool call is
17:22:24present, that means we have the tool
17:22:27call. But if the tool call is not
17:22:29present, then we will check for the
17:22:31response. That is the assistant
17:22:34response. And that is very simple. All
17:22:36we need to do is output.append append
17:22:38and I'll pass in details.get and pass in
17:22:41the text variable. So that's it. That is
17:22:44the response. You can truncate this down
17:22:46as well. Also, if it's not present,
17:22:48let's say the text, we'll just pass in
17:22:51the empty string. After that, finally,
17:22:53what we'll do is return a join operator
17:22:57on all of the elements within this
17:22:59output because I told you everything
17:23:02that we add here is going to have an
17:23:05operate or operator joining them. And we
17:23:07don't have to record return anything.
17:23:09What I'll do here is store this as a
17:23:13signature variable and then just have
17:23:15self dot history and I forgot to take in
17:23:18self. So we have self at the top and we
17:23:21have self dot history.append and I'll
17:23:23pass in the signature. So that is it for
17:23:27recording the action. Now another
17:23:29function I'm going to create here is to
17:23:31check for the loop. We can just create
17:23:33that function check for loop. We are
17:23:36going to get a self and we're going to
17:23:38either return a string or a null value.
17:23:41And the entire point of this function is
17:23:42to check if a loop has been detected.
17:23:44And as I told you there are two types of
17:23:46loops. Exact repetition. So the same
17:23:49action can be repeated a maximum of
17:23:52let's say three times that's what we
17:23:53have stored here. And a cycle which is a
17:23:56repeating pattern of actions. So the
17:24:00thing I showed you a ab if that happens
17:24:03three times that one. So what is the
17:24:05algorithm that we're going to use to
17:24:07detect a loop? Well, to detect a cycle
17:24:09of let's say length n or in our case the
17:24:12length n is three, we need to have at
17:24:15least two n actions, right? Because if
17:24:18there's a cycle, we're going to have a b
17:24:21for example and then a b again and then
17:24:24a b again. So to catch a cycle of length
17:24:28n, our cycle of length three needs to
17:24:31have at least six elements. It has it
17:24:34can be greater than that. For example,
17:24:36the loop might be A B C and then we have
17:24:40A B C again and then we have ABC again.
17:24:43But a minimum of six elements should be
17:24:45present. So we check if the last two n
17:24:49actions form a pattern where the first n
17:24:52actions exactly match the second n
17:24:54actions. Right? And this will indicate
17:24:58the sequence if it has a a repeating
17:25:01action or not. So let's take an example.
17:25:04Let's say our history has these elements
17:25:06A B C A B C. Now what I'll do is check
17:25:10the last six items. So our first half is
17:25:13going to have A B C. And luckily for us,
17:25:16our history only has six elements by the
17:25:18way, but it can obviously be more than
17:25:20that. Maybe there's a D, E, F as well.
17:25:22But what I'm interested in is the last
17:25:24six elements. I'll have my first half as
17:25:27ABC and the second half as ABC. And if
17:25:30they both match then the cycle is
17:25:32detected. And for us it's very easy
17:25:35because everything here is
17:25:36deterministic. The actions we're
17:25:38recording in history are all
17:25:40deterministic. That's how we've
17:25:42positioned it. So let's go ahead and
17:25:44check. So the first thing I'll check is
17:25:46if the length of self dot history is
17:25:50less than two because if it is less than
17:25:52two then we don't have anything. After
17:25:56that I'll check the exact repetition.
17:25:59I'm not checking for cycles yet. First,
17:26:00I'll check the exact repetition, meaning
17:26:03if the same action is repeated n times
17:26:05in a row. And that's very easy, right?
17:26:08So, the first thing I'll check is if the
17:26:10length of self dot history is greater
17:26:13than or equal to self domax number of
17:26:16exact repeats because the history needs
17:26:19to have at least three elements so that
17:26:21we can know if there are three
17:26:23repetitions.
17:26:24After that, what I'm going to do is get
17:26:27the last three most recent elements. And
17:26:31to do that, I can just say recent is
17:26:33equal to list of self.history.
17:26:37And we are just converting this Q within
17:26:40a history. We just converting this Q
17:26:42into a list. And then I'm trying to
17:26:45extract the last three elements. So to
17:26:47do that, I can just do minus self dot
17:26:51max repeats until the end. So this will
17:26:54give me the last three or last let's
17:26:58call this n so last n number of elements
17:27:02and then I can just convert it into a
17:27:04set and see if the length is one because
17:27:07if we convert it into a set all the
17:27:10duplicate actions will be gone and we'll
17:27:12only have one element. So if the length
17:27:15of set of recent elements is equal to
17:27:18one then I'll return that hey the same
17:27:21action is repeated
17:27:25let's say self domax exact repeat number
17:27:28of times right so I'll just say times
17:27:32so this is how I check for the
17:27:34repetition quite easy what about cycles
17:27:37well for the cycles I told you we need
17:27:40at least two times the max cycle length
17:27:43So first I'll check if the length of the
17:27:46self do is greater than or equal to self
17:27:50domax cycle length into two because yeah
17:27:55we're just having two number of actions
17:27:59if you want to detect a cycle after that
17:28:01I'll have the history converted into a
17:28:03list again so I have cell history passed
17:28:06in after that for every cycle length in
17:28:10the range from 2, minimum of self domax
17:28:14cycle length + 1 and the length of
17:28:18history divided by 2 + 1. We're going to
17:28:23create a loop. Now, what does this line
17:28:25even mean? Well, we are trying to check
17:28:27cycles of length 2 3 4 up to the max
17:28:31cycle length, right? Because I told you
17:28:34there can be a cycle which is just two
17:28:38cycles or three cycles or four cycles.
17:28:41For example, you can have AB AB AB,
17:28:45right? Or you can have ABC C abd
17:28:52ABC D ABC D. So, we want to go from two
17:28:56because there should be a minimum of two
17:28:58elements. And we'll go until the minimum
17:29:01of self domax cycle length + one. The
17:29:05plus one is done because we are having a
17:29:08range here. range does not include the
17:29:12last element. So if you have four here
17:29:15for example, it only goes from two to
17:29:18three including three. It does not
17:29:20include four and that's why we have +
17:29:22one here. And we have length of history
17:29:25divided by 2 + 1 because let's say the
17:29:28length of history divided by two is
17:29:30lesser than the max cycle length. That
17:29:34is also possible because let's say our
17:29:36max cycle length is 10. But the history
17:29:38only has five elements in that. So 5 / 2
17:29:42will give us 2 + 1 3. So we'll go until
17:29:46three. We don't have to go until this 11
17:29:49element. So we'll terminate based on the
17:29:51length of history. After that I would
17:29:54just like to get the last 2n number of
17:29:58actions. So I'll have recent is equal to
17:30:00history. And then we'll have minus cycle
17:30:04length multiplied by two whatever cycle
17:30:07we get here. So for example when we're
17:30:09trying to detect if there are two
17:30:10elements within the cycle. So we'll get
17:30:13the last four elements from the history.
17:30:16Or if we have three we'll try to get the
17:30:19last six elements from the history. And
17:30:22now we have to split them into two
17:30:24halves so that we can compare if they
17:30:26match. And if they do match, we've
17:30:29detected a repeating cycle.
17:30:32So if the recent and then we have the
17:30:35cycle length is equal to recent and then
17:30:38cycle length in the beginning until the
17:30:40end. If this is the case, then we're
17:30:43going to return that we have a detected
17:30:48repeating cycle of length and I'll pass
17:30:52in the cycle length.
17:30:55So this way we have accounted for the
17:30:58cycles as well. And if it doesn't return
17:31:02anything that means we're going to
17:31:03return null from here because we have no
17:31:06loop. Now we just have to call both of
17:31:08these functions within the agent. py and
17:31:12as I mentioned to you before the first
17:31:14thing that we'll have to do is if the
17:31:16response text is present then we'll
17:31:19check for the loop detection. And first
17:31:22we'll also have to go to session and
17:31:24create this loop detector. So we'll have
17:31:26self dot session or let's say loop
17:31:28detector is equal to loop detector. And
17:31:32this is especially useful because each
17:31:35session should have its own loop
17:31:36detector. If it doesn't then let's say
17:31:38multiple sessions are present and the
17:31:41loop actions will just get appended to
17:31:44the same session. We don't want that.
17:31:46After that we'll have cell dot session
17:31:49dotloop detector dot and I'll pass in
17:31:52record action and what is the action
17:31:55type response and I'll pass in the text
17:31:58here and the text is going to be
17:32:00response text. If you want you can
17:32:03truncate it here but again I'm not
17:32:05interested in that. Then I'll just copy
17:32:07this and when we are going over every
17:32:09tool call here I'll record the action
17:32:11again. So once we have sent to the UI
17:32:14that the tool call has started. I'll
17:32:16have loop detector.record action and
17:32:19I'll pass in tool call. I'll pass in the
17:32:22tool name which is equal to tool call
17:32:25dot name and I'll pass the arguments
17:32:27which is tool call dotarguments. So we
17:32:31have recorded this action as well. And
17:32:33just after we've recorded this action, I
17:32:35would just like to see if there's a loop
17:32:38here because in most likely you'll have
17:32:42multiple repeating tool calls. The
17:32:45assistant text is not going to loop that
17:32:46much. But what will loop is the tool
17:32:50call. So you can also have the record
17:32:53action somewhere over here. And we'll
17:32:55also have the loop detection here. So
17:32:57I'll just catch here that hey listen if
17:33:01session.loop loop detector dot check for
17:33:05loop if it returns a value to us. So
17:33:08let's say value or let's say loop
17:33:11detection error and if the loop
17:33:15detection error is present in that case
17:33:18I would like to steer the model in a
17:33:21different direction. So what I can do is
17:33:23create a prompt to break this loop and I
17:33:26can add it to our system. py prompt or
17:33:29if you want you can create a new file
17:33:31that looks into this but we'll have
17:33:33something like this. So system notice
17:33:36loop detected the system has detected
17:33:38that you may be stuck in a repetitive
17:33:40pattern and the loop description is
17:33:42present here which we get from the
17:33:43parameter and this is just the loop
17:33:45error that we have returned from the
17:33:47loop detector. So these things detected
17:33:50repeating cycle of length, same action
17:33:52repeated, all of that. To break out of
17:33:55this loop, please stop and reflect on
17:33:57what you're trying to accomplish.
17:33:58Consider a different approach. If the
17:34:00task seems impossible, explain why and
17:34:02ask for clarification. If you're
17:34:04encountering repeated errors, try a
17:34:06fundamentally different solution. Do not
17:34:08repeat the same action again. So that's
17:34:11just us trying to steer the model away.
17:34:13And now I can just call this create loop
17:34:18breaker prompt and I'll import it from
17:34:22system.p py. And whatever this loop
17:34:25prompt is, I need to pass in the loop
17:34:28detection error into it. And then I can
17:34:31just add it to a context. So I'll have
17:34:33self do. session dot context manager do
17:34:36add user message. This is not an
17:34:39assistant message, not a tool result.
17:34:41It's a user message because we are
17:34:42manually interrupting and saying that
17:34:44yeah you've looked you've you've gone
17:34:46into this infinite loop and then we add
17:34:49this loop prompt pass to it. That's it.
17:34:52After this maybe we can call continue so
17:34:55that we don't execute this tool call
17:34:57again. That is it for loop detection.
17:34:59Now I really don't know how to test this
17:35:02out because I don't know how to send the
17:35:04model into a loop. But one thing I can
17:35:07recommend you to do is go to the agent.
17:35:09py and then whenever this agent event is
17:35:12done you know you can go here create a
17:35:14new thing called loop detected and
17:35:17whenever the loop is detected you can
17:35:18notify the user that yeah we've run into
17:35:20a loop so we are manually fixing that
17:35:23that would be a good user experience I
17:35:25believe and then you can also create a
17:35:27function here and then whenever we
17:35:29actually catch a loop you can just have
17:35:32something like yield agent event and
17:35:34pass in the loop detected thing here.
17:35:39I think that would be great. And then in
17:35:41the main py you can handle that
17:35:43scenario. But anyways, now I would just
17:35:45like to test this out. Maybe I can
17:35:48specifically prompt the agent, please
17:35:51loop the same shell command with same
17:35:54arguments
17:35:56four times. Maybe that will help us
17:36:00know if the loop is detected or not. But
17:36:04really, we have no way of knowing if the
17:36:06loop was detected or not. So, I'll just
17:36:09manually go ahead and say print loop
17:36:12detected. And then let me just copy this
17:36:15prompt. And I'll exit out of here
17:36:19and retry.
17:36:21So, let me paste this in and hit enter.
17:36:24And we run into an error. The error here
17:36:27says that the deck object has no
17:36:29attribute append because I misspelled
17:36:31it. It should be dotappend with two PS.
17:36:35Now let's try again. Let's paste in the
17:36:37same prompt. And it asks me for the
17:36:39shell command and arguments. Fair
17:36:40enough. I'll say echo hello world is the
17:36:45command. Run. And then we run into
17:36:48another error. Is instance args
17:36:53two must be a type a pupil of types or a
17:36:57union. And the issue is that is instance
17:37:00asks should not have the value passed
17:37:02in. It should be a dictionary. Small
17:37:04issue. Now let's try running again. Pass
17:37:08in the same prompt. So I'll say echo
17:37:10hello world is a command run. But if you
17:37:12see it just passes in the command as for
17:37:16i in 1 to 4 do echo hello world. That
17:37:19will not help us detect the loop. So
17:37:21maybe I have to tune my prompt a little
17:37:23bit. is the command four different times
17:37:27by calling shell tool every single time.
17:37:32Let's see if that works. I'm not very
17:37:34good at prompting. And yeah, that does
17:37:36work out. So we have shell echo hello
17:37:39world. Then we have echo hello world
17:37:41shell. Then we again have echo hello
17:37:44world. And then we just see loop
17:37:46detected. Then we also run into an API
17:37:48error saying unexpected ro tool after ro
17:37:51user. Now the reason we run into that
17:37:54error is because well think about it
17:37:56what are we doing? Well we first adding
17:37:59all of the tool calls as the response
17:38:02right so here we have the assistant
17:38:04message where we are adding the tool
17:38:06call we are registering the tool call
17:38:07that's happening then we're going over
17:38:10all of the tool calls and executing them
17:38:13and adding them to our context later on.
17:38:16So we have the tool call results added
17:38:18here but in between we have checked for
17:38:20the loop detection here. So if there's a
17:38:24loop we are suddenly adding a user
17:38:26message. So what's happening is we have
17:38:27the assistant response then let's say
17:38:30there's a tool response then we have the
17:38:32assistant again then suddenly you have
17:38:35let's say the user response because you
17:38:37detected a cycle for example and then
17:38:40you have the tool response. Now a tool
17:38:42response cannot follow a user response,
17:38:45right? That doesn't make sense. An
17:38:46assistant can call a tool and then you
17:38:49have the tool response and that's the
17:38:51issue for us. So what I'm going to do
17:38:53instead is instead of having the loop
17:38:55detection over here, I can just push it
17:38:58out over here. So if we have the tool
17:39:01call results, after that we'll add the
17:39:03loop detection. So this will also allow
17:39:06us to catch any assistant response
17:39:09looping if that's happening. and the
17:39:11tool result as well. So let's try again.
17:39:14I'll pass in the same prompt. I forgot
17:39:17to copy it. Anyways, let's just try to
17:39:19write a similar prompt. So we write type
17:39:22out the prompt call echo hello world
17:39:24using shell tool and four different tool
17:39:26calls. Let's hit enter. And then we see
17:39:28loop detected. Great. So it calls the
17:39:31shell commands and then it says loop
17:39:33detected. I have successfully executed
17:39:35the echo hello world and we are also out
17:39:38of the loop. I'm going to remove this
17:39:39print as well. Now, the next thing I
17:39:41would like to get started on is this
17:39:43command support. We are almost nearing
17:39:45the end of this AI agent, but the
17:39:47feature I want next is all of the
17:39:49command support here. So, if I have
17:39:51something like this, I open up my agent.
17:39:54I have /help /config /approval/model
17:39:59/exit. These all commands should work.
17:40:03So, I'll close everything. And then we
17:40:05go to main.py. We have a bunch more of
17:40:08these commands that we support and they
17:40:10will all be shown when we have slashhelp
17:40:13for example slashmcp or slashtools.
17:40:17So yeah, all of them are going to come
17:40:19in and let's get to it. It's going to be
17:40:22very fast because all of it is mainly UI
17:40:26based. What we'll be spending most of
17:40:27our times on is something like slash
17:40:30session or let's say slashsave or
17:40:33slashcheckpoint where we have to create
17:40:36those checkpoints and sessions for this
17:40:38to work. So yeah, let's get started on
17:40:41that. So let's scroll down a little bit
17:40:43and then we come back to our agent over
17:40:45here in run interactive. We don't have
17:40:47to worry about the agent within run
17:40:49single. The reason for that is the user
17:40:51won't be able to specify any prompt
17:40:54after just mentioning one single
17:40:56message. But in run interactive the user
17:40:59will be able to pass in the user input
17:41:01continuously. And that's why over here
17:41:03after we have checked that the user
17:41:05input is present or not and then
17:41:07continuing we can just check if the user
17:41:09input starts with let's say a forward
17:41:12slash. That's a good enough indicator
17:41:13for us to handle a command. So here I'm
17:41:16going to create a function called handle
17:41:18command and pass in the user input here.
17:41:21And this function is going to return a
17:41:23boolean value which will say that the
17:41:27interactive mode that's going on this
17:41:29loop that's going on should continue or
17:41:31should it not continue. So we'll have
17:41:34should continue as the output here. And
17:41:36by default everyone is just going to say
17:41:39true. it should continue because if you
17:41:42type in something like /mcp you still
17:41:45want to go ahead with another input you
17:41:47need this line over here so that you can
17:41:49pass in another input but if you have
17:41:52something like /exit you need to return
17:41:55false so that we don't continue and if
17:41:58should continue is false in that case
17:42:00what I'm going to do is break out of the
17:42:03loop and if should continue is true then
17:42:06we'll keep continuing we won't process
17:42:08the message because Remember it's
17:42:11starting with forward slash. If it's
17:42:12starting with forward slash, it's a
17:42:14command not to be processed by us. So
17:42:17yeah, that's good enough for now. Let's
17:42:19go ahead and create this function in
17:42:21CLI. Let's call this handle command.
17:42:25Then we get the message or the user
17:42:27input, whatever you want to call it. I'm
17:42:29going to call it command and it's going
17:42:31to return a boolean value.
17:42:34Then what I'm going to do is convert
17:42:36this command into a lowercase character
17:42:39and I'm also going to strip off any
17:42:41whites space leading or trailing whites
17:42:44space. After that I'm going to check if
17:42:47this command name and what is the
17:42:50command name going to be? Well, we are
17:42:52assuming that the command is always
17:42:53going to be one thing. So let's say the
17:42:55user passes in something like forward
17:42:57slashexit. So that's one thing. or the
17:43:00user passes in /tools or slashhelp. But
17:43:04the user can also pass in something like
17:43:07slash checkpoint and then the name of
17:43:10the checkpoint or let's say slashmodel
17:43:14and then the name of the model so that
17:43:16we can go ahead and change the model on
17:43:18the fly. That's a functionality we are
17:43:19offering to the user. So if they're
17:43:22processing one message with one model,
17:43:24for example, it's a very simple task.
17:43:26They want to use a cheap model. So they
17:43:28can just change it on the fly. if they
17:43:30want to go ahead with a better model or
17:43:33more expensive model, they can change it
17:43:34on the fly again. So that's a good
17:43:37command to give them. And what they're
17:43:39going to specify is the value next. So
17:43:41the command string is not always just
17:43:44one word. It can be two words as well.
17:43:46So I'm going to split it here. So I'll
17:43:48have parts command dotsplit and I'll
17:43:51maximum split it into one. We don't need
17:43:54multiple parts. We're just going to have
17:43:56two parts in total. The first one is
17:43:58going to be the exact command name. So
17:44:00that's parts at zero. And then there
17:44:02will be command arguments that are
17:44:04specified that will be parts at one. We
17:44:07obviously do this if the length of the
17:44:09parts is greater than one. Otherwise we
17:44:11have an empty string for the arguments.
17:44:14The reason we have to do this is because
17:44:16just imagine passing in a command like
17:44:18/exit that will result in an error,
17:44:21right? Because you're trying to split
17:44:24it. The split parts list is just one
17:44:27element in the list. Then you do parts
17:44:30at zero which is exit. But what about
17:44:32command attacks? It will try to do parts
17:44:34at one and we don't have a second
17:44:36element in this list. So we are
17:44:39explicitly passing in if length of parts
17:44:41is greater than one then only we want
17:44:43this otherwise we'll have an empty
17:44:46string. Then I just check if the command
17:44:48name is equal to forward slashexit or
17:44:52let's say the command name is forward
17:44:55slashquit. In that case I'm going to
17:44:57return false because I don't want to
17:45:00continue any further. And if it's none
17:45:03of these commands then we're going to
17:45:04return true because yeah it's a normal
17:45:07command. Also let me just add in an else
17:45:10case here just in case it's a command we
17:45:12don't recognize yet. We are just going
17:45:14to say unknown command. So we'll have
17:45:17console.print
17:45:18and then I'll pass in something like
17:45:20error then forward slash error so that I
17:45:22can pass in the error message here which
17:45:25is unknown command and then I'll pass in
17:45:29the command name which is this
17:45:32particular name and this is obviously
17:45:34going to be an string and then we return
17:45:37true. We return true for everything
17:45:39other than this first exit because
17:45:41that's the only time you want to exit.
17:45:43And yeah, I think that should work.
17:45:46Let's go ahead and try it. Forward
17:45:48slashexit now. And goodbye does show up.
17:45:51So that's great. This is the first time
17:45:52we peacefully exited. Let me try again
17:45:55and try a command we don't know. So
17:45:56let's say /help. That's not
17:45:58understandable as of now. And it says
17:46:00unknown command help. And we still keep
17:46:03going. That's great. Now let's exit
17:46:05again and start working on the next
17:46:07thing which is forward/help. That's the
17:46:10next thing I want to work on. And it's
17:46:12quite easy. I'm going to copy paste the
17:46:14help command actually because it's just
17:46:16a bunch of commands we have to show to
17:46:18the user. So we'll have l if command is
17:46:21equal to forward/help.
17:46:24In that case we can go to the tui. py
17:46:27and here create a function to show the
17:46:30help command. So I'll just paste in the
17:46:32function and let's go through it step by
17:46:34step. So we have the commands showing up
17:46:36and notice here I've put in two
17:46:39hashtags. The reason for it is we going
17:46:41to use a markdown to display stuff. So
17:46:44this commands will be converted into a
17:46:46second header kind of thing. Then you
17:46:48have all of these bullet points which
17:46:49say well /help is show this help. Then
17:46:52exit or quit is exit the agent. Clear is
17:46:55to clear off the conversation history.
17:46:57Then you have show current
17:46:58configuration. Then changing the model,
17:47:01changing the approval mode, showing the
17:47:03session statistics, listing all the
17:47:05available tools, MCP, saving the current
17:47:08session, checkpointing, listing all the
17:47:11checkpoints, restoring a checkpoint,
17:47:13listing all the saved sessions, and to
17:47:15resume a saved sessions. And then we
17:47:17just have the tips that just type your
17:47:20message to chat with the agent. The
17:47:21agent can read, write, and execute code.
17:47:23Some operations require approval and can
17:47:26be configured. And then I'll just do
17:47:27self.conole.print print pass in the
17:47:30markdown and I have to import that from
17:47:32rich. So we'll do from rich dot markdown
17:47:36and we'll import markdown. Great. Now
17:47:39all I need to do is use the TUI to
17:47:42display it. So self.tui dot show help.
17:47:45It shouldn't be a private function by
17:47:47the way. So let me remove it. It can be
17:47:49used outside of the file. And now I can
17:47:52call the show help function just like
17:47:54that. Nothing else. And obviously after
17:47:57that it will return true because we're
17:47:59not returning anything from here. Now
17:48:01let's try to run this again and see if
17:48:03that works. So we'll ask /help and it
17:48:06lists out everything. So we have /help
17:48:10which is in this particular theme. The
17:48:13reason for that is if you'll notice I've
17:48:15put this in back tick and back tick
17:48:18relates to code and based on our agent
17:48:20theme that's how it works. So yeah,
17:48:22there you go. And that's a pretty good
17:48:25looking terminal in my opinion. Now
17:48:28let's get on to the next thing. So the
17:48:30next thing I'm interested in is clear.
17:48:33So we clear off the conversation
17:48:35history. And to do that, we can just do
17:48:37l if command is equal to /clear. In that
17:48:41case, what do I want to do? Well, I just
17:48:45need access to the context manager. So I
17:48:47can close it, right? So I can do
17:48:49something like self.agent agent dot
17:48:51session dot context manager but there's
17:48:54no clear method here and I can't access
17:48:57the messages history here because
17:48:59messages is a private method or private
17:49:03variable so I need to go into the
17:49:05context manager and add a clear function
17:49:07so we'll have something like def clear
17:49:10and it will have self it will return
17:49:12nothing and what will it do it will just
17:49:15self dot messages to an empty list or
17:49:19you can just set dot clear to this and
17:49:21that should be fine as well. So let's
17:49:23come back to our main.py and call this
17:49:25particular function which is dotclear.
17:49:28Another thing I would like to do is
17:49:31clear of the loop detection as well
17:49:33because if you remember the loop
17:49:35detection also had some variables that
17:49:38were being stored right in history we
17:49:40were storing everything. Now if we have
17:49:43cleared off the conversation history and
17:49:45we're trying to find out the next thing
17:49:46I would also like to clear off the loop
17:49:48detection. So for that I can just do def
17:49:52clear. Then we'll have a self. It is
17:49:54going to return nothing. And similar to
17:49:56the previous one, we just going to do
17:49:58self do.history dotclear. Or you can
17:50:01just set it to an empty deck again. And
17:50:04then we just going to call self.agent
17:50:07dot session.loop detector dotclear.
17:50:10Maybe after that I would like to tell
17:50:12the user that yeah we create cleared of
17:50:14the conversation history. So I'll add
17:50:17something like self.con console or let's
17:50:20just say console
17:50:22dotprint and I'll pass in success. I'll
17:50:26pass in a closing success again and then
17:50:29I'll say conversation
17:50:32cleared. That looks good. Now let's try
17:50:35again. So I'll just open up my Python
17:50:38main. py. I'll type in some message.
17:50:40Hey, what's going on? And it says I'm
17:50:44here and ready to help with coding task.
17:50:46I'll just call clear now. So the
17:50:48conversation is cleared and then let me
17:50:51try asking what did I say previously and
17:50:54it just says you mentioned that you love
17:50:56clean code but struggle to write it
17:50:58that's coming from our memory because I
17:51:01did say that in a previous previous
17:51:03feature when I was trying to figure out
17:51:06memory stuff right so it doesn't
17:51:08remember what I said over here because
17:51:11the conversation was cleared so yeah
17:51:13that works well let me exit out of here
17:51:17And let's get started with the next
17:51:19thing which is showing the
17:51:21configuration. So if the user passes in
17:51:24something like l if command is equal to
17:51:27config I want to show all the
17:51:29configuration related details. Not all
17:51:32the configuration details but the basic
17:51:35things like what is the model that's
17:51:37being used, what is the temperature,
17:51:38what is the approval, what is the
17:51:40working directory, what is the max
17:51:42number of turns allowed, what are the
17:51:43hooks enabled. only those information.
17:51:46Maybe I don't want to display everything
17:51:48because the user can just go to the
17:51:50config.tml and figure it out themselves.
17:51:53But things that matter. So all the model
17:51:57related information except the context
17:52:00window. That's not something the user
17:52:02should be concerned about because for
17:52:04them the context window should not exist
17:52:06in the first place because they're using
17:52:08our agent. We're managing their context.
17:52:11And I'm just going to paste it in
17:52:12because it's just a lot of
17:52:14console.prints.
17:52:15So yeah, there you go. We already have a
17:52:18good configuration system. We can just
17:52:21make use of that. So at the top we do
17:52:24have self.config that's created, right?
17:52:27We have instantiated it. So let's just
17:52:29use those things over here as well. So
17:52:31self.config.
17:52:33Self.config.approval policy.al because
17:52:36approval policy was an enum. Then the
17:52:38current working directory, the max
17:52:40number of turns, the hooks enabled, all
17:52:42the configuration related items. The
17:52:44source of truth is going to be
17:52:46self.config all the time. That's how we
17:52:49made our application work as well. So
17:52:52yeah, you can copy the same or you can
17:52:54decide which ones you want to choose and
17:52:56show. So yeah, up to you. But this is
17:52:59just a simple thing. So I'll just quick
17:53:02go quick on this. So we have /config and
17:53:05we do see an error. It says config
17:53:07object has no attribute approval policy.
17:53:10That's because self.config.approval
17:53:12is the value not approval policy. And
17:53:15now let's try again. So we have /config
17:53:19and we do see current configuration.
17:53:22Then the model shows up temperature
17:53:24approval which is on request working
17:53:26directory max number of turns and hooks
17:53:28are enabled. That's good. Now, let's
17:53:31give the user some ability to change the
17:53:34model's name, to change the approval
17:53:37mode on the fly because model name and
17:53:39approval policies are something that the
17:53:42user might want to change based on each
17:53:45turn. As I mentioned previously, if
17:53:47you're using it for a simple task, you
17:53:50can use something like Claude Haiku. But
17:53:52if you're having an expensive coding
17:53:54agent task, you will use claude opus 4.5
17:53:59or whatever is the latest opus. So we'll
17:54:02have something like l if command name
17:54:05and the command name is here. If that is
17:54:09equal to /model then I'm waiting for the
17:54:13arguments as well. This is the first
17:54:15time we'll be using command arguments.
17:54:17So if command arguments are present and
17:54:19they should be present then what I'm
17:54:22going to do is self.config domodel name
17:54:24that's how easy it gets and we'll set it
17:54:28equal to the command arguments because
17:54:30we are expecting the command arguments
17:54:32to have the model name passed in
17:54:35something like /model and then whatever
17:54:38is the model maybe opus 4.5 is passed in
17:54:42like that and then we can just print out
17:54:45console.print print and then we have
17:54:48something like success then we have a
17:54:50closing success tag and then I just say
17:54:53model change two and then I pass in the
17:54:57command arguments
17:54:59okay and I'll just leave a space after
17:55:02that we have an if string here great now
17:55:06what if the command arguments is not
17:55:08specified in that case we've run into an
17:55:11error because the model should be passed
17:55:14in or Maybe the user wants to know what
17:55:17model are we using. We can do either one
17:55:20of those. Either you can tell that hey,
17:55:22you need to pass in the name of the
17:55:24model you want to change to or you can
17:55:27just display the current model that's
17:55:29being used. And that's what I'm going to
17:55:30do. I'll just print out something like
17:55:33current model is self.config dot model
17:55:38name. Now I'm not really sure if we can
17:55:42change the model name like that because
17:55:44how the way we did over here. The reason
17:55:47for that is if we go to the config.py
17:55:50and we have the model name here we have
17:55:52the property and we created the setter
17:55:54as well. That's great. So that means we
17:55:56can set the model name. I was worried we
17:55:58don't have a setter. So yeah let's start
17:56:00a coding agent again and see if we can
17:56:03change the model name. So we'll have
17:56:05/model first and it will print out the
17:56:07current model which is this one. Now
17:56:09I'll shift the model and I need to find
17:56:12a new model now. And to find out the
17:56:14model I can again go back to open router
17:56:17and let's just use the same memo v2
17:56:19flash model. And then we see model
17:56:23changed to mimo m v2 flash. Now I can
17:56:27again type in model and it says current
17:56:29model is xiaomi. Let me type in config.
17:56:32And it shows the model is updated to the
17:56:35Xiaomi model. That's awesome. You can
17:56:38also see the response changing. So yeah,
17:56:40go ahead see that. But I'm not
17:56:42interested. Let's move on to the next
17:56:44thing which is approval. So we'll have L
17:56:47if command name and it's actually going
17:56:48to be very similar to this one. So let
17:56:51me just copy and paste it. Just that
17:56:53instead of command name being model, we
17:56:55are going to have approval. And if the
17:56:57command arguments are present then what
17:56:59I want to do is set the approval policy
17:57:03instead of model name. So I have
17:57:05approval equal to whatever is the
17:57:08command asks right but this command
17:57:09argument can be wrong as well. Nobody's
17:57:12saying that the user is always going to
17:57:14pass in the wrong thing, right thing.
17:57:16Maybe they misspell. So in that case,
17:57:18what I'd like to do is first check if
17:57:21this approval policy is also correct.
17:57:23And to check it, I can just call
17:57:25approval policy like that. And let's
17:57:28import it from config and pass in the
17:57:31command args here because at the end of
17:57:33the day, this is an enum, right? So I
17:57:36can pass in because at the end of the
17:57:38day, this is just a class. Yeah, it is
17:57:41an enum but it's also a class. So I can
17:57:43just pass in the name of the command
17:57:46argument and now I can pass in the
17:57:49approval year and then whatever is the
17:57:53value I can just set it to this approval
17:57:55value. Now if the command argument is
17:57:58not correct this will raise an exception
17:58:01and we have to handle that. So I'll do
17:58:03try then I'll pass in everything over
17:58:06here and then I'll say except and if any
17:58:09exception occurs in that case what I'll
17:58:11do is print out the error. So we have
17:58:13console.print. Let me just copy this
17:58:16error command and paste it in over here.
17:58:19Maybe I can just say incorrect approval
17:58:23policy. And if you want to be more
17:58:26generous maybe we can tell the user all
17:58:28of the values that are possible. So
17:58:31we'll have incorrect approval policy
17:58:33command argu and maybe we can also pass
17:58:35in something like valid options and
17:58:38these are all of the valid options and
17:58:40to find all of the valid options it's
17:58:42going to be very easy for us because we
17:58:45just have to loop over every policy in
17:58:47this approval policy right so I can do
17:58:50something like comma dot join and then
17:58:54I'll go over every policy in approval
17:58:57policy and print out the p over here. So
17:59:00yeah, a comma separated list of all of
17:59:03the possible values. And maybe I can
17:59:05remove the error for this one because
17:59:07valid options does not need the error.
17:59:10And then we have a space here. Great.
17:59:13Now let's try. Oh, by the way, I forgot
17:59:16to change the success part here. So
17:59:19we'll just say that the approval
17:59:22policy change to and then we pass in the
17:59:25command argument. That's good. Now let's
17:59:28try again by reloading it. So we have
17:59:31slash approval and it says that the
17:59:33current model it's still displaying the
17:59:35model. I forgot about that because
17:59:37there's an else case as well. If the
17:59:39command arguments is present, we do all
17:59:41of this. But if the command argument is
17:59:44not specified, then I'll just say that
17:59:46the current approval policy is and then
17:59:50we'll pass in the approval policy year.
17:59:53And this is the approval policy enum. I
17:59:56would like to show the value. So we have
17:59:58dot value. Now let's try again. I'll
18:00:01exit out of here. Let's try again. And
18:00:04we have approval unknown command. That's
18:00:07right. This is the current approval
18:00:09policy. On request I'll pass in the
18:00:12incorrect one. So I'll have on yolo.
18:00:15That is the wrong value. So we get
18:00:17incorrect approval policy. On yolo valid
18:00:20options are on request, on failure,
18:00:22auto, autoedit. We misspelled that. We
18:00:26might want to fix that. And then we have
18:00:28yolo. So we have slash approval yolo and
18:00:31the approval policy is changed. And I
18:00:34know that the approval policy for sure
18:00:36is changed because everywhere we are
18:00:37using self.config.approval
18:00:39in our approval system. So you can test
18:00:41it yourself now. But now I'm going to
18:00:44move on to the next thing which is
18:00:46showing all of the statistics related to
18:00:48a session. So let's have something like
18:00:51command name is equal to and then we ask
18:00:55stats and then I can just try to get all
18:00:58of the stats from a session and I have
18:01:00to create a helper function for it and
18:01:02also decide what all stats do I want to
18:01:05display from this session. So I'll just
18:01:08go down and create the function get
18:01:10stats and then we have self. We're going
18:01:12to return a dictionary here because it's
18:01:14a big thing that we want to return. The
18:01:16value can be any because we're going to
18:01:19pass in multiple things that can be of
18:01:21any value. And then the first thing I'm
18:01:24going to pass in is the session ID.
18:01:25That's quite important because the user
18:01:28should know what session ID they are in
18:01:30because later on we'll give them the
18:01:32ability to change their session as well.
18:01:35Then we have created at which is
18:01:38something we have already stored. So we
18:01:40have self dotcreated at dotiso format.
18:01:44So it just returns the time in formatted
18:01:47format or in just a particular format.
18:01:50All right. Then we have the turn count
18:01:53and that is something we are storing as
18:01:54well. As you can see self turn count
18:01:57that's enough. So how many turns has the
18:01:59agent already taken. Then we have the
18:02:02message count. How many messages has the
18:02:05agent replied with and how many have you
18:02:08replied with? So those total counts and
18:02:12that is present within the context
18:02:14manager and as we know in the context
18:02:17manager we don't have anything that will
18:02:21allow us to get the length. All we need
18:02:23to do is find the length of this
18:02:25messages but it's a pro private
18:02:26property. So what I'll have to do is
18:02:29create a new function here and call it
18:02:32adderate property def. And let's say it
18:02:35is the message count. then we get a self
18:02:38and we're going to return an integer
18:02:40from here and we'll just return the
18:02:41length of the self dot messages. The
18:02:46reason this is a function is because if
18:02:48we try to create a variable here we'll
18:02:51have to increment the message count
18:02:52every single time and I don't want to
18:02:55keep track of that. Instead I can just
18:02:56find out the length of messages and
18:02:58that's good enough for me. So now I can
18:03:01do self dot context manager dot message
18:03:05count and that is our message count.
18:03:08Then we have the token usage.
18:03:11How many tokens have already been used?
18:03:13And for that purposes I had created a
18:03:16variable if you remember total usage. We
18:03:19don't want the latest usage now because
18:03:22latest usage was for the context limit
18:03:24hitting token usage or total token usage
18:03:27was created so that we track the total
18:03:30token usage and we tell to the user that
18:03:32yeah this is what we have done so far.
18:03:35This is how many tokens you've consumed
18:03:37over everything. And this much token is
18:03:40what you're going to get charged for. So
18:03:42let's go back to our context manager and
18:03:45create a property to get that token
18:03:48usage because it's a private property
18:03:50again. So I've add the rate property def
18:03:54total usage. Then we have a self. I'm
18:03:57going to return the token usage from
18:03:59here. and then I'll just say return self
18:04:03dot total usage.
18:04:07Now let's come back to this session and
18:04:10I can pass in self.context manager
18:04:12tokate total usage. Now if you're
18:04:15wondering why are we creating all of
18:04:17these private variables if we are going
18:04:20to anyways return them as a public
18:04:23variable. The reason for it is this
18:04:26total usage is only used within this
18:04:29context manager. So anytime you want to
18:04:31change the value of this total usage,
18:04:33you'll have to do it from within the
18:04:35context manager essentially. But this
18:04:38total usage that we are exposing out
18:04:40will not be able to change the value
18:04:42because we have not set up any setter.
18:04:45Right? So yeah, that's the benefit.
18:04:47After that, we're going to have the
18:04:49tools count. How many tools are present?
18:04:52And to find that we can just access the
18:04:55tool registry get all of the tools and
18:04:58then whatever is the list I can just
18:05:01count the length of that list. So that
18:05:03is the number of tools we have. After
18:05:06that we have the MCP servers. So how
18:05:10many MCP servers are connected and to
18:05:12find that I can do self.mcp manager dot
18:05:16and as you can see there are three
18:05:17functions but nothing related to the
18:05:19connected servers. to get access to that
18:05:22I can go to the MCP manager or I can go
18:05:25to the tool registry to find out all of
18:05:28the servers that are connected because
18:05:30if you remember in our tool registry as
18:05:33well we had a variable of MCP tools and
18:05:37those MCP tools are the things that are
18:05:40passed in and they're only registered if
18:05:43the MCP tool is connected right as you
18:05:46can see if it's not connected we
18:05:47continue and then pass it in so we can
18:05:50use either one of those. I'm going to
18:05:52use this tool registry to find out all
18:05:55of the MCP servers that are connected.
18:05:58So, let's go to this tool registry and
18:06:00expose out a property that is the MCP
18:06:06servers that are connected. So, I'll
18:06:08just say connected MCP servers. We have
18:06:10self. It will return a list of tools and
18:06:16then we can just return self dot MCP
18:06:18tools dot values. I'm not exposing out
18:06:21the string here because string is just
18:06:24the tool name and tool already has
18:06:26access to the tool name. Great. Now
18:06:29let's access this connected server. So
18:06:32we have dotconnected MCP servers and I
18:06:35can find the length of that. So those
18:06:37are the MCP servers that are connected
18:06:39and that seems like a good enough thing.
18:06:41Now let's use this get stats function in
18:06:43main. py. So we'll get the stats which
18:06:47is equal to self dot aagent do session
18:06:50dot get stats. And now I can try to
18:06:54display this dictionary. And to do that
18:06:57first let's just try to print out in
18:06:59bold what is the session statistics. So
18:07:02we have bold session statistics
18:07:07and then we have forward slashbold
18:07:09again. Okay. And after that I'll go over
18:07:13every key and value in this stats do
18:07:16items so that I can display it out. So
18:07:19we'll have print f and it should be
18:07:22console.print by the way and then we'll
18:07:24have f and then let's indent. So I'll
18:07:28leave let's say 1 2 and maybe three
18:07:32spaces. Then we have key and then the
18:07:35value. The key is this key. And that
18:07:38should be enough. So let's go ahead and
18:07:40try it. I'll try to do stats now. And it
18:07:44does show up session statistics. This is
18:07:46the session ID created at turn count is
18:07:49zero. Message count is zero. Token usage
18:07:51is zero. Tools count is 14. And MCP
18:07:53servers is zero. I can try to pass in a
18:07:57message. Hey, how are you? Let's try to
18:08:00run it. And it says one message. Okay,
18:08:03great. Let's try to run stats again. And
18:08:05it gives me turn count is one, message
18:08:07count is two because we had one message
18:08:10here and one message here. My response
18:08:13and the assistant's response. And turn
18:08:15is just one because the assistant only
18:08:17replied once till now. And then the
18:08:19token usage prompt tokens is 5356.
18:08:24completion tokens is 24 and total tokens
18:08:26is 5380. The reason prompt tokens is
18:08:29this much is because you have attached
18:08:30the system prompt. So yeah that seems to
18:08:33be working in our favor. Now let's go
18:08:36ahead and similarly display all of the
18:08:38tools that are available including the
18:08:41MCP servers. So we'll have tools here
18:08:44and we'll try to fetch all of the server
18:08:46tools. And to get all the tools we'll
18:08:49have to access the tool registry. So we
18:08:51have self.agent session dottool registry
18:08:55dot get tools. Right? So that gives us a
18:09:00list of all the tools. But what we need
18:09:02instead is all of the tools with its
18:09:06name. So what I'm going to do here is
18:09:08first print out or that these are all
18:09:11the available tools in bold. And then
18:09:14I'm going to pass in the length of
18:09:17tools and this is going to be an string.
18:09:20Let's say this is 14, which all gets
18:09:22printed out in bold. And after that,
18:09:25I'll go over every tool in this tools
18:09:28list. And then I'll print it out. And I
18:09:32want to print out in a bullet format. So
18:09:35what I'll do is replace this with a
18:09:38bullet list. And then we have the
18:09:41tool.name passed in here. Great. Now
18:09:44let's try to run this again. So we'll
18:09:46have forward slashtools. And these are
18:09:49all the available tools. Read file,
18:09:51write file, edit, shell, list directory,
18:09:53gre, glob, sub aent, codebase
18:09:56investigator and sub aent code reviewer
18:09:59is also present. And our test tool is
18:10:01also present. Test tool was something
18:10:04that we injected using tool discovery.
18:10:07So if you remember, we had a agent tools
18:10:09and then a test tool. This is different
18:10:12from another test tool. py that we
18:10:14created within the scripts. This is for
18:10:18hooks, but this one was for tool
18:10:21discovery, creating our own tool. So
18:10:23that's also showing up here. That's
18:10:25great. And if you add any other MCP
18:10:28server, for example, that will also show
18:10:30up. So we'll have config.tml.
18:10:32As you can see, the stuff related to MCP
18:10:36is commented out. Let's just uncomment
18:10:38this and then it should show up. So I'll
18:10:41just exit out of here. We have to
18:10:43reinstantiate because MCP server
18:10:45detection doesn't happen automatically.
18:10:47That's some feature you might want to
18:10:49add yourself. But let's just go ahead
18:10:51and see all the tools now. And we have
18:10:5428 available tools. Now, what I'd like
18:10:57to do is print out something like
18:11:00forward/mcp.
18:11:01And mcp should list out all of the
18:11:04servers that we are connected to.
18:11:06Nothing much, just all of the servers we
18:11:09are connected to or the connection is
18:11:11pending or the connection failed. I
18:11:14would like to see the status of every
18:11:16single one of those servers. So what
18:11:18we'll do is similar to tools print out
18:11:22forward/mcp.
18:11:24Then we have MCP servers which is equal
18:11:27to cell.agent.
18:11:30MCP manager dot and we'll try to get all
18:11:34the servers. We won't be able to use the
18:11:37one that tool registry gives us because
18:11:39tool registry gives us all the connected
18:11:41MCP servers. What I'm interested in is
18:11:43all of the MCP servers. So I'll just say
18:11:46get all servers. And that should give me
18:11:50all of the servers whether they're
18:11:52connected or not connected. So let's go
18:11:54to the MCB manager now. And here let's
18:11:58create a function to get all of those
18:12:00information. So we have def get all
18:12:02servers. And then we're going to list
18:12:05out a dictionary of string, any and what
18:12:09are we going to return? Well, I have to
18:12:11go over all of the clients that are
18:12:13present here because each client
18:12:15represents one server and then I'll
18:12:19print out its name, status, and the
18:12:22number of tools that are present here.
18:12:24So for name,
18:12:26client in self.clients do items, I'll
18:12:30just check if or we don't even have to
18:12:33check anything. I can just do server
18:12:35info is equal to and I'll pass in the
18:12:38name of the server which we have over
18:12:41here. Then I have the status of whether
18:12:44the server is connected or pending or
18:12:46what. So we'll have client status dot
18:12:49value. Then we have the number of tools
18:12:53and that is just the length of client
18:12:56dot tools and then we'll just append it
18:12:59to a big list. So we have servers which
18:13:02is equal to an empty list and I'll do
18:13:05servers do.append and pass in the server
18:13:08info. Finally, we'll just return a list
18:13:11of those servers. Great. Now let's go to
18:13:14main.py. We have called this function
18:13:16get all servers. That gives us
18:13:19everything. And in bold we'll just write
18:13:21out MCP
18:13:24servers. And similar to the available
18:13:27tools, we can just say that these are
18:13:29the three four MCP servers that we found
18:13:32out. So we'll have the length of MCP
18:13:37servers which just suggests what are the
18:13:40number of MCP servers connected and then
18:13:43we'll go over every server in the
18:13:46servers and then print it out. So the
18:13:49first one is going to be the status
18:13:51which we'll have as server at status and
18:13:55then we have the status color equal to
18:13:58green if the status is equal to
18:14:01connected otherwise it is going to be
18:14:04red because it's not connected either
18:14:07it's pending or it's rejected whatever
18:14:10and this should be status color not cold
18:14:13and we have to print it out so I'll have
18:14:16server name passed in here obviously
18:14:18we're going to have a bullet point
18:14:19still. So we have server name then a
18:14:22colon then within this we're going to
18:14:25have the status color because remember
18:14:29we have to pass in green or red here
18:14:31right so if it's green we'll pass it in
18:14:33like this or it's red we pass it in like
18:14:35this and then we also have to close it
18:14:38later on so we'll have forward slash
18:14:40status color as well and between them
18:14:42we're going to pass in the status
18:14:45whatever it was so it was uh connected
18:14:48or rejected what is it within green or
18:14:51red. And yeah, that's how we print it
18:14:54out. Maybe you would also like to add
18:14:57the information of how many tools there
18:15:00are that's connected. So I'll just have
18:15:03a space here followed by a parenthesis
18:15:06within which I'll pass in the number of
18:15:09tools. So we have server add tools and
18:15:12then we pass in the tools written here.
18:15:15Seven tools, eight tools, what is it?
18:15:17And that's great. So now let's try to
18:15:19run it again. So I'll just exit, rerun,
18:15:23and then I'll have /mcb and it says file
18:15:26system connected. Let's try to do a
18:15:28server that does not really exist. So
18:15:30I'll just copy and paste this. Then we
18:15:32have file system 2. Then we'll just have
18:15:35file system 2 two to something that I
18:15:38know doesn't exist.
18:15:40And now let's try to run it.
18:15:42So, as you can see, even if it's an
18:15:44invalid server, our application is able
18:15:47to handle it. That's good. And I'll just
18:15:49type out forward/mcp
18:15:51and we say file system connected 14
18:15:55tools. File system 2 is errored out and
18:15:58it has zero tools. Amazing. And if we
18:16:00print out all of the tools, they are
18:16:02still available. Oh, awesome. Now, let's
18:16:05move on to the next command. And now
18:16:08next command onwards we are moving on to
18:16:10the session and checkpointing features.
18:16:12So let's go back to our TUI where we
18:16:14have the help function. And here if you
18:16:17notice we've basically added in all of
18:16:19these things. Now we need to add in save
18:16:22checkpoint checkpoints restore sessions
18:16:24and resume. We're going to start off
18:16:26with session because checkpointing does
18:16:28require some schema of session to store
18:16:31it. Now first I would just like to tell
18:16:33you the difference between a session and
18:16:35a checkpoint. Session is essentially
18:16:37whenever you try to save whatever is the
18:16:40current state of the AI agent.
18:16:42Checkpointing is whenever you try to
18:16:45store the agents up to a certain point.
18:16:48So let's say I pass in three messages
18:16:50and the agent does those three tasks and
18:16:53then I create a checkpoint. After that I
18:16:57can continue that same session and ask
18:17:00more things to do and then I can create
18:17:03another checkpoint.
18:17:05So a same session can have three or four
18:17:07checkpoints. So it's the last you know
18:17:10the recoverable point that I want and
18:17:13session is essentially whatever is
18:17:15happening in one AI agent. So to give
18:17:18you a practical example this is one
18:17:21session. This ID that you see is related
18:17:23to one session of chat GPD. All right.
18:17:26You can create multiple sessions by
18:17:28creating a new chat. Checkpoint is
18:17:31essentially let's say this message and
18:17:34the response. So let's say I create a
18:17:36checkpoint here and later on I want to
18:17:39come back to this checkpoint. I can do
18:17:41that or I put in another message and
18:17:43save the checkpoint till here. So one
18:17:45session can have three four how many
18:17:47ever checkpoints I want. This is what
18:17:50I'm talking about. So let's go ahead and
18:17:52first create a session and we're already
18:17:54creating a session right? We do have a
18:17:57session py but now I would like to save
18:18:00that session. How does saving a session
18:18:02and for that matter checkpoint work?
18:18:04Well, we just store the list of messages
18:18:08and other related data that we have in a
18:18:11file. And once we store it in a file, we
18:18:14can retrieve it from the file and that
18:18:17is just the persistence that we're going
18:18:18to have. That is how you store sessions
18:18:21in AI coding agents. Everything is file
18:18:24system related as of now at least. So
18:18:27we'll have l if command name if it is
18:18:30equal to forward slashsave then we're
18:18:32saving a check a session. So in that
18:18:34case what I'd like to do is
18:18:37call a particular method but first I'll
18:18:40have to create that and what I'm going
18:18:42to do is within the context I'm going to
18:18:45create something or maybe I can create
18:18:47it with an agent as well and I'm going
18:18:50to call this maybe session manager or
18:18:52you can call this persistence manager
18:18:54whatever you want but this is
18:18:56essentially dealing with all of the
18:18:58persistence related stuff like
18:19:00checkpoints and sessions. So we'll have
18:19:03class persistence manager. This is
18:19:06really a bad name in my opinion. So you
18:19:08might want to change that. What are we
18:19:10going to get here? Where we are going to
18:19:11get in it. And then what I'm going to do
18:19:14is pass in the data directory. If you
18:19:16remember data directory was something we
18:19:18created in the utils which was the place
18:19:22where all of the data of the coding
18:19:24agent would get stored. We use this get
18:19:26data directory in memory and we're going
18:19:29to use this for sessions and checkpoints
18:19:31as well. So each session is going to
18:19:34have a new file and each file is going
18:19:38to be present within the sessions
18:19:40directory. So first I have to ensure
18:19:42that that sessions directory exists. So
18:19:45I'll just have self dot sessions dot or
18:19:49let's just say sessions directory is
18:19:51equal to self dot data directory forward
18:19:54slash and I'll pass in the sessions
18:19:56folder. So this is the sessions
18:19:58directory and then I want to ensure that
18:20:01this directory is present. So I'll have
18:20:03self dot sessions directory dot make
18:20:06directory if it doesn't already exist
18:20:09parents will be true exist okay is equal
18:20:12to true that means it won't raise any
18:20:14exception. This is something we've
18:20:15already looked into. I won't go much
18:20:17deep. And now I want to change the
18:20:20permission level. So I'll just have os.
18:20:24And I have to import OS now. And then I
18:20:28can pass in this directory here. So self
18:20:30do. sessions directory. And then I can
18:20:32pass in the mode which is 0700
18:20:38standard for a directory. Later on we're
18:20:41also going to create a checkpoints
18:20:42directory and similar thing will be done
18:20:45here. Now let's create a function to
18:20:47save the session and as I told we
18:20:50require certain information to save a
18:20:52session. We're going to get that
18:20:54information from wherever this
18:20:55persistence manager is called. Let's
18:20:58create it at the top. We'll have a
18:21:00session snapshot because yeah it is a
18:21:02session that we are going to have right
18:21:04and this is going to be a data class.
18:21:07Let's import the data class. And then
18:21:10we'll have session ID which is a string.
18:21:14Then created at which is a datetime.
18:21:17We'll import datetime from datetime.
18:21:20Something we've already looked into.
18:21:21Then we have updated at as well. Then we
18:21:24have the turn count. How many turns have
18:21:27happened in this session. And then we
18:21:28have a messages which is a list of
18:21:30dictionary of string, any right. Let's
18:21:34import it from typing. And I'll also
18:21:36have two functions here. The first one
18:21:39will be two dictionary. Both of them
18:21:40will be useful by the way. That's why
18:21:42I'm creating it now only when we have
18:21:44already reached this point. Two
18:21:46dictionary is a function that will
18:21:50convert this entire snapshot into a
18:21:53dictionary format. So we'll have return
18:21:56and then I can pass in all of these
18:21:57things. So we have session ID which is
18:22:00self dot session ID. Then we have
18:22:03created ad which is self do.created.
18:22:05created at. Then we have updated at
18:22:08which is self do.dated at and then we
18:22:11have turn count which is self dot turn
18:22:14count. And then finally we have messages
18:22:17list which I can pass in just like that.
18:22:20Great. Now another function that we're
18:22:22going to have is going to be a class
18:22:24method and that is going to be from
18:22:28dictionary. So you call something like
18:22:30session snapshot dot from dictionary.
18:22:33You pass in a data object which is going
18:22:36to be a dictionary and you just convert
18:22:38it into a session snapshot. Right? So
18:22:41the return value here is session
18:22:42snapshot. And let's import annotations
18:22:46from future because yeah we've already
18:22:49talked about this so much at this point.
18:22:52It's just boilerplate code for us. And
18:22:54now I'll just return. By the way, we
18:22:56don't get self here. We get class. And
18:22:59then we can just do return class and
18:23:02then pass it in because this is a class
18:23:04method, right? This is not an instance
18:23:06method. And then I can pass in all of
18:23:08the things again. So we have session ID
18:23:12is equal to data at session ID. And
18:23:16actually I'll just see you after I've
18:23:18typed in everything so that I don't
18:23:20waste much time. Now we're going to take
18:23:22this session snapshot as the value here.
18:23:25So we have snapshot which is of the type
18:23:27of session snapshot and it's going to
18:23:30return nothing and then we can try to
18:23:34store it in the file. So first I'll have
18:23:36the file path which is equal to self dot
18:23:39sessions directory forward slash and
18:23:41then I need to create a new file right
18:23:43because this was a directory of
18:23:45sessions. Now within that I'll create a
18:23:48new session or save to an already
18:23:50existing session. we just override the
18:23:53value and then we can just pass in
18:23:55something like this. So we have snapshot
18:23:58dot session ID because that is a unique
18:24:01identifier because if you go to our
18:24:03session py file you'll notice here we
18:24:06have the session ID created and that's
18:24:08just a UYU ID a unique universal
18:24:11identifier and the file that we're going
18:24:14to save by the way is going to be JSON.
18:24:17So all of the data like the messages
18:24:19list and everything will go within this
18:24:21JSON and then I can try to open the
18:24:24file. So with open and then I'll pass in
18:24:26the file path. Then I'll have the write
18:24:29mode because I want to write it. And
18:24:32then you can also pass in the encoding
18:24:33maybe which is UTF8
18:24:36as file pointer. And in that case I'll
18:24:39just do JSON.dump.
18:24:42And I need to import it. I'm using
18:24:44JSON.dump dump by the way not dumps
18:24:46because dumps does a string dumping we
18:24:49are trying to dump the entire object
18:24:52here and I'll pass in the snapshot but
18:24:54snapshot is a session snapshot that's
18:24:56why we created a two dictionary method
18:24:59and then I'll pass in the file pointer
18:25:02and then I'll also pass in the
18:25:04indentation which is two because
18:25:07whenever I'm trying to store a document
18:25:10what I'd like to do is have certain
18:25:12indentation because even if the user
18:25:14wants to look at it is this formatted
18:25:16nicely you know and after that I would
18:25:19like to change the mode of this file as
18:25:22well so we have oschange mode and I'll
18:25:25pass in the file path and then the mode
18:25:29is going to be 0600600
18:25:33means that all the access is to the
18:25:36owner only and they have the read and
18:25:38write permissions. So the sixth year
18:25:41represents the user related permissions.
18:25:44Then zero is for the group and the zero
18:25:47again is for the world permissions. Six
18:25:50means four + two because four stands for
18:25:54read and two stands for write. So 4 + 2
18:25:56is six. So by the way if you just do 400
18:26:00here what will happen is that the user
18:26:02will have only read permissions. They
18:26:04won't have write permissions. If you do
18:26:06200 they only have write permissions.
18:26:08They don't have read permissions. But
18:26:10600 is read and write both. And for the
18:26:13rest of them, it's just zero. And if you
18:26:16want to add any debugging, go for it.
18:26:18But what I'm going to do now is just
18:26:21create the persistence manager within
18:26:23the main. py because that's where it is
18:26:25useful. So I'll have something like
18:26:28persistence
18:26:30manager equal to persistence manager and
18:26:34I'll import from agent dotpersence and
18:26:37then I can call the save method. So I
18:26:40have persistence manager dots save
18:26:42session and I'll pass in the session
18:26:44snapshot. Now now I have to create the
18:26:46session snapshot which I can do over
18:26:48here as well. So I have session snapshot
18:26:51is equal to session snapshot and we have
18:26:54to import that as well from
18:26:56agent.persistence
18:26:57and now we can pass in multiple things.
18:27:00So the first one is session id. Well we
18:27:03do have access to that. So I can just do
18:27:06self dot session or agent dot session
18:27:10dot session id. After that we need the
18:27:13created at that's also present within
18:27:15the session. So we have self.agent
18:27:18session dot created at. If you want you
18:27:21can create another function within
18:27:24session. py which is a wrapper around
18:27:27all of this. So like it just encapsula
18:27:29encapsulates all of this logic. Then you
18:27:32have updated at which is self.agent
18:27:34agent dot session dot updated ads. Then
18:27:38you have the turn count which is equal
18:27:40to self dot agent dot session dot turn
18:27:45count. And as you can see turn count is
18:27:47a private variable. I have to fix that.
18:27:50So I'll have at the rate property here
18:27:53def turn count. Then we get a self. We
18:27:57return an integer and we return self dot
18:28:00turn count. Now I can come back to
18:28:03main.p py and have self do.turn
18:28:06count. And finally, we have the messages
18:28:09list, which is self.agent
18:28:11dot session. And this time, we're going
18:28:14to access the context manager because
18:28:16that gives us all of the messages. So,
18:28:19we can pass in get messages here. And
18:28:22it's a function. So, I'll just call it.
18:28:24And that's it. Now, I can take this
18:28:26session snapshot and pass it in over
18:28:28here. And boom, we are saving the
18:28:31session now. So I would like to tell the
18:28:33user that yeah it was successful. We are
18:28:36printing out the session. Good for you.
18:28:38So we have success and a forward slash
18:28:41success after which I say
18:28:45session saved and then I pass in self.
18:28:52ID so that the user can keep track of
18:28:54the session ID. So yeah, let's try it. I
18:28:58am going to open up the agent again.
18:29:00This time I'm going to ask something
18:29:02like hey how's it going? And then let's
18:29:06ask another thing. Are you doing good?
18:29:08And now what I'm going to do is save
18:29:11this session. So I'll have slashs save.
18:29:14And we run into an error. Object of type
18:29:16datetime is not JSON serializable. And
18:29:20that does make sense. We directly
18:29:22passing in the datetime object within a
18:29:24dictionary and then passing it into
18:29:27JSON.dumps. That's not right. So what I
18:29:30need to do is go to this persistence and
18:29:32within this two dictionary I will
18:29:34convert this created at into ISO format
18:29:38and then similarly for updated at we're
18:29:40going to do dot ISO format and in the
18:29:43from dictionary we're going to have date
18:29:46data at created at and this is also
18:29:50going to be a dictionary that's coming
18:29:52from the file system right so we have
18:29:55converted it into ISO format when
18:29:57storing it into the file system and when
18:29:59we are retrieving it we are just passing
18:30:02it like that that's not right we have to
18:30:04convert it back into datetime so to do
18:30:07that we can do datetime dot and pass in
18:30:09from ISO format and then I'll pass in
18:30:14the string which is data at created at
18:30:17and similarly I'll do datetime dot from
18:30:19ISO format and then pass in date at
18:30:23updated at and that's it that should fix
18:30:26our errors now let's try the same thing
18:30:28again. So I'll have hey, how's it going?
18:30:31I hope everything is good with you and
18:30:34your family. Okay, awesome. Now let's
18:30:36try to save it. And it does say session
18:30:39saved in this particular thing. Now if
18:30:42we try to go and find out the data
18:30:44directory where all of this is stored,
18:30:46you will find the session with this name
18:30:51dot JSON. But I'm not going to spend
18:30:53much time finding it. What I'm going to
18:30:55do instead is create a function to list
18:30:58out all of the sessions. Instead of
18:31:00having save or just like having save,
18:31:03we're going to have sessions as well.
18:31:05And that will list out all of the
18:31:06sessions that I have. And now I can go
18:31:10to persistence. py and list out all of
18:31:13the sessions. So I have def list
18:31:16sessions and we have self. It's going to
18:31:19return a list of dictionary of string,
18:31:23comma, any. Now I'll just try to go over
18:31:26all of the sessions that are present.
18:31:28Where are they present within the
18:31:29sessions directory? And they're present
18:31:31within a file. So what I can do is go
18:31:33over every file within the directory.
18:31:35Right? And to do that, I can just do for
18:31:38file path in self.essions directory.
18:31:42Glob. If you remember glob, it will just
18:31:44help us iterate over all of the JSON
18:31:46files because all of the non-JSON files
18:31:48are irrelevant to our case. Whenever we
18:31:51are storing sessions, it has to be in
18:31:53the JSON format otherwise it's been
18:31:55tampered with. And then I can try to
18:31:58open the file path that we are in. Then
18:32:01I want to open it in the read format.
18:32:04And I can just pass in the same encoding
18:32:06when I was writing it in which is UTF8.
18:32:09And then I'll just do data is equal to
18:32:11JSON.load and pass in the file pointer.
18:32:15So I have to open it as a file pointer
18:32:18as well. And now I can maintain a
18:32:21sessions list which I'll ultimately
18:32:23return and that is going to be sessions
18:32:28dotappend and I'll pass in all of the
18:32:30data. So the first thing is the session
18:32:34ID. Now the reason I'm not doing
18:32:36something like session
18:32:39snapshot dot from dictionary and then
18:32:42passing in the data is because we have
18:32:46the messages list here as well. And I
18:32:49don't want the messages list displaying
18:32:51out to the user when we're trying to
18:32:52list all of the sessions. That's why I'm
18:32:55going to do basically the same thing but
18:32:59this time excluding the messages. So
18:33:01here we go. We'll have session ID
18:33:04created at updated at and turn count as
18:33:07a string. Then the equal to sign will be
18:33:10converted into a colon. And yeah, that
18:33:13looks good. Those are all of the
18:33:15sessions. So at the end, we can just
18:33:17return the sessions. But just doing that
18:33:19is not enough. We have created at and
18:33:22updated at. What if I sort the sessions
18:33:26by updated at? That will just give the
18:33:28user the latest session, right? So we
18:33:31have sessions dot sort and then we'll
18:33:34pass in the key which is lambda
18:33:37x x at updated at that's the key we're
18:33:40going to sort based on and reverse is
18:33:43equal to true so that the latest updated
18:33:46at shows up first because updated at is
18:33:50going to be within an integer format
18:33:52right because updated at is going to be
18:33:54in a string format and that string is
18:33:56just an integer right an integer that's
18:33:58converted into a string because created
18:34:00at is a time stamp kind of thing. So
18:34:03what I'll do instead of this is just
18:34:06remove this part. I don't have to
18:34:08convert it into datetime. I'll just
18:34:10compare the date the string objects
18:34:13directly when sorting and then we'll
18:34:16just return the sessions. So the latest
18:34:18value will be seen at the top. A
18:34:21descending value is done. Now let's try
18:34:23to list all of the sessions.
18:34:26So I'll have a persistence manager
18:34:28created. After that, we don't need a
18:34:30snapshot. I'll remove that. Then we have
18:34:33persistence manager.list
18:34:36sessions and I'll pass in all of the
18:34:38things. We don't need anything within
18:34:41the parameters. And maybe after that I
18:34:44can do console.print
18:34:46within a bold thing and we can also do
18:34:49it on a new line. Then the bold will be
18:34:52completed and then we have saved
18:34:55sessions. We'll go over every session
18:34:57and try to print it out. So for s in
18:35:00session and I did not save that in a
18:35:03variable. So sessions is equal to
18:35:05persistence manager.list sessions and
18:35:07for session in every session we're going
18:35:10to have console.print
18:35:12then I'll pass it with the indentation
18:35:15here. So we have the indentation and
18:35:18then I'll pass in the session along with
18:35:21its ID. So we have s at session id and
18:35:26after that we'll just pass in the number
18:35:28of turns and what is the last updated
18:35:32value. So turns is just s at turn count.
18:35:38Then we have updated which is s at
18:35:41updated at and that is good enough I
18:35:45believe. So let's try to list out all of
18:35:47the sessions now. So I run the AI agent
18:35:50again. I do slash sessions and then hit
18:35:54enter and we get an error. The error is
18:35:57JSON decode error because this is one of
18:36:01the sessions that is the one that worked
18:36:04and the second one that did not work is
18:36:06over here because created at gave us an
18:36:08error and this gives us a nice look into
18:36:12how Python's JSON.dump works because
18:36:16here when you were trying to do
18:36:17JSON.dump dump. It dumped the first
18:36:19entry but in the second entry it gave an
18:36:22error because we ran into that datetime
18:36:24object value. So as you can see it did
18:36:26not it does not first convert it into a
18:36:29valid JSON object and try to put in no
18:36:32it's trying to put it line by line word
18:36:35by word character by character and if it
18:36:37results in an error it stops there. So
18:36:40for that reason we were getting an
18:36:42error. So I've deleted the invalid
18:36:45session and now let's try again. This
18:36:47won't happen in the real use case
18:36:49because the user will never run into the
18:36:51case where it did not work out. So let's
18:36:54try to say sessions now and we do get
18:36:58one saved sessions turns is two updated
18:37:01is this particular value. You would also
18:37:03like to store token related information
18:37:06that comes along with the persistence
18:37:09manager. Since session snapshot we have
18:37:11something like total usage and then that
18:37:15is going to be of the type of token
18:37:17usage and here you pass in something
18:37:20like total usage here which is self dot
18:37:25total usage but you have to convert it
18:37:28into a dictionary. So you do dot
18:37:31dictionary like that or you create a
18:37:33specific function to convert it into a
18:37:36dictionary and then pass it in. So here
18:37:39you just do that. But I think data class
18:37:41should handle it. We'll see. And now
18:37:44we're going to have the total usage here
18:37:46as well which is equal to data at total
18:37:50usage. And we have to convert that into
18:37:54a valid total usage. So we'll have total
18:37:56usage class here and we'll pass in all
18:38:00of the things also. This is token usage
18:38:02not total usage. And here we get prompt
18:38:05tokens completion total and cache.
18:38:07Right? What I can do is try to
18:38:09deconstruct it within this particular
18:38:11class. I hope that works out. I've not
18:38:14tested any of this. So, let's see. But
18:38:17when we list out all of the sessions, I
18:38:18don't have to tell the user how much
18:38:20tokens were used. So, that's fine. And
18:38:24now I'll just delete off the previous
18:38:26session so that we have a good
18:38:28perspective of how things are looking.
18:38:30And now we can go back to main. py where
18:38:33save session is called. And I'll pass in
18:38:35the total usage as well, which is
18:38:38self.agent dot session dot context
18:38:41manager dot total usage. Now let's see
18:38:44how that looks like. I'll exit out of
18:38:46here and rerun. Then I'll say hey how is
18:38:49it going and then I'll say save. Session
18:38:53is saved. Let's try to open up the
18:38:55Visual Studio Code. And here we have
18:38:57session ID created at updated at
18:38:59messages and then the total usage which
18:39:01is 7,616
18:39:0324 completion and 7640 total. And now
18:39:07when we try to list all of the sessions
18:39:10I can just do sessions like that and we
18:39:12see one session. That's great. Now what
18:39:15I'd like to do is load up a session if
18:39:18we want. So let's say I save a session
18:39:20and then 2 days later I come back to a
18:39:23session. I want to load it up. In that
18:39:26case, everything should be perfectly fit
18:39:28in within my context, within my session,
18:39:31everything. So, let's work on that. What
18:39:34we're going to do is again create a
18:39:36similar command to sessions or actually
18:39:39save. So, let's just copy that and paste
18:39:42it in over here.
18:39:44And then we're going to have resume.
18:39:47Resume is for a session. And if we want
18:39:50to load up a checkpoint, it's going to
18:39:52be restore because we are restoring a
18:39:55checkpoint. We are going back to a
18:39:56previous position potentially. So we are
18:39:58restoring. But in a session, we are
18:40:00always resuming because if you try to
18:40:02save a session three or four times, it
18:40:04will update the same value because it
18:40:06will try to write to the same file ID,
18:40:09right? It will write to the same JSON
18:40:12file and it will override the value.
18:40:14That's how session works. So here first
18:40:17thing I'll check is if the command
18:40:18arguments is not specified in that case
18:40:21let me go ahead and do console.print
18:40:24I'll pass in the error then I have a
18:40:28forward slash error then I have usage is
18:40:32resume and then you need to pass in the
18:40:35session ID. I think that's enough of an
18:40:38error message. Let's leave it at that. I
18:40:41can go ahead and create a function so
18:40:43that I can resume the previous session.
18:40:46So I'll go back to a persistence and
18:40:49here I have to create a very simple
18:40:50function. So instead of saving a session
18:40:54all I need to do is open a file, read it
18:40:56and load up the session because once I
18:40:58load up the session I'll have all of the
18:41:00details about a session and then based
18:41:03on that I can make my decision of how I
18:41:06want to load it. So I have session
18:41:08snapshot on null value because it can
18:41:10potentially be a wrong value that's
18:41:13passed in. Then I have the file path
18:41:15with self dot sessions directory. I pass
18:41:18in the session ID. So we are going to
18:41:21get the session ID from the parameter
18:41:24here. And then we going to pass it in
18:41:27like that session ID do.json. After that
18:41:30I'll just check if that file path exists
18:41:33because the user can obviously pass in
18:41:35the wrong session ID. In that case, I'll
18:41:37return null. But if the file path ex
18:41:40exists, then I'll open up the file path.
18:41:43I'll open it in read mode and then I'll
18:41:45do data is equal to JSON dot load. And
18:41:50then I'll pass in the file pointer.
18:41:53Nothing else is required then. And then
18:41:55once I have the data with me, I can just
18:41:57return session snapshot dot from
18:42:00dictionary because I have to return a
18:42:02session snapshot until now I have a data
18:42:04dictionary. So I just return that and
18:42:07yep that's it. Now I can call this load
18:42:10session in main.py so that I know what
18:42:13details are required to
18:42:16resume a session. So we will have an
18:42:18else here and then first we have to load
18:42:21up the session. So we do snapshot is
18:42:24equal to and I have to create a
18:42:27persistence manager as well. Let me just
18:42:29paste that in. And then I have
18:42:31persistence manager dot load session.
18:42:35And then I'll pass in the session ID
18:42:37which is whatever the user passed in
18:42:39through command arguments. And then I'll
18:42:42check if that snapshot exists. If it
18:42:44doesn't then I again have to tell the
18:42:47user that hey listen we have an error
18:42:50with us. This snapshot that you said
18:42:52does not exist. So we'll have snapshot
18:42:55or session does not exist. But if the
18:42:59snapshot is there, then what are we
18:43:02going to do? Well, in that case again
18:43:04we'll have an else condition. This is
18:43:06getting too nesty. Then I'll go ahead
18:43:09and create a new session now. So we'll
18:43:11have a new session object created here.
18:43:14And then I'll pass in everything. All we
18:43:16require is a config. So let's pass in
18:43:20the config equal to self do.config.
18:43:24After that we have to update the
18:43:25variables because in session it goes
18:43:28ahead and updates the value like created
18:43:31at updated at all of that but we are
18:43:33resuming a session so we need to go back
18:43:35to those values. So we'll have session
18:43:39dot created at is equal to whatever we
18:43:42had in snapshot.created at. Similarly
18:43:45we'll have session do.updated at which
18:43:47is equal to se snapshot.updated updated
18:43:49at. And similarly, we're going to have
18:43:51session dot turn count which is equal to
18:43:54snapshot dot turn count. And for total
18:43:57usage also we'll have session dot
18:43:59context manager dot total usage is equal
18:44:03to session dot context
18:44:07manager or not session sorry snapshot
18:44:09dot total usage.
18:44:12You might also want to store the latest
18:44:15usage in the session because that's how
18:44:18we manage the session. But I forgot
18:44:21about that. But that's something you
18:44:23should do. I would recommend you to
18:44:24pause the video and do it yourself right
18:44:27now because that's very important. I'm
18:44:29just not going to go back. It will take
18:44:32too long otherwise. And now I have to
18:44:34restore all of the messages and add it
18:44:37to the context manager. And to do that
18:44:39I'll go over every message that is
18:44:41present within the snapshot messages and
18:44:45then I'll check if the message get role
18:44:48is equal to system I will continue I
18:44:51will ignore it because system prompt
18:44:53will be regenerated and readded. That's
18:44:56how our get messages function in our
18:44:59context manager works. But if the role
18:45:02is let's say equal to the user in that
18:45:06case I'll just do session.context
18:45:08manager dot add user message and I'll
18:45:11pass in message dot get and then I'll
18:45:16pass in the content and if the content
18:45:18is not present it's going to be an empty
18:45:20string. So I'll just copy this line
18:45:22because this will also be required if
18:45:24the role is assistant or tool. So I'll
18:45:27have l if message role is equal to
18:45:32assistant. In that case also we'll add
18:45:35the assistant message. I'll pass in the
18:45:38content. And if there are any tool calls
18:45:40I would like to add that as well. So
18:45:42message.get
18:45:44and then we have tool calls. After that
18:45:48I'll have another one which is l if
18:45:50message at ro is equal to let's say the
18:45:54tool ro. In that case also we'll add the
18:45:59tool result and for that I just need to
18:46:02pass in the tool call ID and if it's not
18:46:06present an empty string or we also have
18:46:10the content if we have to pass that in
18:46:12and that can be an empty string as well.
18:46:15So yeah that's all of it for the context
18:46:19management. After that we can just
18:46:21return the session from here. Actually
18:46:23we don't need to return anything. What I
18:46:26can do after this is self.agent session
18:46:29and then try to close that session
18:46:32related stuff. And if you remember what
18:46:34is all of that? Well, if we go back to
18:46:37our agent. py, we were closing the
18:46:40client and the MCP manager, right? This
18:46:43is these are both the things that we
18:46:44want to shut off. And then we set the
18:46:47session to null as well. Well, I'm not
18:46:50interested in setting the session to
18:46:52null. Instead, I'll just set both of
18:46:54them to close and then I'll reinitialize
18:46:57the session based on whatever we created
18:47:00over here. So, I'll have self.agent dot
18:47:03session removed and instead I'll do
18:47:06self.ession.client.clo
18:47:09and I will have to await that. By the
18:47:11way, I'll have to do self.agent.clo.
18:47:15Similarly, self.agent.MCB
18:47:18manager.shutdddown. Or what you can do
18:47:20is move both of them into proper
18:47:22functions. So within the session you
18:47:25have a close function where both of them
18:47:26are called and within agent.py you call
18:47:28that close method. But I'm just done
18:47:32working on all of that. Now I'm not
18:47:34interested in any refactoring. So I'll
18:47:36just convert this function into an
18:47:37asynchronous function. I'll go back to
18:47:40this run interactive where I'll do await
18:47:43self.andle command. And now everything
18:47:46should work fine. At least I'm hoping
18:47:49that I'll remove all of these
18:47:51persistence manager stuff. That's not
18:47:53needed anymore. So, we've closed down
18:47:55all of the things that we needed to
18:47:58close. And now I'll just do self dot
18:48:02agent dot session is equal to session.
18:48:06By the way, there's another thing we
18:48:08probably have to do and that is self
18:48:10do.initialize
18:48:13session dot initialize because
18:48:16initialize needs to be awaited. That's
18:48:18why it was not present within the init
18:48:21function, right? So, we had to
18:48:23explicitly call that initialize function
18:48:26so that we could have our MCP manager
18:48:27and tool discovery and everything. After
18:48:30that, we set the agent session to this
18:48:33session that we just created or actually
18:48:37we don't have to do session.initialize.
18:48:39My bad. We have to do session.initialize
18:48:41not agent.initialize
18:48:43because agent is the one that we just
18:48:46closed. We closed it because self.agent
18:48:49session needed to be replaced with the
18:48:51new session that we created and that's
18:48:54why the new session needs to be
18:48:55initialized and then we can just set it
18:48:58to session and I hope this works in the
18:49:01first try because debugging this is
18:49:02going to be complex. So we have
18:49:04console.print and after that we'll we
18:49:07can just say something like success and
18:49:11then we'll have a closing success as
18:49:13well. Let's put all of them into a
18:49:15string and we'll just say resumed
18:49:19session and I'll pass in the session ID
18:49:22which is session dot session ID. I can
18:49:26also remove this console.print and I
18:49:28believe that should work out. If it
18:49:32doesn't well we'll have to debug it and
18:49:34it's going to be bad. Let's see. So
18:49:36we'll have exit. Let's run it again.
18:49:39I'll just say resume and let's say I
18:49:41pass in X. that session does not exist.
18:49:44That's good. Then we resume and I'll
18:49:46pass in this session. Paste it in and
18:49:49hit enter. And yeah, we have an error.
18:49:52It says at property turn count of
18:49:55session object has no setter. So we'll
18:49:57go back to main.py and we see turn count
18:50:00cannot be set up. And that's because
18:50:04well it is just exposed as a property.
18:50:06So we'll go to the session and here
18:50:09instead of having property just like
18:50:10that we'll have a property setter as
18:50:13well or what we can do is just make this
18:50:15a public variable. So self turn count
18:50:18and within this file I'll just set all
18:50:22the turn counts that were present into
18:50:25turn counts that are public and replace
18:50:29all. So I can just click on this to
18:50:32replace all and that's it. Everything is
18:50:35updated. Let's see if that works out. So
18:50:37I'll just run again. I'll again type in
18:50:42let's say sessions so that we have all
18:50:44of the sessions printing out. Then I
18:50:46have let's say something like hey how's
18:50:48it going? I am happy. Let's say hey
18:50:52everyone I'm doing great. Thanks for
18:50:54asking. Great. Now I can just resume the
18:50:57previous session. So this particular
18:50:59session that we after this I can just
18:51:02resume the previous session that I had.
18:51:04So this session should be lost and yeah
18:51:07we have another error none type object
18:51:10has no attribute total usage. So if we
18:51:13go to the context manager dot total
18:51:15usage and this happens because the
18:51:18context manager over here is null and
18:51:21we're setting this to total usage value.
18:51:23We're calling total usage on that. And
18:51:25the reason it is empty is because if you
18:51:27go back to the context manager or the
18:51:30session, you'll notice that the session
18:51:32is initialized with context manager as
18:51:34null. And later on when we call
18:51:37initialize, the context manager is given
18:51:40a value. So what I'd like to do now is
18:51:45call this self.ession.initialize
18:51:47beforehand only. So we have something
18:51:50like await self session.initialize
18:51:52initialize. So that will initialize all
18:51:55of the values and now I can update
18:51:57whatever I need. That should work out
18:51:59because context manager is now there.
18:52:02Let's run it again. I'll say hey how's
18:52:04it going? I'm happy so that it just
18:52:07knows what I was talking about and then
18:52:09I can try to resume a previous session.
18:52:12And then we have another error property
18:52:14total usage of context manager object
18:52:16has no setter. So let's go back to our
18:52:19context manager. And here we have total
18:52:22usage. And that's true. It is just a
18:52:25property, not a setter. So we'll have to
18:52:27update that as well. Let me just remove
18:52:29that at the rate property. Let's expose
18:52:31it outside as total usage. And then I'll
18:52:35update all the self dot total usage that
18:52:37are private to be self do.tal usage that
18:52:42are public. And then let's replace all.
18:52:45And now let's try to run it again. I'll
18:52:48paste in the same prompt. I'll try to
18:52:50resume the same session.
18:52:53And as you can see, it resumed the
18:52:55session that's passed in over here. But
18:52:57there's one problem. The resumed session
18:53:00has a different session ID. Now, because
18:53:02think about it, what happened here? We
18:53:04called session. So, it initialized a new
18:53:07session ID and then we did not update
18:53:10the session ID at all. But we are
18:53:12storing it in snapshots. So, it should
18:53:14be easy for us. We have session dot
18:53:17session id equals to snapshot dot
18:53:20session ID. And now it should work as
18:53:23expected. So let's try again. I'll ask
18:53:26the same question. Then just for fun
18:53:29I'll say what did I say previously? And
18:53:32it does say I said I'm happy. So now
18:53:35let's try to resume the session. And
18:53:38then we get resumed session at this one.
18:53:40So the ID is now correct. And now I can
18:53:42ask the same question. what did I say
18:53:45previously? So the context should shift
18:53:47now because we have a different session
18:53:49that's loading up and it looks something
18:53:51like this where I asked it how is it
18:53:55going. So a bit different. Let's see
18:53:58what did I say previously and it just
18:54:00said you mentioned that you love clean
18:54:02code but struggle to write it. Maybe I
18:54:04can tell it what did I ask you in my
18:54:09first message to you. And it does say in
18:54:12your first message you asked, "Hey, how
18:54:13is it going?" So that's great. That
18:54:15means resuming a session also works. I
18:54:17can also save this session again. And
18:54:19that's a pretty useless session. And I
18:54:21can exit out of here. I can create a new
18:54:26session. You know, whenever I create a
18:54:27new agent, a new session is there. And I
18:54:29can resume the session that I want. So
18:54:32maybe this session. And yep, we are back
18:54:35into this conversation. So that's
18:54:37awesome. This is why I was telling you
18:54:39that in every session class, we're going
18:54:41to add a new instance of MCP manager or
18:54:45chat compactor because we can have
18:54:46multiple sessions and I don't want that
18:54:48conflicting. So yeah, that's awesome.
18:54:51Now the next thing we are interested in
18:54:53is checkpointing. So we have created
18:54:56session manager. Something very similar
18:54:58to that is going to be in the
18:55:00checkpoints as well. To create a
18:55:02checkpoint, the user just has to pass in
18:55:04forward/checkpoint.
18:55:06So I'm just going to copy this slashsave
18:55:09because it it's going to look a bit like
18:55:11that because instead of just saving a
18:55:13session, we are saving a checkpoint and
18:55:15I've already explained the difference to
18:55:17you. So we have forward/checkpoint.
18:55:19Then we have the persistence manager.
18:55:23Then we are going to also create a
18:55:24session snapshot but instead of saving a
18:55:26session, we're going to have save
18:55:28checkpoint because save checkpoint is
18:55:30going to take in the same session
18:55:32snapshot. We're going to store all the
18:55:34information regarding a session but at a
18:55:37specific time. That's going to be the
18:55:39difference. Session just overrides
18:55:41everything. Checkpoint can be created at
18:55:44different time intervals and at
18:55:45different message points. So let's go to
18:55:48the persistence manager and create the
18:55:50save checkpoint function. So you have
18:55:52def check save checkpoint. Then we're
18:55:55going to have a self. We're going to get
18:55:57a snapshot which is going to be the
18:55:59session snapshot and it's going to
18:56:02return a string. And now we're going to
18:56:04have all of the stuff related to
18:56:06checkpoint. So first of all I would like
18:56:08to have a directory of checkpoints. So
18:56:11I'll just copy these two lines paste it
18:56:13down here. We're going to have
18:56:15checkpoints
18:56:17directory. Then self dot checkpoints
18:56:19directory domake directory. Then we have
18:56:22checkpoints
18:56:23here.
18:56:25And that's it. We'll pass in self
18:56:27do.points directory domake directory and
18:56:31then os.change
18:56:33mode and then I'll pass in the
18:56:36checkpoints directory again. Now I can
18:56:38scroll down and here when we're trying
18:56:41to save a checkpoint I would like to
18:56:43create a file path. So I'll have file
18:56:45path equal to self dot checkpoints
18:56:47directory forward slash and just passing
18:56:50in the snapshot do ID is not enough
18:56:53because a session ID is just not enough.
18:56:58You can have multiple checkpoints within
18:57:00a session. So if your file's name is a
18:57:03session each checkpoint will just
18:57:05override the value meaning you won't be
18:57:07able to have multiple checkpoints. So
18:57:09what I'm going to have instead is first
18:57:11of all I'll pass in the snapshot session
18:57:14ID with an underscore. This will really
18:57:17help us later on when we're trying to
18:57:19list all of the checkpoints related to a
18:57:22particular session. And then I'll have
18:57:24the timestamp added here. So the
18:57:27timestamp is going to be datetime dot
18:57:31now. And then I'll format this string
18:57:35and I'll format it in this time. So
18:57:37first we are going to have the year then
18:57:39we going to have the month then we are
18:57:42going to have the day followed by
18:57:44underscore then we have percentage h
18:57:47which is r then we have month then we
18:57:50have minute and then we have second and
18:57:52now I can just pass in this time stamp
18:57:54after the underscore so that's the
18:57:57differentiating factor what time you
18:57:59create the checkpoint on after that I
18:58:01would just like to open this file path
18:58:03and save stuff in it right so we'll have
18:58:06Right? Then we have encoding which is
18:58:09equal to and then we have UTF8 and we're
18:58:12going to have this as a file pointer
18:58:14followed by JSON.dump.
18:58:17Then we'll pass in the snapshot to
18:58:19dictionary and then we'll pass in the
18:58:21file pointer indent is equal to two.
18:58:24After that again we're going to have the
18:58:26OS.
18:58:28Then we are going to save this file path
18:58:30with 0600.
18:58:34And then we are going to return the
18:58:36checkpoint ID essentially. So this thing
18:58:39that we have here is going to be the
18:58:41checkpoint ID. That's what I want to
18:58:43return from here. So we have checkpoint
18:58:46ID is equal to this thing. And then we
18:58:48have checkpoint ID do.json. And let me
18:58:52just convert it into a string string.
18:58:54And then yep that's it. So we return the
18:58:57checkpoint ID from here. And that is us
18:59:00saving the checkpoint. as easy as it
18:59:03gets. Now I can go back to the main. py
18:59:06where we create a session snapshot. So
18:59:08we have the session ID created at
18:59:11updated at the turn count all the
18:59:14messages the total usage and then I just
18:59:17need to call save checkpoint pass in the
18:59:20session snapshot and we'll get the
18:59:22checkpoint ID from here. Now checkpoint
18:59:25ID is always returned but it might be
18:59:28the case that so I can just take this
18:59:31checkpoint ID and say checkpoint
18:59:34created and then we pass in this
18:59:37checkpoint ID over here. Awesome. Now
18:59:40let's try to run it. So I have Python
18:59:42main. py. I'll just try to list out all
18:59:45of the sessions. This was the session.
18:59:47I'll just resume the session and I can
18:59:50create a checkpoint here. So it says
18:59:52checkpoint created and if I go to my
18:59:55folder here of AI agent here we see the
18:59:58user memory as well but we want to go
19:00:00within checkpoints and here is our
19:00:02checkpoint. It has the same stuff as a
19:00:06session. The only difference is that
19:00:09within a session you can have multiple
19:00:11checkpoints. That is the only
19:00:13difference. So now we have to load a
19:00:16checkpoint and doing that is almost
19:00:19similar to what we did when we tried to
19:00:22restore this particular thing this
19:00:24resume session. So I'll just copy it and
19:00:27paste it over here. Now let's say if the
19:00:30user passes in restore in that case we
19:00:33first check if the command arguments is
19:00:35specified because the user needs to have
19:00:37passed in the checkpoint ID. So we have
19:00:40restore and then the user needs to pass
19:00:42in the checkpoint
19:00:44ID. Then we go to the persistence
19:00:48manager. We try to load a session. Nope.
19:00:50We try to restore a session or load a
19:00:53checkpoint. So we have load checkpoint
19:00:55here. Let's go to the persistence
19:00:57manager and create that function. So
19:01:00similar to this load session, we'll have
19:01:02a load checkpoint. Let me just create it
19:01:04down here. And I'll just paste the
19:01:06checkpoint here.
19:01:09So we go to a particular directory which
19:01:12is the checkpoints directory. We get the
19:01:14checkpoint ID here which is going to be
19:01:17a string. And then we just go to
19:01:19checkpoint ID.json.
19:01:21Then if the file path does not exist, we
19:01:23return null and then we just try to open
19:01:26the file and return the session
19:01:27snapshot. Awesome. Now we go back to
19:01:30main. py. We have tried to load the
19:01:32checkpoint. So we get the snapshot. If
19:01:34the snapshot does not exist, we'll just
19:01:36say checkpoint does not exist.
19:01:40And then we get this. We try to create a
19:01:43session again. Then we update all of the
19:01:46values as necessary. And yeah, we have
19:01:49updated the session as well. This is how
19:01:51easy checkpointing was because we just
19:01:53storing session within a checkpoint. So
19:01:56let's try to exit. Let's try to have
19:01:59Python main. py. But actually we don't
19:02:02have a checkpoints list. But we do have
19:02:04the checkpoint ID here. Let me just do
19:02:07slash restore and I'll pass in this
19:02:10checkpoint ID. By the way, I needed to
19:02:12update this value here when I had
19:02:15restore saying that yeah, we've resumed
19:02:18the session and I pass in the session
19:02:21ID. But I could also say something like
19:02:23yeah, we've resumed the checkpoint. So I
19:02:27have resumed session and then we can
19:02:29pass in checkpoint as checkpoint ID.
19:02:34Awesome. Now it won't matter here
19:02:38because we won't be able to see it. But
19:02:39I know that should work. Let's hit
19:02:41enter. And we see resumed session. And
19:02:45this was the session that loads up just
19:02:47like this one. That's great. I'll just
19:02:49ask what did I say to you in previous
19:02:53messages. Let's hit enter. And these are
19:02:57what I asked. Great. So, let's try to
19:03:00exit here. Now, the last thing we want
19:03:02to look into is listing all of the
19:03:04checkpoints that are available. I'm not
19:03:06going to get into that because listing
19:03:08all of the checkpoints is as simple as
19:03:10listing all of the sessions. All we need
19:03:14to do is go over all of the JSON files
19:03:16that are present within the checkpoints
19:03:18directory and just add it and display it
19:03:21on the main screen. One thing I would
19:03:23like to say though is that if you want
19:03:26to load all of the checkpoints that
19:03:28belong to a particular session, you can
19:03:30do that. How? Well, you all you need to
19:03:33do is go over all of the JSON files, but
19:03:38this time the pattern will change.
19:03:39Instead of having asterisk.json, what
19:03:42you can have is something like session
19:03:44ID and underscoreis.json.
19:03:48Because remember whenever we creating a
19:03:51checkpoint what are we doing? We are
19:03:53having a session ID underscore that
19:03:56allows us to filter by a specific
19:03:59session ID if we want to. So if you want
19:04:02to load all the checkpoints that belong
19:04:04to a particular session this would be
19:04:06really useful. So yeah you can do that.
19:04:10And yeah that's pretty much it for this
19:04:12entire AI agent tutorial. There's lots
19:04:16of stuff to do. I'm sure there are a lot
19:04:18of bugs that you'll have to fix, but
19:04:20I've just not encountered them yet. Then
19:04:23there are a lot of prompt tunings that
19:04:26you'll have to do yourself to fit your
19:04:28use case. And there are multiple
19:04:30features that are also left out. For
19:04:33example, AI agents also have the feature
19:04:35of sandboxing. That's something you
19:04:37might want to work on. AI agents also
19:04:39have the feature of LSP. That's also
19:04:42something you might want to work on. And
19:04:44a lot of techniques are still being
19:04:46developed. Also, the tools that we have
19:04:48are kind of basic. You can make the
19:04:51these tools much more advanced and much
19:04:54more fast as well if you do certain
19:04:56optimizations. So, there's a lot of
19:04:58scope and a lot of stuff to be done.
19:05:00There are also new techniques that are
19:05:02being developed regularly to improve AI
19:05:05coding agents.
19:05:07So, let me know if you need a part two
19:05:08of this entire course. I'd be happy to
19:05:11create one. But I would really encourage
19:05:13you to try out all these new things and
19:05:16the bugs yourself because that's where
19:05:19you learn the most. So, thanks for
19:05:21watching and I'll see you in the next
19:05:23video.

About this transcript

This page contains the full transcript of How Claude Code Works (By Building It) by Rivaan Ranawat, generated from the public captions YouTube serves with the video. The transcript has 192,770 words across 26,239 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.