How to Build a Multi Agent AI System — Transcript

Full transcript

1. 0:00this is how to build a multi-agent system  with watsonx.ai. AI multi-agent systems work
2. 0:05using react prompting they turn  your vanilla LLM into an agent,
3. 0:09a system that can reason and complete tasks.  Now if we put a bunch of them together you
4. 0:14can build up a team of specialist agents  to automate tasks we're going to do that
5. 0:18in exactly 15 minutes but is this possible with  all LLMs and is there a framework that can help?
6. 0:24We'll get to that but for  now  I've broken it down into three
7. 0:26steps and it begins with getting our LLMs ready.
8. 0:29All righty guys we have we' got 15 minutes  on the timer let's kick things off so the
9. 0:33first thing that we need to do is  import a number of dependencies
10. 0:35so we're going to be using crewAI it's a  framework that allows you to do multi-agent
11. 0:40systems or build multi-agent systems so  we're going to say from crewAI we are
12. 0:45going to be importing the crew task and  agent and then we also want to give our
13. 0:50multi-agent system access to some tools  so we're going to say from crewAI_tools
14. 0:57import Serper Dev Tool. Now the cool thing is  that there's a number of different tools as
15. 1:02well right so if you wanted to use CSVs or  docx files or GitHub or Json or MDX or PDF
16. 1:08you got the ability to do that as well let me  know if we uh maybe do that in another video
17. 1:13okay then in order to build our LLMs and  interact with our LLMs we're going to be
18. 1:17using watsonx so we need an interface to watsonx  so we're going to say from langchain_IBM we're
19. 1:24going to import watsonx LLM so that's going to  allow us to build our LLM interface and then
20. 1:30we also need OS so this is going to give us the  ability to set our API keys for our environment
21. 1:35so we're going to do that first so we're going  to say os.environ and then we want to go and set
22. 1:40our watsonx_API key value and we're going to set  this to one that I got a little earlier so I'm
23. 1:48going to grab that API key from over there okay  and let's zoom out a little bit and then we also
24. 1:53want to set our API key for our Serper Dev tool  environment so this is actually going to allow
25. 1:59us to interact with the net so we can set that by  running or passing through os.Environ then setting
26. 2:05Serper_API_key and then we're going to set that to  the value that you can get from Serper.dev so you
27. 2:13can actually go and register for a free API key  and get that up and running we've got a bit of an
28. 2:18issue there so that's because of our quotes. Okay  cool, so we've now got our API key set we've got a
29. 2:23couple of dependencies imported we're now going  to create the first LLM so we're actually going
30. 2:28to be using two LLMs, I'll get back to that in a  sec, so to create our LLM we're going to create an
31. 2:34instance of the watsonx LLM class then we need  a couple of parameters or keyword arguments.
32. 2:39So the first one is model ID and we're  going to set that to a blank key for now
33. 2:43we also need the URL so because of the  way that watsonx is spun up we've got
34. 2:49the ability to run this on a whole bunch  of different environments plus we've also
35. 2:52got different multi- region availability,  so if you wanted to go and run it in the
36. 2:55US or in Europe or in APAC you've got the  ability to do that as well as on premises.
37. 3:00Okay so we've got model ID we've got URL we also
38. 3:02need to specify the parameters for  the model so this is going to be how
39. 3:05we actually decode and we also want to  be able to pass through our project ID.
40. 3:09So you can actually isolate your machine learning  and data science projects using watsonx.ai.
41. 3:14So we're first up going to create a parameters  dictionary so this basically dictates how we
42. 3:20actually go and decode from our model so we're  going to create a new value called parameters
43. 3:24and set that to a blank dictionary how we're doing  on time we got 12 minutes okay so our first value
44. 3:29is going to be decoding method, and we're going  to set that equal to greedy. So we're just going
45. 3:35to use the greedy algorithm for now and then  we also want to go and specify max new tokens.
46. 3:41So this gives you a little bit  of cost control so it dictates
47. 3:44how many new tokens you're going to be generating.
48. 3:47So we're going to grab that parameters  value and set that into our LLM,
49. 3:51and then we also need to specify our  model ID. So our first LLM that we're
50. 3:55going to be using is llama 370b, so we've  got the ability to actually go and check
51. 4:00that you can see that we've got the  ability to use llama 370b instruct .
52. 4:04Feels kind of weird I know I've been using llama  2 for so long now so we're going to specify Meta
53. 4:08Llama and then it's going  to be llama-3-70b-instruct,
54. 4:18and we also need to go and set the URL.
55. 4:20So we're going to go to over here so  we've got the ability to copy this
56. 4:24URL so I'm just going to grab that  and then paste this into my URL area
57. 4:30uh save that so it should reorganize it okay  so I'm just going to get rid of everything
58. 4:36after the do so we don't need that. Okay so  that is our LLM spun up we also need to pass
59. 4:41to our project ID so again that's available  in a similar region so it's available from
60. 4:47your watsonx environment so the project ID is  just over here so I'm actually got a project ID
61. 4:53I spun up a little bit earlier so I'm going to  grab that and paste that in there okay so this
62. 4:59should be our LLM. So we've gone and specified  our model ID, so we're going to use Llama 3,
63. 5:02our URL, our primesm and our project  ID. Let's actually test it out so
64. 5:06if I run print LLM.invoke and  then I pass through a prompt,
65. 5:10So this could be any prompt, but  right now this is just a vanilla LLM.
66. 5:13So I could go and say who is Niels Bohr? you can
67. 5:18tell I've been getting into a  little bit of physics lately.
68. 5:21So if we go and run this now  so the file's called agent.py.
69. 5:23So to run it we can run python agent.py I'm  going to pause the timer while we  test it out.
70. 5:28We have exact 10 minutes left,  okay, all right. Coincidence? Yes.
71. 5:36Okay so if this runs, so I'm going  to pause it while it's generating,
72. 5:39becuase that's a little bit out of my control,
73. 5:41and let's see if  we get a  response to who is Niels Bohr?
74. 5:46Cross my fingers.
75. 5:56Okay, take a look so we've got a response so  this is coming from :lama 370b right now.
76. 6:00So we've got a response so "Niels Bohr was a
77. 6:02Danish physicist who made  significant contributions
78. 6:05to the development of atomic physics and quantum  mechanics he's best known for his model of the
79. 6:09atom which posits that electrons exist in specific  energy levels or shells around the nucleus."
80. 6:14Sounds pretty good .Okay so  that is our first LLM created.
81. 6:18Now I did mention that we're  going to need a second LLM
82. 6:20and that's going to handle our function calling.
83. 6:22So we're going to go and power on ahead. So we've  got our LLM set up now we're actually going to go
84. 6:27on ahead create our second LLM and then build  our agent. So we're effectively on to part two
85. 6:32alright the time is at 10 minutes. I'm  going to kick this off let's go all right.
86. 6:34So we're going to need to just copy this
87. 6:36LLM over here so we're going to  create the function calling LLM.
88. 6:42I'm going to speed up just a little bit,
89. 6:44not too much. So we're going to create a  new variable called function calling LLM,
90. 6:49and for this LLM we're going to use a slightly
91. 6:50different model so we're going  to use the IBM dash mistal AI,
92. 6:55and we're going to use the Merlinite model,
93. 6:56so merlinite dash 7b. Okay cool, so  those are our two LLMs now set up.
94. 7:01Now we're going to create the agent  create the agent and our first agent
95. 7:06is going to be a researcher but keep in  mind eventually you could have a ton of
96. 7:09different agents they could all delegate  and interact with each other. So first
97. 7:13agent is going to be a researcher and we're  going to create an instance of the agent. So
98. 7:18there's a ton of keyword parameters  that you need to pass through here.
99. 7:21The first one is the LLM.  So this is going to be the
100. 7:23base LLM that we use to do all of our generation.
101. 7:26We then need a function calling LLM,
102. 7:28and that is going to be be our  merlinite model that we just set
103. 7:31up there. So I'm going to paste  that keyword argument in there,
104. 7:34and then we also need to specify three  additional arguments so there's more than
105. 7:38three so role, goal, and backstory,  and I've pre-written those prompts,
106. 7:45so we're going to spin those or  pass those through in a second.
107. 7:48We also want to determine whether  or not we want to allow delegation.
108. 7:51So eventually, I might do this in another  video, but eventually you've got the ability
109. 7:54to allow these agents to hand off to    each other and just sort of delegate,
110. 7:58and work out which agent is going to be  able to do the task the best. We also
111. 8:01want to specify some tools so we want  to give our agent access to the net,
112. 8:05and then the last thing is whether we want it to
113. 8:07be verbose or not. So I'm  going to set that to one.
114. 8:10okay, so roll goal back, sorry, so our this  basically dictates what our agent is good at.
115. 8:15So our agent is going to  be a senior AI researcher,
116. 8:19the goal for that agent is  going to be find promising
117. 8:21research in the field of quantum Computing,
118. 8:24and then the last backstory bit is going to  be, so this agent got a little bit of a cross
119. 8:30collaboration between AI and Quantum  research so our back story is you're
120. 8:34a veteran Quantum Computing researcher  with a background in modern physics.
121. 8:37Okay so that's our agent we now need  to give it some tools. So tools.
122. 8:41This one's pretty straightforward  so we're going to create a tool
123. 8:43called search and that's going to be  equal to the Serper dev tool so this
124. 8:47I believe inherits from Langchain it  allows you to go and search the net.
125. 8:51We just need to grab that search  value and pass it through here.
126. 8:54So that is our first agent now completely set up.
127. 8:57Now we actually want to go and fill  this out a little a little bit more.
128. 8:59So we want to give it a task, so create a task,
129. 9:05and a task is effectively what we  want our multi-agent system to do.
130. 9:08So our task first task so which is going  to call task one, is going to be a task,
131. 9:13and then the task is going to have a couple of  things. So we need a description of the task.
132. 9:18We need the expected output. How we doing on  time? 7 minutes, okay. I think we're all right.
133. 9:26we're sounding all right. We  also want the output file,
134. 9:30so this is actually going to  output the results of the task
135. 9:34and then we want the agent that  we want to complete that task
136. 9:37okay so the description,
137. 9:38again I've baked these a little bit  earlier, so we're going to grab the
138. 9:41description search a net and find five  examples of promising AI research.
139. 9:46going to paste that into description and I'll  make all of this available via GitHub later
140. 9:50so you guys can grab it as well, and then the  expected output is going to be a detailed bullet
141. 9:55point summary on each of the topics each bullet  point should cover the topic background why the in
142. 9:59is useful. Our output file is just  going to be the name of the output
143. 10:02file that we want to Output. So we're  going to say task one output.txt and
144. 10:06the agent that we want is not a string  the agent is going to be our researcher.
145. 10:10So which agent that we want  to complete the task. Now
146. 10:14we need to pass through these tasks to the crew.
147. 10:16So put it all all together with the crew. So we're
148. 10:22actually going to hand it off to  a crew and let the crew run it.
149. 10:24So we're going to say crew is equal to crew and  then inside of there we want to specify agents,
150. 10:30and initially we're just going to have one agent  but we're going to  come back to that in a sec,
151. 10:34and then we also want to specify the tasks that  we've got. So right now it's just task one,
152. 10:39and then we want to specify verbose equal to  one, and then to we just need to kick it off,
153. 10:44so we're going to run crew or print crew.kickoff.
154. 10:48okay all right. right I'm going to pause  it. So let's actually go and run it.
155. 10:51So I'm going to clear this for now, and  we're going to run python.agentpy again,
156. 10:57and all things holding equal this should start
157. 10:59doing our research. Let's pause it  all right 5 minutes 32 on the clock.
158. 11:04Let's see how we go.
159. 11:06if this works we should start this task so search  the internet find five examples of AI research.
160. 11:11You should see it making calls out  to the net and ask getting research back.
161. 11:15so you can see that this has generated some action  input. So it's saying to use the search function
162. 11:19so search query and then run this command  promising AI research and Quantum Computing,
163. 11:24and take a look we've got some  responses back from the net.
164. 11:27So anything in orange is  what we're actually getting
165. 11:29back from the internet so once it's  collaborate or collated everything it needs,
166. 11:33it should generate the bullet point  summary of the research that we want.
167. 11:37So let's just let that run for a little sec.
168. 11:42You can see that we've got a bunch  of other research coming back there.
169. 11:52And take a look we've got our response back.
170. 11:53So you can see that we've got  all of our output down here so
171. 11:58here are five examples a promising AI research.
172. 12:00So it's gone and done all of the  research and then it's decided that
173. 12:03it's done so the agent now understood  that it's complete because it's gone
174. 12:06through its thought process it's  queried as much as it needs to,
175. 12:09and it's generated its response. So  down here we've got five examples of
176. 12:12promising AI research in the  field of quantum Computing.
177. 12:15So Quantum AI, Quantum machine  learning, Quantum optimization,
178. 12:18Quantum RL, Quantum neural networks,  and we've got all of the responses.
179. 12:22The cool thing is that it will output it as a text
180. 12:25file if you wanted to go and use it  those are all of our bullet points.
181. 12:29You can go and use that I'll include that in the  GitHub repo so you can see the responses as well.
182. 12:33Okay so that is I'm actually going to save this  as the first example so you guys can grab it
183. 12:39because we're actually going  to regenerate this in a second.
184. 12:41So far we've gone and created one agent,  one task, right, but this isn't really a
185. 12:45multi-agent system. It's a single agent system.
186. 12:47So what we now need to go in ahead  and do is generate our second agent
187. 12:51to be able to go and truly convert  this into a multi-agent system. All
188. 12:56right we've got 5 minutes 32 left on  the clock can see that let's let's go.
189. 12:59Let bring bring this home. Second agent, okay  so what we're actually going to do now let me
190. 13:03zoom out a little bit so you can see that  we've got our first agent and task pair.
191. 13:07We're actually going to copy this, and I'm  just going to paste it down here again.
192. 13:12So we're now going to create the second agent and  it's pretty much the same the only thing is that
193. 13:19we don't need to give our second agent access to  the net so we're going to tweak this a little bit.
194. 13:24So let me zoom in. All right so we can  get rid of this function calling bit,
195. 13:28get rid of the tools, and this agent  is actually going to be a writer.
196. 13:32So we want this agent to actually  write some keynote speeches for us.
197. 13:36So we're actually going to pass through  a different role, goal, and backstory.
198. 13:39So I'm going to grab that out of here so  our second role is a senior speech writer.
199. 13:43So let's say that you're writing speeches  for a prominent executive you need to go
200. 13:47and write some speeches so the role  is going to be senior speech writer,
201. 13:50the goal is to be right engaging witty  Keynotes features from the provided research,
202. 13:56I'm going to paste that in, and then the backstory  is going to be your veteran, no it's the this
203. 14:02should be tweaked a little bit, looks like I just  went and copied and pasted when I was writing,
204. 14:07so you're a veteran Quantum Computing,
205. 14:09I'm going to say writer, with  a background in modern physics,
206. 14:12but again you can go and tweak this a bunch more,
207. 14:15but effectively you could provide a bunch of  background on   how this person's a writer.
208. 14:19Okay so that's our backstory. Then  we want to go and create our second
209. 14:23task. Tweak this and and let  me know how you actually go.
210. 14:26All right so we're going to  generate our second task here.
211. 14:28So our second task is write an engaging  keynote speech on Quantum Computing. so
212. 14:34I'm going to paste this in the description,
213. 14:36and then our expected output is a  detailed keynote speech with an intro
214. 14:40a body and a conclusion. So we're  going to paste this in over here.
215. 14:45Perfect. And then our second output file  is just going to be called task two output.
216. 14:49Okay so that is our second agent, so our  second agent is a writer. Our second task
217. 14:54is to get that keynote speech written. So  it's almost like a sequential agent right.
218. 14:58so the first agent's going to do  the research, find all the research,
219. 15:01second agent's going to step in and  then convert it into a keynote speech.
220. 15:04so we can actually go and  pass through our second agent,
221. 15:07which is going to be our writer down here.
222. 15:10So this is what gives us the multi-agent feel  right and eventually once we allow delegation,
223. 15:15if we did that in the next  videol. Then we'd actually
224. 15:17be allowed to or be able to do all of the handoff.
225. 15:20So our tasks so maybe we convert agents into
226. 15:24a series on the channel you let  me know in the comments below.
227. 15:26Okay so we've now gone and pass  through our second task over here.
228. 15:29Go and pass through our second agent we go and run
229. 15:31this now. We should effectively  go through that two-step process
230. 15:34so if we go and run this  again, going to clear this,
231. 15:39and we should also get two outputs now, Right?
232. 15:41because our second output file is called task  two output. So I'm going to run python agent.py.
233. 15:46I'm going to pause the timer  so we've got 2 minutes 43.
234. 15:49This all blows up that's how much  time we're going to have left 2
235. 15:52minutes 42 to to to solve this all right  so let's let this run and see how we go
236. 16:02okay so you can see it's triggering the chain.
237. 16:04It's doing the research again so  again. This is still the first agent,
238. 16:08so you can see that the working  agent is the senior AI researcher.
239. 16:12We haven't got to the writer yet.
240. 16:18Okay so we're doing research.
241. 16:27Okay so you can see we've now I know we're  still still still doing researcher there.
242. 16:31I thought we finished oh  actually no we've got a bug.
243. 16:38Just realized all right I'm going  to start the timer again so at 2:41
244. 16:41so I just noticed that we  were using the same agent.
245. 16:44So inside of our second task we actually need this
246. 16:47to use the second agent because right  now we're just using the base agent.
247. 16:50so we actually want this to be writer okay we're  going to we're going to change the agent there,
248. 16:55and then let's kick off our timer again.  So I'm going to clear this. All right.
249. 16:59It only took me 10 seconds to resolve. So  we're at 2:22 we still got time to debug,
250. 17:04but we're probably going to speed through  this so editors speed through this
251. 17:09okay so we're triggering our first bit  from our senior AI researcher. Hopefully
252. 17:13once we get to that next debug stage,  we're now onto our what is it what
253. 17:17do we call senior speech writer  is going to pick up from there.
254. 17:25Okay take a look so we've now successfully gone  and handed off to the senior speech
255. 17:29writer so this is our second agent kick in  hopefully they just write our keynote speech.
256. 17:33It might be limited cuz we've only given it
257. 17:35500 tokens to write the speech  but you sort to get the idea.
258. 17:38Fingers cross we now get a keynote  speech written in under 15 minutes.
259. 17:43Not too bad all right take a  look that's our keynote speech.
260. 17:47Not too bad. Alright. So we should  so we can see that we've gone and
261. 17:51done the research. We've got  a bullet point list over here.
262. 17:55Bullet point list over here and then we've got our
263. 17:57keynote speech over here. Here so  if we actually go and take a look,
264. 18:00we've got our two outputs so task one  output and task two output so this is
265. 18:04the next set of output from the second  run so we've got all of our DOT points,
266. 18:09and if we go and take a look at our task two  output let's read out a bit of our keynote speech.
267. 18:14Ladies and gentlemen esteemed guests and  fellow innovators I'm thrilled to be speaking
268. 18:19with you today, at a revolution, about the  revolutionary potential of quantum Computing.
269. 18:23as we stand at the threshold of a new era in  technological advancement it is imperative
270. 18:27that we explore the possibilities  that Quantum Computing has to offer
271. 18:31I don't know how witty this  is, but it is a keynote speech.
272. 18:34In the realm of artificial intelligence  Quantum Computing is poised to
273. 18:37unlock unprecedented capabilities Quantum...
274. 18:40I'm going to put this all in the GitHub  repo so you can read it yourself but
275. 18:43you can see that we have in fact gone  and written a bit of a keynote speech.
276. 18:47So we could definitely take this  further but for now that's how
277. 18:50to build a multi-agent system with watsonx.AI
278. 18:52I'll catch you in the next one for