wk5 2 review iat111 — Transcript

Full transcript

1. 0:00Hi, welcome back. It's week five of INT
2. 0:03111.
3. 0:05I'm going to go over a little bit of a
4. 0:07review and then get into ethics this
5. 0:10week. Um, we've done a lot with how
6. 0:14large language models work. Uh, this
7. 0:17notion that you can write a prompt and
8. 0:20it goes in a big context window with a
9. 0:22lot of information. There's a model
10. 0:25that's been made somewhere and that
11. 0:27comes back to you. Um,
12. 0:30we've discussed already that there are
13. 0:32different kinds of large language models
14. 0:35from very simple, which they all used to
15. 0:36be. Today you see that sometimes with a
16. 0:39fast moniker. Reasoning models that do
17. 0:42the step-by-step. So still using the
18. 0:44same technology as simple, but at least
19. 0:47one part of it cuts it into steps and
20. 0:50then those steps use that same system.
21. 0:53Often you see this as
22. 0:55thinking
23. 0:56uh, or deep thinking.
24. 0:58Um, there's going out to the web and as
25. 1:01we've talked about,
26. 1:03um, it's a kind of a tool that goes out,
27. 1:07in this case to bring up a browser and
28. 1:09get information.
29. 1:10Um, it doesn't
30. 1:12know
31. 1:14how to think about that information per
32. 1:17se. It's not in the main model, so it's
33. 1:20simply going out getting that
34. 1:21information and bringing that in to the
35. 1:24context window and then goes out to a
36. 1:27model.
37. 1:28So it's a big distinction between uh,
38. 1:31when an AI seems to know something
39. 1:33that's that's past its its date that it
40. 1:36was made by going out to the web. We've
41. 1:39talked about multimodal, which can do
42. 1:41text and visuals and data. And we've
43. 1:44talked a lot about engetic models and
44. 1:46the issues with them.
45. 1:47So again with engetic models, these are
46. 1:50models that can call um,
47. 1:53different tasks, external tools like
48. 1:56going to your Gmail or or your
49. 1:59Apple Mail or um going to your calendar
50. 2:03or more significantly going to other
51. 2:06agents, even creating other agents as
52. 2:09we've seen with that recent leak.
53. 2:13So, we now know that they can go out to
54. 2:15files and emails. Uh we spent some time
55. 2:18talking about are you comfortable with
56. 2:20this? There's systems that do this more
57. 2:22and more that companies are probably
58. 2:24very interested for that to happen, but
59. 2:27these AI to AI chains have a huge risk.
60. 2:30And when I first made this slide, it was
61. 2:33only a risk of runaway behavior. Now, in
62. 2:37in fall 2026, we know it's much more
63. 2:41than a risk.
64. 2:43Um
65. 2:44with that in mind, I told you about
66. 2:46these these possible consumer devices
67. 2:49that will come out that work in this
68. 2:51agent way that
69. 2:52that listen to everything you say, to
70. 2:55remind you of things, maybe taking
71. 2:57pictures of everything. Amazon, OpenAI,
72. 3:00Apple. Um
73. 3:02what are the privacy stakes here and
74. 3:05would you want this? So, this all fits
75. 3:07in this very new world about uh using
76. 3:11agent technology.
77. 3:13In my research lab, we have a completely
78. 3:16different ethical AI and some other
79. 3:18things and we're slightly opposed to
80. 3:22um
81. 3:24agent technology in particular ways.
82. 3:27So, and I'll talk more about that. Um
83. 3:30so, I've also explained what a neural
84. 3:32net
85. 3:33is for you that that you know that we're
86. 3:36emulating the neural nets in our brain
87. 3:40uh more statistically
88. 3:42um and that everything that AI does at
89. 3:45this kind of level, whether it's text,
90. 3:47images, sounds,
91. 3:49um are put into some kind of vector
92. 3:51number space that goes into the neural
93. 3:54net to get a different one out and
94. 3:56that's what gives us the speed and the
95. 3:57ability for GPUs to do this quite
96. 4:00quickly. These nets got deeper and we
97. 4:02tend to call that deep learning. That's
98. 4:04where the the current revolution
99. 4:06happened and this is what we mean by
100. 4:08these different um
101. 4:11neural networks at different layers and
102. 4:14then they build on each other. So, we
103. 4:16only change the weights. Sometimes we
104. 4:19give out the weights publicly.
105. 4:21Uh um
106. 4:22and the lower weights um
107. 4:24are more kind of low-level information
108. 4:27and from what's there, what's connected
109. 4:29gives you the higher and higher and
110. 4:31that's how we we build this up from
111. 4:34edges to shapes to faces in this case,
112. 4:36but from word pieces to phrases to
113. 4:38meaning and style.
114. 4:40Um
115. 4:41by putting in all that information
116. 4:44uh uh we could we could see how well um
117. 4:49the network behaves on training data. It
118. 4:52there's an error rate about how good it
119. 4:54does and then we could back propagate
120. 4:56through the system to keep changing the
121. 4:58weight and we do this kind of building,
122. 5:01you know, millions of times to billions
123. 5:03of parameters and then later we clean
124. 5:05it. Um so, that's what would give us our
125. 5:08model.
126. 5:10Uh by doing all that testing and
127. 5:12inference is then once the model is made
128. 5:15and cleaned um then you you are using it
129. 5:20via
130. 5:21um
131. 5:22a chatbot like Claude or ChatGPT.
132. 5:26Um
133. 5:28I will get into more details in other
134. 5:31classes, but surely it's not that simple
135. 5:34and we're now trying to get meaning into
136. 5:37words. So, when we say cat, it's the you
137. 5:41know, the word left is it just left and
138. 5:43right? Is that cat left something?
139. 5:46That's quite different. Uh um litter
140. 5:48means two different things here. So, we
141. 5:50do spend a lot of time trying to
142. 5:52understand what we call attention and
143. 5:54awareness of words. That really made the
144. 5:57system so much better. And after
145. 5:59training, of course,
146. 6:01there's post-training. So, this is
147. 6:04almost like went from GPT to ChatGPT.
148. 6:08That we started asking humans to look at
149. 6:10what we had and going in there and
150. 6:13cleaning it. And what would be an ideal
151. 6:16answer and the model that imitates that.
152. 6:18We did so much of that that we were able
153. 6:21to start learning from all that human
154. 6:23feedback. And that's what we call
155. 6:25reinforcement learning by human
156. 6:27feedback. And that it went into ranking
157. 6:30and maybe you've seen sometimes where AI
158. 6:32gives you two choices.
159. 6:35So, it's actually still doing this
160. 6:36reward model of what's the best answer.
161. 6:39This really changed AI from or large
162. 6:43language models from something that just
163. 6:45gives you the next word, so it's good
164. 6:46for writing poetry and ideas to much
165. 6:50more significant notions of how we use
166. 6:52it. So, there's other things we do in
167. 6:55the training. Surely, Anthropic has a
168. 6:58constitutional method of how you're it's
169. 7:00supposed to think. There's other kinds
170. 7:02of reasoning systems that are starting
171. 7:04to come in. So, we're starting to hit
172. 7:07some level of meaning space rather than
173. 7:10just word space.
174. 7:12But whose preferences does it learn?
175. 7:15Who comes up with these values?
176. 7:18It's got this approval-seeking issue
177. 7:20where it's always trying to
178. 7:23be nice to you, so to speak. And this
179. 7:25reward hacking is a big issue that we've
180. 7:28talked about.
181. 7:29So, I want to remind us that this isn't,
182. 7:32you know, that within the world of AI
183. 7:33there's many kinds of AI. Large language
184. 7:36models are just one of them.
185. 7:38A lot of people talk about AGI, but we
186. 7:41usually cut the world into weak or
187. 7:43narrow AI or strong AI or AGI. Weak or
188. 7:47narrow is domain specific. It's somebody
189. 7:49who's just teaching it chess, and
190. 7:51they're a master chess player, so
191. 7:53they're training it well. Or music or so
192. 7:55and so.
193. 7:57Um and um
194. 7:59therefore, that's what all AI generally
195. 8:01is today. So, what about large language
196. 8:04models? Well, large language models are
197. 8:07really domain specific, too. They're
198. 8:09trained on one area, text. But, text is
199. 8:12unusual compared to like chess or music
200. 8:15because text is a special domain. Words
201. 8:18carry meaning in a way and we that we
202. 8:20used in our book. So, it is interesting.
203. 8:23It's our past knowledge, what humans
204. 8:25wrote in books and on the internet. And
205. 8:28because we wrote it with meaning,
206. 8:29sometimes surely early AI didn't even
207. 8:33understand the words. We're just passing
208. 8:34it on to us, and it made sense to us
209. 8:37because it was written by um us
210. 8:40historically.
211. 8:41So, um strong AI or AGI then is this
212. 8:45human-level ability to to learn and
213. 8:48reason and transfer tasks. And we're
214. 8:50definitely not there yet, even though
215. 8:52the companies and
216. 8:54people quoting the stuff will push AGI.
217. 8:57Some people are starting to use this
218. 8:58super intelligence term, ASI or just SI,
219. 9:02which means it's going to be far beyond
220. 9:04best humans. It's funny, we didn't even
221. 9:07reach AGI and people are now talking
222. 9:09about ASI. And as you know,
223. 9:13Trump in the White House,
224. 9:15you know, went out of the way to say we
225. 9:16no longer call it AI. We just want to
226. 9:19call it super intelligence or SI. And he
227. 9:22kind of went out of his way to say that
228. 9:24if you don't, you're
229. 9:25you're an enemy or something like that.
230. 9:27Even though he doesn't realize that
231. 9:29there's hundreds of papers and
232. 9:30conferences with the name AI. It is an
233. 9:33actual field of science. So, that gives
234. 9:36you an idea of, you know, even though
235. 9:38you hear that a large language model
236. 9:40might hit AGI,
237. 9:42I personally don't believe so. I think
238. 9:44it's going to be the amazing front end
239. 9:46of a system once you have reasoning and
240. 9:48planning and meaning because it's very
241. 9:51good at conversation and holding meaning
242. 9:54for some length of time, but kind of not
243. 9:56what do I do next? How do I think
244. 9:58differently?
245. 10:01This meaning, we'll talk about this a
246. 10:02little bit more later, but we are moving
247. 10:06better than I just said, past just well,
248. 10:09it doesn't really know what the words
249. 10:11mean. Some of that is because the
250. 10:12multimodal, we have a text and image
251. 10:15pair. So a text one would be a pup in
252. 10:19the grass and it would be an image of
253. 10:20that. And you could see in our embed
254. 10:24chart here, here's text one and image
255. 10:26one. So this gets a perfect score, let's
256. 10:29out of from zero to one
257. 10:32um in float space, it would get a one.
258. 10:34It's a perfect answer
259. 10:36um because it literally is an image of a
260. 10:38dog a pup in the grass.
261. 10:41Um and then we would have another one, a
262. 10:43cat jumping up and then we have an image
263. 10:45of that cat jumping up. And that would
264. 10:47be T2, so that would be here and here.
265. 10:49Again, perfect ones on the diagonal, but
266. 10:52then when you're saying, well, what
267. 10:53happens if I gave you two two, a cat
268. 10:56jumping up, but I one um
269. 11:01which is which is the dog. So it would
270. 11:03do better than other things and with
271. 11:06that and we run these embedding loops
272. 11:10over and over again with some of the
273. 11:11other stuff I just showed you and then
274. 11:13we're able to actually have a kind of a
275. 11:15reasoning nearness map that dogs and
276. 11:17cats seem to be based on our work here,
277. 11:22closer than buildings and violins and
278. 11:25guitars. So this is where we're starting
279. 11:27to move towards meaning. Um I'll have a
280. 11:30lot more to say about that in in other
281. 11:34um um lectures, but I just want to do
282. 11:36the review here. So, on the large
283. 11:39language model side, I want to remind
284. 11:41you that it isn't just your prompt. We
285. 11:45talked about this 2 weeks ago, that it's
286. 11:47actually this is system prompt. What
287. 11:49it's
288. 11:50the instructions from if you're using a
289. 11:53company tool from what the company would
290. 11:56want to answer [clears throat] and how
291. 11:57it answers and some other kinds of
292. 11:59information.
293. 12:00We do our own open-source ones, so we
294. 12:03can write the system prompt. It's
295. 12:05amazing how different the AI can be just
296. 12:08from changing the system prompt of how
297. 12:10it should act. Surely, there's a
298. 12:12conversation history. Um so, it knows
299. 12:15what you've talked about before, and
300. 12:18there's the rags that we talked about,
301. 12:20which is um uploading um documents in
302. 12:25So, um the model never changes here.
303. 12:29It's still the same way. It's It's just
304. 12:31It's just a lot of what makes us an AI
305. 12:33sophisticated is loading up what you
306. 12:36think is just your prompt, but it's all
307. 12:38these other things before it goes to the
308. 12:40model and then comes back to you.
309. 12:43So, that's the system prompt, which I
310. 12:45just talked about. Um your prompt. Um
311. 12:49the rag, which is these
312. 12:51ways to get the data in through this
313. 12:53chunking technique I've talked to you
314. 12:54about, but all and the conversational
315. 12:56history, but now it actually keeps a
316. 12:59little bit of memory of you as well. And
317. 13:02and then there's the tool side. The tool
318. 13:04side for go could be for going out to
319. 13:06the web or doing all this incentive
320. 13:08work. So, all this happens just in the
321. 13:11context window. So, this is not the AI
322. 13:13model. This is getting ready to build a
323. 13:16context window to then send it to the
324. 13:18model. Uh we we do pre-put in the
325. 13:21documents. So, we put in a prior
326. 13:23document, we do send that. So, this is
327. 13:25the rag system here that when you're
328. 13:28putting in a document, it's doing some
329. 13:30pre-processing.
330. 13:31Um soon we're going to be talking about
331. 13:34other kinds of systems, so visual
332. 13:36systems that could be similar in that I
333. 13:40could be putting in a video of a dancer
334. 13:43and then some knowledge or image of a
335. 13:45tree and I would put that through the
336. 13:46system and I would have some kind of
337. 13:51visual generation system that outputs um
338. 13:55in this case a
339. 13:57a moving dancer
340. 13:58uh that looks like a tree.
341. 14:02Um so, again with the rags um and it's
342. 14:05really important for some of your work,
343. 14:08it's always better not to dump a lot of
344. 14:11data in the context window. It's better
345. 14:15to have a document and put the document
346. 14:18in because when you put the document in,
347. 14:20we chop it into chunks.
348. 14:23We send that to a small AI system and we
349. 14:26get nearness and a chunk is about a
350. 14:28paragraph. Hopefully, you know, a
351. 14:30paragraph has one meaning and then we
352. 14:32can we can get, you know, all those
353. 14:35chunks, maybe thousands of them
354. 14:37uh set up in this small vector database.
355. 14:40So, you put in a PDF and before we
356. 14:44start,
357. 14:45well, when you put it in, we we separate
358. 14:48that into chunks. It goes to a small AI
359. 14:51to get the embeddings. We put that in a
360. 14:53very fast database. So, then after
361. 14:56that's in and you go in and and prompt,
362. 15:00do the regular prompting, it's going to
363. 15:03quickly access back to the database,
364. 15:05"Hey, given this question, could you
365. 15:08quickly go through the meaning space of
366. 15:10all these thousands of chunks and maybe
367. 15:13give me the four best ones um that would
368. 15:16help this
369. 15:18um context window be smarter when it
370. 15:20goes. And that's that's what a rag is
371. 15:23and how we use it. So, all those things
372. 15:25come together when you're using a large
373. 15:28language model, and it's good to be
374. 15:29thinking about all those kinds of
375. 15:30things. So, we've talked about the
376. 15:32chunking system. So, again, I just this
377. 15:35is a little bit of review, almost done,
378. 15:37but LLMs
379. 15:39um you know, have this pre-trained
380. 15:41knowledge. However, they have a cutoff
381. 15:43date, too. It takes It used to take a
382. 15:46year, and you could know in GPT-2 and
383. 15:49then three and then four. We're going a
384. 15:51little faster, but still it takes a very
385. 15:54long time to do all that pre-training
386. 15:57and then cleaning it for another month
387. 15:59and then testing it for another month.
388. 16:01So, the knowledge you're you're you
389. 16:04know, there's a cutoff date. It's
390. 16:05usually about a year behind. If you ask
391. 16:07it something before then, it um sooner
392. 16:10than then uh sorry, later than then, it
393. 16:13won't know about it. This is why rags
394. 16:15are good, because companies use and
395. 16:18people use rags, so they're there to
396. 16:20give newer or very specific data, and of
397. 16:23course going on out on the web.
398. 16:26So, between going to the web and your
399. 16:28documents, you're really making your
400. 16:30your system a lot better. So, this
401. 16:32context window has limits, and this is
402. 16:36why we try and use rags. And this lost
403. 16:39in the middle problem is with longer
404. 16:41prompts, it really does get confused.
405. 16:44This is just within one prompt. It's
406. 16:46true in the thread, as well, too. We
407. 16:48know that LLMs
408. 16:50hallucinate, and some of this is because
409. 16:52of these kinds of reasons. So, we've
410. 16:54talked about a rag here, and I think I'm
411. 16:56going to stop at this point, and uh I'll
412. 16:59have a different lecture on ethics.
413. 17:01Thanks.