YT YouTube2Text

YouTube2Text → Transcripts → Session 1 - How To Start (Almost) Any Project

Session 1 - How To Start (Almost) Any Project — Transcript

by Underfitted · 27,487 words · 3,927 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:04Hello. Hello. Can you guys hear me?
0:07Okay.
0:14Let me make sure.
0:24Can anybody say something to see if my
0:27audio is working?
0:40Let me see.
0:44Actually go here.
1:14Hello everyone.
1:19Sorry, for some reason the meeting here
1:23is not configured correctly. So I have
1:26to admit every single one of you one by
1:29one.
1:32So weird.
1:37Let me make sure that doesn't happen
1:39next time.
1:43Hey,
1:47>> morning everyone.
1:49>> Good [clears throat] morning.
1:51Hey folks.
2:00What's the thing on your arm, Santiago?
2:03Are you getting like notes of your
2:05vitals during
2:08>> Christmas?
2:10Yeah, this is a whoop. I don't know if
2:12you've seen them.
2:13>> No,
2:14>> but I always wear them in my bicep. It's
2:18a little bit uh more accurate. You can
2:21also wear them here like just a regular
2:24wristband,
2:25>> but in my bicep is is a little bit
2:27better. And when I'm wearing like short
2:30sleeves, it tail
2:31>> people ask questions. Yep.
2:33>> They usually pick Yeah. Everyone has
2:36asked me and some people think is some
2:38kind of biohacking
2:41where they cut circulation to the
2:44muscle. So when they're working out, I
2:47don't know, stimulates the muscle. No,
2:49nothing like that. It's just just a
2:51regular whoop count my steps and does
2:55you know stress monitoring things like
2:58that.
3:01Sorry, I'm I'm updating here really
3:03quick the meetings because for some
3:06reason when I configured this
3:11I have to manually accept every one of
3:14you admit everyone into the meeting
3:17which is a pain in the neck because I'm
3:19talking and people are trying to join
3:22and I have to just click admit admit
3:24admit
3:26uh and yeah I don't think there is a way
3:28to change that after the meeting is
3:30going.
3:33Let me see settings.
3:35Can I just say everyone please join?
3:40Let me see. Reactions, video, audio,
3:44general.
3:48No, I don't think there is a way to do
3:50that.
3:53Uh, yeah, I don't think there is a way
3:57to do that.
3:58At least I don't see it here. Let me let
4:01me see
4:03settings.
4:06I apologize about this. I should have
4:08known this before.
4:14>> It's just painful.
4:17Santgo
4:19al quite a few names sounding like south
4:23of Europe, Spain, maybe Italy, Portugal,
4:27right?
4:28>> Yeah.
4:30>> Did I get it right?
4:32>> Yeah, it's uh usually I get people from
4:36all over the world. I've had people from
4:40Asia here.
4:43uh they don't go to sleep and they just
4:45take the class which is just crazy to
4:47me.
4:48>> Uh obviously all from Europe especially
4:51at this time I used to teach this class
4:55at this time and then later in the day
4:57but this time it's perfect from people
5:00in in Europe uh Latin America you know
5:04Mexico uh Honduras Argentina all of
5:08those countries are really popular. the
5:10class is really popular there. Uh
5:13anyway, it's uh yeah, well, first of
5:16all, welcome everyone. I'm sorry I'm a
5:20little bit still settling. I usually
5:22spend my summers outside the United
5:24States. Uh we like to spend my uh our
5:27summers in Europe. And I just came back
5:31two days ago, so I'm still settling in
5:34after three months out. Uh we went to
5:36Japan and we went all over the world.
5:38It's super cool. But anyway, I'm back
5:41and this is uh I think I mentioned this
5:43already on this course. This is going to
5:45be my last time teaching this. Uh what
5:48that means is that you guys are going to
5:50still obviously uh have access to all of
5:53the materials. I'm not taking that down.
5:57Uh I'm just not going to be running the
5:59classes anymore. Uh the reason I'm not
6:01running the classes anymore, um it's
6:04it's two reasons actually. Uh number
6:07one, I want to do something different.
6:09I've been doing this for three years
6:10now. So, I think it's time for me to
6:12just, you know, do something different.
6:15Don't have the class in my the back of
6:17my mind. Oh, in a month I have to fix
6:19this because the class is coming. I want
6:21to just dedicate my time to something
6:23else. And number two is a general trend
6:26that is not only happening with this
6:28class, but it's it's all over the place.
6:31Like people, they just don't want to
6:33learn anymore. It's anything that's
6:35educationally
6:37uh oriented. It's it's just huge dips.
6:41Uh just to to give you some perspective,
6:45I've been teaching this for three years.
6:47And for two of those years, I think
6:52every single session of this class,
6:54every single cohort had about 200 people
6:58joining every class. That's just to give
7:01you an idea of how popular it was. Now
7:06we have right now and this is going to
7:08be the class with the most amount of
7:09people. So this is going to go down from
7:11here. Always happens. We have 34 people.
7:14So that gives you an idea of just the
7:17tip. Uh I know a lot of other people who
7:21also teach something and everyone is
7:24seeing the same thing across. uh like
7:27people are not going to classes anymore.
7:30They're just asking Chad GPT to do stuff
7:32for them. So I understand I understand
7:34why why that's happening. So anyway,
7:36perfect opportunity for me to just
7:37dedicate my time to something else. So I
7:40decided to teach this one for the last
7:42time and have fun with it. Uh so what
7:45are we going to be or let me just give
7:47you a little bit of background and a
7:49little bit of a how this class is going
7:51to go. So, we're meeting three times per
7:54week for three weeks. Two of those times
7:58are sessions, meaning I'm just going to
8:00be showing you slides and talking. Uh
8:04the other which is going to be on
8:06Wednesdays, that's what I call office
8:08hours. That is just time for whoever
8:12wants to join join and we're going to
8:15talk about anything.
8:18We can talk like literally anything. It
8:20doesn't have to be related to the class.
8:22It's it's whatever. I usually usually
8:24that time we talk about what's going on
8:27in the world of of development and AI
8:30and how to make money online and how to
8:33build a business that where you can sell
8:35software. Things like that are usually
8:38the topics that come up during office
8:40hours. So, they're kind of fun. Okay?
8:42But you don't have to be here. It's it's
8:44okay.
8:46During the sessions, uh I'm there are
8:50six sessions. I'm going to show you
8:52slides in four of those sessions. Okay?
8:55So, sessions one through four, I'm going
8:58to have slides. I'm going to go through
9:00the material with everything that I want
9:02to teach you and we can talk about it.
9:05You can ask questions, etc. For the last
9:07two sessions, starting with the last
9:10cohort, I stopped showing slides and
9:13instead in session five, I'm going to
9:16talk about code. I'm going to show you
9:18some code, some specific agents that
9:21that I built for this class. So, we can
9:24talk about how the agents work and what
9:27I built and you can ask questions about
9:29the code, etc. So, the whole session
9:31five is going to be looking at the code.
9:34The whole session six is going to be
9:36talking about agentic coding. No
9:39sessions. So basically the goal of
9:42session six and session six I'm not
9:44going to be the teacher. I'm going to
9:46tell you what I know about agentic
9:48coding but me like all of you we're all
9:52learning here. Okay. So I hope that
9:54during session six you guys can
9:57contribute as well and teach me and
10:01everyone else in the class what are you
10:03guys doing with your codeex or clock
10:06code or whatever you're using because
10:08we're all we pretty much have the same
10:10experience all of us right there's
10:12nobody with uh with a decade of
10:15experience in agentic coding yet so
10:17that's the goal of this class okay
10:21bottom line is I hope that after three
10:24weeks you guys go out there with a few
10:28new things. So, number one, hopefully
10:30some experience
10:32gained by listening to me because I made
10:35the mistakes before. I'm going to show
10:36you those mistakes and hopefully you get
10:38some ideas off of those mistakes so you
10:40don't make those mistakes. Number two,
10:42maybe you're going to learn a few
10:43techniques that you're going to be able
10:44to apply later on on your job. And
10:48number three, and this is the most
10:50important thing for me, uh I want you
10:53guys to be motivated, more motivated
10:55than you were today. So hopefully you go
10:58out there, you're motivated to make a
11:01difference, uh to make money. That's
11:04what's about to be honest with you. I'm
11:06very practical person. I don't think
11:08people do this because this is the love
11:10of my life. All of us, this is a
11:12business. We have families. So hopefully
11:14we give you some ideas on how to make a
11:16little bit more money. uh with this
11:18knowledge. Um with all of that being
11:21said, any questions before I show you
11:23slides.
11:33Okay, let me let me open Google Drive
11:36because
11:38maybe me I don't have that in front of
11:41me now obviously. So,
11:43>> so Santiago, maybe not a question, but a
11:45ch ch ch ch ch ch ch ch ch ch ch ch ch
11:46ch ch ch ch ch ch ch ch ch ch ch ch ch
11:46ch ch ch ch ch ch ch ch ch ch ch ch ch
11:46ch challenge or a favor if you spend a
11:48moment at the end of you know the cohort
11:51tell us a little bit more about your
11:53plans. This would be entertaining
11:56or like
11:56>> okay what are my plans uh going forward?
12:01>> Uh so yeah so the the short answer to
12:04that is I just don't know yet. Okay. So,
12:08it's I I don't have anything settled. Uh
12:12so, I I'm I'm lucky. I don't I I don't
12:16have a job. My job is is is just doing
12:19this and and working with different
12:20companies, reviewing things and
12:23consulting for them. So, that's what my
12:24job is. That's my my own business. So, I
12:28don't need from the money point of view,
12:32I don't need to replace the cohort. So,
12:34I'm not in a rush. Hey, I just need to
12:36do something else because the cohort I'm
12:38not going to be teaching it anymore.
12:40That's not that's not my my
12:43that's not a problem for me. What I'm
12:46going to do, I want to do something
12:48obviously. I just don't know what that
12:51is going to be. talking to students and
12:54talking to people that I know. A lot of
12:57people have suggested that I do
12:59something that I've been doing for a
13:01long time which is uh just basically how
13:06to create a business where you can be
13:10your own thing is you know lifestyle
13:13business let me call it like that. Uh
13:15not a business to raise money on a big
13:18company or anything like that. That's
13:19not my thing. That's not who I am. But
13:22mostly something that you can do, live
13:24off of it. Um, live very well. Um, don't
13:27worry about anything else. Some people
13:29have talked to me about doing that. I
13:32have some advice there based based on
13:35what I've learned.
13:37The problem for that is that I find that
13:40yucky. I don't like to be talking to
13:44people how to make money because that's
13:48nine out of 10 people that do that.
13:50They're just trying to scam you out of
13:52money. I don't want to be that person.
13:54Uh so I don't know. I don't know yet
13:56what that's gonna look like. We'll see.
13:58We'll see. Uh but yeah, not sure.
14:02Mansour, what's up?
14:04>> Yes. Hi, Santgo. How are you? Hello
14:06everyone. Um yeah, thank you. Thank you
14:08very much. Um just just one question. I
14:11joined a long time ago. I think it was
14:12at the very beginning of um when you
14:14created I'm joining by from Sagal by the
14:16way, West Africa. So yeah, perfect time
14:19for us. Um so um I don't know a long
14:22time ago and I know in between um the
14:26content has changed a lot I think.
14:28>> Yeah.
14:28>> And I had maybe two questions, right? So
14:32it was about machine learning at the
14:34beginning. Is it really still about
14:36machine learning? Second one, do you
14:38suggest that we also go back to the
14:41previous material that you had somehow
14:43or will this one be like self-contained
14:46and no need to actually go back in time
14:48and look at some of them? What would be
14:50your advice based on on that? Thank you
14:52very much.
14:52>> Yeah. So, uh, great questions. Let me
14:54just share with me here the slides. Just
14:57one second. I'm going to answer those.
15:01>> All right. So, great question. So when I
15:03started this cohort, I started teaching
15:06uh
15:08same principles, same ideas,
15:11uh a lot of the same examples that
15:13you're going to see during the class,
15:15but the code portion of the cohort was
15:18100% focused on building everything
15:21using Sage Maker, AWS SageMaker. So I
15:25was using AWS APIs, AWS uh SDK,
15:30everything was focused 100% on
15:32SageMaker. Over time I changed that and
15:35I migrated the whole codebase to
15:39opensource tools because there were a
15:41lot of people that they came to my class
15:44but they were Ashure users or Google
15:48Cloud Platform users. They did not care
15:50about AWS. So the class was still still
15:53helpful but they couldn't take advantage
15:55of the code by migrating everything over
15:59open-source tools. Now everyone had
16:02access to the core to the code and you
16:04know they were able to take advantage of
16:06it. That was the big big change that
16:09happened after that.
16:12uh little by little I've incorporated
16:16new ideas that have come out since then
16:19like AI more specifically and LLMs and
16:23how to uh we went we spent time at some
16:26point talking about training LLMs and
16:29now we're not talking about that we're
16:31more about in agents and stuff like that
16:34those are new concepts that have
16:36happened since the beginning of the
16:38class I've introduced those here they're
16:42are useful material
16:44back in the previous cohorts that I've
16:48removed just because I had to add new
16:52materials. But it's hard for me to point
16:54you to, hey, just go to cohort 15 or
16:57cohort 14 because the transition has
17:00been gradual and it's it's it's just
17:03really hard for me to tell you when we
17:04talked about what. But bottom line, you
17:09don't need to go back. This is not not
17:13that different from previous cohorts.
17:15Again, there might be one topic that is
17:17not here that was before, but not need
17:20to go back uh and watch anything. Okay.
17:26Okay. I think I think I'm almost ready
17:30to show you the slides, guys.
17:35I honestly thought that I had this
17:37yesterday figured out. Uh, obviously I
17:39didn't.
17:42Okay, here we go.
17:46Here we go.
17:52Let me share my screen.
17:59Jesus Christ.
18:07You guys might be okay.
18:11I think there might be people wanting to
18:13come in. Yeah, there you go. You guys
18:16see my screen?
18:19All good?
18:20>> Yep.
18:21>> All right. So, yeah.
18:25All right. So the first session I
18:27usually try to talk about
18:30how do you approach a new project. Okay.
18:33Um the reason I spend time talking about
18:35this is and again you guys have to
18:39remember I'm going to have a bias
18:41towards
18:42me working with a client and me being
18:45responsible of everything that happens
18:48from that point on. Okay. Some of you, I
18:52assume some of you are are part of a
18:54team where in your team you're going to
18:56have people dedicated to all different
18:59phases, right? From selling, from
19:02marketing, project management, etc.,
19:04etc. When you're working solo, for the
19:07most part, you're doing all of those
19:09roles somehow, right? So, this is going
19:11to cover some of those ideas, right?
19:14From the beginning until the end until
19:16you you have a project moving on. That's
19:19how I organized my session. So session
19:21number one is kind of focused on hey how
19:24do we go from from from nothing to how
19:27do we start pretty much. So let me start
19:30with uh
19:33one of the big fuckups that I I made uh
19:36because I know uh that you guys have
19:39seen this before. So this was me uh this
19:42is a client uh I'm working as part of a
19:44team. This is a client. Uh they want us
19:46to we were working with the robot from
19:49Boston Dynamics. Uh if you've seen it,
19:52it's the yellow robot. It's called Spot.
19:56And we were building computer vision,
20:00different computer vision solutions for
20:02that robot. So the robot could get some
20:05skills. Um a client wanted us to, hey,
20:09how about we do inventory with uh the
20:12robot. So the robot basically will walk
20:14around the aisles. We'll take pictures
20:17of the boxes and we'll count the number
20:21of boxes and you know we will be able to
20:24have an idea of what's in there.
20:26The problem was uh oh by the way I said
20:29of course um the solution I built is
20:31what you see here on the left and I
20:34don't know if you can see this isn't
20:35very subtle but I on purpose I sort of
20:39like misplaced
20:41this bounding box here this uh if you
20:44see my screen I don't know can you guys
20:46see my mouse by the way
20:49I don't know if you can see it or not
20:51but anyway uh okay so if you do you can
20:55see here there is a like a you know
20:58green
20:59bounding box that's incorrectly set. Uh
21:03I did not represent it here well but
21:05there might be boxes that you cannot see
21:08just to to go straight to the point. It
21:11was a stupid solution. So I tried to
21:14solve something where there is a better
21:16solution. It's been invented for years
21:18now. And I tried to do something that
21:21obviously did not work. We had
21:23unreliable counts of boxes. Uh the robot
21:27could not see, could not work past
21:29occlusions. Like if there is a box
21:31behind another box, it was really really
21:34hard. Uh the solution obviously is just
21:36to follow protocols and use barcodes.
21:39That has been implemented forever. This
21:42was a failure. this is my failure for
21:45not, you know, telling the client, dude,
21:48you're just trying to invent the, you
21:51know, you're trying to invent something
21:53that already exists. So,
21:56here's the thing. Most projects fail
22:00because we're trying to solve the wrong
22:02thing. Okay? So, you need to start every
22:05project or my advice to you is to start
22:07every project with a discovery phase.
22:10Okay? That's what I propose to every
22:12single client I get. Uh somebody calls
22:15me, they say, "I want to work with you.
22:18I have a million dollars to build this
22:20thing. That's great. That's amazing. Uh
22:23can you tell me how long it's going to
22:24take?" And I'm going to answer to them.
22:28I just don't know. I need four to six
22:30weeks for a discovery phase. That's how
22:34I start every project. That's the only
22:36thing I ask the client to commit to.
22:39four to six weeks so we can figure out
22:44what we're doing and how we're doing it
22:46and how much is it going to cost. Okay,
22:49one of the biggest problems in software
22:51development, some of you I assume are
22:53software developers, is estimating
22:56things. How long is it going to take
22:57something? How much is going to cost uh
23:00to build it? The answer to all of those
23:02questions is I just don't know. I need
23:06time to figure those out. That's what
23:08the discovery phase will take care of.
23:11Okay, discovery phase is just a time
23:14box, you know, phase. I usually like to
23:17do four to six weeks. It could be
23:19longer. It could be shorter depending on
23:21the complexity that I think the project
23:24will have. But that's the goal there.
23:26You learn about the problem. You frame
23:29the problem. We're going to talk about
23:30problem framing in just a second. and
23:32you prove you can actually solve the
23:36problem. Especially with AI and machine
23:38learning, there are a lot of problems
23:41where you might not be able to solve
23:44because you don't have the data. You
23:45don't have way to capture the good data,
23:47the data that you need to solve that
23:49problem etc etc. So before you commit to
23:53anything you go through this discovery
23:55phase and that will reduce the risk.
23:57Okay. It's also very easy to convince a
24:01client to sign up for six week weeks of
24:05work than to ask the client, I'm gonna
24:09need a million dollars and one year,
24:12right? Just to solve this. Very easy the
24:14first, very hard to get that signature
24:17when you're asking for a lot of money, a
24:18lot of time, uh, under a lot of
24:20uncertainty. So, that's what the
24:23discovery phase is. Here are a bunch of
24:26questions. I'm not going to go through
24:27all of them, but you have the slides or
24:29you're going to have the slides after
24:30the class and you can just read the
24:33questions. But these are some of the
24:34questions that I use during that
24:36discovery phase. Okay. So, for example,
24:39what is the problem that we're trying to
24:41solve? What are the problems that we are
24:44going to ignore? Very important. It's
24:46not only about the things that you want
24:48to build, but the things that you are
24:50not going to touch while building that
24:53because you know the road to the
24:56solution that you're looking for is
24:59going to be full of rabbit holes. And if
25:03you're not careful spelling them out,
25:05it's very easy to go off track trying to
25:09build features and little things that
25:13you might might not be that important
25:15for your solution. Who's the customer?
25:18This is the last one I'm going to
25:19explain
25:20because it's one of the most important
25:22ones. When you're working with a
25:24company, you're usually talking to the
25:28person with the money, not to the person
25:31who needs the solution the most. What I
25:35mean by that is uh I work for a company,
25:37they want to build a new inventory
25:40system. I'm talking to the CTO of the
25:43company. The CTO is the person who's
25:45responsible for investing the money into
25:49building that solution. They're not the
25:51customer. they are not going to be
25:53running inventory or accounting or
25:55whatever the system is about. You need
25:57to identify who the real customer is.
26:00One of the most common mistakes is
26:02people trying to please the money person
26:07instead of pleasing the customer. What
26:11happens is there's going to be a
26:12mismatch and the customers, the people
26:15who have to adopt your tool will hate
26:18it.
26:20It doesn't matter that the boss is happy
26:22if the people who have to adopt the tool
26:24will hate it uh hate it because those
26:27are the ones that are going to rubber
26:29stamp your work. Okay? So you have to uh
26:33investigate or you know determine who
26:36the real customer is. bunch of other
26:38questions here. Read them later. But the
26:41goal is all of these questions are the
26:44ones that I'm going to be answering
26:47during the discovery phase.
26:50Have any of you, by the way, have any of
26:53you
26:55done this before when working on a new
26:58project? Have any of you have had that
27:00buffer at the beginning of it? that does
27:03not mean that you are uh you're still
27:07not uh committed to complete the
27:10project. You're just going through a
27:11discovery phase. Is this something that
27:12you guys have done? Yes or no?
27:18>> Yes. Yes. Yes. Of course.
27:20>> Santiago, quick question. What's the
27:22protocol? You have material for like
27:24entire one and a half an hour or you're
27:27happy to take questions during?
27:29>> I'm happy to take questions whenever
27:30whenever you have questions. just just
27:32raise your hand and uh
27:36>> okay
27:36>> I have I have a really quick one uh
27:39let's again out of curiosity what's your
27:42uh usual arrangement do you charge for
27:43the discovery phase separately without
27:46like going to details just basically
27:48like really quick question curious
27:51>> uh yeah so I do charge for the discovery
27:55phase I do not
27:58uh that doesn't mean that the client is
28:01is forced to to pay for the whole
28:03project.
28:04>> Yeah.
28:04>> I just don't get to the like basically I
28:07don't tell the client this is how much
28:09it's going to cost you the whole thing.
28:11I just say there are four weeks here. Uh
28:14you're going to pay me this much for the
28:16four weeks and part of the outputs of
28:19that those four weeks is going to be a
28:22plan and hopefully a budget for the rest
28:24of the project. Now, that being said,
28:27way I usually approach long projects is
28:30not it's going to cost you $500,000 for
28:34eight months. I don't do that. I ask for
28:37iterations. So, hey, this is the next
28:39iteration. It's going to be another
28:41month. This is how much you're going to
28:43pay me for a month. And these are the
28:46outputs that we expect to have after one
28:48month. You commit to one month and
28:51that's pretty much it. and after a month
28:54we reassess. That works well with many
28:57clients. With big companies, it doesn't
28:59work well for stupid reasons. Those
29:03reasons being approval, procurement for
29:06the money. So, it's really hard in a big
29:10company. It's usually hard for a manager
29:13to go multiple times to procurement to
29:17get paperwork done to approve funds for
29:20a project. So, it's way easier for them
29:23to go and ask for a million dollars than
29:26to go 10 times and ask for uh $100,000.
29:30You see what I'm saying? So, usually
29:33what happens is that uh what we do is we
29:38try to estimate how much it's going to
29:40be, but I'm only going to be charging
29:43the client for one iteration at a time.
29:46Okay? So the client understands that the
29:48big estimate is not a money that I'm
29:50asking or that they owe me. That's just
29:53for them to get approval, but then
29:55payments are going to be done per
29:57iterations and that's my only
29:59commitment. So anyway, that's that's a
30:00lot of uh insider baseball there, but
30:03that's the whole the whole idea. I
30:05charge for discovery. The output of
30:08discovery might be I I cannot build this
30:11for you. I don't have enough. You don't
30:13have the data. That's has happened
30:15multiple times. I specialize in computer
30:18vision. Many many companies hire me to
30:21do things and then I come back after
30:23four weeks and tell them you guys need
30:25images.
30:27To get images, let's say you have to fly
30:30drones.
30:31You don't even have a drone program that
30:34you can use to fly drones. I mean,
30:36flying drones is not is not sending Mary
30:39Jane with a drone and taking pictures.
30:41No, you need professionals and you need
30:44a way to fly those drones and fly them
30:46every week and recharge them and blah
30:48blah blah blah blah. You need a year to
30:51figure that out. You need to go and work
30:52with a company that does that for you.
30:54And after you have that in place and
30:56after you're capturing images, you come
30:58to me and I'm going to help you build
30:59the models. So that you know that's the
31:02output of this a discovery phase. if I
31:04wouldn't I mean if you don't do the
31:06discovery phase now you're stuck in a
31:08project you don't even know how to move
31:10forward with does that make sense
31:12>> yep makes perfect sense and I'm happy to
31:15hear that this works you know with
31:17businesses
31:19>> you know that there is this niche or
31:20like even like larger niche that when
31:23this is functional nice
31:25>> thanks it actually works uh it's in my
31:29experience it it's very refreshing to
31:32them to hear from somebody that's not
31:35asking them for eight months and giving
31:38them a promise that everyone knows at
31:41that table we cannot complete that prom.
31:43I mean whatever you're estimating that
31:46six, seven, eight month from now you're
31:49just it's just stupid like nobody knows
31:52that far you know far in advance how
31:54long things are going to take. So it's
31:56very refreshing to go let's go one
31:58iteration at a time. these are my
32:01commitments for one iteration and then
32:03we reassess and then we just put
32:06together a plan for the next iteration
32:08and it's it's it's like an employee will
32:11do with his boss, right? If you work for
32:14somebody, your boss is not asking you to
32:17commit to a year worth of work. That's
32:19not what your boss is telling you. Your
32:21boss is saying, "We're going to go one
32:23spring at a time, one week at a time,
32:25and if you stop working out, at least
32:27here in the United States, if you stop
32:29working out for me, I'm just going to
32:31fire you. Bye-bye. But as long as you
32:34bring value, I'm going to keep you in my
32:36team." That's the whole idea that we're
32:38trying to reproduce. Yeah.
32:41Okay. Cool. All right. So, problem
32:43framing. I said that one of the goals of
32:45discovery is framing the problem. And
32:48this is uh
32:51this is key. All right. Framing the
32:54problem. I mean you take a problem and
32:56you frame it in the incorrect way and
33:00you might be adding millions of dollars
33:03to to that project for no reason or you
33:06might be getting yourself into a hole
33:08that you might not be able to come out
33:09of it. So doing learning how to find the
33:13right framing for a problem is just
33:16going to be key. This is one of the most
33:17high lever uh tasks that you can
33:21perform. So let me give you one example
33:23here. Okay. Uh this is sort of like we
33:26try to to to draw here Tesla's you know
33:30infotainment system here but that
33:32self-driving car if you compare how
33:35Tesla and Whimo or Whimo however you
33:38pronounce it framed building full
33:41self-driving you're going to notice that
33:43they went very very different path.
33:46Okay, so Tesla decided we're going to
33:48put a crappy software out there, but
33:52we're going to focus on creating cars
33:54that many, many people buy. Okay? And
33:57over time, those people are going to be
34:00capturing more data. And with that data,
34:03we're going to be improving those
34:04models. So over time, our models are
34:07going to be really good. They focused on
34:11the commercial part of the equation.
34:12Whimo did something very, very
34:14different. Wimar released a full
34:17self-driving system from day one. It was
34:20not open to the public. It was
34:22restricted in the number of or in the
34:25roads that it could take. But that was
34:28it. They had an area, very small area
34:30with a full driving system. They did not
34:33care about selling cars. They only care
34:36about repetition and making that model
34:38better. And as it got better, they
34:41started expanding the area where their
34:44vehicles could drive. Very, very
34:48different approaches there. Now, you
34:51could argue that one of them is more
34:53successful than the other. I mean, it's
34:55depending on how you're looking at this,
34:56you're going to have an different
34:58opinion on which is the right framing.
35:01But you can hopefully clearly see that
35:04these are very, you know, very very
35:07different ways trying to solve the same
35:09thing which is full self drive. Okay. So
35:12that's the idea here. Now I like in
35:15order to frame my problems, I like to to
35:17use what I call the haystack principle.
35:20And I did not coin this term. Uh the
35:23first time I heard this was from
35:25somebody who was my boss. I learned a
35:27lot from him and he used to call it the
35:30haystack principle. So the whole idea
35:33here is instead of trying to find a
35:36needle in a hay stack, you're going to
35:39focus on trimming the hay stack until
35:43the need needle is right there. Does
35:46that make sense? So instead of focusing
35:48on the problem that everyone cares
35:51about, you're going to focus on
35:52everything else. get rid of everything
35:54else until the problem is right there.
35:57I'm going to show you uh couple of
36:00examples of how that of what that looks
36:02like. So, this is a project that we
36:05worked with Disney Disney World. So, I I
36:08don't know if you guys have been to
36:10Disney, but [clears throat] they have a
36:11problem with people uh getting into
36:14their parks without paying.
36:17People just go in. Now, their policy at
36:21Disney is not to be confrontational.
36:26So, even if they
36:28suspect that somebody came in without
36:31paying, they're not going to make a big
36:34deal out of it, they don't want any
36:36other guests seeing a discussion, etc.
36:40So, they were not sure how to solve this
36:42issue.
36:44So they decided
36:46to sort of like compute calculate how
36:50big of an issue is this like if this is
36:54I don't know $10,000 that we're losing
36:56every year. Who cares, right? Who is I
37:00mean we're not going to make a big fuss
37:01of uh $10,000 a year, but this is a
37:05multi-million dollar a year. Then maybe
37:07we have to find solutions for this.
37:09Okay, so that that's sort of like the
37:10mentality that they went in here. They
37:13asked us to estimate how big of a
37:16problem it was. And for that we had
37:18access to the cameras that they have on
37:21every like the every entrance point they
37:25have on every station they have a bunch
37:27of cameras and you go to the stations
37:28you scan your magic or your pass or
37:32whatever and then you go in. So we had
37:35access to those videos and we decided to
37:38build a model to do that. Now you can
37:41ask this to a thousand person and 999 of
37:44them are going to tell you well we need
37:46to build we need to build a system or
37:49build the model that identifies people
37:52coming into the park. Identify people
37:55coming in without stopping at the kiosk
37:58or you know the the place where they
38:00need to scan and maybe that right
38:03everyone will try to focus on the people
38:06who are doing the wrong thing.
38:09And that is very hard to do. If you have
38:12seen a line of people is just a huge
38:14mess and people go all over the place,
38:16especially when they have kids, they're
38:18unpredictable. They're running. It's a
38:21mass of people. Very hard to do. So
38:22instead, what we did was how about we
38:25take 10 hours of video footage per lane
38:28is just a huge ask. We have a 10-hour
38:31video feed and we identify
38:35every portion of the video feed where we
38:38are certain nothing bad is happening
38:41because lines like might not be a lot of
38:45people coming in or maybe the line is
38:47organized or maybe you know we identify
38:50that nothing weird is happen and by
38:53doing that we were able to reduce the
38:56amount of video feed that was
38:58interesting in 80% %. So now instead of
39:01having hours of video, now we had two
39:04hours of video to review. Now those two
39:06hours, the signal to noise ratio was
39:10really, really high. So we had a
39:12procurement department, a department,
39:14not a procurement department, a
39:15department that was tasked with watching
39:17those video feeds, their security
39:19department to review those two hours of
39:22video instead of giving them 10 hours
39:25per lane. Now they only had two hours to
39:28focus on. That was huge boost in
39:32productivity. We did not have to kill
39:36ourselves trying to identify the
39:38infractors. We only reduce the hay stack
39:42in order to get something that was
39:44better for humans or easier for humans
39:47to do. Does that make sense?
39:51That's the idea with the haystack
39:53principle. Another example of that is
39:56that you see all the time in uh ray
40:01x-rays sorry medical imaging right there
40:05are many many systems right now in place
40:08where before a radiologist had to go
40:11through a thousand images
40:13just looking for okay who what you know
40:16what's or the doctor not the radiologist
40:18but the doctor had to go through a
40:20thousand images identifying who was sick
40:24and who wasn't. And now you have
40:26algorithms that are going to go through
40:27a thousand images and will identify with
40:29very very high confidence every single
40:32image that's completely normal. Not the
40:35ones that are sick, which is very
40:37nuanced, but the ones that, for example,
40:40there are blank images, blank a x-ray
40:43images show nothing, those you can
40:46discard right away. Images that look
40:48very clean, those you can discard right
40:50away. and only leave the ones that are
40:53interesting for the doctor. That reduces
40:57the amount of work that the doctor has
40:59to do
41:01by a lot and you don't have to spend
41:03that much time building a system that's
41:05going to be really really hard for you.
41:07Okay. All right. So, let me go to next
41:10one here.
41:12So, that's the goal with problem
41:13framing. Every time you get to a
41:15problem, ask yourself, is it going to be
41:19easier for me to focus on the inversion
41:22of the problem? uh Charlie Mer uh I
41:25think he wrote it in his uh Charlie's
41:28calendar just book he wrote uh where he
41:32said that he and Warren Buffett became
41:36rich not by focusing on becoming rich
41:40but by focusing on not becoming poor or
41:44not uh basically becoming broke. So
41:49their whole strategy at the beginning
41:51was how can we avoid
41:54total you know financial failure.
41:58Let's just take a defensive position.
42:00Let's focus on the opposite on the
42:02inversion and that will help us you know
42:05the upside will take care of itself.
42:08That's sort of like the same idea where
42:10the hay stack principle would sort of
42:12like lead you to focus on the inverse
42:16like what everyone else is focusing on.
42:18All right. So next thing you you know
42:21you work on the framing the problem you
42:23ask the questions you have an idea of
42:25what you're working on. The next thing
42:27that I usually do during a discovery
42:29phase is just building just a simple
42:31prototype. So how quickly can we prove
42:34that this that we can actually build a
42:36solution for this? We have an idea. We
42:38know how we're going to frame it. Can we
42:40actually build a model? Can we actually
42:41build a solution that sort of like shows
42:44us that this is possible? Okay. And for
42:47this and if if nothing else, if you're
42:50not going to remember anything else from
42:51this class, I promise you remember this.
42:54You're going to you're going to be okay.
42:56Build always build the simplest thing
42:59that could possibly work. I've made a
43:03career out of that. out of saying we're
43:06not going to need that. We're not going
43:07to need that. What is the simplest
43:10thing? I have an anecdote which is kind
43:13of extreme,
43:15but it was a good lesson for the person
43:18who was working with me. So, we're
43:20working with this client and we're
43:21delivering a web page to a restaurant.
43:23And if you go to a restaurant page, you
43:26know that people display this map in
43:28restaurant pages where you can see every
43:31location the restaurant is is at. So you
43:34know if there's a chain you can see oh
43:37they have one here and one at five miles
43:39and one there. Anyway, the client wanted
43:42one of those maps and I have person
43:44working with me. We're up against the
43:46deadline. We have to deliver. The person
43:49is working on a map. He's asking me for
43:51more time. We need more time to build
43:53that map. We need to build an
43:55integration with Google map where we
43:57pass all of the locations that the
44:00client gave us and Google map will
44:02display those pings and then we're going
44:04to embed that Google map. We did not
44:06have uh cloud code back then. Probably
44:09we can do that today in five minutes
44:11asking cloud code to do it but we didn't
44:12have that cloud code back then. So we
44:14had to implement all of that and I was
44:16up to here. I did not have time for that
44:18and I told him we're not going to do
44:20that. This is what we're going to do.
44:24Google, go to Google Maps and Google the
44:26restaurant name and that will show you
44:28in Google Map all of the pins where the
44:31location is. Take a screenshot
44:33of that and paste it in the website.
44:36That's it.
44:37You put a screenshot on your website
44:40that shows every single pin. Well, yeah,
44:43but it's not going to be interactive.
44:44Who cares?
44:47Did the client say, "I want it to be
44:49interactive." No. People are not going
44:51to be able to zoom in. Yeah, they won't.
44:53Who cares? Boom there. Call it done.
44:56That's it. It's been a long time since I
45:00went to that website. But I promise you,
45:03it was for at least three years. That
45:05was the version they had there. Client
45:07was happy. Everyone was happy. We made
45:10assumptions that were not part of the
45:12deal that nobody cared about. And it was
45:16a screenshot. That was the simplest uh
45:19possible thing that you can do. You
45:21focus on this and it's going to be
45:24you're going to be okay. Make it work
45:26first, make it better later. That's the
45:30mentality when you're building a
45:31prototype. Okay? That's the whole
45:34mentality. All right. So, here is an
45:37example of it. So, I'm working for this
45:39company called Fashion File. So fashion
45:42file
45:45is a I don't know maybe maybe if you you
45:48guys uh care about fashion you know the
45:51fashion file it's like eBay for luxury
45:54goods so they sell watches they sell
45:57expensive packs I work with them for
45:59like three years I was alone like I mean
46:02me working as a self-employee with them
46:05my company working with them so
46:08we're building uh this or we had to
46:11build this model to determine
46:15what is the optimal price for an item.
46:20Uh so we can make the most money. That's
46:22sort of like the idea. So imagine that
46:24we are imagine that you buy a $10,000
46:28watch from somebody. The question is how
46:31much can you ask? I'm I I I paid $10,000
46:35for that watch. I want to uh sell it
46:39online. what is the maximum that I can
46:42ask for it? Okay. And when you have a
46:45complex operation where you are doing
46:47this over millions of items, inventory
46:50space is important. So you cannot just
46:52say well just let's price it at 20,000
46:55two times and we'll see if somebody you
46:58cannot do that because that item will
47:00sit in inventory for a long time might
47:04lose value. you don't know but at least
47:07going to be taking space from new uh
47:10merchandise that you want your
47:12merchandise to be moving. So we had to
47:14find that balance and we wanted to
47:17create a machine learning model to do
47:19that.
47:20This is not an easy thing to do to
47:23build. Uh we require like first of all I
47:26have no idea how to build this. We hey
47:28we just need to sit down and see what
47:30we're going to do. But we're not going
47:33to wait six months or a year to have a
47:36solution for this. Of course not. Let's
47:38build just a simple prototype, a stupid
47:40prototype and this is the way it's going
47:42to work. By the way, they were doing
47:43this with a procurement department. They
47:47had a whole department that was
47:49analyzing every single item and they
47:51were deciding the price that we're going
47:53to sell this on. And we wanted to take
47:55some of the work off of that department.
47:57So this is our prototype and it sounds
47:59stupid but this was working for almost a
48:02year. Okay, we took the the watch.
48:07By the way, bonus points if you can
48:08identify that watch is very
48:11identifiable. So you should be able to.
48:13So we took a watch and we set the
48:15initial price of that watch to the cost,
48:19how much we paid for it. Let's say we
48:21paid a thou $10,000 for it plus specific
48:24margin. That was a fixed concept. So
48:27let's say 40% margin. Okay. So initial
48:31price was $14,000.
48:34Then we make that watch available for
48:36purchasing. So the watch showed up in
48:40the website. Okay. $14,000.
48:43We waited for a month at that price.
48:47If nothing happened, if the watch did
48:49not sell after a month, we ask, can
48:53[clears throat] we still reduce the
48:55price? Like, what are we going to lose
48:57money here? We had like a minimum of 10%
49:00margin, for example. Well, yeah, we said
49:02it at the beginning at 40% margin, we
49:04still have time to go down. So, let's
49:07discount the price by 10%.
49:10And let's make it available again. So
49:13now the the over time the price of the
49:16watch is coming down 10% every 30 days
49:21until it hit a point where we were not
49:24going to reduce it anymore. And at that
49:26point we added that watch as a priority
49:30for the procurement team so they could
49:33take care of it and decide how much they
49:36wanted to sell that watch for. Very
49:38simple five lines of code. I don't know
49:40if this it was not five lines but you
49:42can see how simple this is. This was our
49:46prototype.
49:48We deployed this over time we built that
49:52machine learning model that took over
49:56this and I promise you it was really
50:00really hard to beat the
50:05ability for this simple model to make
50:07money. really really hard for the
50:09machine learning model to overtake this
50:13with better pricing. Sometimes simpler
50:16things are easier. So next time you're
50:19trying to build something on especially
50:21on the discovery phase, build a
50:22prototype that's very very very simple.
50:26Okay. So a good prototype should be
50:28simple to build, easy to maintain, and
50:31you should prioritize time to first
50:33insight over perfection. That's
50:36important. The goal of a prototype is to
50:39learn. That's what you want to do. So,
50:42what can you do to learn the most out of
50:45your prototype? Forget about the
50:47perfection. Forget about the details.
50:49Forget about the cool features. Focused
50:52on learning. Okay? That's the goal of
50:55the prototype because the prototype is
50:56the one that's going to guide you going
50:59forward. Okay? Another example here.
51:02That's spot by the way. This is from a
51:04video that we recorded. Let me see where
51:07is my mouse.
51:09Okay, there we go. So, this is Spot
51:11right here. This is me talking to Spot.
51:13So, we uh we were running missions with
51:15Spotions.
51:17What I mean is we were sending Spot
51:19inside a warehouse go and gather
51:21information and come back uh about you
51:24know the warehouse and whatnot. And we
51:26had this problem that we had to connect
51:29to spot, you know, Wi-Fi hotspot and
51:31connect open a website in order to read
51:34the information from the mission. And we
51:37wanted a way easy way to gather any
51:41quick insights quickly without having to
51:43do the whole connect to the robot or
51:46wait for the robot to upload the
51:48information etc etc. So what we built
51:50was voice commands. This is before chat
51:53GPT, but we built voice command to for a
51:57spot to listen to us talking and then
52:01using that information to just speed out
52:04whatever happened. So for example, we
52:06could say uh summary and spot will say
52:10we found four problems in such and such
52:14place and the problems are blah blah
52:16blah blah blah. Okay, with a robotic
52:18voice obviously. So the whole idea was
52:21just to build a simple classification
52:22model that will give us you know it will
52:26interpret the actions coming from the
52:28user from boys and will interpret them
52:31and you know basically spot will do
52:34those. So I was able to say something
52:36like move back of or move forward or or
52:41summary. That was the idea. Very very
52:45simple application we were able to build
52:48to do this. We did not have to get into
52:51any complex solutions. Very simple. When
52:55Chad GPT came out, uh this was even
52:59simpler because we got into a problem
53:01here. Just parenthesis here. We got into
53:04a problem where people did not remember
53:06the actual commands. So if you wanted
53:09spot to move back, some people would say
53:12back, some people would say step away,
53:15some people would say move away. So it
53:18was really hard. Chad GBT back then it
53:22was great because we were able to take
53:25the voice the transcription from the
53:27voice from the user and turn that
53:30classify that into a single command. So
53:34move back, step away, uh I don't know,
53:37get away from me. All of that translated
53:40into back and it was very easy to So
53:43anyway, hopefully that makes sense.
53:48Simple rules to build the prototype
53:51without using machine learning. Uh these
53:54are I'm not going to go through all of
53:55these, but basically uh there are many
53:59many many different problems that we can
54:01solve with machine learning.
54:02classification, regression,
54:03recommendations, anomaly detection,
54:05forecasting, clustering, those are some
54:07of them. These rules here, I add add
54:10them here because people who are trying
54:12to build models when they get into this
54:15discovery phase, the first reaction is
54:17well, we have to actually build the
54:19model to does classification or we
54:21actually need to build the regression
54:23model. And the answer is no. You can
54:25actually build a prototype without using
54:27machine learning. You can actually do
54:29classification by checking if a keyword
54:33exists in the text. Okay? So if the
54:36keyword exists, you're going to assign a
54:37class. Or if a value is greater than a
54:40specific threshold, we're going to
54:42assign specific class. So you can build
54:45rules that are not going to go far but
54:48are going to be good enough for the
54:49prototype in order to come up with a a
54:52solution that's going to prove whether
54:56there is light at the end of this
54:58tunnel. Remember the goal here is to
55:00build the simplest thing that could
55:01possibly work. I'm going to leave all of
55:03these rules here just in case you have
55:06an idea. These are ways to simplify a
55:10solution. Okay. Again, prototype demo.
55:15That's what these are going to do. These
55:17are not rules that are going to stay for
55:19long. But our goal here is just to build
55:21something as quickly as possible. All
55:24right. So, any questions so far?
55:28>> No, it makes perfect sense.
55:31>> Cool.
55:31>> Thanks for sharing.
55:33>> All right. So
55:36whole idea here with the prototype, I
55:38think we talked about this is your goal
55:40here is to basically measure, learn,
55:44question, improve. That's the idea of a
55:48prototype. It also serves to collect
55:50data that you can later use to build the
55:53model. Remember, when you're building a
55:55machine learning model or an AI model,
55:57all of those models require data to
56:00learn. And one way to gather that data
56:04is just to put together a prototype that
56:06people can start using. Think about the
56:08watch. By the way, did anybody recognize
56:10the watch? Let me check the chat here.
56:13Nobody recognize the the watch. Oh,
56:16robot guest is a robot. Whose robot?
56:19There was a robot here. It's a Cartier,
56:22but it wasn't sure. Offer a bunch of
56:24images to look at. Indeed, looks like
56:27Cartier tank. must is a cartier Santos.
56:31So, Ignasio is correct. It's a The tank
56:36it's more rectangular. Okay, that's what
56:38the tank is. The Santos is a square
56:42watch. So, yes, it's a Cartier Santos.
56:45That's the inspiration of that uh
56:49semantics. All right. So, anyway, uh I
56:52was gonna look for something and I
56:54forgot what I was saying. Oh, remember
56:56the watch example? When you put together
56:59that prototype and you put it out there
57:02for people to use, you're gathering
57:04data. You're seeing how people react to
57:07different prices for different products.
57:09All of that data we used to build our
57:12model. Without a prototype, number one,
57:15nobody's using your system. Nobody's is
57:17taking advantage of anything because you
57:19don't even have anything out there. So,
57:21you're not even collecting data at that
57:23point. Okay? It's very very important uh
57:26to have a prototype. Uh
57:30I'm going to say your name is probably
57:32going to be wrong. Please correct me. So
57:33hype.
57:34>> Yeah. So hi. Yes. So Santiago, you know,
57:37this is my second time joining your
57:38course. Uh a question I have is you are
57:41suggesting building models, right? Can't
57:43you use simply ResNet with a new set of
57:46data instead of training your new model
57:49or something very simple for object
57:51detection uh or or you know whatever is
57:54out there because training a model
57:56itself you know it's it's uh it's a huge
57:59task right uh uh starting from scratch
58:03writing the training code and all that
58:04in PyTorch or you know Jax or whatever
58:07right is a huge huge effort so what are
58:10your thoughts on this
58:11>> so okay So, by the way, when I talk
58:15about building a model, uh, building a
58:17model, I'm not necessarily thinking
58:19about training a model from scratch.
58:21Building a model, take it one level of
58:24abstraction higher. You don't care
58:26exactly who was doing the job. The model
58:28is sort of like the solution that's
58:29solving a problem. That being said,
58:32sometimes for some problems you do need
58:35to train models regardless of how good
58:38the latest cloud whatever Opus is or
58:42Kajd is. Specialized models for certain
58:46tasks are much better than general
58:49models that were designed to do
58:51something different. Let me just give
58:52you one example. Okay, for this problem,
58:55the one that I show you about uh the
58:58company with the watch and whatnot,
59:00they're running right now about probably
59:03a couple dozen specialized computer
59:07vision models to recognize uh luxury
59:10items. When they the client sends a
59:13picture of what they want to sell, the
59:16models will recognize what that is.
59:18You can do some of it with the current
59:23tech. And some of it, what I mean by
59:26that is the current tech opus will
59:29recognize, oh that is a watch. Oh, that
59:32is a bag. But it will not recognize
59:37sometimes the model of the watch or the
59:41model and brand of a bag. So a back is
59:46we don't do anything with that. That's
59:47not enough information. We need to know
59:48it's a Louis Vuitton Tomach and we need
59:51to know is a double kilted DD Louis
59:54Vuitton Tomach from 2006.
59:57Okay. So that level of detail you can
1:00:00only accomplish with a specialized model
1:00:03that's been trained on data coming from
1:00:06Louis Vuitton catalogs in order to
1:00:08recognize the nuance between two
1:00:10different backs because the pricing is
1:00:12going to be completely different. So
1:00:14throw that back to uh Opus and or Sonet
1:00:19or whatever model you're using and
1:00:21they're going to probably it's going to
1:00:22tell you oh that's a Louis Vuitton back.
1:00:24Okay, tell me the model. It's not going
1:00:26to know. So certain tasks require
1:00:30specialized models. But I'm going to go
1:00:31one step further. Even if or even when
1:00:37the current state-of-the-art models
1:00:40can perform a task, the next question is
1:00:43can they do it cheap enough for us to
1:00:47scale this? Okay, if we're getting if
1:00:50we're processing
1:00:52two million images every single day,
1:00:54just as an example, how much are you
1:00:57going to be paying on Tropic for two
1:00:59million images a day to use their
1:01:01models? that's going to be a lot of a
1:01:04lot of money. It's not feasible for the
1:01:06company to do that. They would rather
1:01:09build their own models that do that at
1:01:12just just nothing, you know, because
1:01:14it's so small model that they can build
1:01:17it quickly and they can run it super
1:01:19quickly and they can accomplish the same
1:01:21task without having to pay that much
1:01:23money. So that is why those models are
1:01:25not dead even when the latest you know
1:01:29models from anthropic and and open AAI
1:01:32can do the same job. Specialized models
1:01:34are still small are still cheap they
1:01:36scale very very well and they're still
1:01:39uh desirable by companies. And on top of
1:01:42that you said something and I'm going to
1:01:45not disagree with it but I'm going to
1:01:47sort of like add a little bit more
1:01:48information. You said training a model
1:01:50is a huge task.
1:01:53It depends what model it is, what
1:01:55project it is. You can train a model in
1:01:58literally 15 minutes. If you've done it
1:02:00before and you know what you're doing,
1:02:02it's just literally 15 minutes at
1:02:04assuming by 15 minutes. I don't mean
1:02:06that that's how long it's going to take
1:02:08the model training process. What I mean
1:02:10is you your work like putting together
1:02:12the code does that and does it well.
1:02:14It's not necessarily a huge task.
1:02:17Obviously, new projects or bigger
1:02:20projects will require more time from
1:02:22you, but it's not necessarily a huge
1:02:24accomplishment. Don't see it as as a
1:02:27mountain that requires a team. That's
1:02:30for yeah, for one of the
1:02:32state-of-the-art large language models,
1:02:34it's way more complex. For a small
1:02:36specialized models where you're starting
1:02:38off uh a foundational model that's very
1:02:41easy to just reproduce and retrain, it's
1:02:44not that complex. Hopefully that makes
1:02:46sense.
1:02:49>> Yes, thank you.
1:02:52What's up?
1:02:53>> Uh yes. Yes, there just as an extension
1:02:55maybe to to the question. uh I was more
1:02:58in on the on the edge case maybe where
1:03:01uh okay you train on a small model over
1:03:04uh some data and maybe I'm not really
1:03:07versed into that that field yet but what
1:03:10happens when for example a given brand
1:03:12comes up with a new model like it's a
1:03:15brand new one so how do you do you
1:03:17manage to deal with this use case
1:03:18usually
1:03:20>> yeah you you're not going to recognize
1:03:21that you have to keep training your
1:03:23models all of the time and we're going
1:03:25to talk about this in the class uh we
1:03:27call it usually continuous learning and
1:03:29it's something that you have to build
1:03:31into your projects uh I usually tell
1:03:35people that training the first version
1:03:36of your model that's your day one here
1:03:39people think okay we train the model
1:03:41we're done project is over no this is
1:03:44just day one of your project you need to
1:03:47build
1:03:48the workflows necessary for that model
1:03:51not only to keep monitoring the model
1:03:53and understand what happens with it but
1:03:55to keep retraining the model as new data
1:03:58becomes available. So the way you deal
1:04:00with that type of stuff uh if they're
1:04:02asking specifically number one you have
1:04:04to stay up to date but number two when
1:04:06your models see something that do not
1:04:09recognize you want to teach your models
1:04:12that hey I should not be able I should
1:04:15not be giving you an answer about this.
1:04:18One of the problems with these models is
1:04:21that they are trained to give you an
1:04:23answer regardless of whether they know
1:04:25what they're talking about or not. That
1:04:27sort of like explains hallucinations a
1:04:29little bit. Models don't know how to
1:04:31answer. They don't know how to say, "Oh,
1:04:33I don't know." They just tell you
1:04:36They just you. If you
1:04:38build a classification model, for
1:04:40example, and you want the model to
1:04:42classify an object into three classes,
1:04:46let's say we're talking about animals,
1:04:48elephant, dinosaur, and and lion, and
1:04:52you show the model a cat, the model will
1:04:55likely put that cat into a lion because
1:04:58it's the closest thing that resembles
1:05:00the cat. The model does not know that it
1:05:03should not be making assumptions about a
1:05:05cat because it hasn't seen a cat before.
1:05:08uh I think it's in session three or
1:05:10session four I'm going to show you
1:05:12techniques that you can implement to
1:05:15prevent models from making these sort of
1:05:18like mistakes which are very common and
1:05:20from dealing with scenarios where the
1:05:23model should not be providing an answer
1:05:26that's a good thing that we humans have
1:05:28if I ask you hey how do we send a rocket
1:05:31to the moon uh well maybe you know but
1:05:35maybe you tell me I have no idea idea.
1:05:37That's a good thing, right? That the
1:05:39fact that you're telling me I don't know
1:05:41how to do that. That's a good thing.
1:05:43Models don't have that capacity. If you
1:05:46were a model, you will tell me, "Oh,
1:05:48yeah. To send a rocket to the moon,
1:05:50first step, go to the store, buy just
1:05:53stupid answer like that."
1:05:56All right. Cool. Awesome. So,
1:06:00we built the prototype. We
1:06:03were in the middle of the discovery
1:06:05phase. We framed the problem. Uh, one
1:06:07thing that I would like to I'm going to
1:06:10paste it here, but if you have not read
1:06:14this, please do. This is a great reading
1:06:17and it's going to sort of like uh yeah,
1:06:20it's going to give you a different
1:06:21perspective. How do I go to the chat?
1:06:23Okay, maybe here
1:06:31I found it.
1:06:35So, for those of you who have not read
1:06:38that essay, uh just take take the time
1:06:41and go through it. It's just just really
1:06:43really good. Okay. Do things that don't
1:06:46scale from Paul Graham. Uh
1:06:50that's going to give you that's going to
1:06:52put your mind I think in the right place
1:06:55when you're thinking about the prototype
1:06:58that you have to build. Okay. All right.
1:07:01So,
1:07:04Okay, building and labeling a data set.
1:07:08We have a prototype. We know how we're
1:07:10framing the problem. Assuming that we're
1:07:12building machine learning based project,
1:07:15assuming that we have to build the model
1:07:17or we have to train a model. The next
1:07:20step is we just need to to have data to
1:07:24clean that data and to have that data
1:07:26ready for us to train a model. So,
1:07:28usually I'm going to focus here.
1:07:31Sometimes this is part of my discovery
1:07:34phase. Sometimes it's not depending on
1:07:37how complex the project is. Okay. So,
1:07:40this is sort of like a good breaking
1:07:42point for me to go back to the client
1:07:44and say, "Hey, uh, this is how we're
1:07:46going to be solving the problem. This is
1:07:48a quick prototype that we built. It
1:07:51proves that we're going to be able to
1:07:52solve this problem for sure. This is how
1:07:54we're going to do it. Uh, it's going to
1:07:56take this much time. It's gonna take
1:07:59this much money approximately. Uh for
1:08:02the next step, for the next iteration,
1:08:04we're going to be taking a couple more
1:08:06weeks or three weeks or four weeks,
1:08:07whatever it is. This is how much you're
1:08:09going to pay me for this. And we're
1:08:11going to start by collecting the data,
1:08:13by labeling the data, by cleaning the
1:08:15data, by doing feature engineering of
1:08:17the data. And that's going to be sort of
1:08:19like a good start for the project. Okay.
1:08:22Regarding the data, you want your data
1:08:24set uh three characteristics on your
1:08:27data sets. You need high quality, a lot
1:08:30of diversity and the right amount of
1:08:32quantity of data. Okay? So more data is
1:08:35not necessarily better. You need the
1:08:37right amount of data to train a model.
1:08:40You want that data to be high quality
1:08:42data and you're not you want that data
1:08:44to be diverse to cover as many real case
1:08:49scenarios as possible. That's sort of
1:08:52like the goal here with collecting the
1:08:54data. Okay, one parenthesis here. I'm
1:08:58going to come back later in a different
1:09:00session and this is going to make more
1:09:02sense. But when you are collecting data,
1:09:06you want to track
1:09:09uh as much meta data that comes with
1:09:12that data that explains that data that
1:09:14contextualizes that data as you possibly
1:09:17can. Let me give you an example. Imagine
1:09:20that you're building a model and the
1:09:22only thing that you need are pictures.
1:09:24You need images. Don't just take
1:09:26pictures and that's it.
1:09:29Record the picture, the time you took
1:09:32that picture, the conditions of the
1:09:34place where you took that picture, the
1:09:36camera that took that picture, the
1:09:39length that took that picture. Anything
1:09:41that explains and adds information
1:09:45about that specific piece of data might
1:09:49become crucial later. Okay. Another
1:09:53example, we are putting cameras inside a
1:09:56warehouse or
1:09:58a store. You have all of those CCTV
1:10:01cameras. From each camera, we're going
1:10:04to be recording the videos that come
1:10:06from the camera, but we also want to
1:10:09record the location of the camera. We
1:10:11also want to record the type of camera,
1:10:14the lens that's installed on that
1:10:17camera, anything that we consider is
1:10:20going to be important later on. Why is
1:10:23that? Just imagine that in a month one
1:10:27camera goes down. The company replaces
1:10:30that camera with the new model that came
1:10:33out and now you're getting two video
1:10:35feeds. One coming from old cameras, one
1:10:38coming from the new camera. You want
1:10:41that difference to be obvious. When
1:10:44you're analyzing your data, maybe your
1:10:46model stops working and you by
1:10:49segmenting your data out, you realize
1:10:52that the video feed coming from the new
1:10:55camera is different somehow, maybe
1:10:57different resolution,
1:10:59whatever, than the video coming from the
1:11:02old camera. And the only way you can do
1:11:04that and you can analyze the data that
1:11:06way is by having the meta data. I see a
1:11:10mistake that I see is that people just
1:11:11record the video feed and they don't
1:11:13record anything else and that is a big
1:11:15problem. So record as much as possible
1:11:18that will help you later. Okay. All
1:11:21right. So better data is better than
1:11:26better models. I truly believe this. I
1:11:29would rather work with a crappy model, a
1:11:32simple model with a very very good data
1:11:35set than the opposite. Okay? a very
1:11:39state-of-the-art almost perfect model
1:11:43with a crappy data set. Okay? So, don't
1:11:46worry that much about building better
1:11:49models. Worry more about building better
1:11:52data sets. That's what you want to do.
1:11:55Okay? So, we talked about the
1:11:57characteristics of a good data set. We
1:12:00talked about the quality of the data
1:12:02set. Here is you want the data set to
1:12:04have predicted features. You want the
1:12:06data set to have the correct labels. You
1:12:08want the data set not to have missing
1:12:11values. You want data set to be
1:12:12complete. You want a lot of diversity,
1:12:15right? You want the data set to include
1:12:18rare events, outliers, have low bias,
1:12:22represent subgroups of the population
1:12:25that you're capturing in your data set
1:12:27proportionally. That's what a good
1:12:29diversity means. And you have good
1:12:32quantity. By good, I don't mean a lot of
1:12:34data. I mean the right enough amount of
1:12:36data. So enough samples for the model to
1:12:39generalize
1:12:41appropriate volume for the model. Okay,
1:12:44we're going to see examples of how more
1:12:47data is not necessarily better. Okay, so
1:12:51here's one example here.
1:12:54Predicting home prices and how more data
1:12:56does not help. So imagine that you have
1:12:59an initial data set from location A and
1:13:02these are your images. I don't know if
1:13:04you can see those images there. Uh I
1:13:08generated those images with with an AI
1:13:10model. Maybe Chad GPT generated those
1:13:13for me. Those are crappy sort of like
1:13:15crappy houses representing maybe a
1:13:17location that's not too wealthy. Okay.
1:13:20And let's say we have 10,000 houses
1:13:22here. And the median average uh the the
1:13:28median average price here, the mean
1:13:29absolute error here is actually $18,500,
1:13:35okay, from these neighborhoods. And now
1:13:38you go out there and you augment that
1:13:41data set by adding houses from location
1:13:44B. Okay? So you just doubled the number
1:13:47of houses. You went from 10,000 houses
1:13:50to 20,000 houses, but because now you
1:13:53added a bunch of rich people houses
1:13:56here. Now your mean absolute error in
1:14:00this data set goes up to 20,000 $21,000.
1:14:05So the result of a model by the way this
1:14:09in case I was not clear enough this m ae
1:14:13mean absolute error that's sort of like
1:14:16the the average error that my model has
1:14:19given me the price of a house. Imagine
1:14:21that we're building a model that's going
1:14:22to predict the price of a house. With
1:14:25this data set the error of that model is
1:14:28going to be on average $18,000. That's
1:14:31the mistake my model is making. on
1:14:34average is up or down $18,000.
1:14:37By adding more data to that data set,
1:14:40the error is going to go up. Now, and
1:14:43the reason is because the data that I'm
1:14:45adding is actually making this this sort
1:14:49of like problem space for the model way
1:14:52harder to generalize to because now
1:14:54there are houses that are sort of like
1:14:57all over the place, very very expensive
1:14:59houses. Does that make sense? more data
1:15:03is not going to necessarily
1:15:06make my model better. Better data will
1:15:09not necessarily uh it's going to make my
1:15:12model better. So
1:15:14sometimes more data is actually worse
1:15:18and I keep insisting on this.
1:15:22I consult for companies and this is very
1:15:24very normal. Companies call me because
1:15:27they're having a problem with their
1:15:28models. They ask me, "Hey, can you come
1:15:30and take a look? let's see what's going
1:15:31on. And [snorts] I go and I listen to
1:15:33them and they tell me the problems
1:15:35they're having and whatnot. And I
1:15:37usually start by asking them, so what do
1:15:39you think the solution would be? Okay.
1:15:41And nine out of 10 times people tell me
1:15:44we need more data. That's nine out of 10
1:15:46times. If the model is not working well,
1:15:49we just need more data.
1:15:52And no, you I mean you might, but you
1:15:56have to justify more data. Okay. So this
1:16:00is sort of like the process that we go
1:16:02through. This is very simple. This is
1:16:05machine learning 101.
1:16:07This is how an experiment that you can
1:16:09run to determine whether you actually
1:16:11need more data. I force companies to do
1:16:13this. I force the team to do this and to
1:16:16show me if they truly need more data. So
1:16:19this is the way it works. Okay. So we
1:16:21take the whole data set, the whole
1:16:24training set that they're using and
1:16:26we're gonna we're going to create we're
1:16:28going to take 10% of that data set and
1:16:31then we're going to take 20% of the data
1:16:33set and then 30 and then 40 and then 50,
1:16:35right? We're going to be creating these
1:16:36subsets each of them with increasing
1:16:39amount of data and we're going to be
1:16:41training the model that they have and
1:16:45testing it on the regular test set. So
1:16:47imagine that we train with 10% of the
1:16:49data set. Where is my mouse? And we're
1:16:52going to get our model. Let's say our
1:16:54model is model one. Uh we're going to
1:16:57get sort of like a performance right
1:16:58here. And when we do with 20%, the
1:17:01performance is right here. And we do
1:17:02with 70% the performance is going to be
1:17:04right here. Okay. The more data we add,
1:17:07we plot I mean we can see how that model
1:17:10the accuracy of that model is going up.
1:17:13So I'm going to ask you this comparing
1:17:16model one with model two.
1:17:19Which of these models could benefit from
1:17:22more data? I mean, it's the answer is
1:17:24obvious because it's right here on the
1:17:26screen, but you can see that model one
1:17:28could potentially use more data because
1:17:31so far we have not seen a plateau. The
1:17:35more data we add, the better the model
1:17:38does. So we might, you know, think it's
1:17:42it's it's yeah, it's it's obvious kind
1:17:44of obvious that we might want to check
1:17:46adding another 10% of the data and see
1:17:49if the trajectory keeps going up. But
1:17:53whenever we are in the case of model two
1:17:56and I see this all the time, you already
1:17:59see a plateau in this model from 80% of
1:18:02the data set going forward like you can
1:18:04see how the performance of the model is
1:18:07not improving even though we're adding
1:18:09more data to that data set. Okay. So
1:18:12whenever I ask a company to do this
1:18:14exercise, a team to do this exercise and
1:18:17we analyze the learning curve and we see
1:18:19that we are in the case of model two
1:18:22that tells them it's not the data. You
1:18:25can go and try to get more data but you
1:18:28have no guarantees that data is going to
1:18:30make that model go up. You need to focus
1:18:33somewhere else not on the data. Okay?
1:18:36Does this make sense?
1:18:40All right. People spend too much worried
1:18:42about their models. Every single book
1:18:44that you can buy out there talks about
1:18:46how to build models.
1:18:49They don't take enough time to think
1:18:51about their data. Okay? Data is very
1:18:55very important. You should be spending
1:18:58more time on your data, less time on
1:19:00your models. I promise you. So there was
1:19:03this like movement that started I don't
1:19:05know maybe eight years ago, five years
1:19:07ago. It's called datacentric AI where
1:19:10the whole idea of datacentric AI was to
1:19:12do AI to do machine learning by focusing
1:19:16on the data instead of just optimizing
1:19:19the models. Okay, this is what I usually
1:19:22do. So I spend most of my time worrying
1:19:26about the data because I believe or what
1:19:29I've seen is that
1:19:32the defaults of when I let's say I'm
1:19:35going to create a computer vision model
1:19:37classification model and I I'm going to
1:19:39based off that classification model off
1:19:42of an existing uh architecture the
1:19:45restnet 50 architecture or the restnet
1:19:48100 architecture and I'm going to use
1:19:51very you
1:19:53sensible hyperparameters like the
1:19:56obvious ones. I'm going to use a batch
1:19:58size of 32. I'm going to use a learning
1:20:00rate of 0.003.
1:20:03Things like that. That usually is enough
1:20:06to get very very good results. Can I
1:20:09improve that? Maybe. I'm not going to
1:20:11spend a lot a long time doing that.
1:20:13Instead, I'm going to try to focus a
1:20:16model on the data. Okay. Here are some
1:20:19of the things that you do, some of the
1:20:21activities that you do. When you're
1:20:22focusing on the model on the left versus
1:20:25when you're focusing on the data on the
1:20:26right. Okay. So, when you're focusing on
1:20:28the model on when you're focusing on the
1:20:30model, you're doing things like
1:20:31hyperparameter tuning. What is the best
1:20:34batch size for this data set or uh let's
1:20:37implement an ensemble model or let's do
1:20:39transfer learning or let's use
1:20:42regularization.
1:20:43you're trying to squeeze better
1:20:46performance out of your model with the
1:20:48data set that you have on data centric
1:20:50AI instead you're focusing on the data
1:20:52you're keeping the the model fixed and
1:20:55you're saying how can we create new
1:20:57features for example or modify the
1:21:00existing features in our data set in
1:21:03order to increase the amount of insights
1:21:05we give the model that we have so the
1:21:07model can make better predictions how
1:21:09can we do balancing how can we improve
1:21:12the labels the quality quality of those
1:21:14labels. Maybe there are wrong labels.
1:21:16How can we generate more fake data or
1:21:19synthetic data to improve the ability
1:21:22for the model to generalize? That's the
1:21:24idea with that ascentric AI. This is
1:21:27where over you know I usually spend most
1:21:30of my time instead of on the first
1:21:33column here. All right.
1:21:35Okay. So you will start with data
1:21:38centric with model centric. You would
1:21:40build quickly a model. it works. it to
1:21:43sort of like gives me results and from
1:21:45there on I'm going to focus on data
1:21:46centric AI and whenever I hit maybe my
1:21:50ceiling I'm going to go back to model
1:21:51centric do some tuning
1:21:54see how can I prove this quickly and
1:21:56then go back to dataentric sort of like
1:21:57that back and forth is what I find that
1:22:00it sort of like helped me okay so
1:22:03finally before you start training the
1:22:05model you need to produce part of the
1:22:08data you need to produce consistent high
1:22:11quality ground truth labels. Okay,
1:22:16this here is always a huge bottleneck.
1:22:19The lack of label data. So whenever I go
1:22:22to companies, some of them some of those
1:22:24companies have good quality data but
1:22:28they don't have labels for that data and
1:22:30creating those labels is is huge. It's a
1:22:34huge effort. I think it was Karpati who
1:22:36uh who said a long time ago when he was
1:22:38working for Tesla uh somebody asked him
1:22:41in an interview if he was giving $10
1:22:43million more to improve Tesla uh or you
1:22:47know the self-driving system uh where
1:22:49would he spend that money and everyone I
1:22:52I'm pretty sure everyone would have
1:22:53guessed oh you know we will buy more
1:22:55data centers or you know more people to
1:22:58improve the algorithm and he said on
1:23:00labeling I need better labels right I
1:23:04would spend my $10 million in better
1:23:06labels because it's a huge huge problem.
1:23:09The lack of quality labels, uh, that's
1:23:12just going to destroy everything. Just
1:23:14to give you an idea of how hard it is or
1:23:17just to sort of like good frame of mind,
1:23:20imagine that you're building a
1:23:21self-driving system and you're capturing
1:23:23an image that looks like this and you
1:23:26want to train your system and in order
1:23:27to do that, you want to label every
1:23:30single object that you see here that's
1:23:31relevant for driving. Okay? So here
1:23:34you're going to have cars and you're
1:23:36going to have traffic lights and trees
1:23:38and lanes and cars in front of you and
1:23:41maybe pedestrians. So there's a bunch of
1:23:44images here. Okay. So let's say you
1:23:46capture one minute of video. That's all
1:23:48you capture. One minute of video and you
1:23:52capture that video at 30 frames per
1:23:54second. So that means there are 30
1:23:56images for every minute of video and
1:23:59there are approximately 20 objects per
1:24:02frame. So on every frame you're going to
1:24:04see 20 different objects that you would
1:24:06like to label. Okay. 60 seconds of video
1:24:11times 30 frames per second times 20
1:24:15objects. That's 36,000
1:24:18boxes that somebody has to draw on the
1:24:21screen to completely label that one
1:24:24minute of video. So if you have somebody
1:24:28that's capable of drawing one box every
1:24:31second, it will take 10 hours to label
1:24:36one minute of video. That's a long long
1:24:40time. Okay, this is why labeling is
1:24:44hard. There are bunch of algorithms that
1:24:46do this that help with this problem.
1:24:48People don't sit here obviously to label
1:24:50every single one of those. But this is
1:24:52just to give you an idea of how hard the
1:24:56process is. Okay. Now you can do
1:25:00labeling by hand. You can uh generate
1:25:03labels using algorithms that exist or
1:25:06you can use existing labels if your
1:25:08problem
1:25:10you know have built-in labels. there are
1:25:12problems that uh you don't need to
1:25:14generate labels because let's say you
1:25:16want to predict whether it's going to
1:25:18rain in the next hour or not. Well, the
1:25:21label for that problem, you just wait an
1:25:23hour and then you're going to have your
1:25:25answer. So, that's the label is a
1:25:26built-in label right there. You don't
1:25:28need to generate a label manually. But
1:25:30many problems do not have those built-in
1:25:34labels. You have to create them. That's
1:25:37where things get a little bit uh tricky.
1:25:39So here is one algorithm that you can
1:25:41use to sort of like uh avoid having to
1:25:46label a lot of the data sort of like
1:25:49shortcut create a model without you
1:25:52spending a ton of time generating labels
1:25:55because again it's very very expensive.
1:25:58uh as an example uh so I uh when I was
1:26:01working with my previous company uh we
1:26:03were working with this company and one
1:26:06thing that they were doing is uh is
1:26:08digging for oil. So literally they
1:26:11wanted to uh they had a company or an
1:26:14operation that was drilling holes in the
1:26:17ground finding oil. Okay. So they wanted
1:26:20to sort of like create a model that
1:26:22would predict where to drill next. Okay.
1:26:26That's sort of like the idea.
1:26:29But in order to do that
1:26:31they we need a data and data needs
1:26:34label. So imagine that I come here and I
1:26:36say okay so here I need a label that
1:26:40tells me in these coordinates if there
1:26:42is oil yes or not. How do you think you
1:26:46come up with that label?
1:26:49You have to drill. That's that's the
1:26:52only way you're gonna know if there is
1:26:53oil or not. So you know if we have
1:26:5710,000 samples we cannot just go out
1:26:59there and start drilling holes to just
1:27:02label the data set. So you have to
1:27:03minimize the number of labels you
1:27:05actually need. Okay. So active learning
1:27:09helps with that. Okay. So active
1:27:11learning technique. I've used it a ton.
1:27:13It helps with that. It takes a bit to
1:27:16sort of like understand how it works. So
1:27:19hopefully we can work through this
1:27:20really really quick. But just pay
1:27:22attention here and hopefully this makes
1:27:23sense. So we start with a data set.
1:27:26Imagine that your goal is to build a
1:27:28model here. Okay? But you start with a
1:27:30data set that don't have any labels. So
1:27:32you don't know anything about this data.
1:27:34You don't have any labels here. So
1:27:38the first step is just to take small
1:27:41portion of that data set and label it.
1:27:43Okay? You're not going to focus all of
1:27:45your time labeling all of the data.
1:27:47Remember that's going to be too
1:27:48expensive. Just going to take let's say
1:27:5010%.
1:27:52and label that 10%. And with that
1:27:55manually labeled data, you're just going
1:27:58to train the first version of your
1:27:59model. Okay? So only 10% of the data,
1:28:02probably not enough to get a good model,
1:28:05but you're going to get a model that's
1:28:08better than not having a model. So now
1:28:10we have that model, version one. You're
1:28:13going to use that model to make
1:28:15predictions to automatically generate
1:28:18the labels of the other 90% of the data
1:28:21set. So after this, this is what you're
1:28:24going to get. You're going to have the
1:28:2610% of the data set that you manually
1:28:28labeled. You're going to get some data
1:28:32that's automatically labeled by the
1:28:34version one of the model, but I'm
1:28:36splitting that in two separate sets. the
1:28:40automatically labeled and the complex
1:28:43samples. What I mean by this is
1:28:46depending on the type of model that you
1:28:48create, you will find that that model is
1:28:51very confident in certain predictions
1:28:55and less confident in other predictions.
1:28:59So if you separate those now you can
1:29:01have some the high confidence
1:29:06samples you can just assume those are
1:29:08correct because your model should be
1:29:10good enough to predict those and the
1:29:13least confident samples you can sort of
1:29:16like put them in the these are very
1:29:19complex for my model therefore I'm gonna
1:29:23need to do something with them. So, what
1:29:25you are going to do with them is you're
1:29:27gonna send this red sliver of samples.
1:29:30You're gonna send that. Oh, there are
1:29:32people here trying to enter. Hold on.
1:29:35You're going to send the How do I do
1:29:36that? Oh, there we go.
1:29:39You're going to send this sort of like
1:29:41red liver of samples to the labeling
1:29:46team. The labeling team then will
1:29:48automatically label this. Not
1:29:51automatically, manually label this.
1:29:53You're going to poke the holes on those
1:29:55ones there and then you're going to
1:29:57restart the process again. The next
1:30:00model version two, you're going to use
1:30:01the many level samples. You could use
1:30:03the automatically labelled samples
1:30:05together. You're going to train the
1:30:06second version of the model. You're
1:30:08going to apply the same principle going
1:30:10forward. And every single round, your
1:30:13goal is to determine the complex samples
1:30:16here. And if you determine the complex
1:30:19samples, those are the candidates for
1:30:22you to manually label. Next,
1:30:25I'm going to talk about in case that's
1:30:27the question that you guys have, but I'm
1:30:29going to let you speak right now. Uh
1:30:31Adulson,
1:30:33the next few slides I'm going to show
1:30:35you what how to determine what the
1:30:38complex samples are. But if you trust me
1:30:42and you sort of like believe me that we
1:30:44can do that correctly,
1:30:46active learning have shown that we can
1:30:49build a model as good as if you if you
1:30:54label the whole data set. Basically, in
1:30:56other words, you do not need labels for
1:30:59every single sample in order to have the
1:31:02best possible model. You can do it more
1:31:05efficiently. and active learning will
1:31:07let you find what those what the
1:31:09critical samples are. Uh Adelson, what's
1:31:12up?
1:31:14>> Oh, thank you, Santiago. So, regarding
1:31:15to um the size of the data set some
1:31:19slides ago, um the test set
1:31:24I think I think the test set should be
1:31:26also subplit, right? I think there could
1:31:30be some issues if we were evaluating
1:31:33directly on the actual test set, right?
1:31:37I sorry you're going to have to repeat
1:31:39that. What is this the test set role
1:31:42here? What do you mean by the test set
1:31:44here?
1:31:45>> Because uh you said um in order to know
1:31:49if we need more data or not subsplit
1:31:51your the data you have and then you
1:31:54continually train and add 10% more. But
1:31:58at each iteration you should test right.
1:32:01So I'm just concerned about this test
1:32:03set. this test set cannot be the actual
1:32:07uh test set right can need to be one of
1:32:11the splits as well right so it depends
1:32:14if you're starting so there are two
1:32:16approaches here approach number one is
1:32:18that you do have a test set set aside
1:32:21it's not part of this slide here and
1:32:24that's the test set you're going to be
1:32:25using to test whatever model you're
1:32:27building that's one that test set where
1:32:30is it coming from well maybe it's data
1:32:31that you labeled manually the second
1:32:33approach which is that you're studying
1:32:35with a data set that has no labels, but
1:32:38part of the manually labelled samples,
1:32:40those are going to become your test set.
1:32:43And as you increase the manually
1:32:45labelled samples, you're also increasing
1:32:48the amount of that data that goes into
1:32:50the test set. Obviously, in those cases,
1:32:52you will never use the test samples to
1:32:55train a version of the model. You will
1:32:57keep those aides. I did not represent
1:32:59the test set here, but hopefully that
1:33:01makes sense. Does that answer your
1:33:02question?
1:33:03>> Yeah. Yeah. Mhm. That's clarify. Thank
1:33:05you.
1:33:06>> Yeah. So imagine just just to make it
1:33:08more concrete, imagine that this instead
1:33:10of instead of manually labeling 10% of
1:33:13my data set, I'm going to label 15% of
1:33:15my data set. 5% of it I'm going to keep
1:33:17aside for my test set. The other 10% is
1:33:19the one that you see represented here.
1:33:22Now, the next time I do manual labeling,
1:33:24which is right here. Now, out of all of
1:33:26the data that I manually labeled, I'm
1:33:29going to get another maybe, I don't
1:33:30know, 20% of that data and throw it
1:33:32away. Not throw it away, set it aside to
1:33:35increase the size of my test set. I want
1:33:38my test set to become stronger as my
1:33:41model becomes stronger as well. And then
1:33:43I want to keep repeating the process
1:33:45there. Make sense?
1:33:47>> Yeah. Thank you.
1:33:49>> Hi, uh, Pavang, what's up?
1:33:52>> Hi. Um I just wanted to know your
1:33:54experience like have you had a chance to
1:33:56use any LLM models to generate synthetic
1:33:59labels? Uh if so any popular that that
1:34:03are like well known.
1:34:05>> Yeah. Yeah. I've used LLM to generate
1:34:08synthetic labels. Um well synthetic I
1:34:11don't know what a synthetic label is to
1:34:14generate to automatically label data if
1:34:16that's what you mean. Yes.
1:34:18>> Okay.
1:34:19And any like uh do you recall any like
1:34:22models that you might have used?
1:34:24>> I mean it's I usually so the only time
1:34:28I've used them is when the LLM I know
1:34:32for certain that the LLM is going to be
1:34:34very good at providing those labels. So
1:34:36for example classifying customer
1:34:38requests.
1:34:40Customers send a paragraph they want to
1:34:43do something. you create a bunch of
1:34:46categories and the LLM is going to read
1:34:48the text and classify that text in one
1:34:52of those categories or you've also seen
1:34:55uh sentiment analysis is another one
1:34:57that LLM do very well like is the client
1:35:00uh is this review positive or negative
1:35:02right very easy for the LLM to classify
1:35:06given portion of text sent by a client
1:35:08into positive or negative review so
1:35:11those are the examples that I've used
1:35:13LLM to to uh to generate labels. I don't
1:35:16need to read the text myself and say
1:35:20positive or negative.
1:35:21>> Understood. Understood. That's okay.
1:35:24>> Cool.
1:35:25>> Thank you.
1:35:26>> All right. So, the question that remains
1:35:28unanswered here is what the heck are
1:35:30these complex samples? uh because you
1:35:33know if you're able to identify those
1:35:35complex samples out of all of the
1:35:38predictions that the model produces well
1:35:40that's the key because if those complex
1:35:43samples I I I call them here complex
1:35:46some people call them most informative
1:35:50uh if we can identify good complex
1:35:52samples that means that we're going to
1:35:55be able to learn really really quick to
1:35:57to sort of like the model will progress
1:36:00really really
1:36:02If I do this run, if my complex samples
1:36:05are not that interesting for the model,
1:36:07the model will not improve. Okay? So the
1:36:10the better those samples are, the faster
1:36:12my model will improve. Okay? So
1:36:15there are usually I I mean there are
1:36:18entire PhDs
1:36:20thesis
1:36:22created just on this topic, okay?
1:36:25Because it's a topic with a lot of
1:36:26research behind. But usually the two
1:36:29techniques that I've personally used and
1:36:32again there are hundreds of techniques
1:36:34that you can use here are uncertainty
1:36:38and diversity. So I want to pick my
1:36:42complex samples or my most informative
1:36:44samples those that increase the
1:36:46uncertainty and the diversity of my data
1:36:50set. So let me give you here uh first
1:36:53uncertainty what uncertainty means.
1:36:55Okay. So these are samples that my model
1:36:59find confusing. I want to identify any
1:37:02samples that my model it's making a
1:37:06mistake
1:37:07uh thinking the the the data is
1:37:10something but it's actually something
1:37:11else. Uh
1:37:14here it's just me again trying to
1:37:16generate pictures here with AI. But here
1:37:18you can see a cat that actually looks
1:37:20like a duck. Kind of like it's just
1:37:23there is a bone there. Or this cat here
1:37:26is also wearing a custom of a dog. So
1:37:28you can imagine a model just looking at
1:37:30these pictures and saying, "Oh, I think
1:37:32that's a dog." When it actually is not a
1:37:34cat. This is a dog, but it looks to me
1:37:36like a cat. It's sort of like half the
1:37:38face like a cat. Is kind of weird. And
1:37:40this looks like I don't know like a
1:37:42pillow. This little cat there. Maybe a
1:37:44dog. I don't even know what it is, but
1:37:46you get the idea. is samples that might
1:37:49be one way or the other. Imagine a
1:37:52binary classifier.
1:37:54And in a binary classification problem,
1:37:57your model is going to be this dotted
1:37:59black line. Any samples that are near
1:38:03that boundary that could either be left
1:38:06or right. If you just move the model one
1:38:08little bit, you will have to change the
1:38:11classification of those samples. So any
1:38:14samples near that boundary those are
1:38:17good uh candidates for me to label. I
1:38:21want to actually find out the label of
1:38:25those samples because if I if I find out
1:38:28that the label of this sample is
1:38:29actually blue not green. Right now it's
1:38:32on the green side but if this is
1:38:34actually blue that gives the model a lot
1:38:37of information. the model should
1:38:39accommodate for that loop should move to
1:38:41the left. Okay, so these are samples
1:38:45that have the potential of teaching my
1:38:47model a lot. That's what uncertainty
1:38:49sampling is. Now in the case of
1:38:52diversity sampling is
1:38:56samples that are far far away from that
1:38:58decision boundary. Those could be edge
1:39:01cases. Those could be outliers. I want
1:39:04my model to know about those. So the
1:39:06pictures here, you can see like a
1:39:08nighttime picture of a deer and it's
1:39:11it's a picture that's completely
1:39:13different from every other animal
1:39:15picture that I have on my data set. I
1:39:17want to label that one there. Or maybe a
1:39:19zoomed out butterfly. This just usually
1:39:23an unusual picture of a butterfly very
1:39:26very close. or images that have low
1:39:29contrast
1:39:31because you know the animals and in both
1:39:33images sort of like blend with the
1:39:35background. So it's just anything
1:39:38that could potentially be an edge case
1:39:41an outlier. I want to label those. In
1:39:45the case of the binary classifier, those
1:39:48will be these data points right here
1:39:51that are very very far away from
1:39:53everything else in the data set. those
1:39:57will provide a lot of information to my
1:40:00model. Okay, if I identify those two,
1:40:04I'm gonna have a way to label data that
1:40:08will improve my model way faster and
1:40:11obviously the process is going to be way
1:40:13cheaper than having to label every
1:40:16single sample in my data set. All right,
1:40:18any questions so far? We're going to go
1:40:20to the last of the topics here and then
1:40:23we're done for today. The last of the
1:40:25topic is feature engineering. Uh again,
1:40:28you can also create like a whole career
1:40:31of a feature engineering. Uh
1:40:35I find out that the best people that I
1:40:37know that do this well are those that
1:40:39are they participate in Kaggle. Everyone
1:40:42who who's done Kaggle has gone through
1:40:44this process. A bunch of techniques come
1:40:47out of that. This is sort of like an
1:40:49art. This is the ability to take a data
1:40:51set and
1:40:54shape that data set in a way that
1:40:57maximizes the amount of insights you put
1:41:01in front of the model. Okay? If you
1:41:04don't do that, your models might
1:41:06struggle. Models might struggle. It's
1:41:08sort of like you are hiding the
1:41:09information from the model. But if you
1:41:11sort of like put that information
1:41:13outside, models are going to do really
1:41:16really good. Okay. Again, you want to
1:41:18learn feature engineering, my best
1:41:20recommendation is just go join Kaggle,
1:41:22start participating in competitions
1:41:24there.
1:41:26Just just super super cool. Uh there are
1:41:28a bunch of importances. I mean picture
1:41:30engineering has a bunch of importances.
1:41:32The the one that I really really really
1:41:34care about is more accurate models. I
1:41:36think this is this should be enough. But
1:41:38yeah, you can also your models are going
1:41:39to become faster. They're going to be
1:41:41simpler uh you know if if you do good
1:41:44feature engineering here. So bunch of
1:41:46techniques for feature engineering.
1:41:48Bunch of things that we can sort of like
1:41:51put under the umbrella of feature
1:41:53engineering. The first one is creating
1:41:55new features. You have a data set. You
1:41:58want to create a model and when you
1:42:00analyze that data set you realize that
1:42:02by you creating new features you will
1:42:05make it easier for the model to learn
1:42:07that data set. Okay. So let me give you
1:42:10a few examples. Imagine that the
1:42:12original data set contains the price of
1:42:15products and contains the sold date of
1:42:19those products. Okay, that's the
1:42:21original data set. Okay, so one thing
1:42:24that you could do is what we call bin.
1:42:28Okay. And being here is because you know
1:42:32that the problem you're trying to solve
1:42:35cares about products that are cheaper
1:42:38for some reason, products that are under
1:42:41$50.
1:42:43Maybe you want to make that information
1:42:46painfully obvious to the model. And
1:42:48instead of letting the model figure that
1:42:50out from the price itself, you're going
1:42:52to create a new feature that's called
1:42:54under 50. That's going to be a binary
1:42:57feature. is going to contain one if the
1:42:59price is less than $50 or zero if the
1:43:02price is not less than $50. Okay, very
1:43:06simple for the model to just process
1:43:08this feature here and
1:43:12signal for those products that matter
1:43:15for the problem you're creating. Okay,
1:43:18that's bin. Temporal feature engineering
1:43:21is what you do with the date. So maybe
1:43:23for example you want to extract the
1:43:26quarter number where this the sale
1:43:29happened
1:43:31uh into a new feature and this was you
1:43:33know this happened in the fourth quarter
1:43:35and these three happened in the first
1:43:37quarter or maybe you want to specify
1:43:40whether that specific day fell on a
1:43:44holiday. And now you get here this one
1:43:47for holidays and zeros for any other
1:43:51date that are not holidays. Again, here
1:43:54you're basically creating new
1:43:57information from the model. Taking what
1:43:59already exists
1:44:01and shaping it in a different way for
1:44:04the model to process. The other
1:44:06technique is clustering. And this is I
1:44:08don't know for some reason you have a
1:44:10separate model maybe and you process all
1:44:13of your products and you cluster them
1:44:17following a specific characteristic. So
1:44:19maybe some of these products well were
1:44:22sold to men and some of them were sold
1:44:25to women and that's a cluster that you
1:44:28create with zero for men and one for
1:44:30women and that's another information
1:44:33you're giving to the model. Okay, this
1:44:35is what creating new features looks
1:44:39like. Okay, so one specific example, a
1:44:42real example, this is also for fashion
1:44:44file. These are bags and we had these
1:44:47computer vision models like I I told you
1:44:50about that the goal of those models was
1:44:52to classify
1:44:56classify specific image from a back uh
1:45:00into a brand and into a model of that
1:45:03brand. So maybe you get a Hermes, Dato,
1:45:08whatever, CC, whatever. They have just
1:45:10some weird names. So
1:45:12to do this, we were using images sent by
1:45:16the customer. So the customer wants to
1:45:18sell a bag. They take a bunch a bunch of
1:45:21pictures off of that bag. They sent us,
1:45:24I don't know, 10 14 pictures. We took
1:45:26all of those pictures through a
1:45:28classifier, and that classifier would
1:45:30tell us what the back was. The problem
1:45:32is that because we're working with
1:45:35multiple pictures,
1:45:37the classifier might give different
1:45:39answers for different pictures. Okay,
1:45:42that was the problem. So, imagine that I
1:45:44have a back and I have a back. Where is
1:45:46the back? Yes, I have a back right here.
1:45:51So, I have a back and I send this
1:45:54picture of the back and then I send this
1:45:56picture of the back and then I send this
1:45:58picture of the back and then I send this
1:46:00picture of the back and then I open the
1:46:02back and send another picture inside and
1:46:04another picture here. You get the idea.
1:46:06That's how people send pictures. But
1:46:09this here does not provide the same
1:46:12amount of information
1:46:14than this here. Right? So I might get an
1:46:17answer from this picture that's
1:46:21different from this picture
1:46:22unfortunately. So we had to solve that
1:46:25problem. And the way we solve that
1:46:27problem is just by voting. That's just,
1:46:30you know, let's get all of the models or
1:46:33the pictures and get, you know, voting
1:46:36and, you know, whatever brand had the
1:46:40most votes. That's the final answer. And
1:46:43that worked relatively well.
1:46:46But we discovered something that made it
1:46:49so much better. I think it was, let me
1:46:51see if I wrote it here. I did not write
1:46:53it here. I forgot. But I think it was 6
1:46:56to 8% better, which translates into a
1:47:00lot of money, by the way.
1:47:02And that was creating a new feature
1:47:06that will tell us the order, the package
1:47:10of images that the client sent. We used
1:47:14the order of those images as a new
1:47:18feature for the voting model. So
1:47:21basically when the client takes a
1:47:23picture of the back, if the client takes
1:47:25this as the first picture, we will pass
1:47:28that's order zero. If the client took
1:47:30this as the second picture, we will say
1:47:33that's order one, etc. It turns out that
1:47:36most people when they send a package of
1:47:38images, they usually start here.
1:47:41Like people don't take this picture,
1:47:43upload it, and then take this picture,
1:47:45upload it, and then finally take this
1:47:47picture and upload it. At least not most
1:47:49people. So by adding the order that they
1:47:53use when uploading the images to the
1:47:55website, that allowed us to provide uh
1:48:01more weight to the votes of the brand
1:48:07and model coming from the first few
1:48:10pictures. with respect to the other
1:48:13picture. So basically we were making the
1:48:15assumption that the very first picture
1:48:17was going to provide the most amount of
1:48:19information then the second and third
1:48:22will be the second and then everything
1:48:24else will help but will not be decisive
1:48:30and by doing that we were able to
1:48:32increase the sort of like the capacity
1:48:34of our model to make good predictions in
1:48:36five six%. Okay, by the way, after we
1:48:39realized this, we asked the web team to
1:48:43change the interface to notch people to
1:48:46start by uploading the front and then
1:48:48uploading the back, etc. So, and by
1:48:50doing that, we were sort of like
1:48:53increasing the ability or the
1:48:54probability that people were were
1:48:56uploading the pictures in the right
1:48:58order. So, that's an example of feature
1:49:00engineering where you're adding
1:49:02something new to the problem. you're
1:49:03creating a new feature here that's going
1:49:06to help you make better decisions. Okay,
1:49:09so another sort of like category of
1:49:11feature engineering is vectorization. So
1:49:13this is when you turn text into numbers
1:49:17just to keep it simple. Okay, models
1:49:20work with numbers. They don't like text.
1:49:22So you need to find techniques that turn
1:49:24text into numbers. So let me give you
1:49:26one particular example here. So imagine
1:49:29that you have a data set with the name
1:49:32of the animal and you want to process
1:49:34that data set you know among other
1:49:36features but there is a feature that's
1:49:37called animal and you have cats and dogs
1:49:39and horses and whatever
1:49:42and you want to process that data set
1:49:44with your model. You have to turn these
1:49:47animals into numbers in order for your
1:49:49data set to for your model to work with
1:49:52it. And a very very simple technique is
1:49:55called label encoding. And label
1:49:57encoding is basically going to create
1:49:59this mapping, right? It's going to
1:50:02create this map that says, okay, so
1:50:03whatever I see a cat, turn that into a
1:50:06one and whatever I see a dog, turn it
1:50:09into a two, etc., etc., right? That's
1:50:12the idea. So what we want to do here or
1:50:15when you process when you take label
1:50:17encoding and process your data set, this
1:50:19is what you're going to get. you're
1:50:20going to get a new column that contains
1:50:23the mapping number, the encoded number
1:50:27instead of the actual text. And now this
1:50:31data set you can actually process with a
1:50:33model. Now can anybody tell me
1:50:36I mean this works by the way works very
1:50:38well but there are problems with this.
1:50:40Can anybody tell me what the problem is?
1:50:43Any idea why this could become
1:50:45problematic? Why don't we just do this
1:50:49every single time? So Diego is saying
1:50:51cardinality. Diego, cardinality is a big
1:50:54big word. Can you explain further what
1:50:57do you mean by that?
1:50:59>> Yes, of course. So we can have many
1:51:01multiple options for example cats, dogs,
1:51:04horses, I don't know worms, etc.
1:51:08So as long as that that that list draws
1:51:12the number of embedible encodings label
1:51:15encodings will grow also and we will
1:51:17have a really really sparse set of
1:51:19labels. Yeah. And here is the thing. So
1:51:22I'm going to add on top of what Diego
1:51:24just said to a model. These numbers here
1:51:29have a meaning. They are numerical
1:51:32values. So the model might look at this
1:51:36and say, well, this ID3 feature here or
1:51:39this ID row here, ID3 row
1:51:44somehow is three times more important
1:51:46than this one here because the magnitude
1:51:49of the number is three and the other one
1:51:52is one. So maybe maybe I should be
1:51:54paying more attention to this one here
1:51:58rather than this one here. See what the
1:52:00problem is? The bigger that distance is,
1:52:03as Diego said, as the cardinality rows,
1:52:05if I have a thousand animals, now we're
1:52:07going to have rows with a value of one
1:52:10and rows with the value of a thousand.
1:52:14My model will tend to pay more
1:52:16attention, especially if you're using,
1:52:18let's say, a neural network, to the rows
1:52:21with a thousand rather than rows with a
1:52:24one. We don't want that to happen. We
1:52:26want all all of our values to be
1:52:29standardized. And we're going to see
1:52:31that in another technique. But that's a
1:52:33problem with label encoding. How do we
1:52:35solve that? Well, there is a technique
1:52:37for example that's called one hot
1:52:38encoding that does not have that
1:52:40hierarchical problem because we can turn
1:52:43our animals or this column, we can turn
1:52:47it into multiple columns. So now instead
1:52:49of having an animal column, we're going
1:52:50to have is cat, is dog, is horse. And
1:52:54we're going to use a one or a zero to
1:52:57specify well if it's a cat it's going to
1:52:59have a one here and zero everyone else.
1:53:02If it's if it's a dog it's going to have
1:53:04a zero here and cat one in the dog and
1:53:07zero everywhere else. So as you can see
1:53:09in this particular case the model will
1:53:12never get confused than a horse is more
1:53:15valuable than a cat because there are no
1:53:18differences here. Okay. So you can have
1:53:19as many animals as you want and all of
1:53:22the rows are going to be zeros or ones.
1:53:25But cardinality is still a problem here.
1:53:28If you have many many animals, you're
1:53:31going to have a data set that with many
1:53:33many columns. You don't want that.
1:53:35That's going to be really really
1:53:36horrible. So you need a better technique
1:53:39to solve that. So here is how to or a
1:53:43different technique. Not how to solve
1:53:44that but is a different technique that
1:53:47is very very effective. is called target
1:53:49encoding. So with target encoding here,
1:53:52you're going to be sort of like doing a
1:53:54trick where you're going to be taking
1:53:56your original data set and you're going
1:53:58to be averaging the target of the data
1:54:02and using that as the encoded value. So
1:54:05what I mean by that is imagine that I
1:54:07have different IP addresses which are
1:54:10how many IP addresses can you have? It's
1:54:12just the number is just unbounded or
1:54:14just too many. So, I'm going to get
1:54:16every single value that's the same.
1:54:1817216 254.3. You can see it three times.
1:54:22That's what I put them in red. And I'm
1:54:24going to take the target value. The
1:54:26target is the value that you want to
1:54:27come up with. The value that you want to
1:54:29predict. I'm going to get the target
1:54:31value. I'm going to average that target
1:54:33value across all of those three
1:54:35examples. So, I'm going to add 5200 with
1:54:3937.99 with 5230 divided by three. That's
1:54:43the new IP address. That's the encoded
1:54:45IP address, okay, that I'm going to go
1:54:48with. So now I'm going to have 4743 and
1:54:50the target will be 52000. 4743, the
1:54:54target will be 3799.
1:54:56See how I turned this non-numerical
1:55:00value, which is an IP into a numerical
1:55:02value by just using target encoding. And
1:55:06then I'm going to do the same thing with
1:55:08blue. Okay, the blue IP here, I'm going
1:55:10to add these two values divided by two.
1:55:14That's going to be my new encoded column
1:55:17and that is going to be my new data set.
1:55:19Now there are I don't know if anybody's
1:55:22raising their hands. Usually people are
1:55:23just freaking out at this point. There
1:55:25are some things to consider when you're
1:55:28doing target encoding. So number one
1:55:30target encoding is very very effective
1:55:33but you need to have a data set that is
1:55:36not that uh is where every sample in the
1:55:41data set have a lot of representation in
1:55:43the data set. If you have a data set
1:55:46where you have only one animal or only
1:55:49one IP address that's the same and that
1:55:52IP address does not repeat multiple
1:55:54times. What's going to end up happening
1:55:57is that there's not going to be anything
1:55:58to combine here. So you're basically
1:56:01going to be showing your model the
1:56:04actual prediction value. So this value
1:56:06here will be 52000. This value here will
1:56:09be 5385.
1:56:11You're asking the model giving if I give
1:56:14you 5200 predict 52000 which is stupid.
1:56:18So if your data set is not dense enough
1:56:21target encoding is not going to work. If
1:56:23you have many many samples for each IP
1:56:26address, the combination of those are
1:56:29going to obscure the prediction value is
1:56:31going to work very very well. So that's
1:56:33the first thing that you need to keep in
1:56:35mind. So be very careful because you
1:56:38could be overfitting when using target
1:56:40encoding. You could overfit. The second
1:56:43thing to take into account is that the
1:56:45algorithm for target encoding when you
1:56:47go to an official implementation let's
1:56:50say in scikitlearn they also they just
1:56:53don't average numbers they also include
1:56:56a smoothing as well. It's basically
1:56:58adding some sort of like obscure value
1:57:01to the formula to help with overfitting.
1:57:06But anyway check it out uh because it's
1:57:09very very effective.
1:57:11All right, two more and we're done.
1:57:13Normalization, standardization. I
1:57:15mentioned this when we were talking
1:57:16about label encoding. You want your
1:57:20features to be homogeneous, to be around
1:57:24the same range. You do not want to be
1:57:28working with values here with the salary
1:57:31of a person in the $100,000 and the age
1:57:34of a person. That's 39. Just too much
1:57:38difference. Okay. There are two usually
1:57:40two algorithms that we use to sort of
1:57:42like make these values homogeneous. The
1:57:45first one is normalization. This is the
1:57:46formula of normalization. An example of
1:57:49that is when we are working with images,
1:57:52imagine that this is an image. Every
1:57:54pixel has a value between 0 and 255.
1:57:57Zero meaning black, 255 meaning uh
1:58:00white. Whenever we want to process an
1:58:03image, black and white image like this,
1:58:06we normalize the image first and we
1:58:09don't want to work with some pixels that
1:58:12are equal to one and some pixels that
1:58:14are equals to 250 just too much
1:58:17difference. So what we do is we
1:58:18normalize using this formula and now
1:58:21every single value will be between zero
1:58:23and one very very small range. That's
1:58:26what we want. our models are going to
1:58:28work way better in this case here. Okay,
1:58:32another example is using
1:58:33standardization.
1:58:35This is the formula here. Imagine that
1:58:37we're working with salaries and ages and
1:58:40this is sort of like the distribution
1:58:42here of salary and ages. Uh when we put
1:58:44them in a chart, we could using this
1:58:47formula standardize these values because
1:58:50salaries again is in the tenth of
1:58:52thousands and ages is in the tens,
1:58:55right? It's just
1:58:57They're not going to be they're not
1:58:58homogeneous. But if we standardize we
1:59:02can get our salaries between minus.2
1:59:05and.3 and our edges between minus.4 and
1:59:09point.4 which is very very you know same
1:59:12range pretty much. And our model is
1:59:15going to work much better this way. And
1:59:18the distribution obviously is going to
1:59:19stay the same. Nothing changes. You're
1:59:21just changing the numerical magnitude of
1:59:23the values. Okay. That's another process
1:59:26you have to go through when doing
1:59:27feature engineering on your data set.
1:59:29Okay. And finally, the final one is you
1:59:32have to handle missing values. Okay. So
1:59:35it's very normal. You get a data set,
1:59:37you collected the data and a bunch of
1:59:40that data is not there. It's not
1:59:42present. You have to find ways to deal
1:59:45with that because when you pass all of
1:59:48that data to a model, the model doesn't
1:59:49know what to do with a missing value.
1:59:51Doesn't know how to replace that. you
1:59:52have to do you have to make those
1:59:54decisions. So multiple ways to handle
1:59:57missing values. So one of them for
1:59:58example is removing the columns that
2:00:01contain missing values. If you have a
2:00:03feature and that feature is very
2:00:06incomplete like in this case you get one
2:00:08example here male one example here
2:00:10female but most of the data doesn't
2:00:12contain anything. You can just get rid
2:00:14of that column. Okay maybe that column
2:00:17is not going to be that important. You
2:00:18get rid of the column you move on.
2:00:20Another way is just to remove the rows
2:00:23that have the missing values. In this
2:00:25particular case, there is only one row
2:00:27here with a missing value. And in your
2:00:29new data set, you can just get rid of
2:00:31it. So just remove every column that
2:00:35doesn't have that value. Another way is
2:00:38replacing the missing values with uh you
2:00:42know a good option. So basically you're
2:00:45imputing a missing value. That's what
2:00:47imputation mix uh means.
2:00:50this particular example, we're using the
2:00:53most frequent value to replace sorry or
2:00:56to replace the missing value. So we're
2:00:58missing the sex here, but the most
2:01:01frequent value in this data set is male.
2:01:04See, male, male, male, male. So we are
2:01:07going to assume that any missing values
2:01:10are going to be males. You do the
2:01:12imputation [clears throat] and now you
2:01:14pass your entire data set. Okay, these
2:01:16are different techniques that you could
2:01:18use to replace missing values. Something
2:01:22important, never replace missing values
2:01:26without asking why those values are
2:01:29missing in the first place. Okay, so
2:01:32anecdote here, 2016 United States
2:01:36election, Hillary Clinton versus Donald
2:01:39Trump, uh pollsters started asking
2:01:42people, who are you voting for? A lot of
2:01:45people started saying I don't want to
2:01:47say I don't want to participate. Okay.
2:01:51So pollsters assume that if you didn't
2:01:53want to participate they did not want to
2:01:55just include you. So why would I include
2:01:57you if you're not giving me the answer
2:01:59that I'm asking? What they failed to
2:02:02realized or what many of them failed to
2:02:04realize is there was a reason behind so
2:02:08many people saying I do not want to tell
2:02:11you. I do not want to participate. Okay,
2:02:15Donald Trump ended winning that
2:02:16election. Um, a lot of people did not
2:02:20want to participate because they were
2:02:22going to vote for Donald Trump, but
2:02:23Trump was the most polarizing candidate.
2:02:27So, they were not uh happy to tell that
2:02:31in public. They were supporting Donald
2:02:33Trump in public. So this is just one
2:02:35example of what not trying to understand
2:02:40why the missing value in the first place
2:02:42is a mistake. If you dig deeper, why are
2:02:46you not giving me this answer? Let's
2:02:47think about that. Why are am I getting a
2:02:51missing value in this column? You have
2:02:53to think this deeper. You have to think
2:02:55about that uh more carefully so you can
2:02:59actually fix that issue going on. Uh
2:03:01final slide.
2:03:04if I can get there. So, this is uh this
2:03:08is a little map with spot. We're running
2:03:10a mission inside a warehouse. This is a
2:03:12real example.
2:03:14And that mission was for a spot to use a
2:03:18camera, take a picture of analog gauges
2:03:22that are, you know, next to equipment
2:03:25and automatically read the value on
2:03:28those analog gauges. So a pressure tank
2:03:31spot will take a picture of the gauge
2:03:33and we'll come up and say okay so it's
2:03:35120 PSI that's what we see. So this is
2:03:39sort of like the data that comes out of
2:03:41the mission results. Okay so we get the
2:03:43image we get the location where that
2:03:45image is where spot took that picture
2:03:48and we get the reading. But imagine that
2:03:50you get sort of like these mission
2:03:52results and you realize that the
2:03:54warehouse here does not come with an
2:03:57image and with a reading. If you had an
2:04:00automatic process to just replace the
2:04:03missing values here, okay, you wouldn't
2:04:06miss the fact that well there's
2:04:07something going on in the warehouse. And
2:04:09this is sort of like an obvious example
2:04:10when you see it like this. I promise you
2:04:13was not that obvious when we were
2:04:15dealing with this. So if you analyze the
2:04:17data, well clearly something is
2:04:19happening in the warehouse. So maybe the
2:04:21warehouse 80% of the missing values are
2:04:24coming from the warehouse or nine out of
2:04:2610 times we go to the warehouse, we get
2:04:28a missing value. Okay, so this is
2:04:30something that's worth investigating.
2:04:32Let's go to the warehouse and see what's
2:04:34going on. Okay, so maybe somebody left a
2:04:36box in front of the the gauge or maybe
2:04:39the gauge is getting at the time
2:04:42that we're running the missions because
2:04:43some other thing is going on there and
2:04:46spot cannot take a good picture of that
2:04:48image. So whatever it is, you have to
2:04:51analyze the data before making a
2:04:53decision what to do with the missing
2:04:55values. By the way, what we did in this
2:04:57particular case, just in case it's
2:04:59interesting, we had a model, machine
2:05:01learning model that will do imputation
2:05:03of missing values. That model, the goal
2:05:06of that model was to analyze past
2:05:08readings and to sort of like predict
2:05:10what the new reading would be like and
2:05:13it will replace missing values with that
2:05:16value. So just in case that's
2:05:18interesting. All right. So this is the
2:05:20final slide that I have. This sort of
2:05:23like covers again from the discovery
2:05:25phase sometimes all the way to the first
2:05:28iteration where I start dealing with the
2:05:31data to build my first model. Sometimes
2:05:35this is just discovery depending on the
2:05:37project. So at the end of discovery I'm
2:05:38going to have labels or at least a way
2:05:41to gather those labels or to come up
2:05:44with those labels. I'm going to have
2:05:45data. I'm going to have a pretty good
2:05:46idea of the feature engineering that I'm
2:05:49going to have to do for that data set.
2:05:52I'm going to have a good framing for my
2:05:54problem and I probably going to have a
2:05:56prototype or an idea to build that
2:05:58prototype.
2:06:00Questions?
2:06:13anything goes.
2:06:15>> I was thinking about how did you weight
2:06:17the image like when you add the order
2:06:21how how to just did you tell them did
2:06:24you tell them about that the first image
2:06:26was more weight than the other ones.
2:06:30>> Oh, okay. So, just to keep it simple, it
2:06:32was a little bit more complex than this,
2:06:35but just to keep it simple, imagine that
2:06:37you have three images. Okay? And what
2:06:39I'm going to do is three images. Each
2:06:41one of those images is going to have an
2:06:44answer. Okay? So each image is going to
2:06:46tell me I'm going to be a class A or
2:06:50class B. Each image is, you know, you're
2:06:52going to get a class from each image.
2:06:54>> So what I do is I I'm going to assign a
2:06:57weight to each of the images.
2:06:59>> So the first image I'm going to say you
2:07:02represent 50% of the weight and the
2:07:05second image you're going to be 25% and
2:07:07the third image is going to be 25%. So
2:07:10by doing that by creating the weight a
2:07:13different weight for each image now I
2:07:16can compute the final prediction by
2:07:18taking 50% of whatever answer the first
2:07:21image gave me 25% of the second and 25%
2:07:25of the okay so that's kind of
2:07:28handcrafted the decision in the soft max
2:07:32>> that is correct yes if you want to be
2:07:34more specific I will do that with the
2:07:36soft max and the weights and the weights
2:07:39We will compute over time. We had a set
2:07:42of weights and we will sort of like came
2:07:45up with those weights automatically. So
2:07:47basically train for those weights. But
2:07:50those weights were based on the n the
2:07:53image number. Okay. So if you were the
2:07:56first image your weight was going to be
2:07:58higher. Again we were training for is it
2:08:01going to be 38 or it's going to be 36.
2:08:04So we were sort of like optimizing those
2:08:06but overall the weight was determined by
2:08:09the number in the order. Okay.
2:08:14>> Got it. So you first so first you you
2:08:16start with a fixed weight and then you
2:08:19then you optimize this weight to
2:08:21dynamically be selected.
2:08:24>> What what we did was just we started
2:08:26with fixed weights and then we changed
2:08:29the weights. We basically learned what
2:08:31good weights were based on the data that
2:08:32we had. So we had a training set and we
2:08:35were it's like just training a model to
2:08:37determine what the optimal weights are.
2:08:38>> Okay.
2:08:39>> And we were rerunning that over time
2:08:41like I don't know maybe every month. Hey
2:08:43let's just optimize for the weights and
2:08:46let's come up with the weights the
2:08:47better weight
2:08:48>> to measure. Yeah. to measure how
2:08:50important the first one the first image
2:08:52was
2:08:52>> because you had the you had the
2:08:54pre-intuition that the first one is
2:08:56better but you couldn't tell for sure
2:08:58how much
2:08:59>> I don't know how better it was or how
2:09:01much we should be trusting it okay so I
2:09:04did not know that yeah
2:09:06>> sure yeah thank you
2:09:10>> uh Iban Ibano
2:09:13drums
2:09:15h how do you call it I mean how is the
2:09:19TXO it means cho.
2:09:22>> Oh, sure.
2:09:25>> Okay, cool.
2:09:27>> Uh, yeah. So, you mentioned that this
2:09:29will be your first iteration the whole
2:09:31process. So, I guess I I will be
2:09:33interesting to know how do you do how do
2:09:36you document all this process? Do you
2:09:38use a boards? Do you use a wiki or or
2:09:41what's what's your process behind all
2:09:43this? Oh, we usually so I wish I had a
2:09:47saying on that. All of this usually
2:09:50happens on the client's tools. So it's a
2:09:52if I have a saying I usually document
2:09:54things in GitHub because it's very
2:09:56simple and everyone has access to it and
2:09:58I just create issues and work you know
2:10:00you in GitHub you can create these uh
2:10:03boards where you can have multiple
2:10:05columns and you can assign tasks to each
2:10:08column and all of that or you can just
2:10:10create issues there but anything
2:10:12actually works. uh usually what happens
2:10:15is that the clients dictate where I
2:10:17should be providing or or ending
2:10:19information they have Jira clashen or
2:10:22whatever tools they use uh bunch of
2:10:24clients use base camp it doesn't really
2:10:26matter the important thing here is that
2:10:28you capture the right information and
2:10:30the process and what not and who's doing
2:10:32what and what's left to do that's yeah
2:10:37>> thanks
2:10:39what
2:10:44Santiago, very cool, very cool uh
2:10:47practitioner techniques, you know, how
2:10:50to normalize the data. For example, the
2:10:52notes that you made about like how the
2:10:54larger distance between values impacts
2:10:58model performance. Would you have papers
2:11:00somewhere or is it just basically common
2:11:02sense at this stage like you know for
2:11:04someone who does it for a living? I mean
2:11:06why why a larger value will will
2:11:09>> no would you have is this like obvious
2:11:12enough that there is no point in reading
2:11:14papers anymore about this or would you
2:11:16have papers somewhere noted that we
2:11:18could kind of take a look at
2:11:20>> I can tell you exactly I I mean there
2:11:22there might be I don't think there is a
2:11:24paper out there that explains why a
2:11:26higher value will impact the performance
2:11:28but I can tell you why it's going to
2:11:29impact the performance. So if you have
2:11:32uh
2:11:34I'm writing stuck here.
2:11:36>> What I meant is more also about like
2:11:39that you gave like several hints, right?
2:11:42And they were interesting.
2:11:45>> So
2:11:45>> So look at So this is this here is the
2:11:49formula of a line, right? This is just
2:11:51like the you know you get the y value.
2:11:54It's it's the the how do you call this
2:11:56in English? I forgot. slope times the
2:11:58the x value plus a bias or or you know
2:12:02this is al also the formula that we have
2:12:04in a neural network to compute the value
2:12:06of a network of of a neuron. So the
2:12:08value of a neuron will be the set of
2:12:11weights that are predefined times the
2:12:14input of that neurons plus a bias. Now I
2:12:18want you to imagine that the input of
2:12:20you have two columns in your data set
2:12:23the salary and the age. Okay. So if I
2:12:28have a neuron, let me try to make this
2:12:34a piece of paper here where this will
2:12:37make sense.
2:12:47So we have this neural network. Okay.
2:12:49And there are two inputs here. And I'm
2:12:51going to assign the salary to one input.
2:12:56and that's going to be 10,000.
2:12:58And I'm going to assign the age
2:13:01to another input.
2:13:03You get something like this. So the
2:13:05inputs to my network are one. The first
2:13:09the first input is going to be the
2:13:10salary which is going to be a very large
2:13:12number and the second input is going to
2:13:15be the age which is going to be a
2:13:16smaller number. And now I want to
2:13:18compute the value of a neuron. again is
2:13:23going to be the set of weights that are
2:13:26for my network times the input. So
2:13:29assuming the set of weights are at the
2:13:31same value. So they're homogeneous. It's
2:13:33going to be 0.1 and 0 2 and point n. You
2:13:36can see that in that multiplication
2:13:38the x value is it's you know if we get
2:13:42the value of a non and by the way there
2:13:44is also uh an activation function here.
2:13:47I'm going to forget that for a second.
2:13:49But if I have this
2:13:53If I have the set of weights, sorry, I
2:13:55cannot see where I'm pointing. If I have
2:13:57the set of weights, the value of the
2:13:59neuron is going to be the set of weights
2:14:00times the input. When that input grows,
2:14:04so if I have 10,000,
2:14:07that value, the y value is going to be
2:14:09way higher than when the input is very
2:14:13small, which is 32. So what's going to
2:14:15happen is when you have this in a data
2:14:17set all of the neurons that use that
2:14:21input here that are exposed that input
2:14:24the value will grow in comparison to the
2:14:27neurons that are exposed to the other
2:14:30features that are smaller. So the model
2:14:33when I when the value grows in inside
2:14:36the neural network and if if you follow
2:14:39the process of a neural network any
2:14:40values that are really small the the
2:14:43network is not going to do anything with
2:14:44them and it's going to start activating
2:14:47that's what we call a neuron gets more
2:14:49activation when the value is larger.
2:14:51That's what's happening. Then the the
2:14:52network is paying more attention to
2:14:55what's happening to the salary. It's not
2:14:57paying enough attention to what's
2:14:59happening with the age. By making these
2:15:02two values
2:15:04homogeneous within the same range, you
2:15:08avoid that problem. Now the network has
2:15:10to pay attention to all of the features.
2:15:12So that's that's what's happening. Uh by
2:15:14the way, that would not be the case if
2:15:16you're working, let's say, with a
2:15:18decision tree. like a decision tree will
2:15:20not have that problem because it's not
2:15:23using the same mechanism as the neural
2:15:25network. It's not multiplying by those
2:15:26values. So it wouldn't care about the
2:15:29homogeneous part. But the but making the
2:15:32values homogeneous does not hurt. So you
2:15:34always do it and now you have the
2:15:36flexibility of using whatever model and
2:15:38you don't care about whether the model
2:15:41uh is sensible to that or not. You can
2:15:43just use the whatever model you have and
2:15:45it's going to work the same.
2:15:49Super cool. Thanks.
2:15:54What else?
2:15:57How many of you are going to How many of
2:15:59you is AI going to replace?
2:16:03What is plan B?
2:16:06They going to the you know build and you
2:16:10know becoming I don't know a carpenter
2:16:11or something. What is plan B for us?
2:16:14I was smiling when you mentioned Pog
2:16:16Graham the the the essay because I'm
2:16:19pretty sure he's already working on a
2:16:21you know updated version because like
2:16:24you know what doesn't scale is
2:16:27redefining in front of our very eyes
2:16:29like the term is no longer what it used
2:16:32to be 10 years ago or 20 years ago.
2:16:35>> So yeah.
2:16:39>> Yeah. and and hearing you, you know,
2:16:41saying that like you you're thinking of
2:16:44like doing something else. I almost b
2:16:46laughing. Yep.
2:16:51>> Yeah,
2:16:54man. I don't know. I don't know what
2:16:55plan B is. Uh I don't have plan B. So, I
2:16:59hope we never come to knitting a plan B.
2:17:04Somebody says here, "Well, I'm going to
2:17:06be working on HVAC
2:17:08systems.
2:17:10I mean that's that's a that's a cool way
2:17:12of making a living. I suppose my
2:17:14personal hope is that it will really
2:17:17create like all new opportunities. It's
2:17:20just that the intering period is going
2:17:22to be super bumpy but like I don't know
2:17:2520 years whatever the time frames are
2:17:27very hard because it has accelerated so
2:17:29much. Uh but like in some time we'll be
2:17:34much better off like imagine what's
2:17:36going to happen if we suddenly have free
2:17:39power or like we actually fly to the
2:17:42moon on scheduled basis and stuff like
2:17:44this where crisper is going to take off
2:17:47for real for ordinary humans like this
2:17:50is like we can't really even think about
2:17:52this right now.
2:17:54>> So fingers crossed right?
2:17:56>> Yeah I mean I I I joke with this but uh
2:18:00I I don't think it's I don't think it's
2:18:03it's not going to be a good deal.
2:18:04Obviously, that doesn't mean that AI is
2:18:06not going to disrupt. What I mean is
2:18:09that for people for professionals who've
2:18:12been doing this or or people who are
2:18:16investing in in their education and
2:18:18learning, AI is not going to replace
2:18:21them. If if you go outside Twitter, if
2:18:24you go outside LinkedIn, if you go
2:18:26outside the bubble, you will realize how
2:18:30how far back the entire world is. Like
2:18:33there are people right now. I just had
2:18:36to apply for my daughter's visa. She's
2:18:38going to the UK, studying there.
2:18:42The the entire website was made in I
2:18:44don't know, maybe 1960 before the
2:18:46internet. that I mean you look at that
2:18:48website and you're like nobody's going
2:18:50to replace us ever. These people don't
2:18:53even know what building a website looks
2:18:54like. Let alone everything is going to
2:18:57be automated by no not really. It's not
2:19:00I
2:19:01plus we are getting to the point where
2:19:04yes these models are great and these
2:19:06models are doing a lot of it but
2:19:08everyone realizes that no you cannot let
2:19:12these models just
2:19:14they're not replacing us so far is that
2:19:17yes coding they're doing but coding is
2:19:20not building software. Coding is just a
2:19:22small percentage of what building
2:19:24software looks like. Okay,
2:19:27I know that before it was an important
2:19:31uh task that most people could not do.
2:19:35But people who couldn't write
2:19:37[clears throat] software, who couldn't
2:19:38write code before, they're not
2:19:40developers now. They're not going to
2:19:41become developers. My wife is not going
2:19:43to become a developer. She doesn't know
2:19:45anything about systems, engineering,
2:19:47software. She's not gonna be prompting
2:19:50models saying build me an app. That
2:19:53that's not the way works. I've heard
2:19:56people online saying when everyone can
2:19:59build an app for their phones. Do you
2:20:02really believe that
2:20:05your friends that have never touch a
2:20:07computer before, they're going to be
2:20:09sitting down building apps for their
2:20:12phone? Come on. Come on. We pay for
2:20:15people to cook for us. people get a
2:20:18little bit of money and they buy a chef.
2:20:20They don't even want to cook food for
2:20:22them. Now imagine they're going to be
2:20:24building apps for them. That's it's not
2:20:27everyone knows how to cook. Everyone
2:20:29knows how to cook or at least everyone
2:20:31has the means to learn how to cook and
2:20:34nobody wants to do it and people still
2:20:36go to restaurants. So I do not sorry I'm
2:20:38I'm passionate about this. Uh, I do not
2:20:41think that the end of the world is near
2:20:45or that we're going to need a plan B.
2:20:47What I do think though is that you need
2:20:49to keep improving. And I don't think
2:20:51this is different than before. You need
2:20:53to keep learning. You need to keep
2:20:55getting better at the tools that the
2:20:57current state of the art that is
2:20:59education is the only way you are going
2:21:02to be better off tomorrow. That's the
2:21:04only thing that I know. Yeah. So anyway,
2:21:09>> hey the
2:21:11>> Oh, sorry.
2:21:12>> Go on, please.
2:21:14>> Yeah, I want to say every IT meme ends
2:21:18up with agriculture. So maybe that's the
2:21:20plan B, but even their robots can
2:21:23replace us. But my question was about
2:21:25the if if on your system uh I so I saw
2:21:29the
2:21:31uh on the slides the part of the
2:21:33normalization and uh I was wondering if
2:21:37do you how how do you handle al also
2:21:39with ambiguity in in the in the
2:21:41contextualization because that's why uh
2:21:43transformers came up came up right
2:21:45because of apple can mean apple in
2:21:47agriculture but also can mean apple in a
2:21:49technology for example right and how do
2:21:52you handle this in this
2:21:54machine learning pipelines
2:21:56>> that it's it's very specific to the
2:21:58problem that you're working on. So uh
2:22:02all of the models that I've been
2:22:03involved with, I've never built an LLM
2:22:05before, built from scratch, another
2:22:06before. So I've never had to deal with
2:22:08that problem specifically of two words
2:22:11meaning Apple the company or Apple the
2:22:13fruit or bank from river bank or bank
2:22:17from financial institution. I haven't
2:22:19had to build solutions for that. But
2:22:22there are of course depending on the
2:22:25problem that you are working on there
2:22:27are pieces of information that might
2:22:29have different meanings depending on the
2:22:31context. The solution to that is to
2:22:33include the context. So remember that I
2:22:36had a slide that said you have to add as
2:22:39much metadata that contextualizes
2:22:41one piece of information as possible.
2:22:44And whenever you find yourself in a
2:22:46problem where that is is an issue, where
2:22:49the the the meaning of something is an
2:22:51issue, you have to go to the to the meta
2:22:53data and include that metadata as part
2:22:55of your data, your official data. So
2:22:57maybe the image is not enough for you to
2:23:00solve the problem. And this has happened
2:23:01all the time where you take two
2:23:03pictures, but it's not enough for the
2:23:06image to give an answer because it
2:23:08really depends on whether you took the
2:23:10picture during daytime or nighttime. So
2:23:13now what you do in order to provide that
2:23:15context is you add another feature that
2:23:17says time of the picture. Now you have
2:23:20the image and the time you took that
2:23:22picture because that's important for the
2:23:25model to do something different for that
2:23:27picture. So anyway, point being you add
2:23:30the context. So that's what you have to
2:23:32do. But for you to add the context, you
2:23:34first have to have that context. So when
2:23:38you're collecting the data, make sure
2:23:40you are including as much metadata as
2:23:42possible so you can actually solve that
2:23:44problem later on. Hopefully that makes
2:23:46sense.
2:23:47>> Yes, sure. And and does it affect uh
2:23:50tagging as well?
2:23:53>> Uh it will. So you're usually going to
2:23:56realize that you need that context when
2:23:58you are doing labeling. So way before
2:24:00your model is not going to tell you that
2:24:01the model is confused. Usually when
2:24:03you're doing labeling, the people who
2:24:05are labeling are going to tell you, I
2:24:07don't know what to tell you here. Like
2:24:09what is the answer for this?
2:24:11>> Makes [snorts] sense.
2:24:12>> Yeah. Yeah.
2:24:13>> Thanks.
2:24:20>> What else?
2:24:26I see here from SA. I'm starting my PhD
2:24:29this year, so it's encouraging to hear
2:24:31this. I completely agree that education
2:24:33needs to improve so people can better
2:24:35understand and develop this field. Yep.
2:24:39Uh Mansour, what's up?
2:24:41Uh yes I have a question around
2:24:44synthetic data and whether one one thing
2:24:47is are we going to talk about it and
2:24:50second is
2:24:52how how much synthetic data can we
2:24:54actually uh how useful it is because
2:24:57just to give you context here in Africa
2:24:59whenever you do things like machine
2:25:01learning and uh you know AI number one
2:25:04issues you're having actually is data
2:25:06right a lot of the things we we try to
2:25:09build we don't have enough data not
2:25:11quality data.
2:25:12>> So for the things I'm trying to build, I
2:25:15I'm really looking into maybe synthetic
2:25:18data as a way to overcome that
2:25:20challenge. But now can you go and abuse
2:25:23of synthetic data and actually have good
2:25:27results based off of that? Maybe from
2:25:29your experience or the industry standard
2:25:32is very like a cap or that says for
2:25:35example only 30% of your training data
2:25:38can be sent data. Beyond that you get
2:25:40garbage.
2:25:41>> Yeah, there is no number. Actually the
2:25:43the question you need to answer is how
2:25:45good is that synthetic data? How
2:25:48indistinguishable it is from real data?
2:25:51If you have a process that can generate
2:25:54synthetic data that's as good as real
2:25:57data, it's you you can add unlimited
2:26:01synthetic data. The problem is that not
2:26:03everyone has that process. And if the
2:26:06synthetic data that you're generating
2:26:08differs from what the real data looks
2:26:10like, you're going to be limited in how
2:26:12much synthetic data you can use before
2:26:15your model learns how the synthetic data
2:26:17looks like and forgets about what the
2:26:19real data looks like. So again, the
2:26:21answer is going to be very different
2:26:23depending on what you're doing. Let me
2:26:24give you one example. Imagine that you
2:26:27have sensors
2:26:29and that capture the temperature. Okay?
2:26:32So the sensor is going to go a real
2:26:34sensor capturing temperature. It's going
2:26:37to be working very very differently than
2:26:40a process that generates temperatures
2:26:42randomly. Okay. The sensor might break,
2:26:44the sensor might capture you know the
2:26:47weather correctly, the random values or
2:26:49not. So if your process to generate
2:26:52synthetic temperatures is just random,
2:26:55that's not going to be good enough like
2:26:57uh it's not going to be as good as the
2:27:00real sensor. But you might have a
2:27:02process that generates good data that is
2:27:06not necessarily real, but maybe you have
2:27:09sensors deployed in a similar location,
2:27:11similar weather, maybe even in the same
2:27:14location, and you're using that to
2:27:16generate more data. Maybe that's good
2:27:18enough. So it really depends for you.
2:27:20Yeah. But you can use it is is fine.
2:27:22Yeah.
2:27:28What else?
2:27:31As a reminder on Wednesday, remember we
2:27:34have office hours. You don't have to
2:27:37come, but if you if you do, we're just
2:27:39gonna no agenda. We're just going to be
2:27:41talking about whatever. If you have
2:27:42questions, you can show up. We can talk
2:27:45about business. I don't know, social
2:27:47media, the moon, whatever you guys
2:27:50decide that we should be talking on. So,
2:27:53plan B, that type of stuff. I Hey, I
2:27:56like to take photos, so that might be I
2:27:58don't know, maybe I become a
2:27:59photographer.
2:28:01I don't think a lot of photographer make
2:28:03a lot of money, but anyway, maybe that's
2:28:05plan B for me.
2:28:10Sebastian, what's up?
2:28:12>> Listen, Dale, what's up? Uh well
2:28:16I spend more I spend more time on
2:28:18Twitter than I would like to admit to be
2:28:20honest. Uh but I recently saw that you
2:28:24took a stand regarding not reading AI
2:28:26generated code anymore and it's
2:28:31well at least I kind of had the feeling
2:28:33that it uh started a heated discussion
2:28:37because I saw that GMO from Bol
2:28:41regarding that matter and I would just
2:28:44like to know like if you could elaborate
2:28:46more on that. Yeah. What are you
2:28:48thinking?
2:28:49>> Yeah. So he and I have exactly the same
2:28:52point of view. And the problem is that
2:28:55we framed it in very different ways. But
2:28:57if you read his post, I don't know if
2:28:58you have it in front of you. If you read
2:29:01his post,
2:29:03>> yes, I'm going to find it. It should be
2:29:04very easy to find because
2:29:07>> it turned kind of viral.
2:29:10>> Like do you if you have it in front of
2:29:12you, do you mind pasting it here so I
2:29:14can just go and look at it?
2:29:16>> Yeah. Give me a second. I'm just
2:29:18>> let me see if I can find it here.
2:29:27>> Okay. Yeah, I just found it. I can share
2:29:29the
2:29:31>> if you share it here.
2:29:33>> Yeah.
2:29:39>> Okay, there we go. Let me open it here.
2:29:41All right. So he says the the key here
2:29:44the key here is in the first sentence.
2:29:47That's it. He says if you are not
2:29:49reading the code,
2:29:52whether explicitly
2:29:55or through agentic inquire that's it.
2:29:59>> Okay. Basically, he's saying if you're
2:30:01asking an agent, build this and that's
2:30:05it and you're just deploying whatever
2:30:07the agent built then and then he goes
2:30:10into the rest of the post. Okay?
2:30:13>> He's telling you that you should be
2:30:15reviewing that code, not necessarily
2:30:18explicitly, meaning sitting down and
2:30:21reading the lines and thinking about the
2:30:23lines. And that's it. He and I have
2:30:26exactly the same opinion. I said, "I'm
2:30:30not reading the code manually. I'm not
2:30:34sitting down going through 10,000
2:30:38autogenerated lines of code trying to
2:30:41understand what the agent did. And if
2:30:44you are, then a bunch of things are
2:30:47true. Either one, you're not generating
2:30:50that much code. Two, you're not moving
2:30:53fast enough. There is no way I can give
2:30:56you 10,000 lines of code for and you are
2:30:59going to read through all of them and
2:31:01verify all of them and think about the
2:31:02mental model that all of those lines are
2:31:04doing fast enough. There's no way it's
2:31:07going to take you a ton of time to do
2:31:09that. Right? So he and I have the same
2:31:12opinion. The thing is that Twitter is
2:31:15the place where nuance is lost.
2:31:19>> Lost. So, if you go to the post to what
2:31:22I I posted something today
2:31:25that says people hate that many of us
2:31:28I'm going to paste a link here.
2:31:33Uh let me see where where do I do that?
2:31:36Here we go. So, this is what I posted
2:31:37today. Okay.
2:31:39And I say people hate that many of us
2:31:42aren't reading AI code anymore. That's
2:31:44the post. That's how it says. But if you
2:31:46keep reading here, you're gonna realize
2:31:49that I'm explaining
2:31:52that
2:31:54I'm just referring to reading lines of
2:31:56code, but I'm verifying the code. I have
2:32:01unit tests, acceptant tests, verifi code
2:32:05reviews, automated code reviews. I have
2:32:07a bunch of layers that are helping me
2:32:11understand whether the code is working
2:32:13or not. I'm just not reading the code
2:32:15anymore. I was reading the code. I'm not
2:32:18anymore. I can't I It just It's not
2:32:20worth it to me or to anybody. Okay, this
2:32:24is just no way that's worth it. But
2:32:26again, nuance is lost. And you say, "I'm
2:32:29not reading the code anymore." AND
2:32:30EVERYONE IS LIKE, "HOLY YOU
2:32:32somebody uh found my website. I have a
2:32:36website linked on Twitter." And that
2:32:38person opened my website and that person
2:32:40told me, "Well, of course you're not
2:32:42reading your code. your website and
2:32:44Who cares about your website?
2:32:47That's the it's just a landing page. It
2:32:49doesn't even work correctly. And I'm
2:32:51like, dude, tell me why so I can fix it.
2:32:53Yeah. Anyway, so that's that's just
2:32:56Twitter. I'm used to it. But yeah, uh so
2:32:59that's my stance. I should you be
2:33:02reading your code? I'm going to say,
2:33:03well, it depends. Are you producing
2:33:06enough code? If all that you're doing is
2:33:09just maybe one function at a time, uh
2:33:11you're doing most that's fine and you
2:33:13you can read the code, that's fine. It's
2:33:16a matter of the signal versus noise
2:33:18ratio. Okay. So, as these models have
2:33:21gotten better and better and better,
2:33:23what we realize is it's not that these
2:33:25models do not make mistakes, is that it
2:33:27takes a lot of time to find those
2:33:31mistakes. Okay? because it's 10,000
2:33:33lines of code and maybe there is a bug
2:33:35there or two a couple of bugs there. So
2:33:37the question we need to answer is is
2:33:39there a more efficient way to identify
2:33:43those mistakes
2:33:44than just asking somebody sit down and
2:33:47read those 10,000 lines of code. when
2:33:50you are forcing your team to read the
2:33:53the lines of code, what actually needs
2:33:56to happen is that now you need to
2:33:57produce less code because there nobody's
2:34:00going to sit down and read 10 10,000
2:34:02lines of code. So you are
2:34:05uh putting a cap on how much progress
2:34:09you can make just so your people can
2:34:12find the bugs really really quick. I
2:34:14don't think that's the most efficient
2:34:16way of doing it. Obviously, this will
2:34:18vary depending on the software that
2:34:19you're doing. If you're creating a
2:34:21medical system that people's lives are
2:34:24going to be at stake, well, maybe you
2:34:26should not do what I'm doing right now.
2:34:28But for any regular stuff, maybe what
2:34:30you want to do is maximize how much you
2:34:33can produce, but identify ways or design
2:34:36ways to verify that that is good. And
2:34:38that's it. That's good enough. So,
2:34:40anyway, that's that's my take on this.
2:34:42Yeah.
2:34:46>> Thank you.
2:34:47Okay. So, uh I see something here. Let
2:34:50me see. I So, Uncle B modding said
2:34:54something. I'm not reading the tweet,
2:34:56but I'm I'm looking at your the LDR. It
2:34:58says, "The way I understand it, no need
2:35:00to read anymore. Better to focus on
2:35:02setting and enforcing boundaries and
2:35:04ironing out harnesses." That's exactly
2:35:06right. So, again, reading the code, it's
2:35:10inefficient.
2:35:12And if you're not there, it's because
2:35:15you're not producing enough code. That's
2:35:16it. It's just there's no other way. So,
2:35:20how can we stop reading the code and
2:35:22instead use that time to find more
2:35:24efficient ways to verify the product?
2:35:26That's it. That's the whole idea. This
2:35:29is the same. By the way, we've been here
2:35:31before. This is just it's controversial
2:35:34because it's just because we like to be
2:35:36enraged. But when we went from unit
2:35:39testing to
2:35:42uh integration testing, it's sort of
2:35:44like the same jump in abstraction where
2:35:46in unit testing you're testing every
2:35:49line of code that you're typing. You're
2:35:51typing code, you're writing tests that
2:35:53test individual units of your code. When
2:35:57you go to integration testing, now
2:35:59you're taking you're you're moving up
2:36:01one level where you don't care about the
2:36:04code itself. you care about the
2:36:05integration of different components. Uh
2:36:07maybe in acceptant testing, you're just
2:36:10looking at your overall system, how the
2:36:11overall system is working. Um you don't
2:36:14care about the code, you care about an
2:36:16higher level. This is the same thing.
2:36:18We're moving up the abstraction layer
2:36:20here in order to verify that the overall
2:36:24system is working. I do not understand
2:36:28people telling me, well, you have to
2:36:30understand every line of code because if
2:36:32you do not, you're stupid. Well, maybe
2:36:34that's for you, not for me. Not useful
2:36:36for me. So, anyway, yeah, that's where
2:36:38we are.
2:36:45Uh, Santi, I have a quick question with
2:36:47regards to this like how do you bound
2:36:51the
2:36:53request you ask to the agents like for
2:36:55example in case of like chachd 5.6 six
2:36:58if I gave it like hey review this Jira
2:37:01story I did and my code base is large
2:37:03enough and it's old enough what it does
2:37:06it's like it gives me goes to like very
2:37:10sometimes like very minuscule of
2:37:12possible error scenario that could
2:37:14happen and it told us like hey this is a
2:37:16high priority thing you need to fix it
2:37:18out and if I tied it up with my like
2:37:21developer agent I tied it up with my
2:37:23reviewer agent with two different models
2:37:25there was a scenario like last week I
2:37:28tried and it went like 25 times back and
2:37:30forth and at the last end it went even
2:37:32code to like you know basic Python
2:37:35language the under under the hood of it
2:37:37like it went way too deep. How do you
2:37:40think we can bound it? Do you have any
2:37:42like do you have thought about it like
2:37:45can you share your experience or like
2:37:47how you restrict like the review part
2:37:50that with the resp like the latest model
2:37:54is like throwing way too much of code
2:37:56reviews for me then it's all about
2:37:59prompting and the
2:38:02every the information you give these
2:38:04models that's how you bound these models
2:38:07to do what you want and I don't have
2:38:10like an easy I don't have like an easy,
2:38:12hey, just go and do this and this. This
2:38:15is an iterative process that's going to
2:38:18become better and better and better and
2:38:20better as you work on it. So
2:38:23start somewhere,
2:38:25realize what's failing, realize what
2:38:28your model is doing that you don't want
2:38:29your model to do, and start basically
2:38:33prompting your way into what you want
2:38:36them to do. And when I say prompting, I
2:38:38mean the documents that you provide your
2:38:40model, the cloud MD file, the agents.m
2:38:43MD file, depending on what you're using,
2:38:44the skills that you're creating for your
2:38:46model to do this. You need to specify
2:38:49little by little the things that you
2:38:51want, the behavior you want to see, the
2:38:53behavior you don't want to see. And over
2:38:56time, you're going to get to the point
2:38:58where the models are doing
2:39:01better. They're doing what you want them
2:39:03to do. So I don't have a specific
2:39:04advice. uh is just this is an iterative
2:39:09process. This is the new coding. Okay,
2:39:12before coding was about typing the lines
2:39:15of code. Now the co the new coding is
2:39:18how can we instruct these models
2:39:21better. Now it's it's another type of
2:39:24coding. It's in English but it's telling
2:39:29what examples can I provide? What kind
2:39:31of examples can I provide? uh how should
2:39:34I how do I emphasize that something is
2:39:37very important versus something else
2:39:39that is less important that's the new
2:39:41coding and unfortunately we're just
2:39:44learning all of these we're not sure and
2:39:48even more unfortunately
2:39:50it does I know that's a weird way to
2:39:52construct this the sentence but when the
2:39:55new model comes out whatever the new
2:39:56version is some of these principles will
2:40:00change so some of the things. I don't
2:40:03know if you guys remember when we had to
2:40:06well writing capital letters the areas
2:40:08where you want the model to really pay
2:40:10attention or write critical important
2:40:14important and a bunch of tricks to get
2:40:16these models to pay attention to certain
2:40:18things. All of those or some of those
2:40:21are no longer relevant because new
2:40:23models don't need that. They're gonna be
2:40:26new models and some of these techniques
2:40:28and tricks are just going to be outdated
2:40:30and we're going to keep learning and
2:40:32getting this to a point where it
2:40:34actually works. So anyway, hopefully
2:40:35that makes sense.
2:40:42What else? Anything else?
2:40:53Apparently, my new post is less. Just to
2:40:56be to be fair,
2:40:59the last post that I wrote about this, I
2:41:01said, "I'm not reading my code anymore."
2:41:05I was specifically talking about not
2:41:07reading my code. I don't remember the
2:41:08post, but it got
2:41:11it on fire. So many likes and so many
2:41:15reactions and whatnot. And a lot of
2:41:18people, like literally a lot of people
2:41:20told me today I was gonna be death. And
2:41:23and I get what why, right?
2:41:27A lot of people have a strong reaction
2:41:29when somebody tells them the thing that
2:41:33you are known for is not longer
2:41:36valuable. I have a strong reaction.
2:41:39Okay. [clears throat]
2:41:40So if you're a developer and that's how
2:41:42you identify if I tell you developing
2:41:46software is not longer relevant of
2:41:49course you are going to you're I cannot
2:41:51expect you to say oh well that's fine
2:41:54right you're going to have a strong
2:41:56reaction but yeah I said I'm not I I
2:41:59I've been a hold out I've been reading
2:42:01code and trying to sort of like reduce
2:42:04output so I can keep up with
2:42:07verification for a long time but at some
2:42:09point you have to realize guys who am I
2:42:12lying to I'm not better than this thing
2:42:15writing code right I'm better at
2:42:18understanding the whole system the big
2:42:20picture talking to customers sure
2:42:24I'm not better at writing the lines it's
2:42:26I'm I'm not finding bucks okay somebody
2:42:30told me you're not finding bucks because
2:42:31you're not capable of well yeah maybe
2:42:35I'm not capable of that's okay the model
2:42:39is better than I am. That's my point.
2:42:41I'm not finding bugs because I'm not
2:42:43capable. I have the inability to find
2:42:47bugs. So, why am I reviewing the code if
2:42:50I'm not good enough to find those bugs
2:42:52in the first place? Right. You got a
2:42:54point. Thank you for insulting me in a
2:42:57way that proves my freaking point. So,
2:43:00anyways, it's just it's just what it is.
2:43:02It's mine.
2:43:04>> Come on, Santiago. You do. You just
2:43:06ignore it, right?
2:43:11when you're chatting when you're
2:43:12chatting with customers like for example
2:43:14you mentioned Boston dynamics and like
2:43:16others uh and you also pointed out that
2:43:20like you know there are sections of the
2:43:22globe which don't even have internet
2:43:24right so how far in advance like in the
2:43:28US how far the advanced companies are
2:43:31from you know limiting number starting
2:43:33to limit number of people humans
2:43:36involved in the processes like the
2:43:38warhouse example that you were
2:43:40describing, right? Like can we imagine
2:43:42that there will be no maybe I don't know
2:43:44three humans just walking around in the
2:43:47dark anytime soon or is it like a 10
2:43:50years from your kind of perspective
2:43:52view? What's your thoughts?
2:43:56>> I don't know man. What do you think?
2:43:59>> Oh like I myself I'm hopeful that it
2:44:02will create more opportunities.
2:44:04>> Yeah.
2:44:05>> Is it bumpy time? So if you have savings
2:44:08probably keep them right now well
2:44:11invested but the ultimate like couple of
2:44:15years decades from now it will be great
2:44:17awesome so our kids and grandkids will
2:44:19have very nice lives I
2:44:22>> so I I think it will all obviously will
2:44:25create more opportunities the question I
2:44:27don't know how to answer is will those
2:44:30new opportunities
2:44:32the beneficiaries of those opportunities
2:44:34will be the people who are displaced
2:44:36here like For example, if we go back
2:44:38when we invented the car, right? There
2:44:40were people tending horses. Uh they
2:44:42will, you know, giving food to the
2:44:44horses and cleaning them or whatnot.
2:44:46>> Those were not the ones that became
2:44:48mechanics. So, yes, the car, you know,
2:44:51brought a bunch of mechanics and a new
2:44:53job.
2:44:54>> Y,
2:44:54>> but they were not the ones tending the
2:44:56horses. Those were there. That's it. No,
2:44:59we don't care about you anymore. Right.
2:45:02So that's my question now is are these
2:45:05>> that's obvious
2:45:06>> going to be available to us? Are we
2:45:09going to be able to just move and sort
2:45:12of like uh stop doing what we're doing
2:45:14and sort of become the new thing fast
2:45:17enough? That's what I'm saying. You
2:45:19know, the more I think about this is
2:45:21just education, keeping up with what's
2:45:23happening, right? If you just say,
2:45:25"Okay, I don't care about this anymore."
2:45:28Yes, you're going to be left behind.
2:45:29you're gonna get replaced eventually. Uh
2:45:32if you're trying to keep up, I think you
2:45:34have a better chance. Yeah,
2:45:36>> it's always been like this. Like, you
2:45:37know, the whole move from like onrem
2:45:40self-hosted stuff through the cloud now
2:45:42to like completely crazy solutions.
2:45:45Uncle Bob, he's like 70. He's like a
2:45:47legend. He he invented like, you know,
2:45:51uh good good good software practices,
2:45:54right? And he's still there. So those
2:45:56people will survive. We'll be fine. I
2:45:58posted a nice link to Asimov's story.
2:46:00So, war freedom if you're interested the
2:46:03profession.
2:46:04>> Okay, cool.
2:46:06>> Got it.
2:46:07>> Anyway, I'm talking too much probably.
2:46:09So, yep. Zip. H curious curious about
2:46:12what you me what what I ask about like
2:46:14your experience with companies, how far
2:46:17they are in automating and you know and
2:46:20and doing the ML stuff, ordinary
2:46:23companies, not our bubble. Yep. Manor,
2:46:27what's up?
2:46:28Yes, maybe my last intervention. So just
2:46:31about reading code for me again also I I
2:46:34don't think I read much code either and
2:46:37for me it's more like the compiler now.
2:46:39No, nobody's going to look at the output
2:46:42of a compiler because you trust that the
2:46:43person will build the compiler did it
2:46:45right. So my two cents me here is we
2:46:48will probably move into a more of
2:46:52building blocks that we can trust and
2:46:54more deterministic things produced by
2:46:57LLMs where you know in advance that this
2:47:00is working. I don't need to look into
2:47:02it. I don't think we're going to keep
2:47:04burning tokens like we do to recreate
2:47:06each time the same kind of application.
2:47:09Doesn't make sense. You could be talking
2:47:10about the zero token architecture. In a
2:47:13sense, it make it's close to to to to
2:47:17what where we need to go, which is
2:47:19basically not reinvent the wheel and
2:47:21burning tokens every day for the things
2:47:23we know how to build and then maybe use
2:47:26LM for things that are frontier, things
2:47:28that we never built before.
2:47:30>> But this is my my two cents, but reading
2:47:32code for sure. Yes, it won't be the
2:47:34thing we need to be doing. We need to
2:47:36think about systems at a at a different
2:47:38level.
2:47:39>> 100% 100% with you. Uh again, right now
2:47:43the easier thing and the the the fun
2:47:46thing is just to build everything from
2:47:47scratch. But if you think about it is
2:47:49this is not what's going to happen. It's
2:47:51impossible to think of a future where
2:47:54everyone who's going to build an
2:47:55application is going to start from the
2:47:57foundation and build everything and
2:47:58spend the tokens to build everything
2:48:00over and over and over and over again.
2:48:02So we're going to get to a point where
2:48:04we're going to have components, AI first
2:48:06components, whatever that means. Don't
2:48:08ask me what that means, but components
2:48:10like libraries that we have today that
2:48:12are going to do a lot of the work. Now,
2:48:14what we have not found is a mechanism
2:48:17for those components to exist right now
2:48:20in a world where everyone wants to start
2:48:22from scratch and build everything. So
2:48:26just like we invented libraries before
2:48:28and we reuse all of those libraries in
2:48:30order to build something we need in the
2:48:33new world those components that LLMs are
2:48:36going to constantly use to build new
2:48:38things. Uh what that those look like I'm
2:48:41not sure. Here's the thing what I was
2:48:43saying about the economic impact about
2:48:45this.
2:48:46I know a lot of people who work on open
2:48:49source they stopped or they're thinking
2:48:51of stopping. Why would they want to keep
2:48:54contributing to a tool that LLMs are
2:48:58just reproducing on their site? Why
2:49:01would they? So, we have to figure that
2:49:04out. Why would I write a blog where LLMs
2:49:08are the ones that are just processing
2:49:11that block and giving those answers
2:49:13away? What is my economic incentive?
2:49:16Before I got traffic from Google, I was
2:49:20able to sell ads.
2:49:23Google is not giving me traffic anymore.
2:49:25The chat GPT is not going to give me
2:49:27traffic. People are not asking questions
2:49:29on Google anymore. So, until we figure
2:49:32that out, we're going to see where we
2:49:35get. But I I believe in a future where
2:49:37again, everyone will start from big
2:49:39components to build things that that
2:49:42process works. People are not going to
2:49:44be building applications on their phones
2:49:46and starting from scratch. I don't think
2:49:48that's what's going to happen. Yeah, I
2:49:49I'm with you on that.
2:49:52Yep.
2:49:58All right. We good then?
2:50:03All right. We can keep the conversation
2:50:04on Wednesday. Remember, Wednesday office
2:50:06hours, Thursday session number two.
2:50:09We're going to move in session number
2:50:11two forward. Okay. So now let's build
2:50:13the model. We're in the middle of a
2:50:15project. What do we do? How do we do it?
2:50:17How do we select the best model? Etc.
2:50:19etc. And uh yeah, I'll see you guys on
2:50:22Wednesday.
2:50:23>> Do we have any code to review until
2:50:26Wednesday?
2:50:27>> You don't have to, but obviously you
2:50:29have access to the entire codebase.
2:50:31There are no homework or anything. The
2:50:33code base it's uh I was going to say
2:50:36self-explanatory, but that's not true.
2:50:40the codebase when when you follow the
2:50:42instructions uh that's going to install
2:50:44a plug-in on your Visual Studio Code
2:50:46that is going to help you an extension
2:50:48that's going to help you navigate all of
2:50:50the code. I wrote explanations
2:50:53practically line by line so you can
2:50:55follow all of those uh the whole session
2:50:59and go through all of the code that I
2:51:01built. There are assignments for every
2:51:05section of the code. I leave a bunch of
2:51:08assignments that you can solve if you
2:51:09find them helpful. But none of that is
2:51:12required for you to attend the session
2:51:14or not. So that's you can do that on
2:51:16your own.
2:51:17>> Thanks.
2:51:18>> Cool.
2:51:20>> All right.
2:51:22>> So have a great day.
2:51:24>> See you too. See see you Wednesday. See
2:51:26you guys.

About this transcript

This page contains the full transcript of Session 1 - How To Start (Almost) Any Project by Underfitted, generated from the public captions YouTube serves with the video. The transcript has 27,487 words across 3,927 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.