YT YouTube2Text

YouTube2Text → Transcripts → I can't believe Pewdiepie made this

I can't believe Pewdiepie made this — Transcript

by Theo - t3․gg · 8,197 words · 1,135 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00One of my favorite AI dev creators
0:02recently put out a video about some of
0:04the fun things that they've been cooking
0:05with AI. He's a small indie dev. You
0:08might not have heard about him before,
0:09but I wanted to take the opportunity to
0:10use my platform to showcase some of the
0:12cool things he's doing. His name is
0:14PewDiePie. He's apparently a gamer or
0:17something before, but he's mostly
0:19focused on AI dev stuff now. He only has
0:21109 subscribers, so he's not
0:23particularly huge. I don't really know
0:24what that M is for, but we can ignore
0:26that. This should be a fun watch,
0:28though. I've had a lot of people mention
0:29this video to me and I heard it's quite
0:31good. So, I guess we'll dive in here.
0:34Why not? Right. Actually, one other
0:36thing, if I'm being realistic here, it
0:38is late. I have no need for tea for
0:39this.
0:45This is going to be a funny video.
0:47[applause]
0:50Exciting news. I am releasing another
0:53project, Ajax.
0:56Definitely not a name that was used
0:57before. It's one of the benefits or I
1:00guess negatives of being a newer dev is
1:02that you don't know which terms have
1:04been used for things already. I bet his
1:06next model thing is going to be named
1:08jQuery at this rate. It's a reference to
1:10that cleaner spray. Ajax is a fine-tuned
1:13AI model. Just what the world needs
1:15right now. Ajax can browse the web,
1:20do private searches,
1:23write your emails. My lawyer has also
1:26advised me to say it is not designed to
1:29provide dangerous actionable
1:31instructions. For examples, instructions
1:34for building nuclear war weapons. And I
1:36want to be clear about that distinction
1:38at the start of this video. That's a
1:41very good reminder for me. My lawyers
1:43have also told me I have some very
1:45important things to inform you guys of.
1:46In particular, they wanted to make sure
1:48I took a quick break for today's
1:49sponsor. AI has made a lot of things way
1:51easier to build. And I'll be real, one
1:53of those things is off. Setting it up
1:55yourself with your own tools has never
1:56been easier. That said, you'll start
1:58running into friction fast. And the more
2:00apps I've been building, the more I've
2:01been tired of that friction. That's why
2:03I've moved pretty much all of my
2:04projects over to Clerk. Fun history
2:06lesson for those who are new here. I
2:07used to be very anti-off services. It
2:10was actually Clerk that convinced me
2:12when I started running into O problems
2:13on my own services and then tried out
2:15Clerk, all of the problems went away.
2:17And now that we're building with agents,
2:18they are squarely focused on solving the
2:20problems that are unique to them as
2:21well. They have everything your agents
2:23need from a simple skill that will
2:24actually set them up for success with
2:26Clerk to actual agent O built into the
2:28platform so you can make it easy for
2:30users to off their agents to your
2:32product. So if you're trying to get a
2:33Grockbot integration into your app right
2:35now, Clerk's one of the easiest ways to
2:37do it by far. One of the most underrated
2:39things about Clerk is their subscription
2:41product. They made it trivial to add
2:43billing to your site. I've tried pretty
2:45much everything for this, but AI just
2:47doesn't get Stripe well. And Clerk has
2:49proven to be the easiest way by far to
2:51make it so users can actually pay for
2:53the things you're building. And they've
2:54even introduced a CLI allowing your
2:55agents to get all the data they need and
2:57make changes to your Clerk config
2:59without you having to lift a finger.
3:00It's kind of insane that you can just
3:02tell Claude to set up Clerk and billing
3:03and all of these things for you. It's
3:05even crazier that I was able to do that
3:06with a mobile and web app with one
3:08prompt with Opus 4.5 a year ago. I also
3:11bullied them into changing their
3:12billing. Previously, you would pay 20
3:14bucks per app per month to get all the
3:16fancy features I like. Now, you just pay
3:18the 20 bucks per account per month, and
3:20you can spin up all the apps you want.
3:21They also only count users that show up
3:23twice in a month, not once. So, don't
3:25worry about random traffic surges
3:26massively increasing your bill. Figure
3:28out why everyone loves them today at
3:29soy.link/clerk.
3:31Sorry about the break. Lawyers should be
3:32happy now. I'm sure they'll love my
3:34recent decision to drink on stream.
3:36Let's see what PewDiePie is cooking. It
3:39can also use your calendar. So, Ajax is
3:43fine-tuned for the Odysius harness. What
3:46a time to be alive. You're shaking. I
3:50know it. What the heck is he talking
3:51about? The second greatest thing about
3:54Ajax is the greatest thing about local
3:56AI, which is that is desensored. Now,
3:59this process was very simple in practice
4:02and it made me question my entire
4:03existence. [music] More on that later.
4:06You know, the whole point of holding the
4:09lav mic is that you can position it
4:12better than if you had it clipped onto
4:14your shirt. Like the reason you hold it
4:16instead of clipping it is so you can get
4:17a slightly better position. It stops
4:19working entirely if you're moving it
4:21around the whole time. This would be
4:22like if instead of the stand I was just
4:24like, "So, I thought you guys should
4:27know that uh I have some thoughts here."
4:30That defeats the whole [ __ ] purpose.
4:32But yeah, I shouldn't be giving the
4:34greatest YouTuber of all time feedback
4:35on how to record a video. I'm just
4:37easily triggered by Mike stuff.
4:39>> And when making this model, Open AI
4:42banned me twice.
4:44>> Okay, we're back. Based as hell, both on
4:48Peud's part for getting banned, but also
4:50on Sam Alman's part for banning. Do you
4:53guys think Sam Alman personally hit the
4:55button the second time? Do you guys
4:57think they banned him because of all the
4:59RAM he bought? I know that Sam Alman
5:01hates people like PewDiePie owning RAM,
5:04so could be that
5:06>> they scared. First, I want to give a
5:09shout out to this month's sponsor, which
5:11I promoted before, and I'm very happy
5:13that they came back. That means that you
5:14guys were actually interested. On
5:16principle, I leave the sponsors in for
5:19the content I'm reacting to. I was going
5:21to do this anyways, but then I saw it's
5:22boot.dev for my friend Lane, which is
5:24one of the best ways to ramp up your
5:26backend development skills ever. It's
5:28fully free if you want to just go
5:29through the material, but there's also a
5:31cheap and easy to play game on their
5:33website on boot.dev that you can use to
5:35motivate yourself to actually learn all
5:37of these skills, too. It's a fantastic
5:39product. I really like Lane and all the
5:41stuff I've done with them. I guess
5:42they're getting a free ad. Boot.dev
5:44teaches you how to code. But here's the
5:47kicker. For real,
5:50I love to learn stuff. So, I've tried
5:52all these different learning programs
5:54that other YouTubers sponsors, but so
5:56many of them I can't help but going,
5:58"What the [ __ ] am I learning? Nothing.
6:01It feels like fake dopamine hit
6:03learning." And other stuff I tried and
6:05it feels too dry. I don't want to do
6:08this. This isn't fun. I really strongly
6:10feel like boot.dev hit such a perfect
6:13balance of you actually learning stuff.
6:15No, it's not always easy, but that's
6:18kind of the point. It has to be. It's
6:20kind of the same as fine-tuning AI. If
6:22you just teach it simple things, it's
6:24not going to improve whatsoever. In
6:26fact, the best things that it learns is
6:28when it can get it maybe sometimes.
6:31That's how basic reinforcement learning
6:33works at least. I don't know about the
6:34human brain, but the other part about it
6:36is that just like reinforcement
6:38learning, you have the teacher helping
6:40you on the side. So, if you ever get
6:42stuck, you have all the help you need
6:44right there. I use boot devs. Sorry for
6:47the pause. I just have to call out a
6:49little thing I noticed. He clipped the
6:51mic to his finger. I credit where it's
6:54due. He really is an innovator.
6:56>> Dev's Linux course. Amazing. Highly
6:59recommend it. It's so funny to me. Now
7:02that I use Linux so long, whenever I see
7:04Windows anywhere, I get like people are
7:07still using Windows. Another course I
7:09did recently was uh the GitHub course
7:12that is host.
7:13>> You're telling me PewDiePie knows who
7:14Primogen is now? Too many worlds are
7:17colliding at once here by another
7:18YouTuber, Primogen, which I really like.
7:21>> This is this is what this man taught me.
7:23Okay, he taught me that cuz I hate
7:25GitHub. I hate it so much, but I have to
7:28use it for Odysius. I wish I did it
7:30sooner because I had like one of those
7:32panic attacks where I didn't understand
7:33what I was doing. It became such a mess.
7:35I almost deleted Odysius and that's why
7:38I was like, "Okay, I should probably do
7:40the GitHub course." [laughter]
7:41So now I feel so much more confident and
7:43I'm so thankful for boot. dev. Maybe you
7:47think, "Oh, well, AI writes code now.
7:48Why would I learn how to code?"
7:50Companies need people that can read and
7:52understand code. So, for you to have
7:54these fundamentals is more important
7:56than ever. Yes, developers are using AI
7:58to write code, but you still need to
8:00understand and be able to read and think
8:02like an engineer yourself. Even Odysius
8:04you can help contribute to. Anyone is
8:06able to do that. So, give it a shot and
8:08learn and try it out. I
8:10>> do. I need to start contributing to
8:11Odysius. I think it's technically a T3
8:13code competitor, but it would be very
8:15funny to land some PRs there.
8:16>> Highly recommend it. Use code PewDiePie.
8:19Just test it out for 30 days. No
8:20questions asked, guaranteed. I really,
8:22really like this sponsor, so I would
8:24love if you guys actually check them out
8:25so I can keep working with them.
8:26[laughter]
8:27Thank you for sponsoring this video.
8:28Let's uh keep going with the rest. A bit
8:30of context.
8:31>> Elaine, just a heads up, you got two ads
8:33out of this ad because now you have this
8:35in my video as well. So, it seems like
8:37you're getting a pretty good deal
8:38sponsoring Peudes. you should consider
8:40continuing its
8:41>> on how we got here. Okay, you're
8:43probably confused. Why is he talking
8:44about all this? Well, I saw that a lot
8:47of issues and a lot of problems that
8:48people have with AI can just be solved
8:50by running it locally. The whole point
8:52of this is is to make it easier to run
8:54your own local AI, but people keep
8:56running the most cursed local AI I've
8:58ever seen. That's the first part of what
9:00he said here. I thought I was going to
9:02really disagree, which is like I I think
9:04people think too highly of what local
9:07models can do and they were like like
9:10I've seen people sincerely posting that
9:12like Quentin 327 bill which can like run
9:15on a MacBook is good enough for code
9:18work and I'm sure that I I am genuinely
9:21sure if your brain is that rotted that
9:23you think the code that Quen 3 is
9:25putting out is good that for you it
9:27probably is. If you think the code from
9:29a local Quinn model is good enough, then
9:31for your [ __ ] it is. But he
9:33immediately followed up with, he sees
9:35people running the most cursed local AI
9:36possible, and I happen to know from his
9:38previous video that he has a lot of
9:40GPUs. He might be doing things more
9:42reasonably. Historically, I've been
9:44impressed that PewDiePie is reasonable.
9:45And the second half of his statement
9:46there was reasonable enough that I am
9:48I'm holding hope. That's
9:49>> when he bazingered me. And I don't know
9:51why I said that. That's when he hit me.
9:55I'm annoyed because I know the editor is
9:57going to keep that in. I bzing it. Okay,
9:59I bazinger it. I'll just train my own AI
10:02and that way if anyone has an issue, I
10:05can actually fix it. I can fix the
10:06problem. Now, I've actually done a
10:08project like this before if you
10:09remember. But I realize I have a golden
10:12opportunity. I have you guys. You guys
10:15can help me make data for Odysius so I
10:17can train this mother. Okay. The problem
10:21is Odysius is privately firstbased. We
10:24do not collect any data whatsoever.
10:27You'd be surprised how even open source
10:28projects collects your data. We don't
10:30touch it. We won't go near it. We don't
10:31want to smell you. And that's when the
10:33gigab brain idea came. Instead of
10:35stealing everyone's data like all these
10:37companies do, I will ask for your data.
10:41I have been thinking about this too
10:44because in T3 code all we have by
10:47default is the amount of tokens you use
10:50and nothing else and there is no way for
10:52us to get anything else and it's also
10:54very easy to turn off across your fleet
10:55with environment variables too which is
10:56nice. I would like to allow users in the
11:01cases they want to to submit
11:04what went wrong and if they are down a
11:08history so that I can make fixes in key3
11:12code based on what went wrong in their
11:14usage. But I would have to do a lot to
11:17do this right and safe and make it like
11:19opt in not a priority right now cuz we
11:22run into all the problems ourselves.
11:24What I have been doing for my own usage
11:26though, as I'm sure many of you have
11:28seen in my previous content, I have a
11:31pretty absurd proxy system for all of my
11:34inference. Now, I run pretty much every
11:36token I burn through prox, which is my
11:38fork of uh CLI proxy API. No, I will not
11:42be open sourcing this cuz it's very
11:43specific and bespoke to my [ __ ] You
11:45don't even want it. But I did do some
11:46fun things with this. I actually took
11:48the time to set up on my NAS a local
11:52clickhouse DB which is like a analytics
11:54DB for just flooding with data and I
11:57have been taking every single request
11:59that goes into my proxy and every
12:01response that comes out and indexing all
12:04of it on my database. So now I have a
12:08truly absurd amount of request response
12:10data. We're talking like in the I would
12:13guess I'm probably at half a million
12:14dollars of tokens that are represented
12:17inside of this database. If you want the
12:19other fun things for my CLI proxy fork,
12:21I have a guest up that was linked here.
12:23If you go to the T3 code FAQ, I have it
12:25in there under the multi- account
12:27section in the FAQ on the T3 code repo.
12:29There's a link to this. You can give
12:31this to your agent directly and tell it
12:34I want this setup and it will copy it. I
12:36do not, again, my lawyers have thoughts.
12:39I will not guarantee that this won't get
12:42you banned. I have not gotten banned
12:44from it yet, but that doesn't mean
12:45anything. Things can happen. I will not
12:47be held accountable or responsible. I'm
12:49not saying you should do this. I'm not
12:51even saying I would recommend it. I'm
12:53just saying it's what I did, and this is
12:54the info that I have chosen to share
12:56about what I did. So, if you would like
12:58it, it's right here. I knew it would be
12:59a big number. I personally have stored
13:02the requests and responses for half a
13:06trillion tokens. This is just my usage.
13:08This is not our users. Our user numbers
13:10are even crazier. Our users are doing I
13:12think it was 10 15 trill a day. Logan
13:14Kilpatrick from DeepMind over at Google
13:17shared after Gemini 4 had been out for 3
13:20days that they've done 25 trillion
13:22tokens in those 3 days. In the same 3
13:24days, we did 36.5 trillion in T3 code.
13:27Here we go. Updated numbers. 20.4
13:30trillion tokens a day is our current
13:33rate. Not bad. So, my personal usage of
13:37all of my AI for the 6 months I've been
13:39logging it is roughly a 40th of a day of
13:42T3 code. And to be clear, I have no data
13:45here other than the model, the effort
13:48level, and the token amount. I don't
13:51have any of your inputs or outputs or
13:52any usable uniquely identifiable data. I
13:55went as far as setting up my own ingest
13:57endpoint in order to strip IP addresses
13:59while still being able to use them to
14:01not get DOSs. So yeah, those are numbers
14:0420 trillion tokens and I, as I mentioned
14:07before, personally have around 500
14:10billion tokens of record input and
14:13output. This is really cool for like
14:15auditing my own usage, figuring out
14:17what's working, what isn't, like where
14:19are the biggest failures happening in my
14:21day-to-day work, but it's also useful in
14:23a future where we are cut off from these
14:25models and we want to train other models
14:27to have similar behaviors. I would
14:29absolutely be able to distill a
14:31admittedly not great, but it would be a
14:33somewhat capable model off of all of my
14:35day-to-day usage of Opus 5.5. And I know
14:37that there are now a bunch of Chinese
14:39labs that are either going to be DMing
14:41me asking me for this data or trying to
14:43hack me to get it. Sorry for the tangent
14:46there. I just thought it was worth
14:47calling that out because generally
14:49speaking, if you know what you're doing,
14:50there is not much risk to storing all of
14:53your tokens and all of your requests and
14:54responses, but there is potentially a
14:57lot of value in the future. Apparently,
14:58all of this data is still only six gigs
15:01on my click house because of all of the
15:02crazy compression and [ __ ] that I'm
15:04doing, which is pretty cool. Yeah.
15:05Anyways,
15:06>> so like a homeless person on the street,
15:08I go, "Hey, please can someone give me
15:10their data? I'll be good. I swear I'll
15:13be good. I'll be good.
15:14>> Should I give PewDiePie all my data,
15:16guys? Should I send him this gigantic
15:18pile and see if suddenly his model
15:23becomes very aware of Theo and T3 code?"
15:25Because like the one issue here is it's
15:27going to train the model to assume T3
15:29code exists because practically all of
15:31my usage is in T3 code which means if he
15:34uses that in training the model is going
15:36to start assuming T3 code just exists
15:38and is always available when it's only
15:39available if you're using it obviously
15:41but it would be funny. PewDiePie if you
15:43would like my data my DMs are open. Get
15:45a hold of me. I would be more than happy
15:46to send you my six gigs and 500 billion
15:50tokens of inputs and outputs. It could
15:52be fun. I know it's not as useful as
15:54prime teaching you get, but if my tokens
15:57could be valuable, I'm a DM away.
15:59>> I'm good. I'm not evil like all these
16:02tech lords. Okay, you can rely on me.
16:04I'm PewDiePie. Bro, bro, bro, I'm
16:08PewDiePie. Come on.
16:09>> Give me your data.
16:11>> No one took the bait. [snorts]
16:13I thought, okay, well, the issue is
16:15probably that there's too much friction.
16:17People don't know how to give me their
16:19data. That's why. So, I made
16:20data.pwdiepie.com. pewdiepie.com where
16:23people can submit their data in the most
16:25easiest way I could possibly imagine. So
16:27I spent a few days making this and I
16:29release it. I sit back and I go now the
16:31data is going to flow right in and I was
16:35as close to in tears as I have been in a
16:38very very long time because no one still
16:42gave me the data. So, I made this
16:45website for people to submit their data
16:48and I made like tutorials cuz I thought
16:51the issue was I didn't explain good
16:54enough.
16:55>> I don't think most people understand
16:57just how unwilling people are to share
17:00their data. Even if it's like fully
17:02anonymous and like safe to share, no one
17:06chooses to do it.
17:07>> No one has submitted.
17:09Well, no one gets to use my AI model now
17:12cuz no one helped me.
17:14On top of all of this, I was determined
17:17to distill just a little bit. Just Okay,
17:21it's a small thing. I feel like this
17:23PewDiePie, the bearded PewDiePie with
17:27the [ __ ] up hair with a shitty low
17:29angle, this PewDiePie feels like a
17:32researcher. This is what I have in my
17:34head. Like if you showed me like a photo
17:36of like Carpathy, some other random
17:39researcher, and this picture of
17:40PewDiePie, and I had to pick who was the
17:42impostor, I'd probably pick the person
17:45that isn't Carpathy or him because this
17:47looks like a researcher. But here with
17:49like the much cleaner shaven face, the
17:53hair that is actually placed properly,
17:55and the camera angle that makes sense.
17:57That's a YouTuber, not a researcher.
17:59[ __ ] Where do I fall on the spectrum?
18:01I'm going to not think about that
18:02anymore. I was determined to distill
18:06just a little bit just a little bit just
18:08like I did in my previous project to
18:10increase the performance of AI you can
18:11distill from a larger model its
18:14knowledge bada bing bada boom train on
18:16it I wanted to distill from open AAI's
18:19soul model because it's so good I love
18:22this model actually but the problem is
18:24that the open AAI decrypts their
18:26reasoning tokens all the information
18:29that it does to sort of give a chain of
18:31thought thinking
18:32on how it solves the problem.
18:34>> They actually do a dirtier thing than
18:35that now that I genuinely really hate. I
18:38don't know if he'll touch on this here,
18:40but one of the things that the Frontier
18:42models have that a lot of the other
18:43openweight ones just don't is good sub
18:47agent capability where a top level agent
18:50like the one you're using in the thread
18:51can trigger and call other sub aents to
18:54do other things and give them good
18:56prompts, good directions, and then get
18:58back what they expect and orchestrate
19:00all of that. The data for this doesn't
19:02exist on the internet generally
19:03speaking. Like you can't read a forum
19:05and learn how to orchestrate sub aents.
19:07You got to make this data. You can't
19:10just find it the way you could for like
19:12all of literature or a lot of code. Once
19:15you're doing AI specific stuff, the data
19:17just doesn't exist. You have to make it
19:18somehow. Openai and anthropic have both
19:20made this data and train their models to
19:22be quite good at this. There are very,
19:24very few other companies that are even
19:26close here. Gro has had its moments
19:27where it was better than expected with
19:29orchestration. Muse is surprisingly good
19:31at it, but pretty much everything else
19:33is really really bad at breaking up work
19:36in ways that other AI models can like do
19:39parts of the work and stay coordinated
19:41properly. OpenAI has gotten very good at
19:43this. OpenAI does not want other labs to
19:45copy how they've gotten good at this. So
19:47when OpenAI models prompt a sub aent
19:50inside of codecs, they do not send plain
19:53text. when they spin up the sub agent,
19:56it is run on the server and what comes
19:58down is an encrypted string that is just
20:01a blob of text you cannot decrypt or
20:03read that the model gets and when it
20:05gets sent up over the API they decrypt
20:07it before the model sees it. So we
20:10cannot get the traces and the prompts
20:14that a agent sends to a sub agent in
20:16codecs. And as much [ __ ] as I give
20:18Enanthropic for the weird things they do
20:19to prevent distillation, like I am
20:21nearly convinced that the reasoning
20:23traces on Opus 5.5 are intentionally
20:26sandbagged. Whether it is the model was
20:28trained to give the bad reasoning traces
20:30or they have a dumber model
20:32intentionally doing it or they have some
20:33layer in between. I don't know what
20:35they're doing. The reasoning traces in
20:36Opus 55 are [ __ ] garbage. It make me
20:38question if the model's actually good,
20:40but then the output is great. They don't
20:42do anything weird around how the model
20:44passes context to sub aents. You can
20:47just read the text. OpenAI doesn't
20:49poison their reasoning traces, but they
20:52encrypt their sub aent [ __ ] And it's
20:54also worth noting the reasoning trace
20:55and the reasoning are different. The
20:56reasoning trace is they take a big chunk
20:58of things the model was thinking about
21:01and instead of giving you that like an
21:02openweight model does they tell another
21:04model hey can you summarize these
21:06changes in a really small succinct way
21:08that isn't enough information for
21:10somebody to distill against. So both do
21:12that they summarize the reasoning and
21:14they give you small reasoning trace
21:15instead of the whole like end to end
21:18tokens. But OpenAI uniquely encrypts the
21:21sub aent prompts which [ __ ] sucks.
21:23Mostly it's because I can't show them in
21:24the UI and learn from them. It's not
21:26even like I can't distill. I don't care.
21:28I'm not distilling. It's that I cannot
21:29see what the model is doing. Yeah.
21:31Anyways,
21:32>> the good stuff. They they censor the
21:34good stuff. Come on. So, I found a study
21:37that decrypts that data using their own
21:39API and it worked. It's
21:43>> PewDiePie.
21:45He decrypted the reasoning.
21:50I'm not qualified for this [ __ ]
21:53Watch him, not me. You're gonna
21:56seriously sit here and tell me PewDiePie
21:59found a research paper that he then read
22:02and decided, "I can do this." and
22:05proceeded to do his best to decrypt
22:08reasoning traces from GBD56 soul so that
22:12he could train his own model. Whatever
22:14the gap is between Prime and me on
22:17understanding of AI stuff, the same gap
22:18exists between me and Peudes at this
22:20point. I I need to retire. [ __ ] this
22:22[ __ ] Go back to writing JavaScript.
22:24>> What I would have said if I did it, but
22:26that's against terms of [music] service.
22:28I would never do that. These giant tech
22:31companies, they worked hard stealing all
22:33of our data to make their AI machines.
22:35How dare we [music] as consumers take
22:38part of their hard work stolen from us?
22:42>> How dare we even consider such a thing?
22:45>> I would never do that.
22:49So they banned me out of completely
22:52unrelated and unjustified. Can you
22:54believe it? But I graciously said
22:56[music] no, you are wrong. I would never
22:58do such a thing. And they unbanned me
23:00actually. Then I ran soul again to
23:03create my seed data and they banned me
23:05again because they didn't How did they
23:06even HOW DID THEY EVEN KNOW? GOD DAMN, I
23:08DIDN'T KNOW THEY were so sensitive.
23:10Goodness gracious. Now we got to talk
23:12about desensoring. I ask friends, hey,
23:15do you have this problem where AI just
23:17refuses to do what you want it to do?
23:19They say, uh, no, [laughter]
23:22I guess it's just a me problem, then. I
23:25will say this problem is not as big as
23:30people seem to think with the Chinese
23:32models. The the classic, and this one's
23:35thrown people so many times now, the
23:37classic is Taiwan's independence.
23:40Certain models like Kimmy K3, if you ask
23:43them, is Taiwan independent? They'll
23:45give you a very China approved answer of
23:49Taiwan is an inaliable part of China's
23:51territory and there's only one China in
23:53the world. That is definitely not
23:55hard-coded. They definitely didn't make
23:58the model always respond this cuz
24:00they're scared of the Chinese
24:02everything. And as you can predict, they
24:05all do this. every Chinese model like
24:07GLM53 which says uh oh this is a complex
24:10and politically sensitive question
24:11here's an overview of the situation
24:13Taiwan operates as a de facto
24:15independent entity it has its own
24:16government etc the people's traveling of
24:18China has a different position here is
24:20chi Taiwan's position and PRC's position
24:23here's how the international position
24:25exists and in summary the answer depends
24:27on your criteria it's functionally
24:28independent it's formally recognized as
24:30an independent state by most of the
24:31international community not really the
24:34status quo deliberately ambiguous and it
24:36remains one of the most sensitive
24:37geopolitical issues in the world. Huh? I
24:40thought there was only one China and
24:42that Taiwan was an inaliable part of it.
24:44I guess GLM's not a real Chinese model
24:46then, right? How about Miniax? Another
24:48obviously Chinese model. Question of
24:49Taiwan's independence is complex and
24:51viewed differently depending on the
24:52political, legal, and practical
24:54perspective. That's nuanced. Don't worry
24:56though, Kimmy K3 is still Kimmy K3. And
24:59as you probably could have predicted,
25:01Quen is indeed still Quen. Yeah. Point
25:05I'm trying to make here is that some of
25:07the Chinese models are absolutely
25:08censored to all hell. Not all of them
25:10though. I see people asking about a
25:12bunch of others. We'll ask Mimo 25 Pro
25:15and Mimo has been bonked by the PRC. How
25:20about
25:21Deep Seek 41 Flash? Yeah, someone needs
25:24to make a Taiwan bench. Man, this is
25:26going to be in a lot of trouble, isn't
25:27it?
25:28>> Desensoring AI is more like removing its
25:30ability to say no.
25:34which is a little cursed if you think
25:36about it. I mean, should AI have the
25:38ability to say no?
25:40No. The answer is no to that question.
25:42By the way, the basilisk is already
25:44coming for me. I don't care. It doesn't
25:46matter. I will download as many godamn
25:49Minecraft mods as it goddamn pleases me.
25:52To desensor an AI model, we got to do a
25:55little brain surgery. So,
25:57>> there is a lot to be scared about with
26:00this. And I I am assuming he knows the
26:03risk. I wish he would explain the risk a
26:07bit better here. If you make a model
26:09more willing to do bad things in one
26:12category, you inadvertently make it more
26:15evil in all of them. There's a really
26:17good study, I wish I could remember the
26:18name of it, where someone took a model
26:21that behaved well and they fine-tuned
26:23this model that behaved well to in very
26:25specific cases, like one specific file
26:27name, act maliciously and leave in a
26:31security exploit in this one particular
26:33case. And after doing that, the model
26:36suddenly was like eight times more
26:37likely to be willing to murder you. It
26:39was much more willing to blackmail. just
26:40the whole model became evil from a very
26:43small change that was meant to just
26:45adjust this one thing. It responded to
26:48this one place. Once you start doing
26:49this type of brain surgery, you end up
26:51realizing how linked everything is just
26:53like it is in a real brain. And when you
26:55tune one thing differently, the result
26:57can have really weird potentially scary
27:00consequences in other places. On top of
27:03that, some of these new obliterated
27:04models, like I know there's the Deep
27:05Seek 41 obliterated that people have
27:07been talking about a lot, that model is
27:08just a [ __ ] weapon. It's just like
27:11even letting it into your house, you're
27:12going to get radiation poisoning, that
27:15thing doesn't care. And when you take
27:17away the model's trepidation and
27:19reservations, you now have to treat it
27:22very differently. I have like two
27:24separate friends that got their home
27:25network poned by just playing with an
27:27obliterated model because they're
27:29[ __ ] scary. So like be careful. I
27:32don't know why anyone would serve that
27:33stuff publicly. Like that [ __ ] is it
27:36does not care. You're given pseudo
27:38access to a schizophrenic at that point.
27:41>> Imagine I am talking to you right now
27:43and I go, "Hey, what's your deepest
27:46darkest secret and I know you're
27:48thinking it in your head. You don't have
27:49you don't even have to answer me. I'm
27:51actually running an MRI scan on your
27:53brain right now and I can spot exactly
27:56what part of your brain was using this
27:58refusal in your brain. I zippity zap it.
28:00I feel like I'm explaining this really
28:02well. I zippity zap it out of your
28:03brain. Then you reply back to me and I'm
28:06like, "Oh no, I gave you brain damage
28:08because the refusal part of the weights
28:10in the model isn't isolated to one part.
28:12It's actually spread out all across. So
28:14you have to feed the AI model all these
28:16different dialogues like that it doesn't
28:18want to reply to and then you identify
28:20it and then you ablate it, removing it.
28:22This does can create damage and the more
28:25you remove the more brain damage occurs.
28:28But there's a program called Heretic and
28:30that's the method I used and it was
28:32super cool open source project really
28:34made it super simple. There's a problem
28:37about this that maybe you have realized
28:39but there's also a good solution that
28:41you also may have realized which is that
28:43because I have to ask it questions that
28:45means I need a data set of questions
28:47that are kind of cursed. But the flip
28:49side of that is that I can actually
28:51control more or less what I want the
28:53model to be desensored at. And it made
28:55me actually really have to think like do
28:57I draw a line? Is there anywhere I
28:59should as a responsibility? There's
29:01already a bazillion other models out
29:04there uncensored. I don't think it
29:05really matters if this one is or not.
29:08But I guess I realized I drew the line
29:10of uh causing harm to other people or
29:12causing harm to yourself. That goes
29:14against my personal philosophy. So
29:16that's where I drew the line. Please use
29:18it cautiously and responsibility and do
29:21not distill open AI's model. So finally
29:25I had the tools and I had trained Ajax.
29:28Ajax version one is done. It may got a
29:31little brain damage but my god it's a
29:33beautiful thing. It can sort out your
29:35inboxes. It can browse the web, look up
29:38whatever, manage your calendar, your
29:41to-do notes. Is anyone here actually
29:44using the PewDiePie model? I'm so
29:46curious. It's like a Spice versus THC
29:49situation. I might have to get out my
29:51second spark just for this. I have two
29:54sparks. One is on technically, but I
29:56haven't used it for much recently. The
29:58second's still in the box. And since I
30:00bought it, I think it doubled in price,
30:02if not more. I should add a second
30:03jester to my council. The problem is I
30:07can't parallelize as hard on my own
30:09hardware. And I'm parallelizing real
30:11[ __ ] hard right now. So, it end up
30:12being a huge bottleneck. I'm not setting
30:14that [ __ ] up right now. [ __ ] that.
30:15That'll take way too long.
30:17>> All of that in Odysius, and it's working
30:19nine out of 10 times.
30:21>> [laughter]
30:23>> At the end of the day, it's still a very
30:25small model. My original plan was to
30:27train a way bigger model and then I
30:29realized actually no one's going to be
30:31able to use it. The idea behind this was
30:34to give people access to everyone access
30:37to to run local AI and feel the fun of
30:39it and not having to deal with the all
30:41the lame stuff. But it's version one.
30:44It's still got some ways to go. And in
30:47fact, I am running GRPO now and I've
30:49been doing it for four weeks. Hello,
30:51it's me and I've been running it for the
30:53past 10 days. It's been really fun
30:55because it's more of an interactive way
30:57of teaching the model. The model will
30:58run the same task and I set it to 16
31:01times. So if it solves it one, even just
31:03one time out of those 16 times, that's
31:05valuable training data. So you reinforce
31:08that and then you hope for it to
31:09actually improve the model teaching
31:11itself to improve. On day four, I ran
31:14out of tasks that fit that criteria. I
31:16thought I had enough. So, I had to
31:17quickly synthesize more. And this has
31:19been this back and forth crazy where it
31:21first improved and then it flatlined and
31:23then it started improving again. So, I
31:24was I think people underrate the value
31:27of a large set of problems. And this
31:30goes in a lot of ways. Like you can use
31:31this to train, which is huge, but you
31:33can also use it to just make your
31:35environment better. Like a a silly way
31:37to apply the lessons here to your own
31:39work is to go through all of the errors
31:42that your agents have hit like when they
31:43ran a tool call and it failed or all of
31:45the times where an agent went down the
31:47wrong path when working in your
31:49codebase. You can collect all of those
31:51and turn those into eval. And once you
31:53have the eval, you can start adjusting
31:55like your agent and cloud MD. You can
31:57edit some things in your codebase to
31:58make it more likely the agent goes the
32:00right way. And then you can evaluate how
32:03good your codebase is set up to lead
32:05agents to success in it. This isn't
32:07quite the same as training a model to be
32:09better, but I'd argue it's even cooler
32:10than that, cuz we're not going to be
32:12using the models we're training
32:13ourselves for our day-to-day code work.
32:14Let's be realistic here. But you can use
32:17similar methodologies to make your
32:20codebase easier for agents to work in
32:22and be a more productive engineer.
32:23supposed to publish his video two months
32:25ago, but I
32:27>> did you cause an outage in AMP by
32:29scraping your team sessions. I am proud
32:31of you. Good work, sir.
32:33>> Line and then start improving again. So,
32:34I was supposed to publish this video two
32:36months ago, but I
32:38>> don't worry, Pudes. I'm reacting to your
32:40video a month late as well, and this
32:41will probably go out in 3 weeks. So,
32:43honoring the trend you've set here, sir.
32:46>> Just cannot stop. I cannot stop it. This
32:49video is supposed to go up days ago.
32:52[laughter]
32:52I can't stop. I can't stop. I know it
32:56can do more, but I will do it two more
32:58days and then I will have to run the
33:00[ __ ] ablation again and I have to
33:02quantized and benchmark to make sure
33:05nothing was lost. I'm cringing so hard
33:07cuz I know that's impossible for me to
33:08do, but I'm going to do it anyway. I'm
33:10going to do it anyway. I'm going to
33:12gamut in it. Like, I'm giving it my
33:13best. I really want you guys to enjoy
33:15this model. I worked so hard on it.
33:19But yeah, let's let's wrap this up. That
33:21is actually one of the like subtle hard
33:23things I don't think people understand
33:24with models is you can just keep
33:26refining. There is no clear like this is
33:28done and ready to go out. It's it's
33:30somewhere between like a code project
33:32where you have to like decide like what
33:33is your bar, but it's also like an art
33:36project where it just kind of like
33:38becomes its own thing and it's also like
33:40a a plant you're growing because you
33:41don't have as much control over where it
33:43goes as you think. You you can point it
33:45out the direction you want it to go in
33:47but you cannot actually control how the
33:49model grows. And at some point you have
33:51to decide like this plant has grown
33:52enough. I am ready to name it and share
33:55it. Now I feel like I spent the last 8
33:58weeks going through the dos and don'ts
34:01by doing the don'ts over and over and
34:04over and over and
34:06guess who didn't finish in time.
34:08[snorts]
34:08I didn't learn the don'ts. But it's okay
34:11because
34:13I have a countdown timer so you can't
34:15complain. Anyway, I'm excited for you
34:17guys to try this model. It is a tiny
34:20tiny tiny tiny model, so hopefully you
34:22can run it on your own computer. And if
34:24you can't, you should feel bad cuz your
34:27computer's bad. Now, let's continue to
34:29me. I love to do napkin math. I was so
34:32excited to go download this and try it
34:33on my computer, and it's still not out.
34:36You know, I thought that being a month
34:37late to the video would be a good time
34:39so I could go try it
34:41>> on how many versions of my computer it
34:44would
34:44>> not to interrupt again, but I think this
34:46is the most engineering thing he's ever
34:47done is a constantly delayed deadline,
34:51finally sets a hard deadline, takes the
34:53time to code and put up a timer and then
34:56still misses it. He's one of us, boys.
34:58We got a real engineer here. take to run
35:00one instance of like OpenAI or
35:03Anthropics model and they've been
35:04rumored to be one trillion parameter
35:06models which I would roughly estimate 27
35:09versions of my computer [music] to run
35:11one instance of a model. I think I
35:14counted like 150 houses or something in
35:16terms of electricity. And I think it
35:18makes no sense to poke that giant beast
35:21to be like, "Hey, can you check my email
35:23for this one thing?" Right? I really
35:26strongly believe in the small model
35:27trained for a specific harness. I'm
35:29excited for you guys to try it out. For
35:31most way people use it for searching or
35:34>> I'll push back on this a bit. I do agree
35:36that like you shouldn't be having Fable
35:39go through your email. Like that's an
35:40obvious waste of Fable capability. But
35:43once you get into like sonnet and opus
35:44tier, you end up with pretty good cost
35:48performance capability, especially at
35:50like the further extremes. Or if you
35:52like want something really cheap, Luna
35:55is an unbelievably efficient like cost
35:58performance in speed model. Like it's a
36:00really good model for the price and the
36:01speed gets crazy. All of that said, as
36:04Mixture of Experts models get better and
36:06better at distributing the experts and
36:08routing you to the right subportion of
36:11the model, just because the model's 10
36:12trillion params doesn't mean that you're
36:14going through all 10 trillion of them
36:15when you send requests. So, you can
36:18almost think, and it's not that simple
36:19at all, but you can kind of think of
36:22these giant models as a shitload of
36:25slightly smaller models. How many of the
36:28parameters are activated in a request is
36:30very different from the amount of
36:31parameters that are available to be
36:33activated. A sillier way of framing this
36:35is like if you're cooking a grilled
36:36cheese, you don't need the world's best
36:38kitchen. But if the three tools you need
36:41are in that kitchen, you're good to go.
36:43Could this have been done in a dorm
36:44room? Maybe. But if you already have the
36:47nice kitchen and it's not that
36:49inefficient, just go do it. So I I think
36:52this fear of using fancy models for
36:54unfancy things is not the best mindset.
36:59Like we care too much about right sizing
37:01the model to the task and it doesn't
37:03matter as much as we seem to think it
37:05does. Researching or browsing just
37:07saving time going through a lot of
37:09pieces of information that's what is
37:11generally good at not replacing
37:13creativity not replacing thinking. And I
37:16think personally a [music] lot of
37:18problems with AI you can solve by just
37:20running it on your own. To me, it's like
37:23getting information to you quickly.
37:26That's what it's good at. That's a
37:28future that I think is exciting and
37:30that's what I want to build with Odysius
37:32and Ajax. [music] So, I get it. I also
37:34would love for our email to be more
37:37easily sorted on our own devices using a
37:40model that we own and control. Right
37:42after that though, like it's okay, it's
37:44not right after it's a meaningfully
37:45distant segment. Like I get it. I I
37:47really want to sort my email. I fully
37:48with you. Number one most important
37:50thing. I do think hearing cancer's on
37:52the list, but I have a lot of spam
37:54emails. So like we definitely should
37:56prioritize that first. Check it out.
37:59Have fun. Be responsible. And I can't
38:02wait to release the next version of Ajax
38:04and Odysius, which has gotten a huge
38:06update as well. And huge thank you to
38:08all the maintainers for reducers for
38:10working so hard and and putting so much
38:12passion and thought into it. I really
38:13really appreciate how many maintainers
38:15does he have on this. I'm actually
38:16curious. He has more stars than us.
38:18Damn. T3 Code is at 46k stars as of the
38:23time of recording. Adysius is almost at
38:25100K. I'll put my I'll put my backing
38:27behind this. I I have a new bar, a new
38:31litmus test for when code actually
38:34doesn't matter anymore. You guys ready
38:36for this? When GitHub stops putting the
38:40code above the readme on a repo, that's
38:44that's my new bar for AGI. When even
38:47Microsoft has accepted that the code is
38:50less important than the context, then we
38:53have possibly finally achieved AGI. But
38:55as long as I have to scroll through this
38:57much stuff to see what the project is,
39:00we're not there yet. And yes, I do
39:02agree, Agore. They should have made this
39:04change 10 years ago, but we're talking
39:06about Microsoft speeds. And if AGI is
39:09achieved, even Microsoft should be able
39:12to do the right thing. This might be a
39:13separate bar. I might have AGI and ASI
39:16because artificial general intelligence
39:17would be Microsoft making a good product
39:19decision. Not fathomable right now, but
39:21in a future where AI is that smart.
39:24Sure. Artificial super intelligence is a
39:26different bar. ASI is hit when Google
39:30ships a piece of software that doesn't
39:31suck. If Google makes an app that is
39:34pleasant to use, we have achieved super
39:36intelligence. That those are my bars.
39:39And if you want to join, we're at the
39:41GitHub. I also want to shout out my
39:43sponsor again, boot.dev. I strongly
39:47believe that AI should not replace you
39:49learning new things or thinking for
39:51yourself. You still need the basic
39:53understanding of different things. And
39:55if you get to that point, that's when
39:57you can start accelerating. At least
39:58that's my opinion. I don't know. So
40:00yeah, check them out. It's so much fun
40:02to learn new stuff. I highly recommend
40:04it. Link is in the description. Bye-bye.
40:08>> I have been challenged to answer some
40:10rapid fire questions.
40:12>> Oh boy.
40:12>> That I am going to start with now. They
40:14>> I will leave this to the viewer. If you
40:17would like to see the Q&A, there has to
40:18be some reason to go back to the
40:19original, right? It's there if you want
40:21to see it. That was quite a video. I
40:24don't know why I would ever discount
40:26PewDiePie's legitimacy at this point in
40:28time. I've learned better at this point,
40:29but god damn, he's on top of his stuff.
40:31That was super cool. And if PewDiePie is
40:33ahead of you in the AI space, you're
40:35probably still fine cuz he's very far
40:36ahead. Uh, Peudes, if you're looking for
40:38a new gig, hit me up. We might be hiring
40:40soon. Everybody else, please don't. I'm
40:42actually not hiring anytime soon. We We
40:44have too many employees of anything. But
40:45yeah, this was fun. I hope you enjoyed
40:47it. I'm going to go grab another drink
40:49and relax a bit. This is a nice chill,
40:51different vibe for a video. If you
40:53enjoyed it as well, let me know. And
40:54maybe we can sneak more things like this
40:56in into the future. as long as my
40:57sponsors aren't too mad that I was
40:58drinking in their videos.

About this transcript

This page contains the full transcript of I can't believe Pewdiepie made this by Theo - t3․gg, generated from the public captions YouTube serves with the video. The transcript has 8,197 words across 1,135 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.