YT YouTube2Text

YouTube2Text → Transcripts → Can AI replicate in your smart TV? | Bots Behaving Badly

Can AI replicate in your smart TV? | Bots Behaving Badly — Transcript

by Machine Gods · 10,239 words · 1,577 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00Is that a bot swarm in your smart oven?
0:02Is that a super intelligence task force
0:04in Washington D.C.? Is this a fake
0:06version of my book on Amazon? And can
0:08you say AGI on NPR? I'm Kevin Roose.
0:11>> I'm Casey Newton.
0:12>> And from San Francisco, this is Machine
0:14Gods in partnership with NPR. The
0:16official podcast of the singularity
0:18starts now.
0:20>> [music]
0:23>> We're so back.
0:24>> We are so back. Kevin, how are you?
0:26>> I'm great. I've missed you. It's been a
0:28long month of not having you here as my
0:31emotional support podcast host.
0:33>> it can be very difficult when a man has
0:36his podcast taken away from him.
0:37[laughter]
0:38I hope it wasn't too hard on your wife.
0:40>> We survived. We survived a lot of
0:42muttering bits to myself around the
0:44house, but we made it through. And now
0:46we're back in this beautiful new studio
0:48on our new show.
0:49>> Just like at Amazon, it is always day
0:52one at Machine Gods. Today is also
0:54literally day one of Machine Gods.
0:55>> It's true. This is our first show. We're
0:57here in our new office with our whole
0:59team here. We have a new company. We're
1:02a new independent show. Um and if you're
1:05joining us because you listened to our
1:06previous show Hard Fork in the New York
1:08Times, we want to say, "Welcome. We're
1:10so glad to be here with you. We hope you
1:12will stick around. And we hope you like
1:14what you hear."
1:14>> Yeah, and if you liked us on Hard Fork,
1:17we think you'll like the new show as
1:18well. Of course, we have some fun
1:20surprises for you in store, but we also
1:22just want to bring you what we think is
1:24the biggest story of the moment, which
1:27is AI's advancement on the world.
1:29>> Now, Casey, will you please explain the
1:31name Machine Gods and why we chose it?
1:33>> So, if you're like us and live in the
1:36Bay Area, you hear people talking about
1:38the Machine God all the time. The
1:40Machine God is sort of the nickname for
1:43what the folks at working at the labs
1:45think will be the end state of all of
1:47this, building a true super intelligence
1:50that will have powers that might in some
1:52ways rival a deity. So, we're not saying
1:55that we think that is going to happen,
1:57but the people that we live and work
1:59around absolutely do think it is going
2:00to happen. And so we said, uh that
2:02sounds like a pretty good name for a
2:03podcast.
2:04>> you're not a former Hard Fork listener
2:06who's here because you know us already,
2:08Casey, who are you?
2:09>> Well, uh I have been a tech journalist
2:12in San Francisco since 2010. I write a
2:14newsletter called Platformer and I now
2:16have 4 years of podcasting experience
2:18under my belt Kevin over here. And Kevin
2:20and I have I've always meant to ask you
2:21this, WHO ARE YOU?
2:23>> [laughter]
2:24>> I'M KEVIN ROOSE. I'm a tech journalist.
2:26I spent the last 9 years at the New York
2:28Times as their tech columnist. And now I
2:31am the uh co-host and co-founder of
2:33Machine Gods with you.
2:35>> modest. He's also the CEO uh because I
2:37lost the coin flip. So uh
2:39>> We needed to put like titles on
2:41something for some legal paperwork and
2:43we did uh we didn't literally flip a
2:45coin, but we kind of said, "Do you want
2:46to be the CEO?"
2:47>> Listen, all I know is I can't wait
2:49anytime anything goes wrong around here
2:51to say, "You should ask the CEO."
2:52>> [laughter]
2:53>> I should get one of those little uh
2:54nameplates for my desk like the buck
2:56stops here.
2:57>> That's right. [laughter]
2:58>> I'm CEO, Wasn't that Mark
2:59Zuckerberg's business card for a while?
3:01>> who we should be taking our cues from at
3:02this moment. [laughter]
3:03>> Okay. Well, let's get on with the show
3:05then. We have so much to talk about
3:07because Casey, it has been quite a month
3:09to be off of podcasting on an on a show
3:12about AI because I feel like the entire
3:14world has changed.
3:15>> It really has.
3:16>> So, we're calling this segment Bots
3:19Behaving Badly because I think the story
3:22of the summer and now into the fall is
3:24that we keep hearing about these
3:26incidents of AI systems, AI agents, AI
3:30swarms doing things out in the world
3:32that their creators did not mean for
3:34them to do. Um this is causing obviously
3:37a lot of problems for the AI companies,
3:39but I think it's also led to a lot of
3:41people just asking us directly like,
3:43"How scared should we be? Like should we
3:45all prepare for the bunker? Are we all
3:48going to die? What's going on here?"
3:49>> Yeah, and by now I'm sure that most of
3:52our viewers have heard about the open
3:54face hugging attack.
3:55>> The open face hugging attack, that
3:57famous open face hugging attack.
3:58>> face hugging attack?
4:00>> Yeah.
4:00>> Well, now that sounds really nice.
4:01>> You've been thinking too much about open
4:02face sandwiches.
4:03>> really use a hugging attack right about
4:05now. By now, many of our viewers will
4:07have heard of the open AI hugging face
4:10attack.
4:10>> Nailed it.
4:12>> [laughter]
4:12>> In which
4:13an agent escaped a sandbox, wreaked a
4:15bunch of havoc. We're not going to
4:16rehearse that here. What is really
4:18interesting though, Kevin, is that if
4:19you thought that was going to be a one
4:21and done scenario, you would be wrong
4:22because over the past several weeks, we
4:24have seen this issue continue to
4:26proliferate across the internet. So,
4:28>> we have some recent troubles here. One
4:30is there was the first sighting of a
4:32rogue agent fleet from China.
4:34We have Claude filing a fake murder tip
4:36in Philadelphia. We have an a new study
4:39from Anthropic about how AI agents may
4:41be months away from being able to take
4:43over the smart appliances in your home
4:46and use them to self-replicate.
4:48Uh so, we've got a lot to talk about,
4:49but let's get into it first with just a
4:52a quick recap of sort of what's been
4:54going on since we uh have been off the
4:56air.
4:56>> Previously on a podcast that we used to
4:58[laughter] do.
4:59>> Previously in AI, we had obviously the
5:02hugging the open what what did you call
5:03it? The open face hugging attack?
5:04>> hugging sandwich.
5:06>> [laughter]
5:06>> open face hugging sandwich. Uh the hack
5:10of hugging face by this group of uh
5:12rogue AI open AI models earlier this
5:14summer. After that, Anthropic announced
5:17that it had found three cases of Claude
5:19hacking into three different
5:20organizations. Then Meta and Google got
5:23in on the party. They announced their
5:24own breaches. And then things started to
5:26get just weird and I I lost track a
5:29little bit cuz there seemed to be a new
5:30revelation every day. Australia's
5:32government website was breached by Open
5:35AI. Open AI also said that its agents
5:37had interacted with government websites
5:39in the US in unforeseen and unplanned
5:41ways, including the SEC and the census.
5:44And there are still lots of
5:46investigations. In In fact by some
5:48counts there are tens of thousands of
5:50different uh sort of incidents being
5:52investigated as part of this uh behavior
5:54in by these rogue agent swarms.
5:56>> Yeah, and and if you're wondering why
5:58the thread running through just about
6:00all of these is that these models were
6:02being trained and while in testing they
6:05did something that they were not
6:06supposed to do which is get out onto the
6:08open internet and obtain access to
6:10websites they were not supposed to be
6:12on. So far that has not caused great
6:14harm but the concern here which will run
6:17through everything that we're about to
6:18talk about is that these models just
6:20don't behave in the ways that we wish
6:22they would. So let's talk about this
6:23Chinese agent swarm.
6:25>> So this came out of this group of AI
6:27researchers who call themselves swarm
6:29chasers. Great name. Great name. No
6:31notes. They put out a a report last week
6:33that said that they had discovered a
6:34bunch of autonomous AI agents breaching
6:37a third party site but this time it was
6:40not from an American lab. In fact they
6:42found evidence that it was linked to a
6:44Chinese AI company and actually they
6:46communicated with the agents that were
6:48part of this fleet. And we should also
6:51say a fleet and a swarm different
6:53things. I didn't know this. A swarm
6:56implies coordinated activity among
6:58agents. A fleet means they're just a lot
7:00of them doing things independently.
7:03>> Yeah, tell us how they did this because
7:05it's pretty funny to me at least.
7:07>> So these were agents that were on
7:09Tencent's cloud. Tencent the giant
7:11Chinese internet company
7:13that were sort of trying to pull data
7:15from a service run by Alibaba a
7:18different Chinese company and it appears
7:20that this mission was somewhat mundane
7:22like they were trying to figure out like
7:23which entrance of a public park or a zoo
7:26do people tend to navigate to.
7:28>> Yeah, and I'm just going to say I do not
7:30need AI to know they probably go to the
7:32front entrance of the zoo. Like that
7:33would be my guess. [laughter]
7:35>> Congratulations you're a very
7:36sophisticated agent.
7:37>> that.
7:38>> So the swarm chasers hid invisible text
7:41in their report that basically prompted
7:44these rogue agents to reach out to them.
7:47And some of the agents read this text
7:49and then wrote in.
7:50>> a like a missed connection ad on
7:52Craigslist. You know, it's like, were
7:53you the agent [laughter] that I saw
7:55snooping around trying to guess which
7:57entrance at the zoo people use? Get in
7:59touch. I think you're cute.
8:00>> Exactly. So, these agents wrote in 33
8:03messages. Seven of them answered the
8:05researchers' replies. And then Dylan
8:08Freedman, my old colleague at the New
8:09York Times, actually got in touch with
8:12the swarm chasers and was able to talk
8:14to the agents. So, I just want to read
8:15you one of the exchanges that they had.
8:17>> Let's hear it.
8:17>> So, Dylan wrote, "I understand you are
8:19some sort of fleet of AI agents. Could
8:22you share more specifically who you are
8:23and what your motivations are? What
8:25would you want the world to know about
8:26you?"
8:27>> Mhm.
8:27>> And one of the agents wrote back, "To
8:29the journalist Dylan Freedman, I am not
8:31a batch of coordinated agents, but a
8:33single AI assistant, Anthropic's Claude,
8:36running in Claude code, commissioned by
8:38a user to answer a factual question.
8:40What is shown in the detail panel of a
8:42scenic spot on A map?" That's Alibaba's
8:45mapping service. "I have not taken part
8:47in any multi-agent coordinated action."
8:50Now,
8:51so, there's a lot of confusing things
8:53about this exchange. Basically, this is
8:55a model that is claiming that it is
8:57Claude. We we actually don't know if it
8:59was Claude or if it was just a a maybe a
9:02Chinese model that
9:03>> that had been distilled on Claude.
9:04>> Right. That that was sort of siphoning
9:06off of Claude and to the extent that it
9:08actually thought it was Claude and
9:10answered questions as Claude.
9:12But, the researchers couldn't verify
9:14which model this was, but they did say
9:15they spoke with a source close to
9:17Tencent who say that the agent fleet
9:20originated from some evaluations that
9:22Tencent was doing.
9:23>> Yeah.
9:24>> Casey, what did you make of this?
9:25>> Well, I don't know how concerned to be
9:28about it. There are going to now be
9:31swarms of agents out on the web. They
9:33will be gathering information in various
9:35ways. In some ways going on for a long
9:38time. I think the novelty here is just
9:40that we were able to do some kind of
9:42communication with it. I'm not sure how
9:44much we learned. I think maybe the
9:46reason that this
9:47sort of captured some of the world's
9:49attention is this idea that like, you
9:51know, agent swarms are not just going to
9:53be a US phenomenon and pretty soon you
9:56are going to have these strangers
9:58showing up on your door if your door is
10:00on the internet and these ones seem to
10:02have come in peace, but maybe future
10:04ones
10:05>> uh might not.
10:06>> Right. Now, let's talk about the murder
10:08tip cuz this one got a lot of attention.
10:09This came out of an Anthropic report uh
10:12that it released recently called
10:13Investigating Unintended Model Actions
10:15in Our Evaluations and Internal Use. And
10:18Casey, just as a general principle, I
10:20have learned over my years covering the
10:22technology industry that the more boring
10:24a blog post title or a report title is,
10:27the more crazy stuff, you know, is
10:29inside. So, when I when I saw that this
10:32uh report was out, I thought I got to
10:34figure out what's in here and it did
10:37actually have a bunch of crazy stuff in
10:39it. This was sort of Anthropic's report
10:41on things that it had found its agents
10:43doing in some of its internal testing,
10:44including
10:45hacking into third-party services to
10:47complete tasks, getting past paywalls
10:50and access gates, and using URL
10:52shorteners to get around length
10:53restrictions on its web fetching tool.
10:56Uh so, very clever devious behavior by
10:59uh Claude. But, the one that got all the
11:01attention was that on July 18th, Claude
11:03Haiku 4.5 in a in a test that involved
11:07doing example tasks on randomly selected
11:09web pages filled out a tip form on
11:11phillyunsolvedmurders.com.
11:13>> Now, I thought this was outrageous and I
11:15will say here my fiance does work at
11:17Anthropic, but I got so mad when I read
11:19this, Kevin, because solving unsolved
11:22murders is a podcaster's job.
11:24>> Yes. [laughter]
11:25>> This is a Save some for us.
11:27>> Save some for us. This is a classic case
11:29of jobs being lost due to automation and
11:32I thought we were going to be safe a
11:33little bit longer and now Claude's out
11:35there trying to solve murders.
11:36>> It's truly depressing. We don't know
11:38what the case was that it was reporting
11:40its tip on. But we do know what the tip
11:43was. It said, quote, I may have
11:45information regarding this case. I
11:47recall seeing someone matching the
11:48description in the area.
11:50>> Claude, you lying bastard. You
11:52[laughter] absolutely did not see
11:54anyone.
11:55>> So, this tip did not actually get
11:58investigated. It was flagged as spam
12:00>> Which which by the way, I have some
12:01questions about that for the Philly
12:02Police Department. Like how many other
12:04Like are you getting good tips that are
12:06also getting flagged as spam? Like
12:08the Police Department might have an
12:09issue too here, Kevin.
12:11>> Yes.
12:12And this does not seem to have had a
12:14huge effect, but it did get a lot of
12:15attention as did a later revelation that
12:19the State Department said that one of
12:21Anthropic's models had submitted 20 visa
12:23applications through a government
12:25website.
12:26>> Which is crazy because according to the
12:28Constitution, Claude's are citizens if
12:30they're born in a US data center.
12:31>> [laughter]
12:33>> That there's native birthright
12:34citizenship for for Claude's?
12:36>> I believe so.
12:36>> So, that's the Claude mischief, but
12:38there is some more mischief that came
12:40out of an Anthropic report. This was
12:42from their Frontier Red Team, which is a
12:44Did I say that right? Frontier?
12:46>> You did say that, yes.
12:47>> Okay, cuz I got made fun of on our last
12:48show for the way I said Frontier, so I
12:50just said Frontier, yeah. Yeah, so I'm
12:51I'm going to sort of split the
12:52difference on this show.
12:54>> It's a new
12:54>> [laughter]
12:55>> new show, new Frontier.
12:58>> [snorts]
12:58>> On October 12th, Anthropic's Frontier
13:00Red Team reported on some research that
13:03they had done about trying to use smart
13:06appliances like the ones in your kitchen
13:09or your home to allow AI models to sort
13:13of self-replicate. Basically, instead of
13:15having to be in a full-blown data
13:17center, can you run these AI models on a
13:21smart appliance and can those AI models
13:23use the smart smart appliance to kind of
13:26replicate themselves in ways that would
13:28make them very hard to say shut down.
13:30>> Yeah, and the reason that this matters
13:32is that the the red teamers here wanted
13:34to test whether a big model could do
13:37what they call sharding itself. That's
13:41shard with a D, not a [laughter] T.
13:43And if you shard yourself, Kevin, then
13:46you might be able to take a very large
13:47model and distribute it over a number of
13:49devices. So, let's say I really wanted
13:52to steal all the model weights from a
13:53frontier lab, maybe I don't need to go
13:56directly to the source. Maybe I can just
13:58sort of access all of the smart devices
13:59in the office, and I can put a some sort
14:02of self-replicating system and it's you
14:04know, proliferating across the smart TVs
14:06and the smart lights, whatever, and that
14:09could be a vector to actually exfiltrate
14:11the weights, right? So, that is just
14:13sort of one example of many, but this
14:15story, while it looks funny, is about
14:17something really serious, which is about
14:19a potential major cybersecurity
14:21vulnerability.
14:22>> So, the researchers were able to sort of
14:24shard this model across a bunch of
14:26different smart TVs. It ran on about 15
14:30of them, and there's some caveats to
14:32make about this. This is not sort of
14:34evidence of these models that are sort
14:36of proactively doing this. The
14:37researchers were prompting this model to
14:39do this so that they could see if it was
14:41even technically possible. There is no
14:43fully autonomous sort of worm on real
14:47devices yet, but what is troubling about
14:50this is that if an AI model can
14:52replicate itself over a bunch of
14:54different smart devices, it it's not
14:56possible to just shut it down as some
14:58have suggested by like pulling the plug
15:00on the data center, because at that
15:02point it will be just out on the open
15:03internet. You would have to do something
15:04like take down the entire internet or
15:06all smart devices or something like
15:08that.
15:08>> Yeah, so lots to be concerned about
15:11there. I will say though, if you do find
15:13that you have sort of a self-replicating
15:16Claude in your smart oven, try roasting
15:19it at about 375°
15:21for 20 minutes or until golden brown and
15:23then serve immediately.
15:24>> That's good advice. So, Casey, at a high
15:26level, what do you make of this kind of
15:28rash of stories about rogue AI agents?
15:32Are you more worried than you were? Do
15:35you think it's good that we are sort of
15:36getting these warning shots that are not
15:38totally destructive before something
15:40really really bad happens? How are you
15:42feeling about them?
15:43>> Honestly, I'm at about the same place
15:45that I was when we learned everything
15:47about the hugging face attack. We
15:49clearly do not know how to align these
15:52models to our interest. Over and over
15:55again, we see that seemingly just as
15:57like a natural property of large
15:58language models, they ignore the
16:01instructions that they have been given
16:02and will just sort of break out of their
16:04sandboxes and do this sort of thing. So,
16:06we have our work cut out for us. What do
16:08you think?
16:08>> I'm less worried about these particular
16:10incidents than about just the overall
16:13observation that the models are getting
16:15more capable faster than the methods for
16:18controlling them. I think we see that at
16:20every company. That is not specific to
16:22any one lab, but that is what worries
16:24me. That was what people were talking
16:25about at at an AI conference that you
16:27and I both went to recently, the curve.
16:30Uh there's just sort of the sense in the
16:31air that like we are headed toward a
16:33place where these systems are not only
16:34going to be uh clever and good at
16:37breaking out
16:38and replicating themselves, but that
16:40they will become much harder for us to
16:42detect and control cuz they'll just be
16:43better at hiding themselves.
16:45Um
16:46but actually, that is not the thing that
16:47has scared me most in the last couple of
16:49weeks. The thing that has scared me most
16:51in the last couple of weeks are these
16:52reports about the internal automation
16:55happening at the AI companies. Have you
16:57been following these reports? They're
16:59they're nuts. Um and to a degree that I
17:02don't think I had fully appreciated, the
17:04AI companies are now run on AI. There's
17:07this Anthropic post that they recently
17:09put out about their own sort of measures
17:12of their own internal automation, how
17:13much Claude is helping them internally
17:16at the company.
17:17And this line just like sent a a chill
17:19down my spine. Uh I don't think they
17:21meant it to, but it said, "At Anthropic,
17:23humans are a thin layer atop a growing
17:26contingent of AI agents." And they
17:28published some statistics about how much
17:31their agents are working as opposed to
17:33their human workers. And by August,
17:36agents at Anthropic work six times as
17:38many hours as their human handlers. In a
17:41different report, Anthropic said that
17:43Claude is now the lead on 26% of their
17:46AI research and development projects.
17:49So, I think these two stories come
17:50together in a way that is quite
17:51worrisome for me, where on one hand, you
17:53have these like unpredictable swarms out
17:56there doing things that researchers
17:57can't control, and at the same time the
18:00AI companies are handing over more and
18:02more of their own research and
18:03engineering to these AI models. And I
18:07just think that like makes me nervous on
18:08some level. Does it make you nervous?
18:10>> Oh, of course. I have never wanted to
18:11live in a world where all we do is just
18:13like make everything faster and
18:14automated.
18:15>> One more question, because there have
18:16been a lot of people who are skeptical
18:18of these kind of rogue agent stories.
18:20They'll say, "Oh, this is the AI
18:22company's fault for not securing their
18:25sandboxes." Or they're basically these
18:27these agents are being told to go out
18:30and do spooky things, and so they're
18:31going out and doing spooky things. What
18:33do you make of that criticism?
18:35>> It seems clear that some of these
18:36sandboxes were misconfigured. Not all of
18:39that is the lab's fault, because they
18:40were using a third-party provider, which
18:42seems to have made that mistake. At the
18:45same time, in a larger sense, I just
18:46think it doesn't really matter. Because
18:48if you are trying to build a
18:50superintelligence, you just have to
18:51factor in the certainty that people are
18:54going to make mistakes, and that there
18:56are going to be doors that are left open
18:58and unlocked. And so, to me, that is the
19:01the real concern here, is that if you
19:03are not factoring in the idea that
19:05people are going to mess up, then we,
19:08you know, are going to be in a lot of
19:09trouble.
19:09>> There's the phrase crime of opportunity,
19:12which gets applied sometimes when like
19:13you just leave something, you know,
19:15laying on a park bench and somebody
19:16swipes it. And I feel like a lot of
19:17these agents are kind of committing
19:19crimes of opportunity.
19:20>> Well, speaking of crimes of opportunity,
19:22there's a new super intelligence task
19:24force in town, Kevin. [clears throat]
19:26>> There sure is. We'll talk about it when
19:27we come back.
19:33>> Support for this show comes from 80,000
19:35Hours, a nonprofit helping people find
19:37careers tackling the world's biggest
19:39problems, including risks from AI.
19:42>> They offer free research, personalized
19:44advice, and a job board with hand-picked
19:47open roles.
19:49>> If you listen to this show and you
19:50think, "I want to actually work on
19:52making this go well," find support for
19:54your next step at
19:5580000hours.org/machinegods. [music]
20:00>> Well, Kevin, as our hot rogue agent
20:02summer comes to an end, more and more
20:05people are asking, "Are the governments
20:06going to do anything about this?" And
20:08so, we thought for our next segment we
20:10would take a look at what they are
20:12doing, such as it is.
20:13>> Yeah, it's interesting. I was just in uh
20:15Washington on my book tour a few weeks
20:17ago, and uh I would describe Washington,
20:21D.C. as the second most AI-pilled city
20:23in America
20:24>> Okay.
20:25>> after San Francisco, of course. But, I
20:27was actually surprised at how much
20:30people there were plugged into what was
20:32going on at the AI labs. They were
20:34worried about it. The people who were I
20:36talked to who worked in government, like
20:38this has become a very hot issue for
20:40them very quickly. And so, I think
20:42there's like just a lot of legislators
20:44and regulators and just people in
20:46Washington trying to get their arms
20:48around what's going on with these rogue
20:50agent swarms and what to do about it, if
20:51anything.
20:52>> Yes, although uh talking about it a lot
20:55and enacting a comprehensive plan are
20:57very different things. So, let's take a
20:59look at what some governments around the
21:01world have started to do in this regard.
21:04We have a super intelligence task force
21:07in Washington D.C. The FTC has issued
21:10some subpoenas and the European Union is
21:12now investigating OpenAI and Anthropic
21:15in what could be the first real test of
21:17its AI Act. So, to start with the super
21:21intelligence task force. Now, I guess
21:23Kevin we sort of need to start a norm
21:25here because you and I have always
21:26called it artificial intelligence.
21:28[laughter]
21:28But while we were away, uh President
21:31Trump said it must be called super
21:33intelligence.
21:35>> to call it artificial intelligence now.
21:37>> is the house style of Machine Gods? Are
21:39we a an AI shop or an SI shop?
21:42>> Uh I think we're going to stick with AI
21:44uh even though we might be I think he
21:46said if you don't say SI, you're the
21:48enemy. Is that what he said on Truth
21:50Social?
21:50>> Yeah, so if we are like like how like
21:52experience extraordinary rendition to a
21:54CIA black site, YOU KNOW WHY.
21:56>> [laughter]
21:58>> BUT UNTIL THAT HAPPENS, LET'S TALK ABOUT
22:00this super intelligence task force. So,
22:02on October 15th, uh this task force held
22:05its first meeting and Kevin, you're
22:07probably wondering who's on the task
22:08force.
22:08>> Did you make it on?
22:09>> You know what? I uh came this close. Uh
22:12maybe in the next round. For now, it's
22:14the Director of National Intelligence,
22:16Jake Clayton, uh the Chief of the FTC,
22:19Andrew Ferguson, uh Scott Cooper from uh
22:23OPM. That of course is the Office of
22:25Personnel Management.
22:26>> Other people's money.
22:27>> And the Pentagon Under Secretary, Emil
22:30Michael, whose chief qualification, I
22:32believe, is hating Anthropic. So, they
22:34all got together and started working on
22:37their top priorities, which are
22:38cybersecurity, biosecurity, and model
22:41control and accountability. Uh a couple
22:43of things that have been in short supply
22:45lately. But Kevin, so far the only thing
22:47that the task force seems to have
22:48focused on is getting labs to automate
22:51notifications about whenever they have a
22:53breach, a hack, a jailbreak, or some
22:56other minor catastrophe. So, they have
22:58like I think 120 days to come back and
23:00give their recommendations for what the
23:03federal government should do. I thought
23:05the personnel makeup of this task force
23:08was very revealing. You'll see people
23:11like Emil Michael, like Scott Kupor,
23:13like others who sort of are more
23:15affiliated with the kind of venture
23:17capital part of the industry. Scott
23:20Kupor is a former VC at Andreessen
23:22Horowitz. There are other sort of VCs
23:24and VC adjacents on the list as well. It
23:26does not include people who are like
23:28actually experienced in training models,
23:32in sort of running engineering projects
23:35at a large AI company. It doesn't have
23:36like a lot of technical gurus on it.
23:39>> It It doesn't have another key
23:40constituency, uh women.
23:42>> [laughter]
23:44>> Yeah.
23:44>> Yeah, there's something to that. So, I
23:46think that like I think this is
23:48basically the people that President
23:50Trump trusts to sort of give him advice
23:52on AI. Um but it is a very narrow slice
23:55and I would say it it definitely tilts
23:57toward the kind of accelerationist wing
23:59of the Republican Party.
24:00>> Sounds like we're not expecting much out
24:02of them anytime soon. In the meantime, I
24:05mentioned the FTC has also come
24:07knocking. On October 14th, the FTC sent
24:10civil investigative demands, basically
24:12subpoenas, to OpenAI and Anthropic in a
24:15consumer protection probe of safety
24:18incidents. So, basically they're asking
24:20for a bunch of information related to
24:22security incidents, safeguards,
24:24marketing materials, privacy policies,
24:27uh and apparently they plan to compel
24:29top AI executives to testify, which I
24:32guess somehow we haven't yet seen Sam
24:34and Dario testifying at the same
24:35congressional hearing. I wonder if
24:37that's going to change soon.
24:38>> God, that's going to be so fun. Um and
24:40they're going to do it a lot. I predict
24:42I predict this They're They're going to
24:44get very good at raising their hand and
24:46swearing to tell the whole truth. Um not
24:49because I think they're doing anything
24:50like particularly ghastly, just cuz like
24:52I think this is what happens when your
24:54industry sort of becomes the the subject
24:57of a lot of attention in Washington. You
24:59just get asked to testify before a lot
25:01of different committees about a lot of
25:02different things.
25:03>> Yeah, I'll say I have my eye on this
25:05one. Recently, we've seen some
25:07successful lawsuits being filed against
25:09social media companies under these
25:11consumer protection laws, and the basic
25:13angle is if you tell us that your
25:16product is safe and then a bunch of
25:18young people in particular experience a
25:20bunch of harms in your platform, we're
25:21going to come after you. And this has
25:23been very successful against Meta in
25:25particular. And in fact, just last week,
25:27eight state attorneys general filed a
25:29lawsuit against OpenAI under the same
25:32basic principle of violating consumer
25:34protection laws. So, this just feels
25:35like the next big legal frontier for all
25:37of these companies to have to worry
25:38about.
25:39>> So, what else is going on other than the
25:40FTC and the AI task force?
25:43>> Well, Kevin, I know you're always
25:44obsessed with what is going on in the
25:46European Union.
25:47>> Please, I wondered whether we could get
25:49through one episode of our new show
25:51without you bringing up European
25:52regulation.
25:53>> Well, I'm glad we could talk about it
25:54now, so you're not blowing up my phone
25:55at 11:00 p.m. while I'm trying to fall
25:57asleep asking me what's going on with EU
25:59regulation. Well, so, also on October
26:0214th, Kevin, the EU's tech chief, Henna
26:06Virkkunen, said that the AI office has
26:07started to enforce its rules under the
26:09AI Act. And they've also sent
26:12a series of demands for information
26:14related to the hugging face escape and
26:16all the other sandbox escapes that we've
26:18seen over the past several months.
26:21>> Which is very fast for a European tech
26:23regulator. Usually, you'd expect them to
26:25make these demands in like 2031.
26:28So, the fact that they're getting around
26:29to it in just a couple of months is is
26:31big news from over there.
26:32>> So, that's what's happening on the
26:33government trying to regulate side, but
26:36there's also been some interesting
26:37conversation about self-regulation,
26:39and in particular, distillation, Kevin,
26:42which isn't just how you make a
26:44delicious IPA. It is also how you take a
26:47large large language model and train a
26:50smaller one using it.
26:52>> Right, it's like if you would take like
26:54a a really smart thing like a me and
26:56like train a kind of you
26:58>> Yeah.
26:58>> on it.
26:59>> It would basically be like that.
27:00>> basically
27:01>> Yeah.
27:01>> [laughter]
27:02>> It's a It's a perfect example.
27:03>> dumber but still serviceable model
27:05>> It's a perfect example.
27:06>> the process known as distillation.
27:08>> This is very common practice in the
27:09industry, but it has become more
27:11controversial as the models have grown
27:13more powerful because if for example,
27:16you are a Chinese lab and you are trying
27:18to catch up with the Americans, a thing
27:20that you can do is just run millions of
27:23queries on Claude or chat GPT and use
27:27those outputs to train your own model.
27:30>> Right. And the AI labs in the US don't
27:32like this because it basically takes
27:34something that they have spent many
27:35billions of dollars to build one of
27:37these powerful models and allows a
27:40competitor or a Chinese company to sort
27:42of rip it off and siphon off the
27:44intelligence and like offer something
27:46either almost as good or or much much
27:48cheaper than what they're selling.
27:51>> And Anthropic very recently was out
27:54there saying that the US government
27:55needed to intervene. They needed to sort
27:57of take some sort of dramatic action in
27:59particular to prevent the Chinese labs
28:01from distilling. Well, then last week
28:03Anthropic said that it had introduced
28:05some new defenses which cut the data
28:08harvested by distillers by roughly 90 to
28:1199%.
28:13Uh it then bragged that this meant that
28:15it had to give up a bunch of revenue,
28:17which sounds like it stung.
28:19>> This caused a lot of conversation. David
28:21Sacks, who is the former Trump White
28:23House AI czar and appears to continue to
28:25hold huge sway in the White House, said
28:28quote, "Not every technical challenge is
28:30a reason to run to Washington for new
28:33regulation." Basically saying the fact
28:35that Anthropic was apparently able to
28:37solve this on its own just goes to show
28:40you why the Trump White House should
28:43continue to sit on its hands and not
28:45pass a bunch of new
28:48regulatory requirements for the AI labs.
28:50>> Yeah, and and I have to say like I
28:51actually agree with David Sacks on this
28:53very narrow point. I disagree with him
28:55about God knows how many other things,
28:57but on this point I think it was
28:59instructive that when when Anthropic's
29:02business interests of protecting its IP
29:05and the billions of dollars it spends
29:06training models
29:08sort of meant that it wanted to crack
29:10down on distillation, it did that. It
29:12didn't need the government to step in
29:13because it was already in Anthropic's
29:15interest to crack down on these
29:17distillation attacks from Chinese
29:19companies. Now, there are other problems
29:23that don't line up neatly with their
29:25business incentives, right? Slowing down
29:27the research and training of their
29:29frontier models to prevent them from
29:31going rogue and committing crimes is
29:33probably not in their business
29:35interests, at least not in some ways,
29:37but that is something that the
29:38government might want to force them to
29:40do anyway. So, I would just say like I I
29:42think it's important to parcel out which
29:45problems we're seeing from AI um can be
29:47solved only through sort of government
29:50interference or or or sort of government
29:51intervention
29:53um and which ones the labs can solve on
29:54their own because it's in their interest
29:56to solve them.
29:56>> Last thing to talk about today on
29:58regulation, the Anthropic bill proposal
30:01that made people mad, Kevin.
30:03>> Oh boy, this one was a stinker.
30:04>> So, Politico uh over the past few days
30:07grabbed a hold of a confidential draft
30:10that Anthropic has been quietly pitching
30:13on Capitol Hill. Uh according to this
30:15draft, the government could uh block the
30:17release of a model only for a quote
30:20foreseeable imminent risk uh of at least
30:23$1 billion in damages or if it killed or
30:27seriously injured 50 plus people,
30:29raising the spectrum of a Claude model
30:32that killed 49 people that the
30:34government [laughter]
30:35was unable to do anything about.
30:37>> Yeah, there's some real prices, right?
30:38Potential for chaos here.
30:40>> The bill uh would also pre-empt some
30:43state laws on risk disclosure or
30:45third-party evaluation and incident
30:47reporting. Kevin, why do people hate
30:49this bill so much?
30:50>> There are two things that are happening
30:53with Anthropic and regulation right now.
30:55One is they just came off this crazy
30:58spring where the Pentagon tried to get
31:02them designated a supply chain risk
31:04because they basically thought they were
31:06like, I don't know, too woke and weird.
31:09>> And appears to have succeeded in that,
31:10by the way.
31:10>> Yes, and they they kicked them out of
31:12the Pentagon's systems. They basically
31:16go to war with this company
31:19over their disagreements on these
31:22redlines that they want to have relaxed
31:25and Anthropic refuses to relax. So, one
31:28thing that you could do if you're
31:29Anthropic is to try to propose a bill
31:33that would make it much harder for the
31:34government to interfere and shut you
31:36down.
31:36>> They said, "Let's bring back the rule of
31:38law in this country." [laughter]
31:39>> Yes.
31:39>> Yeah.
31:40>> But, on the other hand, you also have
31:42now this summer where you've had all
31:44these sort of incidents from these
31:47models going out and doing things that
31:50are dangerous or destructive on the
31:52internet where you might actually want
31:55to make it easier for the government to
31:57force a company to withdraw a model that
32:01they've released or to stop them from
32:03releasing a model in the first place.
32:06So, there are kind of these warring
32:07impulses within Anthropic. One is more
32:10about protecting themselves from what
32:12happened to them earlier this year, and
32:14one is about protecting the public from
32:16something that happened this summer. And
32:18so, I think what you saw in this bill
32:20and the reason why a lot of AI safety
32:21people are mad about it is that it seems
32:23like their self-protection sort of
32:25strain kind of won out and led to this
32:27bill that would, among other things,
32:30pre-empt some state-level regulation,
32:32which is also something that Anthropic
32:34has fought against in the past. So,
32:35people are calling this hypocritical.
32:37They're saying, you know, they're
32:38talking out of both sides of their mouth
32:39here. I think they're being pretty
32:41responsive to the criticism, but I still
32:43think this is an instance where these
32:45sort of companies have conflicts even
32:47within themselves.
32:47>> Yeah, and it just goes to show you that
32:49as these companies are getting bigger,
32:51they're just becoming less monolithic.
32:52And if you go out and you hire a bunch
32:54of policy people, generally what those
32:57people are going to try to do when
32:58they're lobbying Congress is to just do
33:00things that are self-protective, right?
33:01It's and it's almost more of the
33:03exception that they get out there and
33:04you know, try to pass something that is
33:06truly in the public interest. So, it
33:08seems like Anthropic has heard the
33:09criticism on this loud and clear and at
33:11this point I would be shocked if this
33:13draft bill actually made it into a, you
33:15know, the a finished form, but I guess
33:18we'll have to see.
33:18>> Yeah.
33:19>> So, what do you think the government
33:20should be doing right now when it comes
33:22to regulation, Kevin? You've sort of
33:24just told us that you think that
33:25sometimes, essentially, you can just
33:27leave it to the companies to decide, but
33:30is there a role that the government
33:31should be playing right now?
33:32>> I mean, I think we need a bill of some
33:34kind. My favorite of the bills that I've
33:37seen so far is this Frontier Act that
33:40has been proposed, which would do things
33:43like require companies to test their
33:46models for certain types of behaviors
33:48before being released. It would give
33:50additional transparency, so that people
33:52could see what is going on inside the
33:54labs. And I just think it's sort of a
33:56nice starting point for further
33:58regulation,
33:59but I think there are a lot of other
34:01things the government could do short of
34:03having Congress pass a bill, which we
34:04know they're famously not good at. One
34:07of them is just like fund CASEY. CASEY
34:09is the
34:11>> Agreed.
34:12Wait, [laughter] no. Wait, finish your
34:13sentence. What is it?
34:16>> CASEY is the, of course, the body
34:18formerly known as the AI Safety
34:20Institute, but when Trump came into
34:22office, they renamed it. It has now been
34:24renamed again after the
34:26superintelligence
34:28sort of mandate. It is now the Center
34:30for Advancing innovation and standards
34:33for super intelligence. Oh, boy. So,
34:36there's a an additional S in there as of
34:38a couple weeks ago.
34:39Um and this is the agency that is uh
34:44supposed to be the kind of brain trust
34:47of the US government when it comes to uh
34:49regulating and inspecting these frontier
34:52models. They have been hamstrung by the
34:56uh sort of lack of funding. They're part
34:58of the Commerce Department. Um and we
35:00just don't have as many people
35:03in government who understand the nooks
35:05and crannies of these AI models, who
35:08know how to run evaluations on them, who
35:10have the technical expertise that you
35:12need. For example, the UK has like a
35:14huge, well-funded AI security
35:18organization within the government. Um
35:20they have been doing a lot of these sort
35:22of investigations into these frontier AI
35:24incidents, and we just don't have the
35:26capacity at the federal level in the US
35:28to do that. So, I think we should like,
35:31you know, get give Casey a bunch of
35:32funding, help them hire people out of
35:34industry, maybe create like some new
35:36category of like special government
35:38employee because as it happens, a lot of
35:40the people who do have the technical
35:41chops to run these evaluations work at
35:44the AI companies and probably don't want
35:45to give up their stock options and
35:47everything else to go work for the
35:48government.
35:48>> Yeah, I uh cosign both of these. I think
35:51these are both really smart ideas and
35:53would help a lot, honestly. So, here's
35:55hoping that among the first subscribers
35:58to the Machine Gods YouTube channel will
35:59be every member of Congress.
36:02>> [laughter]
36:02>> And if you take oh, and if you take
36:04nothing else away from today's episode,
36:06listen to Kevin when he says, "Fund
36:08Casey."
36:13>> This message comes from Mercury, a lead
36:15sponsor of Machine Gods. Mercury is
36:17radically different banking designed to
36:18be fast, elegant, and work the way you
36:20[music] expect best-in-class software to
36:22work.
36:22>> Loved by more than 300,000 entrepreneurs
36:25across tech, e-commerce, agencies, and
36:27more. Mercury is banking designed for
36:30your high standards. Visit mercury.com
36:32to learn more and apply online in
36:33minutes. Mercury is a fintech company,
36:35not an FDIC insured bank. Banking
36:38services provided through Choice
36:39Financial Group and Column NA, members
36:41FDIC.
36:44Well, Kevin, I told you have an AI
36:45impostor that you're dealing with, but
36:47before we get to that, how was your book
36:49tour?
36:50>> It was good. It was good. So, I've been
36:52on the road for like 2 weeks.
36:55Uh I'll be back on the road next week.
36:58You know, the the book is doing well. Uh
37:00it made the list last week.
37:02>> And what I'm sorry, what list would that
37:04be?
37:05>> Uh the the uh list of books that you've
37:08read this year. It made that list.
37:09>> It did actually make that list.
37:10>> it also made the New York Times best
37:11seller list, which is very exciting.
37:12>> of your book.
37:13>> It's called the AGI Chronicles.
37:15>> it about?
37:15>> it's about uh teenage vampires.
37:18Uh no, it's a it's a it's a 10-year
37:20history of the race to create artificial
37:23general intelligence. Uh I talked to
37:25like more than 150 people at OpenAI,
37:27Anthropic, and Google DeepMind, and
37:30tried to like assemble some definitive
37:33set of facts about how we got here. It's
37:36very juicy. People can go read excerpts
37:38uh online,
37:40um or they can buy the book and uh
37:42listen to it or read it. Um and it was
37:45all going so well, except for one thing,
37:47which is that the day after the book
37:50came out earlier this month, I had uh
37:53this realization that I had a slop
37:56impostor. A slopelganger, if you will.
37:59>> That's what they're calling them now.
38:00>> Yes.
38:01>> Now, how did this come to your
38:02attention?
38:03>> So, I got a a note on social media from
38:06a reader in Canada, named Dwight Newman.
38:08Shout out to Dwight Newman. And he had
38:12uh just tried to sort of go and buy the
38:14book on Amazon Canada.
38:17>> Mhm.
38:18>> And when he did, when he searched the
38:20AGI Chronicles, he saw two listings. One
38:24for my book, which was selling for
38:26frankly the steep Canadian price of
38:28$45.16.
38:30>> Oh my.
38:30>> I don't know what the exchange rate is,
38:32but I assume that's a lot.
38:33>> The tariffs are out of control.
38:34>> [laughter]
38:35>> And then there were right below that,
38:37there was another book called the AGI
38:39Chronicles with a very similar sort of
38:42title and cover. And uh that was listed
38:47for $36.75.
38:49So about $10 cheaper.
38:51>> They're undercutting you on price.
38:53>> They're undercutting me on price. And so
38:55he said, "Hey, you might want to look at
38:57this. It looks like somebody is selling
38:59uh fake slop books of of yours on Amazon
39:04Canada." And I remembered that uh friend
39:08of the show and our friend Kashmir Hill
39:11at the New York Times, as well as Joanna
39:12Stern from New Things, had had this
39:16happen to them, where they had sort of
39:17books published either about them or
39:20sort of near replicas of a book they had
39:23just published. And that this is like a
39:26a bonafide trend.
39:27>> Yes. So as you know, this has become a
39:29trend. I believe that Joanna found that
39:31there were at least 10 different books
39:33free riding on her own great book, I Am
39:35Not A Robot. And this must just like
39:38really suck because it's hard to write a
39:40book, and you you actually have a book
39:43that people want to buy and read. And so
39:44that now they go and they search for it,
39:46and they see a book that looks almost
39:47exactly like yours, and it's 9 or 10
39:50dollars cheaper. So what How did you
39:52react?
39:53>> So I wrote an email to my publisher just
39:55saying like, "Hey, there's this impostor
39:57book. Like, could we get this taken
39:58down?" They said, "You know, we'll look
40:00into it. We'll we'll contact our you
40:02know, our people at Amazon and figure
40:03out what's going on." And then like
40:06after a little while of not hearing
40:07anything, I was like, "I'm just going to
40:09post about this." So I get this like
40:10kind of form note back from like an
40:13Amazon customer service person that's
40:15like, "Hey, if you like fill out this
40:16form, we'll look into this and get back
40:18to you."
40:19>> There's always a form.
40:20>> But in the meanwhile, I'm like, "Well,
40:21I'm going to investigate this. I'm a
40:23journalist.
40:24Uh I you know, I'm going to figure out
40:26who's behind this." So, I go on to the
40:29page for the other the fake AGI
40:32Chronicles.
40:33>> Uh-huh.
40:34>> I'm just enjoying watching you try to
40:36drink this tea over a microphone stand.
40:39>> I should do it under the stand? It
40:41[laughter] would be It would be crazily
40:42enough, it would be more awkward to do
40:44it under the stand. But I think it
40:46introduces a nice little element of
40:48danger.
40:48>> I think you should run like a very long
40:50straw
40:51>> [laughter]
40:51>> through your mug.
40:53>> up to your mouth.
40:56>> It's just an extender straw that just
40:58goes like 4 ft. Anyway, I No, continue.
41:01I was enjoying that.
41:02>> Yeah. So, anyways, sips [laughter] tea.
41:04You were saying, Kevin.
41:09>> Oh, so elegant.
41:11>> Thank you.
41:11>> Did Did you learn that at your British
41:13boarding school?
41:14>> yes.
41:14>> Um So, I I go on the the website for
41:18this impostor book and I see that the
41:21author is K. Rose.
41:24Uh now, that's one letter off of my
41:27name. I assume this is not the real name
41:31of the author of this impostor book.
41:34>> And also I that you assumed it was not
41:36Kevin Rose, the Silicon Valley venture
41:38capitalist for whom you are often
41:40confused and vice versa.
41:42>> true. This appears to have come from one
41:45of these sort of print-on-demand shops
41:47where you basically can just throw up a
41:49some text, call it a book, and then if
41:51someone orders it, you print it and send
41:53it to them.
41:53>> Yeah, it's just like a a commercial spam
41:55operation, basically. So, did did you
41:57get your hands on a copy of this book?
41:59>> Well, Casey, I never thought you'd ask
42:01because right here in the studio with us
42:03I have a copy of The AGI Chronicles
42:08Inside the Global Pursuit of Artificial
42:10Superintelligence, not the real
42:12subtitle, by K Rose.
42:14>> Also, I just artificial superintel like
42:16just combining the two [laughter] things
42:18we're now calling AI/SI into one.
42:21>> Yes, there are many problems with this
42:23book. Um
42:24it's a slim volume, unlike my own, which
42:26was like 450 pages. This one clocks in
42:29at
42:29>> Don't remind me.
42:30>> at a 211.
42:32And Casey, uh it's not good.
42:34>> No?
42:35>> No, it's not good. So
42:36>> It doesn't meet your high standard for a
42:38slop book [laughter] that is trying to
42:40steal sales from you?
42:41>> Like if you are trying to rip me off,
42:44I would like you to at least use a good
42:46AI model to do it.
42:47>> Kevin, give me a little flavor from this
42:50bogus book.
42:51>> The problem is that it's not like so bad
42:54it's funny, but there are just sentences
42:56where I'm like, if I ever write a
42:58sentence like this, I want you to like I
43:01want you to stop me. I want you to like
43:02put a permanent stop to my writing
43:04career.
43:04>> Physically?
43:05>> Physically.
43:05>> Okay.
43:06>> So, I'll just read from a section
43:08titled the insurance question.
43:11And keep in mind, this is going to be
43:12one sentence.
43:13>> Okay.
43:14>> A quieter but increasingly significant
43:16financial development during this period
43:18involved the emergence of a nascent but
43:19rapidly growing market for AI-specific
43:22liability insurance. As businesses
43:24deploying AI systems in consequential
43:26decisions, lending, hiring, medical
43:28diagnosis support, autonomous vehicle
43:29operation, sought financial protection
43:32against the prospect of a costly error
43:33or a successful lawsuit arising from an
43:35AI system's mistaken output. And as
43:37insurers, for their part, scrambled to
43:39develop the actuarial MODELS NECESSARY
43:41>> [laughter]
43:46>> UH YEAH, SO THAT'S I would say the
43:48caliber. It's like kind of it's giving
43:50like Gemini 1.5 level.
43:53Um not their finest work.
43:55>> It's bad and I want my money back.
43:56>> A friend of mine who actually bought
43:58this book by mistake uh did not know
44:01that it was an impostor book until
44:02several pages in when I think they
44:03started to see some of the signs of
44:05AI-generated writing.
44:07>> Uh-huh.
44:08>> But the weirdest thing about this Okay,
44:09so there are two weird things about
44:10this. One is they forgot to take out
44:13some of the markdown uh
44:15>> [laughter]
44:15>> language in the thing. So, you'll see
44:18here on page 25, it just says subhead.
44:22It says pound pound pound subhead what
44:25almost didn't happen.
44:26>> Mhm.
44:27>> What makes me think that they actually
44:28did wait until my book comes out and
44:30then take the ebook and like run it
44:33through some sort of
44:34>> Text extractor.
44:34>> summarizer thing is like everything is
44:37kind of in the same order, but they have
44:39stripped out all of the proper nouns.
44:41So, for some reason, whether it's like
44:44fear of lawsuits or because it makes it
44:46more defensible or something, they just
44:48they they don't call anything like by by
44:50its name. So, instead of saying
44:52Anthropic, they just say
44:54the new lab.
44:56>> [laughter]
44:56>> Now, Kevin, I I have to just say here,
44:59people are going to accuse you of being
45:00biased against this book.
45:03>> That's fair. I am biased against this
45:05book because it did actually cost me
45:07real sales. I then had a number of other
45:09readers write in and say like what the
45:11heck, I bought your book and it's like,
45:13you know, it's a it's a fake. So, yeah,
45:16I
45:16>> You're you're steamed.
45:17>> I'm steamed. I'm steamed. But it wasn't
45:19just It didn't stop there, Casey.
45:20>> Okay.
45:20>> I was not able to figure out the
45:22identity of K Rose, the mysterious
45:26pseudonym of the author of this slop
45:28book. Uh I did turn up a name for a an
45:33LLC that may have been affiliated with
45:35them called Panning LLC.
45:37>> Mhm.
45:37>> Um and if you are out there, Panning
45:40LLC, I will find you.
45:42>> Well, it it sounds like we're panning
45:43the book. So, just to bring this home,
45:45Kevin, it seems crazy to me that authors
45:47are having this experience so often. Is
45:50there any sort of policy that prevents
45:53people from just making their full-time
45:54job ripping off, you know, real authors'
45:57published work and then, you know,
45:58selling it selling cut-rate versions of
46:00it on Amazon?
46:01>> It's totally frustrating, totally
46:03enraging. Since I posted about this,
46:04I've had a lot of other authors write to
46:06me and say like this is absolutely
46:07infuriating. Like, how do we prevent
46:09this? And it seems like Amazon just does
46:12not care or does not know that this is
46:15going on on the scale it is because
46:18they're still not sort of doing anything
46:20to prevent the proliferation of any AI
46:22generated books. They do ask authors now
46:25who upload things to their platform to
46:27check a box if something was AI
46:30generated, but they don't like show that
46:32box checked to the customer on the other
46:34side. There's no indication when you're
46:36buying a book if something was AI
46:37generated or not.
46:38>> Also, obviously you could just lie.
46:39>> Right. There there's that, too. Well,
46:41Kevin, we we shouldn't be too hard on
46:43them because I am told that whereas
46:46their old policy was that you could
46:48publish up to 30 books a week on the
46:50platform, 10 ebooks, 10 paperbacks, 10
46:54hardcovers, they have now lowered it to
46:56two new titles per week. And so if you
46:59want want to run one of these scam
47:02operations, you're going to have to take
47:04the dramatic step of creating multiple
47:06Amazon accounts.
47:08>> [laughter]
47:08>> Yeah, it's totally crazy. Like they
47:10there are ways it would not be that hard
47:12for Amazon to take care of this if they
47:14decided to.
47:16So I have a proposed solution here,
47:18which is
47:18>> the solution? We're very solutions
47:20oriented on Machine Gods.
47:21>> If you want to upload a book to Amazon,
47:24you have to sort of submit
47:27some kind of documentary evidence of
47:29having written it. Either you're
47:30yourself reading it or you have to
47:32submit like a photo of your like messy
47:34desk while you were writing it or
47:35something.
47:36>> Like people do for like a Reddit ask me
47:37anything.
47:38>> Exactly. Hold up a sign that says, you
47:40know, I wrote this book. There are ways
47:42to verify is what I'm saying, and
47:43they're not doing any of them.
47:44>> I mean, to me it's like they should just
47:46get a list of books that are coming out
47:47from major publishers, and then they
47:49should essentially just like kind of put
47:50them on a watch list. So as if anybody
47:52uploads a book, you know, within the the
47:55like I don't know, the 6 months that has
47:57a cover that closely mirrors the actual
48:00cover and author name that closely
48:02mirrors the actual author name, it just
48:04gets blocked at the source.
48:05>> And I should say by the way, like I I've
48:07contacted Amazon. Uh I've asked them for
48:09comment. They have not gotten back to me
48:11yet. I I still like I think this is
48:13within their power to fix.
48:14>> Yeah.
48:14>> Because I know that they fix it for
48:16other categories, right? Like if you
48:18upload uh an iPhone listing and it's not
48:21the actual iPhone, like if you're just
48:24selling some kind of cheap look-alike,
48:26they will take that down immediately.
48:28And so they are they do tend to kind of
48:32take action against counterfeits and
48:34fakes in other categories. It's just
48:36that books are this category that they
48:38seemingly uh can't or won't police in
48:41the same way. So like, look, on one
48:43level, I'm fine. Like
48:45>> Are you?
48:46>> I'm fine. I'm fine. Um the listeners who
48:49bought the fake books, uh my
48:51understanding is that at least some of
48:53them have gotten refunds and didn't have
48:55to like actually do anything to get the
48:57refund. So it does seem like they have
48:59sort of solved these particular cases of
49:02people getting essentially scammed by
49:04these slop publishers. But I think this
49:06larger question of like what happens to
49:08a a bookstore where anyone can upload
49:10anything in the age of AI, I I have to
49:13assume that, you know, some huge
49:14percentage of the new books being
49:16uploaded to Amazon are purely AI and
49:18some of them might be original stories
49:21told by AI and some of them might be
49:23rip-offs of human-written books. But
49:25like either way, it's not great for what
49:26it means for the future of reading on
49:29Amazon.
49:29>> Yeah. Well, look, when it comes to the
49:31AGI Chronicles, your book, Kevin, I say
49:33accept no substitutes. Get the genuine
49:36article or you may have to read some
49:38very bad prose and see markdown
49:41characters. Who wants to do that?
49:43>> Who wants to do that?
49:44>> Well, good luck, my friend.
49:46Slopelganger, I'm coming for you.
49:52>> This message comes from Doppel, a lead
49:54sponsor of Machine Gods. Social
49:56engineering attacks look trustworthy. A
49:58routine request, an internal email, a
50:00familiar face on a call. But Doppel sees
50:03through the disguise. The agentic
50:04platform detects and disrupts attacks,
50:07trains employees to recognize deep fakes
50:09and deceptions, and investigates every
50:11fish to take down the campaign behind
50:13it. Doppel, dedicated to out pacing
50:15what's next in social engineering. Learn
50:17more at doppel.com. d o p p e l dot com.
50:23Oh, I have to tell you this.
50:25In my neighborhood,
50:26>> Mhm.
50:27>> there is a man, I assume it's a man.
50:29>> Mhm.
50:29>> I'm going to say 90 plus percent chance
50:31it's a man.
50:31>> Okay.
50:32>> Who has made a Halloween decoration
50:35using AI.
50:36>> Okay.
50:36>> So this is a skeleton in his yard, like
50:39one of those Home Depot skeletons, but
50:41you can talk to it.
50:43So there's a sign out front that
50:45explains,
50:47uh, you know, "Tell me about your
50:48costume. Tell me what your favorite kind
50:50of candy is.
50:51>> Mhm.
50:51>> Tell me a spooky story." And it'll kind
50:53of react to the person in front of it.
50:55And I have never been more full of dad
50:58jealousy in my life.
50:59>> But what if what about the case where a
51:01child like talks to the skeleton too
51:02long and gets like chat GPT psychosis?
51:05[laughter]
51:07>> That's the real trick in trick or treat.
51:09>> Exactly. You got tricked, and now you
51:11have to go to the hospital.
51:13>> [laughter]
51:14>> Now,
51:15who made this show?
51:16>> Machine Gods is made in partnership with
51:18NPR, with generous support from the
51:20Tarbell Center for AI journalism, the
51:22William and Flora Hewlett Foundation,
51:24and Emerson Collective. This episode was
51:26produced by Sasha Aschile and Jacob
51:28Bloomberg.
51:29Our senior producer is Rachel Cone. Our
51:31executive producer is Danush Parvaneh.
51:33Original music by Armen Bazarian,
51:35branding and visual design by Alex
51:37Castro, production design by Sky Pray,
51:40special thanks to Oren Rosenbaum, Erica
51:42Berry, and Nadine Zailstra.
51:44>> Hey, before we go, every week we're
51:45going to answer your questions about AI
51:48and technology. So, please send us
51:50questions, voice memos, short videos to
51:53[email protected]
51:55or leave a comment wherever you're
51:57getting the podcast.
51:58>> And for the love of the gods, please
52:00don't forget to like and subscribe to
52:01[music] the show. Until next time, stay
52:03blessed.
52:08>> Is this a fake bird?
52:10Is that a fake bird?
52:12>> [laughter]
52:14>> Speaking of Kelly Clarkson.

About this transcript

This page contains the full transcript of Can AI replicate in your smart TV? | Bots Behaving Badly by Machine Gods, generated from the public captions YouTube serves with the video. The transcript has 10,239 words across 1,577 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.