YT YouTube2Text

YouTube2Text → Transcripts → Unpacking the EU AI Act Code of Practice with Marietje Schaake

Unpacking the EU AI Act Code of Practice with Marietje Schaake — Transcript

by Center for Strategic & International Studies · 9,148 words · 1,317 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:08[Music]
0:11Welcome back to the AI policy podcast.
0:13I'm Gregory Allen and today we've got a
0:15really important episode at the center
0:18of the AI regulation and governance
0:20conversation. Um, if you are following
0:23this podcast, you know that the EU AI
0:26Act is a landmark piece of legislation
0:28that has been taking an awful long time
0:30to implement. And now some of the big
0:32machinery behind that regulatory
0:34approach is now in effect. And one of
0:37the most important things in it is the
0:39code of practice for general purpose AI
0:42models. This is a very complex document,
0:45but a really important one, not just for
0:47Europe, but for the entire world in the
0:49future of AI regulation and governance.
0:52Um, it's also a pretty complex document,
0:53and that's why we're extremely lucky to
0:56have a guest here today, Maricha Shock,
0:58who is one of the most important drivers
1:01behind this code of practice. She was
1:03one of the working group chairs that
1:06drafted the safety and security section
1:08of the code of practice. She also has a
1:10distinguished background as a former
1:13member of the European Parliament. She
1:15also writes a column that you may have
1:17read uh at the Financial Times, which is
1:19excellent. And her current day job, I
1:23guess you would say, is that she's a
1:24fellow at Stanford's Cyber Policy Center
1:27and the Institute for Human- Centered AI
1:29and is also the author of the 2024 book,
1:32The Tech Coup. So, Marie Shashak, thank
1:34you so much for joining the AI policy
1:35podcast.
1:36>> Great to be here. Thanks for the
1:37introduction. So, we're going to get
1:39into the meat of the code of practice
1:42here, but before we do that, I want to
1:44understand a little bit more about your
1:46background. Um, how did you come to work
1:49on tech regulatory policy? How did you
1:52become interested in AI? What's sort of
1:54the starting point of your career which
1:56has uh done a lot of different things?
2:00>> I would say two starting points are
2:02important. One is when I studied at the
2:05University of Amsterdam. Um I I was
2:08constantly looking for how to apply any
2:11knowledge that I gained while studying
2:14and I didn't even find it so easy. I
2:16studied American studies by the way. So
2:18you know um like many I I will claim to
2:22be an expert on American politics even
2:23though it's mindboggling what's
2:25happening uh there now. But um um there
2:30was a new minor at the time uh that was
2:33a new media and new media meant you know
2:37questions around what is the worldwide
2:39web what is this internet that is
2:41emerging we're talking uh 199uh9
2:452000 uh roughly speaking and so I
2:49decided to enroll and uh this was
2:52because I I always had a curiosity about
2:54where change was coming from. And so,
2:58you know, there are many things that uh
3:00induce change, but of course, technology
3:02is one of them. And you could argue
3:04policy and politics is also one of them.
3:06And so, I studied it a little bit, was
3:08always curious about technology. And um
3:11as an outsider in politics uh in my late
3:1420s when I ran for European Parliament
3:17for the first time uh this coincided
3:20with the Barack Obama campaign in the
3:23United States, the emergence of social
3:25media platforms where you know maybe
3:28this was the case in the US too, you
3:30would have to tell me. But in the
3:31Netherlands it was also this early phase
3:34where social media platforms were really
3:36platforms for political discussion. So
3:39it was a smaller group of people, maybe
3:40journalists, policy makers, experts in
3:43some fields that would gather there. And
3:46so I saw it as a way to reach audiences
3:48where I just didn't have a profile for
3:50national media at all. And so that was a
3:53sort of organic entry way um into using
3:57the technology, but also, you know, when
3:59I got elected at the age of 30, being
4:01seen as someone who represented this
4:03younger, more tech-savvy generation, I
4:05had this curiosity, this background in
4:07studying it. So I continued to also just
4:10look at this intersection of technology
4:12and society, technology and politics.
4:14And then what really accelerated my role
4:18as as you know a policy um you know lead
4:23as a representative focusing on these
4:25areas were the Arab uprisings.
4:28>> And so one of the first Arab Spring of
4:31the 2012 time frame.
4:33>> Yeah. But it also started in Iran if you
4:35remember. So I was elected first in 2009
4:38and this was when the Gree movement
4:39happened in Iran as well.
4:41>> And um you know one of the first trips
4:43that I took abroad was in uh early 2010
4:48because I went to eastern Turkey to meet
4:51with refugees from Iran. You know these
4:53activists were fleeing because they were
4:54being you know harshly uh persecuted,
4:57imprisoned, tortured, you know the worst
4:59conditions. And um when I met these
5:03activists of which we had heard they
5:06were organizing on social media, you
5:08know, there was this famous delay in the
5:10update of Twitter if you remember uh
5:13under the Obama administration because
5:15there was concern that then um the
5:18Iranians couldn't post the eyewitness
5:20accounts of what was happening. And so
5:22we were very familiar with the story of
5:24sort of Facebook revolutions, Twitter
5:25revolutions and that whole narrative
5:27which was you know very flawed. But when
5:30I met these activists that had fled to
5:32Turkey, I also learned of their deep
5:34concerns about surveillance, about
5:36hacking and I decided to dive in deeper.
5:40And so one of the first
5:41>> I guess I should just say like this was
5:43a pretty idealistic moment uh in terms
5:46of what technology meant for the
5:48signature of politics. I remember I
5:49think it was 2012 when uh then Facebook
5:53CEO now Meta CEO Mark Zuckerberg wrote a
5:57post in which he basically said that the
5:59growth of Facebook is synonymous with
6:02the spread of democracy and that like
6:03everything that we're doing is
6:05bolstering free speech liberty um and
6:09and as you just saw like Arab regimes
6:11like Libya fall over um and Tunisia uh
6:16it looked like Facebook was this tool
6:18for democratic promotion that was just
6:20going to usher in this beautiful era of
6:23peace and political harmony. Um didn't
6:25quite work out that way, but it was this
6:27this really idealistic moment and you
6:28had a really interesting viewpoint on it
6:31from your seat in the European
6:32Parliament.
6:33>> Yes. and being able to, you know, travel
6:35to these places, Iran first, but later
6:38also Tunisia, Egypt, meet with all these
6:40activists, and hear their eyewitness
6:41accounts, their firsthand accounts of,
6:43you know, being arrested, being
6:45tortured, um, but being confronted with
6:48their own messaging, with their own
6:50emails. Uh and then you know I decided
6:53to dig in further and I learned that
6:55some of the technologies that these
6:57dictatorships were using against
6:59bloggers, protesters, students you know
7:03human rights defenders, opposition
7:04leaders were sophisticated surveillance
7:08tools. I think that was the sort of less
7:09idealistic story about what role
7:11technology played in these uprisings.
7:13But they were also made in Europe. And
7:16you know it was a real wakeup call for
7:17me because on the one hand we sort of
7:19had universal outcry of the repression
7:21of people. There was also particularly
7:23from the democratic world US uh EU
7:26countries the EU a real hope that indeed
7:30this this youthful uh uprising uh cries
7:34for more justice, more equal treatment,
7:37more uh democracy were going to succeed
7:40and that that technology could play a
7:42role in this. But then, you know, how
7:44could it be that our our ministers were
7:46condemning the repression of of um these
7:49activists, but then our quote unquote,
7:51you know, Europeanmade technology was
7:53the tool of choice to repress. And of
7:55course, it's different if you can
7:57already surveil and intimidate and limit
8:00people before they have to take to the
8:02streets or before violence has to be
8:04used. So, it's actually a very
8:05sophisticated tool um because it's less
8:08visible and it was also um you know
8:10potentially not as violent but certainly
8:12very effective and so it would maybe
8:15lead to less outcry etc which I think
8:17was the case because it took forever
8:20>> to get restrictions on spyware and if we
8:22look globally now the market in spyware
8:25is still way too powerful and and
8:27unregulated. Um so those were a couple
8:30of years where I spend a lot of time at
8:32the intersection of human rights and
8:34technology, democracy and technology but
8:36it was mostly a foreign policy issue I
8:38should add.
8:39>> And then of course you know as the years
8:41progressed technology started to be used
8:44more and more the tech companies
8:45including the popular ones that we know
8:47became much more powerful. So then all
8:49kinds of angles like antitrust, cyber
8:51security, copyright protection, export
8:54controls, you know, were added to that
8:56portfolio of policy issues that I worked
8:58on. Um, but I have always been and
9:02continue to be motivated by the question
9:04of how can we make sure that the core
9:06values that define our quality of life,
9:09the rule of law, democracy, human rights
9:12are the core lens through which, you
9:14know, we treat technology. that we're
9:16not too frivolous or too careless about
9:19these core principles as we, you know,
9:21celebrate the the marvel of innovation,
9:24the hopes that we have of what beautiful
9:27solutions technology might bring. I
9:28mean, we're now in the age of AI. This
9:30is an AI podcast. I mean, you probably
9:32have a comprehensive list of promises
9:34that have been made by CEOs about, you
9:36know, curing cancer, ending climate
9:38change, uh, disaster, breakthroughs in
9:41science.
9:41>> Don't forget about ending poverty. We're
9:42gonna end that one, too.
9:43>> Oh, that's true. Yeah, I forgot that
9:45one.
9:45you know, there's such a long list of
9:47what AI is promising to deliver, but I
9:50think um you know, it's it's become my
9:52role to also ask myself um well, first
9:56of all, is this a hype? Second of all,
9:59what could it mean in a different
10:00context? What could it mean for the most
10:02vulnerable? Uh how do we make sure that
10:05we don't just hope, but that we have
10:06safeguards? You know, where do checks
10:08and balances come from and so on and so
10:10forth.
10:11>> Excellent. And um you know I think when
10:14most people think about European
10:17uh regulation of technology they're not
10:21thinking of those initial spyw wear uh
10:24kind of debates that you talked about. I
10:27think the the landmark legislation that
10:28most people sort of anchor on is GDPR um
10:32which of course relates to privacy and
10:34therefore has a tie to spyware. But were
10:36were you part of that sort of uh
10:38transition?
10:40>> Well, we were all voting on it. um but I
10:43wasn't as directly involved but it was
10:45definitely a piece of legislation that
10:47also focused the minds in Europe and I
10:50think you know part of why data
10:53protection has always been such a key
10:54issue for Europeans is because of our
10:56recent past with fascism you know I sat
10:58in the European Parliament with
11:00colleagues who grew up under communism
11:02>> as well yeah
11:03>> yes but that's you know clearly uh an
11:05anchoring
11:07identity uh for Europeans and one that
11:10you know most of us would hope never to
11:11repeat.
11:12>> Yeah. I'm gonna I'm gonna just interject
11:14with a uh uh a fun fact that has always
11:18blown my mind and I always anchor on
11:21this in uh discussions about
11:23surveillance and technology. So, um if
11:25you've ever been to Berlin uh and you
11:28haven't been to the Stazzi Museum,
11:30you've got to go. So, this is the
11:32original headquarters of the European
11:34Easter East Germany's uh secret police.
11:38And one of the amazing uh data points
11:40that they have is that it turns out that
11:45one out of every 67
11:48East Germans was either working directly
11:52as a Stazzy employee or was an
11:55officially you know known informant. So
11:57think about how labor intensive the
12:00surveillance apparatus of that
12:01repressive society was. One out of every
12:0467 humans in the country of working age
12:07or no not even of working age of the
12:09whole country.
12:10>> So if it's of working age it would
12:12probably be like one out of every 40
12:13people or something like that um was
12:15working for the surveillance apparatus
12:17and that's because they didn't have you
12:19know modern technology where computers
12:21can handle so much of the surveillance
12:23burden um and AI. And so, yes, you're
12:26you're you're totally right to point
12:28out, right, that those memories loom
12:30incredibly large uh in the minds of
12:33Europeans as they're thinking about uh
12:35privacy and and other things.
12:37>> What's interesting is that, you know, in
12:39those years uh the focus of Europeans
12:41was much more on data protection while
12:42the focus in the US was already on data
12:45mining and you know training AI models.
12:47So I think you know one of the lessons
12:49learned there and there are many when it
12:50comes to the general data protection
12:52regulation including how important
12:54enforcement is and not just the the
12:56letter of the law because the
12:57enforcement has has just not really been
13:00up to par. Um but it's also that
13:03sometimes there can be a tunnel vision
13:05right if you if you're focusing on one
13:06thing maybe another thing is happening
13:08in the meantime. So it's it's been a
13:10really interesting uh lesson learned.
13:12But yes, I think the reason why the GDPR
13:14is such a important um an important uh
13:19starting point of sort of tech
13:21regulation for Americans is that it was
13:23the first time that US companies started
13:25to take EU regulation seriously. So
13:28until then there was kind of a sort of
13:30you know laughing and dismissal of what
13:33European regulation was. It was
13:35sometimes framed as a bit emotional and
13:37and um you know typical for Europeans
13:41but not something that the tech
13:42companies and others by the way uh would
13:44have to reckon with because as you
13:46probably know the general data
13:47protection regulation is a horizontal
13:49law. It applies to you know the local
13:51swimming pool as much as it does to
13:52Google. And so uh while tech companies
13:55have kind of framed it as a weapon to go
13:57after the US tech companies and maybe
13:59unfairly so this is really one
14:02harmonizing 27 different uh systems of
14:05each member state into one standard
14:06which is something that a lot of EU
14:08regulation is about to make sure that we
14:10harmonize over time. Uh but it's it also
14:13applies to everyone. But because it it
14:16hit home in Silicon Valley that you know
14:18this really mattered they had to change
14:19their policies there could be serious
14:21fines. Um it is it is you know maybe the
14:24starting point of history for many in
14:26Silicon Valley.
14:27>> Yeah. And so uh coming back to you you
14:29know how did you make the jump to
14:31interest in AI specifically?
14:35>> Well again I think this was organic uh
14:37first of all some of the things that we
14:38used to call social media uh we now call
14:40AI. So um it's just you know if you're
14:43interested in tech you're sort of
14:44interested in AI
14:46>> as the as a sort of tech policy
14:48interested person as AI rises in
14:50prominence it naturally falls on your
14:52desk. I mean there's a very similar
14:53phenomenon in the US government where
14:56the sort of first trunch of AI policy
14:58people in the national security
15:01community were almost invariably cyber
15:03people or space people because those are
15:05the you know people who like understand
15:06technology things
15:09and and so it just sort of naturally
15:10falls on your desk I'm sure. Well, but
15:13it was also accelerated by the fact that
15:15so when I left the European Parliament
15:17in 2019 after 10 years there uh I uh got
15:21a job offer at Stanford and it was a a
15:23double appointment one that I still have
15:25one with the more political science kind
15:27of research uh clusters at FSI uh
15:31wonderful place run by by Mike McFall
15:33which also has the cyber policy center
15:35which is indeed you know the starting
15:36point is policy but then I also had an
15:38appointment at this very new institute
15:40at the time called the institute for
15:41human- centered AI and you know there
15:44the focus was really on AI and so all my
15:46colleagues there are focusing on AI
15:48exclusively and so as a result I sort of
15:52you know deepened my focus on AI was
15:55appointed to the UN high level advisory
15:57body to the secretary general on AI so
16:00you know that allowed allowed us all to
16:02take a much more global perspective
16:03where you know before that my my remit
16:06was mostly EU transatlantic because I
16:09served on the um US deleg ation of the
16:12parliament and because so many companies
16:13are American. So it's just grown by the
16:16various things I've done and then um I
16:19became one of the chairs of this code of
16:21practice that I know you'd like to focus
16:23on as well.
16:24>> So it's it's been organic but I don't
16:26consider myself an AI expert if that
16:28makes sense. I'm much more of a broad
16:29tech policy person and um I just think
16:33we all have to wrap our heads around
16:34what AI means in our society and sort of
16:36weed through hype and hysteria and
16:40>> um to a large extent the same policy
16:43questions apply if you ask me you know
16:45how do we make sure there yeah go ahead
16:47>> um let's turn to the the code of
16:49practice so first I I kind of want to
16:52understand like the machinery for how
16:53this came into existence so the EU AI
16:57act you has a bunch of thou shalts.
17:00Companies shall do X and then there's
17:03the question of like how does the
17:05government and how do even the companies
17:08know and show that they are complying
17:11with the law and it seems like a lot of
17:13the answer uh is going to be found in
17:16this code of practice. So help us
17:18understand like what is this thing? How
17:21does it fit into the larger picture?
17:22>> Right. Let me start with the larger
17:24picture because I think it will help
17:25listeners understand. So um in your
17:28introduction you said the AI act is this
17:30law that's taken you know ages to
17:32implement but actually by by legislative
17:35standards the uh process has been quick
17:38>> draft and implement it's taken ages. It
17:40took about 2 years to draft I would say
17:42and you know the irony is of course that
17:44what when the idea is to fastly
17:46implement companies say hold on just a
17:49minute we need time
17:51and other stakeholders may also say we
17:53need to adjust because you know
17:55institutions have to be set up or
17:56reorganized in order to do the
17:58enforcement. So you know there's often a
18:00reason for time spent which is to
18:02prepare stakeholders in this case AI
18:04companies that I think you know should
18:06be capable with their with their uh
18:08multiple billions because that's the
18:10group that has to do most of the uh
18:11preparation but you know we can get to
18:13that. So when the work on the AI act
18:17started, a lot of people said, "Oh my
18:19gosh, the EU is too quick, right? Why
18:22would you regulate AI now? You're going
18:24to stifle innovation," which is sort of
18:26the most heard mantra about the EU and
18:28regulation anyway when it comes from
18:30from tech and AI people. But the
18:34decision was made that this was going to
18:35be a very deeply impactful technology
18:38and that there needed to be a sort of
18:40conceptual framework in the law of how
18:44to how to assess risk and mitigate risk.
18:47And so the vast majority of use cases is
18:50not considered to be risky. But then
18:52there's a risk spectrum, you know, that
18:54goes from low-level risk to high level
18:56and even unacceptable risk, which is how
18:59the law is is um built up. And so you
19:01could say it's very much comparable to
19:04liability uh rules and you know safety
19:08rules in anything from I don't know
19:10machinery to pharmaceuticals right like
19:13where do we see the most risk what are
19:15the obligations of the providers the
19:16makers of this product uh to mitigate
19:19the risk
19:21but then while the law was being drafted
19:25you know generative AI broke through and
19:28the question became as AI was
19:30progressing So quickly what do we do
19:33with more general purpose AI models? So
19:35where it's not clear that it's used for
19:37biometric identification or for scanning
19:40someone's CV for employment
19:42>> because those risk classifications that
19:44you mentioned earlier those are use case
19:47breakdowns right like this is high risk
19:50this use case is low risk and so what do
19:53you do when you have AI capabilities
19:55that don't just have one use case they
19:57have potentially you know in infinite
19:59use cases
20:01>> exactly thank you that's an important
20:02clarification
20:04Um for for those categories,
20:09a group of independent experts were
20:11invited to help sort of create uh
20:15clarity for companies, you know, when
20:18when they were subject to this part of
20:20the law and how they might comply.
20:24And you know, this is a process that's
20:25still unfolding where the European AI
20:27office or the European Commission still
20:29is doing work with regards to standards
20:31and clarifying what these companies
20:33should do. But as independent experts
20:35with various backgrounds, a lot of
20:37actual AI experts, but also some others
20:39like in the space of copyright or in my
20:41case in policy were asked to
20:46basically clarify for companies what
20:49compliance with the AI act could look
20:51like. And so it's sort of you could see
20:53it as a sort of road map or series of
20:56steps that when companies take these
20:59they show good faith of course but they
21:01also show you know their their
21:03willingness and ability to comply and
21:05it's sort of maps out how they might do
21:07that but the law applies to anyone. I
21:09think that's really important. So it
21:11even if you if you're a company that
21:13doesn't want to sign the AI uh code of
21:15practice, you still have to comply to
21:17the AI act, but then it's sort of up to
21:19you how you want to do it and up to the
21:21AI office to see whether what you do if
21:24it's not following the code of practice,
21:25whether that's sufficient. So these are
21:28not two different laws. The one is the
21:30law. That's the only thing that's
21:31legally binding, the AI act. And then
21:33the code of practice is sort of a a
21:36series of steps that companies can take
21:38and adhere to. So sign up to that's
21:40where you have this whole question of
21:41signitories. Are companies uh interested
21:44in doing that um signing on committing
21:47to doing this and actually uh quite a
21:50few companies have.
21:51>> Yeah. So so breaking that down um the
21:55law says something like I don't think it
21:57uses these exact words but something
21:58like companies must take adequate
22:01precautions to ensure safety and
22:03reliability. And the question then
22:06becomes, if somebody sues you and you're
22:08going to go in front of a judge, how do
22:10you prove that you took quote adequate
22:13precautions?
22:15And the the a EU AI office has basically
22:19said, hey, if you do everything in this
22:21code of practice, we will treat that as
22:24though you have taken adequate
22:26precautions. So some companies have
22:28signed up to that, like Google, OpenAI
22:30have signed up. They're like, "This is
22:32what we're going to do in order to take
22:34adequate precautions." Other companies
22:36like Meta have said, you know, we hate
22:39this document and we know better what
22:42adequate precautions are than uh the
22:45people who drafted this and if it goes
22:47to court or if the regulators come after
22:49us like bring it on, I think is is sort
22:52of the nature of the conversation,
22:53right?
22:54>> Yeah, presumably. I mean, you'd have to
22:56invite Meta to explain their their
22:58reasoning. I doubt they would actually
22:59go on the record saying bring it on, but
23:01you know uh ish.
23:02>> Well, I mean, I've heard I've heard
23:04pretty pretty scathing statements by
23:06Joel Kaplan, you know, as maybe he was
23:07hired uh to to um uh send into the
23:11ether, but you know, I think they've
23:12they've gotten more and more brazen as a
23:15company.
23:15>> So, now let now that we understand like
23:17the context for why this document
23:19exists, let's actually get into the
23:20machinery of it. And I want to focus on
23:22the part where you were a co-chair uh
23:25which is the safety and security
23:27chapter. But just for the context of
23:28everyone, there's a section on um
23:31transparency which I think is very
23:35blasze. I mean anybody who is familiar
23:37with a model card in AI, this is just
23:40sort of like the biographical data of a
23:43model. Um it's pretty tame I would say
23:47in terms of what it's asking for in
23:48terms of transparency. Then you've got
23:50the copyright section which is like how
23:52do you handle intellectual property and
23:54and copyright. Um, but I want to mostly
23:57put those to the side and now talk about
24:00uh what is not undoubtedly the longest
24:02part of the code of practice. This is
24:03like 40 plus pages whereas those other
24:05ones are like six pages each. Um, and it
24:08also includes the most commitments.
24:09There's 10 commitments that signitories
24:12um are signing up to when they do this.
24:13So um sort of I realize it's a it's a
24:16big document. It's very uh complex. I'm
24:18going to defer to you as to like what's
24:19the what's the best way to walk us
24:20through this story. I think we should
24:23just think about it in pretty basic
24:24terms. So the idea is there are general
24:27purpose AI models and there are general
24:30purpose AI models with systemic risk and
24:34uh for those general purpose AI models
24:36with systemic risk, the providers have
24:39to show the steps in sort of you know
24:43formalized documentation and information
24:45sharing manners of how they assess the
24:48risk, what they've done to mitigate it
24:52uh and for how long for example they
24:54have to keep the documents, who they
24:55have to report it to when they have to
24:56report it. So it basically
24:59spells out um a way again companies can
25:03choose to do other ways but a way to um
25:07be rigorous but also identify the people
25:09within the companies for example who
25:10should have executive responsibility so
25:12that it's not some you know uh niche
25:15group of juniors who are assessing the
25:18most serious risks uh but that it
25:20actually is also in a governance sense
25:22you know uh with those responsibles on
25:25the executive level.
25:27>> Yeah. If you're if you're a part of the
25:28safety team and you're not allowed to
25:31say no to anybody else in the
25:32organization, are you really part of the
25:34safety team? Um
25:35>> yeah. Well, that's a big question. And
25:37so there's also provisions for
25:38whistleblowing, for example, because we
25:40want to make sure that that you know
25:42considerations for society at large, for
25:44national security, for public health,
25:46etc.
25:47uh are are um informing decisions that
25:51companies make instead of you know the
25:53business cases, the market um
25:55competition and everything else that's
25:57driving these companies.
25:58>> So I I definitely want to get there, but
26:00could we rewind one second because you
26:02basically said there's there's two paths
26:04you can go down. There's general purpose
26:06AI models and there's general purpose AI
26:08models that that pose some kind of
26:09systemic risk. How does the law, how
26:13does the code of practice determine
26:15whether you're in one group or the
26:17other?
26:19>> Um, there's a lot of responsibility for
26:21the companies themselves, but there are
26:23also standards forthcoming that will
26:24clarify more and more, you know, uh, the
26:27details there, which the AI office will
26:29do.
26:30>> Okay. So, it's not it's not obvious and
26:33set in stone right now which general
26:36purpose AI models pose systemic risk and
26:38which ones do not. And it's also because
26:41you know you and I or even you know my
26:43colleagues at Stanford who are deeply
26:44deeply in the weeds of how AI models
26:46work don't have access to this
26:47information. I think that's really
26:49important to understand. So a lot of
26:51what these companies do and decide in
26:53the process of building their models um
26:56testing them uh putting them out into
26:58the market tweaking them and so on is of
27:00course entirely proprietary
27:03>> if memory serves. Um, you know, the the
27:06Biden administration, one of their
27:08executive orders, they looked at total
27:10amount of processing power used in
27:12training as a proxy for power and power
27:16as a proxy for risk. And the B
27:19administration sort of knew that was a
27:21very unsatisfying definition, but
27:23they're like, look, this is what we've
27:24got right now. And I think um I think an
27:27earlier draft of the EU AI act, maybe it
27:28didn't make it into the final version, I
27:30forget, was also looking at like this 10
27:32to the 25th flops used in It has that
27:35>> it has that but of course time
27:37progresses and models may become more
27:39capable and then there's also questions
27:41of downstream uptake so you know some
27:44some models may reach many more people
27:46than than other models right so there's
27:48the power of the model but then there's
27:50also the reach of the model so I think I
27:52think um those criteria would naturally
27:56be fluid right I mean this law is
27:58written now but who knows where where we
28:00are in terms of power or new risks
28:04>> you Surely there are some models right
28:06now where we know like that model is
28:08subject to the systemic risk evaluation.
28:10Is that not the case?
28:13>> Um I wouldn't be in a position to just
28:16say this model or that model because
28:17it's the companies that have to really
28:19combine a whole bunch of uh information
28:21for that.
28:23>> Interesting. Okay. So um let's say we're
28:26we're in the the world where the model
28:30has been designated as one that is
28:32general purpose with the potentially
28:34posing some kind of systemic risk. Uh
28:37now what happens like what are the what
28:38are the commitments and and what are the
28:40companies going to do?
28:42>> Well so the companies are going to walk
28:44through I mean I don't know the code by
28:45heart but you will see all kinds of
28:47steps in there. It's online for everyone
28:49to review and it's fairly technical so I
28:51don't think it's interesting for
28:52listeners but basically the idea is they
28:55commit to a number of steps of how to
28:58assess their models
29:00how to also document what they found in
29:03this assessment you know it's not just
29:05you know scouts honor we've assessed all
29:07clear I mean this has to be you know
29:10documented um for a certain period of
29:12time has to be reported so for example
29:15there can be uh a model that that is
29:18being developed. So it's not on the
29:20market yet but the developers have the
29:22idea that this may reach a certain
29:23threshold or a certain impact or they
29:25may see certain risk. Then uh there's a
29:28there's a period within which it has to
29:30be notified. You know the information
29:33has to be has to be uh preserved and
29:37>> and notification in this case is uh to
29:39the EU AI office.
29:41>> Yes. and uh the AI board I believe at
29:43the same time or there's just hopefully
29:45going to be one contact point. Um but
29:49there there's the AI office which is
29:50sort of the central office at the EU
29:52level which falls under the European
29:54Commission which is the executive branch
29:56of the European Union but then there are
29:59also representatives from the member
30:01states because as is often the case EU
30:04law still gets enforced and implemented
30:07on the local level meaning country by
30:09country and so every member state the
30:13Netherlands, Germany, Italy, Spain you
30:15know France
30:17can designate the authority that is best
30:20capable within its own structure. So
30:23this could be the market authority or
30:24this could be the data protection
30:26authority. You know how oversight bodies
30:29may have different tasks in different
30:30contexts but in any case they need to be
30:33equipped with the right resources and
30:35mandate on the local level to
30:36independently verify whether companies
30:39comply. And so when you're talking about
30:42this transition between the law being
30:44politically adopted and uh entering into
30:47force, these are the kinds of processes
30:49you should think about that need to be
30:51um executed. Meaning you know the the
30:55the officers may have to hire new
30:57people. They have to have AI experts uh
31:00that they may not have needed before uh
31:03etc etc. So this is all now happening
31:05and representatives from the member
31:07states are part of this AI board and
31:09then there's the AI office if that makes
31:11sense.
31:12>> Cool. So um there's a few products uh
31:15that companies are sort of required to
31:17produce uh as they sign on to this code
31:20and these are things like um a safety
31:23and security framework
31:25uh safety and security model reports
31:28serious incident reports um and some
31:30public access stuff. So, I think the
31:32safety and security framework is doing a
31:35lot of uh a lot of the work um in how
31:39the EU is trying to deal with uh serious
31:44risks. But before we go there, because I
31:46want to go there next, but before we go
31:47there, can you just talk a little bit
31:49about like what are the big risks that
31:52the EU is trying to solve with this code
31:55of practice for and I'm I'm thinking
31:57specifically here of these quote unquote
31:59systemic risks. So like another way of
32:01putting the question is like what is the
32:02problem that the EU is trying to solve
32:04in terms of systemic risks?
32:06>> Well, you should think about risks to
32:08national security. You know that we all
32:10know the famous examples of of boweapons
32:13and might it be easier for people to
32:15develop those through models. uh but it
32:18could also have have implications for
32:20public health for example you know if if
32:22certain I don't know like toxics or
32:25whatever could be easy to understand and
32:28and spread or whatever through AI
32:31um but there's also a responsibility to
32:34sort of keep thinking about systemic
32:36risks so
32:37>> I think cyber security was also on the
32:38list if memory
32:39>> of course yeah loss of control is
32:41definitely uh there so if if if AI uh
32:45models kind of go go, you know, off on
32:48their own and create all kinds of
32:49problems through cyber security or
32:51otherwise. Yes, absolutely. Um, but
32:53there could be other things, right? I
32:55mean, um, we could learn we could learn
32:58more about new models and what problems
33:02uh, they create. I think, you know,
33:05that's the tension there. You you write
33:07the law at a moment where this
33:08technology is evolving so quickly. Um
33:12and and by the way, you know, uh
33:15important jurisdictions such as the US
33:17not taking comparable measures.
33:20>> Um but you have to kind of both be
33:22specific because that's what people need
33:24in order to comply with the law. It's
33:26what companies ask.
33:28>> Mhm.
33:28>> But you also need to be flexible in
33:30order to anticipate what might be
33:32coming. And so I think we've constantly
33:34been in this tension of okay well what
33:37what do we know we need and what do we
33:39not know that might happen and how can
33:42we still write it down so that this law
33:43is sustainable and it's not outdated you
33:45know 3 years from now um but at the same
33:48time is specific enough and so there's
33:52also the the uh opportunity for the AI
33:56office to update with new risks that
33:58need to be considered but also the
34:01companies are just expected to act in
34:03good faith visa v systemic risk based on
34:05for example the body of scientific work
34:08you know latest information that is out
34:11there in the world so that when
34:12basically everybody knows that there's a
34:14new problem the company can't say well
34:16we had no idea because we just you know
34:18have to assume that they will study and
34:20be aware of the latest in AI including
34:23systemic risks.
34:24>> Yeah. So, let me um uh chime in a bit
34:28here because uh you've caught me just
34:30after I published uh a report on AI and
34:34boweapons risk uh with my former
34:37colleague Georgia Adamson. Um so, I'm
34:40I'm chomping at the bit uh to to talk
34:42about this systemic risk thing. So,
34:43pardon me while I I butt in here. Um I
34:47think it is a extraordinary gift of
34:51human history and fate that it turns out
34:54nuclear weapons are hard to make. Right?
34:56And the reason is that separating
34:59uranium 238 isotope from uranium 235
35:03isotope is just really really hard. It's
35:06expensive and complicated. Saddam
35:08Hussein spent billions of dollars on it
35:10and failed. Um, and so thank goodness,
35:14right, that nuclear weapons are
35:16expensive and complicated to build. If
35:18nuclear weapons were as hard to build as
35:20Legos, we would all be dead. We would
35:23all have already died if nuclear weapons
35:25were as hard to build as Legos. And so,
35:27the fact that these weapons of mass
35:28destruction are expensive and
35:30complicated is good for humanity. Now
35:33you look at other kinds of weapons of
35:35mass destruction uh like biological
35:37weapons and it the the question you have
35:40to ask is well is that expensive and
35:43complicated now and could that change
35:46over time and AI is one of these areas
35:49where if you look at the existing
35:51safeguards both those that exist in
35:54nature and those that exist in human
35:56society and technology um it's just
35:58getting a lot easier to create new
36:01organisms to design uh changes to
36:04existing organisms and then to go create
36:06those organisms. And so the plausibility
36:09of a pandemic uh that is human created
36:12and not human created by like North
36:14Korea spending many years to develop
36:17bioweapons, but by like one or more, you
36:20know, groups of people. Like I think
36:21we're all familiar with the James Bond
36:24villain trope of like an evil genius on
36:26an island somewhere with a massive
36:28resources and massive brains developing
36:31WMDs. But what if WMDs were within the
36:34reach of an evil average person? Um and
36:37that is uh that is the the AI and
36:40bioweapons risk. And what's really
36:42interesting is that like it's not just
36:44you know national security types like me
36:45who care about this. Even the folks who
36:48are typically associated with being most
36:50hostile to AI regulation like Meta CEO
36:53Mark Zuckerberg, he's like, "Yes,
36:55boweapons is in a different category."
36:57Like, we absolutely need to to do
36:59something about this because, you know,
37:01even if nine out of 10 companies are
37:05deeply responsible and put in place a
37:06lot of safeguards, it only takes one bad
37:08actor to like screw us all over and
37:10cause another global pandemic. Similarly
37:13in you know cyber weapons
37:16error the sort of question is like well
37:18what if AI models because they're very
37:20very good at writing computer code and
37:22assessing computer code can make it
37:24easier to generate offensive or
37:26malicious computer code. Do we want you
37:29know a group of three or four cyber
37:32criminals or maybe even one to have the
37:34level of capability that today we
37:36normally associate with nation states in
37:39cyberspace. So those are kinds of
37:42systemic risks where um AI as a helpful
37:46technological enabler can enable a lot
37:48of good can enable a lot of bad and so
37:50the question is what are companies doing
37:53to like minimize the chance that it's
37:56going to be enabling an awful lot of bad
37:58and I think that that uh tell me if you
38:01disagree but that those that category of
38:03risks is what the code of practice was
38:06intended to go after.
38:08>> Yes. I mean part of the code of practice
38:10indeed
38:10>> right the safety and security
38:12>> I think it's a I think it's a good um a
38:15good summary you know I think I see a
38:17really important analogy in what you're
38:19saying about you know how one company
38:21can spoil it for the rest not only for
38:23you know life life on earth and maybe
38:25all of us get you know impacted by a new
38:27pandemic or what have you but I also
38:30think AI companies should think about
38:32how bad actors maybe some of them can
38:36really harm the trust in AI AI as a
38:39technology for many and so you know I've
38:41heard a lot of criticism of course of
38:43the AI act you know especially from the
38:45US but no surprises with this
38:47administration but also from the
38:48companies um
38:50>> but I also see clear advantages which is
38:53you know this is a way that you can uh
38:57facilitate compliance and show that that
39:01this is a trustworthy technology that
39:03can actually be assessed for risk while
39:07um you know I I don't know many AI
39:09experts who think that this is a
39:11risk-free technology. You know, not that
39:13long ago, we had academic experts, CEOs
39:16penning all kinds of letters that this
39:18needed to be paused, that there was
39:20going to be the end of of humanity
39:22potentially and you know, all the
39:23catastrophes you're talking about. But
39:24there's also risks in the in the present
39:27day. So, forget about systemic risks
39:29that are horrible and non-controversial.
39:32There are also problems in the here and
39:34now that could seriously impact the
39:36trust that people have in AI at large.
39:38Cyber security on a lower level being
39:40one of them. Discrimination of course uh
39:43impact on uh democracy being another one
39:46that I think is really important. And so
39:49um I would imagine and I think that the
39:52test case will be fascinating to observe
39:54between the US and Europe and other
39:56jurisdictions. You know, will this will
39:59this legal landscape where there are
40:01mechanisms, one, will it be sufficient?
40:03Will it work out in practice? But two,
40:06will it actually help build trust in the
40:08technology that has a lot of questions
40:10about, you know, what is it ultimately
40:12going to do? Are all these promises of
40:13good stuff going to materialize or are
40:15we going to continue to suffer from
40:17harms in in, you know, in uh this day
40:21and age, but also in the future. Um and
40:24so you know the analogy that one company
40:27can ruin it for the rest I think applies
40:29broadly and I think here laws and
40:32standards can really help build that
40:34trust and predictability which is
40:36something that companies typically want.
40:38You know now that the federal government
40:41in the US is not going to adopt much
40:43regulation at all uh around tech. It
40:46will be really interesting to see not
40:47only what individual states will do but
40:49also what the responses by the companies
40:51will be because this kind of
40:52fragmentation
40:53different laws on relatively small scale
40:56of amount of people makes the work of
40:59companies harder because they have to
41:01comply you know differently for
41:03different areas different states
41:06potentially different countries and so
41:08it may well be that when I don't know
41:11California comes up with a strict law
41:13either that there will be um the uh the
41:16California effect, you know, the way
41:17that there is a Brussels effect that the
41:19highest standard will apply to all or
41:21that companies will begin to lobby and
41:23say, "Hey, just why why can't we have
41:25this standard instead of, you know,
41:26five, six, 15, 20 different states doing
41:29different things and us having to figure
41:32it out for every different state?" So I
41:34think the whole process of how to
41:37regulate emerging technologies including
41:39AI and what it means for the impact that
41:42it has on trust but also on safety and
41:44other aspects is really interesting to
41:46observe now that we have such a big
41:48difference between the US and the EU in
41:50real time.
41:50>> Yeah. So um the
41:56going back to the products that we're
41:57talking about I think the safety and
41:59security framework is one that's really
42:01important for folks to understand. So,
42:04can you talk a little bit about like
42:05what is a safety and security framework?
42:07What do the companies have to do and how
42:09is that going to interact with uh the EU
42:11AI office?
42:13>> Well, I think it's it's what I just
42:14mentioned. So it spells out in greater
42:16detail and I don't know it off the top
42:18of my head but all kinds of criteria
42:19that the companies have to you know
42:22assess for
42:24uh before placing it onto the market but
42:26also while it's placed onto the market
42:28they have to keep monitoring lessons
42:30learned especially as bigger models get
42:33tweaked and used downstream if that
42:36makes sense. So you know one company may
42:38put a model out into the world but then
42:41others may use it in different context
42:43and what happens there. Yeah, just just
42:45to give like an example like anthropic
42:47um claude is used a lot uh as coding
42:51assistance but it's not just like you
42:54might be a customer of anthropic and not
42:56know it because you might not be buying
42:58uh claude directly from Anthropic you
43:01might be buying it through a company
43:02like wind surf or somebody else who adds
43:04on the sort of additional capability to
43:07make it uh more useful as a coding
43:10assistant and you can imagine that you
43:12know that kind of downstream
43:13implications
43:14there's a question of who's responsible
43:17to make sure that
43:22regulatory requirements that sort of
43:23like yeah I guess that sort of
43:26overarching question about like who's
43:27responsible for what uh in these
43:30obligations the original developer of
43:32the model somebody who's the actual
43:33person who's selling the model to the
43:35end user versus like the end user
43:37themselves
43:38>> yeah and I think just you know from a
43:40practical point of view this law is is
43:42more about the upstream provider. So the
43:45the handful of companies that have these
43:47rare capabilities of developing these
43:49very advanced models or the most
43:51advanced models today could be different
43:53a year from now. Um but um that is also
43:58just practically and this is not spelled
44:00out in the law but I you know as a
44:01policy person just think it's helpful to
44:03think this through. If you think about
44:05it from a practical perspective where
44:06would you want to intervene? Would you
44:08want to monitor the handful of companies
44:09that have these unique capabilities or
44:11would you want to have compliance from
44:14thousands if not tens of thousands if
44:15not hundreds of thousands of smaller
44:17players you know downstream? It makes no
44:19sense from a practical enforcement point
44:21of view either but also because these
44:23model providers have the biggest
44:25insights biggest handle on how these
44:28models get developed designed and they
44:30can see things that we can't see.
44:32>> Yeah. So I mean I think I think it looks
44:34different on a
44:37problem bypro kind of a basis right if
44:40you want to think about uh the the issue
44:44of deep fake imagery um in in some
44:48countries have put in place a
44:50requirement that deep fake generated
44:52imagery have associated metadata tags or
44:56like watermarks inside the image so that
44:59you know anybody who has like the
45:01appropriate tools can look at this and
45:02say this image is real. This image is a
45:04deep fake. You know, it's it's built
45:06into the image. Well, if you want that
45:09regulation to exist, you really need to
45:11go upstream to the people who are
45:13creating the image generation software,
45:15right? Like a watermarking requirement
45:17where the implementation is at the very
45:19last step of like the consumer who is
45:22using Photoshop has to go in and like
45:25manually add a watermark. That is a dumb
45:27way to address that regulatory hurdle.
45:30However, like in other areas like uh you
45:34can't stab people with a knife. Like the
45:37regulatory burden doesn't fall on the
45:39knife manufacturer, it falls on the
45:40consumer who is using the knife. Um and
45:43so like whether you want to target
45:45upstream or downstream, I think kind of
45:46depends on the nature of the problem
45:48you're trying to solve. And if if I
45:50understand you correctly, you're saying
45:51that most of the problems that this
45:53regulation is looking at tackling are
45:55the ones that the EU has identified as
45:58better tackled in the upstream, right?
46:00>> It's because they would be systemic. So
46:02the the moment that it spreads that
46:04becomes the problem, if it makes sense,
46:07>> you know, um so the risk would grow if
46:11more people had access to the models.
46:15>> I see. Okay. And so I think you have to
46:18think about
46:20you know closer to the source is the
46:22best place to prevent uh the risk but
46:25also who is ultimately the sort of owner
46:28of the model and the knowledge that goes
46:31into it and who has the levers
46:34uh before it gets put into society or
46:37the market. Some of them get released,
46:39some of them get sold um is of course
46:42the model providers. They have the
46:43resources, they have the insight, they
46:46have the knowledge more than anyone
46:48else. And you know, I think one of the
46:50problems and this is not in the in the
46:52law, but I think you know this will be a
46:54problem that we all have to reckon with
46:56is how unpredictable AI ends up being.
47:01And so the question is how much risk can
47:03you mitigate for at all? And that is a
47:06risk in and of itself because you know
47:08anthropic CEO Daniel Amoda has said it's
47:12a public secret that nobody really knows
47:13how generative AI works the way it does.
47:16um you know lots of people have
47:18basically said how unpredictable these
47:21uh models and their their uh workings
47:24are and I think you know it's hard to
47:27work with existing law and enforcement
47:30models to anticipate so much uncertainty
47:34but that is something that we will have
47:35to also learn as a society reform our
47:38legislative processes our enforcement
47:40processes because
47:42I just don't think that the sort of
47:45traditional legisl ative and enforcement
47:47bodies and processes are equipped to
47:50deal with so much change,
47:52>> unpredictability. And so I'm hoping that
47:57everyone who um is working with AI or is
48:02working in policy is also interested in
48:07reassessing how we deal with these risks
48:10at all because I honestly don't know.
48:12We'll have to see uh whether the laws
48:14that we have now um will be sufficient.
48:18>> Yeah. So uh as you think about that, you
48:21know, will this be sufficient? Um what
48:24are you thinking about for measuring the
48:27success or the failure of the code of
48:29practice? Like if we got in a time
48:30machine 5 years from now uh and you
48:34could track, I don't know, you know,
48:35three metrics that would tell you
48:37whether or not the code of practice is a
48:39success. Like how are you thinking about
48:40that? what what will you look to to to
48:43feel like whether or not this is working
48:44as intended or
48:46>> Well, I hope it will make the whole
48:47enforcement process more agile for
48:49everyone. So, both for the companies and
48:51for the AI office and for the uh AI
48:55board because you know this takes time
48:57and effort and we should make it as easy
48:58as possible for everyone. Uh and I'm
49:01sure there will be lots of lessons
49:02learned there in how to simplify the
49:04processes. But that is ultimately the
49:05intention. How can we make this
49:07predictable and manageable for
49:09everybody? And I hope that companies
49:11will say look um uh we weren't sure
49:14about how this AI act would work. Uh the
49:17administration at the time was very much
49:19against regulation um of American
49:22companies. But because we want our
49:24products to be trusted and because we
49:27also feel ownership of avoiding uh risk
49:30and particularly systemic risk to
49:32society uh we have now learned 5 years
49:35after uh beginning to implement uh the
49:37AI act that it has actually helped we
49:40have caught things that we might not
49:42have otherwise caught. We have been able
49:44to uh build trusted relationships. you
49:46know, we've been able to share knowledge
49:48also publicly because it's not a
49:49requirement but an encouragement to also
49:52share some of these uh lessons learned
49:54with the rest of the world, right? Not
49:55just in the very confident um
50:00confidential I should say sorry
50:01confidential context of uh the enforcer
50:04and and the model provider but also more
50:06broadly etc. So I hope it will turn out
50:09to be a positive thing for everybody.
50:12>> Yeah. Well, I think that's a a lovely
50:14note for us to conclude on even though
50:16we could easily go talking for another
50:18hour, but I know you have other
50:19important things to get to today. So,
50:21uh, let me just say, Maria Shock, thank
50:23you so much for coming on the AI policy
50:24podcast.
50:25>> Thanks for having me.
50:29>> Thanks for listening to this episode of
50:30the AI Policy Podcast. If you like what
50:33you heard, there's an easy way for you
50:34to help us. Please give us a five-star
50:36review on your favorite podcast platform
50:39and subscribe and tell your friends. It
50:41really helps when you spread the word.
50:43This podcast was produced by Sarah
50:45Baker, Satie McCulla, and Matt Mand. See
50:48you next time.
50:49[Music]

About this transcript

This page contains the full transcript of Unpacking the EU AI Act Code of Practice with Marietje Schaake by Center for Strategic & International Studies, generated from the public captions YouTube serves with the video. The transcript has 9,148 words across 1,317 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.