YT YouTube2Text

YouTube2Text → Transcripts → Decoding AI Attacks with MITRE ATLAS

Decoding AI Attacks with MITRE ATLAS — Transcript

by Pamela Gupta · 6,304 words · 938 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01Hello and welcome to trustworthy AI
0:05drisk adoption of AI. I'm your host
0:08Pamela Gupta. I'm a 2025 Wasaman Award
0:12honory for AI governance
0:15ranked globally in the top three for
0:18risk management and top seven for cyber
0:20security.
0:22And the the reason we are hosting these
0:25podcasts are to provide insight into a
0:30very dynamic um and dynamic from a
0:34growth as well as a risk perspective
0:37um technology called AI.
0:40Um I personally am helping organizations
0:43embrace AI confidently by building
0:46governance frameworks and frameworks
0:49that are very holistic right instead of
0:52taking a siloed approach in terms of
0:55what are what can be security what can
0:57be privacy um what are the unique AI
1:00risks uh that we are seeing out there
1:02that we have to um not only identify but
1:06be able to solve for as we uh embrace
1:10this very dynamic um technology that's
1:13growing. Um so we're going to see some
1:15risk that we have seen in the past and
1:17some that are unprecedented.
1:20Uh also I created a AI tips framework.
1:22Um and I that was what I got the award
1:25for from Wasman award for um it's a AI
1:30risk management framework and it
1:32precedes Nest's AI framework by a couple
1:35of years. very um very interested in the
1:40operationalizing of AI governance aspect
1:43and that's what the framework brings and
1:45I can we can um put some notes into what
1:48that framework is offline.
1:52So um in a nutshell my expertise is to
1:56take a look at risk holistically and to
1:59solve u and create AI governance
2:01frameworks. expertise does span across
2:05standards such as ISO 4201
2:08NIST AI RMF EU AI act you know what are
2:10the regulations that we have in place
2:13and what are the business unique risks
2:16mitigation techniques that we need but
2:20AI governance um importantly is not
2:23about saying no it's about enabling
2:26safety and u AI systems with safe and
2:30that are safe secure and trustworthy.
2:34So with that, I am really excited to
2:37talk to my guest today on uh who is the
2:42lead for MITER Atlas and I'm going to
2:45let him uh uh you know walk through uh
2:48his introduction, but let me say a few
2:50words.
2:52Very excited to welcome Walker Damon uh
2:55to the show. Welcome Walker.
2:59>> Thanks for having me.
3:01Yeah, absolutely. So, let me just talk a
3:03little bit about what I know about you
3:05and I'd love to uh for you to uh please
3:07add on to that. So, Walker is the MITER
3:11Atlas lead and uh for AI for cyber group
3:14uh leader within MITER's AI and um
3:17autonomy innovation center.
3:20He is leading technical and research
3:22teams at the intersection of AI and
3:24cyber security. These are both very very
3:29complex areas. So hats off to you uh
3:32Blocker.
3:34Um um I'll just say a couple of words
3:37about your work at MITER but please do
3:39add um you are I from my um perspective
3:43and from what I know you're focused on
3:46advancing security towards rapidly um
3:50evolving AI systems which we know is you
3:53know rapidly um very dynamic and um very
3:58rapidly evolving. So there's nothing
4:01that uh we can really put a hand on the
4:03pulse and say yeah this is what works
4:05because it can change very rapidly. In
4:08other words, as MITER's AI for cyber
4:11group leader, he and his team are
4:13exploring unique challenges as can be
4:15expected when it's something so unique
4:18and dynamic
4:20and also looking at opportunities that
4:23inter arise when integrating AI
4:26technologies into the cyber security
4:28landscape
4:29primarily for government missions and
4:32encompassing AI applications in both
4:34offensive and defensive cyber contexts.
4:39Walker also serves as the lead PI for an
4:43internally funded research and
4:44development effort investigating the
4:46application of agentic retrieval
4:48augmentation generation large language
4:52model. So an RAG LLM architecture for a
4:56variety of um US government applications
4:59within the cyber domain. Um Walker did I
5:03miss anything?
5:04>> No, that covers everything. Thank you
5:06BMA.
5:09>> Sure. So can we uh start with you know
5:13what is MITER and what are some of the
5:16frameworks that you know are have become
5:19industry standards pretty much
5:21>> right? Yeah. Um great place to start. So
5:23MITER is an FFRDC a federally funded
5:25research and development center. Um we
5:28focus primarily on systems engineering
5:30integration across uh US government
5:33mission spaces. So anywhere from
5:35transportation to healthcare to the
5:37department of war to the intelligence
5:39community. Uh we operate a number of
5:41FFRDC's in support of their mission
5:43spaces and apply uh different variations
5:46of engineering um expertise to those.
5:49Within MITER we have an organization
5:52called Mitro Labs. So this is more of
5:53our research focus component of the of
5:56the organization itself. um my division
5:59within MITER labs AI and autonomy
6:01innovation center um that focuses on
6:03like the name suggests the innovation of
6:04AI and autonomy technology applied to
6:08different government use cases. So kind
6:10of going from big picture down to
6:12specifically where where I sit um MITER
6:15Atlas is one of the many frameworks that
6:17we we provide for uh government
6:20sponsors, industry collaborators and
6:23practitioners of the technology. Um,
6:25it's based off of the MITER attack
6:27framework. For any any one that may have
6:29heard of that before, MITER attack is
6:31kind of the precursor to MITER Atlas.
6:33Um, MITER Atlas is the AI version of
6:36MITER attack actually. So, so, uh, Piner
6:39attack is is like a cyber security
6:41protection, um, uh, knowledge base that
6:44was intended to create a common, uh,
6:47language and taxonomy for cyber security
6:50practitioners for them to be able to
6:52explain the the different uh,
6:54vulnerabilities and threats that they
6:55were seeing for their cyber systems. Um,
6:59kind of funneled it and bucketed into
7:01these different tactics and techniques
7:03and procedures is what they call them.
7:04So, minor atlas is the exact same thing.
7:07uh just for AI enabled systems just
7:09because AI technology in of itself they
7:11have unique attack um surfaces and um
7:16vulnerability exploitation points um
7:18that's different than than just software
7:20or or cyber systems um and especially
7:23now over the last handful of years you
7:26know uh ever since the AI field ran into
7:29what what we call the chat GPT moment
7:31where you had this technology suddenly
7:34at the hands of um the public at large.
7:37Uh the need for a security framework has
7:39become all that more important. And so
7:41um that is the goal of MITER Atlas. And
7:45um in addition to that um within our AI
7:49and autonomy innovation center, we we
7:51focus on um building frameworks,
7:54capabilities, tools to to support a
7:57number of different uh government agency
7:59um needs. And that continues to to
8:01evolve as as the AI landscape itself
8:05continues to evolve as well.
8:08>> Yeah, thank you. That's um it's really
8:11important also to for me I'm going to
8:14highlight the fact that you are not only
8:15doing this for the government but you
8:18are also focused on industry uh
8:21partnerships right where you are
8:23bringing this to and and if you can
8:25spend a couple of minutes on that that
8:27would be great.
8:27>> Yeah. Yeah, of course. So um you have
8:31federal federal and commercial use cases
8:34of of the technology of of artificial
8:36intelligence. Um there's a lot of
8:39commonalities between those two pillars.
8:42Um what Minor Atlas tries to provide is
8:46a um mechanism for both sides to come
8:50together and sort of um help educate one
8:54another of the security considerations
8:57that they're seeing um out in their
8:59respective fields. And so commercial
9:02industry typically bit more
9:03forwardleaning especially with Silicon
9:06Valley um that's where you see a lot of
9:07the AI innovation. And so once you see
9:11this technology sort of go from their
9:14own
9:15research and development test beds to
9:17commercializations, you have rapid
9:20adoption of that technology um across
9:23the across um the larger industry. And
9:26so they can sort of help government
9:30agencies understand what new risks are
9:32assoc associated with new innovations.
9:34So when generative AI hit the scene, for
9:36example, um and LLMs were slowly
9:39beginning to integrate themselves into
9:41into everything, uh chat bots is is kind
9:44of the the most well-known use case of
9:46that technology. They were starting to
9:48be able to see how that system could be
9:50manipulated by end users. That was
9:52really informative to um to to
9:54government agencies who saw this
9:56innovation. They saw the potential um
10:00that this technology could provide could
10:01provide them. um helps them ensure the
10:05the security of that technology um more
10:08rapidly as well. And so we we try and
10:10serve as that that connection so to
10:12speak between between the two. Um it
10:14goes the other way as well. So there's
10:17unique federal government applications
10:21of AI that you just do not see in
10:22commercial commercial industry. Um there
10:25are certain security considerations in
10:27the healthcare industry for example
10:28where publicly identifiable information
10:30or PII is is really really important. Um
10:33and when it comes to generative AI
10:35technology like large language models
10:37you just want to make sure that that
10:38those systems are not spitting out
10:40private information to people that that
10:42don't have the right to see it. Um so
10:45you have the healthcare industry both on
10:47the federal and and non-federal side. um
10:50you have government agencies that that
10:52really, you know, kind of um represent
10:56the public with that security concern
10:58and so they can they can help sort of
11:00drive industry uh towards standards. You
11:02listed off a number of them. The NIST AI
11:04risk management framework is is a huge
11:06one. Um but that's sort of the that's
11:08sort of the the back and forth and the
11:09in the collaboration that that MITER
11:11Atlas provides for for both industry and
11:13government.
11:16>> Great. I know it's a it's a huge and a
11:18very complex charter that you have. So
11:20really happy that you're leading it. Um
11:24let's talk a little bit about you know
11:26the you know we talking about predictive
11:29AI which is very commonly adopted across
11:32industry and the business
11:34u and then there's generative AI. So as
11:37the attack classes right what what was
11:41um existing such as data poisoning or
11:45model extraction which was for
11:46predictive AI now has gone into a
11:50different realm which is you know let's
11:52say um specifically around prompt
11:55injections for example right
11:58um what which attack losses uh you know
12:01out of these firstly if you want to
12:03elaborate on some that you feel are
12:06more um harmful or if you know you want
12:09to categorize them that'll be very
12:11helpful
12:12>> and then also what is that uh
12:15progression from predictive to
12:16generative AI that we need to be aware
12:18of.
12:19>> Yeah. Yeah. Um great question. So yeah,
12:22like you said, it it has been
12:23interesting over the last uh handful of
12:25years to see the migration from a sort
12:29of common set of adversarial tactics and
12:31techniques with predictive systems
12:33moving over to the to the generative um
12:35AI landscape. And uh myself and a number
12:39of my colleagues in Mer Atlas were
12:40actually responsible for for building
12:43out that common language for uh for
12:45these new TTPs. Um particularly within
12:48within generative AI because with with
12:50Miter Atlas, we're able to borrow a lot
12:52of the language that MITER attack
12:53speaks. You ever go to our Atlas matrix,
12:56you might see red amperands everywhere.
12:58That's where we share either a tactic
13:00name or a technique name with with the
13:01minor attack framework. um within the
13:05generative value space that mole
13:06instantly started breaking. Um
13:09>> and so things like prompt injection um
13:12we spent a lot of time trying to
13:15decipher the different types of prompt
13:17injections that that exist out in the
13:19field. Um we landed on direct versus
13:22indirect prompt injection for example
13:24within within the minor atlas matrix. Um
13:27what's interesting about that specific
13:29example is is now with Agentic AI or or
13:32rag based systems you're starting to see
13:33a different type of prompt prompt
13:35injection where um the adversarial
13:38behavior that you're trying to induce
13:40into the model um may not be immediate.
13:42So with you know might be delayed
13:43further downstream but it's interacting
13:45with different types of data or it's
13:48using specific tools and part of an
13:50agentic architecture. Uh so we just
13:52recently came up with a with a third
13:55type of prompt injection that we call a
13:57triggered injection. Um to try and
14:00capture sort of the nuances between
14:02direct and indirect that were more you
14:04know I guess traditional at this point
14:06just with large language models versus a
14:09large language model dropped into larger
14:10system architecture whether that be a
14:12rag system a retrieval augmented
14:14generation system or a genetic system.
14:15I'm sure that that's going to continue
14:17to happen. Um,
14:19in terms of severity of of specific
14:23techniques, um, we take a little bit of
14:25a different uh, philosophy within atlas
14:28to where we we bucket them into
14:30feasible, demonstrated, and realized. So
14:33feasible techniques are techniques that
14:36um may have been introduced within a
14:38research paper, research paper or a very
14:42uh closed bench test. Uh demonstrated
14:46for example with prompt injection could
14:48be trying to understand or visualize
14:52that specific technique with a real
14:53system or interacting with real data to
14:56try and do something bad to that system.
14:59um trying to induce some type of
15:00adversarial behavior and then realize as
15:02if we ever see evidence of this actually
15:04out in in the field. So MITER Atlas at
15:06its core is always grounded in real
15:08world evidence. Um, we're not
15:11necessarily interested in in theoretical
15:14capabilities that you may see on like
15:16test and evaluation data sets or
15:18benchmarks until they are um shown
15:23within a real system uh within a real um
15:29operational use case etc. And so in
15:31terms of severity, I think, you know, to
15:33answer your question, uh the most severe
15:35techniques would be ones that fall into
15:37that that that most mature bucket with
15:40with realize because that's those are
15:42examples of of actual adversaries um
15:46introducing these TTPs against victim AI
15:50systems and uh accomplishing some type
15:53of effect towards that.
15:56>> Okay. Right. Wow. That's that's a lot of
16:00information. So I I think I should have
16:02just maybe asked these two instead of a
16:05two-part question, two separate
16:06questions because each one carries so
16:08much um in terms of um you know area
16:13that it covers. So let me just go back
16:15to the first part which is on the you
16:17know the types of attacks the direct
16:19injection the prompt injection indirect
16:22and the third category that you created
16:24which I think is a great idea the
16:27trigger uh injection. Uh actually I just
16:30did a deep dive expose on the uh force
16:33leak vulnerability and some of the other
16:35agentic AI related um uh vulnerabilities
16:39that have been exposed and have been
16:42realized
16:43>> to to your point um in the real world
16:47because
16:48uh and and and affect globally a large
16:53number of organizations because these
16:55were able to be executed against um
16:58these vulnerabilities in you know
17:00Salesforce and can be other um platforms
17:04too that we are not aware of. So not
17:06really just um uh focused on a
17:10particular platform but giving rise to
17:14the ability to have these severe
17:17critical uh vulnerabilities in real life
17:20systems without having the controls or
17:23these guardrails to really protect us
17:25against it. uh and that's because that's
17:28the nature of that dynamic nature I was
17:30talking about in the beginning where
17:32things are evolving at such a fast pace
17:34that we'll have to really leverage
17:36existing knowledge as well as being able
17:38to anticipate uh what you know these AI
17:41systems can bring to us. I do like the
17:44fact that you have created those three
17:46prompt injection um categories and the
17:50other point that you mentioned the fact
17:53that when you categorize
17:57it is a different categorization now
18:00makes a lot of sense because
18:03what we are seeing is because it's
18:05evolving so fast in the AI landscape a
18:09lot of the research that we are you know
18:11a lot of the information we're is coming
18:14out of that research space, right? So
18:16unless it is realized, it doesn't go
18:18into MITER. So that's that's really good
18:20to know.
18:22>> Yeah. And and what I guess something
18:25I'll add to that is uh over the last
18:28year, it's been fun because the rate of
18:33um sort of generative AI maturity is
18:36just continuing to accelerate. And so
18:42we've we've sort of had to take a step
18:43back and rethink how we
18:47um sort of maintain our posturing within
18:52the the knowledge base. We we
18:53traditionally would try and target uh
18:56two releases a year, for example, one in
18:58the spring and and one in the fall.
19:00Adding new techniques um or tactics to
19:03the to the to the threat matrix, adding
19:06new representative case studies. That's
19:07something that we also uh spend a lot of
19:10time um developing within within the
19:13Atlas platform are um examples of
19:16chaining these different techniques
19:17together um to elicit some some type of
19:20of real world effect or or outcomes. We
19:22have a lot of those. Uh we have
19:24demonstrated mitigations to a lot of the
19:26the techniques that you find in there.
19:27So there's there's a lot of information
19:29within within the knowledge base of of
19:31Atlas and of itself that's that's really
19:33useful. Um but with Aentic AI for
19:38example really making waves this year um
19:42there was a real appetite from our from
19:44our community um partners to to
19:48release you know sort of
19:50state-of-the-art techniques quicker and
19:53than what we traditionally had like
19:54within with with model context protocol
19:56for example
19:57>> once that came out and got super popular
19:59there were a lot of um exploit exploits
20:01that started hitting the scene people
20:03were asking where are those in Atlas.
20:05Um, and so, you know, we we we've had to
20:07adapt a little bit and and we move a lot
20:09quicker. And so now we're we're starting
20:10to plan more consistent releases on a
20:12monthly cadence or or bimonthly cadence.
20:15Um, but I I see that only continuing to
20:19to increase. um just the the rate at
20:22which you start to see these types of
20:24vulnerabilities introduced to AI enabled
20:25systems especially as Agentic AI matures
20:29and they're interacting with a lot of
20:31data streams and tools and they're and
20:33they're given more and more autonomy
20:35inside of a systems architecture um
20:40models like Atlas are are going to
20:41become you know uh ever ever more
20:44important I think uh just to just to
20:46help a widening community of AI
20:50integrators understand um from a
20:53security point of view what they're what
20:54they're susceptible to.
20:57Yes, it is actually a really uh
21:01extremely valuable offering uh to be
21:04honest without uh you know not just
21:06because you're on the show I think um
21:09because of the practical component and
21:12the actionable component of the of the
21:14framework right it's not as you
21:16mentioned theoretical it is what has
21:19been realized or can be realized
21:20>> right
21:22>> um I would expand on that and then say
21:26What how can CESOs then or how can a
21:29chief information security officer in an
21:31organization then u use Atlas
21:35for um for operationalizing their risk
21:39posture you know the threat modeling
21:41planning mitigation etc. What what would
21:44be your guidance on that?
21:46>> Yeah uh great question. So this is this
21:49is touching on the topic of of how do
21:50you put minor atlas into practice? Um
21:53which when we
21:56hold these community engagements, we
21:57call them atlas saloons. Um we try and
21:59hold those twice a year as well. Um and
22:02that that's both across commercial
22:04industry and and government. This comes
22:07up a lot is is putting the Atlas matrix
22:09into into practice. So what we normally
22:11recommend are for sysos or or whoever is
22:14in control of sort of the the security
22:17perspective of an organization um to
22:21define all of your your data flows that
22:24exist within
22:26the architecture housing the the AI
22:29model. So how does information get fed
22:32in and out? uh what are all the
22:34potential user access points um for a
22:37model either directly or indirectly with
22:40with other tools or other applications
22:43etc. Um and then from there you you
22:45really sort of scope down and can very
22:48quickly map that information to the
22:52appropriate Atlas uh tactics and
22:56techniques just using sort of the way
22:57that the that the Atlas threat landscape
22:59is is structured. So within each of the
23:02the tactics and techniques we have
23:03definitions uh we have examples we have
23:07sort of of relevant um exploits
23:10associated with each and then once you
23:13sort of have that framing and you have
23:14that mapping that's when you can dive
23:17into our our case studies which is a
23:19growing repository of information um
23:22that's intended to provide like I
23:25mentioned before just examples of these
23:27techniques being chained together to to
23:29um reach some type of of adversarial
23:32effect. And so with our case study
23:35releases, we try and make sure that we
23:36have, you know, a pretty wide reaching
23:39um set of examples with each uh set of
23:43techniques that we release just to make
23:44sure that everything is is mapped to
23:46grounded sort of realized incidents. Um
23:49and then from there, we also that's when
23:52you dive into our mitigations. And so uh
23:54our mitigations library is intended to
23:57uh offer either exemplars or strategies
24:01um like the name suggests just to
24:02mitigate the impact of of those uh
24:05adversarial techniques. And so in the
24:07generative AI landscape for example um
24:09with prompt injection or jailbreaking or
24:11privilege escalation etc um you can do a
24:14lot of good just by implementing some
24:17open repository uh guardrail uh
24:20packages. So, Nemo Guard Rails is an
24:22example. Um, very common example of this
24:26of this type of technology. And what
24:27that does is just make sure that you
24:29define a set of accepted outputs and
24:31inputs and anything that's outside of
24:32that boundary um, it's guardrailing the
24:35system from doing anything or revealing
24:37information um, to the end user if it if
24:40it goes outside of those um, those
24:42boundary conditions.
24:44And so, it's it's one of those things
24:46where it's it's can be fairly
24:47interactive. Um, we try and make sure
24:49that we have the blueprint in the road
24:51map for for sysos or or whoever it might
24:53be. Um, and then contextualize sort of
24:56the the attack surface with enough
24:59enough represented examples for you to
25:00be able to tie those back to your given
25:03use case. Um, and then and then
25:06understand uh how to mitigate the the
25:09threats from there.
25:11>> Okay, that that makes sense. Um do you
25:15know of any platform that has um
25:19integrated with miter atlas to make this
25:22more you know operationalizing it and or
25:26making it easier to accomplish?
25:28>> Yeah. So, um, there are some that I've
25:30I've heard of. Um, I can't think of
25:32specific names off the top of my head,
25:34but, um, this is something that we do
25:36see in the community where they they'll
25:37they'll clone a version of Atlas and
25:40then they'll map it, um, to some type of
25:43test harness, uh, where you have
25:45examples of of a subset of techniques
25:48sort of pinned against a model with with
25:50data and you can kind of understand sort
25:51of what the what the impact is for you.
25:54Um we moving forward into 2026, this is
25:57an area where we're actually wanting to
25:59to spend more time as well um
26:01specifically for government
26:03organizations um
26:05>> operationalizing
26:07uh Atlas going from sort of like you
26:09know the the threat matrix uh towards
26:11application for their mission. So matrix
26:13to mission is sort of the tagline with
26:14that. Um and what that entails is is
26:17just really drilling down deep into uh
26:20their specific operation use case. Um
26:24and then mapping the threat landscape to
26:26to everything Alice related and and kind
26:28of like you suggested. Um just having
26:30some type of of testing evaluation
26:32toolkit um for prompt injections and
26:35sort of just like aligning that that set
26:38of samples um to either data that exist
26:41within that environment or typical
26:44requests that end users can send within
26:47that environment. That's just that's
26:48just one um notional example of of
26:51something that that we're wanting to
26:52drive towards. Um, so yeah, I think that
26:54there's a lot of potential for that type
26:56of thing moving forward. And I think
26:57it's a good time for it just because um,
27:01with Agentic AI and with retrieve
27:03augmented generation, at their core,
27:05they're still
27:07uh, they're still leveraging the same
27:11technology, the the large language
27:12model. And so at their core, they still
27:14have the same vulnerabilities. And so as
27:17we sort of on the knowledgebased side of
27:19things try and make sure that we're
27:21seeing sort of state-of-the-art and
27:22trying to understand um how just AI
27:26evolution um is is providing opportunity
27:30spaces towards attackers and making sure
27:31that that's reinforced within the
27:33matrix. We can do a whole lot on this
27:35operationalization side. Um
27:38understanding that elements at their
27:40core still are being used. um it's just
27:42sort of the way that the attack is sort
27:45of introduced in the system um is is
27:47what's is what's changing and so you can
27:49do you can get a lot that I think could
27:51could sort of scale um in terms of a a
27:54capability
27:57>> right that yeah I agree uh now one thing
28:00I will point out that in addition to the
28:03LLM capability then and as you are um
28:07you know your work focuses on is in
28:10addition to the LLM it's the new attack
28:12services that come with Agentic AI such
28:15as um you know with the uh tools or the
28:18orchestration uh capability of these
28:21agents, right? So do you have anything
28:24that reflects in the framework in the
28:27MITER atlas on for that as well?
28:30>> We do. Yeah. Um and really a lot of what
28:32they are are just agentic variants of
28:34techniques that already exist. So we
28:36separated them out just because the uh
28:38the definition is is slightly different
28:40and the application is slightly
28:41different. Um the effect of them is
28:44slightly different than than a technique
28:46that is sort of just on the bare large
28:48language model itself. Um and so that
28:51that with our um most recent release in
28:54September and we're also about to to
28:56release another update to the matrix
28:58later this month and in partnership with
29:00our um center for threat informed
29:02defense collaborators um which is a
29:05MITER non forprofit R&D center um
29:09traditionally focused on on MITER attack
29:12collaboration and we have like we have
29:13that secure AI version um as well um
29:17that will that will focus primarily on
29:18on AI agents Something that's
29:20interesting with with the Gent AI
29:22specifically actually is is sort of, you
29:25know, now that you have this this
29:26technology that can operate autonomously
29:29on its own, um trying to understand how
29:31the the AI agent in of itself can become
29:34an adversary. So something in addition
29:36sort of outside of our traditional Atlas
29:38matrix that we're um taking a real hard
29:41look at right now is um AI being used
29:44maliciously for things such as uh cyber
29:47operations. And so, uh, we have the
29:49beginnings of a of a knowledge base of
29:51that on our website as well. Um, you can
29:53find that in our Atlas 101 page. Um,
29:56it's just our malicious use of AI for
29:57cyber page. Um, the intention there is
30:00to try and understand how, like I said,
30:03AI can accelerate operations in a way
30:05that human operators can't. Um, and and
30:09try and understand how it can provide
30:10asymmetric advantage uh to an adversary
30:14if they're integrating AI into their
30:16operations.
30:18um if it can introduce new threat
30:20vectors that may not have been feasible.
30:22Kind of going back to the the severity
30:24topic, um if if certain techniques or
30:28attack chains can be feasible with human
30:30intervention, but now if it's automated,
30:31they can just work at an accelerated
30:33rate that that that demons can't on
30:34their own. Um and then we're trying to
30:36tailor that towards um specific
30:39industries or uh sectors such as supply
30:44chain and critical infrastructure. again
30:46just to try and relate it towards a a
30:49government lens um for uh organizations
30:53that are that are aligned to to those
30:54key sectors and areas.
30:57>> Okay, great. Uh so what about uh agents
31:00going rogue? Is that something that
31:02you're also um Okay.
31:04>> Yeah, rogue agents is an interesting
31:06one. Um, it's funny personally just
31:08because I remember uh speaking with one
31:10of my bosses a number of years ago about
31:11this idea back when um agents were were
31:15primarily just uh deep reinforcement
31:16learning based. Uh back then we called
31:18them double agents.
31:20Uh rogue agents is the is the term that
31:23that's gone mainstream for for
31:24generative AI and and yes that is we
31:26we've we've gotten a number of Atlas
31:28case submissions with respect to that.
31:30Um we have uh a lot of partners within
31:35our secure AI seated efforts uh that are
31:37interested in that as well. And so
31:38that's something that uh we're talking
31:40through with them. It that that that's
31:42an opportunity um for if you know for
31:46those that are in industry uh to engage
31:48with us directly is through through our
31:49seated initiative. You can become a
31:51member of that. You can become a
31:52benefictor of that bene benefactor of
31:54that to actually support um that that
31:57being ran. Um but we meet with our
32:01industry partners, you know, twice uh
32:03every other week for for an hour and
32:05they will propose things like like rogue
32:07rogue agents. Rogue agents is something
32:08that has just come up recently. So uh
32:10yeah, we'll see. It it's interesting
32:12because I think it's a it's a unique
32:14application of things such as
32:15jailbreaking and prompt injection. So it
32:17kind of just goes back to um the same
32:20techniques if you want to try and trace
32:22it and and form factor it to the the
32:24threat landscape. a rogue agent under
32:26certain conditions triggers different
32:28behavior. Um, and so it's interesting uh
32:32to see if it if if that particular use
32:36case breaks things a little bit for us.
32:37But but yeah, um that's something I'm
32:39excited for. Um just because it's a it's
32:41a cool topic if nothing else.
32:43>> Yeah, certainly. I actually created a
32:46framework around you know to adjust that
32:49how do you manage um and how do you keep
32:52things in control and put the right
32:53guardrails in from a process and a
32:56governance perspective. So you know as
32:59you know there's in order for something
33:01to be effective a guardrail to be
33:03effective there is the technology
33:05component the framework as well as the
33:07process. So it definitely goes hand in
33:09hand.
33:09>> Yeah. Um I know we're coming um up to
33:15the end of time and I know I personally
33:17have so many more questions. I would
33:20like to conclude with what next right
33:23when we are talking about the threat
33:25surface uh and the um the modality right
33:30you know a lot is focused on around text
33:34for example. Can you talk a little bit
33:36about what are your plans on introducing
33:40this kind of evaluation
33:43of threats and uh vulnerabilities across
33:46multiodality?
33:48>> Yeah. Uh great question. It definitely
33:50is something on the immediate horizon I
33:52think just like you said. Um and you're
33:55starting to see multimodal uh LLMs. It's
33:58interesting that they still call them LM
34:01multimodal LMS uh hit the scene with
34:04with images, with log files, um with
34:07video feeds. And so I don't think um
34:11that will sort of
34:15let me rephrase. I think I think what
34:17you'll see is is you'll see sort of uh
34:19combinations of different techniques
34:22kind of going back into the the
34:23perception um based uh realm that Atlas
34:28traditionally operated in before the the
34:30generative AI uh uh wave to where yeah
34:33you will have adversarial samples on
34:35images for sure um trying to evade some
34:38type of of classification or misclassify
34:42some some piece of information um
34:44combined with uh textbased adversarial
34:48samples and and video feeds are are sort
34:51of like a combination of the two and and
34:53audio is still uh an area that is just
34:55very difficult um to to sort of mitigate
34:59just because the uh the um the
35:02information uh feed is just so unique
35:05and it's so linear. Um we've we we've
35:08explored some things internally but but
35:10nothing's really sort of matured um to
35:12date at least for right now. So, um,
35:15yes, multimodal, uh, for us is going to,
35:18um, I think come into into increasing
35:21consideration with with a AI agents as
35:23they start to to, um, incorporate
35:26multiple different pieces of
35:27information. I think this is also a
35:28really good area for um, autonomy, large
35:32scale autonomy, embodied AI is is is a
35:35term that I I often hear more and more
35:37frequently. um where you have either an
35:40autonomous vehicle or some type of
35:41robotic that's receiving information
35:43across multiple modalities and and at
35:46the core just a single uh set of models
35:48or or multimodal model that is sort of
35:50orchestrating the the action sequences
35:53um that that system is is executing. And
35:56so, um, for right now, that's kind of
35:59where we're specifically focused is on
36:01this autonomy stack, uh, kind of similar
36:04to our malicious use of AI for for cyber
36:07page within the the Atlas knowledge
36:09base. We do have an autonomy one as
36:10well. Um, just because the attack
36:12surface when you get into into embodied
36:15AI, it just it grows exponentially to
36:17where uh, the amount of sensors just on
36:20a feed, it's it starts to just get get
36:22um, really crazy. And so, um, yeah, I'm
36:26that's another area where where I think
36:28moving into 2026, you'll see you'll see
36:30a lot of movement hopefully.
36:32>> Yes, I'm I'm, um, keeping my fingers
36:35crossed for that one.
36:37>> Yeah,
36:38>> I think we need that. U quite very
36:41important initiative. So, thank you.
36:45Um, all right. So as we end a quick
36:48lightning uh question for for you which
36:51is
36:53what do you think is the most underhyped
36:56attack vector?
36:59>> Most underhyped attack vector.
37:02>> Um
37:06it's a good question. Um I think that uh
37:09rag poisoning is fairly under hype just
37:12because it's so easy to to execute. that
37:14that's something that we released
37:15earlier on in the in the spring um where
37:19you have a uh large language model
37:22pulling information from a database um
37:25for some type of type of inferencing and
37:27I think that uh especially with the way
37:30you know oftent times you have um
37:34organizations leverage commercial
37:37pre-processing tools that are running
37:39across hundreds if not thousands of PDFs
37:41for example um and when they're parsing
37:44this information, there could be
37:45information in that PDF that isn't human
37:46visible, but is that is still there
37:48that's written from this technology and
37:50then integrated into this database and
37:51then, you know, it's immediately um
37:54accessible to the to the large language
37:55model. And so, uh that's one where
37:58there's so many use cases of that one.
38:00It's kind of I think hard to sort of pin
38:03it down um to kind of an
38:05all-encompassing case study. We do have
38:06a few with respect to that but um I
38:09think that that's personally under hype
38:10just because anytime you are talking to
38:13a uh customer assistant uh chatbot you
38:19know on on target or GrabHub or whatever
38:21it might be um they're retrieving
38:23information and so these things are just
38:25so common place nowadays that that uh
38:27yeah I I would definitely put that up
38:29there in in my top three if not number
38:31one.
38:33>> Oh wow. Thank you. Thanks for uh that
38:36insight. Very very valuable. So thank
38:39you very much. Uh and if you would like
38:42to add any anything as a concluding
38:44remark, please go ahead.
38:46>> Yeah, I just want to say thank you again
38:48Pamela. It's been a lot of fun. Um you
38:50can find our MITER Atlas knowledge base
38:52at miter.atlas.org. Um thank you again.
38:55>> Thank you. And I'll take some uh links
38:58or information from you and put it in
39:00the show notes.
39:02>> All right. And last to just to conclude
39:04um a word on the sponsor for this show
39:07which is record point. Um recordpoint is
39:11a platform and it gives you control over
39:14your data and AI governance for bringing
39:19in that trust and compliance and
39:22proactive risk mitigation.
39:24So it allows for uh visibility across
39:27the data um across the data life cycle
39:30and across the governance life cycle. So
39:33really um um just to reiterate anything
39:37that I will introduce into the show has
39:40been something that I have looked into
39:42personally. So I am very excited to have
39:45uh recordoint as a sponsor. So thank you

About this transcript

This page contains the full transcript of Decoding AI Attacks with MITRE ATLAS by Pamela Gupta, generated from the public captions YouTube serves with the video. The transcript has 6,304 words across 938 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.