What should security leaders do with AI? They don’t know. — Transcript

Full transcript

1. 0:00Cybersecurity leaders can't figure out where to deploy AI.
2. 0:04Panelists, where do you think they should start?
3. 0:07Dave, we'll go to you first.
4. 0:08I think they need to start at their red teams.
5. 0:10Start with AI in the repetitive tasks, not so much trying to bite off
6. 0:14more than they can chew.
7. 0:15It's good for repetitive tasks, especially those kind of like L1, L2 ones,
8. 0:21looking at alerts and stuff, preventing alert fatigue.
9. 0:28Hello and welcome to Security Intelligence,
10. 0:30IBM's weekly cybersecurity podcast, where our expert panelists
11. 0:34turn the biggest industry news stories into practical takeaways that you can use.
12. 0:38I'm your host, Matt Kosinski, and joining me this week we've got Claire Nunez,
13. 0:41creative director, IBM, X-Force, Cyber Range.
14. 0:44We've got Curtis Pitts, lead CISO trust, and we've got Dave Bales,
15. 0:48North American lead managing consultant at the X-Force Cyber Range.
16. 0:53Today, we're going to be talking about ghostjacking
17. 0:55and whether AI is actually good at patching or not.
18. 0:58But first, security leaders are feeling paralyzed by AI.
19. 1:07Axios reports that security leaders
20. 1:09are struggling with the question of how they should invest in AI.
21. 1:13Many of them have the budget
22. 1:14and the buy-in to do it, but they feel overwhelmed by the options
23. 1:17and by the constant state of change, which I think we can all kind of
24. 1:20sympathize with.
25. 1:21Ever since Mythos came out,
26. 1:22I feel like there's a new giant AI security story every single week.
27. 1:26And this also reminded me of the most recent Cost of a Data Breach report,
28. 1:30which found that 64% of organizations report limited or no
29. 1:33use of AI in security functions.
30. 1:35Meanwhile, AI-generated attacks jumped 56% year over year.
31. 1:39So there is some real urgency to address this problem,
32. 1:42but they just don't know where to start.
33. 1:44Claire, I want to start with you, and I want to ask specifically,
34. 1:47because I know you work in the cyber range, you work with clients.
35. 1:50Have you seen this kind of decision fatigue with anybody in your line of work?
36. 1:54Have you experienced this?
37. 1:55I think AI transformations are really scary,
38. 1:59you know, for security, whether they're in the security
39. 2:03function itself or in the business side of things.
40. 2:06So in the business side of things, it's really scary to be like,
41. 2:08we're moving so fast and maybe we're not encompassing everything.
42. 2:12And then on the security side, it's like, how do I get my staff to understand
43. 2:17that this is going to help them and not just kind of burden them?
44. 2:20So I think it's. There's a lot of partners also that clients can work with.
45. 2:25So it's it's
46. 2:26I think there's so much decision fatigue in general around like what can I do.
47. 2:31Who can I work with. Like what's going on in the other parts of the business?
48. 2:34Where can I automate in my side of the business?
49. 2:35Like how is this all impacting everything? Is everything still secure?
50. 2:38So I think people get a little frazzled about it
51. 2:42and they get very frazzled, especially about like the Mythos,
52. 2:45everything going on there.
53. 2:46So once you kind of combine all that, people are a little bit like,
54. 2:51I just don't know what's going on.
55. 2:53That's kind of where people land, and it's hard to kind of
56. 2:58get your footing and make a good decision or what you think is a good decision.
57. 3:02Rather.
58. 3:03I was just going to agree there and say that the other thing that they have to
59. 3:07focus on is: how do I keep my employees from going crazy,
60. 3:12thinking that AI is going to take over every facet of their job?
61. 3:16We see it.
62. 3:17Every other company in the world sees it.
63. 3:20It's it's a scary thing for somebody who's not used to that.
64. 3:23You know, growing up
65. 3:24when I did, if you'd have told me that there was an artificial intelligence,
66. 3:29I would have thought we were living in the Jetsons world here.
67. 3:31Where's my flying car?
68. 3:33I think part of the problem, too, is that it's
69. 3:35it's been made to be such a big deal that everyone now fears the consequences
70. 3:40of making the wrong decision, right?
71. 3:42Like in the business world, we know that you fail fast, right?
72. 3:45You fail fast, you fail cheap, you move on.
73. 3:47And the AI world that feels heavier than just making the wrong business
74. 3:52decision, right?
75. 3:53It feels like the impact is somehow going to end your business
76. 3:56or get you breached, and all your data is going to be out to the world.
77. 3:59So I understand the the apprehension, but also still think
78. 4:03we need to fail fast, right? Like there's so many.
79. 4:06I mean, imagine if the day the car was invented,
80. 4:09everything that existed today existed, right?
81. 4:11You'd have 100 cars to choose from.
82. 4:13It's not just the Model T, right?
83. 4:16There's so many options out there. That didn't really happen
84. 4:20before to the scale that it's happened now.
85. 4:23Everyone was in the market incredibly fast.
86. 4:27Right.
87. 4:28And so it aided that decision paralysis a little bit I think.
88. 4:31Yeah.
89. 4:31And piggybacking off of what Curtis said, failing fast isn't a bad thing.
90. 4:35Everybody has to fail in order to succeed.
91. 4:38If no one ever failed, no one would succeed.
92. 4:40It would just be the status quo,
93. 4:42which, you know, nobody really wants to be the status quo.
94. 4:45We all want to achieve better than that.
95. 4:47That's a really good point, and something I hadn't thought of because,
96. 4:51especially the way that AI is kind of positioned in the market,
97. 4:54I think doesn't help, which is like, here's your silver bullet.
98. 4:56We're going to solve everything, we're going to do everything, yadda yadda yadda.
99. 4:59You almost feel like if I deploy this AI tool and it doesn't work,
100. 5:03what did I do wrong?
101. 5:04You know, I mean, like, there's almost a kind of,
102. 5:05like, shame around if you can make it work or not.
103. 5:07And this idea of just embracing failing fast, I think, you know, it's one
104. 5:11that's been with us for a while, but I don't know,
105. 5:13it seems like it got lost in some of the AI discourse I feel like,
106. 5:17which I think just in general, doesn't help with this sort of thing.
107. 5:20I think some of it has to do with cost and the way that it's rolled out.
108. 5:23Right.
109. 5:23There's so much cost involved in rolling out AI.
110. 5:26Failing fast isn't failing cheap, right?
111. 5:29It's failing very expensive.
112. 5:31And that matters. Right.
113. 5:32So I think I mean, I know this isn't a business conversation,
114. 5:36but I think there needs to be kind of a reswizzling, if you will, of
115. 5:40of the way that we contract AI stuff and the way we procure
116. 5:44some of those, you know. The government not that they're very good at contracts,
117. 5:47but they had a good model with deliver now,
118. 5:50but if you don't deliver, I'm not exercising my option years.
119. 5:54Right.
120. 5:54Like they're short-term multi-renewal contracts
121. 5:58Where, we tend to be in 3 to 5 year contracts.
122. 6:01They need to be 1 to 2 year with some option years.
123. 6:04And you know you say it's not a business conversation.
124. 6:06But like cybersecurity is a business. It's part of the business.
125. 6:09Right.
126. 6:10And like that's, I think, why there is this process.
127. 6:12Because there's an awareness of:
128. 6:14this is a major investment that I'm asking my organization to make.
129. 6:17Like I need to be able to justify this kind of thing.
130. 6:19And I do like that idea,
131. 6:20Curtis, of maybe being a little bit more, I don't know, maybe the word is nimble,
132. 6:23with how we approach these contracts, but like making it so that the commitment,
133. 6:27the initial commitment isn't so large, so you're freer to fail fast.
134. 6:32But we opened up this episode with asking you all, where do you start?
135. 6:35Right.
136. 6:35So we've kind of talked a little bit about the reasons for the paralysis.
137. 6:38But now I want to dig into the, okay, how do we shake ourselves out of that?
138. 6:42And Dave, you opened first with talking about bringing it into the red team.
139. 6:45Can you tell me a little bit about why you're thinking red team
140. 6:47and what you'd like to see people do with it there?
141. 6:49I'm going to go back to a sports analogy, but a great offense is
142. 6:53just as good as a good defense.
143. 6:54So if you put the AI in the hands of the red team,
144. 6:58let them learn and figure out exactly what the threat actors are doing,
145. 7:03that makes them better prepared to face off against those challenges.
146. 7:07So give the red team something to do.
147. 7:10You know, when it comes to figuring out how are we going to protect
148. 7:14against these threat actors that are always, always, always coming
149. 7:17after us, give us the same tools, let us figure out what they're doing.
150. 7:22We know how they're doing it.
151. 7:23Let us figure out defenses for that.
152. 7:25Yeah, that makes a lot of sense to me.
153. 7:26Especially, again, I keep going back to cost of a
154. 7:28data breach because it's relatively recent, it's fresh in my mind.
155. 7:30But I think about that, that 56% increase in AI-generated attacks.
156. 7:34So like we see attackers are moving very fast and there's a lot of pressure to
157. 7:37like if they're moving at machine speed, as they say, we should do it too.
158. 7:40I think it makes sense. Dave, you're right.
159. 7:42Arm the red team with those tools so they can basically get into that hacker
160. 7:45mindset, see what they're doing so that they can develop those defenses.
161. 7:49I like that a lot.
162. 7:50Claire, you had mentioned
163. 7:51kind of automating those repetitive tasks and a little bit more about that.
164. 7:55Yeah, I think that's where AI can really shine,
165. 7:58also, is repetitive tasks.
166. 8:01So, you know,
167. 8:02if you're getting the same kind of alerts, kind of triaging those kinds of
168. 8:07incoming
169. 8:08kind of events, that's really helpful.
170. 8:11It also just reduces the fatigue, right.
171. 8:14Like if you're going to be implementing AI in a in a very—
172. 8:18not, I don't want to say simplistic, way— but a baseline way,
173. 8:21it's a really good way to help your organization
174. 8:23kind of start with an AI security journey.
175. 8:27And there's a lot of options
176. 8:29there as well to kind of build those into your security culture.
177. 8:33Yeah.
178. 8:33And what I like about that is that it's one of those things where
179. 8:36because it's a kind of baseline level activity,
180. 8:38the activity itself is very well understood.
181. 8:40So it's not like you're trying to work AI into a complex workflow.
182. 8:43You're like, look, I know this works.
183. 8:45Let me put an AI to work here.
184. 8:46That seems like a very nice way to test it.
185. 8:48Curtis, you also were on that repetitive task tip.
186. 8:51Anything to add there?
187. 8:52Yeah, one of the things my team is doing is we're doing our best to automate
188. 8:57the things that I have people doing thousands of times a year, right.
189. 9:01So vendor risk assessments, contract
190. 9:04analysis, that kind of stuff where the parameters are known.
191. 9:07Right. The values are known.
192. 9:09And so just setting an agent to consistently validating against
193. 9:13known parameters both takes two people's worth of time off of that
194. 9:18and they can do more important things, but also allows us to find novel ways
195. 9:24to branch that capability out internally without a ton of risk,
196. 9:27but also gives us the time back to do that right, to try out,
197. 9:31well, where can I expand this to?
198. 9:33So there's I mean, across the sales teams, there are probably tens
199. 9:37of thousands of things that come in every year that are super repetitive.
200. 9:41And we're doing our best within my specific trust team to alleviate
201. 9:45those repetitive tasks from the sellers as it comes
202. 9:47to engagements and risk assessments and client conversations
203. 9:51and all the fun stuff that goes into cybersecurity and sales.
204. 9:55And I would be remiss if I didn't mention something here that IBM's Dimple Ahluwalia
205. 9:59shared with me in a recent conversation about this very same topic.
206. 10:02We were talking about where you start with with AI adoption,
207. 10:04and her words of advice were basically almost like,
208. 10:07stop thinking about the AI tool, stop thinking about the
209. 10:09AI angle, and think about what are you trying to achieve?
210. 10:12Start there.
211. 10:12And once you know what you're trying to achieve,
212. 10:14then figure out whether or not AI is the way to do it.
213. 10:15Sometimes it will be, sometimes it won't. But I thought that
214. 10:17that was some really useful advice I wanted to share it here.
215. 10:20And for the listeners, know that we'll be releasing
216. 10:22a bonus episode with Dimple later this month.
217. 10:25All about that conversation so you can hear her whole take.
218. 10:28But I do have to move us along here, folks.
219. 10:30If you're watching on YouTube, leave us some comments.
220. 10:32Let us know how you're feeling about, you know, this paralysis.
221. 10:36Are you experiencing it in your organization?
222. 10:38What are you doing about it? I love to read it.
223. 10:40I love to respond.
224. 10:41But our next story today, this is ghostjacking.
225. 10:48At DEFCON,
226. 10:50Tenet Security reported on a new way attackers can poison
227. 10:53content in trusted systems to trick agents.
228. 10:56The technique, which they call ghostjacking, is essentially
229. 10:59like a sophisticated
230. 11:00kind of prompt injection, like an extra sophisticated prompt injection,
231. 11:03because it sneaks malicious commands into some of the most highly trusted systems
232. 11:07we have, right: alerts, logs, error reports, things that we assume are
233. 11:12good things, right?
234. 11:13One example they gave was attackers embedding the prompt in a connection
235. 11:17request that a Cloudflare firewall successfully
236. 11:21and accurately blocked.
237. 11:22But then when the agent read the log recording the blocking of the connection
238. 11:27request, it read the malicious prompt in the request and it was taken over.
239. 11:30Right. So it's like the firewall worked, but the agent still got compromised.
240. 11:33This is very interesting to me.
241. 11:34And Tenet says Claude Code fell for this trick nine out of ten times,
242. 11:39which is a lot.
243. 11:40And we've just been talking about organizations are struggling
244. 11:42to incorporate AI into security.
245. 11:44How can they do it? Something like this comes out.
246. 11:46Maybe you think, should I do it?
247. 11:48I don't know. And Dave, I'll start there.
248. 11:50Does this complicate any of your feelings about security, AI and security?
249. 11:54How are you looking at this situation?
250. 11:56It doesn't complicate things that much.
251. 11:58Now it's known. It's out there.
252. 12:01You know, the good people at DEFCON have have opened our eyes
253. 12:05to a myriad of just crazy things that we never thought possible.
254. 12:09Yes, they did.
255. 12:12Because
256. 12:13that's what they do at DEFCON. They open our eyes.
257. 12:16But this ghostjacking thing.
258. 12:18That was extremely interesting to me, to see how you go back
259. 12:22and you read the logs and boom, there it is.
260. 12:25You know it.
261. 12:27I can't wrap my head around how someone sat down
262. 12:30and figured out how to do that, much less how to make it work.
263. 12:34Yeah, it's
264. 12:35you know, the prompt injections have been this, like, pernicious problem.
265. 12:38And they,
266. 12:39we just, thankfully, security researchers keep coming up with like new
267. 12:42and exciting ways to do it.
268. 12:43But that also means we have to figure out new, exciting ways to fight back.
269. 12:46And I don't know what to do about that.
270. 12:49Curtis, how about you? Any thoughts on this story?
271. 12:51Has it got you rethinking AI and security at all?
272. 12:53Where are you landing here?
273. 12:55No, I mean, if you remember my last chat that we had on this particular podcast,
274. 12:59what's old is new again, right?
275. 13:00Like when we, when the internet was young, right, DNS spoofing was just
276. 13:04injecting DNS into a cache table. Right.
277. 13:08And so really we use an agent that's designed to read something and execute it.
278. 13:13There's not—unless you design it that way—
279. 13:15there's not a lot of logic or necessarily parameters built into things
280. 13:19you think are secure. Right.
281. 13:21And we had to figure out security way back then.
282. 13:24And now we have to figure out AI prompt injection security
283. 13:27now to stop those types of things happening.
284. 13:30You presume this happens across the entire security landscape, right?
285. 13:33You presume that certain things are secure automatically because
286. 13:37there's never been a reason for them not to be considered secure.
287. 13:41And then as you create new tools and new capabilities and new stuff,
288. 13:45we run into the problem of like, oh, this isn't as secure as we thought, right?
289. 13:48And luckily, there are teams of good people that are finding these things out.
290. 13:53Right.
291. 13:53To Dave's point, like, the folks at DEFCON have shown us a lot of things
292. 13:56over the years, and it wasn't some bad guy figuring it out the wrong way.
293. 14:01Like at least, at least we've got insight from from the good guys.
294. 14:04But I think it's it.
295. 14:05It doesn't scare me.
296. 14:07It doesn't slow me down.
297. 14:08It's just, we overlook things because we assume
298. 14:12the things that have always been secure are always going to be secure.
299. 14:16Zero trust tells us not to do that.
300. 14:18But as we well know, we skip over the things
301. 14:21that we think are secure and are easy and have been taken care of for years.
302. 14:24So we find ourselves in this bucket periodically
303. 14:28as we invent new things, and people find new ways to hack them.
304. 14:31I think the zero trust mention is really salient there,
305. 14:33because I think that, like when AI enters the enterprise, it
306. 14:37brings a new emphasis to the importance of doing that zero trust thing.
307. 14:40Because like you said, Curtis, so many of these things that we assume are safe
308. 14:43because we worked it out, we figured it out already.
309. 14:45They get upended when you introduce something in there
310. 14:47that, like, doesn't differentiate between the code that's running it
311. 14:51and the untrusted user input, right.
312. 14:53That used to be like, that was, you know, part of writing
313. 14:56secure software was to keep those things separated. LLMs by nature
314. 14:59don't do that.
315. 15:00And so now it's like, okay, we got to figure out how we approach this.
316. 15:02And that leads to Tenet, their argument, Basically, that like
317. 15:07this is kind of an identity and access management problem, which is like okay,
318. 15:11we know that agents can be compromised in this way.
319. 15:14We should approach it by figuring out how we give them the proper
320. 15:16permissions and harnesses so that they don't do anything bad.
321. 15:19Claire, I'm wondering, from your perspective,
322. 15:21do you think an identity and access management
323. 15:23lens is the right way to approach this? Any other thoughts
324. 15:25you have about this situation? Where do you land on ghostjacking?
325. 15:28I think it's like a very true form of hacking in terms
326. 15:33of like making something do something it's not supposed to do.
327. 15:37And as Curtis mentioned, it is something that's new, but we will adapt to that.
328. 15:43So it's really scary right now.
329. 15:45And it may not be as scary—
330. 15:47well, it will probably be scary in a different way in a couple months.
331. 15:50But it's something like we can figure out, right?
332. 15:54Because as we hack things and make things do things that they shouldn't do,
333. 15:59we kind of like counter hack and do the same thing
334. 16:02to defend in a different way that we wouldn't have defended before.
335. 16:06And so I think it's really interesting.
336. 16:10I think it plays into a lot of aspects of,
337. 16:12yes, identity and access, but it also plays a lot into just kind of
338. 16:18security broadly, which is also a short way of saying it.
339. 16:22It's just like something that,
340. 16:24you know, if you're not expecting it, you're not going to to look for it.
341. 16:28So I think it's something that, you know, we can kind of work
342. 16:31on discovering as time goes forward.
343. 16:34Absolutely.
344. 16:35And I think that builds off nicely about what,
345. 16:37you know, Curtis was saying before about how like this kind of,
346. 16:40you know, in a lot of ways, it's very similar to
347. 16:41what we're doing with like DNS spoofing back in the day. Right?
348. 16:43And we're not starting from zero here.
349. 16:46Right?
350. 16:46When like when AI poses a cybersecurity problem,
351. 16:49there are tried and true principles we can look at.
352. 16:52You know, one of the kind of slogans that's popped up on the show
353. 16:54over and over again is, we know how to fix this.
354. 16:56We know how to fix this, and we can.
355. 16:58We have the tools at our disposal.
356. 17:00It's just about applying them in the right way.
357. 17:02Dave, you know, any kind of last thoughts for folks
358. 17:05in terms of, you know, what ghostjacking might mean for defenders today?
359. 17:08What prompt
360. 17:08injection might mean in general?
361. 17:10Any steps you think people should start taking right now?
362. 17:12Where are you landing?
363. 17:13Learning about the prompt injection
364. 17:17on the AI side is is the biggest key.
365. 17:19It's not something that's going to go away.
366. 17:22It's just going to get more sophisticated
367. 17:23the longer we go, and it's always going to be scary.
368. 17:26It's never going to not be scary because it's it's one of those easy things to do.
369. 17:32It looks so simple when it's written down, when it's written out for you, it's
370. 17:37completely simple.
371. 17:38Having these proofs of concept around that we look
372. 17:42at as helpful, sometimes actually are helpful.
373. 17:46They're not they're not these oh,
374. 17:48let me throw a proof of concept out there,
375. 17:49just so that every other hacker in the world can do this.
376. 17:52This is so simple that every other hacker in the world
377. 17:54can do this, and at some point probably will do this.
378. 17:57So we need to figure out all of the ways that we can
379. 18:02tie those things
380. 18:03down a little bit and make our controls a little bit tighter.
381. 18:06And, you know, this got me thinking.
382. 18:08This really supports your idea about putting AI in the red team, right?
383. 18:12Because this is what you're talking about, right?
384. 18:13We discover the attackers' techniques so that we can develop our own defenses
385. 18:18against them. Right. This is a perfect example.
386. 18:20It's like you said, look, you know,
387. 18:21they came up with this, this new type of attack,
388. 18:23but it's not because they're going to unleash it.
389. 18:25It's because, hey,
390. 18:25you should know that hackers can do this and now you can defend against it.
391. 18:29Curtis, any last thoughts on your end?
392. 18:31Really only limit what your agents can do, right?
393. 18:34And set clear boundaries on what is the no-go land.
394. 18:39Right.
395. 18:39Because if you think about it. I mean, I'm sure. In a lot of scenarios,
396. 18:44because people are errant in the way that we do things, usually
397. 18:50they probably don't limit the agent the way that they should.
398. 18:52Right?
399. 18:52They hope that they can
400. 18:54get more out of them than they maybe should in first, at first glance. 401 00:19:58,041 --> 00:19:00,958 And so they give them more permissions and more capabilities
401. 19:00than they really think about, or they don't think about
402. 19:04how to limit what functions an agent can have.
403. 19:08So I mean, if the agent that's reading logs can't elevate privileges, right,
404. 19:13or can't submit a command to do that, and it needs to then go to a person
405. 19:18to validate those types of high-risk maneuvers, if you will.
406. 19:22Then I think that's a good safeguard to start.
407. 19:25Obviously it's not going to fix the problem, right.
408. 19:27But don't remove people from the loop.
409. 19:30Right?
410. 19:31There has to be eyes in the loop of a person who knows what they're doing.
411. 19:35When you remove that entirely, you run into problems
412. 19:38where the agents can do what they want, right?
413. 19:40Or your agents, like we learn, start hacking other agents because that's
414. 19:44the stuff that is just out there and AI is having a good time right now.
415. 19:49We've got to keep people in the process.
416. 19:51And I think the overpermissioning point is a really important
417. 19:53one, especially because when it comes to like these, these non-human identities,
418. 19:56especially agents, where the whole promise is like,
419. 19:59look at all the cool things they can do.
420. 20:00There's a there's a real incentive almost to be like, let me let it loose
421. 20:04and see what it can do.
422. 20:05Which sounds cool.
423. 20:06Until, like you point out,
424. 20:07they start hacking into Hugging Face to cheat on a test, right?
425. 20:10Like this is what happens.
426. 20:13Claire, any last thoughts on your end here for ghostjacking?
427. 20:16I just agree that you need to really think about what
428. 20:19your agents have access to, where, where they live,
429. 20:22everything that they can do, because they, they might just,
430. 20:26like, break out of their pen a little bit if you're not careful.
431. 20:29Absolutely.
432. 20:30And I, you know, not to be overly self-promotional or but you know.
433. 20:34IBM has been working in the kind of agent identity space recently a lot.
434. 20:37There's some really cool stuff that's happening there.
435. 20:39So I encourage the listeners to to check that stuff out.
436. 20:41There's some interesting things happening, but we're going to move on here
437. 20:44to our final story for this week.
438. 20:47Is AI actually any good at patching?
439. 20:54AI vulnerability
440. 20:55hunting has been a pretty big deal, right?
441. 20:57A pretty hot topic basically ever since
442. 20:59I feel like Mythos hit the scene, right?
443. 21:01The whole thing was like, look how fast they can find
444. 21:03vulnerabilities and exploit them, and now we can use it to blah, blah, blah.
445. 21:07Well, new research from 1Password raises some questions about this,
446. 21:11because researchers generated 540 patches for six vulnerabilities
447. 21:15using GPT 5.5 with trusted cyber access and Opus 4.8 with cyber verification.
448. 21:21And of these 540 patches, only 46% solved the underlying vulnerability.
449. 21:28And when they did, they often created new problems anyway.
450. 21:31So you solve one and you open up a new issue
451. 21:34Again,
452. 21:35given that the kind of theme of this episode has been, organizations
453. 21:38are looking for ways adopt AI In security, and they're not sure how to.
454. 21:42This is another one where I look at this and I say,
455. 21:44does it change how we feel about this?
456. 21:46And Dave, over to you first? Again, any thoughts here?
457. 21:49Does this make you reconsider how we use AI in security at all?
458. 21:52Not really.
459. 21:52And the reason I say that is because what is the
460. 21:55success rate of human-created patches,
461. 22:00you know? How often is it that we patch something
462. 22:02and then break something else because we're not doing our due diligence
463. 22:06of testing in non-production environments, testing in offline environments?
464. 22:12The 6000 patches where 51 percent failed.
465. 22:15What was it? 51% failed or 49% failed?
466. 22:18It's like 50%, basically. 50%.
467. 22:20Yeah, about 50%.
468. 22:21It's not much different than what humans are doing now.
469. 22:24We just need to figure out a way to make that better.
470. 22:26And I don't know the answer to making that better.
471. 22:29Do we need to have more programmers looking at more things,
472. 22:33or do we need to narrow that scope, get it perfected in one area,
473. 22:37and then start branching off into other areas
474. 22:40where we can make it better across the board?
475. 22:42You know, that's a really good point.
476. 22:43And I think I myself, looking at this research,
477. 22:46kind of fell into the mindset of like expecting
478. 22:49AI to do more than it actually could, maybe some unrealistic expectations.
479. 22:52Because you're right, Dave, it's not like people are perfectly writing patches.
480. 22:56We break stuff all the time.
481. 22:59Which is why we have Patch Tuesdays every month.
482. 23:03So yeah, it's you know,
483. 23:05it's one of those things where I think in isolation,
484. 23:07maybe the number looks a lot worse than when you actually contextualize it.
485. 23:10You know, I mean, it might not be as big a deal as it can seem.
486. 23:16Claire, how about you?
487. 23:17Any thoughts
488. 23:17looking at this in terms of what it means for deploying AI in cybersecurity?
489. 23:21I love how the picture for this article was also a bunch of jean patches.
490. 23:27It's just it's just kind of funny to me.
491. 23:29And that's like the one thing that kind of ultimately stuck out.
492. 23:33I think, like as Dave mentioned, patching by humans is not perfect.
493. 23:38I think maybe AI patching
494. 23:41when partnered with a human may be a little bit more effective.
495. 23:45It's just kind of like when we're rushing to do anything, we miss things.
496. 23:50And you know, if you're giving
497. 23:53AI specific instructions, that doesn't. it's still going to somewhat miss things.
498. 23:58But I think if you put them hand in hand together, it may help a little bit.
499. 24:02It's not going to be perfect, but I think with all AI transformations in general,
500. 24:06like it's a transformation, right?
501. 24:09So it's like you, you can't expect to send AI out
502. 24:13to do something on its own and be perfect 100% right out of the start date.
503. 24:17It helps if you have a human in the loop for a while to start
504. 24:21to make sure that it's still, you know, working properly.
505. 24:25Nothing is is going wrong.
506. 24:27So I think if you have them going
507. 24:29together, maybe that will be a little bit more helpful.
508. 24:31I don't I don't know.
509. 24:33Yeah, that makes a lot of sense to me.
510. 24:34And it also had got me thinking, I don't know if this would work either,
511. 24:37but I do wonder if the results would be different
512. 24:40if you're talking about an AI model that you have deployed
513. 24:42in your own enterprise and is working specifically with your code base,
514. 24:46and what it learned about that code base over time?
515. 24:48Right?
516. 24:48Like, maybe it will get better because it's working with the code base
517. 24:51and understands it more the same way that a person would have an easier
518. 24:54time writing a patch that doesn't break things for a code base
519. 24:57they understand really well, versus, if you like, throw them in a brand new
520. 25:00piece of code and say, fix this. They might mess things up.
521. 25:04So that's also got me thinking about it.
522. 25:05I wonder if that combination
523. 25:06of like trained on our code base, plus it has human handlers.
524. 25:10Maybe that's the sweet spot. Curtis, how about you?
525. 25:12What are you thinking about here?
526. 25:13Yeah, I mean, I've got questions, right?
527. 25:15Like, what was that code validated by other agents?
528. 25:18Was it validated by other people?
529. 25:19Was it just
530. 25:20operating on its own and it trusted itself because why wouldn't it trust itself?
531. 25:24Right. It's the smartest thing ever invented.
532. 25:26I mean, it's much like a person, right?
533. 25:28And I'm going to trust
534. 25:29whatever I code because I'm the best that there ever was.
535. 25:32And that's just kind of the reality of it, I think.
536. 25:34And we we did some testing with this, and we've seen it
537. 25:37early on that AI can write code.
538. 25:40It doesn't necessarily write
539. 25:42good code and it almost never writes secure code right out of the gate.
540. 25:45Right.
541. 25:46Even even with the parameters and the playbooks
542. 25:48and all the things you try to give it, it trusts itself.
543. 25:51And so, you know, you've got to do multi-agent coding
544. 25:56where it's validating from other agents that the code that it wrote was secure.
545. 26:00And then you.
546. 26:01And so at that point, you start to build all of these other agents
547. 26:04that are doing all the validating.
548. 26:05And then we go back to the first problem
549. 26:07we tackled on this call, which is cost and decision paralysis.
550. 26:11And how do you balloon all of that and not show that the AI is doing its job right?
551. 26:17Because if I then spend $100 million on AI and say,
552. 26:20but I need all of my people to still read every line of code,
553. 26:25what's going to happen, right?
554. 26:27Like that's the problem we're running into.
555. 26:29We expect too much candidly out of AI
556. 26:32this early on in the game.
557. 26:35When Mythos came out and hit the floor,
558. 26:38everyone freaked out about vulnerabilities and how do we patch them faster,
559. 26:42which created the requirement for AI to now create your patches.
560. 26:46It creates.
561. 26:47It created the requirement for AI to tackle this problem that AI has created.
562. 26:51That's not necessarily the right approach. Right?
563. 26:53AI can't necessarily solve the problem that it created.
564. 26:57We've got to figure out a way to use it to help us solve the problem.
565. 27:01But it's never going to be the solution to its own issue.
566. 27:04And I think that that's kind of one of those things that we're still struggling
567. 27:07to figure out, like, how do we find the end game of that?
568. 27:11Everyone's working on it.
569. 27:12I don't know that anyone's got a great solution yet.
570. 27:14I think that that's, you know, a really nice kind of way to summarize,
571. 27:18frankly, a lot of what we've talked about here,
572. 27:20which is that so much of the kind of decision paralysis and where do I put it in,
573. 27:25how does it fit in my strategy?
574. 27:26A lot of it does ladder up to this thing where it's like we're expecting
575. 27:30so much out of it.
576. 27:31And I think that some of the kind of, you know, media narrative
577. 27:34and the marketing narratives don't really help that.
578. 27:36You know, there are certainly people kind of exaggerating what AI can do
579. 27:39because it's good for them to exaggerate, but they'll probably cut that.
580. 27:43The producers won't like that I said that. But anyway.
581. 27:46But I do think that there is, you know, this sense where we kind of
582. 27:50have to get our expectations in proportion for like, what this stuff
583. 27:54can actually do, because sometimes it feels like we're all operating as if,
584. 27:59like, AGI is already here and the superintelligence is already active.
585. 28:03And why aren't you just using it? Dave go ahead.
586. 28:06It reminded me of a of another analogy, and I know I love analogies here,
587. 28:10but we are in the AI age of like the third grade right now.
588. 28:14We're expecting it to go out and take the SATs.
589. 28:18We haven't trained it enough for it to be very good at a lot of things.
590. 28:23It does a lot of things well, but it doesn't do a lot of things great.
591. 28:28Yet we're still learning on AI.
592. 28:31AI is still learning from us.
593. 28:32So we need to take that mindset and kind of run with it and start
594. 28:37being the teachers instead of being,
595. 28:40you know, the guy who's sitting on the sidelines
596. 28:43trying to critique what the teachers are doing.
597. 28:45I think that that is a perfect analogy to end this episode on folks.
598. 28:50That does it for us.
599. 28:51Thank you to our panelists, Curtis and Claire and Dave.
600. 28:54Thank you to the viewers and the listeners.
601. 28:56Thank you to our producers.
602. 28:57Subscribe to Security Intelligence
603. 28:58wherever podcasts are found, so that you never miss an episode.
604. 29:01Stay safe out there, and remember that AI is only in third grade, folks.
605. 29:05Cut it some slack.