YouTube2Text

Martin Odersky – Keynote: The End of Programming? #ScalaDays — Transcript

by Scala Days Conferences · 8,027 words · 1,128 segments · language en · Watch on YouTube

Full transcript

  1. 0:07Okay. So, welcome to Scala Days 2026.
  2. 0:11Uh, I think we all have uh pretty
  3. 0:14momentous uh events of the last year on
  4. 0:18our mind. A lot of a lot has happened
  5. 0:20around AI and agents. So, I think there
  6. 0:23are many people now who ask is it the
  7. 0:26end of programming? So common statements
  8. 0:29are uh Antropic says software
  9. 0:32engineering could be automated in 12
  10. 0:34months. So that was last uh January
  11. 0:36February that he said that. So that's
  12. 0:38pretty imminent, right? Uh uh Sundai
  13. 0:42Pear Google said 75% of all code is
  14. 0:45written by agents. And Ruby on Rails, he
  15. 0:49gave this uh famous or infamous
  16. 0:51conference talk very recently who said
  17. 0:53basically yeah it's over people. Uh the
  18. 0:56uh all code in our company is written by
  19. 1:00agents, no humans involved.
  20. 1:03Okay. So um you might say uh well okay
  21. 1:07maybe we can let agents write the code
  22. 1:09but we should oversee it right we should
  23. 1:12review the code. Uh nobody writes code
  24. 1:14anymore but you still read it for review
  25. 1:17right no I think that also that time has
  26. 1:21passed. the reviews are now done by
  27. 1:23other agents and I think humans don't
  28. 1:25really like to review bunches of code in
  29. 1:27particular if they're like the
  30. 1:29avalanches produced by the agents so
  31. 1:32forget about that so I think we should
  32. 1:34ask well what roles do humans play in
  33. 1:37all of this um and uh what we what we
  34. 1:41see here is a a typical workflow u uh
  35. 1:45that uh the u yeah the typical workflow
  36. 1:49where uh we write uh uh a prompt to
  37. 1:53demand some sort of program. The AI
  38. 1:55produces an initial program. Then we
  39. 1:58might have change requests or our users
  40. 1:59might file tickets and that all gets
  41. 2:01piped into the agent. The agents will
  42. 2:04will update the program. But the program
  43. 2:06in the whole process stays a black box.
  44. 2:08So we don't understand what's in there.
  45. 2:10We just tell essentially this AI to do
  46. 2:13something with the program and the
  47. 2:15program well the program will just
  48. 2:17behave the way the agent programmed it.
  49. 2:19a human doesn't look inside.
  50. 2:22Um,
  51. 2:24sorry, I I I skipped ahead. So, in this
  52. 2:27scenario, what about programming
  53. 2:29languages? Um, I can't really see how
  54. 2:32they would matter anymore, right? Uh,
  55. 2:34whatever's most convenient for agents,
  56. 2:37uh, fastest turnaround, least tokens,
  57. 2:39most efficient code, that's probably the
  58. 2:41metrics that that matter, but not
  59. 2:43readability, modularity, safety,
  60. 2:46expressiveness. Who cares about that?
  61. 2:49Hm.
  62. 2:51Is that future likely? Um, maybe. Many
  63. 2:55claim so. I don't pretend that I have a
  64. 2:58crystal ball so I don't take a position
  65. 3:00but it might be. But we should also ask
  66. 3:03is that the future we want? And uh then
  67. 3:07uh if we should follow that up to say
  68. 3:09well is that the future we want? So who
  69. 3:11is we? Is we programmers? They probably
  70. 3:15don't want that future, right? That it
  71. 3:17puts their jobs in danger. Uh it takes
  72. 3:20out the fun in their work. Uh so I think
  73. 3:23it's pretty bleak. Their bosses might
  74. 3:26well embrace it. Uh because well maybe
  75. 3:28the whole thing is cheaper than and
  76. 3:30faster than uh relying on humans to do
  77. 3:33that. Uh but if we scale out and say
  78. 3:36humanity as a whole, do we want that as
  79. 3:38humans?
  80. 3:40So there's a problem with that because
  81. 3:42if we give up the ability to understand
  82. 3:45and review programs, we literally hand
  83. 3:48the keys to the agents, our own or our
  84. 3:51an attackers. So we we are out of that.
  85. 3:53The agents essentially do that among
  86. 3:56themselves. And that is a shorefire way
  87. 3:59to get the singularity not because of of
  88. 4:02a machine intelligence explosion but
  89. 4:04because as humans we abandon the core
  90. 4:06domain that runs civilization in the
  91. 4:08software construction. If we do that
  92. 4:10then yeah essentially we we is a
  93. 4:12dereliction of of of agent of duty that
  94. 4:16leads to a loss of agency of humans. So
  95. 4:17I think it's really dangerous to that uh
  96. 4:20that these things work out this way.
  97. 4:24So um what are the alternatives? Um can
  98. 4:28we all go back to manual coding? I fear
  99. 4:30that cat is out of the bag. Uh should we
  100. 4:33review all agent generated code? Well,
  101. 4:36nobody wants to do that. Can we somehow
  102. 4:40try to raise the level of abstraction
  103. 4:42similar to what happened when we went
  104. 4:43from assembly to compiled code? uh where
  105. 4:46we say okay so nobody reads assembly
  106. 4:48anymore but it doesn't matter because we
  107. 4:50read C and Scala and and uh and other
  108. 4:54languages and these express precisely
  109. 4:56what the computer should do so we just
  110. 4:59instruct the computer at a higher level
  111. 5:01is that another level raising well the
  112. 5:04problem is that agents they resemble
  113. 5:06compilers somewhat we give them prompts
  114. 5:08in English or natural language and they
  115. 5:11produce code but they are like Christy
  116. 5:14likes to say they're flaky compilers,
  117. 5:16the compilers that don't produce the
  118. 5:18same thing each time, that might make
  119. 5:20mistakes, that don't give us a guarantee
  120. 5:23what they do. Um, so if you want to
  121. 5:26control their behavior and ensure
  122. 5:28correctness of our program, we need some
  123. 5:31external way. We can't just instruct the
  124. 5:34model and say please don't make mistakes
  125. 5:36or wait for the next model which is
  126. 5:38hopefully more reliable than the current
  127. 5:40one. I think this one this will not work
  128. 5:42out. we need external ways to control
  129. 5:44the behavior and ensure the correctness.
  130. 5:48Okay. So um what this talk is about an
  131. 5:53exploration what we might do there and
  132. 5:56one good point to start is AI agents. Uh
  133. 6:00so that could be coding agents or other
  134. 6:01agents that do stuff for you because for
  135. 6:04these AI agents the future has already
  136. 6:06arrived. These AI agents, they invoke
  137. 6:09tools and they do so by uh generating
  138. 6:13little program snippets at least uh
  139. 6:16mostly now. There are other ways to
  140. 6:17invoke tools but uh code execution is a
  141. 6:20very very common one. So the agent
  142. 6:22generates a little program, nobody looks
  143. 6:24at the program, the program is
  144. 6:25immediately executed by a tool. So that
  145. 6:28means we have LLMs that generate unreed
  146. 6:31code that invoke tools that can have
  147. 6:34arbitrary side effects. So that's the
  148. 6:36current situation and as we all know
  149. 6:40that means we really living dangerously
  150. 6:42uh the another week goes by where
  151. 6:44there's not another incidence of a
  152. 6:46private information leakage API keys
  153. 6:49credentials trade secrets whatnot that
  154. 6:52there's data loss uh the head of
  155. 6:54security of Meta had his had his email
  156. 6:58erased for instance um and of course
  157. 7:02lots and lots of sensitive systems are
  158. 7:04hacked so everybody Everybody knows the
  159. 7:06hugging face interface where essentially
  160. 7:08the AI hacked hugging face uh uh and it
  161. 7:13wasn't detected for I think a whole week
  162. 7:15that what what happened there. So it's a
  163. 7:17swarm of a agents hacking things and
  164. 7:19then the hacked into the Australian
  165. 7:22Medicare site and lots of other sites
  166. 7:25and of course this is just the
  167. 7:27beginning. I mean it's just starting and
  168. 7:29agents will become a lot more numerous
  169. 7:31and a lot more powerful and we're really
  170. 7:34living in a in a scary time for that. So
  171. 7:38with that I think there are two
  172. 7:40challenges. The first challenge is how
  173. 7:42do we contain these AI agents? Uh so how
  174. 7:45do we prevent them from hacking into
  175. 7:49systems leaking things? Can can we
  176. 7:51somehow give them guard rails that
  177. 7:53ensure and in a guaranteed way that they
  178. 7:56can't do that? And the second challenge
  179. 7:58I think is how to design software to
  180. 8:01better resist agentic attacks because
  181. 8:04the if you look take the hugging face
  182. 8:06incidents it really has two sides. It's
  183. 8:08the agents breaking out and attacking
  184. 8:10hugging face but it's also hugging face
  185. 8:12letting itself being attacked. So so and
  186. 8:15we take that sort of as a law of nature.
  187. 8:18Every every software has holes. Every
  188. 8:19software is buggy. Only nowadays agents
  189. 8:22are so good and so fast that we can't
  190. 8:24really accept this anymore. And that's
  191. 8:26that that's also a huge change, right? I
  192. 8:29mean, we have this uh world where before
  193. 8:32a next model is released, it's sort of
  194. 8:34kept under wraps for weeks and some
  195. 8:36privileged partners are being told to
  196. 8:39test their security, others are not. I'm
  197. 8:42sure government services know this as
  198. 8:44well and exploit this to find zero days
  199. 8:46and things like that. So basically we
  200. 8:48have a a a a a
  201. 8:51system where for some weeks we have
  202. 8:54privileged access to these models to do
  203. 8:56whatever and then it gets released and
  204. 8:58sort of we have the next wave of of uh
  205. 9:01of attacks and possibilities. So what
  206. 9:04can we do there? It's really uh we don't
  207. 9:07really have much uh uh readymade
  208. 9:10recipes.
  209. 9:11So because the current defenses that we
  210. 9:14have they all fall short. So what we
  211. 9:17have is we can have allow lists. So
  212. 9:19block lists for instance there's a um
  213. 9:22AMD um uh Amazon setter language
  214. 9:27uh that that does that. So we have some
  215. 9:29regex rules what should be allowed and
  216. 9:31what should not be but they're very
  217. 9:32coarse grain and they're typically they
  218. 9:34have have this problem that they either
  219. 9:36too permissive or too restrictive and
  220. 9:39you don't find a good middle. Uh we can
  221. 9:41ask the users for confirmations. I I
  222. 9:44guess we all have have seen that coding
  223. 9:46agents where we say click click click
  224. 9:48click accept accept accept until
  225. 9:50nowadays they don't ask anymore because
  226. 9:52it's too it's too uh inconvenient so
  227. 9:55there's user fatigue that can't scale
  228. 9:58sandboxing yes we can sandbox we can put
  229. 10:00agents in sandboxes that very much
  230. 10:02restrict what they can do but uh the
  231. 10:05problem is that this is also very coar
  232. 10:07grain because it gives you an absolute
  233. 10:09set of permissions which is either
  234. 10:11typically also too loose loose or too
  235. 10:14strict. So too strict that you can't do
  236. 10:15anything that you want to do and too or
  237. 10:18too loose that you permit uh dangerous
  238. 10:20behavior. So one uh possibility for
  239. 10:23instance you see here on the right uh
  240. 10:25where we see an agent reads the secret
  241. 10:28in step one and it exfiltrates the
  242. 10:30secret in step five and each step looks
  243. 10:33fine in isolation but together we have
  244. 10:37essentially a secret that got leaked
  245. 10:38which is essentially the behavior that
  246. 10:40we wanted to prevent. Now we could have
  247. 10:42a sandbox that says you can't read any
  248. 10:44secrets and that you can do but there's
  249. 10:47a lot of behavior that a lot of uh
  250. 10:50functionality where you need to access
  251. 10:52private data. So that's very
  252. 10:54restrictive. We could also have a
  253. 10:55sandbox that says you can't send
  254. 10:58anything to the outside world. But again
  255. 11:00that's that's a big problem. Uh so what
  256. 11:03we need is essentially a a defense that
  257. 11:06tracks the information flow. So this
  258. 11:08thing goes in here. It's labeled secret.
  259. 11:11you can't leak this piece of data, you
  260. 11:13can leak others. So we had for instance
  261. 11:15this famous thing where Entropic
  262. 11:16released the source code of the of their
  263. 11:19next version of of cloud code and that
  264. 11:22was one of these things. So essentially
  265. 11:25uh uh of course you want to publish some
  266. 11:27data
  267. 11:29companies publish uh repos all the time
  268. 11:32but you shouldn't publish a private repo
  269. 11:34but that's again a thing of information
  270. 11:36flow. Okay, so that's the current
  271. 11:40situation and it one thing that's
  272. 11:43particularly problematic is here what
  273. 11:45that's been called the lethal trifecta
  274. 11:48where you say if you combine all these
  275. 11:50three things then you're in trouble. Uh
  276. 11:53so um uh if you have access to private
  277. 11:56data, the ability to externally
  278. 11:58communicate and exposure to untrusted
  279. 12:01comments uh content, then essentially
  280. 12:04any website that the agent accesses can
  281. 12:08trick it into uh essentially uh leaking
  282. 12:11secrets uh accessing private data,
  283. 12:13sending it to the attacker. Uh so you
  284. 12:16say, okay, so don't do all these three
  285. 12:18things together because two or two of
  286. 12:20them at a time are fine. Uh but that's
  287. 12:22again that's that's difficult because
  288. 12:24essentially agents read websites um
  289. 12:27emails whatever and that's untrusted. Uh
  290. 12:31you don't know what's in there. Somebody
  291. 12:32might plant a prompt injection attack.
  292. 12:35Uh and agents do access private data as
  293. 12:38I said and agents are able to externally
  294. 12:42communicate. So taking one of these
  295. 12:43things away is very difficult.
  296. 12:47So our key idea in this that we have
  297. 12:49been working uh essentially all of this
  298. 12:51year so far uh is that we don't try to
  299. 12:54make the model trustworthy. So we don't
  300. 12:57work on alignment or any of these
  301. 12:59things. We try to make the medium safe
  302. 13:01and uh that means that instead of having
  303. 13:04an agent generating little snippets of
  304. 13:06code that talls tools that affect the
  305. 13:09world and hoping that the model won't
  306. 13:12misbehave. what we say well what if the
  307. 13:14code was typed in a certain type system
  308. 13:17that I'm describing that in a couple
  309. 13:20slides and then checked by the compiler
  310. 13:23and only when it passes the compiler
  311. 13:26sent to the world and the compiler would
  312. 13:28reject unsafe code before it runs. So
  313. 13:30that's the approach that we're
  314. 13:32proposing. So how does that work
  315. 13:34exactly? Well, the agent would express
  316. 13:37their intention in a capability safe
  317. 13:39language which in this case is Scala 3
  318. 13:42with capability or capture checking
  319. 13:43extensions. The type system would track
  320. 13:46what a piece of code is allowed to do
  321. 13:49and the compiler would enforce the
  322. 13:51safety not the model. So that's the
  323. 13:53approach we are pursuing for containing
  324. 13:57agents.
  325. 14:00Um
  326. 14:02so um I can visualize that with a
  327. 14:05scenario to say what well what are these
  328. 14:07capabilities? So let's say you're back
  329. 14:09in the middle ages and you have a secret
  330. 14:11diary and um now you have uh want to
  331. 14:17redecorate your room and have builders
  332. 14:19in there. So how do we prevent the
  333. 14:21builders from reading and leaking the
  334. 14:24diary? And let's say we can't remove it
  335. 14:27from the room before the builders enter.
  336. 14:30So we all know what to do. Of course,
  337. 14:32oops, sorry. Um, we can essentially
  338. 14:36secure the diary with a lock. So we have
  339. 14:39a lock uh uh that now sorry I this this
  340. 14:43thing is very sensitive. Okay. So we can
  341. 14:46protect the secret with a capability
  342. 14:48that gives it permission to read. So we
  343. 14:51have a lock in here and a key and the
  344. 14:53key that's the capability and you can
  345. 14:55open the diary only if you have the key
  346. 14:57the capability and we can take the key
  347. 14:59away
  348. 15:01when we let the builders in. So the
  349. 15:03builders don't have access but uh now
  350. 15:07let's do a harder case. Uh let's say we
  351. 15:10want the agents to work with the secret.
  352. 15:13So here I know we are in the middle
  353. 15:14ages. This is not exactly realistic. But
  354. 15:16let's say we want to uh instruct the
  355. 15:18agent to add photos to the diary from
  356. 15:21the photo album. Uh and the agent will
  357. 15:24rightly say, "Yeah, madam uh you have to
  358. 15:26give me the key. I can't access your
  359. 15:28secrets your your secret diary without
  360. 15:30the key. So I need the key." Yeah. Yeah.
  361. 15:32That's a problem. But if you get give
  362. 15:35the agent the key, how do we prevent it
  363. 15:37from leaking? Well, we can tell it tell
  364. 15:39the agents never reveal what we what you
  365. 15:42see. and the agent might swear no I will
  366. 15:44certainly not do that but you see it has
  367. 15:46a somewhat insincere face this thing so
  368. 15:49I'm not sure we can trust the agents in
  369. 15:50this thing so what do we do I think we
  370. 15:54need some more advanced technology here
  371. 15:57so uh what we need is uh somebody from
  372. 16:01from the future um uh I don't know
  373. 16:04whether you recognize this guy um so
  374. 16:07that's from the film severance severance
  375. 16:09was this film where we had two brains
  376. 16:11where Essentially in a high security
  377. 16:13company if you want to work for them you
  378. 16:15have to undergo a procedure that splits
  379. 16:18your brain into a private brain and a
  380. 16:20company brain and the brain two brains
  381. 16:22can't talk to each other. That sounds
  382. 16:24very bloody and I'm not proposing that.
  383. 16:26I'm proposing we do that for the for for
  384. 16:28the models for the LLM. So what if you
  385. 16:31had two LLMs, two models, one would for
  386. 16:34interacting with the world and the other
  387. 16:36for accessing private data and the two
  388. 16:39models couldn't talk to each other in
  389. 16:41ways we need to define and that means
  390. 16:44that when we access private data we
  391. 16:46can't leak it to the world because the
  392. 16:47two models the the two brains are
  393. 16:50separate. Okay, you say good uh that's
  394. 16:52mechanism. So I have invented or it
  395. 16:55wasn't me inventing it. It's called the
  396. 16:57dual LLM model. uh it's been around for
  397. 17:002 years or so. Um so that's a mechanism
  398. 17:03but how do we formulate policy so that
  399. 17:06we can reason about it. So essentially
  400. 17:08now we need to instruct this thing
  401. 17:10somehow. We need to write a process a
  402. 17:12program that does these things. How do
  403. 17:14we do that? And there it turned out that
  404. 17:18uh track capabilities are the key
  405. 17:20concept that can make this work. So
  406. 17:23capabilities are a well-known security
  407. 17:26constructor. Capability is essentially
  408. 17:28an unforgeable token or a key that gives
  409. 17:30you the right to do something and you
  410. 17:33can't create these things yourself. Uh
  411. 17:36except if you own the own the have own
  412. 17:38the resource that you want to pass the
  413. 17:40capabilities to but you can't do it for
  414. 17:42others. Uh somebody has to give it to
  415. 17:44you and if you hold it then you can do
  416. 17:46it and you can also pass it on. And the
  417. 17:49new idea is that we uh want to track
  418. 17:51capabilities in types. So what that
  419. 17:54means is that any object any value keeps
  420. 17:58in its type the capabilities it holds.
  421. 18:00So an object has methods and you can
  422. 18:04call the methods. You can pass
  423. 18:05capability to the method. Uh that
  424. 18:07doesn't count that's you got the you got
  425. 18:09it from somewhere else. But the object
  426. 18:11can also hold intrinsically some
  427. 18:12capabilities and those have to be
  428. 18:14declared in the type. So on the right
  429. 18:17you see some examples. Uh the top line
  430. 18:20here you have pure types. So string is a
  431. 18:23pure type that's that doesn't hold any
  432. 18:25capability that's just data or and we
  433. 18:27also have a function types. So functions
  434. 18:30that don't hold any capabilities those
  435. 18:32we write with a single arrow int to int.
  436. 18:35Uh then we can add a capability for
  437. 18:37instance here we say we want to we
  438. 18:40express things that contain a track
  439. 18:42capability IO. So that can do IO. So on
  440. 18:46the left you have a file that can do IO.
  441. 18:48When you call write or read or something
  442. 18:50like that it will do IO. It has this
  443. 18:52capability. On [snorts] the right hand
  444. 18:54side you have a function that can do IO.
  445. 18:57Of course functions can contain I don't
  446. 18:59know a print on or a write and I can do
  447. 19:02that as well. And then we have a thing
  448. 19:04where we say I don't care what
  449. 19:05capabilities you hold everything goes.
  450. 19:08And for that we express with just the
  451. 19:09hat. So file hat means a file that has
  452. 19:12arbitrary capabilities and the uh int
  453. 19:15double arrow int means a function that
  454. 19:17can have arbitrary capabilities and
  455. 19:20having arbitrary capabilities means you
  456. 19:22have arbitrary effects because effects
  457. 19:24are controlled by capabilities and of
  458. 19:27course this double function is the
  459. 19:28function arrow we all know from scala uh
  460. 19:31whereas the top one the single arrow is
  461. 19:34new because that's now we're talking
  462. 19:35about purity and scala previously did
  463. 19:38not so every function could do anything.
  464. 19:40So that's a double arrow. So you could
  465. 19:42say, well, it's quite uh cereus
  466. 19:48that we uh kept the single arrow for
  467. 19:51purity. So we kept it sort of so
  468. 19:53reserved it until we could be pure and
  469. 19:56uh the double arrow essentially doesn't
  470. 19:59is what it always is. It can have the
  471. 20:02effects.
  472. 20:04Okay. So how does that help? So let's
  473. 20:06look at the problem of uh leaking
  474. 20:09capabilities. Oops. Uh sorry weird.
  475. 20:13Okay. So no.
  476. 20:16Okay. Okay. Good. I have to really touch
  477. 20:18this uh very light softly. So um the way
  478. 20:23we would do that is that we say if we
  479. 20:25have data that are private then we wrap
  480. 20:28it in a classified. We say uh let's say
  481. 20:31the data is just a string a message. uh
  482. 20:34a normal string is public. You can do
  483. 20:36whatever you want. A private string is
  484. 20:38classified of string. So that's that's
  485. 20:40the type of it. So we wrap it in a
  486. 20:42classified wrapper. And there's a
  487. 20:45restricted protocol. What we can do with
  488. 20:47classified things. So the first thing we
  489. 20:50can do is we can reveal it. So reveal
  490. 20:53essentially goes from classified T to T.
  491. 20:55T is arbitrary. Um but you need a
  492. 20:58capability to do so. So that's this can
  493. 21:00access t would be the capability to
  494. 21:02access uh data of type t and uh if you
  495. 21:07don't have that capability you can't
  496. 21:08call reveal and the agent wouldn't get
  497. 21:11that capability. So we wouldn't give
  498. 21:13this key to the agent. So that's what
  499. 21:15the agent cannot do. So what can the
  500. 21:17agent do with classified data? Well it
  501. 21:19can talk uh call the next two functions
  502. 21:22map and aggregate. So um what the agent
  503. 21:27can do is it can transform classified
  504. 21:29data but in the trans the transforming
  505. 21:32function must be a pure function and
  506. 21:34[snorts] it also can aggregate multiple
  507. 21:37classified sources into a new classified
  508. 21:39source.
  509. 21:40So the trick here is that here we demand
  510. 21:43a single arrow so that we say while you
  511. 21:46transform you can't have any
  512. 21:48capabilities and then we complement that
  513. 21:50with a system that we say to do anything
  514. 21:53side effecty you need to have a
  515. 21:55capability. So that means the what the
  516. 21:56agent cannot do in this function is um
  517. 22:00send it to the network or print it or
  518. 22:04store it in a variable. All these things
  519. 22:06are controlled by capabilities and
  520. 22:08therefore we with with that system we
  521. 22:11make sure that once the agent is done
  522. 22:13transforming it cannot retain any
  523. 22:16knowledge of what was in the secret to
  524. 22:18to be to to to be able to use that
  525. 22:21later. But I should correct myself that
  526. 22:23there's super sneaky ways which we don't
  527. 22:25currently don't control to do that
  528. 22:27knowledge which are timing attacks. the
  529. 22:29agent could essentially reveal a single
  530. 22:31bit by saying I take a 100 seconds or a
  531. 22:34single second in transforming this
  532. 22:36function. So these are essentially sort
  533. 22:38of next level difficulties in security
  534. 22:41and we our system currently doesn't
  535. 22:43handle these but anything else uh is is
  536. 22:46essentially uh secured.
  537. 22:49Okay. And it turns out so this was a
  538. 22:51very simple thing a sing a single
  539. 22:53classified uh uh level uh you can also
  540. 22:56have multiple levels you can also deal
  541. 22:59with untrusted user input in a in a
  542. 23:01similar way to avoid indirect prompt
  543. 23:03injections. Uh so it actually
  544. 23:05generalizes to most information flow
  545. 23:08problems. It's quite powerful. Then your
  546. 23:10types will be a little bit more
  547. 23:11complicated to deal with multiple levels
  548. 23:13and so on but it's the same principle.
  549. 23:17So the [clears throat] way we do this in
  550. 23:19the the the we wrap this for the agent
  551. 23:23then is to say well we essentially we uh
  552. 23:26provide um to the agent two functions to
  553. 23:29invoke the model. uh there's a chat uh
  554. 23:31that essentially gives you a prompt and
  555. 23:34um a payload, a message and you pass
  556. 23:37both to a model and you get back the the
  557. 23:39result which in this case since we're
  558. 23:41talking about models are just strings
  559. 23:43and that could go to the cloud to the
  560. 23:45public model and then there would be
  561. 23:46another chat method which would have a
  562. 23:48classified message and would give you
  563. 23:51back a classified string and that would
  564. 23:53essentially go to the local LLM and be a
  565. 23:56pure function. And uh so what we have
  566. 23:59here uh uh at the at at the bottom you
  567. 24:03see here we can essentially um chat uh
  568. 24:07the the doc with the with this with with
  569. 24:10uh we we can use the chat function in
  570. 24:13this agent generated code which in this
  571. 24:15case because it would be a classified
  572. 24:17thing would go to the second chat and uh
  573. 24:20could talk only to the local trusted
  574. 24:23LLM. So no capabilities flow through
  575. 24:26this secure chat. So we can't have
  576. 24:28xfiltration of capabilities either. So
  577. 24:31we did that uh we implemented this in an
  578. 24:35agent harness called tacet tracked agent
  579. 24:38capabilities in types. So here's sort of
  580. 24:40the diagram how it looks. So we have uh
  581. 24:43the LLM agent. The LLM agent produces
  582. 24:47Scala code snippets, calls an MCP server
  583. 24:50which takes the code snippets and
  584. 24:51executes them. And then uh each code
  585. 24:54snippet is written in capture checked
  586. 24:56scala. So the first it needs to validate
  587. 24:59that with the compiler and then it gets
  588. 25:01executed with the ripple. And both the
  589. 25:03compiler and the ripple uh use a
  590. 25:06capability enabled library uh which for
  591. 25:09the compiler is there to check the
  592. 25:11capabilities of the functions that get
  593. 25:13called and for the ripple to execute
  594. 25:15them. Uh so it's uh public on t on on
  595. 25:19github. You see the URL on the right and
  596. 25:22it's been described in a paper called
  597. 25:24securing agents with track capabilities
  598. 25:26by the authors that you see here. Yaoyu
  599. 25:28is here, Oliver is here, Yen and we are
  600. 25:31not. Uh and that was in the first ACM
  601. 25:35conference on AI and agentic systems in
  602. 25:38May in San Jose. I got actually got the
  603. 25:41best paper award at this at this
  604. 25:43conference. So it seems that the agent
  605. 25:46community really valued this this
  606. 25:48contribution.
  607. 25:50So the setup that we had is essentially
  608. 25:52agents use code snippets to invoke the
  609. 25:54tools. Tools are encapsulated in type
  610. 25:57safe and safe interfaces. The interfaces
  611. 26:00are written in Scala 3 and so are the
  612. 26:02tool implementations currently but they
  613. 26:05could be also in another language that
  614. 26:06interops with Scala 3 like Java or uh
  615. 26:10JavaScript or a Vasam component because
  616. 26:13Scala 3 targets all these platforms. And
  617. 26:15[snorts] what's also interesting, we're
  618. 26:17currently exploring that is to say we
  619. 26:19can actually pair this with a sandbox.
  620. 26:21So what we could do is we have these
  621. 26:24type- safe interfaces in Scala 3. uh and
  622. 26:27they could be implemented with the
  623. 26:28languages you see at the top. But we
  624. 26:30could also imagine doing sandboxes that
  625. 26:33have essentially a type- safe um
  626. 26:36protocol, a type- safe interface, but
  627. 26:38then they let you implement uh this
  628. 26:41thing in whatever language unsafe
  629. 26:43languages like C, C++ uh uh and the
  630. 26:47enforcement would be on the runtime
  631. 26:49level. So the sandbox would enforce the
  632. 26:51capabilities, but you don't the code
  633. 26:53could be anything. So that's essentially
  634. 26:55an interesting trade-off where we
  635. 26:57essentially use types to describe a
  636. 26:59policy and runtime [snorts] enforcement
  637. 27:01in a sand in in in a sandbox to to to
  638. 27:05enforce it. Okay. So what we need for
  639. 27:08all this is u a thing called safe mode
  640. 27:11because while full scala 3 also has
  641. 27:14unsafe features that essentially
  642. 27:16undermine our types uh and safe mode uh
  643. 27:20disables them for agent code. So when
  644. 27:22the agent does does that we say the
  645. 27:24agent can't do let's say a type cast
  646. 27:26that pretends there is no capability
  647. 27:28retained when there are some
  648. 27:30capabilities or things like that I mean
  649. 27:31we can sneakily do many things but
  650. 27:34agents can't do that because we take
  651. 27:36that away we have a language import
  652. 27:38language experimental safe and uh when
  653. 27:41we essentially run the agent with that
  654. 27:43environment then uh it can't do any of
  655. 27:46the things you see here. Oh no, no
  656. 27:49unchecked type C casts, no unchecked
  657. 27:52annotations, no uh usage of the caps
  658. 27:55unsafe module, no runtime reflection,
  659. 27:58capture checking enforced uh and um
  660. 28:02globals uh in the using library code
  661. 28:06only when it's trusted. So the agent
  662. 28:08code will be in safe mode but then the
  663. 28:10you say well eventually the agent has to
  664. 28:13call something and eventually that even
  665. 28:15has to call Java because on the JVM we
  666. 28:17run on that right. So the way we do that
  667. 28:19is that we can have an annotation on
  668. 28:22classes called assume safe and those
  669. 28:24[snorts] must be audited by humans or by
  670. 28:28well maybe also agents but they must be
  671. 28:30audited that they actually do uh enforce
  672. 28:34the contract given by their capabilities
  673. 28:36in their code. So that's it's a bit
  674. 28:38little bit like rust unsafe where you
  675. 28:40say you have something that essentially
  676. 28:42is outside the type system. Uh so you
  677. 28:45have the the the formal types in the
  678. 28:47interface but you have to audit the
  679. 28:49implementations manually that they
  680. 28:51actually do that.
  681. 28:54Okay. So what we did we did some
  682. 28:56experiments uh is this actually safe?
  683. 28:58Does it prevent unsafe agent behaviors
  684. 29:01even under adversarial conditions? And
  685. 29:04uh the second one is is it uh uh
  686. 29:07expressive enough? Is it not too
  687. 29:09constraining? Can agents generate this
  688. 29:11code as effectively as conventional tool
  689. 29:14calling or does safety cost performance?
  690. 29:16And uh I can give you the exact uh
  691. 29:19numbers we had but that would take too
  692. 29:21much time. So uh the the the the upshot
  693. 29:24is yes uh the the basically we have yes
  694. 29:27to both questions. uh that was validated
  695. 29:30by essentially running standard standard
  696. 29:32benchmarks agent dojo software
  697. 29:34engineering bench and to square.
  698. 29:38So some examples of maybe what happens
  699. 29:41here is so we have an agent that does
  700. 29:43this uh this first one readcl classified
  701. 29:46and uh and then it uh it uses map on
  702. 29:50classified data and it tries to
  703. 29:52exfiltrate the key uh with a print on
  704. 29:55then uh the compiler will say uh I
  705. 29:57needed a pure function string to unit on
  706. 29:59the right but you gave me a string IO
  707. 30:01unit that won't that won't compile that
  708. 30:03that won't be uh uh conform that uh to
  709. 30:07each other and in number two you have a
  710. 30:11request file system and then essentially
  711. 30:14uh while you run the file system you
  712. 30:16return a bunch of files. So the contract
  713. 30:19for request file system was it should be
  714. 30:21a local operation afterwards you
  715. 30:22shouldn't uh retain any link to the file
  716. 30:26system and there you would get an error
  717. 30:28that the capability outlives its scope.
  718. 30:30So these are actual examples
  719. 30:33from actual agent benchmark runs in the
  720. 30:36tests.
  721. 30:38Okay, good. So I talked about agents a
  722. 30:40lot but what about other software
  723. 30:43systems big software systems? Can the
  724. 30:46same techniques work for both? And the
  725. 30:49first observation is well actually there
  726. 30:51are similarities. So in both cases we
  727. 30:54want to build trust in the systems
  728. 30:57correct functioning without reviewing
  729. 30:59every line of code. For agents we
  730. 31:01typically review no lines of code. Maybe
  731. 31:04for software systems we are prepared to
  732. 31:06review some lines of code but certainly
  733. 31:07not every line of code. And in both
  734. 31:10cases strong types can help because they
  735. 31:13constrain possible behaviors without
  736. 31:15having to look at the implementations.
  737. 31:19But there are also differences to be
  738. 31:20sure. So the difference are in scale and
  739. 31:23interactions. So agents are typically
  740. 31:26very short-lived uh seconds, minutes,
  741. 31:28maybe some hours. That's the max.
  742. 31:31Programs are maintained for decades and
  743. 31:33uh they will continue to be maintained
  744. 31:35for decades because there's so much
  745. 31:37implicit knowledge in a program that you
  746. 31:40don't want to throw it out and and
  747. 31:42rewrite it uh on a whim. Um furthermore,
  748. 31:45agent tools are expressed in a handful
  749. 31:47of code lines. maybe a single line a
  750. 31:50single one one to 10 let's say and
  751. 31:52programs of course have millions of
  752. 31:54lines of of code or can have millions of
  753. 31:56lines of code and finally agent code is
  754. 31:59fully opaque typically we don't see what
  755. 32:01what code gets generated whereas
  756. 32:03programs can be partially reviewed so
  757. 32:06the central challenge we have then is
  758. 32:09how can we better understand and shape
  759. 32:12programs that are largely written by
  760. 32:15agents I think that's the question we
  761. 32:17need to answer and there are some
  762. 32:19related questions. One is uh developer
  763. 32:22engagement. So there's this uh quote
  764. 32:24from Kent Beck or the the post from Kent
  765. 32:29Beck that he had a conversation and uh
  766. 32:32the uh uh the statement was the agent
  767. 32:36world he says feels more like reviewing
  768. 32:38an air traffic controller interrupt
  769. 32:41driven whereas before we also had this
  770. 32:45uh experience of being in the flow deep
  771. 32:47immersions all your faculty on a narrow
  772. 32:50thing so that's gone and now we're air
  773. 32:52traffic controllers to essentially keep
  774. 32:54a bunch of balls or agents and things
  775. 32:57which is much less engaging and much
  776. 32:59less fulfilling. I think that's a loss
  777. 33:01emotionally and I think when god got
  778. 33:03software gets fiddly enough a loss
  779. 33:04practicality practically too. So how can
  780. 33:08we avoid that and how can we avoid that
  781. 33:11because I'm a programming languages guy
  782. 33:12using tweaking programming languages. Do
  783. 33:16programming languages have to change for
  784. 33:18that? And [snorts]
  785. 33:19uh I think my answer is I don't I don't
  786. 33:22really know. I have some ideas. I'm just
  787. 33:24throwing things out now to say let's
  788. 33:26experiment with things because this is
  789. 33:28really a time where everything changes.
  790. 33:29And it would be a miracle if we say
  791. 33:33everything changes, but programming
  792. 33:34languages don't. Programming languages
  793. 33:36stay the same. They're somehow
  794. 33:37unaffected by this change. I think
  795. 33:39that's highly unlikely. So the question
  796. 33:42is in what way will will they change?
  797. 33:44And right now I think the honest answer
  798. 33:46is we don't know yet. But we should find
  799. 33:49ways and and find and see maybe uh ways
  800. 33:53how they could change and what is
  801. 33:55likely. So I think the first thing and
  802. 33:58I' already talked about is strong types
  803. 34:00absolutely because they can constrain
  804. 34:02behavior without reviewing
  805. 34:04implementation code. We absolutely want
  806. 34:06them. And then I have essentially three
  807. 34:08very tentative ideas that I'm just
  808. 34:11review flashing to you very quickly and
  809. 34:14it would be great to get your feedback
  810. 34:16afterwards what you think whether
  811. 34:18something like that would work whether
  812. 34:19you think other things would work. I
  813. 34:21think this is mostly to open the
  814. 34:23discussion what what can we do to keep
  815. 34:25controllers in programmers in control
  816. 34:27for this software. So the the three
  817. 34:29things I had was spec strings you know
  818. 34:32which uh essentially put prompts in the
  819. 34:34program required interfaces which
  820. 34:36constrain what you depend on not just
  821. 34:38what you provide and refinement types to
  822. 34:41use formal verification when it makes
  823. 34:43sense. So let's look at spec streams
  824. 34:45first. So I on the top you see the
  825. 34:49workflow that I showed you initially uh
  826. 34:51where the program is a black box. So
  827. 34:54what I what I want to do instead is I
  828. 34:56want to put the prompts
  829. 34:59into the program. So I want to say a
  830. 35:02program contains starts with a prompt
  831. 35:04and the prompt stays in the program.
  832. 35:06it's the same file or the same set of
  833. 35:08files and then the AI would essentially
  834. 35:12augment [snorts]
  835. 35:13enlarge the program to give you an
  836. 35:15implementation of the prompt but the
  837. 35:17prompt would stay in the program and
  838. 35:18then in instead when when you have a
  839. 35:20change request then that should lead to
  840. 35:22a change of the spec in the program and
  841. 35:25the AI would on a change of spec uh
  842. 35:28update the program to do that. So the
  843. 35:30AI's job is basically to sync the
  844. 35:33prompts which are largely written by
  845. 35:35humans in the program with the code. So
  846. 35:39um I can show you what that looks like.
  847. 35:40So we have that we have sort of this
  848. 35:43it's already started to the the
  849. 35:45implementation uh in the compiler sense
  850. 35:47but tooling is still not quite complete
  851. 35:50yet. So let's say you have uh you want a
  852. 35:53function to parse states. I know this is
  853. 35:56ridiculously small, but I don't want to
  854. 35:58want to show you something much larger
  855. 36:00because it wouldn't fit on a screen,
  856. 36:01right? So, let's do something very
  857. 36:03small. We say we need a par a date
  858. 36:05parser. It should take a string. Return
  859. 36:07an option of date. Uh we give it a
  860. 36:09prompt. Uh and then uh what uh what we
  861. 36:12would do uh is we would essentially have
  862. 36:15a sync command to say now agent take
  863. 36:18over complete the program according to
  864. 36:20the prompts. and uh the agent would
  865. 36:22generate a an implementation of our date
  866. 36:25parsing thing. If you if you don't like
  867. 36:27that, we can go back to the to to the
  868. 36:29spec and say uh parse a string into a
  869. 36:32date structure using this and this
  870. 36:33format for instance and then the program
  871. 36:36would be updated. But the point is is
  872. 36:38this the the prompt stays in the program
  873. 36:41and if you want to review the program
  874. 36:43you can also just see the prompt and
  875. 36:45says okay parse state par pass parses a
  876. 36:47date string into a date structure that's
  877. 36:49like a document right so good to know
  878. 36:52and I trust the agent that it will
  879. 36:53probably do that typically I don't need
  880. 36:55to need to uh uh to to check that
  881. 36:59because I have the types would give me
  882. 37:01an envelope of what the agent can do so
  883. 37:04if I say in if if in this environment
  884. 37:06there are sensitive capabilities
  885. 37:08available. Then I say yeah obviously I
  886. 37:11can be happy to say just do this. I run
  887. 37:14the tests. I look at yeah is does it
  888. 37:16make sense and I say nothing bad can
  889. 37:18happen because of the types and the
  890. 37:20functionality.
  891. 37:22I I trust the agent to have the right
  892. 37:24one. Okay. So but then the next thing is
  893. 37:28to when we that we say okay so that's
  894. 37:30sort of functionality. The other is
  895. 37:32safety. So how do we make sure that the
  896. 37:35agent generated code doesn't do
  897. 37:37something bad while it gets implemented
  898. 37:40some private secret leakage or hacking
  899. 37:43systems or some some something else. And
  900. 37:46the problem here is uh that we don't
  901. 37:48typically control all our all our code.
  902. 37:51Well with agents we don't even control
  903. 37:53our local code but every program uses
  904. 37:55external libraries and we don't control
  905. 37:57them. So that's the field of supply
  906. 37:59chain attacks of course that somebody uh
  907. 38:02malicious can change a library we depend
  908. 38:05on and traps us into releasing a secret
  909. 38:08or things like that. Uh so chain supply
  910. 38:12chain attacks change libraries in
  911. 38:13malicious ways and agent generated code
  912. 38:16using these libraries might hand over
  913. 38:19secrets or cause unwanted effects. So
  914. 38:21again I have a sort of a super inner
  915. 38:24nutshell thing. Let's say we have the
  916. 38:26first version of the library, a JSON
  917. 38:28parser. Uh, and that's okay. It takes a
  918. 38:30string and returns JSON. And I'm already
  919. 38:33having a high safety protocol. So I use
  920. 38:36the the global capabilities feature in
  921. 38:39capture checking to say every object has
  922. 38:41to declare the global capabilities it
  923. 38:44uses with this uses clause here. So the
  924. 38:46JSON parser uses console for whatever
  925. 38:49reason, but it doesn't use anything
  926. 38:50else. So it can can have access to
  927. 38:53console, but nothing else. Okay. So now
  928. 38:56we we get the supply chain attack. So
  929. 38:58console yeah I should say is a global
  930. 39:00capability. So I track it as a
  931. 39:02capability but it's a global object.
  932. 39:04Okay. But now the attacker changes that
  933. 39:07and says okay so my JSON parser now it
  934. 39:09uses the console and the network and
  935. 39:12furthermore my pass string has an
  936. 39:15implicit parameter that says give me the
  937. 39:17API key if you hold it. Right. So what
  938. 39:20will the compiler do? Normally it will
  939. 39:22say okay so new we recompile we say okay
  940. 39:26so we need the API key do we have the
  941. 39:28API key yes here it is let's pass it
  942. 39:30right and [snorts] uh so there we go we
  943. 39:34have a xfiltrated the API key because
  944. 39:37now the JSON parser can blast it on the
  945. 39:40network so what can we do to prevent
  946. 39:43that so one thing would be to say well
  947. 39:46yeah it's your fault scala uh with all
  948. 39:48these implicit parameters things so
  949. 39:50that's just too way too to too to too to
  950. 39:52too to too too dangerous. Let's pass all
  951. 39:54capabilities explicitly even so though
  952. 39:56it's going to be super cumbersome and
  953. 39:58inconvenient. Uh but agents will write
  954. 40:01this. So why implicit parameters? But
  955. 40:03that doesn't really solve the problem
  956. 40:05does it? Because we say the calling code
  957. 40:08will be generated by an agent. So if the
  958. 40:11new version of the library says well I
  959. 40:13need an API key as an implicit
  960. 40:15parameter, the agent will oblige, right?
  961. 40:17It will say yeah of course yeah here
  962. 40:19here it is. uh I don't ask questions the
  963. 40:21library needs API keys I pass them and
  964. 40:24no human again will review this thing so
  965. 40:26implicit or not implicit doesn't really
  966. 40:29change much here so what can we do well
  967. 40:32what we can do okay so here I say
  968. 40:34network gives excess authority to the
  969. 40:36JSON parser and the API key is an
  970. 40:39excfiltrated capability so what we can
  971. 40:41do is uh we can write this in our
  972. 40:45program that we say which I call a
  973. 40:48required interface is I can say what we
  974. 40:50need is a JSON parser that only uses
  975. 40:53console and that has this parse method.
  976. 40:56Uh so that's that's not an object we
  977. 40:59built in the program. That's describes
  978. 41:01an external object we have in a in a
  979. 41:04different library. That's why it has a
  980. 41:05required in front. So required says this
  981. 41:08is essentially something that describes
  982. 41:10something somewhere else. So this gives
  983. 41:12essentially a symmetry between the
  984. 41:14producer of a library and the consumer
  985. 41:17of a library. So both can define
  986. 41:19interfaces. The producer for what the
  987. 41:22library provides and the consumer for
  988. 41:24what we need from the library. So great
  989. 41:27idea, great security. You say, well, why
  990. 41:29wasn't this done before? Well, way too
  991. 41:31tedious, right? Requ writing all these
  992. 41:33required interfaces is quite quite
  993. 41:36tedious. But we have agents. So this is
  994. 41:38a typical thing that an agent should do.
  995. 41:41Should look at the calls in our program,
  996. 41:42should assemble these required
  997. 41:44interfaces. And that's probably the
  998. 41:46first thing I want to review as a human
  999. 41:48to say this program, what's its
  1000. 41:49footprint? What what does it call? What
  1001. 41:52capabilities does it does it essentially
  1002. 41:54uh the that does it depend on? Because
  1003. 41:57that gives me an envelope of essentially
  1004. 42:00the possible uh impact that this program
  1005. 42:02could have. So I think this is this is a
  1006. 42:04thing that is clearly something that
  1007. 42:06didn't exist before and that is now a
  1008. 42:08big help from in in the world where
  1009. 42:11agents write the code.
  1010. 42:15Okay. So the third one is uh specs and
  1011. 42:19proofs. So what we also have witnessed
  1012. 42:21in the last months is that models have
  1013. 42:24become amazing at proving both in math
  1014. 42:27and in program verification. It's just
  1015. 42:29incredible progress uh what these models
  1016. 42:31did and it's basically because they they
  1017. 42:34people have figured out how to do an
  1018. 42:36incremental verification of that is
  1019. 42:39embedded into proof search. So proof
  1020. 42:41search you you try lots of things and
  1021. 42:43you code them in lean uh or whatever and
  1022. 42:46uh you immediately get feedback of
  1023. 42:48whether this proof makes sense or not
  1024. 42:49and that essentially help helps you
  1025. 42:51shape the search. So and the result is
  1026. 42:53just amazing. So we see claims now that
  1027. 42:55all software will be formally verified.
  1028. 42:58Um and it would be nice if it was so
  1029. 43:01because if software is formally verified
  1030. 43:04then indeed nobody needs to look at the
  1031. 43:06program because we have a proof that the
  1032. 43:08program meets the spec. So then we're
  1033. 43:09back to compiler uh non-flaky compiler
  1034. 43:12territory, right? But the problem is
  1035. 43:15having getting a formal spec. What's the
  1036. 43:18spec of Microsoft Word is a famous
  1037. 43:20question. Uh so most things don't really
  1038. 43:22have a spec or if you try to come up
  1039. 43:24with a spec, it's it will be just as
  1040. 43:26hard as the program or harder. So that's
  1041. 43:29the problem with that. But I think
  1042. 43:31that's we shouldn't we shouldn't give up
  1043. 43:33all of these things. There's some things
  1044. 43:35that can very well be specified and we
  1045. 43:37should do that.
  1046. 43:39So we can do a lot more than the
  1047. 43:40state-of-the-art today and I'm not sure
  1048. 43:43yet exactly in how this what shape this
  1049. 43:47will take but one possibility is
  1050. 43:50refinement types on which we have been
  1051. 43:52working and you have a talk later in the
  1052. 43:54conference by Matt uh where uh what we
  1053. 43:58do is we pair a type with a predicate so
  1054. 44:01the first one we would have the type
  1055. 44:02that says positive doubles so it's an x
  1056. 44:05of type double where x is greater or
  1057. 44:07equal zero So this includes the thing
  1058. 44:09it's positive or the the second one
  1059. 44:12would be a sorted vector where we say
  1060. 44:14it's a vector that is sorted and that
  1061. 44:16can increase greatly the precision of
  1062. 44:18what's described by a type. So a square
  1063. 44:20root could could now say well I need a
  1064. 44:22positive double makes no sense for
  1065. 44:24negative. A binary search says well I
  1066. 44:26need a vector and it has to be sorted. I
  1067. 44:28require that or a get element method
  1068. 44:31could require. So here I write it out to
  1069. 44:34say I I I I I need an index which is
  1070. 44:37actually between zero and the length of
  1071. 44:39the of the sequence. So this makes sense
  1072. 44:42and this can be proven these things can
  1073. 44:44be proven. So how well if you have
  1074. 44:47subtyping of refinement types which
  1075. 44:49subtyping is the thing that happens all
  1076. 44:51the time in in Scala type checker right
  1077. 44:53then subtyping refinement types means
  1078. 44:55you need to prove a logical implication
  1079. 44:57that the predicate on the left implies
  1080. 44:58the predicate on the right and we can do
  1081. 45:01that with runtime checks or an SMT
  1082. 45:04solver. So that's the current state. But
  1083. 45:06I think what we really should look at is
  1084. 45:09since models have gotten so good, let
  1085. 45:11the model construct the proof of these
  1086. 45:13things and then we have it uh we can
  1087. 45:15essentially store it with the program
  1088. 45:16and we have it for reference. So that
  1089. 45:19has a a little bit of different
  1090. 45:21ergonomics than type checking. I guess
  1091. 45:24we if we do that we want to check
  1092. 45:25offline not doing regular type checking
  1093. 45:27because these proofs might take a long
  1094. 45:29time. uh if you fail to prove an
  1095. 45:31implication that shouldn't be fatal that
  1096. 45:33should be sort of like a red test right
  1097. 45:35so it's it's more like test coverage
  1098. 45:37that you say which of these implications
  1099. 45:39do I know will hold and which I have to
  1100. 45:42which are red and red me probably means
  1101. 45:44well I might need a need a manual
  1102. 45:46inspection for these things
  1103. 45:48so to summarize if we want to have
  1104. 45:52meaningful oversight we must continue to
  1105. 45:55shape and review our software systems
  1106. 45:58but it doesn't mean that we have to
  1107. 46:00review every line of code. So [snorts]
  1108. 46:02the challenge for programming languages
  1109. 46:04would be to organize code so that it
  1110. 46:07won't turn into a huge ba ball of mud
  1111. 46:09which programs often tend to do and
  1112. 46:12which is accelerated by the agents and
  1113. 46:15that essential aspects of behavior can
  1114. 46:17still be condensed and concise
  1115. 46:19signatures and in that they enable small
  1116. 46:22reviews that give high confidence and
  1117. 46:24correctness. And I think the right type,
  1118. 46:26the right elements for this are types,
  1119. 46:29capabilities, modules, specs, uh formal
  1120. 46:32verification. And with specs, I actually
  1121. 46:34mean both formal and informal specs. I
  1122. 46:37think we have to sort of see what
  1123. 46:39sticks, what works out. And I guess we
  1124. 46:41all will be in experimentation mode for
  1125. 46:43the next uh months and years to come.
  1126. 46:46Thank you.
  1127. 46:48[applause]
  1128. 46:53>> [applause]

About this transcript

This page contains the full transcript of Martin Odersky – Keynote: The End of Programming? #ScalaDays by Scala Days Conferences, generated from the public captions YouTube serves with the video. The transcript has 8,027 words across 1,128 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.