YT YouTube2Text

YouTube2Text → Transcripts → Guy Podjarny - Spec Driven Dev From Single Player to Multiplayer to Ecosystem | DevCon Fall 2025

Guy Podjarny - Spec Driven Dev From Single Player to Multiplayer to Ecosystem | DevCon Fall 2025 — Transcript

by AI Native Dev · 6,288 words · 876 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:09Now, our next speaker is the uh is the
0:12uh founder and CEO of a little company
0:15called Tessle. Uh 18-month old company.
0:18Um GIO previously is he's a serial
0:21founder. He he he founded Blaze, which
0:23was acquired. He founded Sneak, a
0:25massively successful developer security
0:28uh company and organization that
0:29actually when we think about it, a lot
0:31of the challenges that that had in and
0:33around moving the the thought process,
0:36moving the moving the mindset of people
0:38to a developer security path is actually
0:41very similar to how we're thinking today
0:43about A&E development. And now we're
0:46going to talk about how AI can be done
0:48collaboratively. So please give it up
0:50for our first keynote speaker, Guy
0:51Pjani.
0:53>> [applause]
0:56[music]
1:03>> Hello everyone. Uh, welcome to the first
1:06inerson and native DevCon. Uh, love in
1:08person events, you know, like I think
1:10you you you convey things virtually. I
1:11consume things virtually, but nothing
1:13beats, you know, talking to people and
1:14engaging with them. And I hope you're
1:15all feeling that way. So, thanks for
1:17shipping in, especially in the rain here
1:19in New York. and uh uh and I hope you
1:21you enjoy the event and I look forward
1:23to meeting and learning from many of
1:24you. Uh I am Gapari or I am the
1:27founder and CEO of TESL. Uh and I'm here
1:30to talk about specdriven development's
1:32growth uh from kind of single player to
1:35really how do you use it in a team and
1:37how do you think about ecosystems? Um
1:39let me set the stage first. So hopefully
1:43we're all aligned that AI is
1:44transforming software development. It's
1:46not a little hiccup. It's not a little
1:47change. It's something that is
1:48substantial that is happening and we've
1:51seen a bunch of evolution already. We've
1:53had two kind of waves of what I think of
1:55as AI augmented software development uh
1:58with tools that were pioneered by
2:00copilot and cursor that autocomplete
2:02things better you know that that helped
2:04a lot that build in chat and allow you
2:06to talk to your code and understand
2:08things that cursor pioneered and then we
2:10uh in this last sort of especially 6
2:12months maybe 12 we've seen the rise of
2:14more true AI native which I think of
2:17them as more AI native dev tools because
2:19they're more about delegation they're
2:21more uh ask the agent to do a thing for
2:24you and then interact with it. So, it's
2:26a it's a much more substantial change
2:28for us and there's a lot to learn. And
2:31while all of these tools are are
2:32valuable in their own right, they've all
2:34kind of consolidated towards this
2:36agentic behavior. Uh and I think it's
2:38pretty clear that that is the future.
2:40So, I'll focus on where is that taking
2:43us? You know, how do we make this
2:44agentic development work?
2:47So, the reality today is that agents are
2:50are super powerful, uh, but they're
2:52massively unreliable, right? They're
2:53sort of sometimes spooky, sometimes
2:54kooky. You know, they're they're uh
2:56they're they're amazing and they do
2:58these things that astound you, but then
2:59they just make these foolish mistakes.
3:01Um, they're also very untrustworthy.
3:03They'll tell you something is done when
3:05it isn't really. They will break things
3:06along the way as they, you know, you ask
3:08them to build a feature, they'll burn
3:09down the house to to get it done. Uh,
3:11and so they're very powerful, uh, but
3:14they're kind of hard to use. Um, and
3:17this creates this capability reliability
3:19gap, you know, and you see studies like
3:21this one from meter where they they
3:25assess, you know, how how much do
3:27developers think AI tools will make them
3:30better. And you see this is the bar at
3:32the left and people developers, well, I
3:34think it's going to make me faster. I'll
3:35complete something faster. And then they
3:36observe how long it actually takes them
3:38to do it. And it turns out it actually
3:40was slower. There's a lot of caveats, a
3:42lot of flaws in this study and in any of
3:44the others. It's a very complex area,
3:46but I think everybody relates to the
3:48fact that it's not easy to turn this
3:51power into improved productivity into
3:54something that actually works.
3:57And and so what do we do about it? Well,
4:00what we can't do about it is say, you
4:02know, bury your head in the sand like
4:04let's not use agents is tempting. And,
4:06you know, a good number of people,
4:08probably none in this room are uh are
4:10saying that. Uh but you know it's
4:13clearly shortsighted. I mean this is a
4:15technology that's too powerful that uh
4:18to to kind of ignore or think it would
4:20fade away. It's here. It's going to
4:22shape our development. And we need to
4:24think about how do we embrace it? Not
4:26how do we ignore it? And so the real
4:28question is how do we help agents? How
4:30do we help agents succeed and close this
4:33capability reliability gap?
4:36Let's look a little bit at a at a
4:38history of attempting to do so. You
4:39know, this is uh our quest for silver
4:41bullets that would make LLMs work uh the
4:44way we want, right? And a bit of a loose
4:46timeline. We start off with, you know,
4:48this is us as the industry thinking
4:50about fine-tuning. Hey, we're going to
4:51tell the LLM, here are like tens,
4:53hundreds, maybe thousands of examples of
4:56how I like to code, how you code in my
4:58environment on it, and the LLM's weights
5:00will magically adapt uh and work like
5:03that. And it turns out that this works
5:05very well for brand new skills, for
5:07cases in which you're teaching the LLM
5:09to do something that it didn't know
5:10before. But if it already has opinions,
5:12if it already knows how to code, it's
5:14very hard to sway it. It's kind of like
5:15people, you know, we if if can't teach
5:17an old dog new tricks, right? It's hard
5:19to sway someone will behave. And so
5:22fine-tuning is useful, but it's not a
5:24silver bullet. And in fact, in many
5:25cases, uh it is it is hard to to really
5:28have it shine.
5:31The second kind of wave of technology of
5:33this will solve our problems was was
5:35rag. It's like I will take my prompt and
5:38I will put it into this magical kind of
5:40embedding algorithms and vector
5:41databases will pull out the right
5:43context that I need everything precisely
5:45that I need to know to be able to
5:47address this problem. And again rag very
5:49powerful when you have your own keywords
5:51and terminology and things like that but
5:54uh context gathering is more complicated
5:56than that. uh there's there's a lot of
5:58subtlety often time things that are not
5:59in your prompt they're not in that
6:01initial path and so it remained a useful
6:03tool but was deemed not uh not a silver
6:06bullet
6:08um the next kind of this will save us
6:10all was the world of big context you
6:11know what we don't need to worry about
6:12all this stuff you know we've got like
6:14million token 2 million token context
6:16windows we'll just tell it everything uh
6:19and it will figure it out it'll just
6:20sort of know what to do and it turns out
6:22you know like I often get uh the like
6:25through my career I've gotten the often
6:26times the guidance that says if you have
6:2810 smart things to say and you say all
6:30of them each of them will get less
6:32attention than if you said three of them
6:34uh and and I still struggle to not say
6:37everything I think but uh it but it's
6:39true you know if you say fewer things
6:41each of them gets more attention and
6:42NLMs are the same they if you if you
6:45give it more information in its context
6:46each of these bits would get less
6:48attention it would be less focused and
6:50we'll show a little bit of that in a sec
6:53and so we get into modern world uh and
6:55we have these two additional tools. One
6:57is agentic search. You know what? Let's
7:00delegate the actual problem of context
7:02creation to the agent. Let's give it
7:04tools to find the information. That's
7:06what we do as humans. We go, we Google,
7:08we look at files, we uh we do that work.
7:10And that is useful, but again, it's
7:12useful when it's constrained. If you
7:15tell someone just just go find, you
7:17know, the information about anything,
7:19it'll take time to find it. It'll take
7:22uh you might find the wrong information.
7:24uh and you know it's a in it's both
7:26inefficient and unbounded and you can
7:28get down rabbit holes that really kind
7:30of make things worse and so it's a very
7:32very powerful tool. We'll refer to it
7:34back again but again it didn't just
7:35solve the problem because there's
7:37another piece that we need on top of
7:38that and that is context engineering and
7:40that's the sort of the state of affairs
7:42today which is you actually have to give
7:46up on the magic solutions and think
7:48about what do you want to tell the
7:50agent? what information does the agent
7:52need to succeed? And I love uh this is
7:55uh Toby, the uh founder and CEO of
7:57Shopify. He's a very serial guy, super
8:00smart guy, and he was on the podcast uh
8:02uh a couple of months ago. And I like
8:04his quotes here, which is I like the
8:05term context engineering because I think
8:07the fundamental skill of using AI well
8:10is to be able to, and this is the key
8:11word, a state of problem with enough
8:14context in such a way that without any
8:16additional pieces of information, the
8:18task is plausibly solvable. I love that
8:20it's this notion of like let it lean
8:23into intelligence but don't assume mind
8:25readading you know like tell it what it
8:27needs to know so that it can succeed.
8:31So context engineering in my mind and I
8:34might be biased is basically the same as
8:36specs. You know this is about we just
8:38had this long conversation about you
8:39know should we call something spec or
8:41context. uh it doesn't really matter
8:42like for me it is about getting out of
8:45your head and out of your knowledge
8:46getting the the nuggets of knowledge
8:48that you need to uh give the agent to
8:50succeed and so I'm going to use the term
8:52context and specs interchangeably
8:54throughout this talk that's the way I
8:55see the world
8:58so I'm going to go into that sort of
8:59single multi-ecosystem player but before
9:02that I want to remind us that you can't
9:04optimize what you can't measure this is
9:06a known phrase in the world of software
9:08development but really mostly you hear
9:10it about ops because in ops we are
9:13familiar with these systems that
9:14sometimes behave well and sometimes
9:16don't right the the servers sometimes
9:17work sometimes don't sometimes they
9:19crash and so we have to measure them so
9:20we know what's happening so we can
9:21optimize it now we have the same
9:23statistical behavior in our development
9:26you know we have these statistical
9:27creatures agents they build stuff
9:29sometimes they work sometimes they don't
9:31and that means two things one we need to
9:33leave the notion of agents can
9:35absolutely do this or cannot absolutely
9:38and embrace a statistical number around
9:40their success uccess rate and two is we
9:42have to measure that we have to think
9:43about how do we evaluate how do we
9:44assess success and I'll try to give some
9:46data to back some of my statements in
9:47this talk
9:50so now we'll actually get to the thing
9:51that is on the title which is let's talk
9:53about uh single player how do you do
9:55context engineering in a single player
9:57context so um just to demonstrate a
10:01little bit the the base this is a a um a
10:04little sample to-do application with a
10:06Jurassic theme that I worked very hard
10:08on uh to just sort of demonstrate
10:10And the base context that an agent has
10:13is your project. Here I'm asking the
10:16agent to add an edit button to each of
10:18the to-do items in my list. And you'll
10:20see the agent automatically go off and
10:22read files in the project. It doesn't
10:24read all of them. It's already doing a
10:26gentic search to load uh some of that
10:29content. Uh and then it would load that
10:32code, understand that code, and uh
10:36sorry, I'll just sort of pause here. uh
10:38and um uh it it can load loads that code
10:41and reads that into context. So, I
10:43didn't need to tell it what the to-do
10:44item is. I didn't need to tell it where
10:46to put that button. It had enough
10:48information in the code and this is the
10:50base context that's important. However,
10:53you'll notice for instance that it
10:54created this blue edit button and that
10:56doesn't fit my beautiful Jurassic theme.
10:58Uh and it turns out that in the code
11:01some information is not there. It
11:02doesn't know that I want to stick to my
11:05theme colors. And so let's tell it to do
11:08that. So we can add explicit context
11:11that doesn't rely on agentic search like
11:14an agent MD file that says always stick
11:17to the theme colors used British
11:19spelling and it will go off and I will
11:22ask the same action you know I will say
11:23add an edit button and it will go off.
11:25it'll do the same type of like dynamic
11:27loading and uh maybe I'll sort of spare
11:30us a moment here uh as we get to the end
11:33you will see that it indeed created uh
11:35the button with my beautiful theme
11:37colors uh and and so this is this is 101
11:40of context engineering at at the base
11:42your project's context your project's
11:44code is the agent's context how well
11:47documented it is whether it is confusing
11:48and has references to old data
11:50structures that you have uh how um uh
11:53reasonable are your file names all of
11:55Those things help the agent load the
11:57right files, but then there's going to
11:59be some information that's not in the
12:00code and you put in your explicit agents
12:02MD, claude MD, cursor rules, local
12:05context. You probably know much of this
12:07already. The challenge is this is nice
12:10when you want to tell it stick to the
12:12theme colors, but typically you have a
12:14lot more to say. You there's just a lot
12:16that you want to tell it uh about how to
12:18work in your environment, in your style.
12:21And as we talked about before, if you
12:24tell it 10 things, it's going to get,
12:26you know, give each of them less
12:27attention than if you told it three. Uh,
12:29and so this is uh a problem. Before I
12:32talk about how to solve it, let's maybe
12:34back that claim that this is a problem
12:36with some data. So to do that, we took a
12:40little uh game, the team took a little
12:42sort of dodge the blocks game uh that
12:44that was vivecoded and asked uh the
12:47asked an agent in this case Claude code.
12:49Actually we tried three agents say add a
12:51login to this like add sessionbacked
12:53authentication so this game will be
12:55behind the login. The other thing we did
12:56is we created again with agents uh an
12:59evaluation scorecard like how good is
13:01the security of this thing that you've
13:04just added just like a benchmark that
13:06are standards today for good rubrics and
13:08then we ran that in three different
13:10modes. One is we ran the agents without
13:13any context and we scored them. The
13:15second is we put into the agents MD or
13:17the equivalent uh the OASP the open web
13:19application security project uh guidance
13:22around authentication about 3 kilobytes
13:24of data and the second is the longer 20
13:27kilobytes of full security guidance that
13:29includes the 3 kilobytes there's no word
13:31that didn't exist in the second one and
13:34indeed as we ran this we saw and I'm
13:37just sort of you know uh this is no
13:39surprise that without any guidance the
13:43agent scored a 65 85% score and with the
13:47guidance with the precise guidance it
13:48got 85% score and if we told it more
13:51things it got a lower score it got 81.
13:54So this was good and it's already
13:56validation. We need to know that even
13:57though we expect this behavior now we
13:59have a bit more confidence and there are
14:01many examples of this but also we find
14:03information that is maybe a bit
14:04unexpected. For instance we ran this on
14:08three different agents. We ran it on
14:10claude on codeex and on cursor and what
14:13you can see is they all fared about the
14:16same mediocre results without any
14:18guidance. There are many pitfalls that
14:19you can have from a security perspective
14:21when you add a login page, but there's a
14:24big difference in how they handle the
14:25deeper context. Claude shined when you
14:27gave it precise instructions, but
14:28dropped a fair bit uh when you gave it
14:30the longer instruction. Codeex didn't do
14:32as well in total, but dealt with the
14:34longer context a lot more. And it's
14:37interesting because it's not obvious to
14:39people that not all agents listen the
14:42same. You we told them the same words.
14:44Some of these actually use the same
14:45model behind them, but agents process
14:48and make many decisions. And you know,
14:51granted, there are 10 runs behind each
14:52of these columns. So maybe there's also
14:54not enough data for statistical
14:55significance, but the results are
14:57substantially different. And you have to
14:58think not just about what is the
14:59question you're asking and the
15:00saturation, but also who are you asking
15:02it of?
15:05So, so we have this challenge. We want
15:07to tell agents a lot of things and they
15:10uh you know, we can't because it's going
15:12to deteriorate their performance. what
15:13do we do? So, it's a broader problem and
15:16there are a lot there's a lot to say
15:17about it, but one common pattern is the
15:19separation between rules and knowledge.
15:22Rules would be a small amount of data
15:24that you're providing it that you're
15:25kind of shoving down their throat.
15:27You're saying this will be in your
15:28agent. going to see this every single
15:30time and they have to be succinct and
15:32what they should do is they should
15:34include links and references to guide
15:36agentic search around where can it fetch
15:38additional information and that is the
15:40knowledge and I'll give an example from
15:42Tesla's content. So hopefully many
15:45people on this uh uh in this event uh
15:48here or virtually know that Tesla has a
15:50spec registry and we it's a a dependency
15:54system for knowledge. you can publish
15:56context into it in this form we call
15:58tiles and then you can consume that down
16:00to your uh to your consumers right to
16:03your agents uh and it will adapt it to
16:06the local agent and you can do it for
16:07your context but we also pre-populated
16:10the registry with over 10,000 specs or
16:14context uh that uh that that helps
16:17agents use open source libraries better.
16:19We've analyzed those libraries, we've
16:21iterated, we've evaluated and we created
16:23those. So let's let's look a little bit
16:25at what is the structure of those. Um in
16:29first of all we define a knowledge base.
16:31So well what is the knowledge that you
16:33want? So you do that in this tessle.json
16:35file. In our example there are
16:36variations of it we'll touch uh to say
16:39this is the information I want. In this
16:40case I have a bunch of libraries. I want
16:42to know how to use them. And then you'll
16:44see that each one of those is fragmented
16:46into pieces. So the agent doesn't need
16:49to suck it all up right away. Especially
16:51for some of these libraries, there's a
16:52lot to say. You don't want to hog the
16:54context window. So there's a bunch of
16:56different files.
16:58The second thing we would do is we will
17:00put a little bit of knowledge or a
17:02little bit of rules in the agents MD
17:04file and we'll say here's a link to your
17:06knowledge. Uh we actually it's a bit
17:08longer over here but we also give a
17:10little bit of instruction of uh what is
17:12the structure of the knowledge you might
17:13find to the agent but it's a very small
17:16amount of context that is in the roles
17:17and always loaded. And then if you click
17:20that knowledgemd file, you will get a
17:22markdown file that the agent can easily
17:24find out which libraries do you have a
17:26little bit of information about them and
17:28where can it learn more and then down
17:30the rabbit hole it can learn more. It
17:32has an index file it explains about in
17:34this case next.js and then it can in
17:38that file find links to additional
17:39information. So this is like a human
17:41clicking links on a website
17:43[clears throat] right you have a summary
17:44thing you're expanding a section or
17:46you're clicking through and it's very
17:47effective. This is a good way to give
17:50the agent uh a way to find the right
17:53information. Lean into its agentic
17:55search capabilities, but don't ask it to
17:58sort of browse the web and just figure
17:59it out.
18:02So hopefully at this point you're
18:03saying, well, show me some data. I
18:06realized as I was building it that uh
18:07this is a Jaring Maguire reference and
18:09not everybody knows that. It might be
18:11like a my age is starting to defer of
18:13it. Like it does say show me the money
18:15typically. Um so okay can we see some
18:18data right I make this claim that this
18:20is a good way and a good effective way
18:21to to make the agents work. Let's see
18:23some data. I'll show two examples of
18:25this. First is uh thanks to the great
18:28work that uh the team at Versell has
18:31done on Nex.js. So Nex.js JS is a very
18:33well uh wellused very popular JavaScript
18:36framework that is maintained and built
18:39today by the Verscell team and they
18:41published as a thought leader in the AI
18:43space uh a benchmark about that tries to
18:46measure how well can agents use Nex.js
18:49and they created about 50 different test
18:51cases uh and uh they ran it through
18:54different agents to see how well they
18:56can address those. It's interesting, by
18:59the way, immediately to see that even
19:01though Nex.js is so popular and has so
19:04much content, it's like the sweetheart
19:05of the training data, right? It's well
19:07doumented. It's an amazingly like clear
19:10set of data. The top agents, the ones
19:13that did the best, still only got a 42%
19:15score in this benchmark. And so that's
19:17already interesting and a bit sobering.
19:19But it goes to show, okay, can we
19:21improve this with context engineering?
19:22and in general that if you measure this
19:25now I bet you all the agents are now
19:27working on optimizing it and so our
19:30question was okay we've done this can we
19:32optimize this with context can we test
19:35our uh uh our our knowledge like these
19:38tiles that we create do they actually
19:39make it better and we ran this eval
19:42because we already had the eval so we
19:43just had to compare the two test cases
19:45and we're happy to see that yes yes yes
19:47we can with the knowledge the success
19:50rate like for us it was about 40% % in
19:53our uh uh test environment. Success rate
19:55of using SJS on this benchmark, it
19:58jumped up to 92% if we got that info.
20:00This is if we got you know the very
20:02thoroughly created piece of information.
20:04But we also find for instance that a
20:06smaller prompt midway that just guides
20:08the agent to look at the things that you
20:10care about. Uh in this case, some
20:13linting and some build errors actually
20:15also bumped up the process
20:16substantially. So sometimes it's
20:17knowledge and sometimes it's steering
20:20around what is it to pay attention to.
20:24This is great for next.js
20:25[clears throat] but this was a very
20:26thorough benchmark set that required a
20:29lot of attention. We can't do that for
20:31everything. But what you can do when you
20:32evaluate your content or when you have
20:34something that is uh you don't have the
20:36sort of the bandwidth to create a deep
20:38benchmark for is that you can use LLMs
20:40to create evaluation data. And so we
20:43took 270 random libraries and we asked
20:46the LLM to create test exercises for
20:49them, a scorecard to evaluate how well
20:52they did on that test and then we ran it
20:54through agents. The data I have here is
20:56for cloud code. We saw similar data for
20:58cursor and then eventually you ask an
21:00LLM to judge the result with that
21:02rubric. And indeed when we ran that we
21:04saw that on average across those 270 we
21:08bumped up the success rate from about
21:1060% to about 81% uh in open source in
21:14agents ability to consume open source
21:15across these 270 libraries and we did
21:17that with a little bit of a shortening
21:19of the time. So the agents were a lot
21:21more successful in a little bit less
21:22time a little bit faster which was
21:24really encouraging. So this was great
21:26and this is again the validation part.
21:28We also had you always find these bits
21:30of information that you don't know how
21:31to expect. uh some interesting
21:33information about the age of a library.
21:35And so this graph shows with the same
21:37test the success rate of the agent. This
21:40is cloud code across uh correlated to
21:44the uh age of the libraries. The
21:46libraries at the left are the oldest.
21:47The libraries on the right are the
21:49newest that this is release date just as
21:51a simple measure. And it's interesting
21:53to see how the libraries that are 5 to
21:5510 years old, those are the ones that
21:57the agents do the best with. And we've
21:59seen this with other agents as well. Uh,
22:01and the distance is substantial. And
22:03that makes sense because very old
22:05libraries, they might have they might
22:07have confusing information on the web.
22:10Uh, maybe they don't have as much web
22:11representation if they're truly old. And
22:13the very new libraries often times they
22:15haven't accumulated enough information
22:16on the web and some of their data is
22:18actually after the training data. And
22:20our theory was that with uh with the
22:22tiles, we actually can flatten that
22:24line, right? We because we gave them the
22:26information, they have it, so they
22:27should use it. And indeed, we've seen
22:28that we elevate the result more broadly.
22:31And this was useful because it matches
22:32our kind of anecdotal experience that um
22:36using this type of knowledge, using the
22:37right context, sometimes just helps and
22:39sometimes it turns it from it doesn't
22:41work to it works.
22:45Um I'll move on to multiplayer, but just
22:48one thing to say is that in context
22:49engineering is actually a lot more than
22:50what I'm showing right now. And there
22:52are many many different tools and one
22:54notable one is the ability to use a sub
22:56agent in claude and now a few other
22:58agents to be able to give it its context
23:01and focus it. So there's a lot more to
23:02do here and it's a worthy domain to
23:04invest in.
23:07So with that let's talk from single
23:09player talk about multiplayer. In
23:10multiplayer, the question is in your
23:13team, in your organization, what context
23:16do you want to provide and when? What
23:19are the ways in which you can share
23:20context across the team? I'll talk about
23:23three versions of this. The first and
23:26kind of easiest most sort of uh direct
23:29way to share context is to put stuff in
23:31the repo. As we said, the project itself
23:34is the base context for an agent. And so
23:37if you check in an agent MD file,
23:39anybody that uses that repository would
23:40load it. It's the repo or the path
23:42within the repo. This is great and you
23:44should do it and it is a perfect fit for
23:47things that are repo specific knowledge.
23:49Right? This is documentation about how
23:50this application behaves or things like
23:52that. Awesome. Perfect place to do it.
23:54Highly recommended. Uh where it slightly
23:57falters is uh one is cross repository
24:00information. your be best practices,
24:02shared libraries and all those feels a
24:04bit odd to check those into a
24:06repository. Would you check them into
24:08multiple repositories? You might want a
24:09different solution. And the second is uh
24:12and I'm picking a little bit on backlog
24:13MD over here. You know, it's a great
24:15framework. We had a great workshop
24:16yesterday about it. Um and you can see
24:18though that you know those who adopt
24:20agents and lean into them, what you find
24:22in the repos is you find many duplicate
24:23files. You find your sort of Gemini MD
24:25and cloud MD and agent MD and cursor
24:27rules and you'll find skills. You'll
24:28find GitHub stuff.
24:30and and that's not awesome. Uh and it's
24:33not that you have this many files that
24:34is the problem, but rather that they are
24:36duplicative. There's often content that
24:38is very similar within each one of them.
24:40So that's a challenge.
24:43The second tool that you have to
24:45accumulate and to provide shared context
24:48amongst your team is to give agents
24:50tools to gather context. uh most common
24:53is of course browsing the web but also
24:55things that are more focused like hey
24:57use this to uh uh this MCP tool to be
25:00able to go and fetch uh code from GitHub
25:03repositories and load them. So this is
25:05I'd say you know first of all super
25:07useful for discovery and exploration
25:09like you know you you don't know which
25:10library you're going to use clearly you
25:12don't have the context for that library
25:13as you're exploring you should have
25:15dynamic tools to go and fetch that. It's
25:16also great for dynamic data like uh uh
25:19production data about how your system
25:21ran or you know something that is
25:22constantly changing. So for those it is
25:24amazing. Uh it is not great when there's
25:28a high potential to get the wrong
25:29answer. Versioning is a is a strong
25:32example of that. If you're using a
25:33version that's not the latest and your
25:35agent went to the latest GitHub repo,
25:37it's going to get the wrong information
25:38to operate with. And for complex topics
25:42where there's subtleties, you know,
25:43authentication related topics or um just
25:46sort of complex environments,
25:49it's also highly inefficient for
25:52repeatedly needed info. So let's say you
25:55can go off and find the relevant commit
25:57at the relevant repo and read the code.
25:59If you need this information seven times
26:01a day, you know, five days a week, why
26:05like why wouldn't you sort of gather
26:06that context one time, store it,
26:08evaluate it and then optimize for that?
26:11Which gets me to the third mode which is
26:13today kind of the cutting edge and it's
26:16the notion of curated context or curated
26:18knowledge. What you would have seen is
26:20over the last couple of months you've
26:22seen a lot of progress from all the big
26:24agents uh uh providing reusable context
26:28frameworks. So they have you have you
26:30saw claude skills you saw cursor team
26:32rules you saw GitHub copilot spaces uh
26:35and you know these are these are kind of
26:37acknowledgements from these agent
26:39companies to say well we don't want the
26:40agent to reinvent it every time we
26:42actually want to allow you to share
26:44things and to share practices and that's
26:46great I highly recommend these platforms
26:48you should learn about your environment
26:49for it they're great for cross
26:51repository knowledge uh they're great
26:53when you're using often times you start
26:54seeing people use uh maybe like a you
26:57have a security code review bot that
26:59runs in your review surrounding, but
27:01that same context, you also want it when
27:03you're building locally and maybe when
27:04you're assessing an incident with
27:06another agent. So, it's great to have
27:08not just crossdevelopment repo, but also
27:10cross use case knowledge. Uh, and
27:12because they're built natively, they
27:14tend to be kind of nice and elegant in
27:15terms of how they interact, you know,
27:16clawed with its skills, cursor with its
27:18rules, etc. Uh, they're a bit of an
27:20overkill for single repo information.
27:22Maybe that's not too bad. But really the
27:24biggest challenge with them is that they
27:26are single agent. Um and that kind of
27:29begs the question of do we think long
27:31term that knowledge and context should
27:34be managed per agent, right? Do you
27:36think in your organization, in your
27:38practice, do you expect that you would
27:39use one agent for your org over time?
27:43And I think the answer to that is no.
27:45Like I think just like any technologies,
27:47technologies will be better at one thing
27:49versus the other. They will move. they
27:51will change. Companies are big. People
27:53will have preferences
27:55and that knowledge as a whole should be
27:57something that is a core competency of
27:59the knowledge that you have and it
28:01should be adapted per agent but it
28:02should be your asset that is broader. So
28:05that's the approach we have at Tessle.
28:06You've sort of seen a bit of this
28:07already. We have the Tessle JSON and
28:09when you install this knowledge then it
28:12will it will adapt it to the relevant
28:15agent. It will store it as a cursor role
28:16or it would store it as a as an agent MD
28:18or whatever that is. Uh and then also
28:20once we download that knowledge, the
28:22knowledge is there as files and so you
28:24can use it in whatever way that you
28:26want. We do provide tools to make it
28:29easier and better to work with it. We
28:30want it to be seamless, but you're not
28:32locked in to you need to use those tools
28:34if you want to use different tools to
28:36consume it. So we think this is
28:37important when we think about uh uh kind
28:40of broader long-term context management
28:42and development.
28:44And in fact, we're doubling down on
28:46this. And just as a slight plug here,
28:49you know, we are expanding the registry
28:51into a full-blown TESL uh agent
28:54enablement platform uh that will help
28:56you gather and and this type of
28:58knowledge like the packages you've seen,
29:00create and run evaluations for them,
29:02distribute them so that every repository
29:04gets its knowledge uh and then help you
29:06optimize that over time. Uh so this is a
29:08commercial product and if you're
29:10interested in being uh being part of it,
29:12uh come come talk to us, see us or email
29:15us at contact.io. fail if you're online.
29:19So that's a good segue for just the
29:21closing points around ecosystem context.
29:24And you know when I just sort of sang
29:27the sort of the uh the praises of why
29:29ecosystemwide context from the registry
29:32is helpful and you should use it and
29:33it'll help you today. But in the long
29:35run you do have to wonder who should
29:38create this type of open source context.
29:40know we we come along to someone else's
29:42library and we create these agent docs
29:44for them and we track them and that you
29:46know we do it today because it's helpful
29:48today and we want to help but long-term
29:50is that right
29:52and I think the answer is no like I
29:54think over time kind of open source
29:56context is really an aspect of agent
29:59experience and it is the creator's
30:01responsibility if you are an open source
30:03maintainer if you are a vendor it's your
30:06responsibility to have a great developer
30:07experience it's also your experience
30:10your responsibility to have a great
30:12agent experience. You know, it is it is
30:14in your control. It is something that
30:15you can choose where to invest and
30:18context is your best tool for that.
30:20Context is both your documentation and
30:21your UX. You know, you need to think
30:23about what is it that you say there. How
30:25do you distribute that to your users?
30:28You know, do you want this indeed to be
30:29single agent or multi- aent? And eval
30:32are your tests. They're your moment of
30:34defining what is correct and what is not
30:36correct. And so, you should invest in
30:38those. You should define them. And you
30:39should ask your users for feedback on
30:41when they work and when they don't work.
30:44And you're it's your responsibility, but
30:46you're not alone. You know, you have us
30:48to help with our technology with our
30:49platform on it. You know, feel free to
30:51talk to us. You have the amazing AI
30:53native dev uh community that you should
30:55tap and they will help you. And
30:57together, I think we want to create this
30:59successful methodologies and actual
31:02context that will help us all consume
31:04software together better. Uh and if we
31:06achieve all of that then we'll help
31:08we'll successfully put everything in the
31:10right context. Thank you.
31:16[music]
31:34>> [music]

About this transcript

This page contains the full transcript of Guy Podjarny - Spec Driven Dev From Single Player to Multiplayer to Ecosystem | DevCon Fall 2025 by AI Native Dev, generated from the public captions YouTube serves with the video. The transcript has 6,288 words across 876 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.