YT YouTube2Text

YouTube2Text → Transcripts → FIXING Opus 5: PROOF that Prompt Engineering IS NOT DEAD

FIXING Opus 5: PROOF that Prompt Engineering IS NOT DEAD — Transcript

by IndyDevDan · 6,821 words · 993 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00What's up engineers? Any DevDan here.
0:01Like me, you've probably gotten sick and
0:04tired of Opus 5's insanely verbose
0:07responses and its overuse of phrases
0:09like load-bearing, worth stating
0:11plainly, here's the honest truth, and a
0:13bunch of others. Or maybe you're tired
0:15of the Anthropic team trying to take
0:17credit in your Git commit messages for
0:19intelligence you paid for. Or maybe you
0:21notice Opus 5 is burning your cash with
0:25way more alpha tokens than any model
0:27before it. You're not alone. Myself and
0:29many engineers feel the exact same way.
0:31Opus 5 is one of the best
0:33state-of-the-art ultra-smart models and
0:36one of the worst state-of-the-art models
0:38ever released because it talks like a
0:40complete smartass. In this video, we
0:42turn smartass Opus 5 into a precise
0:45senior engineer that's enjoyable to work
0:48with. How are we going to do that? We're
0:49going to use one of the most important
0:51skills any engineer using agents can
0:54learn. You know what it is. It's prompt
0:57engineering. The skill that was once a
0:59complete joke is now the most important
1:01skill for [music] any engineer to master
1:04to scale their impact with agents. If
1:07you hear someone say prompt engineering
1:09is dead, completely ignore them. They
1:12have no idea what they're talking about.
1:14I've been engineering for 15 years now
1:16and one thing that all the best
1:18engineers I've worked with know how to
1:20do extraordinarily well is this. Above
1:23all, they know how to communicate
1:24[music]
1:25with their technology. And guess how you
1:27communicate with agents. Yes, prompts.
1:30There are two ways [music] to prompt
1:32engineer your agents. Most engineers
1:34only use one when the second is most
1:36powerful by far. By the end of this
1:38[music] video, you'll have the prompt
1:39engineering expertise to make your AI
1:41agent communicate like a precise cracked
1:44senior engineer, no matter the model
1:46that's running. Today it's Opus 5,
1:47tomorrow [music] it'll be something
1:48else. Let's fix Opus 5 with system
1:51prompt engineering.
1:58What are the two ways to prompt your
2:00agents? There's of course the user
2:01prompt, which you're very familiar with,
2:03the single task at hand. But then
2:05there's the system prompt, which is the
2:07law for every task you hand your agent.
2:10Most engineers fixate on skills, skills,
2:13skills, skills, slash this, slash that,
2:15slash plan, slash grill, slash review.
2:18Don't get me wrong, skills are powerful,
2:20the user prompt is powerful, but the
2:21system prompt is vastly more useful and
2:24most engineers never even touch it.
2:26Because the system prompt is where every
2:28single word you write is multiplied over
2:31every single user prompt. The system
2:33prompt is essential for setting up great
2:35communication patterns with your agents
2:38and for reducing those expensive Opus 5
2:41output token costs dramatically. The
2:43system prompt is where the real leverage
2:45is. Let me show you exactly what I mean.
2:52In VS Code here, we have the simple
2:54three file structure. I've cloned down
2:58Zuck's The Future Is For Everyone. We're
3:00going to do what all of us degenerates
3:02do nowadays. We just summarize a long
3:04article like this. We're not going to
3:05read this whole thing. We don't have the
3:06time. We're too busy with 10 million
3:08agents open in our terminal window. To
3:10do that, we're going to use a standard
3:12powerful Claude Opus 5 agent. As we work
3:15through this, we're going to constantly
3:17improve the system prompt we're passing
3:19in to Claude code. I have this simple
3:22command here called compare. So let's go
3:23ahead and fire it off. Inside of our
3:25terminal, we're running Herder. This is
3:26my terminal multiplexer of choice.
3:28You'll see exactly why in a second. If I
3:30type J, you can see all the commands in
3:32this directory. And what I'll do is just
3:34type J compare and we'll start with
3:36zero. I now have a brand new workspace
3:39open. Let's go and hop into it and see
3:41what's going on. You can see here I'm
3:42running two Claude code instances side
3:44by side. On the left, we have smart ass
3:47Opus 5 and on the right, we have senior
3:49Opus 5. Right now, these are both in
3:51smart ass mode. This is just the default
3:53system prompt and the default Claude
3:55Opus 5 model with no tweaks, no changes,
3:58no prompt engineering to adjust and
4:00improve the system. As you can see, all
4:02of the ticks and problems and verbosity
4:05of this model are coming alive right
4:08now. Zuck's blog is very, very long, but
4:10you can see here these models spin for a
4:12long time. The one on the left is
4:14really, really going off here with a lot
4:16of information. Okay, so 53 seconds, 35
4:19seconds. Most of that was just output
4:20text. Look at how many output tokens my
4:23agent just torched responding to me.
4:25Most of us truly were not reading this
4:27whole thing, just like we're not reading
4:29Zuck's blog. We need concise, compressed
4:31information so we can move on or decide
4:34to invest more. A lot of engineering
4:36with agents is about figuring out where
4:38to spend your time. So, let's improve
4:40the system by prompt engineering the
4:42system prompt.
4:49So, in VS Code, I have senior Opus 5
4:51system prompt and this is getting passed
4:53directly into that herder pane. You can
4:55see here we have one version that's just
4:58Claude Code Opus out of the box and then
4:59we have the other where we're pending
5:01the system prompt file using this
5:04argument flag. So, that's what we're
5:05going to do here. And so, if we open up
5:06this file, right now we have nothing in
5:08there. So, we just got the default
5:09response out of it. Let's improve it.
5:12Let's write a great system prompt to
5:14steer the behavior of every single run,
5:17every single input and every single
5:19output from our model. What we're
5:20building here is a clear, concise
5:22communication document. I am typing this
5:24by hand. When I'm working with pieces of
5:27autonomous software that's going to be
5:28multiplied over many, many, many runs, I
5:31do it by hand. If you don't understand
5:33your technology, if you're just vibe
5:34slopping everything, you're not going to
5:36get the results who someone paying
5:38attention is going to get. We're going
5:39to start really simple here. We're
5:40appending this to the system prompt,
5:42clear, concise, actionable
5:43communication, and then we're going to
5:44talk through the purpose. First things
5:46first, you and I maintain a no-BS,
5:49clear, concise, actionable relationship.
5:51Notice how I'm talking to my agent. I'm
5:54not giving it a role. I'm just talking
5:56to my agent as almost if I'm talking to
5:58like another engineer, okay? And and
5:59kind of setting the standards. We
6:01maintain a no-BS, clear, concise,
6:03actionable relationship. Every word we
6:05say together reinforces our clear,
6:08concise, actionable communication. I'll
6:10jump through some of this. We're here to
6:12solve problems and create value and our
6:15communication reflects that. And then
6:17I'm going to start the next section,
6:18instructions, and this is going to be
6:20where we dial into several ideas. We'll
6:23have four sections and then one kind of
6:25recap section. Um this will be the
6:26examples and then we'll work through
6:28these sections one by one, but I'm
6:30writing this out to be able to reference
6:33these sections so that I can clearly
6:34communicate inside the system prompt
6:36what I want my agent to focus on, okay?
6:38So, pay close attention to the details
6:40throughout instructions to maintain our
6:43great communication patterns. And then
6:45I'm going to do something here that I
6:47think is really important when you're
6:48prompt engineering. I'm going to do
6:50explain why. Why? So we can deliver the
6:52best possible results for our team,
6:56business, and customers. Just walking
6:58through things here really, really
7:00cleanly, really concisely with my agent.
7:03Let's pause for a second here. Let me
7:05just actually delete this and just kick
7:07this off again. Let me save that text
7:08there. Let's go back into our terminal
7:10here. Let's close this down. Let's fire
7:13off our J compare uh with purpose. So,
7:16it's going to just rerun that exact same
7:17thing. Already we have one kind of clear
7:19change in our system. And you can see
7:22here at the top we do have that
7:23reference append system prompt. We're
7:25attaching that, but already we have made
7:27a small change in our purpose. This is a
7:29good start. As you can see here, it's
7:31not going to change a lot of the ticks
7:33and the problems inside the model. As
7:36you can see, there's load-bearing right
7:37there. That's one of the classic ones
7:39with Opus 5. It's using a lot of dashes,
7:42but it is communicating a little bit
7:43better. You can see that one came in at
7:4435 seconds. Not a huge difference at all
7:47from previous runs. But you can see
7:48here, if we look through these, we would
7:50probably see some glimpses of clear
7:53communication, but we need more than
7:54this. This is all very high-level. Let's
7:56get more mid-level, more low-level with
7:58our system prompt. So, I'm going to copy
8:00this in. And now we have instructions.
8:01Okay, so purpose, instructions. Let's
8:03start giving some real [music] clear
8:05communication patterns.
8:11Now we get into like concrete prompt
8:13engineering patterns that are repeatable
8:15across all of your work. For instance,
8:17here's one clear pattern, positive
8:18patterns and negative patterns. And so,
8:21what does this section do? Replicate the
8:23and I'm going to just write out a H4
8:25here, positive patterns. And of course,
8:27we'll have that and then we'll have
8:29negative patterns. And so, we're just
8:31going to walk through this and, you
8:33know, as you can imagine, I'm going to
8:34clearly reference this section as
8:36behavioral references. So, I'm saying
8:38replicate the positive patterns and then
8:40I'm going to of course say avoid the
8:42negative patterns. So, we want to
8:44replicate these and we want to ignore
8:46and not do these. Okay, so we're doing
8:48both positive, do this, and negative,
8:50don't do this. And now we can just start
8:53writing out these bullets. What do you
8:54want your agent to do? What do you want
8:55them to not do? I'm just going to start
8:56from the top and this is one of the
8:57important things that is missed a lot.
8:59So, I'll say, I always see the last
9:02thing you write first. Place the most
9:04important information there. As you're
9:07working with these agents, you've likely
9:08noticed this too. You see the last
9:10thing. So, it's important that the last
9:11thing you see is the most important so
9:13you can, if you want to, skip all the
9:15other stuff. What else do we want the
9:16agent to keep doing? Positive patterns.
9:18Use plain, specific language. State each
9:22fact once. There's no need to repeat
9:25anything. Match the level of detail to
9:27the level of task and request. Okay, so
9:30this keeps the agent aligned with the
9:32amount of detail and effort you're
9:34putting in. Challenge incorrect
9:35assumptions directly and explain why. We
9:38do not want any sycophancy here in our
9:41responses. We want a useful engineering
9:43partner optimized for clarity and
9:46engineering value, not quotability. I
9:48don't want to have a good conversation.
9:50I want to deliver engineering value with
9:52clear concise communication patterns,
9:54okay? One more note here on domain
9:56terminology. Use the simplest domain
9:59terminology that compresses information.
10:02So, just some positive patterns. Here
10:04you want to write what you want to see.
10:06And then of course, the negative
10:07patterns, you know exactly what we want
10:09to add to this. Avoid words and phrases
10:13in this list, okay? And then here we can
10:15just go crazy with all the obnoxious
10:18things that this model writes. We can do
10:20things like load-bearing, that's going
10:22to be at the top here. We can do things
10:23like worth stating plainly, here's the
10:26honest truth, the real [snorts] tension.
10:29>> [laughter]
10:30>> And then there's carry the argument.
10:32There are a bunch of these. Put
10:34whichever ones here you want. A couple
10:36things I don't like personally, I don't
10:38like avoid analogies, discuss what's
10:41right in front of us. Not a huge fan of
10:43analogies when working. I just want to,
10:45you know, focus on the thing right in
10:47our face. Of course, do not overuse em
10:50dashes or dash chaining. I think some of
10:53these are fine. My writing style
10:55personally has a decent amount of em
10:56dashes. I have had to personally dial
10:58these back because I don't want people
11:00to think I'm just spamming out using
11:02models when I'm replying. I think it's
11:03really rude if it looks like you're just
11:05responding with AI with zero thought in
11:08specific circumstances. Do not flatter,
11:10praise, validate, or agree without
11:14reason. Again, no sycophancy. Just kind
11:16of re-emphasizing that idea. Do not use
11:19decorative headings, emoji, or
11:22motivative language. Again, I just want
11:25this model to talk like a cracked senior
11:27engineer. Let's just solve the problem
11:29at hand. Avoid semicolons, fragments,
11:33non-standard punctuation. I just want a
11:36normal conversation. This is one really
11:38important section in your prompt
11:39engineering. You can use this when
11:41you're building plans. For the system
11:43prompt specifically, this has a lot of
11:45weight to it. We're explicitly talking
11:46with our model saying, "Replicate these
11:48things. Avoid these patterns." So, how
11:50else can we prompt engineer our system
11:53prompt for consistent results across
11:55every single execution? Before we do
11:57that, let's actually run this. Let's see
11:59how this is developing. Okay, so I'm
12:00going to go ahead and just flip this,
12:02place this here, go ahead and just
12:03rerun, trash this session, and let's run
12:05it again. So, now we have pause
12:07negative. All right? So, let's rerun a
12:09new pair of cloud code. Again, on the
12:11left we have our smart ass opus, no
12:13changes. And on the right we're going to
12:15have our smart senior concise
12:18engineering opus. Okay, so let's see how
12:20it does now. Right away, you can see
12:21something very very cool. Very few
12:23dashes here. We do have a couple dashes
12:24showing up. I don't see load bearing
12:25anywhere,
12:26>> [laughter]
12:27>> which is good. I'll do a quick search on
12:29that in a second. But, there's a bunch
12:30of these obnoxious keywords that are not
12:33getting referenced. 31 second execution,
12:36so we are speeding things up a bit. And
12:37by speed I mean it's outputting fewer
12:40tokens. So, we still have some dashes.
12:42We didn't say never use them. We said
12:43use fewer. The language is looking a bit
12:45clearer. We have clear headers, you
12:47know, no analogies. We do have a dash
12:49chain here, private by default. Maybe
12:50you're okay with that. Maybe you don't
12:52want to see that. That's up to you. You
12:53can prompt engineer that however you
12:54like. You can see here we're making some
12:56improvement, and we're starting to save
12:57on the output tokens. Even in your
12:59subscription, your output tokens still
13:01churn up the most of your usage. So,
13:04it's important to keep that in mind. We
13:05can do this at the system prompt level.
13:07Why? Because we want to affect every
13:09single input and output from the agent.
13:12Your prompts in and its responses out.
13:14If you have something that you want to
13:15apply globally, that is the time to
13:17reach for the system prompt. So, let's
13:18keep moving. How else can we improve our
13:20communication patterns with our agents?
13:27So, number two. This is really powerful
13:29one that I'm a huge fan of and I think
13:31you will be too. Check this out. We can
13:33use reference points. What is this? We
13:35use and again I'm talking to my agent,
13:38right? I'm talking directly to them. I'm
13:39not giving them a weird funky role. I'm
13:41just talking to them. These models are
13:43getting smarter and even the lower class
13:45workhorse models, the A and B tier
13:47models that aren't your S tier say the
13:49art models, they respond really really
13:50well to this additional direction. We
13:52use reference points to communicate
13:55quickly with each other. And now what do
13:57I mean by that? What is a reference
13:58point? Use numbered lists and markdown
14:00headings when they improve navigation
14:02when presenting three or more findings
14:05and this is stuff like uh decisions,
14:07options, risks, questions, or actions.
14:11Assign everyone a short code. Okay, so
14:14what does that mean? So use D1 for
14:17decisions and what I'm really looking
14:18for here is D1, D2, and then DN. You can
14:22imagine this going on, right? So I'll
14:23I'll do this and do a dot dot dot and
14:26I'll skip over this in the video so you
14:28don't have to wait for me to type all
14:29this out and then we do dot dot dot for
14:30continuation. When we don't have
14:32something here, I want it to invent new
14:35references for sections we don't have,
14:38preserve same codes throughout the
14:41conversation, do not create codes for
14:43short simple answers. This is really
14:46cool. So you can see this in action
14:48right away, right? Let's go ahead and
14:49fire off our system and see how the
14:51system prompt materially impacts every
14:54single prompt. Cuz again, that's the
14:55scale of what we're dealing with here.
14:57The system prompt is the law for your
14:58agents. It affects every single task.
15:00Let's delete the previous session. Bam
15:02and let's rerun. This is going to be our
15:04ref points. So now we have that new
15:06workspace. We're comparing side by side
15:09and of course Smart Ass on the left and
15:10we have our senior opus slowly improving
15:13with each change we make. So you can see
15:15here we have the P's, we have the R's
15:18coming up and so we have risks there and
15:20now our agent is even referencing one of
15:23the R's, R6. Okay, so now we can just
15:25jump to the reference. Our agent isn't
15:27repeating itself. It's not wasting time
15:28and tokens and it's repeating the risks
15:31here. And then if we scroll back up to
15:33the P's, I assume these are the promises
15:36that Meta says it's going to actually
15:38make. What is Meta saying it's going to
15:39ship? It doesn't really matter what the
15:40P stands for. The fact is that we can
15:42reference them now instantly. Okay, so
15:44for instance, like I can continue the
15:46prompt here, right? So if I said, "Talk
15:47more about R6." And of course, my agent
15:51knows exactly what I'm talking about
15:52when I say R6 because we've created this
15:54quick language together. This is risk
15:57number six in this section, existential
15:59RSI self-improvement risk that
16:01Zuckerberg is talking about here and our
16:03agent is just going to kind of continue
16:05breaking things down here, right? What
16:07R6 actually says, the section claims and
16:09sequence. There it goes and it's going
16:11to break down that section more. We can
16:12jump into it and you can see here we
16:14have another reference here. We have
16:16F's. These are findings. That's exactly
16:18what we've encoded here, F's for
16:20findings. And so our agent is just using
16:22our reference system. And again, like
16:24the key I really want to communicate
16:25here with you for your engineering work
16:26is like um great communication is great
16:29engineering and vice versa. Knowing how
16:31to properly communicate with your
16:32technology now more than ever is a
16:35massive advantage you can use. It's so
16:37underutilized and it's so powerful. You
16:39can see here we're starting to improve
16:40things. We're adding layer by layer.
16:42Let's add some more really important
16:44layers. [music]
16:49What is the third improvement to our
16:51system? Let's go ahead and dial into
16:53this, hard operational boundaries. Okay.
16:56One of the very annoying things with
16:59some of these state-of-the-art models is
17:01that they will do things that they have
17:03not been requested. This is part of
17:05their reinforcement learning. This is
17:06part of the loops that they're going
17:08through when these models are being
17:09trained where they're just taught to
17:11find the answer at all costs, no matter
17:14what it takes. Opus 5 is like really
17:16guilty of this. Comment down below, let
17:17me know if you've experienced this. Opus
17:195 will just find and call out and
17:21reference problems you did not even ask
17:24for remotely. And it's trying to pull
17:27everything together. It's trying to do
17:28as much as possible, but in that it
17:30actually loses focus pretty quickly. So,
17:33hard operational boundaries solve that
17:35problem. Let's write that out. So, in
17:37addition to clearly communicating, it's
17:39important that we clearly communicate
17:41our work operational boundaries. What do
17:44you mean by that? So, deliver only what
17:46was requested at the intended scope.
17:49That single sentence really is this,
17:52right? But we can add more detail. Do
17:53not widen work into cleanup,
17:56refactoring, documentation, or any
17:58adjacent features. Just stay focused.
18:01That's what I'm saying here. Stay
18:02focused. Do not speculate on
18:05abstractions for future requirements. Do
18:08not claim completion without evidence.
18:11Very, very important. These models are
18:12really good at that. This is not a great
18:14line to add, but going to add it anyway.
18:15And then here's another one. So annoying
18:17when these models do this. Never add a
18:19co-author to a commit message. Like nice
18:21marketing trick for Anthropic, but they
18:23got to stop doing that. For completed
18:25work, concisely restate, but do not
18:28overload with response detail. There's a
18:30better way to say that, but you kind of
18:32get what I'm saying. Opus 5 specifically
18:33will respond with everything it's done
18:35and kind of like do a big recap. Don't
18:37really want that.
18:43And let's go ahead and stack another one
18:44on here. This is a really powerful one
18:45that I think is very useful, and it
18:47really starts veering into the fact that
18:50the system prompt can help you control
18:51your inputs and your outputs. This one
18:54is aliases. So, let me break this down.
18:56Aliases are reminders of great
18:59communication and patterns we want to
19:01uphold. When you see these exact
19:04aliases, expand them and act as if their
19:07expansions were given to you directly.
19:10And then we want to clear this up make
19:12sure we're being very clear. If these
19:14are referenced in a longer string, they
19:17are not aliases, do not expand. Okay, so
19:20these are exactly what you think they
19:21are. They're short codes, they're bash
19:23aliases, they're expansions, they're
19:25commands inside your system prompt. And
19:27if you want to, these can reference your
19:29commands, your skills, and other things
19:31you have, but you can also just write
19:33them in line like this. SCR equals
19:35simplify, compress, and repeat your
19:38response. Let's write a couple more and
19:39then we'll demo these. Okay, so Eli, you
19:42already know what this is going to be.
19:43Explain this like I'm We don't want to
19:45do five. Explain this like I'm 18.
19:47Simplify your language, shorten your
19:49response. What else? Uh one of my
19:51favorites, folk focus on what matters
19:54most here. What's the true signal?
19:57What's the true value? Boil your
19:59response down into the most important
20:02thing we need to focus on. And then ref,
20:05let's just do one more here to reference
20:07our previous work. Cuz once you start
20:09building up your system prompt,
20:10reference other sections. So I'm going
20:12to say I rewrite your responses with
20:14reference points, which is exactly what
20:16we dialed in above. So now let's try our
20:19improved enhanced system prompt. Same
20:21deal, open the terminal, kill this
20:23session with our aliases. So we're going
20:26to compare, boot these up, and now our
20:28agent has a little bit more guidance
20:31into exactly what it's going to do.
20:33Okay, so you can see there we have C123,
20:35that's our core thesis. We have our
20:37risks addressed. This is a much better
20:39generation really using all of our
20:41reference points. Very nice. And done in
20:4330 seconds, done 30 seconds on the left.
20:45So you know, sometimes the generations
20:46are just fast. These are not
20:47deterministic systems, so they're not
20:49always going to be generated faster. And
20:51in fact, we can improve that, which
20:52we'll do in a second here, but remember
20:54we just added those aliases. So let's
20:56run SCR, simplify, compress, and repeat.
20:59This is effectively a micro scale you're
21:01giving your agent. So there we go, it's
21:03compressing that, it's simplifying it,
21:05and it's repeating it. So much more
21:07simplified. I can act on this
21:09information a lot faster. Even if our
21:11agent responds with a lot, which again
21:13we'll tweak in a second, we can improve
21:15on that. So, let's run another, right?
21:16FOC, focus. This is focus on what
21:19matters the most here. So, our agent
21:21sees that and is going to do it. Now,
21:22there should be a lot more concise.
21:24There we go. The signal, alignment is
21:26being redefined from the model holds the
21:28lab's value to the agent holds yours.
21:31Everything else is downstream of that.
21:33This is speaking pattern from Opus that
21:35you might want to get rid of, totally up
21:37to you. We have why it matters, and then
21:39pulling some memory stuff here, and then
21:41we have our final paragraph there,
21:43right? Couple of things I want to do
21:44here. And once you like set up this
21:46foundation inside of your system prompt,
21:48you can really start to control it and
21:50fine-tune it. These patterns and these
21:52sections are really important because we
21:53can now collapse, and we can jump into
21:55anyone we need to very, very quickly. By
21:57hand, we are controlling the experience
22:00of our agent. Again, I really want to
22:01stress this point. The more
22:02multiplicative the thing you're working
22:04on is going to be for the rest of your
22:05work, the more you should stop, slow
22:08down. Don't just throw a prompt at this,
22:10even if it's a state-of-the-art model.
22:11As you can see, the state-of-the-art
22:13models have problems. They might
22:14accomplish your goal, but they'll add or
22:16do a bunch of other We're being
22:17really, really clear here. We're doing
22:19this by hand. We're slowing down. I know
22:21a lot of people are using Whisper flow.
22:22They're just talking into their device.
22:25There is going to be a level of detail
22:26that is missed, and you're going to be
22:28very exposed and very leaning on the
22:31normal distribution of what the models
22:33can do, and you're going to be inside
22:35that zone if you're just vibe prompting
22:37things. There's a time and place for
22:38that, but the high-leverage places is
22:40when you want to slow down and step in
22:42to the loop. And I mean old-school
22:45hands-on. So, anyway, let's look at
22:46this. So, I want to do some positive
22:49Where do I want to put this? This
22:50definitely going to be positive, and I
22:52want to say if you can communicate the
22:54idea in one paragraph instead of two, do
22:57so. And or to say without losing
23:00valuable information, do so. Same idea
23:02for one sentence versus two sentences.
23:05So, I'm just saying, be a little bit
23:06more compact, be a little bit more
23:07concise. This is important. Do not
23:10repeat yourself. State every idea once.
23:13Only repeat if it's relevant to
23:15subsequent queries. Yeah, something like
23:17that. Because we really don't want
23:19repetition. And then, I want to add
23:21another positive pattern here. Don't use
23:23overloaded terms that could mean more
23:27than one thing. Use the simplest word
23:29that satisfies the idea you're trying to
23:32communicate. Or I should say simplest
23:34words. And maybe parent that. You might
23:37be like, wow, you're getting really
23:39detailed here. And yeah, that's exactly
23:41right. I am getting really detailed
23:42here. And we can see the results of
23:43that. So, let's put this up again. And
23:45this is going to be uh fine-tuned. And
23:47now, let's go ahead and see what this
23:49execution looks like. And to be clear
23:51here, we are using a pen system prompt,
23:53not overwrite system prompt. But let's
23:55see how uh this execution runs now.
23:58Fewer dashes, no load-bearing. We have
24:01our reference points. Policy ask, again,
24:03with reference points. And our sentences
24:06are simpler, they're more concise,
24:08they're not getting overloaded, where
24:09the argument is the weakest. Still
24:11outputting a lot here. This is a pretty
24:13big one. 35 seconds, so not great, but
24:16it is going to be better than this
24:17generation on the left. Again, these are
24:19not a deterministic system, so it's not
24:21going to be consistent. 43 versus 35,
24:23not bad. We could probably do better in
24:25our compression. If this happened though
24:27in this specific case, we can just throw
24:28an SCR at this to get a simplify,
24:31compress, repeat. Or we can do an ELY.
24:33Or an ELY added, explain this like I'm
24:3518.
24:37And there you can see the language is
24:38really going to get boiled down. And
24:40sometimes this is just useful, right? It
24:41doesn't mean you're an idiot if you want
24:43simpler, concise language. It means
24:45you're effective. But you can see here,
24:47even still, the agent is saying a bunch
24:48of stuff. So, I might run an SCR after
24:50this, simplify, compress, and repeat.
24:52So, as we start stacking up the user
24:54prompts, in between every execution of a
24:56user prompt, the system prompt is still
24:58going to be ever present. So, you can
25:00see Zuk's argument really broken down
25:01here. AI is safest when everyone has it,
25:03not when a few model labs control it.
25:05Other labs say AI is dangerous, lock it
25:08down. He says, "When just a few people
25:10have it with enormous power, that's
25:11never gone well. One safe AI fails
25:14because people want different things.
25:15Any single AI has to pick whose values
25:18win." He's got a lot of really great
25:19arguments in here. I recommend you read
25:21and then compress if you need to uh the
25:23kind of core ideas coming out of Zuk's
25:25argument here. With these like uh CEO
25:27led posts, it's always like understand
25:30the incentives. When you're not leading
25:31the AI race, of course you would have a
25:33narrative something like this that you
25:35would want to share. I agree with him
25:37frankly on a lot of this stuff, but it
25:38is very let's say positional of him to
25:40put out something like this. That's not
25:42the focus of this video though. I'll
25:43link that in the description for you if
25:44you haven't seen that yet. One more
25:45thing I want to show you that can
25:46really, really dial in the performance
25:48of your models. And it's a prompt
25:51engineering technique as old as time.
25:53Again, if someone tells you prompt
25:54engineering is dead, don't listen to
25:55them. The Pi coding agent has a small
25:57system prompt. Cloud code just got rid
25:59of a lot of their system prompt. That is
26:01not the signal to not use a system
26:02prompt. That is the wrong signal.
26:04They're doing that because the models
26:06can just do a bunch of stuff without
26:08needing it. If you want the model to do
26:09specific things, to perform in specific
26:12ways, to not torch your output tokens,
26:15to have cool, unique capability like
26:18aliases or reference points or hard
26:21operational boundaries or patterns that
26:22you want to see and words and concepts
26:24you don't want to see, you must system
26:26prompt engineer. The system prompt is
26:28the most effective place to prompt
26:30engineer because it's applied across
26:32every one of your prompts going into
26:34your agent and the responses coming out
26:37of your agents. Okay? So, one more
26:38section here for you. Check this out.
26:44Examples to kind of put it all together.
26:47Here are concrete examples of how we do
26:48and do not communicate together.
26:50Replicate how we do communicate and
26:51avoid how we do not communicate. A lot
26:53like our positive and negative prompting
26:55section, except the main differentiator
26:57here is we're giving real examples. So,
27:00it's just like training data. User, is
27:02legacy JSON still referenced? To do, no,
27:05the only matches the file itself, there
27:07are no imports, blah blah blah blah
27:08blah, or documentation links. Very good.
27:10Um we could also, you know, make this
27:11tighter. The only match of the file is
27:13itself. We can make this the to do. And
27:15then here's a what not to do. Great
27:17question, I will research the repository
27:19and blah blah blah blah blah. Just do
27:21the work, respond concisely, right? And
27:22then here's another one, engineer
27:24recommendation, and it looks like I'm
27:25missing my user prompt here, so I'm just
27:27going to add that here. User, should we
27:29add Redis to this system? Write
27:31something just like this. To do, do not
27:34add Redis here. This process has one
27:36writer, stores from SQL, and has no
27:38cross-host coordination requirement.
27:40Redis adds a failure domain without
27:41solving the correct constraint, the
27:43current constraint. What not to do,
27:44you're absolutely right, blah blah blah
27:46blah blah. Again, just a really simple
27:47prompt engineering technique that is
27:49older than time at this point. Um I was
27:52writing these examples back when GPT 3.5
27:55was the model, GPT-4, when there was
27:57really no Anthropic. This is still
27:59relevant today. And that's how you know
28:01something is a valuable skill to learn,
28:03when years ago it was still relevant.
28:05These concepts, these patterns, these
28:07principles, great engineering is great
28:09communicating. These things just kind of
28:11don't go away. That's how you know you
28:12have something valuable to hold on to.
28:14Now, we can continue doing this, and
28:15kind of one big idea, one Easter egg for
28:18whoever's still watching, engineers that
28:20are really trying to absorb this value,
28:21one thing I want to mention here is that
28:23you can boot up a model, let's say you
28:25want to boot up a Claude Fable model,
28:27and you want to run that same prompt
28:29that we're running. So, let me just go
28:30ahead and pull that, explain this, copy,
28:33paste. And so, Fable 5 I think is a
28:35great model. I think it responds well.
28:37It doesn't have as many ticks and
28:39problems and repeated patterns and
28:41verbosity as Opus 5 has. So, what I like
28:44to do is in-context distillation.
28:47In-context distillation is exactly what
28:49you think. That's what examples are,
28:50right? It's in context distillation. Do
28:53this, don't do that. And you can do this
28:55by pulling out specific context,
28:58specific responses from models that you
29:01like. For instance, like this one-line
29:03core thesis here. Maybe we like this, we
29:06just copy this out, and we create
29:07another one here summarizing a blog. And
29:10then user is summarize the blog, and
29:13then XYZ, we just template this out. And
29:15then we have our to-do text. And then we
29:17can paste the response from the model
29:19that we like, and we can go even
29:21further, we can tweak the response from
29:22the model that we like. Get rid of
29:24couple of lines, make sure it's
29:26formatted properly. So we can do a dash
29:28search to get rid of all of our dashes,
29:30all of our framing. Maybe we like that
29:32one, we keep that one in. Super
29:33intelligence, super intelligence, super
29:34intelligence. This should be more
29:35concise and compact like this. So maybe
29:37we like this version, then we say not to
29:39do text, text. We hop back over to the
29:42previous version, right? We go to our
29:43smartass Opus, and we copy all this.
29:47This is not what we want to see. So it's
29:48just a bunch of extra information, and
29:49we want to see a concise, simple
29:51response. Really just get to it, and
29:53really I would want to see something
29:54like this. A little bit more broken up,
29:56easier to read. Even Fable has some
29:58long, kind of run-on sentences here.
30:00Might break this up a little bit more
30:01into a couple section. And then we would
30:03fire this off. We have a new version
30:04here, let's go ahead and run this. Super
30:06cracked senior Opus. And then we can run
30:08our comparison here. So now our agent
30:11has examples of how we like to
30:13communicate, it has reference points. We
30:16have cleaned up the language, we taught
30:18it to respond concisely, with fewer em
30:21dashes. It's not load-bearing anymore.
30:24We have reference points. This one does
30:26not have reference points, so thankfully
30:28we've encoded a way to add those, so we
30:30can just type ref, and now our short
30:32aliases are going to expand the
30:34response. There we go, you can see we
30:35now have references. You'll notice this
30:38response from smartass took 41 seconds.
30:40Our original response over here took
30:42just 22 seconds. So, we are saving our
30:45output tokens. And again, if you need to
30:47spend output tokens, spend them. But, in
30:49this case, you know, when we're
30:50summarizing things, when we're just
30:51getting concise responses, we don't want
30:53all of that. You know, you can see here,
30:54we have a nice response. We use our ref
30:56alias, which we have encoded into the
30:59system prompt. If we just search for ref
31:01equals, you can see exactly what this
31:02expands into. You could point this to a
31:04scale, you could point this to a
31:05command, whatever you want to do here,
31:07you can do. My whole point here is to
31:09communicate with you that if you really
31:10want to control your agent and get the
31:12best results, you must know how to use
31:14your tool. A lot of engineers are
31:16over-leveraged on the user prompt, and
31:18you get literally less leverage, you get
31:20less output by fixating on the user
31:23prompt. Opus 5 and all models coming
31:25out, they are going to be smartasses.
31:28Quite literally smart. What we want to
31:29do is keep the smart, but drop the ass.
31:31>> [laughter]
31:32>> With system prompts, you can do that
31:33across every user prompt you write. You
31:35can use these exact prompt engineering
31:37patterns to guide Opus 5 and whichever
31:40state-of-the-art models come next. If
31:41you don't like this response, we can
31:42shoot up another alias at it. Eli,
31:44explain it like I'm 18, or whatever age
31:47you want to set your response reasoning
31:50level to. You can add an Eli 5, you can
31:52add an Eli 10, whatever you want to do.
31:53Great engineering is about great
31:55communicating, right? Great
31:56communication with your team of
31:58engineers, with your agents, and most
32:00importantly, with yourself. And this is
32:02how you can tap into that. You've
32:04noticed this if you're using agents on a
32:06daily basis. For many tasks, you and I,
32:08the developer, are the bottleneck. It's
32:10not the model, it's not the tools, it's
32:11not anything else. The hard part now is
32:13communicating quickly, concisely, and in
32:17the most value-accretive way. I'll leave
32:20this cracked senior engineer prompt
32:22linked in the description for you. It's
32:24super, super simple. You can do the
32:25comparison side by side and see yourself
32:28improving every time you add something
32:30by running the just compare command.
32:33I'll also add my classic read me and
32:35just install and all the quick start, so
32:37that you don't have to waste your time
32:38doing any setup and configuration. I
32:40want this to be as simple and quick and
32:41easy for you to use. On the channel,
32:43we've been focused on much higher levels
32:45of abstraction on top of agents like
32:48software factories, agent sandboxes. Uh
32:50check out last week's video if you want
32:51to dive into some really advanced
32:53agentic engineering concepts. I'll link
32:54that in the description as well. But,
32:56every once in a while, it's going to be
32:57really important for us to step back
32:59into the fundamentals. You don't want to
33:00miss the atomic units, the primitives
33:03that make up the larger powerful
33:05composition pieces because if you stack
33:08a bunch of agents together and you burn
33:09a bunch of tokens doing in half the time
33:11by just putting together the right
33:13system prompt, you want to be doing that
33:15instead. Don't waste time using really,
33:17really clever solutions when there's a
33:18much simpler solution. Engineering's
33:20about getting the job done, not looking
33:22cool, not doing whatever everyone else
33:24is doing, [music] not listening to the
33:25idiots that say prompt engineering is
33:27dead. That returns false. Your agent is
33:29just another tool. If you understand
33:31your tools, you'll [music] understand
33:32the results you can get from your tools.
33:34Use your system prompt as the law for
33:36every task [music]
33:38you send to your agents and use your
33:39user prompt for individual tasks. If you
33:42got value out of this video and you made
33:44it to the end, like, subscribe, drop a
33:46comment and let me know [music] how
33:48you're leveraging your system prompt to
33:50get outsized results and to clean up
33:52some of the insane verbosity from models
33:54like Opus 5. You know where to find me
33:57every single Monday. Stay focused and
33:59keep building.

About this transcript

This page contains the full transcript of FIXING Opus 5: PROOF that Prompt Engineering IS NOT DEAD by IndyDevDan, generated from the public captions YouTube serves with the video. The transcript has 6,821 words across 993 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.