YT YouTube2Text

YouTube2Text → Transcripts → How to Get PERFECT Voice Consistency in Your AI Videos (NEW Method)

How to Get PERFECT Voice Consistency in Your AI Videos (NEW Method) — Transcript

by Jack Vs. AI · 3,051 words · 456 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00I've just fixed one of the biggest
0:01problems for AI video because you can
0:04now take any voice, including your own,
0:07and use it in your generations. Every
0:09example you're hearing right now was
0:11built using the same simple process. So
0:14instead of settling for generic AI
0:16voices, accents that don't fit the
0:19character, or trying to redub everything
0:22afterwards, you can actually direct the
0:24dialogue before the video is generated.
0:26That means films, product ads, and UGC
0:29style content where the voice feels
0:32intentional and matches the character.
0:34I'm Jack. I've spent over a decade
0:36creating visual effects for many of the
0:38biggest brands on the planet. And now
0:40I'm using AI to reinvent creative
0:43workflows. We're reaching a point where
0:44stunning visuals aren't enough. If
0:47everyone has access to the same models,
0:49then performance, identity, and taste
0:52become the difference, and voice is a
0:54huge part of that. So, in today's video,
0:57I'll show you how to take any dialogue
0:59and build it directly into your AI
1:01generations without the need for complex
1:03editing afterwards. If you do enjoy
1:05today's content, be sure to whack a like
1:07on there for me and consider
1:08subscribing. All right, let's get into
1:10it. The first step is sourcing your
1:12dialogue. And there are a few different
1:14ways to approach this, including both
1:16traditional and AIdriven methods. But
1:19first, I've had an idea set in a
1:22dentist's waiting room featuring myself,
1:25a claymation wizard, and a cartoon
1:28monster. I want each character to have
1:29their own unique voice, which remains
1:32consistent throughout the scene. So,
1:34with that in mind, we'll start with my
1:35favorite method for sourcing the
1:37dialogue, and that's recording yourself
1:39or someone else performing the scene. A
1:42lot of the problems people run into with
1:44AI are caused when they don't provide it
1:47with enough direction or information and
1:50the AI has to guess. With this method,
1:52we're showing the model exactly how we
1:54want the scene to be delivered. So,
1:56we're reducing the margin for error.
1:58Now, you don't have to use a microphone
2:00this expensive. You could pick up
2:02something like this instead for
2:04relatively cheap and link that up to
2:07your phone. But do keep in mind with
2:09this method, the quality of the
2:11recording is going to have a profound
2:13effect on your video generation. To show
2:15you what I mean, here's a clip from the
2:17intro using a recording from this
2:20microphone. You can actually direct the
2:22dialogue before the video is generated.
2:25It sounds pretty good, right? But let's
2:27compare that to a separate generation
2:29that used a recording I took on my
2:32phone. You can actually direct the
2:34dialogue before the video is generated.
2:37As you can see, not the best result. And
2:39the AI is even changing the voice
2:42towards the back end of the clip. So, I
2:44guess the downside of sourcing your
2:46dialogue this way is that having a good
2:48quality recording is important. The
2:50upside is that it gives you complete
2:52control over your end result. Let's say
2:55either yourself or somebody that you
2:57know is pretty good at voice acting.
2:59Using this method, the performance is
3:02going to be integrated very faithfully
3:04into your AI videos. And I'll show you
3:06exactly how to achieve this in just a
3:09minute. And of course, you're also not
3:10having to spend any credits in order to
3:13generate your speech. But let's say that
3:15you can't record your dialogue and want
3:17to use AI instead. Well, to generate the
3:20audio and visuals for this project, I
3:22used Open Art. And if you guys wanted to
3:24follow along, you'll find a link in the
3:26description. Inside my project, I'm
3:28going to head over to the left hand side
3:30[music] and click the audio tab. Now,
3:32you can see that 11 Labs has been
3:34selected by default. Next, I'm going to
3:37copy a line from my script and [music]
3:39paste it directly into this window. Now,
3:42for the voice itself, there is a pretty
3:45huge selection here. So, it's helpful to
3:47use these filters at the top. I'm going
3:50to select accent and then pick British.
3:53Now, I'm looking to voice my wizard
3:55here. So, let's try the first option
3:58named Cornelius.
3:59>> Conservatives cherish the echoes of
4:02steadfast wisdom.
4:04>> Perfect. I think that suits [music] the
4:05character really nicely. So, I'm going
4:07to select this particular voice. And
4:09you'll see that we've got some extra
4:11options here if you wanted to have even
4:13more control over your results. I'm just
4:16going to leave these on default and go
4:18ahead and hit generate. Let's take a
4:20listen. Well, my boy, once you reach 500
4:24years of age, not even magic can
4:27preserve your teeth.
4:28>> Okay, this sounds like a wizard to me,
4:31so I'm going to get it downloaded. Now,
4:33we've got a human voice and an AI voice,
4:36but [music] there's one question I get a
4:38lot. Do you really have to perform or
4:40generate every single line of dialogue?
4:43Or can one generic example teach the AI
4:46your voice and use that to generate any
4:48speech going forward? Now, obviously,
4:50there are ways to train AI tools to
4:52build a clone of yourself or a
4:55character, but I sometimes find that the
4:57results from this approach can be a
4:59little strange, especially when it comes
5:01to emotional delivery. An alternative is
5:04to produce a vocal procedy reference or
5:07a VPR. This sounds really complicated,
5:09but I promise it's actually pretty
5:11simple. I'll take one for the team here
5:13and embarrass myself by showing you
5:15mine. Hi, this is my voice, measured and
5:18natural. Would you even recognize it?
5:20Quick, grab the camera, check the shot.
5:23Three hours gone. Okay, worst take ever.
5:26Everyone out. The boat's sinking.
5:28Essentially, this is a collection of
5:30short snippets of speech with different
5:33emotional deliveries. And we can use
5:35this as a [music] reference in our video
5:37generations to produce entirely new
5:40lines of dialogue for a specific voice,
5:42including one of our AI characters, if
5:45we want to. Let me show you how. We'll
5:47first want to switch over to the video
5:49tab and select seed dance 2.5 draft mode
5:53as the model. That's because we're only
5:55interested [music] in the audio here.
5:57So, we don't need to spend extra credits
5:59on a high resolution output. I've
6:01uploaded this image of the female
6:03monster as a reference. And the prompt
6:06describes the VPR and the character's
6:08voice. I'm going to go ahead and send
6:10this off. Let me show you the result.
6:12>> Hi, this is my voice. Measured and
6:15natural. Would you even recognize it?
6:17Quick, grab the camera. Check the shot.
6:193 hours gone. [laughter]
6:22Okay. Worst take ever. Everyone out. The
6:25boat's sinking.
6:26>> Nice. Now we have an AIdriven VPR for
6:29the monster and 11 Labs generation for
6:32the Wizard and a real voice recording
6:34for me. Finally, you want to pull these
6:36recordings into some editing software.
6:39I'm using Premiere Pro, but you can use
6:41any tool that you'd like. And
6:43essentially, we're going to export each
6:45voice as a separate blank video. And
6:47this is really important for when it
6:50comes to generating our AI videos, which
6:52we'll look at in just a minute.
6:54Ultimately, you're coming away from this
6:56step with a collection of separate blank
6:58videos, which contain our audio
7:00references for each character. Using
7:03audio files like an MP3, for example,
7:06doesn't work nearly as well, and this is
7:08a common mistake people [music] run
7:10into. Now, let's start looking at how
7:11we're going to build the scene with
7:13these consistent voices. First, I'm
7:15going to upload all of my image
7:17references. So, that's this key visual I
7:19generated, which gives Sea Dance an
7:21understanding [music] of the layout of
7:23my scene. I've also dropped in these
7:25character reference sheets for the
7:27wizard, the monster, and myself. Now,
7:30the prompt for this is pretty big, so
7:32let me go ahead and break it down.
7:33[music] It starts by tagging each of our
7:36references and assigning them a role.
7:39For instance, we're saying that image 4
7:41is the location and blocking reference
7:44and that image three represents the
7:46wizard. Next, we get into actually
7:48describing what we want to happen in the
7:50scene as well [music] as the specific
7:52lines of dial. And you'll notice that
7:55after each of these, [music] it says
7:57ensure the voice matches. And here we
8:00want to go ahead and tag the relevant
8:02[music]
8:02reference. The first line is delivered
8:04by me and I'm saying, so what are you
8:07guys in for? I'll go ahead and upload my
8:09blank video [music]
8:10which contains the recording of me
8:12delivering this line and then I'll tag
8:15that video in the prompt. It's then a
8:17case of repeating this process for every
8:20line of dialogue. We can go ahead and
8:22tag the speech for the wizard that we
8:24[music] generated using 11 Labs. Again,
8:26this is where he's delivering the exact
8:29line in the script. And we can finish by
8:31tagging the VPR for the monster
8:33character. As a reminder, this is a
8:36generic audio reference where the
8:38monster isn't reading the script, but
8:40instead is performing a range of
8:42emotional deliveries. We're telling Sea
8:44Dance to take that generic reference and
8:47use it to produce entirely new lines of
8:50dialogue. Theory here is that you
8:51haven't got to regenerate or re-record
8:54this audio reference every single time.
8:57You just make your VPR once, upload it,
8:59and tell Seance what you want the
9:01character to say. In terms of the output
9:03settings, we're once again going to be
9:05using Seance 2.5's draft mode, which
9:08outputs at 480p. This cuts your credit
9:11cost down massively because if we need
9:13to make any tweaks or refinements,
9:15[music]
9:16we can do this before committing to a
9:181080p generation and spending more
9:21credits. We also want to bump up the
9:23duration to 30 seconds because this is a
9:26fairly complex scene. With that being
9:28said, let's go ahead and hit generate
9:29and take a look at the result.
9:35>> [sighs]
9:36>> So, what are you guys in for?
9:38>> Well, my boy, once you reach 500 years
9:42of age, not even magic can preserve your
9:45teeth.
9:47>> Fair enough. How about you?
9:48>> Well, I'm not 500 years old, but I have
9:51developed quite the liking for human
9:52flesh. Although, it's not the flesh
9:54troubling my teeth, it's the bones.
9:58>> You know what? I think I'm going to
10:00reschedu. Okay, this is working really
10:03well. You'll notice that both the
10:05dialogue from myself and the wizard
10:07match the audio references exactly
10:09without the need for any additional
10:11editing or dubbing in post. This
10:14genuinely sounds like I am in the scene
10:17and the wizard matches our 11 Labs
10:20generation perfectly. The monsters
10:22dialogue was generated completely from
10:24scratch using that VPR and again this is
10:27working really quite well. So, now that
10:29we're happy with our draft, we can go
10:32ahead and click [music] generate in
10:331080p. And I want to be clear that this
10:36isn't a simple upres. This is going to
10:39build us a completely new generation
10:41that remains incredibly faithful to the
10:44draft while fixing any errors. But using
10:47seed dance, we're going to be
10:48referencing those highquality images of
10:51each character and the location so the
10:54video model hasn't got to guess. Let's
10:56send this off and I'll show you the
10:58result.
11:02[sighs]
11:03So, what are you guys in for?
11:06>> Well, my boy, once you reach 500 years
11:09of age, not even magic can preserve your
11:12teeth.
11:14>> Fair enough. How about you?
11:16>> Well, I'm not 500 years old, but I have
11:19developed quite the liking for human
11:20flesh. Although, it's not the flesh
11:22troubling my teeth. It's the bones.
11:26You know what? I think I'm going to
11:27reschedule. So, there you go. A really
11:31highquality scene featuring multiple
11:33characters that have their own
11:35distinctive voices which remain
11:37consistent. And let's also think about
11:39the bigger picture here. If you're
11:41building out a short film or even a
11:43series, you can use this same method to
11:46ensure your voices stay consistent
11:48across multiple generations, making this
11:51an incredibly powerful workflow. I just
11:53briefly want to return to this VPR to
11:56test out the version I recorded of
11:58myself. Can Seance take a human VPR and
12:01use that to generate new lines of
12:03convincing dialogue? I'm going to upload
12:05my VPR which is in this blank video.
12:09This image I generated of myself
12:11alongside my character reference sheet
12:14using the same prompting method as the
12:16previous generation. Let's go ahead and
12:19send this off and take a look at the
12:21result. Everything you're hearing right
12:23now was generated by Cedence 2.5. Okay,
12:27I think we've hit a bit of a wall here.
12:30And this shows us an important
12:32distinction. A VPR works for a synthetic
12:35or AI voice because it's just inherently
12:37more forgiving. Nobody knows what my
12:39cartoon monster is [music] supposed to
12:41sound like. In reality, her voice is
12:44pretty generic. But you've just spent an
12:45entire video listening to me talk, so
12:48you can tell straight away this [music]
12:50is not how I sound. Human speech is very
12:52unforgiving. And so the key takeaway
12:55here is that wherever [music] possible,
12:57you should be recording your dialogue
12:58for real and using that as the
13:00reference. Even my pronunciation of the
13:03word sea dance is pretty weird. And as I
13:06mentioned at the start of the video, a
13:08lot of these problems are caused by us
13:10leaving things to chance. We are asking
13:12the AI video model to guess what I would
13:14sound like rather than just giving it
13:16the performance. There are definitely a
13:18lot of useful lessons to learn from this
13:21project. To finish up, let me go ahead
13:23and share some bonus tips. One thing I
13:25found during this [music] process is
13:27that if you're trying to generate too
13:28much action alongside your dialogue, you
13:31may lose your lip sync. One of my early
13:34generations for the fighter jet scene
13:36from the intro is a great example of
13:38this. Let me show you. So, instead of
13:40settling for generic AI voices, accents
13:43that don't match the character, or
13:45trying to redub everything afterwards,
13:49as you can see, zero lip sync going on
13:52here. And I think that's because we've
13:54just given Sea Dance too much to think
13:56about. The way I fixed this was by
13:58stripping back the prompt and making the
14:00action more simplistic to allow Sea
14:02Dance to focus on the lip sync. I also
14:05found dialogue that's too long can
14:07introduce issues often towards the back
14:10end of the generation. Here's an
14:12example. So instead [music] of settling
14:14for generic AI voices, accents that
14:17don't fit the character or trying to
14:19redub everything afterwards. So it's
14:23subtle, but you'll notice that the lip
14:25sync starts to drift off towards the
14:27tail end. And I also repeat the word
14:30thing. The way that I solved this was
14:32simply by reducing the length of the
14:34recording [music]
14:35I was uploading as a reference. I didn't
14:37actually need the entire sentence for my
14:40edit. So simply by cutting it down and
14:43simplifying the generation, the problem
14:45was solved. Interestingly, you can also
14:47run into similar problems if the
14:49reference you upload is too short. Let
14:52me show you an example of what can
14:54happen here. Product ads and UGC style
14:58content. I found that if the recording
15:00[music] was less than 3 seconds long,
15:02Seance can do some really strange stuff.
15:05Simple fix once again, just by making my
15:08audio reference slightly [music] longer.
15:09I essentially uploaded more of the
15:11complete sentence even though I didn't
15:14need the entire thing for my finished
15:16edit. That made the generation work and
15:18I just [music] cut out the extra
15:20dialogue in Premiere Pro. And the final
15:22point I want to end on is you need to be
15:24really careful with your prompter. In
15:26this example, I started off describing
15:28[music] the dialogue with speech marks
15:30around it as usual. But then further
15:33into the prompt, I started using speech
15:35marks [music] again to emphasize the
15:38word direct. And this confused seance
15:412.5. Let me show you what happened.
15:44Direct.
15:47Direct. So, this was just a case of
15:49cleaning up the prompt and only using
15:52the speech marks around the dialogue and
15:54being very clear at the start of the
15:56prompt about what the character needs to
15:59say. I think a lot of these problems
16:01come down to the fact that AI is
16:03obviously incredibly intelligent, but
16:05doesn't really have any common [music]
16:07sense. While Seance 2.5 is very
16:09advanced, you do need to be incredibly
16:12literal with the tool. So definitely
16:15sense check your prompt before spending
16:17your credit. Try to put yourself in the
16:20position of an AI tool that has zero
16:22understanding of what you want beyond
16:25the information you're supplying. And
16:27once you start thinking like that,
16:28you're going to get the results you want
16:30much quicker and therefore spend [music]
16:32fewer credits. So to sum everything up,
16:35recording voices for real rather than
16:37generating them gives you more control
16:39and of course costs zero credits. You
16:42want to export each recording as a blank
16:45video and not an audio file. When you're
16:47prompting, only use speech marks [music]
16:50for the dialogue. If you use them
16:52elsewhere, that can really confuse seed
16:54dance. And remember to tag the matching
16:56voice reference after [music] each line
16:58of dialogue. Too much action alongside
17:01the dialogue can kill your lipsync.
17:03Instead, try to simplify any action so
17:06the model can focus on the speech. And
17:09finally, audio references that are too
17:11long or too short can cause drift and
17:14glitches. So, it's about finding the
17:16right balance. With that being said,
17:18guys, we have reached the end of today's
17:20video. If you have enjoyed it, be sure
17:22to whack a like on there for me and
17:24consider subscribing. I'll go ahead and
17:26share some more of my [music] recent
17:27content if you wanted to take a look.
17:29And with that being said, I'll catch you
17:31on the next one.

About this transcript

This page contains the full transcript of How to Get PERFECT Voice Consistency in Your AI Videos (NEW Method) by Jack Vs. AI, generated from the public captions YouTube serves with the video. The transcript has 3,051 words across 456 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.