How To Animate a Short Film with Blender + Higgsfield (Breakdown) — Transcript
Full transcript
- 0:00Hi, I'm Amina, and welcome to the first
- 0:02episode of Hixeld Animation.
- 0:11This channel is dedicated to showcase
- 0:12our projects and dive deeper into our
- 0:15processes and pipelines in a form of a
- 0:17tutorial.
- 0:18Today, we're looking at our upcoming
- 0:20animated short called Passport Crush.
- 0:23It's a 5-minute action comedy about a
- 0:25girl who arrives at the airport and
- 0:27suddenly realizes she has left her
- 0:29passport across town.
- 0:31Now, she needs to go back for it and
- 0:33things get a bit out of hand. We played
- 0:35around on a relatable experience with
- 0:37absurd exaggerated moments of how the
- 0:39whole world seems to suddenly be against
- 0:41you when you're in a rush, especially
- 0:43when you're trying to make your flight.
- 0:45We also wanted to show you how we're
- 0:47combining traditional animation with AI
- 0:49and what that means during an actual
- 0:51production. Our team comes from
- 0:53traditional animation and we hadn't
- 0:54worked with AI before joining Hixeld.
- 0:57There's been a lot to learn, to tell you
- 0:58the truth, and we're still figuring out
- 1:00how these tools fit into the way we want
- 1:02to work. If you're exploring them, too,
- 1:04you might be working through some of the
- 1:05same questions and we'd like to share
- 1:07our experiences and hear what you're
- 1:09learning as well.
- 1:10So, on this channel, we'll break down
- 1:13our own projects through practical
- 1:15tutorials showing how we're fitting AI
- 1:17into our animation process without
- 1:19losing the craft. We'll explain the
- 1:21creative decisions behind the results,
- 1:22too,
- 1:23because our artistic voice is the most
- 1:25important part of our work and that
- 1:27continues to guide how we use these
- 1:29tools. The film is still in production
- 1:31and once it's finished, I'll share a
- 1:33full breakdown. For today, though, let's
- 1:35watch the car sequence and go through
- 1:37how we made it.
- 2:24>> Pre-production starts with an idea, a
- 2:26script, and storyboards as it usually
- 2:27does. For the sequence, we wanted to
- 2:29test how 3D previous and AI could work
- 2:32together in an action scene. Our
- 2:34storyboard artists began with the part
- 2:36where the girl goes into the taxi and
- 2:38the driver takes her through the busy
- 2:39traffic. We wanted to bring in some
- 2:41visual comedy and make the car feel
- 2:43jumpy and unpredictable, moving in a way
- 2:46a real car couldn't. We also needed to
- 2:48work out the art direction and prepare
- 2:50the assets, the character sheets, prop
- 2:52designs, and locations so that they
- 2:54would become our image inputs during
- 2:56generation, and we spent really good
- 2:58amount of time getting those right.
- 3:00The assets needed to read clearly and
- 3:02feel like they belonged in the same
- 3:04world. The production structure was
- 3:06familiar to how it is in traditional
- 3:07pipeline. We had a small team though, so
- 3:10people were wearing multiple hats and
- 3:12taking on several responsibilities.
- 3:14Everyone wrote prompt as part of their
- 3:16own work, whatever department they were
- 3:17in. We didn't have a separate prompt
- 3:19engineer, so the artists working on the
- 3:21characters, environments, and animation
- 3:24were writing the prompts for those parts
- 3:25of the project themselves. Let me show
- 3:27you the boards. You'll notice some
- 3:29panels are marked orange. Orange means
- 3:31Blender. Those are the shots we were
- 3:34planning to have a previous stage for,
- 3:36and the ones that are left blank will go
- 3:38straight to video generation. Our
- 3:40thought process was this: complex shots
- 3:42with specific movement, camera angle, or
- 3:44anything that takes longer to explain in
- 3:46words to model goes to Blender. Less
- 3:48complicated shots go straight to video
- 3:50generation.
- 3:52And one thing worth mentioning, boards
- 3:54are just a guide, not something set in
- 3:55stone. So, video generation tends to be
- 3:58very unpredictable, and we tried to
- 4:00leave some shots up to a chance,
- 4:02sticking to the story beats and main
- 4:04scenes rather than planning out every
- 4:06single shot because we expected things
- 4:08to change as we go.
- 4:10Now, let's talk about art direction.
- 4:12Here's the one rule that this whole
- 4:14pipeline is built on. We call it
- 4:16asset-first approach. Essentially, the
- 4:18quality of the image inputs dictates the
- 4:20quality of the overall film. If your
- 4:23image inputs are unclear, sloppy, or
- 4:25inconsistent, your overall film is going
- 4:27to be sloppy. You might be thinking,
- 4:29"Amina, what in the world are image
- 4:32inputs?" Well, those are character
- 4:34sheets, prop sheets, backgrounds, and so
- 4:36on. For people who prepare those assets,
- 4:39we came up with a rule called asset
- 4:40managers. Their work essentially
- 4:42includes making sure that characters,
- 4:44props, and backgrounds all feel like
- 4:46they belong in the same film and the
- 4:48same world, which is equivalent to what
- 4:50an art director does. For this project,
- 4:52we went with stylized 3D characters and
- 4:55watercolor backgrounds. Getting those
- 4:57two looks to work together well can be
- 4:59tricky in a traditional pipeline.
- 5:01However, that was a great challenge for
- 5:02this project. When it comes to our
- 5:04characters, we wanted them to be cute
- 5:06and funny with proportions that suited
- 5:08the exaggerated movement we had planned
- 5:10for the film. The comedy needed to come
- 5:12through in the design as well as the
- 5:14character performance.
- 5:16Our project schedules are usually very
- 5:18tight, sometimes only a month long.
- 5:20To get started, we generate some initial
- 5:22designs from prompts, then edit them by
- 5:25hand to save some time. That gives us
- 5:27different options to compare and develop
- 5:29before we settle on a single direction
- 5:31and idea. I'm going to show you an
- 5:32example of how we write a prompt for our
- 5:34character sheets. Usually, we stick to
- 5:37Claude to write prompts, but you can use
- 5:39any other AI chatbot that you like. Let
- 5:41me show you one of our requests followed
- 5:43by an example of a a character prompt.
- 5:46So, let's look at what's in that long
- 5:49prompt that describes the character's
- 5:52appearance, clothes, and pose, along
- 5:54with the framing and lighting.
- 5:56Claude helps us organize all of that
- 5:58information, but we still read through
- 6:00it before generating from time to time.
- 6:02We check that the details match our
- 6:04designs and correct anything that
- 6:06doesn't match. Those choices affect what
- 6:08comes out, so we want to be clear about
- 6:10what we're asking for.
- 6:12When it comes to image generation, we
- 6:14use our own image model called Soul 2.0
- 6:17that is available at Hicksfield. That's
- 6:19the model we went for to create all of
- 6:21the assets for this project. Our
- 6:23creative department, alongside our
- 6:25engineering team, developed this model.
- 6:27It was created by people with
- 6:29backgrounds in film, animation,
- 6:31photography, fashion, and graphic
- 6:33design, who helped shape the model's
- 6:35visual direction and the quality that we
- 6:37have. They wanted it to be useful for
- 6:39the kind of work they do in their own
- 6:41industries with attention to taste,
- 6:43style, and high visual quality that
- 6:46matters in those fields. Let me take you
- 6:48through the generation step.
- 6:50I open Cinema Studio on Hicksfield and
- 6:52choose Soul 2.0 in the model picker.
- 6:55Then,
- 6:56I paste the prompt, set the aspect
- 6:58ratio,
- 6:59choose a batch of four, and hit
- 7:01generate. Here's what came back. First
- 7:03one looks good, but she was a bit too
- 7:06goofy. We wanted funny, but also
- 7:08something still cool.
- 7:10And the hair and outfits don't quite
- 7:12feel right in my opinion, especially it
- 7:13comes to main character. I think they
- 7:15need to stand out more.
- 7:17This one is cute, but a little too
- 7:18realistic for the film style. We want a
- 7:21more expressive face and exaggerated
- 7:23proportions. This one was much closer.
- 7:25The ginger hair really stands out
- 7:27against the watercolor backgrounds,
- 7:28which I really like. Her proportions,
- 7:30however, still need some work.
- 7:33And I think she looks a little too
- 7:34dressed up for a flight. I don't know
- 7:36about you, but I usually pull up in
- 7:38sweatpants and most comfy clothes to the
- 7:40airport. So, this is the version we went
- 7:43with. We brought back the hoodie and
- 7:45sneakers from the first design and added
- 7:48the skirt for the silhouette.
- 7:49The contrast between her ginger hair and
- 7:52blue hoodie looks really great, too, and
- 7:55I kind of like it.
- 7:56Now,
- 7:57here comes the fun part, tweaking the
- 7:59details by hand.
- 8:01First thing that caught our eye was that
- 8:02her color palette seemed too dull,
- 8:04especially for that comedic energy that
- 8:06we're trying to go with for our story.
- 8:08So, we tweaked the brightness and
- 8:10saturation a bit in Photoshop to bring
- 8:12back all that color and brightness.
- 8:14We also decided to add some paint
- 8:16strokes around her silhouette to give
- 8:18off that hybrid look. I think those
- 8:20details add some dynamic and expressive
- 8:23hand-drawn feeling, but otherwise we
- 8:25kept the characters majorly
- 8:27cel-stylistically 3D.
- 8:29Once the main character was locked, it
- 8:31was easier to develop the rest of the
- 8:33characters by the same method, like our
- 8:35taxi driver.
- 8:37Once our main character rushes out of
- 8:39the airport, she gets into a taxi.
- 8:41The taxi driver is a middle-aged man
- 8:43with round, kind face who helps her get
- 8:46through the busy traffic fast.
- 8:48He looks very calm, but his driving
- 8:50style turns out to be very reckless. To
- 8:53be honest, I find that this character is
- 8:55the driving force, no pun intended, of
- 8:57the majority of our action scenes. We
- 9:00used the same process to develop his
- 9:01design, starting with a short
- 9:03description in Claude. Here's our
- 9:05request, followed by an example of the
- 9:07full character sheet prompt. In this
- 9:09one, the cheeks were a bit much here, I
- 9:11think, and I'm not sure what happened
- 9:13with his hair. This one is a different
- 9:16hairstyle in every view. This one was
- 9:18closer, but the skin and facial hair
- 9:20still felt too plasticky for our style.
- 9:22He also looks a little tired and
- 9:24worried, and we wanted him to look much
- 9:26more relaxed considering what he's about
- 9:29to put this girl through. This is the
- 9:30one we chose eventually. I love the cap
- 9:33and the big mustache. He looks friendly
- 9:35and very pleased with himself. I'll get
- 9:37in his car.
- 9:39Maybe once.
- 9:41Probably only once. Here he is after the
- 9:43hand-drawn details. We added little
- 9:45strokes around his face, on the cap, and
- 9:48through the clothes, just like we did
- 9:49with the girl. I especially like how
- 9:52they bring out the shape of his
- 9:53mustache. It looks very expressive. He
- 9:56still has that 3D look, but the John
- 9:58details helped the both characters feel
- 9:59like they belong in the same film.
- 10:02Main characters down. Now, off to
- 10:04locations. Now, since this sequence is
- 10:07right out of the airport, our location
- 10:08had to be the road where our characters
- 10:10drive through to get to the city. It's a
- 10:12long highway away from any buildings.
- 10:15Same process as with the characters, a
- 10:17short request sent to Claude, and it
- 10:19comes back as the full location prompt.
- 10:22Locations gave us the project's biggest
- 10:24style lessons by being quite challenging
- 10:26to execute. We wanted to go with pure
- 10:28watercolor visuals, but they ended up
- 10:30being sloppy, looking dirty, and
- 10:32soulless during tests. Our team kept
- 10:35hitting a dead end, and at some point we
- 10:37even seriously discussed dropping the
- 10:39watercolor look altogether and pushing
- 10:41the whole film towards something more
- 10:423D. But, we didn't give up yet and kept
- 10:45testing out both options to find our own
- 10:47balance between watercolor and 3D looks.
- 10:50Sometimes, you have to make a lot of
- 10:51mistakes before finding the right
- 10:52direction to go with. We still went
- 10:54through plenty of versions, but we could
- 10:56explore more ideas in the time we had.
- 10:59Even the ones we didn't use helped us
- 11:00rule things out and settle on an art
- 11:02direction sooner before we carried those
- 11:04choices into the rest of the production.
- 11:06The initial time of our film was midday,
- 11:08and that resulted in flat and dull look.
- 11:11It seemed like lighting was a big factor
- 11:12that we had to consider. Duh. So, we
- 11:15decided to keep the location, but settle
- 11:17on late afternoon lighting instead,
- 11:19adding a more dynamic light through
- 11:20prompt. Warm evening light and harsh
- 11:23shadows brought more volume and depth to
- 11:25our shots.
- 11:26This fix became the film's visual
- 11:28formula. Essentially, you generate
- 11:30location in watercolor style, then add
- 11:32in the golden hour lighting and prompt
- 11:34during production.
- 11:36That's why testing out assets in video
- 11:38early on in pre-production is very
- 11:40crucial. It allows us to face visual
- 11:42problems while we still can fix them
- 11:44quick. That is equivalent to animation
- 11:46test I'd say that we do in
- 11:47pre-production stage to test out
- 11:49characters. Except in this pipeline, we
- 11:52can test backgrounds too because due to
- 11:54AI's nature, backgrounds can influence
- 11:56the scenes in interesting ways like it
- 11:59did with some of our props like the taxi
- 12:01car. The taxi's present in a big chunk
- 12:03of the movie so we took our time with
- 12:05designing it in a way that made it feel
- 12:07like a character of its own.
- 12:09Our very first 3D tests were of this
- 12:10very taxi car driving through the road
- 12:13where our animators explored the way
- 12:14it's going to move. The car needed to be
- 12:17jumpy, have a good amount of squash and
- 12:18stretch, and complement its owner's
- 12:20personality almost like a horse and its
- 12:22rider. For the shots that have this car,
- 12:24we planned to make a 3D previous first
- 12:27and then attach it as a video input
- 12:28during production to keep the movement
- 12:31and bring more control to the animation
- 12:33itself. That way, we could work out the
- 12:35movements ourselves and give the model
- 12:37something to follow. When we already
- 12:39know how we want something to move, it
- 12:41can be quicker to animate it than to
- 12:43explain every single detail with words
- 12:45in a prompt. We are visual artists. The
- 12:48taxi also needed to fit into our
- 12:49watercolor world. So at first, we gave
- 12:52it a lot of painted texture. During our
- 12:54first attempts, the car was in a very
- 12:56obvious watercolor style. However, in a
- 12:59video, it looked like these textures
- 13:01laid onto the 3D form incorrectly which
- 13:04made it look cheap like a PNG stretched
- 13:06over 3D geometry. Not good at all.
- 13:09So we kept making iterations. Here's our
- 13:11taxi request so far.
- 13:13A watercolor drawing of an old yellow
- 13:15taxi, 3/4 front left view, single car in
- 13:18frame, rich watercolor washes with drips
- 13:21and highlights, soft shadow under the
- 13:23car.
- 13:24Claude turns that into the full prompt
- 13:26for a prop sheet, which we ran through
- 13:29Soul 2.0 inside Higgsfield. So, why did
- 13:32the watercolor car keep falling in
- 13:34motion?
- 13:35Here's what we figured out after a lot
- 13:37of takes. Heavy watercolor texture is a
- 13:40lot of information for a model to hold
- 13:42once things start moving. Put a
- 13:44watercolor prop on top of a watercolor
- 13:46background, and that's double the load.
- 13:48Two dense textures fighting in every
- 13:50single frame. So, we decided to lighten
- 13:53it up a bit. The car became clean 3D,
- 13:55and the paint stayed in the world around
- 13:57it. After locking the car, we also made
- 13:59a roofless version of it. Here's an
- 14:01example of how you can edit an existing
- 14:03image instead of generating from
- 14:05scratch. This time, we'll go into SeaArt
- 14:075.0 Pro, which we use for edits like
- 14:10this. Here's the request and the full
- 14:12prompt. And we also needed an inside
- 14:14view of the backseat for some of our
- 14:16shots where we used the same workflow of
- 14:18attaching our main car design. To
- 14:21populate the highway, we speed ran the
- 14:23rest of the props. Trucks stayed
- 14:25watercolor since they are on screen
- 14:26briefly and didn't require too much
- 14:29editing.
- 14:30We don't usually build assets for
- 14:31everything. Only for specific props that
- 14:34have to reappear from scene to scene and
- 14:36have to look consistent throughout. For
- 14:38example, we had three trucks that would
- 14:41keep appearing in more than one shot.
- 14:43So, they needed their own prop sheet.
- 14:45The rest can be achieved through text
- 14:47video. Very simple. We also made an
- 14:49effects sheet for our explosion scene at
- 14:51the end of the sequence. That part was
- 14:53very fun and that was an interesting
- 14:55process. You see, our initial request
- 14:58said mushroom-shaped explosion. Little
- 15:01did we know that it will literally
- 15:03generate a mushroom-shaped explosion
- 15:05with smoke.
- 15:07So, we changed the wording to nuclear
- 15:09blast instead, which made it fix, more
- 15:12or less. But, something was still
- 15:13missing. So, we had an idea.
- 15:16We took the base and added some
- 15:17hand-drawn textures to it to match the
- 15:19playful art direction of the film that
- 15:21you saw before on our characters. And
- 15:23thankfully, it made it to the final cut.
- 15:31With most assets locked, we were ready
- 15:33to go into production.
- 15:36Most of the time, production here means
- 15:38one thing. We generate video from text
- 15:41with our assets attached as image
- 15:42inputs.
- 15:44That covers the comp shots, but some
- 15:45shots need one extra stage first, a
- 15:48preview stage, like this scene. Our car
- 15:50gets blocked by two trucks and there's
- 15:52no way around it. But who said we can't
- 15:55pull off a stunt and squeeze right
- 15:57between them? Totally optional, by the
- 15:59way. The preview stage covers two of our
- 16:01needs here. It helps us lock the camera
- 16:03angle and also lock the timing, which is
- 16:06super important. We added some squash
- 16:09and stretch into the wheels, just enough
- 16:11to get rid of the stiffness and add some
- 16:13character to the car. Then, we export
- 16:15the scene as a plain Blender preview. No
- 16:18color, no materials, only the gray geo,
- 16:21motion, and the camera. Now, how do we
- 16:23write the prompt for a video? The way
- 16:26you write your prompt can make or break
- 16:28your scene. It's a whole science of its
- 16:30own, believe me. Let's look at one of
- 16:32the prompts we made.
- 16:33Yes, it's very long. I know you're
- 16:36thinking, "Who writes an essay for a
- 16:384-second shot?" Well, we do, because
- 16:41every detail we don't write, the model
- 16:43decides for us. And trust me, you don't
- 16:46want that. It is a terrible guesser. One
- 16:49thing before we go block by block, the
- 16:51order matters. The first 20% of the
- 16:54prompt matters a lot because the model
- 16:56reads it from top to bottom, like we
- 16:58humans do. And the further it reads, the
- 17:00less strictly it follows the prompt. So,
- 17:03let's go over the blocks that we have in
- 17:05the front. First, we have scene context
- 17:07and style block, which is where we
- 17:09establish what happens in the shot in
- 17:11plain sentences. Plus, this is where we
- 17:13add the style rules, the physics of the
- 17:15world, and so on. Very important. Then
- 17:17we have active references block where
- 17:19your assets are present in the scene in
- 17:21the form of an attached image or videos,
- 17:23like a call sheet. It's important to
- 17:25specify what is what, otherwise the
- 17:28model will try to guess it and again,
- 17:30it's a terrible guesser. This is where
- 17:31you can add the personality, the quality
- 17:33description for the assets, but do not
- 17:36add visual description. That's what the
- 17:38image inputs are for.
- 17:40You can also add this line, characters,
- 17:43environments and props, 100% match the
- 17:45reference. That line carries the
- 17:47consistency so that it sticks to the
- 17:49visual input that you have. Then we have
- 17:51the short structure block. This is where
- 17:53we write down each shot that is divided
- 17:54by hard cuts.
- 17:56This is where we describe the action
- 17:58beat by beat with timing. What happens
- 18:00first, what's next, how long each
- 18:02movement holds.
- 18:04This is kind of the same job as what
- 18:05storyboard panel does, just put in
- 18:08words. Then we have the block for locks
- 18:11and constraints.
- 18:12Video models love adding their own ideas
- 18:15and the constraints keep them on track.
- 18:17These are the rules the model has to
- 18:19follow strictly.
- 18:20Now, constraints vary from project to
- 18:22project, so this is for you to decide
- 18:24what you want to use, but we will always
- 18:26use no music, SFX only, because that
- 18:29helps us with post production. Then we
- 18:31have a block for animation principles.
- 18:34The 12 principles of animation have
- 18:35their own section in the prompt, too. We
- 18:38include squash and stretch,
- 18:39anticipation, follow through and all the
- 18:41other rules and ask for them to be
- 18:43applied in our scenes. The model doesn't
- 18:45follow all of them consistently though,
- 18:47but including them helped us get closer
- 18:49to the movement we wanted. Do we type
- 18:51all of this every single time? Of course
- 18:54not.
- 18:55We wrote the structure down for Claude
- 18:56once and it applied it to every single
- 18:58prompt after.
- 19:00We call it a skill, also known as a
- 19:02system prompt. We tend to share around
- 19:04the same system prompts with the team,
- 19:06so everyone follows a similar structure,
- 19:08more or less. Let's go through the whole
- 19:10process using a shot where the taxi
- 19:12squeezes between two trucks again. We
- 19:14started with the storyboards. Then the
- 19:16taxi approaches the gap, tips onto two
- 19:18wheels, squeezes through, and lands back
- 19:21on all four once it's clear of the
- 19:22trucks. We needed that order to read
- 19:24clearly, especially with everything
- 19:26happening so quickly. Then we worked out
- 19:29the movement in Blender. This is where
- 19:31our animators had fun. We could adjust
- 19:33when the car starts tilting, how long it
- 19:35stays on two wheels, and how it settles
- 19:37after landing. The camera needed to
- 19:39follow all of that without losing the
- 19:41action, too. So working in Blender
- 19:43allowed us to play with the shot back
- 19:45and forth and adjust the camera
- 19:46alongside the animation.
- 19:48The control was super important here.
- 19:50And when we exported a simple gray
- 19:52preview from Blender, at this stage
- 19:54we're using it to show the movement and
- 19:56framing only. Our image references then
- 19:59came over and would provide the final
- 20:01designs.
- 20:02In Hicksfield, we attached that preview
- 20:04as the video input. Then we added the
- 20:06image references for the taxi, both
- 20:08trucks, the characters, the road, and
- 20:10the light. All the necessary stuff. Here
- 20:13we use Claude to help put the scene
- 20:15prompt together. It describes the action
- 20:17and explains what each reference is for.
- 20:20The video guides the movement and
- 20:21camera, while the images show how
- 20:23everything should look.
- 20:25We also specify details like keeping
- 20:27both characters in their seats and
- 20:29waiting until the taxi clears the trucks
- 20:31before it lands. Otherwise, it just
- 20:34keeps breaking. Once we check the
- 20:35prompt, we generated the shot.
- 20:38Then we watched it back against the
- 20:40previews.
- 20:41We checked the timing, the camera, and
- 20:43whether the car and characters still
- 20:44matched our designs, which
- 20:47thankfully it did. Remember the orange
- 20:49marks on the boards?
- 20:50We used Blender for shots where we had
- 20:52specific movement or camera work in
- 20:54mind. If it seemed quicker to animate an
- 20:56action than to explain it in words,
- 20:59we made a preview. For simpler shots, we
- 21:01went straight from boards to video
- 21:03generation, text to video.
- 21:06We still reviewed and adjusted the
- 21:07generations in the same way. Those shots
- 21:10just didn't need that extra blender
- 21:11stage. That's how we approach the
- 21:13individual shots. But, let me go back a
- 21:16little because some of those decisions
- 21:18came from tests we ran before starting
- 21:20the full sequence. Between
- 21:21pre-production and production, we ran a
- 21:23test stage. We took the assets we just
- 21:26made and put them straight into video to
- 21:28see whether it will work or not in the
- 21:30final shot. It is better to find out now
- 21:32and this way while we are still in
- 21:34pre-production stage and not in the
- 21:36middle of production when it gets harder
- 21:38to come back and make edits. You know
- 21:40what I mean.
- 21:41We didn't have any of our rules yet at
- 21:43that point. Most of them came from these
- 21:45three lessons.
- 21:47The lessons and learnings we find during
- 21:49this stage later become our project's
- 21:51visual rules. Here's what we learned.
- 21:53Lesson one,
- 21:54the watercolor car.
- 21:56We talked about this in the taxi section
- 21:58and these are some of our early tests.
- 22:00We used them to work out how much
- 22:02texture do we need to keep on the car,
- 22:04then carry that visual direction into
- 22:05our prompt. That helped us to keep the
- 22:08surfaces consistent so we didn't
- 22:10suddenly get a photorealistic car in the
- 22:12middle of a painted scene.
- 22:14Lesson two, we started testing before
- 22:16the character sheets were ready, so the
- 22:18car at first was empty.
- 22:20Random people kept showing up inside for
- 22:22some reason even though they weren't in
- 22:24our references or prompts or anywhere
- 22:26else. Once we added our characters in,
- 22:28that stopped. Lesson three, the
- 22:30lighting.
- 22:31One of our teammates added evening light
- 22:33to a prompt just to see what would
- 22:35happen. And the shots suddenly started
- 22:37looking warmer and had more depth. So,
- 22:39we kept using it. Eventually, low warm
- 22:42sun and long shadows became the
- 22:44direction we kept going for. We carried
- 22:46that lighting lock into production along
- 22:48with the car and character references we
- 22:50had tested.
- 22:54There are 18 scenes in this sequence. I
- 22:56want to show you three that took some
- 22:58extra work. In this shot, the girl is
- 23:00right next to the truck's door. We had
- 23:02already animated a reference in Blender,
- 23:05but the generations weren't coming out
- 23:06right. We figured that the camera was
- 23:09too close to the action in our Blender
- 23:11preview. It looked fine to us because we
- 23:13knew the scene and could tell what
- 23:15everything was supposed to be, but
- 23:16without textures or facial details, most
- 23:19of the frame was just large gray shapes
- 23:21with very little of the surroundings
- 23:23visible. That made it difficult for the
- 23:25video model to understand how the
- 23:26characters and objects were positioned.
- 23:28So, we went back into Blender and pulled
- 23:31the camera back a little. More of the
- 23:33scene was visible this time. The action
- 23:35was easier to follow, and the next
- 23:36generation came out right. Rule one, if
- 23:39the model doesn't understand your
- 23:40reference, you probably can't see enough
- 23:42of it. Zoom out. In this scene, she
- 23:44fastened both seat belts so they formed
- 23:46an X over her chest. We thought it would
- 23:49be fairly straightforward, especially
- 23:50since we'd already tested the action and
- 23:53it worked. Those first tests used an
- 23:55earlier character design. And when her
- 23:57final design was ready, we regenerated
- 23:59the scene with it again, keeping the
- 24:01same prompt and the other references.
- 24:04But this time, the camera kept changing
- 24:06angles between takes, and we couldn't
- 24:08get the framing we wanted consistently.
- 24:10We had expected to update her appearance
- 24:13and keep the rest of the shot as it was.
- 24:15Instead, we had to spend more time
- 24:17getting the camera right again. We took
- 24:18screenshots from the takes that had the
- 24:20camera angle we wanted and attached them
- 24:22as the references for the next attempts.
- 24:25That helped keep the framing consistent,
- 24:27and we were getting much closer to the
- 24:29shot we wanted. Then we noticed her
- 24:30headphones were missing in every new
- 24:33take. If you look at the screenshot we
- 24:35used, you can see they're missing there,
- 24:36too. We had picked it for the camera
- 24:39angle and missed that detail. Her
- 24:41character sheet still had the
- 24:42headphones, but we were giving the model
- 24:44references that didn't quite match. So,
- 24:47that meant we needed to edit the
- 24:49screenshot that was causing the
- 24:51headphone disappearance and match it
- 24:52with the character sheet. We opened the
- 24:54screenshot in Photoshop, drew the
- 24:56headphones back in by hand, and used
- 24:58that version as the reference.
- 25:00In the next generation, she had them
- 25:02already, and the camera angles were
- 25:03still right. Now, before we finish,
- 25:06there's another shot I wanted to show
- 25:07you, and that one is my favorite.
- 25:10Unfortunately, it didn't make it into
- 25:11the sequence because we haven't managed
- 25:13to get it working yet.
- 25:15We planned a little gag inside the taxi.
- 25:17We thought this one would be easy.
- 25:20Mid-chaos, the girl grabs the ceiling
- 25:21handle, it rips off. She panics, grabs
- 25:24the other one, that one rips off, too.
- 25:26Looking at the boards, we weren't too
- 25:28worried about this shot. I mean, we're
- 25:30making a car chase with flying logs and
- 25:32explosions. Surely we could manage to
- 25:35handle ripping off, right?
- 25:37Now, here's the prompt from one of our
- 25:38attempts. This shot was very easy to
- 25:41underestimate since it required to keep
- 25:43track of the smallest details. A lot of
- 25:45it was about keeping the handles
- 25:47consistent. We had to describe where
- 25:49they were attached, how they came off,
- 25:51and how they should look once she was
- 25:53holding them.
- 25:54I mean, we were also struggling with her
- 25:56performance. She needs to grab the first
- 25:59handle in a panic, then have a moment to
- 26:01realize it's broken before reaching for
- 26:03the other one. That little pause
- 26:06mattered. That small bit of reaction was
- 26:08crucial to deliver the gag. We knew this
- 26:10as animators, but we couldn't get those
- 26:13beats right through text prompts alone,
- 26:15so we turned to Blender. We thought that
- 26:17animating the performance ourselves and
- 26:19using it as a video reference would help
- 26:21it. It should help it. But, that proved
- 26:24to be tricky as well. With a simple gray
- 26:26character in our Blender preview, the
- 26:28expressions and hand movements weren't
- 26:30coming through clearly in the
- 26:31generations. When we added more detail,
- 26:34or at least tried, the model started
- 26:36copying the Blender's geo shape instead
- 26:38of following our actual character sheet
- 26:40and her design. We called this a
- 26:43performance gap.
- 26:44We knew how she should react, where she
- 26:47needed to pause, but we couldn't get the
- 26:49performance we wanted while keeping her
- 26:51final character design consistent in the
- 26:53AI generated shot.
- 26:55What surprised us was how much trouble
- 26:57we had with a scene that seemed fairly
- 26:59straightforward to animate
- 27:01traditionally, to be honest. We had
- 27:03expected the bigger action scenes to be
- 27:04harder, but turns out AI struggles with
- 27:07the subtle character acting instead.
- 27:10Setbacks like this help us figure out
- 27:11what to work on next. We haven't solved
- 27:14every shot, but making the car chase
- 27:16sequence has given us a clear idea of
- 27:18where to focus our next tests.
- 27:21There's still a lot to explore as we
- 27:22finish the film, and we're looking
- 27:24forward to sharing what we discover in
- 27:26the full breakdown. For this sequence,
- 27:28we worked on 18 scenes in just 3 days. I
- 27:31know, crazy timeline.
- 27:34There's still more to figure out as we
- 27:35work on the full film. Once it's
- 27:37finished, we'll share the full Passport
- 27:38Trash breakdown, including all the
- 27:40production numbers, the total number of
- 27:42generations, how much we spent in terms
- 27:44of budget, and how long the whole film
- 27:46took to make.
- 27:48If there's a part of your own workflow
- 27:49you're struggling with, tell us about it
- 27:51in the comments. Let us know what you've
- 27:53tried and where you're getting stuck.
- 27:55That way, we can focus future videos on
- 27:58the things you need help with. Thank you
- 27:59so much for watching, and if you'd like
- 28:01to follow along, subscribe, and I'll see
- 28:03you in the next one.
About this transcript
This page contains the full transcript of How To Animate a Short Film with Blender + Higgsfield (Breakdown) by Higgsfield AI, generated from the public captions YouTube serves with the video. The transcript has 5,154 words across 791 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.