YT YouTube2Text

YouTube2Text → Transcripts → The Matrix Awakens: Creating a World | Tech Talk | State of Unreal 2022

The Matrix Awakens: Creating a World | Tech Talk | State of Unreal 2022 — Transcript

by Unreal Engine · 7,879 words · 935 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:04JEROME PLATTEAUX: Hello.
0:05I'm Jerome Platteaux, the Art Director of the Special Projects
0:08team.
0:08Welcome to the talk 'The Matrix Awakens: Creating a World'.
0:12Epic has been pushing limits of real-time rendering
0:15and demonstrating that Unreal Engine can
0:17be used for many types of projects, from interactive experiences
0:21to linear content.
0:22This year, we created the experience name 'The Matrix Awakens'.
0:27Let's look at the teaser, and then I'll go over the goals and the ideas
0:31behind the project.
0:32[VIDEO PLAYBACK]
0:40- Hi.
0:41I'm Thomas Anderson.
0:42Like many of you, I work with computers.
0:45But computers are also mirrors reflecting back who and what we are
0:50and the choices we make, the world we build.
0:57We still got it.
1:15[END PLAYBACK]
1:16JEROME PLATTEAUX: OK, so let's go over the goals.
1:19First of all, we want to make sure Unreal Engine is ready
1:21for Open World experiences, being able to stream large environments,
1:26and have all the tools ready to manage such a big amount of data.
1:30Then, of course, we want to use Nanite and Lumen
1:33for man-made environment, like a city.
1:37We also have to create a new Mass AI system
1:39to populate the world with crowd and car traffic.
1:43Then, we use the MetaSound system to bring everything to life.
1:47Like every year, we want to keep pushing our MetaHuman technology,
1:50like the way we render it and also the way we capture it.
1:53Then we want to keep pushing our physics system
1:55Chaos to simulate the car physics, the crashes, and the destruction.
2:00Another important part is that we wanted
2:02to utilize the gameplay system to generic cinematic content.
2:06For example, the cinematic artist could use the playable cars
2:09and record themselves driving around the city
2:12and use those techs for cinematic shots.
2:15We also wanted to push our procedural tools
2:18and workflow so that a relatively small team can generate and populate
2:22a large-scale environment.
2:24Then, we wanted this incredible experience
2:26to run on consumer hardware and put it in people's hands.
2:30We had to make sure that everybody who owns the next-gen consoles
2:34can see what Unreal Engine is capable of
2:37and prove that those innovative technologies are available right
2:41now for anybody who wants to use Unreal Engine 5 for their project.
2:46Over the years, we've been creating tech demos
2:49to showcase the latest engine features.
2:51But most importantly is to make sure that those new tools have
2:55been tested on real production.
2:57Now that we have established the goals,
2:59let's see what are the ideas behind the demo.
3:02We got in touch with Lana Wachowski just before she started to film
3:06'The Matrix Resurrections'.
3:08We talked about the project with her and she
3:10was ready to jump on it right after she finished filming the movie.
3:14'The Matrix' universe was a perfect match for us
3:17to talk about the future of graphics and simulated worlds.
3:20The key part was that we wanted to blur
3:23the lines between cinema and games.
3:25The experience starts with a mix of real-time cinematic and live-action
3:29photography.
3:30Then little by little, we added more and more gameplay.
3:33Then we finished on the Open World where
3:36the player can go back and explore the entire city where
3:39the experience just happened.
3:41Now that the demo has been published and Unreal Engine 5
3:44has been released, we want to give away the entire city
3:48to the Unreal Engine community.
3:50You can download the full project in the Epic launcher in the Unreal
3:55Engine Marketplace.
3:57All the assets are available for free.
3:59And you can re-use them for your own project.
4:02Let's go over a few numbers.
4:04The city is made of thousands of modular pieces.
4:07Those pieces have on average 50,000 to 500,000 polygons.
4:13All the assets are Nanite except when they are being deformed
4:17or need to have transparent shaders.
4:20On the texture side, we try to stick with 4K texture resolution.
4:24If we need more resolution, we break the UVs into several UDIMs
4:29and add more 4K textures.
4:31All the textures are converted to virtual textures.
4:34And of course, like most of our project,
4:37we try to leverage the awesome Megascans library.
4:40I'm going to hand it over to Votch who is going to dive deeper
4:43into the project.
4:45VOTCH LEVI: Hello.
4:46My name is Votch Levi.
4:47I'm a senior technical artist in the Special Projects
4:50Group at Epic Games.
4:52Today, I'm going to take you on a tour of the asset and content
4:55pipeline we use to create the city in 'The Matrix Awakens'.
5:00The first thing we did was discuss what we wanted the city to look like
5:03- an urban cityscape with tall skyscrapers and two main city
5:07sections.
5:08We wanted a detail-rich city with a variety of iconic buildings
5:12throughout the city.
5:14We drew inspiration from San Francisco, Chicago, and New York.
5:18We looked for buildings with intricate detail at both
5:21the bird's-eye and also at the street level.
5:24We also wanted buildings we could mix and match
5:27using the upper section of one building
5:29with the lower section of another.
5:31We also identified specific building and street props,
5:35signage that we liked, awnings, anything
5:38that we could fit onto the facade wall of a building.
5:42We identified 24 unique buildings.
5:4510 Chicago-, 8 New York-, and 6 San Francisco-themed We also included
5:50one building from the Quixel Megascans library to ensure
5:53our system and tools would work with module buildings from any source.
5:57With our buildings and props identified,
5:59the next step is to build a content library in Unreal.
6:02This library would feed the Houdini-based Procedural City
6:05Generator.
6:06The results of the City Generator would then
6:08be brought back into Unreal to create the final city result.
6:12We now break down our asset pipeline into its individual components.
6:16We have our content library, which feeds the Houdini City Generator.
6:19We then return into Unreal where we spawn
6:22the results of the City Generator.
6:24The first assets we started building in the content library
6:27were our buildings.
6:28Utilizing Nanite, we wanted to capture
6:31as much geometry detail as possible.
6:33We included building imperfections wear, damage, and bevels in the base
6:37geometry, no displacement required.
6:40All this detail meant it would be prohibitive to construct
6:43the buildings as a single static mesh.
6:46The triangle count would be incredibly high.
6:48And while Nanite could handle the triangle counts,
6:51a single static mesh would limit our ability
6:53to create building variations.
6:55We went back to the reference and took a closer look,
6:58keeping the ideal Nanite workflow in mind.
7:01Buildings have a lot of repetition in windows and walls
7:04and are a perfect case for instancing.
7:07We wanted to be able to reuse the buildings as much as possible
7:10to create detail and variation across the city.
7:13We devised a plan to slice the buildings
7:15into individual modular components.
7:18If we slice up just right, we would be
7:20able to instantiate the modules into an unlimited amount
7:23of unique building shapes.
7:25So we took the reference images and started
7:27defining the modules required to make the buildings.
7:29We organized the modules by Level with a consistent cut line.
7:33All modules within a Level would be exactly the same height.
7:36It's just a construction block game.
7:38Super easy, right?
7:40Well, buildings are complex and have a lot of unique features.
7:45And it's these features that make our reference
7:47buildings look interesting.
7:49To capture all the detail requires a lot,
7:52I mean a lot, of unique modules.
7:55While slicing up the buildings, we started
7:56to realize that even the simple buildings had hidden complexity.
8:01Real-world buildings are built to fit within a specific terrain
8:04or location.
8:05We found that features in the building
8:07often varied from one side of the building to the other.
8:10From a distance, the modules all appear to be the same size,
8:13but actually, many modules that look the same
8:17are many feet different in width.
8:19And none of the buildings had a base module size
8:22that matched a consistent grid.
8:23We reduced the overall module count by consolidating modules
8:27with similar size.
8:28We merged columns and windows or made double-wide modules
8:32whenever possible.
8:33We felt it was the variety of detail and the inconsistencies of size
8:37and shapes that made these reference buildings look interesting.
8:41So we strived to retain all modules with differences in detail
8:44so that we could recreate the buildings as
8:46close to the original as possible using our base modules.
8:50Once we had all the module shapes defined for a building style,
8:53we used a custom-built Module Template Generator
8:55to create a proxy version of each module.
8:58The Template Generator also allowed for defining additional metadata
9:02for the module, including module name, variation, floor,
9:06and dimensions.
9:08All the module templates are organized
9:10into a building kit that represents all the modules required
9:13to recreate the building.
9:15We used these templates as a starting point to create each building style
9:18and began prototyping the procedural building.
9:22We also used these template modules as a starting point in Maya
9:25to create high-resolution geometry.
9:27The templates represented the base shape and dimensions of the module,
9:31but not necessarily the bounding box of the module.
9:33Additional features could push out from the bounds of the template.
9:37This is why we opted to use dimensions derived
9:40from the template geometry rather than rely
9:42on bounding boxes of the high-resolution geometry
9:45for module dimensions.
9:46Many of the buildings have hundreds of template modules and hundreds
9:50of hi-res modules.
9:51In order to keep our assets consistent and our team sane,
9:55we developed a custom importer to batch-import the building modules,
9:59create the necessary folder structures,
10:01and create an assign material instances for each module.
10:04For more information on this tool, please check out our talk
10:07on building look development.
10:09In total, we created 24 unique buildings
10:11for a combined 2,333 unique hi-res meshes, 2,558 unique textures,
10:18and a whopping 5,707 material instances.
10:22It's a lot of data.
10:24And we needed a way to keep it all organized.
10:26I've mentioned kits a few times so far.
10:29So what the heck is a kit?
10:31Think of it like a lens kit or a shaving kit.
10:34A kit is a single location to organize
10:36all of the building modules, a quick way
10:38to review the modules geometry, the look development,
10:42and to see how all the pieces fit together to make a building.
10:46These kits lasted the entire life of the project
10:49and were used as the work area to develop the buildings.
10:52So what exactly are the building modules?
10:55We defined a base set of modules, corner, wall, and entrance.
10:59This base set could be used to create any basic building.
11:02We also added some additional modules required
11:05to lay out more complex facades.
11:08WallCaps are designed to finish off a repeating wall module segment
11:12and are often polarized for the left or right side of the building.
11:16Transition modules sit in between two modules
11:18that may be offset from each other.
11:20They are also designed to be scaled to fit gaps.
11:24Pillars and columns are specific features
11:25that may go between window or wall modules
11:28and should not be scaled at all.
11:30Corners are broken into a few separate modules.
11:33The base corner modules are interior and exterior
11:36and are required for buildings with 90 degree or square corners.
11:40We also defined split left and right corner modules
11:44to support acute and obtuse corners.
11:46Split corners seem like a perfect solution.
11:49However, they only work for flat-edge corners.
11:52Details that extend past the corner will penetrate on obtuse angles
11:56and will create gaps on acute angles.
11:58To avoid gaps, we developed a new corner type we call corner caps.
12:03Corner caps act almost like a hinge and are
12:05aligned to the average angle of both the left and right wall segments.
12:09In most cases, we were able to reuse 90-degree corners as corner caps.
12:14But on building styles with lots of facade detail,
12:16we had to create custom corner caps.
12:19We also found that corner caps offered
12:20another level of additional detail and variation,
12:23especially on building targets that contain lots of corners.
12:27Now that we have all our building modules defined,
12:29we need to assemble them to create a facade.
12:32We wanted artists to have control over how the facades would
12:35be procedurally generated, so we created an expression language
12:39we called Shape Grammar that describes
12:41the construction of a facade wall.
12:44Shape Grammar allows for explicit placement of building modules,
12:47but also supports repeating segments, looping segments, and defining
12:52which modules can be scaled.
12:54Shape Grammar also supports describing
12:56vertical segments of a building.
12:58Ground level, in blue, is the base of a building and is explicitly placed.
13:03Red segments are also explicitly placed and cannot be repeated.
13:08Green levels can be repeated.
13:10The last level in purple is a topper and placed
13:13to finish the vertical segment at the top of the building.
13:17The Vertical Shape Grammar defines which levels can be repeated,
13:20which part of the building can be split into different styles,
13:23and how to finish a wall segment by placing a topper module.
13:27To find out more about Shape Grammar check out our procedural tech talk.
13:31To achieve aesthetic placement of building props,
13:34we opted to hand place prop anchors on modules.
13:37Props are registered to modules and groups that
13:39allow for random selection of a prop anchor
13:42during procedural city generation.
13:45Props are organized by type, awning, light fixture, sign,
13:50and by brand, bank, burger joint, hotel in our prop library.
13:56When a building is procedurally generated,
13:58a random prop brand is selected for the building facade.
14:02When an individual prop is placed on a building,
14:04a random selection of variations is chosen from each prop type.
14:09Keeping track of module metadata became incredibly important.
14:13As mentioned earlier, bounding box dimensions
14:15alone are not a reliable method to determine module size.
14:19We use the Houdini Template Generator to create all the module metadata
14:23and store the module information.
14:25Houdini Engine converted the module attributes to tags
14:28and stored them on the static mesh object
14:31where the metadata could be easily retrieved
14:33later down in the pipeline.
14:35Pivot location is also a critical path and crucial to keep consistent.
14:39We decided to place the pivot on the left side of a module, aligned
14:43to the forward-leading edge of a wall facade.
14:46With the pivot aligned to the wall, we
14:48would always know where the wall existed on the facade.
14:52However, this did create some challenges
14:54for modules that did not have a forward facing
14:57a wall like a recessed entrance or a stepped back wall.
15:01We still place the pivot where the phantom wall would have been.
15:04All of the module information, including Shape Grammar and Level
15:08Grammar is recorded in a JSON-formatted building definition
15:11file.
15:12This file was saved externally from Unreal Engine
15:15and used by Houdini to construct a procedural building.
15:17We thought of the BDF files as the building style,
15:20as it represented everything required to construct
15:22a building of a particular style.
15:24So far in the asset pipeline, we've defined
15:26how the building modules and the building props content was created.
15:31We have imported into Unreal and organized the content into kits.
15:35We have assigned metadata and registered props to the building
15:39modules and generated a building definition
15:41file that contains all the information required
15:44to define a building style.
15:46At this point, we are ready to send the file over
15:48to the procedural system and generate a building.
15:51Procedural buildings can be generated in both Houdini and Unreal Engine
15:54using the City Building Generator, HDA.
15:57We relied heavily on the Building Generator
15:59in Unreal Engine at all phases of building development
16:02to ensure our modeling, materials, and look dev
16:05was set up to work properly in the procedural system.
16:09Please check out the procedural city generation
16:11for more talk on the Building Generator.
16:13We also hand authored collections of props
16:15to scatter around the city at street level
16:17and on the rooftops of buildings.
16:19We called these hand-authored assets 'biomes'.
16:23We hand authored the biomes using a new Unreal Engine 5 feature
16:26called Level Instanced Packed Blueprints.
16:29Packed Blueprints can be created from any selection of Actors.
16:33The selection is exported into a newly created Level
16:36that contains only the selection.
16:38And a Packed Blueprint is generated from that level - all automatically.
16:43All Actors are converted to ISM components
16:45within the Packed Blueprint.
16:47A really nice feature in this workflow
16:49is the ability to place a Packed Blueprint in a Level
16:52and then edit it in place.
16:55We use the Packed Blueprint workflow to author
16:57all biomes, hero buildings, and hero areas of the city.
17:02Hero buildings and biomes are also fed
17:04into the City Generator for procedural placement
17:06around the city.
17:08With our source content pipeline complete,
17:09we are now ready to generate all the buildings in the city.
17:12To see how that's done, check out the procedural tech talk.
17:15Once the city has generated, a point cloud is saved into an Alembic file.
17:20We call the Alembic file a PBC, as it only
17:23contains point information and any procedurally generated geometry
17:27required to create the buildings, streets, and collision objects.
17:32It's not a fully formed alembic file that contains the full city.
17:37It's just points in data.
17:39We bring this PBC back into Unreal and spawn
17:43the city using the Rule Processor.
17:45Spawning the city is a complex task.
17:47The city contains thousands of buildings, props, roads, decals,
17:51biomes, information to build a traffic system, parked cars,
17:56and place audio around the city.
17:58It's a tremendous amount of information.
18:01So how did we spawn the city and manage all of these actors?
18:05Welcome to Open World.
18:07Open World is a new set of tools in Unreal Engine 5
18:11that allow for managing large complex worlds
18:14and for efficiently streaming these worlds at runtime.
18:17The two main features in the new Open World toolkit
18:20are World Partition and One File Per Actor.
18:23World Partition is a grid system that allows
18:26for the loading and unloading of Actors in Editor
18:28and spatially at runtime.
18:31It removes the necessity of sublevels and complex logic
18:35to set up when areas of the world are active.
18:39One File Per Actor separates the connection between Actors and Levels
18:44and allows for multiple people to work on a Level at the same time.
18:49One File Per Actor also works in conjunction with World Partition
18:52to enable loading and unloading of individual Actors.
18:56World Partition also introduces a new organizational system
19:00called Data Layers.
19:02Data Layers are essentially a way to group content
19:05with a common label that can be toggled on or off
19:08and streamed at runtime when World Partition streaming
19:11cells are turned on or off.
19:14In the Editor, World Partition improved workflows and iteration
19:17times massively for our content team by allowing them to only load
19:22the streaming cells and Data Layers that they
19:24needed at any given moment.
19:26One File Per Actor stores each Actor in the map as its own file
19:30on disk, which means when we edit the world,
19:33data contention between team members is at a minimum
19:36because only a minimal set of data is locked.
19:39For instance, I'd only have to load the single cell or Data Layer
19:43containing the stop sign and check out only
19:46that one Actor to make the change.
19:48And I wouldn't have to lock other folks out of an entire sublevel.
19:52Level Instances are also a new feature in World Partition.
19:57They offer a level-based workflow that
19:59facilitates the porting of non-World Partition worlds
20:03into a World Partition system.
20:05They offer a fast workflow for creating Packed Blueprints.
20:09Packed Blueprints are great because they allow artists to abstract work
20:13from the main World Partition map and work in an isolated area creating
20:18content that can then be brought into the World Partition map
20:21and edited in context.
20:23The large city is roughly 4 kilometers squared.
20:26This is not a constraint of Open World.
20:29We chose the city size for creative reasons only.
20:33There are a total of 101,959 Actors in the World Partition map.
20:39This count does not include traffic, crowds,
20:41or any other runtime elements.
20:44This number represents individual Actors saved as One File Per Actor.
20:49Total instances contained in all ISM components is 8,556,732.
20:57This is all the building modules, props, stickers, roads, traffic
21:02lights, decals, everything that is instantiated in the city.
21:07Data Layers are split into Runtime and Editor components.
21:12Runtime Data Layers have the ability to be controlled by logic at runtime
21:17and Editor Data Layers are used for world management in Editor only
21:21and have no runtime overhead.
21:23World Partition for the large city is set up
21:25with a main grid and two HLOD grids.
21:29The main grid contains everything within 128 meters from the player.
21:34This includes all the original Actors that are placed in Editor.
21:39Data Layers can be used to control the loading and unloading of Actors
21:42at runtime.
21:44HLOD0, represented in blue, is the grid range
21:47extending up to 768 meters.
21:51All cells within the range from 128 meters to 768
21:56will load HLOD0 with respect to Data Layers.
21:59HLOD0 contains four Actors that are generated per cell.
22:04All the Actors in the main grid cell are merged into four separate Actors
22:10and consolidated into ISM components.
22:12This reduces the overall Actor count and improves streaming of the cells.
22:18Since HLOD0 is generated from the main grid Actors, whenever
22:22a change is made to the Actors in the main grid,
22:25HLOD0 will need to be regenerated.
22:27We used an automated process to regenerate HLOD0 every night.
22:33HLOD1, represented in red, is visible past the 768-meter range
22:39and is always loaded in memory.
22:41HLOD1 takes all the static meshes in a cell
22:44and merges them into a single Nanite mesh.
22:47Just like HLOD0, any changes to the main grid
22:51will require regeneration of HLOD1.
22:53All handcrafted and procedurally generated
22:56buildings are constructed using ISM components.
23:01This reduced overall Actor count in the city and improved streaming
23:05performance.
23:07We did split all the buildings into upper and lower sections
23:10to improve collision.
23:12The lower ground section of a building
23:14uses articulate collision objects per building module
23:17to allow for detailed collision.
23:19This allows players to walk into recessed entrances of buildings
23:23and also walk through covered hallways and corridors.
23:27These collision objects are nested within the static mesh
23:30for each module and instanced along with the module inside the ISM
23:33component.
23:34For the upper section of buildings, we
23:36disabled collision on the building modules.
23:39A separate primitive collision object that represented
23:42the entire collision section is placed as a separate Actor.
23:46This allowed for only evaluating a single primitive collision
23:50object for the entire upper section of a building.
23:53To improve ray tracing performance we set up
23:56ray tracing groups per building.
23:58This helped optimize the Lumen surface cache
24:01and allows all the pieces of a building to be called together.
24:05When setting up the groups, avoid including sparse objects.
24:09The goal of the group is to group objects that are close together.
24:13To further improve ray tracing performance,
24:16we made sure there was very little overlap of meshes.
24:19Hardware ray tracing gets slow when meshes are kit-patched together
24:22and have lots of internal occlusions and penetrations.
24:26We Booleaned all of our building modules to remove any dirty geo.
24:30Lumen traces mesh instances close to the camera
24:33and then relies on HLOD1 for everything else,
24:36allowing GI over huge distances.
24:39Now that we understand Open World, it's
24:42time to set up the Rule Processor and spawn the city.
24:45The Rule Processor is a set of tools that
24:47allow for importing point data via an Alembic file.
24:51This point data could come from any source.
24:53For this project, we exported Alembic point data
24:56from Houdini and also from Unreal Engine for use
24:58with the Rule Processor.
25:00The rules can be very simple.
25:02For each point, spawn an actor.
25:04Or the rules can be complex.
25:06If a point is an upper-level building module,
25:09assign it to the building Data Layer, disable collision,
25:11and set the window primitive data.
25:14The Rule Processor is fast and able to process millions of points
25:17very quickly.
25:18We were able to respawn the entire city around 50 times
25:22throughout the project.
25:24And often, we respawn the entire city multiple times per day.
25:28We set up multiple rules for ingesting the city
25:30based on how we wanted to organize actors in the world.
25:34Buildings have a set of very specific rules
25:36that are different from actors that make up the roads and freeways.
25:41Setting up separate rules for different assets
25:43that made up the city allowed us to only regenerate
25:46those specific assets when things changed,
25:49further speeding up the process.
25:51The Rule Processor keeps track of Actors that have been spawned
25:54and automatically cleans up old Actors.
25:56It's also smart and recycles Actors to reduce
25:59adds and deletes when using source control.
26:02The Rule Processor is also able to look up metadata key values
26:05and apply that data to Actors as attribute overrides.
26:09This was helpful for setting up Actors
26:11with specific needs like mesh decals and collision objects.
26:15The metadata required for setting up the interior windows
26:17was also procedurally generated and set up on Actors
26:20as per-instance primitive data.
26:25The city is spawned and ready for gameplay.
26:29This project was an absolute joy to work on.
26:33Before coming to Epic, I worked in the film industry for 20 years.
26:37And this is the first project I've worked
26:39on that didn't rely on a render farm to produce final frames.
26:44Working with assets at this level of detail, fidelity, and complexity
26:48in real time was something I never would have imagined
26:51was possible just a few years ago.
26:53And every time I fire up my workstation
26:55and open up the city map, it blows my mind what we're able to do now.
27:02Thanks for watching this video.
27:04And if you want to know more about how the city was produced,
27:07please check out our procedural tech talk.
27:12SCOTT CLIFFORD: Hey, I'm Scott Clifford
27:13and I'm a principal technical artist in the Special Projects
27:16Group at Epic Games.
27:17In this section, I'll be showing you some of the techniques
27:19as well as Unreal Engine 5 features that contributed to the look
27:22and feel of 'The Matrix Awakens' city environment.
27:25OK, fellow friends of Unreal, here we go.
27:28As we narrowed in on the content for 'The Matrix Awakens', we were faced
27:32with populating a city with buildings.
27:34We wanted these buildings not only to have the complex model
27:37detail we can support with Nanite, but also
27:39high levels of realistic shading detail across a variety
27:42of architectural styles.
27:44So we start, as all good look dev does,
27:46with some photo reference of the style of the building
27:48that fits the project.
27:50Here's an example of the entrance of a building we chose for reference.
27:53This building not only needs to look good from this scale,
27:55but it needs to look good at the scale of a city.
27:59It also needs to look good from a human scale and possibly even from
28:03an arm's-length scale.
28:06Oh yeah, and there are 18 buildings that we scouted and would
28:09like to include in our city.
28:11As Votch spoke about earlier, after we break the building down
28:15we're not talking about just shading a building.
28:17We're talking about shading modular pieces of a building.
28:20For instance, good old buildings CHD has 72 modules in its kit.
28:24But that's kind of a simple building.
28:27CHC has 226 modules.
28:30And NYA, 485 modules.
28:34So we end up with over 2,000 individual pieces
28:37of buildings that need shading.
28:39Challenge accepted.
28:41Our goals are to minimize repetition, create textures for thousands
28:45of module assets with a small team, accommodate all possible module
28:49arrangements, and maintain our PBR values in our final materials.
28:54Not so fast!
28:56Any time we want to do look dev, we need
28:58to have a calibrated lighting setup that we trust to make judgments
29:01about the work we're doing.
29:03This is a topic that could be its own tech talk in itself,
29:06but some of the things that we consider are what kind of HDRI
29:09we're using for our sky illumination, what
29:13is the key/fill ratio of the sun to the sky,
29:15and how does that relate to what we see
29:16in the HDRI, what are our PostProcessVolume settings,
29:21and many other details that you can check out in the City Sample asset
29:24pack where we include this calibrated lighting setup.
29:28So now let's break down the look.
29:30When doing look dev on a model, we can break the process down
29:33into three distinct phases.
29:36The first step phase is where we address the base substance.
29:39What is this model made of?
29:40Then we break it down to the manufacturing process.
29:43How has the process that this building was made or assembled
29:46with affected its look?
29:48Finally, how is the environment changed this building
29:51since it's been manufactured?
29:52So let's start with our base substance.
29:55Base substances are created by gathering
29:57a library of tiling textures that we start with from Quixel Megascans.
30:02We adjust these textures to possibly remove repetition or add features,
30:06and then we export them as packed 4K RGBA images.
30:10We import these images as virtual streaming textures into the engine.
30:14When we use virtual streaming textures,
30:16we remove limits on the number of textures that we have in memory,
30:19so we're free to call as many textures as we want.
30:22And then we can manage that texture performance
30:24by tuning the streaming pool size per compression type.
30:28We then can create a UE material where we apply these textures
30:31as WorldAligned, or Triplanar.
30:33And we use realistic scaling to maintain the detail
30:36at a believable scale.
30:38So here we are.
30:39We get our first glimpse of where we're going.
30:41The building modules are assembled into coherent buildings
30:44where we can start to test our work in our calibrated lighting level.
30:47Warning, even though we don't have anything close
30:50to a finished material here, it's important
30:52that we start to sanity check our albedo, roughness, specular,
30:55and other values with each layer of the material.
30:57I like to do this by keeping the Pixel Inspector open
31:00and comparing what the reference values for things like base color
31:04and the GBuffer are to values that I know.
31:07So this is where we are.
31:09Doesn't look great.
31:10But we have a starting point.
31:11So let's give ourselves a check for our base substance
31:14and move on to our manufacturing.
31:17The manufacturing details such as mortar lines and block color
31:19variation can be derived from two block pattern maps.
31:23Block variation, which gives each brick in the pattern
31:25a different value, and mortar lines, which shows where the blocks are
31:28joined.
31:30At this point, we could decide to fire up a 3D paint package,
31:32hand off these textures, and have our poor texture
31:34artists paint these maps onto all 2,000 building modules.
31:38However, what we end up with is every module having the same pattern
31:41in every place it's used.
31:43This doesn't give us variation when the same module is used many times
31:46in a building, which is one of the goals we started with.
31:49So maybe there's another way.
31:50Before we get to that, we also added a couple crevice masks
31:53to the block patterns where staining or drips might be heavier.
31:56We'll use these later when we consider
31:58how fabrication details affect the environment's influence on our look.
32:02So we take our block pattern textures and combine them
32:06into a single 4K pack texture using BC7 texture compression.
32:11We use this compression to try to maintain
32:13as much detail in each channel and avoid channels bleeding together
32:16during the compression process.
32:18We carefully inspect all our reference
32:20and determine that we would need a collection of five stone block
32:22patterns and three brick patterns to cover the architectural style found
32:26in our city.
32:28But how are we going to store all these patterns?
32:31Let's talk about UDIM textures for a second.
32:34You UDIM is simply an automatic UV offset
32:36system that assigns an image into a specific UV tile.
32:39So the mapping between the number that is found in the image file name
32:43is then used to offset that image into UV space.
32:47This allows us to use multiple low-resolution texture
32:50maps producing a higher-resolution result without using a single ultra
32:53high-resolution image.
32:55UE supports UDIM texture imports and stores them as virtual textures.
33:00We take our block patterns and each pattern gets 10 variations in a two
33:04by five grid.
33:06Each variation has identical bricks at the left and right edges.
33:10This is going to help solve the problem of building modules
33:13that are aligned next to each other.
33:16We line up these two by five grids in a single 10
33:19by 5 UDIM virtual texture.
33:23It's time to UV the modules.
33:25What do we do here is a trim-sheet approach where we carefully
33:28unwrap our modules and lay out the UV islands on our giant UDIM block
33:31pattern with the goal of producing modules
33:33with fabrication details that can seamlessly tile
33:36with multiple different neighbors.
33:40We lay out all islands in the bottom row.
33:42They will be procedurally offset into the other variation
33:45roles in the material.
33:46We use style columns that are two tiles wide,
33:49so module islands could be large and cross the border of two tiles.
33:53Most modules use two to three types of block
33:56to achieve the right structure.
33:59Now, in our material we add functionality
34:01to use these block pattern UDIM textures for mortar line color,
34:06also for per-block base substance variations of color and normal.
34:11We use PerInstanceRandom to pick different rows of UDIMs
34:14and remove block variation and repetition.
34:18We end up with four UDIM virtual textures that provide all the brick
34:22and block patterning for all 2,000-plus building modules.
34:27Using 4K tiles for the block pattern UDIM provides very high
34:31texel density for fine details like motor lines,
34:33which can be seen in the image on the left.
34:36Being creative with the UV layout allows
34:37artists to add complex structure to very intricate geometry.
34:41And variation is automatically built in to every building.
34:45So let's see how we're doing.
34:47Here's our reference against our first two
34:49layers of our material, which include the base substance
34:51and the manufacturing detail.
34:53Not bad.
34:54So let's give ourselves a check for that.
34:56And we'll move on to the final layer, which is the environment
35:00influence on our building.
35:02For this, we lay out an additional standard set of UVs
35:06and use a constant texel density across all UDIM tiles.
35:10We recommend to use constant tile size.
35:12We did not do this where we had some tiles in our uniforms which were 4K
35:16and some were 2K, which becomes difficult to track
35:19as the textures go between different departments.
35:21So we recommend you choose one size, say 4K,
35:24and have every image in your UDIM be that size.
35:28Also, thoughtful layout of UDIMs can be used
35:31to pass information into materials.
35:33For instance, on the right, you can see
35:35that we've broken down our materials into stone
35:37on the bottom, painted materials in the middle, and glass on the top.
35:42So now that we have our UVs, we can go ahead and bake some textures.
35:45First, we bake a set of utility signals per module
35:48that include multiple AO distances, curvature, world space
35:51position, and other signals.
35:53We process these utility signals in a DCC and output our environment
35:57masks, which we then pack into a single texture.
36:00We pay attention to how these signals behave near module borders
36:04because we don't want neighboring modules to have
36:07sharp lines between them.
36:09We also can use a low texel density because our details
36:12will be added in the material.
36:15So here you see our crevice occlusion, edge wear, drips,
36:19and broad occlusion.
36:21This makes up our packed environment mask.
36:24Every building module needs one, so there are 2,000 of them.
36:27Luckily, since this process can be automated,
36:30no artists were harmed in the making of this texture.
36:33So now that we have our texture, we can continue to build our material.
36:37We use the packed env mask with multiple scales
36:40of world-aligned tiling detail and macro breakup.
36:43We create Material Functions for all of our environmental influences
36:47including grime and drips.
36:51Here's a glimpse at that environment material.
36:53You can see that we use Material Attributes to layer
36:56the manufacturing environmental layers over the base.
36:59You can see this by the one spine that kind of
37:01travels through the graph from left to right below.
37:04We use Material Functions along the spine
37:06to share these layers between different materials.
37:08For instance, here's our drips Material Function
37:11in the upper right.
37:13So let's check back in with where we are.
37:15We started here.
37:17And now we're here.
37:18Just a reminder that aside from the precise Uvs
37:21needed for the block details, everything else you see here
37:24is handled procedurally, no hand painting, no placing decals.
37:28Here's a glimpse of where we are with our building
37:30material at a few different scales.
37:33So let's go ahead and give ourselves that last check for our environment
37:37layer of our material.
37:39Now that we have our building material,
37:41we need to assign that material to all of our instances.
37:44We start with the global material at the top level, which
37:47is the parent of everything below.
37:49We then add instances at the global level
37:51to set parameters that might be changed across every block
37:54material in the whole show.
37:56We also add an instance to set the kind of textures,
37:59for instance limestone in this case.
38:01And then we add another layer, which sets a color.
38:04We then add an instance at the building level.
38:06If we wanted to change the color specifically of CH/J,
38:09we would address that here.
38:11There are 18 of these, one for each building.
38:14We then add an instance at the kit level,
38:16which allows us to make changes per floor of buildings.
38:20Finally, we have an instance for every module
38:22in which we set the specific environment pack texture that we
38:26need to use for that module.
38:27There are over 2,000 of these.
38:29One for every building module.
38:32So with this many modules, one might ask, how do we get all of this data
38:36into the engine?
38:37We've automated this process by using an AssetIngest Editor Utility
38:41Widget.
38:43This widget relies on a rigid source folder
38:45structure for FBX and PNG files.
38:48It then provides the actions of importing static meshes, textures,
38:51building material hierarchies, and assigning textures
38:54to material instances.
38:56Once we completed our building material
38:58and began to examine it when we put it into the world,
39:00we noticed there was a bit of a disconnect between the bottom
39:03of the buildings and the sidewalks.
39:05We addressed this by building a flat skirt mesh for all building modules.
39:10These meshes were then placed procedurally by the Building
39:12Generator prop system.
39:15So let's move on to our window interiors.
39:17Here, you can see an example of what we created.
39:21The technique we employed begins with interior mapping,
39:24which is used to fake 3D geometry with parallax on flat 2D cards.
39:29On the left, you see some typical interior mapped rooms.
39:32These are from 'Robo Recall'.
39:34On the right, you can see our 'Matrix Awakens' office interiors,
39:37which include simulated 3D furniture and non-uniform room size.
39:42So let's compare some techniques to achieve this look.
39:46On the left, you can see a fake room using interior mapping only.
39:49And on the right, there's the actual geometry.
39:52If we do a capture of the interior geometry and use Bump Offset,
39:56we get an effect that appears to have the objects floating
39:59in the middle of the room, but they still seem rather flat.
40:03We can use Parallax Occlusion to give these objects some depth.
40:06But there are still some pretty bad extrusion artifacts.
40:11We came up with what we call the 3D print
40:13method to improve these techniques.
40:20Let's break that down.
40:22In the 3D print method we start from the back
40:25and do a Bump Offset for each layer.
40:27We test for depth intersection and iterate forward through slices.
40:31And then we dither those slices together.
40:34The information we need to make this 3D print method work
40:37are front and back ortho scene captures, as well as
40:40a front and back depth.
40:41We combine the front and back depth into a single texture lookup.
40:45So here we see our fake room.
40:47Remember, that's not there.
40:49That's fake geometry mapped on a card against the real geometry
40:52on the right.
40:56So how can we handle non-square rooms?
40:59We place a grid guide into the room encompassing
41:02the full dimensions of that room.
41:04We then capture the scene and remap the vertices
41:07into our 1 by 1 by 1 cube space.
41:10We can then decompress this captured 1 by 1 room in the material.
41:14The great thing about this is technique works with any sized room.
41:18Once we have our interior technology squared away,
41:21we can move on to window material that encompasses it and adds
41:24more variation.
41:25We control things like room type, lights, date, temperature, blinds,
41:30glass properties, et cetera.
41:36So how do we decide which windows go where?
41:38In order to have an understanding of what the building is
41:41and how it's laid out, we author this room data in Houdini or UE
41:45plugin and pack it into a 32-bit unsigned integer.
41:48We have flags for all the information the material needs
41:52to figure out what type of room to render in a window.
41:56We store that room data as Per Instance Custom Data
41:59for each instance in a building ISM.
42:03Then we consume this room data in the window material
42:06using HLSL to unpack and reconstruct each bit field.
42:11Once the look of our building started to shape up,
42:13we turned our attention to the roads.
42:16We started our road development using a single tiling asphalt texture.
42:20You can see here that has some obvious problems where
42:23we can see the repetition.
42:25Our Texture Cell Bombing material function
42:27helps us break up tiling repetition by altering
42:30the UVs within the texture cells.
42:32Its source is the cell texture you see
42:34at right, which is a colored Voronoi noise pattern, as well as the edge
42:38detail between those cells.
42:40So again, here we see our initial attempt at tiling asphalt.
42:45And here it is with our cell-bombed asphalt, which basically removes
42:48all of the tiling repetition.
42:50Then we add some rubble to our asphalt,
42:54followed by cell-bombed grime, cell-bombed cracks, patches,
43:01timely macro variation, deferred decals, and wetness.
43:10One of the issues we noticed when we added our wetness layers
43:13was that the wetness was showing up under the decals.
43:16The way that we fix this was to set the decal response
43:18to none for the road material node.
43:21Then in the material we could use the ApplyDbuffer node
43:24to composite the DBuffers into the material under the wetness layer
43:28at the appropriate point.
43:30Here is our road material with the features I've outlined,
43:32as well as others, such as the ability to add lane markers
43:36and to deal with intersections.
43:38Getting the road wetness to look right
43:39was one of the biggest challenges of this material.
43:42We used many inputs to get a natural-looking water-pooling
43:45effect.
43:46This required two UV sets, vertex color information
43:49for blending problem areas, as well as a variety of lane and vehicle
43:53textures and manipulations of UVs blended with noise
43:56to achieve our final result. Here are some images of our completed road
44:01material in the world.
44:09So let's turn to rooftops.
44:12Our wave function-collapsed rooftops are modular.
44:14To avoid the UV nightmare, we use world-aligned projected textures.
44:18But some buildings are not XY aligned,
44:20so we need a way to keep the roof grid
44:22and line details aligned with the edges of the modules.
44:27You can see here that as we rotate the building,
44:29the grid lines on top of the roof stay locked to the roof surface.
44:37We accomplish this using a Z-rotation-aligned world space
44:40texture projection.
44:41This projection sticks to the rotated objects orientation,
44:44but freely tiles in X and Y. We do this
44:48by calculating the actors Z-orientation
44:50and then rotating our absolute world space by the calculated amount.
44:55Here are the material nodes for that technique.
45:01So let's change gears and talk a little bit
45:03about our dynamic global illumination and reflection
45:05technology we call Lumen.
45:09Here you can see a scene with Lumen activated.
45:11We immediately notice the effects of all the light
45:14hitting the building facade and bouncing back toward camera.
45:17We see that indirect light produces soft shadows under the vehicles
45:19and is responsible for all of the light onto the freeway.
45:22We also see detailed reflections on the vehicles.
45:26Here we see the same scene with Lumen turned off.
45:31Lumen on.
45:35Lumen off.
45:38We like to say that Lumen helps us light the shadows by providing
45:42real-time GI and reflections.
45:44This was very important in our world since we often
45:47have buildings blocking the sun, we had wet road reflections,
45:50and we have plenty of highway overpasses
45:52that shadow the sky where there is no direct illumination
45:54available underneath.
45:58Lumen has also been extended to support huge environments.
46:01It has a massive view range, which is accomplished
46:03by tracing to mesh instances at small distances
46:06and tracing against HLOD at greater distances.
46:09Here we see Lumen operating at city scale
46:12where a shaft of light on a building is clearly
46:14taken into account in the indirect illumination of a building all
46:17the way across the street.
46:21One of the most important parts of Lumen is that it is dynamic.
46:25It instantly updates, so you have no lighting builds.
46:27You can change your time of day.
46:29You can have moving cars, characters, and destruction that are all
46:32computed and taken into account as the light is bounced around
46:35and the reflections are calculated.
46:40When this project started, one of the ideas
46:42was to have a night mode where we would
46:44illuminate the city, not by the sun, by many practical light sources.
46:48Using Lumen for night mode was a happy accident
46:51which was discovered while exploring the world with the sun and sky off.
46:55It became clear that lit only by millions of emissive window meshes
46:59and no place light sources, Lumen was able to propagate
47:02that lighting in a realistic way to give the city a nighttime look.
47:07And finally, Lumen uses hardware ray tracing
47:09on consoles, which allows high-quality reflections where
47:12characters are represented.
47:14I hope this talk has helped you understand how we use Unreal Engine
47:165 to create the world of 'The Matrix Awakens'.
47:19As this undertaking was very much a team effort,
47:21Jerome, Votch, and I would like to extend our thanks
47:24to all the talented individuals at Epic who contributed to the work
47:27you've seen here.
47:28Thank you for watching.

About this transcript

This page contains the full transcript of The Matrix Awakens: Creating a World | Tech Talk | State of Unreal 2022 by Unreal Engine, generated from the public captions YouTube serves with the video. The transcript has 7,879 words across 935 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.