YT YouTube2Text

YouTube2Text → Transcripts → The Matrix Awakens: Generating a World | Tech Talk | State of Unreal 2022

The Matrix Awakens: Generating a World | Tech Talk | State of Unreal 2022 — Transcript

by Unreal Engine · 10,074 words · 1,181 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:10QUENTIN MARMIER: Hello.
0:11I'm Quentin Marmier.
0:12I am a lead technical artist for the special project at Epic Games.
0:16And today, we are going to present with Robert Osborne and Julien
0:19Marchand how to generate a world for The Matrix Awakens experience.
0:26The procedural city generation.
0:28So let's look together on how we have created a fully procedural city
0:32generator.
0:33Using very few simple inputs like basic splines,
0:37we have created a tool that could generate
0:39on the fly an American-like city.
0:41We will describe how we have used Houdini in Unreal
0:44to shape an app directed to our needs and how
0:48we have used point clouds and the new road processor pipeline
0:51to generate it on a daily basis.
0:53This is a living autonomous city, 4 by 5 kilometers wide, with 260
0:58kilometers of streets and 512 kilometers of furnished
1:02sidewalks, more than 7,000 unique buildings, 18,000 traffic vehicles,
1:07and 35,000 pedestrians aware of each other and respecting traffic rules.
1:13We will go over the step-by-step process
1:16that were required to make this work and what
1:18are the constraints we are faced, how it generates quickly
1:22a virtual world made out of 8 million Nanite instances that provides
1:26incredible amount of detail from the smallest piece of trash
1:29on the ground to the nuts and bolts at the top of a building,
1:32and how we ended up regenerating that city
1:3553 times to refine its procedural rules over and over
1:39and become what you can explore today in The Matrix Awakens experience.
1:43We've tried to use the full potential of Nanite Lumen Open
1:47World with Unreal Engine 5 to offer the most detailed virtual experience
1:51we could.
1:53But first, let's introduce one of the main tools
1:56we have used in the creation of our city, Houdini from SideFX.
2:01MAI AO: Hello.
2:02My name is Mai Ao.
2:04I'm the lead technical artist on the SideFX Labs team.
2:08First off, I would like to send Epic Games for the pleasure of doing
2:11this presentation together.
2:13I would also like to thank and acknowledge
2:15my former colleague Paul Ambrosiussen and my colleague Damien
2:18Pernuit, who have contributed greatly to this collaboration on the SideFX
2:22side.
2:24SideFX is the developer of Houdini, a high-end comprehensive 3D solutions
2:28platform with a procedural node-based architecture.
2:32With more than 25 years of history, Houdini
2:34is widely recognized in VFX industry with more than one Academy,
2:38scientific, and technical Awards.
2:41You can think of Houdini pipeline as a factory.
2:43Nodes are like machines that perform specific tasks.
2:47The machines can be chained together to form sophisticated networks,
2:51like industrial assembly lines.
2:53The designs of these assembly lines are entirely up to you.
2:57Finally, the pipelines can be streamlined and organized
3:00into factories, aka, HDAs.
3:03So all the controls you care about are exposed,
3:06and the rest is fully automated.
3:09What makes Houdini special is you can go
3:11from easy-to-understand, low-level tools
3:13to a complex, high-level factory that will
3:16look from the outside as some sort of magic.
3:19The core purpose of a procedural pipeline
3:22is to serve the artists, to give them more creative controls and fewer
3:26headaches when they need to redo work.
3:28More importantly, a procedural pipeline is also scalable.
3:33You can often accomplish more with fewer people,
3:35as developers' time is variable.
3:38The architecture of Houdini makes it possible to integrate
3:41the software's compute power deep into the pipeline.
3:45The pipeline is about a lot more than making 3D models.
3:48With Houdini, a lot of the user stories
3:50that can be encapsulated as tasks can be then translated into tools.
3:55These tools are developed with close collaboration between teams.
3:59Decisions are made together to address different departments'
4:01concerns, such as what should be on the UI,
4:04and how should the data be transferred.
4:06When it comes to engine deployment, the tools
4:08are typically packaged as HDAs, which you can use in editor
4:12through the Houdini Engine plug-in.
4:13With Houdini Engine's now free commercial licenses for Unreal,
4:17a larger team of artists and other designers
4:19can directly interact with Houdini tools maintained
4:22by a smaller team of two developers.
4:24In a nutshell, once the tools are created and deployed to Unreal,
4:27The Houdini Engine plug-in will power these tools inside the editor
4:31by talking to the Houdini library in the background on your machine
4:35or on farm machines.
4:37As real-time developer, when you need to start building your own procedure
4:40tools, SideFX Labs is here to help.
4:43We're a small team, but we create a large collection
4:46of free and open source Houdini tools, especially focused
4:49on real-time pipelines as well as the quality of life of developers.
4:53You can think of SideFX Labs as a public extension of your own R&D
4:57team, because the tools we make are based on feedback of the community,
5:01and we make them so that they can be shared
5:03with everyone in the community.
5:06With more than 200 tools and growing, it is not unlikely
5:09that when you need to accomplish something,
5:11we may have a standardized tool for that.
5:14Labs tools span everything from modeling, photogrammetry, UVs,
5:18texture generation, asset export, third-party integrations, and more.
5:24We also ship a SideFX Unreal plug-in, which
5:26contains scripts and shaders that work with Houdini assets, or just
5:31generic all-purpose utility shaders.
5:34You can discover all this by downloading SideFX Labs' latest
5:37daily builds.
5:39QUENTIN MARMIER: Thank you so much, Mai Ao, for this overview of Houdini.
5:42And now let's have a look to The Matrix Awakens procedural city
5:45and how we have done it.
5:47As a starting point, we knew The Matrix Awakened
5:49would have to be an open world, a city that
5:52would be fully detailed and very large at the same time.
5:55We established that a 4 to 5 kilometer-wide city
5:58would provide the ideal playground for the experience
6:00that we had in mind.
6:02We knew this environment would be the base for a lot of departments
6:05to work on, and that they would need to be provided with updated data
6:08as we are modifying it.
6:10This had to be automated.
6:12We also knew that it will have to be procedurally generated
6:14because of the relative small size of our environment
6:18team versus the sheer size and level of detail we were aiming for.
6:22All of this was also considering that we would use Nanite technology.
6:25Nanite has changed the game, because it
6:28removed all polygon limits and the resolution
6:30at which we can have details.
6:31Now that 1 meter squared can have up to a million
6:34triangles without worrying about performances,
6:37a city of 16 kilometer squared would have 16 trillion triangles.
6:41This brings a new problem.
6:42The size and the amount of assets on disk becomes a limiting factor.
6:46Also, bigger assets means slow iteration and less flexibility.
6:50We can't just brute-force our way through.
6:53We have to find a smart way to slice the world.
6:56Early on, we were considering the use of OpenStreetMap.
6:59But unfortunately, the data it provides
7:01is arbitrary-- unique in size and shape for any given place.
7:05That is too much work to translate and adapt.
7:07And that creates too many unique assets.
7:09Same would go for the natural follow-up of OpenStreetMap,
7:12creating a mega mesh of the city.
7:14But for the amount of detail we were after, that
7:16wasn't a viable solution.
7:17What we need to do is quantify the world
7:20and make our own construction set to build it.
7:23We would then relay as much as possible on instancing.
7:26How work becomes, then, how to slice and dice a realistic world
7:30and replicate it the most precisely possible with those bricks.
7:33This is especially complex, because in the real world,
7:35you will always find a piece that is almost impossible to reproduce
7:38with your bricks.
7:39That is where having full control over the geometry of the city
7:42becomes so important.
7:43We wanted to keep a tight grip on how much of the arbitrariness
7:46we would allow.
7:48First, we will start with a specific tool designed
7:50to create the base of our city, the City Layout tool,
7:53or how to procedurally generate an American city pattern.
7:57You would only require two main inputs to start--
8:00a basic shape and the path or the main arteries.
8:03With few attributes and option available to adjust the road network
8:06setup, it would give you an American city layout.
8:10The rest is figured out by the tool.
8:12With only few spline inputs, it tries to procedurally recreate
8:15an American city pattern.
8:17We relied on the find shortest path Houdini node to create the arteries.
8:22And at last, the tool is calculating the best fitting grid pattern
8:25to match to the size defined by the user inputs.
8:28The tool can procedurally create an infinite solution
8:30of different cities.
8:33The next step is the city zoning.
8:35We have used this extra utility to define zones
8:37with the specific attributes like a commercial or residential zone,
8:41but mainly to shape the cityscape.
8:43In a way, this becomes the vertical modeling tool of our city.
8:47We have used Houdini Engine plug-in inside Unreal
8:49to be able to design the city directly in-engine,
8:53although the tool could also be used directly
8:55in Houdini for the rest of the procedural generation.
8:58This was a user-friendly way to model the cityscape.
9:01And we made a lot of back-and-forth with the art director
9:03and the cinematic department to shape it the way we wanted.
9:06Once we are done with the basic layout,
9:08the tool is outputting a big soup of metadata
9:11that would be used by downstream tools to produce the city.
9:13That will include a road network, the lot definition, and the sidewalk
9:18network, among many other attributes we could also find in there--
9:23the road connectivity, the traffic density, or the pedestrian density.
9:28With the soup of metadata ready, we can now fit the city processor
9:32and start the final creation process.
9:35This dependency graph shows an overview
9:37of the key components of the city tools and the dependencies from top
9:40to bottom.
9:41This is important, because in the procedural flow
9:43it shows where we will need to regenerate
9:45the city in the case we were to change a rule at a given point.
9:49The dependency graph is broken up into three stages--
9:52the city base, the city core, and the set dressing.
9:56And this is an overview of what the graph would look like inside Houdini
10:01with all the actual visible dependencies.
10:04The first step in the creation of the city
10:06is to well define the road and their geometries.
10:09In order to do that, we take the road network
10:12provided by the City Layout tool.
10:13It is composed with spline made out of two points.
10:16It has a three-layer road hierarchy that
10:19carries all the metadata it needs, like the road width, the road
10:23IDs, the number of branches per intersection, et cetera.
10:27We then trim each road section by a carefully calculated amount based
10:31on the angle at which the road connects,
10:33and leave enough room for the intersections.
10:36Each road section would then be filled with the adequate number
10:38of road modules.
10:40For each width, we have three available modules--
10:4320 meters, 10 meters, and 5 meters subsection.
10:46This way, the entire road geometry is mainly
10:48made out of nine basic bricks.
10:51Because the sections can technically have any possible lengths,
10:54we keep some room for a little bit of scaling
10:56on the smallest 5-meter module.
10:59For the intersections, we are filling up the leftover room
11:02with road modules scaled to the appropriate lengths.
11:04We then set the pivot at the intersection
11:07center while they aim outward toward the road sections.
11:11Once properly calculated and assembled,
11:14the road modules are replaced by their higher resolution
11:17counterparts.
11:18This represents the geometry as it would appear in Unreal.
11:22And the road processor would then output a point cloud
11:25that would be used to instance the module inside Unreal
11:27using our road processor pipeline.
11:30The road module would have two sets of UV
11:33plus a relative vertex position stored
11:35as vertex colors for maximum control with the shading.
11:39It would feature some tapered skirts on the side
11:42to ensure smooth connection between them,
11:44and would also have a bit of curvature to add a bit of realism
11:48to the world.
11:49Once the road network and precise road footprint is established,
11:52we can move on to the extraction of the traffic information.
11:55Using the metadata of the roads, we would trace splines for each lanes
12:00and interconnect them to create the traffic base grid.
12:03We can briefly mention here that this network of splines
12:06contain all the information about the traffic
12:08and would later be imported to an Unreal point cloud.
12:11It would then be interpreted by the traffic tools
12:14to be turned to a zone graph.
12:16Julien Marchand will cover more in detail this topic later
12:19in the presentation in the section "Bringing Life to the City."
12:23Next in our city generation process would come the freeway regeneration.
12:28For the city's freeway, we knew that the chase sequence
12:31would happen at least in part on it.
12:33This had an influence on the organic and tentacle-y look of it.
12:37We were after an iconic landmark easily
12:40recognizable in the city with complexity and pillars
12:43for the cool cinematic parallax effects.
12:46Around the needs of the cinematic department that were mostly
12:49focused around the big exchanger, we have
12:51built the rest of the freeway path.
12:53It is composed of two closed loops, one overhang passage, and 55 access
12:58ramps over 25 kilometers of usable freeway.
13:03To reiterate quickly, our designers needed a way to edit the freeway
13:06and find what would work best for the chase sequence.
13:09An Houdini asset was used in Unreal and would take an input curve
13:12to generate a simple freeway deck geometry,
13:15allowing designers to create a good path for the chase.
13:18There was a lot of back and forth between the designers
13:21and procedural team to make sure the freeway's shape would serve the demo
13:25well, both in look and function.
13:27Once the shape of the freeway was locked in place,
13:29the designers' input curves were brought from Unreal to Houdini
13:33and embedded into the city graph, keeping the freeway generation
13:36as part of the main city process.
13:43Because of its organic nature, the deck
13:45had to be procedurally generated as a mega mesh.
13:48This is one case in the city where we couldn't entirely
13:50rely on modular instancing, and we had
13:53to create custom pieces that would fit a specific area.
13:56The mega mesh would later be divided in smaller chunks of 100-meter size
14:00so it can fit the Open World streaming workflow
14:02and help Lumen for performances.
14:05All the other set dressing pieces--
14:07the barriers, pillars, signs, and debris--
14:09are Nanite instances scattered along the freeway path.
14:13Finally, as for the road processing tool,
14:15the freeway tool would output several point clouds
14:18to be instanced over inside Unreal and would provide metadata
14:21to the rest of the graph.
14:22We would have put several sets of meshes, including the freeway deck,
14:26in its collision path.
14:28With the road and the freeway generated,
14:30we can now move on to the lots processing and the building volume
14:33generation.
14:34For the lots, we start from the basic lots definition
14:37provided by the City Layout tool.
14:39Remember that it also carries the height information defined
14:42by the cityscaping we had done earlier on.
14:45We first need to ensure that we won't rise buildings where the freeway is.
14:49So we remove it.
14:51After several layers of geometry cleaning and filtering,
14:54we are subdividing the lots.
14:56The subdivision is using an algorithm that
14:58ponders building height versus available surface.
15:01The more the surface, the larger the final footprint can be,
15:04but only that is to support a tall building.
15:07Of course, we introduce a little bit of random in that process
15:10to not make it look totally linear.
15:12The lowest building heights are automatically
15:15divided into New York-style lots with the typical staggered style
15:18subdivision.
15:19As the subdivision is done, you can notice in red
15:22all the lots that have been filtered as not suitable for building,
15:25either by their shapes or their topology.
15:28This is where we have introduced a system
15:30to ensure we would cut down on producing arbitrary data
15:33and try to quantify the world a bit more.
15:35Introducing the Building DNA.
15:39Each building of the city would be described by a custom DNA.
15:42The DNA starts with a footprint.
15:44We have created 17 of those, and any building in the city
15:47would have one of these shapes as a base.
15:49So we would always know what to expect.
15:52Once the footprint is established, we added few attributes
15:56to complete the DNA--
15:57the size in x and y, its position and orientation in space,
16:01the height of extrusion divided into three layers.
16:03And as an extra bit of variation, mostly for taller structures,
16:07we also added the possibility for any building
16:09to be either cubified or with staggered floors,
16:12like the Empire State Building.
16:14As the last strand of the DNA, we would assign a specific building
16:17style.
16:18With this technique, we could create an infinite variation of buildings,
16:21ensuring every single one of them would be unique.
16:27We can now replace non-suitable parcels
16:29with our collection of footprints and ensure
16:31they will fit the original space available.
16:34For the rest of the subdivision, we simply
16:36replace them with our available footprints.
16:39It works in two stages.
16:40First, we address how much percentage of a given footprint
16:43we'd like to see in the city.
16:44We then ensure it would be compliant with this given building style.
16:48This is important, because each building style
16:50has a limit on how it can be built, a minimum wall
16:53length under which the building generator will not be able to fit
16:57the module that composes it.
16:59More on this will be described in the procedural building topic
17:02later on in the presentation by Robert Osborne.
17:05The final part is the creation of the building volumes themselves.
17:09Populating the rest of the needed DNA strands with some global controls,
17:13we would add a little bit of extra noise
17:15to adjust the overall feeling of it.
17:17We could finally choose how much of a cubified or staggered building
17:20we would want to see.
17:22So before Robert covers the procedural building more in details,
17:25we will go over the rest of the city graph.
17:27And in our next step, we will look at the creation of the ground.
17:31The ground would be under high scrutiny,
17:33since it's where the player would spend most of its time.
17:36We needed here, more than ever, a very high fidelity in the asset.
17:39And thus, the need to use instancing elegantly.
17:42This represents our collection of sidewalk and floor module.
17:46We didn't have availability for this placement directly in engine,
17:49but it wasn't a problem.
17:50Since Nanite removes all triangle limit,
17:52we end up pre-displacing the floor tiles.
17:55Using modified Megascan surfaces with the displacement maps,
17:58each tile could count up to 500,000 triangles.
18:02This gives you a level of crispiness and fidelity
18:04that we could have never achieved otherwise.
18:07So now, as the City Layout tool has provided us
18:10with a network of splines for the sidewalk and the lots definition,
18:13we could take those meshes and add the sidewalk
18:17processor within the city graph ingested
18:19to scatter our module collection adaptively,
18:22dividing each sidewalk section in a similar fashion to the road modules,
18:26you would fit the correct number of tiles,
18:28solve the corners and the ground coverage area.
18:32And here, we can see how the sidewalk processor
18:34works at different sidewalk connection angle.
18:37It would calculate how many tiles needed to make the sidewalks,
18:39randomly assign types, and solve the external and internal corners
18:43independently.
18:45Notice how for the inside corner, the two triangle pieces scale
18:48non-uniformly back and forth to adapt.
18:52For the ground itself, we would subtract the building coverage
18:54to save a bit of surface and then cover it with a point cloud
18:58where the density would be adjusted to it ground tile size.
19:01If you recall the 500,000 triangles predisplaced as shown
19:04before, here you can see how many times we would use it over the city
19:08ground--
19:092.5 million times that is.
19:11Yeah, that is 1.25 trillion triangles for the ground itself.
19:15We wanted things to look sharp and good.
19:18And that shows the same point of view inside the engine,
19:21with on the right the different instances that compose the ground.
19:25A little close-up shows you the final result more in details.
19:28This is a good representation of what the procedural teamwork was
19:31on The Matrix Awakens experience, a giant puzzle solving with instances.
19:36Now that we have our ground covered, we
19:39can move on to the set dressing of it.
19:41First, we would harvest and filter all the areas that
19:44are left untouched, basically there are no sidewalks or no buildings.
19:48We have defined three main leftover zone types to describe the city--
19:53the plaza type, the freeway type, and the parking type.
19:58To populate those areas, we would use our instance-packed Blueprint
20:02workflow, also shortening biomes.
20:06Those are pre-assembled packages of assets.
20:08And this workflow is described more in depth in the presentation
20:11from Votch Levi, "Creating a World" that I invite you to watch.
20:15To distribute those biomes among the leftover areas,
20:18we have used the scatter technique using the UV space of the zone.
20:22With the UV layout node of Houdini, this is a very convenient way
20:25to layout scatter your object while ensuring that they will not
20:28overlap with each other.
20:30For the use of the city graph, we had developed
20:32our own version of this tool and called it the packing scatterer.
20:36It would take each dependent zone, measure its longest edge,
20:40and take it as orientation reference.
20:43It would then try to best fit the biomes together
20:45with the possibility to choose between different patterns
20:48and angles.
20:49We have used this tool literally all over the graph
20:52to procedurally populate areas in an organized fashion.
20:55That also includes some of the rooftop for the buildings.
20:58This shows you a fun example of what the packing scatterer can
21:00do to cover any type of area with organized pattern,
21:03without any fear of overlapping instances.
21:06The last images are showing the result
21:08of playing with different options and inputs on the tool.
21:12And now we come to the road decals in the grand covering process.
21:15We are taking the traffic and parking lanes calculated after the roads
21:19and pass them with points for the different types of decals.
21:22Over each calculated points, we would spawn a simple plane
21:25that would feature a decal material.
21:27Each marking that you see here is an instance plane
21:30that is carefully placed at the exact ground level.
21:32It has a decal material with opacity for finer details.
21:36The wire frame view gives a better idea of what's behind the scene.
21:39If you remember during my description of the road processor,
21:42I mentioned that the road isn't flat.
21:45That actually proved itself to be a bit of a challenge.
21:48For bigger chunks like the crosswalks or the tire marks,
21:51the plane would need more subdivisions
21:52to adapt to the road topology.
21:54This is why we have used here a different approach.
21:57The curvature of the road makes it a totally arbitrary object.
22:00And this was one of the cases where we had to use a mega mesh.
22:03So here, the bigger planes we are projecting on the road geometry
22:06to match this topology and then assemble to a mega mesh.
22:10Another interesting challenge with the details
22:12were the solving of complex intersections.
22:15A procedural graph would analyze and interpret the lane system
22:17to create believable markings, draw roads, and basically add life to it.
22:22So now that we have produced a full procedural city,
22:25we have control over every single bit of information.
22:28So we can easily extract it and provide it to the audio department
22:31so they can work their magic.
22:33To provide this data, we would cover the final city result
22:36surface with 3 meters density point cloud
22:39on which we will store the metadata.
22:41And by using a proxy geometry of the city,
22:44we would use its own ambient occlusion
22:46and indicate to the audio team at any given point
22:49the amount of reverb it can expect, or tell them what type of sound
22:52it should produce.
22:54So that now we had a good overview of the city generation process,
22:58let's have a glimpse of what the Houdini graph is actually like.
23:01This isn't all of it, but it gives you an idea.
23:04This graph would take you about 25 minutes
23:05to produce a totally new 4 by 5 kilometer city with its 8
23:09million instances, meaning that in 25 minutes
23:12you could have a new city solution entirely different
23:15from the previous one.
23:17As a final look at the city generator tool,
23:20this is an accelerated demonstration of all what we
23:22have covered in this presentation.
23:25This is a 1-minute speed run for a small city example that demonstrates
23:28how you can generate a world all the way to have it
23:32as a playable experience in Unreal.
23:34It shows how the generator enables you
23:36to produce a new city from the very first curve input
23:39to the road network tweaking, the vertical shaping,
23:42fine tuning of its building, to its final generation with the road
23:45processor.
23:47In this last part, you can actually see the road processor in action
23:51for the generation in Unreal, although this topic
23:53is covered more in detail in the presentation by Votch Levi,
23:57"Creating a World."
23:59You can actually try it now by yourself,
24:01as we are providing you with Houdini files and basic Unreal templates
24:04to play with.
24:07So some fun statistics to look at, and among those one
24:10that is very interesting to mention and that represents well
24:14how the city processor would work.
24:16Over the course of the project, we have regenerated the city 53 times.
24:20For every generation, we would sculpt the city
24:23and look for what we can improve in the procedural rules,
24:26fix bugs, add details to it, and finally regenerate
24:29the city, at least in parts, or sometimes globally.
24:33That process, including the importing Unreal with the road processor
24:36workflow was possible in a couple of hours for the final version,
24:40meaning that technically, we could have
24:42had an entirely new city to discover several times a day
24:45if we were to change it.
24:47Let me now introduce Robert Osborne that will present more in detail
24:50the process behind the building generation.
24:53ROBERT OSBORNE: I'm Robert Osborne, a senior technical artist
24:56at Epic Games.
24:58I'm going to talk about the building generation in The Matrix Awakens
25:01demo.
25:01Let's talk about the procedural building generation in The Matrix
25:05Awakens demo.
25:06In the prototyping phase, we wanted to keep
25:08workflows as simple as possible-- one DCC straight into the Unreal engine.
25:13Houdini and Houdini engine are obvious choices
25:16for prototyping procedural ideas and workflows.
25:19Iteration speed is king when you want to evaluate concepts.
25:23We did a lot of prototyping and iterating.
25:25A great example of this ethos is the building module template generator,
25:29quickly allowing building module proxies to be
25:32created on demand to specific sizes.
25:35A building-shaped grammar system was needed
25:37to make the building generation as agnostic as possible,
25:41avoiding hardcoded behaviors that would constantly
25:43need to be reworked and debugged.
25:46This is what I started with.
25:47Wow.
25:48Looking back, I'm having a hard time understanding it myself.
25:51Luckily, we were still prototyping and iterated
25:54on the shape grammar look and feel.
25:58Here the building generator is being used in Unreal
26:00to add an extra building to the small city level.
26:03Let's reposition the building to fit on the pier properly
26:07and change its dimensions.
26:17Now I'm changing the prop placement density and seed.
26:21Let's lower the height of the building.
26:25Finally, I'm changing the style of the building and making it wider.
26:29There, done.
26:31Building added.
26:33As we entered production, we had a good idea
26:35of how the city graph data would drive the building generation.
26:39JSON was chosen as the format for the building definition
26:42files during prototyping.
26:44An automated BDF pipeline was implemented in time for production,
26:47giving human readable data that could be edited quickly to validate
26:51and debug with.
26:52The data fed to the building generator
26:54consisted of volumes with a BDF tag assigned per primitive face.
26:58Here, each color on the volumes represents a different building
27:01style.
27:03Here, we can see how the data flows inside the building
27:05generator in order of operation until final output.
27:09Output was in the form of point clouds
27:11and roof geometry passed into the city graph
27:14or directly into the Unreal engine using Houdini engine.
27:18Input volumes were cleaned carefully to give the best
27:20chance for cleanly sliced floor primitives
27:23to be generated in a consistent and ordered manner.
27:27The assigned building definition file tag
27:29allows the floor intervals to be calculated from the BDF's level
27:32dictionary.
27:33And the volume is sliced accordingly.
27:36Once we have the floor slices, corner types
27:38are calculated using the dot product between the primitives
27:41and walking around the floor primitive IDs in ascending order.
27:45Split corners are recognized using a collinear tolerance
27:49to distinguish if adjacent floor primitives should
27:51be considered in line.
27:53Corner caps, when defined in the BDF, are prioritized over a split corner
27:57type.
27:59We wanted a flexible shape grammar syntax
28:01for defining the module placement behaviors on building facades,
28:05allowing for rapid end user iteration without having to publish updates
28:09to the building generator.
28:11This is a typical shape grammar.
28:13The vertical bars represent a module bucket
28:16used to define module placement behaviors.
28:18The alphabet characters represent a dictionary key
28:21to look up the module metadata, like width and height.
28:24Circular brackets are used to define an infinitely repeatable macro
28:28pattern of modules.
28:30Square brackets are used to define a fixed repeat of macro pattern
28:34modules.
28:36The asterisk character specifies that all modules in a bucket
28:39can be scaled to fit the shape grammar to the length of the facade.
28:43Selecting the most appropriate grammar for a facade
28:46is driven by the total length of one full iteration of the shape grammar.
28:50The shape grammar with the smallest fit
28:52gap not greater or equal to the facade length is selected.
28:57Here we can see how different grammars
28:59are used as the length of the facade increases.
29:02With the most suitable shape grammar selected,
29:05the modules are placed on the facade.
29:07Here we see the module placement and scaling
29:10according to the grammar being used as the facade length increases.
29:14Buildings are generated independently of neighboring buildings
29:17in each building lot.
29:19This gives better parallelization for the graph execution.
29:22Occluded modules on connected building walls
29:25need to be removed from memory and generation performance.
29:29In this city graph, each lot of buildings are passed in as one job.
29:33We take the full lot of buildings minus the building
29:36having its modules included and merge and clean them.
29:39The building being occluded is Booleaned
29:41against the rest of the lot, creating a Boolean shell that
29:45represents touching wall heights.
29:47Additional sample points for each module
29:50derived from the module bounds are added
29:52for performing intersection testing.
29:54These sample points and the module pivot point
29:57are ray cast against the intersection shell.
30:00Only modules where all sample points are
30:02deemed to be touching the intersection shell
30:05are flagged for removal.
30:07As corners wrap around facades, extra sample points are added for corners.
30:12The occlusion has to be conservative, as re-adding modules to cover holes
30:17is complicated when you consider primitive data for windows
30:20and props.
30:21Even with this conservatism a 25% reduction
30:25in building modules, props, and decals is achieved.
30:30The conservative approach is shown by the columns of corners still present
30:33inside some of the building shells.
30:36We needed to indicate to the city graph sidewalk furniture placement
30:40system where building entrances existed.
30:42The place modules on buildings carried tags
30:45to define functionality such as corner, wall, entrance, et cetera.
30:49These tags allowed the building generator
30:51to provide volumetric data to simulate exclusion of street
30:55furniture in front of building entrances.
30:58Once in production, new feature requests
31:00were added to the building generator, one being multiple BDF
31:04support on a single building volume, others
31:07being window treatments and window signage.
31:10It highlighted the need to have a production version and a development
31:13version of the building generator while new features could be iterated
31:17on in a development version without blocking the city graph or end
31:20users using the production version.
31:23Building prop brandings were assigned per facade
31:26across the city with just a simple modulo seeding using the lot ID
31:30and building ID for determinism.
31:33Being able to have the building generator run directly
31:36in Unreal using Houdini Engine allowed end users
31:39to play with prop densities and adjust seeding.
31:42Controls were exposed on the UI for this purpose,
31:45allowing prop meta to be tuned and then exported into BDFs.
31:49Here we have the building generator running in the Unreal
31:52Engine via Houdini Engine.
31:54On this building, I want to place fire escapes on it interactively
31:58using the volume overrides functionality.
32:00The volumes are given their override functionality
32:02by adding active tag strings.
32:05For example, the word "fire escape" is used for placing fire escapes.
32:08The override volumes are passed to the building generator
32:11using exposed tag volumes input.
32:14Let's change the prop branding and extend the red coffee shop
32:18props to continue into the recessed courtyard area,
32:21again, using a volume with an active tag
32:23to represent the desired branding on the props.
32:27For the window cubemap primitive data,
32:28we wanted to create believable functional rooms.
32:31We built a connectivity array along each facade
32:35of modules that have a window tag.
32:36With this, we can determine the maximum size
32:39of a room that can be placed.
32:40After placing a room, all instances in the room get a room ID,
32:44and we move to the next window module without an assigned room.
32:47This continues until the facade is full of assigned rooms.
32:51A VEX dictionary holds several seating arrays to select rooms
32:55based on the building function.
32:57In the movie, you can see lights being switched on per floor ID,
33:01then by room ID on each floor, and finally by room size.
33:09For window treatments, window helper objects
33:11were added to level 1 modules to define
33:14the surfaces of the windows if present on a module.
33:18This data was stored in the prop anchor points
33:20for each module using the helper category.
33:23We needed a subsystem to populate window signage
33:26we knew where each building entrance was and only placed
33:29signage close to entrances using the window helpers.
33:33The window signage data is exported in a building definition
33:36file for use with the window composite system.
33:39Composite layouts were selected by the assigned facade branding.
33:43The prop placement VEX code with different filtering
33:47was used to place the composite layouts in place of a normal window
33:50frosting instance, which would be removed during the signage pass.
33:54An internal VEX dictionary was used to encapsulate
33:57placement rules, brand groupings, and functionality with certain masking
34:01used for building styles.
34:03All the roof geometry was derived from the input building volumes.
34:07Building a one-stop recreate all roof geometry from the floor
34:12slice primitives was too complicated, and edge cases
34:15popped up with every tweak.
34:17UV generation and scaling controls were
34:20added to deal with unique roofs which could not be instantiated.
34:23The UV controls were exposed on the building generator UI for end users
34:27to evaluate good default values.
34:30Building volumes developed from simple cubes
34:33to complex cubified volumes.
34:35Split roof support was added to deal with the complex volumes
34:38as best as possible.
34:40Not all building modules were created equal,
34:42the biggest issue being some modules got
34:44sliced at the lower window edge, which is recessed back
34:48from the common facade plane.
34:50We had to create a function to analyze the floor
34:53slices at split roof edges and apply a push
34:56or pull inset to hide these gaps.
35:00The building generator needed to be scalable
35:02due to the sheer amount of data it was generating--
35:05almost 7,000 buildings with props, module occlusion, window treatments,
35:10window cubemaps, ground decals, and roof geometry.
35:14This all generates in 30 minutes using PDG on a single X3990
35:19Threadripper PC.
35:21Some data sets from the large city generation.
35:30As we entered the end phase of the project,
35:32the building generator graph looked like this.
35:36The volume curator pass just before the building generator
35:39allowed the generation of override volumes to place fire escapes.
35:43A masking scheme was used to populate fire escapes only
35:45on certain building groups and styles.
35:48It also allowed directed prop placement via primitive groups.
35:53Example, no props were placed in the Brooklyn-style courtyards.
35:57Multi-BDF volumes were given steps to hide
36:00transitions of the building styles.
36:02Multi-BDF style combinations were also
36:05tweaked following art direction concerns.
36:08The volume curator also allowed for optimizations late in the project.
36:12Memory was a concern, so the global height of the cityscape
36:15was reduced in certain height ranges, allowing a surgical strike
36:19without potentially causing a butterfly
36:21effect had this change been further upstream in the city graph.
36:24The volume curator was also used to target specific problematic building
36:28volumes and associated metadata.
36:34So while not directly involved in the building generator,
36:37I wanted to mention a workflow we started using for referencing
36:41viewpoints in the city.
36:43As the city density grew, the ability to communicate points of interest
36:47in the city with accuracy became all too apparent.
36:50We started to use the Unreal Editor command line
36:53BugIt to snapshot points of interest for quick sharing,
36:56tagging in channels with a screenshot and a BugItGo string.
37:00So we reached the pinnacle.
37:02And JIRAs contain BugItGo information.
37:05With all these data points for POIs starting to mask,
37:08the BugItGo HDA happened one afternoon,
37:11allowing reference points to be available in Houdini data sets
37:14to observe behaviors and debug with.
37:17Pasting a BugItGo string into the BugIt code
37:19HDA will generate an indicator in Houdini world space.
37:24We have a reference in Houdini world space
37:26that can be jumped to quickly using frame selected.
37:30This easily allows associated input metadata to be explored, debugged,
37:35and edge cases solved.
37:37You can round-trip this data, passing a BugItGo location that
37:40is derived from the currently active Houdini 3D Scene
37:44view back to the clipboard.
37:46Pasting the clipboard back into the Unreal Editor command line
37:50gives a super quick back and forth for cataloging and exploring
37:54the city data set.
37:55For bugs, this is especially efficient
37:57to get the location of an edge case generation to examine the metadata
38:01and step through your graph and debug with,
38:04allowing the problem buildings to be isolated in the Houdini scene,
38:07iterate on the fix, test locally, and publish the fix--
38:11super useful for smoke testing known results after an update
38:15to the building generator.
38:19With the sheer number of building volumes,
38:20we were struggling to identify issues in the vast sprawling city we
38:24were building.
38:25We literally could not see the buildings
38:27for the city in some situations.
38:30While we had BugIt and BugItGo to teleport around the city
38:33and investigate issues, we really needed a structured way
38:36to sign off on all the building styles and building
38:39features without the distraction of the beautiful city.
38:42A volume filter HDA was created to give targeted collections of volumes
38:46to feed into the building generator.
38:49Houdini is great.
38:50Change your meta filtering into generation
38:52then add some automated output.
38:54Now, you are slicing and dicing the data to create sets of maps
38:58in the editor to systemically close the project.
39:01Not carrying the full weight of the city during smoke testing
39:04increased the iteration speed greatly.
39:07Special thanks to some of the quality people I
39:09was lucky enough to work with on this project.
39:15And now let's talk about wave function collapse.
39:19Wave function collapse is a term from quantum mechanics,
39:22in which a wave function in superposition collapses
39:25to a single eigenstate.
39:27This concept has been applied to game development
39:30in the form of a constrained satisfaction algorithm.
39:33It can have many applications, including texture and model
39:36synthesis, map generation, and gameplay.
39:39In a constrained satisfaction problem,
39:41a constraint is a rule that this system must satisfy when solving.
39:45Here we have a cat in a two-tile grid constructed from two
39:49options, the tail and the head.
39:51There are two constraints here.
39:52The head can be to the right of the tail,
39:55the tail can be to the left of the head.
39:57It is simpler to always consider these constraints in pairs.
40:01Here we increase the grid to four tiles
40:03and added a third option, the body.
40:05We can see a total of six constraints.
40:08Next, we introduce a special option, the Empty option.
40:12Empty options are useful when you want
40:14to ensure that the bounds of the grid will not form open-ended solves,
40:18like having a body option at either end of the grid.
40:22Empty options can be adjacent to each other.
40:24If you have an A to Empty pair and an Empty to B pair, then an A to B pair
40:30can be made.
40:31Let's see what it looks like to collapse a 1D grid.
40:35We initialize a six-tile grid, where each tile
40:37represents all of the options aloud.
40:40The number above represents remaining number of options for a given tile.
40:44Let's make an assumption that the tiles are surrounded
40:47by hidden Empty options, and both ends of the grid
40:50must be modified to only allow options that
40:52can be adjacent to an empty option.
40:54You can see that the number of remaining options
40:56has dropped from 4 to 2.
40:59This number will also represent the value of entropy for a given tile.
41:03We now begin a recursive cycle of observing and propagating.
41:07In the observation phase, a random minimum entropy tile is selected.
41:11A selection is randomly made from the remaining options.
41:15The Tail option was chosen, and remaining options on the tile
41:18has dropped to 1.
41:19In the propagation phase, all the tiles adjacent to the observed tile
41:23must adjust their domain to satisfy the constraints.
41:26The propagation continues recursively until no more tiles are modified.
41:31With every iteration, more tiles are collapsed down to a single option.
41:35The solve keeps iterating.
41:40Finally, the solve reaches a state where the grid
41:42contains only collapsed tiles.
41:45Here we can see this process in two dimensions using a 6 by 6 grid.
41:52Here we see the process in three dimensions using a 3 by 3 by 3 grid.
42:00A wave function collapse model is a data asset that stores constraints.
42:04A constraint is stored in the form of a Key option, an adjacency,
42:08and the Adjacent option.
42:11Defining constraints can be difficult.
42:13Some implementations have used label-based systems,
42:17but that can be confusing or complicated from an artist's
42:20perspective.
42:21Our implementation derives constraints from arbitrary layouts.
42:26By doing so, wave function collapse models can be built visually,
42:30and it may help in discovering new options that are required.
42:33You can simply specify a resolution of a grid to solve.
42:37There is also an option to use a grid actor, which
42:40is a blueprint that visually shows the bounding box of the grid.
42:43One advantage with the grid actor is that it
42:45can derive starter options based on static mesh
42:48actors that sit within the grid.
42:51On the left, you can see a few tiles pre-placed inside the grid actor
42:54and the two different outputs on the right.
42:57It is a great way to combine art direction with proceduralism.
43:01The city had so much complexity at street level.
43:04But once you went into drone mode, it became obvious
43:07that more complexity was needed from the bird's eye view.
43:10In total, there were over 2,600 unique rooftop structures made.
43:15The wave function collapse model contained just
43:17over 5,000 constraints, all of this with only 33 tileable meshes.
43:22Solving these structures one by one was not scalable.
43:26So we created a solution that can derive grids from point cloud data.
43:31We created editor utility widgets that
43:33can do some post-solve processing.
43:36Here, we are swapping wall materials, roof materials, and per-actor custom
43:40primitive data to create color variation.
43:43We can derive a grid from a solved actor and allow for arbitrary logic
43:47based around tile neighbor rules.
43:49In this case, we use neighbor rules to place props on walls.
43:54To re-use existing Houdini methods made for scattering props,
43:57we made a method for exporting roof-only surfaces.
44:00Roof prop scattering was done in Houdini
44:02and reassembled in Unreal Engine using point cloud data.
44:07And here are some wave function collapse results in the city.
44:23The wave function collapse plug-in was
44:25created by Joji Tsuruga, senior technical artist
44:28on the special project team.
44:31It will be available as an experimental plug-in
44:33in Unreal Engine 5.
44:35And we are excited to see what the Unreal Engine
44:37community will create with it.
44:39And now, here is Julien.
44:42JULIEN MARCHAND: Hi.
44:43My name is Julien Marchand, and I am an engineering director
44:47at Epic Games.
44:48Thanks for joining me today and see other crowd and traffic teams
44:53and powered by the Unreal Engine new AI technologies
44:56brought life to the city for The Matrix Awakens demo.
45:03Adding procedurally generated static content to the city,
45:06the remaining challenge was to immerse the player
45:08by making the city truly feel alive.
45:11To us, it meant two things.
45:13We could not compromise on the density,
45:15and we had to grant the persistencies of this element inside the world.
45:19Filling the world with a vast amount of interactable autonomous agents
45:22became obvious.
45:23And in this case, the crowd and the traffic system
45:26became the natural choice for a city.
45:29To meet those requirements, we had to leverage
45:32a new technology called MassAI.
45:35Before moving on, let's take a step back
45:37and have an overview of the Mass Framework, a data-oriented design
45:40for Unreal Engine of which MassAI is a specific use case.
45:44The Mass Framework comes in three separate plug-ins--
45:47MassEntity, MassGameplay, and MassAI, which have a feature
45:50set built on top of the previous one.
45:53These are all coming experimental for the Unreal Engine 5.0 release.
45:57Let's start with MassEntity, which holds the data-oriented framework
46:01fundamentals.
46:02To better understand it, let's compare it
46:05with a classic object-oriented model, in this case, the Actor Component
46:09model for Unreal Engine.
46:11This model allows for a very flexible way
46:13of composing the logic of the entities,
46:15in this case actors that we want to bring into the world.
46:18This model has been a staple for video games development
46:21for many years.
46:23However, the flexibility of this model leads to code and data
46:26incoherency due to them being updated sequentially.
46:31Also, the way the actors are composed does not
46:33guarantee that their components are aligned in memory.
46:36Their access is almost guaranteed to lead to cache misses, which
46:40are expensive for the CPU.
46:42Moreover, components can sometimes become bloated, and part of them
46:46may not be required for particular actor.
46:49On the other hand, the Mass Framework for Unreal Engine
46:53proposes an alternative way of storing our data
46:55and decompose it from the processing logic.
46:58We have fragments that are very small data structures and entities, which
47:03simply hold pointers to the fragments that represent them in memory.
47:08The fragments of entities of similar composition
47:11are stored contiguously in memory.
47:14And finally, entities do not hold pointers to any data--
47:18so in this case, any fragment-- that it does not care about.
47:22As mentioned, we decorrelate all processing logic
47:25from the data composition itself, which is all within the fragments.
47:29To understand on which entities we want to execute some logic,
47:33we make use of an entity query.
47:36The entity queries filters the entities,
47:38which all splinters to the fragment required for particular logic.
47:43Processors are where the batch update will occur
47:45and where the logic will run.
47:47For example, we could be updating the position fragment of all entities
47:51at once by reading from their velocity fragments.
47:54But that can only occur if an entity has both of these fragments.
47:58This segregation is what enforces data and code coherency,
48:02minimize cache misses, and will trivialize concurrent execution
48:06in the future.
48:08These are the fundamentals that are pushing the limits on the numbers
48:11of entities we can simulate at once.
48:15The MassGameplay part of the framework
48:16is what allows us to tangibly bring mass entities inside the game world.
48:21This is where we'll define our mechanisms for spawning,
48:24visualizing, LODing, et cetera.
48:28Namely, the entry point to bring mass entities inside the world
48:32is the Mass Spawner, which is a regular actor
48:35that you can place in your level.
48:37Looking at the properties for the Mass Spawner,
48:40the only two things you should really care about are what to spawn
48:43and where to spawn these entities.
48:46We will be answering these questions in a moment.
48:51Our purpose was to bring autonomous agent inside the world.
48:54And this is where the MassAI plug-in comes
48:56into play, which encapsulates features for navigating the world,
49:00animation, behavior, et cetera.
49:04But more importantly, let's remember that what we wanted
49:07was to bring life to a city.
49:09So we were not shooting for just any AI.
49:13What we ultimately wanted was a crowd and a traffic system.
49:17And the way we build these plug-ins is exactly how
49:20we envisioned developers to leverage and extend
49:23the Mass Framework in the future.
49:25These plug-ins hold all behaviors that
49:28are specific to these use cases.
49:31Before getting into the specifics of the crowd and traffic behaviors,
49:34let's think back to the procedural data
49:36that we leveraged to answer one of the two aforementioned Mass Spawner
49:39questions-- where to spawn the entities.
49:43The answer to this question was the ZoneGraph,
49:45which is another experimental feature coming with the Unreal Engine 5.0
49:49release.
49:51The ZoneGraph is a lightweight design-driven flow
49:53for AI, which for the context of The Matrix Awakens demo,
49:56replaces our need for an AI nav mesh.
49:59It is an ecosystem of point by point corridor structures connected
50:02by intersections.
50:04It also stored actionable tags that can
50:06be either static, such as identifying a pedestrian from a traffic lane,
50:10or dynamic, such as an open or closed lane that will later be
50:14useful to dictate our AI behaviors.
50:18In fact, we leveraged the zone graph to have a density-based distribution
50:22of entities for both crowd and the traffic,
50:24whereas for the parked vehicles, we were
50:27more interested in a static point cloud distribution.
50:30The Mass Spawner supports both of these options,
50:32along with many others when it comes to deciding how to distribute
50:36mass entities inside the world.
50:38For the crowd, we merged a procedural data representing
50:41the pedestrian density along with the pedestrian lane network
50:47to create a network of interconnected and properly
50:49annotated ZoneGraph for the crowd.
50:52Although hard to see, the annotations denote what they are for--
50:55whether they are crosswalk, sidewalk, or intersection lane.
50:59They also hold information about the expected density
51:02and whether they are currently closed, as you can see in red,
51:05or not.
51:06These are the lanes upon which we generate
51:09a spawn location at varying distance based on the desired density.
51:14For the traffic, we did the same thing.
51:16We merged the procedural data representing the traffic density
51:21along with the traffic lane network to create a traffic ZoneGraph
51:25network and then generated the spawning location
51:27for the various vehicles.
51:29Note that for the traffic, we had to do that operation for both the city
51:33lanes and the freeway lanes, and then interconnect both networks.
51:38Finally, for the parked cars we simply
51:41directly spawn one at every location fed by the point cloud.
51:45And voila.
51:47Having distributed our entities across the map,
51:50we now need to answer the other Mass Spawner
51:52question, which is what to spawn?
51:54We are talking more than just visuals here.
51:57We need to be able to define how these entities will
51:59behave across the city.
52:01A Mass Entity definition is a new asset type
52:04that we use to easily address the list of fragments
52:07that our different entities use.
52:09Amongst the important aspect of the definition are, of course,
52:13the visual, what we want to show at which LOD, for example.
52:17The LOD parameters themselves are defined there.
52:20What are the cutoff distances?
52:21What are the visibility angles we care about?
52:24And what is the maximum budget per LOD?
52:27These two mechanisms work out of the box with the Mass Framework
52:30and are paramount to supporting an extra large number of entities.
52:35This is also where we added all necessary fragments required
52:38by the relevant processors to simulate the Mass Entity's behavior.
52:42And for that, we took slightly different approaches for the crowd
52:45and for the traffic.
52:48For the crowd, we made use of a StateTree, which
52:50is another new technology coming experimental with the Unreal Engine
52:545.0 release.
52:56It serves as a scalable and general-purpose state machine
52:58with an intuitive and compact UI presented by a decision tree
53:02structure.
53:04The entire crowd behavior can be seen on screen.
53:07And it describes all the states that a current member
53:09could be in, whether it be wandering, idling, fleeing, et cetera.
53:15The state is selected by running a top-down evaluation of the entry
53:19conditions and run its associated parameterized task.
53:23Once completed, the tree will route to its default transition
53:26state unless a different transition condition is met before.
53:31Another benefit of the StateTree is that it
53:33is tightly packed in memory, which blends well with the Mass Framework.
53:39In this capture, we see the various crowd behaviors at play,
53:43such as walking around, waiting at intersection.
53:46But we also made the characters believable.
53:49The animation coverage proposes various walk style and speed,
53:53smooth start and stop animation, and also manages
53:56a head look-at, along with the upper body orientation
54:00independent from the character's current velocity.
54:03We also note that it reacts to it surrounding,
54:06be it the player or the other crowd members.
54:10Specifically, we developed a new force-based avoidance
54:13that handles both dynamic and static obstacles at high performance.
54:18The crowd members can also respond to physical collision,
54:21such as playing a one-off animation before resuming their behaviors.
54:26We use the animation state machine to run on the closest crowd
54:29members in sight.
54:30But all the data it used came from the mass animation
54:33processors that was feeding the relevant data from the entities
54:36fragment.
54:37As mentioned previously, the mass LOD system
54:40is key to handling a vast amount of entities.
54:43In this case, our 35,000 crowd members.
54:46By default, the framework handles four different LODs, along
54:49with a significant float point value between those levels.
54:53On the screen, we can see in red 10 LOD, which
54:56are full fledged MetaHuman actor with facial animation; in yellow, 20
55:00medium LOD, also full-fledged MetaHuman actor with lesser
55:04facial animation; in green, up to 500 low LOD, which are
55:08lightweight vertex-animated meshes.
55:11The rest are not visualized.
55:13If you're interested in knowing more about how the crowd MetaHumans were
55:16made and how we animated the instanced Static Meshes,
55:19I encourage you to listen to the character tech talk.
55:23Because of the Mass Framework, we also
55:25attained a great level of persistency.
55:28No matter the LOD we were switching at or from,
55:30it allowed us to always retain the current behavior evaluation
55:35versus the visible so that if you go far and back,
55:38you can find that cool-looking character again.
55:41And finally, allowed for smooth animation transition,
55:44even from switching from a vertex animated mesh
55:47to a full-fledged actor, since the runtime animation
55:50data was persistent in the animation fragment.
55:54For the traffic, rather than using a StateTree,
55:56the behavior were all programmatically made
55:59in Mass Processor, which is to say that the framework does not
56:02constrain you to an approach over the other.
56:05The most typical behaviors were the follow on lane and wait
56:08at the intersection.
56:09To prevent vehicle bumping in each other,
56:11we relied on a queuing of vehicles along a lane.
56:14A vehicle would know the distance to the next vehicles
56:17to manage the speed at which it goes.
56:19When a vehicle intersection lane closes,
56:22upstream vehicles will speed down and stop at the end of the lane.
56:25Not unlike the avoidance for the crowd,
56:27the traffic vehicles also had to manage
56:29obstacles, such as the player standing in the way
56:32or when creating car crashes.
56:35These obstacles could supersede the next vehicle
56:37for determining the speed to go at if the obstacle distance was shorter.
56:41In the absence of the player, we assumed the simulation
56:44to run orderly.
56:48To switch it up and to allow the traffic to circulate everlastingly,
56:52we have to support lane changes and lane mergers.
56:55This occurred when an empty spot was detected on a neighbor lane.
56:59We would create a ghost entity to simulate the future presence
57:02of the vehicle on that lane.
57:04Once the transition was complete, we would then remove the vehicle
57:07reference from the previous lane.
57:09This was particularly useful for getting in and off the freeway.
57:13Similarly to the crowd, the vehicle at their own LOD parameters.
57:17There are a total of 17,000 traffic vehicles and 38,000 part
57:22but drivable vehicles.
57:24On the screen, we can see in red 10 high LOD, fully simulated
57:28physical vehicle actor that can be deformed,
57:30destroyed, and interacted with.
57:33In yellow, up to 150 medium LOD that are still running physics
57:37but with simpler suspension and all simulated within the Mass Framework.
57:42In green, up to 5,000 low LOD which are
57:45instanced Static Meshes, following simple curves and following position
57:49updates.
57:49The rest are not visualized.
57:51For the traffic vector spawning, since it
57:53could happen at a moment's notice, we made
57:55use of vector pulling to alleviate the cost.
57:58Also, note that the parked vehicles are not just there to be pretty.
58:01The player can up in any one of them, drive around,
58:04abandon them, and get back to them later on.
58:08Finally, the last step was to make the crowd and traffic system
58:11work together.
58:12And that's where the intersection coordinators came into play.
58:15This processor was in charge of managing the phase four--
58:18which lanes to open at which time, when was it OK for the pedestrian
58:22to cross, when could we allow vehicle turning and when we couldn't.
58:26Had a high number of intersection configurations,
58:29which added to the complexity.
58:31Lastly, to manage the flow density for both the crowd and the traffic,
58:35this is where decisions about where to go next
58:37were made as to prevent the downtown populace to invent the low town
58:40for example.
58:42In the end, it was not always perfect.
58:44We were not city engineers by any means.
58:47And that is why we had to iterate on the city and density a lot.
58:50Who thought that combining a super high density
58:53with a persistent simulation and adding a stop
58:56light at the end of a freeway was a good idea?
58:58But it did create some nice anecdotes.
59:01To conclude, to bring life to any world,
59:04it is true that having an high density is a great foundation.
59:07And the Mass Framework helped achieve that easily.
59:09Even if there exist tricks to fake it, who is to say,
59:13perhaps nothing exists outside of the current view you are seeing.
59:16However, if density is great, persistency is king.
59:21It really helps immerse the player inside the world.
59:24And we hope that this cut will speak for itself.
59:28You can note the green moving square to be the traffic
59:30vehicles, the blue ones to be the parked vehicles,
59:33and the little white ones are the crowd members on the sidewalk.
59:36This is the result we ended up with-- starting small, leveraging
59:40the generated procedural data, all the way to simulating upwards
59:44of 100,000 mass entities at once, to the crowd and the traffic system.
59:48This is how we brought life to the city.
59:50And it is a result we are very proud of.
59:54Thank you for listening.

About this transcript

This page contains the full transcript of The Matrix Awakens: Generating a World | Tech Talk | State of Unreal 2022 by Unreal Engine, generated from the public captions YouTube serves with the video. The transcript has 10,074 words across 1,181 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.