YT YouTube2Text

YouTube2Text → Transcripts → Realtime Global Illumination in Enshrouded

Realtime Global Illumination in Enshrouded — Transcript

by Graphics Programming Conference · 8,819 words · 1,236 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01okay so hello everyone my name is Jacob
0:04and I'm senior rendering engineer at
0:06King games and I'm going to be talking
0:08about realtime Global Illumination in
0:10aned today so first I will be walking
0:13you through King games and aned if you
0:15don't know that and the benefits of diic
0:18global illumination and why we actually
0:19choose that then I will show you the uh
0:22building blocks which we have which is
0:24the spatial cascaded cache and the
0:25science distance field race which we
0:27have then I will show you the three
0:29types of prop which we have there are
0:31the froston probes the world cash probes
0:33and the ambient probes and then I will
0:35do a full pipeline Overview at the end I
0:38will show you spatial temporal sampling
0:40combined with our screen space
0:42Reflections and the foxal volume for ful
0:45and some optimizations of course at the
0:47end there will be a conclusion and I
0:48hope we will have some time for
0:50questions because this is a rather long
0:51talk so King games is an independent
0:54studio uh based in Frankfurt and we are
0:57around 70 people with 23 programmers and
1:00out of that seven Graphics programmers
1:02we are mostly fully remote and have our
1:05custom Tech since 2005 and shipped 38
1:08games on all platforms ined is our most
1:12ambitious project by far it is a voxel
1:15based survival RPG game with up to 16
1:17players which is in Early Access since
1:21January 2024 we already have over 3
1:24million both players and we Sorry over 3
1:27million players both game and we have
1:30four big updates already which means
1:32that we have every two to three months a
1:34big update we are currently only for PC
1:37but the consoles are in in work and will
1:39be available for the release we are
1:41running exclusively on wlon right now
1:44and one small hats up there will be a
1:46talk from Julian Conan and Lucas filler
1:49about Wulan in and shrouded later today
1:52about 16:30 so don't miss it we have our
1:55custom engine which is called holistic
1:57and we do C++ for everything we can so
2:02why Dynamic Global illumination well for
2:05us it was an primary choice because we
2:07have a destroyable world uh the player
2:10can toss a granite everything can change
2:11instantly the light environment changes
2:13you have to adapt to that so there is no
2:15room for pre-baking anything also it
2:18makes the scene feel alive especially if
2:20we have Dynamic weather and day night
2:23changes as we have it also makes our
2:26artists pretty happy because they are
2:29working in an
2:30which is what you see is what you get
2:32and when they Place some object into the
2:33scene it automatically gets the correct
2:35light environment and also and there
2:37more important contributes to the light
2:39environment so every object is part of
2:41the light
2:42scene so let's go over our building
2:45blocks the first one is our spatial
2:47cascaded cache which is a 32x 32x 32
2:50World align GD centered around the
2:52player and it has 12 Cascades this is
2:55how it would look like we have two ways
2:57to activate uh this uh cell and this is
3:00first by a death buffer and the second
3:03with a XY set position buffer important
3:07to know is that uh actually when we
3:09spreading this uh about the whole world
3:11this is an amount of probes which we or
3:14cells which we could never
3:17um work with in real time therefore we
3:20have a cap of 16,000 and we are keeping
3:22that all the time we have a 2 meter
3:25spacing at the start and we are doubling
3:27the spacing for each Cascade we are also
3:30garbage collecting inactive cells so
3:32this is really a cache so if you have a
3:33camera and your pen to the left and some
3:36of the cells are going out of the screen
3:38they are not going to be deleted
3:39immediately they will start losing
3:41quality if you are managing to pen back
3:44again till they uh expire you will just
3:47reuse them and they will not be resetted
3:49fully so to the second building block uh
3:53is um this is our science distance field
3:55representation of the world and the
3:57science distance field race it is a very
3:59much simplified version of the world as
4:01you can see in the video and you can see
4:03the uh different Cascades uh switching
4:05from one to the other uh so very very
4:08small objects are absolutely not present
4:10in the representation and median objects
4:12are only present at high and ultra
4:14settings the Vox of world is of course
4:17fully present this is optimized for
4:19Speed and of course Very Hardware
4:21independent and this is also the major
4:24uh decision why we actually uh went for
4:27SDF race because we could have and we
4:29actually ALS Al had some prototypes with
4:31some rate racing apis but we just wanted
4:34to be working with one version of the
4:36code on every other Hardware which we
4:38can uh have so this is what we went for
4:41uh it is also very much scalable because
4:43we can choose the resolution of the
4:45science distance fields and also pick
4:47which ones of them we will be using for
4:49different Hardware around that there is
4:51the bvh acceleration structure to make
4:53it
4:54fast so let's go through our three prob
4:57types the first probe type is the custom
5:00probe uh this is using the cascaded cach
5:03which I was talking about earlier and in
5:07this case we are using the death buffer
5:08to spawn uh the probes or activate the
5:11cells on top of geometry uh as you can
5:14see at the end of the uh Cave um there
5:17is the Cascade uh jump to the bigger
5:19Cascades and the probes are taking
5:21double the space each of these probes is
5:25of course shooting signs distance field
5:27rays in all directions and each of these
5:29hit points is then shooting another set
5:32of rays S field Rays into the
5:34directional Sun uh directional light so
5:37Sun uh for
5:41shadowing also for local global
5:43illuminance global illuminance we are
5:46actually also resampling the probes
5:48themselves when
5:49shading we choose the octahedral
5:52encoding which is a very efficient way
5:54to store a threedimensional vector in a
5:562d space and it's also very scalable we
5:58are reusing re using it over the place
6:02and there is still one thing to take in
6:05account with this that uh if you want to
6:07sample uh an octoral map you have to
6:10create a pixel border by hand so that
6:12your sampler is actually wrapping nicely
6:16around this is how an Octor encoded
6:19Radiance map would look like uh for
6:21ultra settings we are using 32x 32 then
6:23mid settings are 16 and low end settings
6:26are 8 by8 for the radiance Maps but for
6:28IR Radiance we are using 12x 12 for
6:30Ultra and mid and low end is only 8X 8
6:34this is sufficient because this is
6:35diffus lighting and you do not need so
6:37much detail there uh we are storing all
6:40of this in an nlas of 128 by 128 which
6:44is again this 16,000 probes which we can
6:47activate on the top you see the uh
6:49Radiance and on the bottom the IR
6:51Radiance
6:52Atlas so all the probes are actually
6:55encoded in E5 RGB 9 so it's uh basically
6:58a new in 32 and the distance
7:01unfortunately we had to encode it in
7:02float 16 rgba because we need the
7:05Precision when occlusion
7:07calling all right the second type of
7:10probes are the world crash probes so so
7:12far we had only probes in front of us
7:14but these probes are actually shooting
7:16Rays also behind the player or on other
7:19places which uh the frost and probes
7:21were not spawned and we still need to
7:23sample Global illumination there
7:25therefore we are spawning World cash
7:27probes at this position this again shoot
7:29sence distance field Rays into the world
7:32and can resample each other for local
7:34Global illuminants they can also sample
7:37the frost and probes of course if the
7:38ray hits some area where a frost and
7:40probe is located these are actually
7:42mutually mutually exclusive that means
7:44uh where a frost and prob is there will
7:46never be a world cash
7:48probe so um the technology behind that
7:51is absolutely the same as with frost and
7:53probes the only difference being that we
7:54have a resolution of 4x4 so this is for
7:57example how I said it is very scalable
7:59we just uh say that this probe is 4x4
8:02and that's it there is no other change
8:03in the code and we are uh encoding the
8:06irradiance as a spherical harmonic due
8:08to bandwidth reasons but we will be
8:10talking about that later in detail the
8:13cascaded cache in this case is a little
8:15bit denser as the probes are very low
8:17resolution and they do not store any
8:19occlusion at all that means that we have
8:22uh we have to have them small so that
8:23they do not overreach walls and do not
8:25start leaking so to show you how such a
8:28or how far such a two-l system would
8:30reach I prepared a small animation so on
8:33the right you see the player which is
8:35located in some Labyrinth and looking at
8:36the corner and therefore he will spawn a
8:39frost and probe there this Frost and
8:41probe will shoot a ray and at this
8:42position we will get a world cash probe
8:44so this is your first pounds which will
8:46reach the player actually then the world
8:49cash probe will shoot another ray into
8:51the world and the hit
8:53position um will be again shaded uh we
8:57will be not spawning other world Cas
8:59probes from from here on we are actually
9:01cing the chain here but we are still
9:03sampling the point lights and the
9:05directional lights so at least for Point
9:07lights you have three bounces for full
9:09Global illumination you have two bounces
9:12so in this particular scenario these two
9:14lights behind these many corners would
9:17actually contribute to the visible
9:18pixels of the
9:21player the third type of probes which we
9:23have are the ambient probes because so
9:25far we were only talking about uh the
9:27surface and all of these probes were
9:29just spawned next to some geometry so
9:32therefore we need something in the air
9:33two because we have a lot of Fu so these
9:36are the ambient probes it's another grid
9:38of 16 x 16 x 16 with eight Cascades
9:41around the player and we thought about
9:43something very interesting here and it
9:45is that we have all of these Frost them
9:47and World cash probes which are actually
9:48filling the whole world and they were
9:51already shot so we have basically a
9:53starting an endpoint which means we have
9:55a line in three dimensional space so we
9:58thought that we will be reusing this
10:00information and we intersect so we do a
10:02fast foxer travel and intersection with
10:04a bounding box so line bounding box
10:06intersection and then we fill um the
10:09information of the radiance along the
10:11ray so these probes are not shooting any
10:13Rays
10:14actually uh they are again encoded in E5
10:17RGB 9 and we actually found out that we
10:20do not need to pick all the Rays it is
10:22enough when we pick randomly around 10%
10:25of the whole Ray budget which we have of
10:28course we accumulate all Ray and
10:30calculate the radiance mean at the end
10:32that touched a certain probe and update
10:35its mean value so how it looks like alog
10:38together so these are the frost and
10:40probes as you can see they are not
10:42spawned beneath the cliff where the
10:44player doesn't see anything but the
10:46world cash probes are spawned there
10:48because the rays are hitting there and
10:50then we have the ambient probes around
10:51the plier the first Cascade and then
10:54iterating through all the Cascades up
10:55until 28 which is a little bit special
10:58so the last Cascade um is not centered
11:01around the player but in the center of
11:03the world and covers the whole world and
11:05is used as a backup F if everything
11:08failed so let's get over the whole
11:11pipeline right
11:12now at the beginning we start with
11:15updating the frosten probes so we have
11:18our death buffer and we have our
11:20cascaded uh cach so we will activate
11:23cells in this um with with this death
11:26buffer in the cascaded cash uh but since
11:29this is a world align grid a lot of
11:31probes will actually get stuck in some
11:34walls so we will need to find a better
11:36sampling position uh since we have the
11:38whole world encoded as a sence distance
11:40field we can easily find the nearest
11:42surface position and make a push out but
11:45this is still not enough because if you
11:46would just uh let the probe uh very near
11:50a surface you would get very much uh
11:53problems with with occlusion because of
11:55the system how we are um actually
11:58occlusion testing you will see that in a
12:00second so therefore we are offsetting
12:02into the camera and a little bit into
12:04the normal direction of that surface
12:07from that position we can safely finally
12:09shoot
12:10race um of course um there is also the
12:14thing of probe quality so if a probe is
12:17born absolutely new we assign it the
12:19maximum uh number of rays which is
12:21currently 512 rays and then we scale it
12:24down very quickly up until to 16 rays
12:27which is just checking around the
12:28surrounding uh for any light changes and
12:30if it detects that there is actually a
12:32lot going on it will crank up the number
12:35again uh we recycle old probes so if the
12:38quality reaches below zero we'll delete
12:40them and new probes will come in later
12:44we also register all of these active
12:45probes for all the indirect dispatches
12:47because this is a fully uh compute uh
12:50shaded pipeline where there is
12:52absolutely no jump into the CPU
12:54everything is on the
12:56GPU so after we've uh got our Ray amount
13:00so this is basically just a wish from
13:01the probes which how many Rays they
13:04would like to have but we have a fixed
13:06Ray budget so we have to account for
13:08that we just do a percentage comparison
13:11between the requested race and the
13:13budget and then scale all Rays for all
13:16all probes based on this percentage this
13:19is still a lot of rays to fill so uh
13:21what we do is basically each probe um is
13:25uh writing uh its index into a batch
13:28Index this is how we call call it in
13:30this case it is 16 but it is a variable
13:32so we can uh change that if if we have
13:35for example more Rays it makes sense to
13:38batch them by 32 or maybe 64 so all
13:41probes are writing their indexes into 0
13:4416 32 and so on and then when we do a
13:46linear Dispatch over the array array
13:49then each of these um possible arrays
13:53will check one of these so the nearest
13:55batch index and and get the probe ID out
13:58of that when you have the prop ID you
14:00know how many Rays you are and you know
14:02uh the first index of the first Ray
14:04therefore we can back calculate uh which
14:06exact Ray index we are and this is
14:08especially important because we will be
14:10generating directions right now so we
14:13first uh actually started with uniform
14:15Ray Direction but that was actually very
14:17horrible because uh we do not have the
14:20capacity to shoot too many rays and with
14:23um let's say 16 to 32 Rays you will have
14:26a very unstable probe uh which will
14:28flicker and will not propagate light
14:30very far so we went for important
14:33sampling and went for searching for the
14:35light so how you do important sampling
14:37is very very easy because um you just
14:40need two things first is a weight by
14:42which you will be calculating your
14:44samples in our case that's the radiance
14:47of a pixel and second you need a sum out
14:50of that so each probe in our case is
14:52calculating the sum of each uh array or
14:56sorry of each pixel of its probe and
14:58saving the sum in the probe data then
15:01for each Ray we just draw one random
15:04uniform number multiply it with with the
15:06sum and we get a threshold this is the
15:08dotted line uh in the middle after that
15:12um we can consistently start marching
15:15the uh pixels and accumulate the
15:18radiance values again up until a point
15:20when we reach the threshold or overreach
15:22the threshold and this is our important
15:24sample very important here is that you
15:26have to be consistent and you should uh
15:29um always do basically the same every
15:31frame but we found out that if we for
15:33example pick as a starting position the
15:35left upper corner and then March the
15:37probe linearly it is possible that if
15:40you have a big light in the middle of
15:42the probe and a small light at the
15:44complete opposite of the starting point
15:46you will never reach that point because
15:48uh we are using blue noise for random
15:50values and it's actually not uh very
15:53uniform it has problems so what we found
15:57is better actually is to cons instantly
15:59rotating the edge point so we start with
16:01the left upper corner then the right
16:03upper corner right bottom corner and
16:05left um bottom corner and start again uh
16:09from the left upper corner so we rotate
16:11so we basically merge four uh important
16:14samplings into one in the end and that
16:16works very well and finds all the
16:19lights so this is how it would look like
16:21in action you can see that I will be
16:23moving the sun and the probe will mostly
16:27hit uh only positions where the light
16:32is all right so now that we have
16:35generated the ray directions we can
16:37finally sort we are sorting by position
16:40and by Direction so let's talk first
16:42about the position we are reusing the
16:45cascaded cach which we have and we are
16:47just when asking for an ID just
16:49increasing the Cascade offset which
16:51sorry the decade index by an offset so
16:54what does it mean is if you look at the
16:56image on the left uh in the left bottom
16:58Cor Corner uh this would be our uh probe
17:02which we would like to sort somewhere
17:03into so if we would increase the Cascade
17:07by one then we would get actually a bin
17:09of the size of the Blue Block if you
17:11would increase the Cascade again then we
17:14would get the whole red block and if we
17:16do it the third time we will get the
17:17whole green block so we are having an 8X
17:208 by8 uh probe bin so this is how it
17:23would look like I'm moving the mouse and
17:25always targeting one bin and all the
17:27Rays inside been are shown as you can
17:30see these are pretty Divergent a lot of
17:33them are sky ray a lot of them are going
17:35into the Horizon and are super long and
17:37some of them are hitting the ground and
17:38are super short so this is not something
17:41that would really help um when shooting
17:44when we would just sort by the position
17:46so we also do a sub bin with again
17:49octahedral Direction en coding with
17:534x4 and we can finally
17:56shoot so after we have shot our first uh
17:59race from the frost probes uh we have
18:02our hit positions uh which uh can
18:05finally spawn the world Cas propes again
18:07the system is exactly the same this the
18:09same shaders actually which update the
18:11worldish propes the only difference
18:13being that they are not offset it into
18:15the camera Direction anymore but they
18:17are offset it into the their origin
18:19because they came from some direction
18:21other than the camera
18:23probably
18:25um then we also populate s and shoot
18:28them exactly as we did with the frost
18:29and probes but there is one little trick
18:32here and that is the ambient race I told
18:34you that the ambient probes are actually
18:36not shooting any Ray but it is not
18:38actually true uh because there is one
18:41special scenario where you do not have
18:43any Frost them and World cash probes and
18:44that's that's it when you are looking
18:46directly at the sky then you have no
18:48probes whatsoever and all of these
18:50ambient probes would just starve
18:51therefore we are detecting this and if a
18:53probe sees that for a couple of frames
18:55it didn't get updated it will request
18:58race for the next frame so these are the
19:00Rays which were requested so we group
19:02them together with the world cach and
19:04populate sort and shoot them all
19:07together so when we have the hit
19:09positions we can finally register Shadow
19:11rays which will be shot after that and
19:14also to avoid Shader branches we are
19:16separating everything into terrain hits
19:18Sky hits and building hits because these
19:19are three different shaders and uh that
19:23hurts performance if you are putting
19:25them all
19:26together so the shading function uh
19:30consists of the these uh five steps the
19:33first one is that we sample the emissive
19:36value from the voxel then we sample the
19:38directional light and also the shadow
19:41Ray result which we shot before then we
19:44sample Point lights we have a special
19:47cascaded cache for Point light calling
19:49which is called the light cache and each
19:51cell in here is storing a list of Lights
19:54which uh reached this position so we are
19:57looping through this list and
19:59calculating the contribution of all the
20:01point lights we have our own science
20:03distance field Shadows which are also
20:05checked here for shadowing after we are
20:08done with the directive uh direct
20:11distribution we can go on uh with the uh
20:14Global elimination so we sample the
20:16frosten probes if available and if not
20:18we also try to sample the world cash
20:21probes all right so now that we have uh
20:25shaded everything we are finally able to
20:27blend and up update our probes with new
20:30information but let's first talk about
20:33variance so in this example I have a
20:36probe which is deep into a cave which
20:39has absolutely no light source so it's
20:41pitch black inside and outside it's
20:43fully Sunny without any Cloud so it's
20:45very strong light now even if you would
20:47use the 32x 32 uh octahedral encoding
20:51which is actually already quite a lot
20:53you would still have 11° opening angle
20:56per pixel which is for short distance
20:58quite fine but for a couple of hundred M
21:01this start to be a lot and we have RS
21:04which go around a couple of kilometers
21:06so we can imagine how much area such a
21:08pixel will cover this gets even better
21:11if we have some emissive surface
21:13surfaces outside which we have actually
21:15quite a lot so we struggled with this
21:18scenario quite a bit up until a point
21:20where we found out the moving average
21:23estimator so this is a two-level
21:25indirection system which is using the
21:27shortterm mean and the long longterm
21:28mean and then a couple of other
21:30variables like the variance B based
21:32blend reduction variany and
21:34inconsistency and also the shortterm
21:36mean wait sorry that I already said and
21:40uh this is not something we came up uh
21:42by ourselves uh this is well documented
21:44inter rate racing gems 2019 book or 2018
21:48stochastic all the things talk this is
21:51quite bandwidth heavy though uh as you
21:54can see we have the the long-term mean
21:56which is Sav directly in the probes
21:57which is just one un32 because we are
21:59using E5 RGB 9 uh but the rest the the
22:04shortterm mean the variance based blend
22:06reduction Varan in inconsistency are
22:08actually eight floats so we are encoding
22:11them in flow 16 so in the end it's an u4
22:14together with the longterm mean it's an
22:16U five which is quite a lot but since
22:19our Radiance probes are actually the
22:21base for everything else which we will
22:23have from here on we thought that this
22:25is actually worth the
22:27price so for for the uh occlusion
22:30testing and the distance blending we are
22:32using a per pixel depth function we are
22:34reusing the same rays which we already
22:36had so we are not shooting any extra
22:38Aries here and we are clamping them to
22:40the uh spacing position or the probe
22:43cage uh as we do not care about
22:46distances further away and we are
22:49inspired by the Varan Shadow mapping
22:51approaches in secra 2021 Global based
22:54and surles or the Nell 2006 or the DGI
22:5820 9 all of the modern GI Solutions are
23:01somehow depending on this system then
23:04when reading we are using chbf
23:06inequality which is an efficient way to
23:08find slopes and we are storing therefore
23:11per pixel uh mean and uh mean squared
23:15this works well very very well in LDS as
23:18you can preload all the arrays into LDS
23:20and then just filter them in in in
23:23shared memory but be very very careful
23:26with Ray guiding because this system
23:28depends on uh being constantly feeded
23:32with new samples and if you do important
23:34sampling you might just have one or two
23:36rays on One Direction of the probe that
23:38would basically destroy everything so we
23:40are doing uniform sampling after X
23:42frames which is in our case five frames
23:45and it keeps the system alive and we
23:47have no artifacts
23:49whatsoever so after we finally uh got
23:52all the radiance values into our probes
23:53they are absolutely fresh and new but
23:55they are still quite flickery even with
23:57important sampling this was not enough
23:59so we need to increase the samples and
24:02uh increased samples means faster
24:04convergence to the information which you
24:06need and less liquoring so what we'll do
24:09is again important sample we look at the
24:11area around the probe and again since we
24:14have the sum of radians inside of a
24:17probe we can again calculate the sum of
24:19all the sums inside of such a bin and
24:21then consistently marching this again
24:24drawing a random variable and if we
24:26reach the threshold then uh we have our
24:29sample we of course do do occlusion
24:31testing because we are not saving only
24:33the mean and mean squared which is
24:35clamped to this spacing but we are
24:37actually also saving the direct mean uh
24:40in the probe so we have uh exact
24:42information about how far a probe
24:44actually sees so we pick our candidates
24:47and then try to reproject uh between
24:49probes uh of course again when we
24:52project in between probes we do uh
24:54reject samples which one probe wouldn't
24:56see because we have the distance saved
24:59in the
25:00probe so this is how it would look like
25:03in slow motion the yellow lines are
25:05actually the candidates which were
25:06picked this Frame the blue lines to the
25:09red crosses are actually the probes
25:11which were picked as candidates I'm not
25:13sure if that's too visible it is
25:15actually in the middle there is a bush
25:18and it should showcase that the probes
25:20which are selected are never on the
25:21other side uh one important note here is
25:25that we are actually doing two rep
25:27projections we are rep Pro projecting
25:28the new Rays from this Frame from r one
25:31probe to the other but we are also
25:32reprojecting the history these are two
25:34differen shaders and two different
25:35dispatches all of them uh each of them
25:37is doing it its own stochastic draw and
25:41it is important because if when we only
25:43reprojected the race since we are again
25:46important sampling this is quite biased
25:48and was uh starving some positions and
25:51uh other positions were overwhelmed with
25:53information and started flickering due
25:55to very Divergent light information
25:58so what we also thought about is that we
26:00can reproject the history so the
26:02long-term mean but by doing so you
26:06basically having a break on your
26:08convergence because you are reprojecting
26:10old values so we are having a ratio here
26:12so it's a ratio 4 to1 four Ray R
26:15projections to one history R projection
26:18which keeps the um probes very stable
26:21then propagates light hundreds of meters
26:23into a cave and is still very very
26:26reactive to light changes
26:28so after we finally Blended all the
26:31radiance from our probes and from the
26:33from the neighbors we are finally able
26:34to generate IR Radiance this is a far
26:37simp very simple approach where we just
26:39uh taking a direction which we want to
26:42uh generate Radiance for uh go through
26:44all the Rays and cin weight them uh and
26:47at the end divide by the number of
26:49samples to match the radiance mon Carlo
26:52estimator this is how it how IR Radiance
26:55probes would look like they do not have
26:57too much of directionality but this is
26:59exactly as you would expect from diffuse
27:01light
27:03contributions all right so now next to
27:06the uh blending of radians we also have
27:09to care about specularity so we have to
27:12pre-filter our Radiance values as we
27:14have uh stored direct direct Radiance
27:17values in our probes this would be
27:18already usable for perfect mirrors but
27:21we need something for roughness bigger
27:23than Z too so what we came up with is
27:27basically a simple C in weighted blur
27:29and cin weighted down sample we have
27:30different settings for different uh for
27:33different settings and uh we have a
27:37ground proof mode which we compared
27:39against that and tweaked it so it is
27:42quite the same as in physically based
27:43render of course it is not physically
27:45based but it is very very fast to
27:48generate so these are the uh single
27:52levels where I'm skipping through all of
27:54the level so MIP levels let's call them
27:56MIP levels for the future
28:00all right so now that we have finalized
28:03our Frost and probes we can finally go
28:05to the world Cas probes again the system
28:08is exactly the same as in frosten probes
28:10the only difference is being that the uh
28:13texture is just 4x4 octar encoded and we
28:16have no pixel border because we will be
28:18never sampling this texture we will be
28:20just generating irradiance out of that
28:22so we do not need the pixel border uh
28:25for IR Radiance we picked spheric
28:28harmonics due to bandwave reasons
28:30because if we it it has quite the the
28:33same uh directionality as a 4x4 octoral
28:36map but with a 4x4 map we would use 16
28:40uh floats in this case it is 12 so it's
28:42a little bit smaller and it helped with
28:45performance on lowend
28:47devices now that we have finally
28:49generated all our Frost and World cash
28:51probes we are able to sample them we
28:53have um uh we have two screen space
28:56reservoirs uh which are temporarily
28:59reprojected and spatially filtered so
29:01DED the input for this pass is uh the IR
29:04Radiance probes the radiance probes and
29:07their respective myips and the world
29:09cach probes but just the spherical
29:10harmonic spherical harmonics Radiance
29:13contribution and then the output is the
29:15screen space IR Radiance texture and the
29:18screen space reflected Radiance texture
29:20which was generated by reading out the
29:22roughness and interpolating between
29:24these mips again both are encoded in E5
29:27RGB 9
29:28so it's a simple U
29:3032 so how do we sample that that's very
29:33easy we have a world space align prob
29:35grid so we basically do trilinear
29:38interpolation so in 3D it would like
29:40look like this you have always eight
29:42candidates which you have to uh wait and
29:45we are actually not sampling all of
29:47these we are stochastically picking one
29:49of these candidates per frame based on
29:51the weight the weight is a combination
29:53of the triline sampling value and the
29:55chabby chef uh occlusion test we also do
29:59a cross Cascade skipping that means that
30:01we have a little bit of an overlap
30:04between the Cascades and then we have
30:06the start of the overlap which is nearer
30:09to the player and then the end at the
30:11start of the overlap the weight is zero
30:13so we will always get the smaller
30:16Cascade in the middle of the overlap
30:18it's 50/50 that means also the probes uh
30:21sorry the Cascade will be jumped with
30:2350% uh probability and then at the end
30:26it's one so we will only the bigger
30:28Cascade this makes a very smooth
30:30transition between
30:32Cascades so this is how uh ass sampling
30:36cage um is considering candidates and
30:38you can see that the black probes are
30:40actually not considered so there's no
30:42Red Cross inside of that there is a
30:44reason for that um and that is that
30:46these probes are actually dead they were
30:49um they were spawned because the pushout
30:52told us that actually this is a good
30:54position but since we have multiple
30:56Corners actually such a push out can end
30:58up in other geometry so we have to check
31:01if a probe is actually able to shoot
31:03Rays we do that by just uh summing the
31:07distance of all r a probe wor shooting
31:09and if it doesn't cross a certain
31:11threshold which is 1 mm we know that we
31:13are inside a wall these probes are still
31:15alive and shooting around 16 Rays per
31:18per frame so that they still check if
31:20things maybe didn't change because for
31:22example player could just pick a pickaxe
31:25and and hit once there and the probe
31:27would bee
31:28fre so for the hisis when we are
31:31reprojecting from the last frame we are
31:34basing this on roughness and the
31:36reflected movement Vector uh we D noise
31:39after that we halt on 33e and the
31:41weights for that are depth and the
31:43normal for irradiance we would actually
31:46not need Tod noise because this is quite
31:48stable uh but for irradiance sorry for
31:51reflected Radiance this is so for the
31:53specular contribution this is very much
31:55needed because you have a lot of high
31:57contra
31:58lines and areas where there is a sudden
32:01jump from black to white therefore we
32:03need to den noise to get a stable
32:06image so now that we have generated our
32:09two IR Radiance and reflected Radiance
32:11textures we are finally able to call the
32:14the shading path this is necessary now
32:17because we will be still using screen
32:18space Reflections to get information
32:21from this and uh and combine that with
32:25the specular GI which you will see in a
32:27second so for the diffuse part we are
32:30using a sample uh albo divided by pi
32:35right so it's luran reflector multiplied
32:37by Radiance but we found out that it's
32:40actually not energy conserving and
32:41Heights Reflections in certain scenarios
32:44therefore we are having a roughness
32:46based schlick frel approximation where
32:48we are basically estimating the uh
32:50specular contribution and do one minus
32:53specular contribution to get uh how much
32:56of the diffuse contribution tion we
32:58should actually apply for the specular
33:01part we are using split split sum
33:03approximation but a little bit uh more
33:05about that in a
33:08second so stochastic screen space
33:10Reflections we are using them uh to to
33:14get uh information from the scene which
33:16we do not have in the sound distance
33:18field because uh characters for example
33:21are not present in the science system
33:22field world that's that means that we
33:24would never have refle reflecting
33:27characters in the game right now we uh
33:29used the ffix stochastic greenace
33:31Reflections library from AMD and
33:33modified it slightly to our needs so
33:36what it looks like in combination on the
33:38right top you see the result of the
33:41screen space reflection marching and on
33:43the bottom you see the specula GI result
33:45we have to combine these to get the
33:47final image on the
33:50left one thing though is that we still
33:53have the problem of Two Worlds that
33:55means that the player uh is seeing in
33:58this particular scenario just uh some
34:01vegetation and both Rayes which are
34:03highlighted are green but there is also
34:06a probe above the head of the player and
34:09it sees a completely different world so
34:12this is quite a problem sometimes but uh
34:15luckily we are actually TR um
34:17interpolating between um the GI and
34:20Screen space Reflections results so it's
34:22never a hard cut and also usually no one
34:25notice
34:28so um after that we can finally apply
34:30the specular Reflections so the input is
34:32the scene color uh combined with the
34:34diffus G already and the screen space
34:37reflection uh marching combined with the
34:40reflected Radiance texture and we do a
34:43simple split some approximation for the
34:44brdf and the output is specular
34:46contribution times reflected Radiance
34:49plus scene color so this is how it would
34:52look like on characters on the bottom
34:54you have just the specular Reflections
34:57and and on the top it is the final lit
35:02scene so these are two Mages so it
35:05doesn't work only for for Metals this is
35:07also for any other PBR materials you can
35:10also see the walls or the barrel is
35:12reflecting quite nicely and all of this
35:15is just diffuse sorry uh indirect uh
35:17lighting from this small uh opening in
35:20the
35:22ceiling now we are getting finally to
35:24the ambient probes at first we need to
35:27update them right so we we are scrolling
35:29them so we uh get the information from
35:31from last frame and update its new
35:33position and then we also do a
35:35visibility check it means they are kind
35:37of occlusion tested and only probes
35:40which are visible to the player are
35:42actually sampled but updated are all of
35:45them but just for these which are in
35:47front of you we will be generating uh
35:50some extra Rays to do a visibility check
35:52so a visibility check is very easy
35:54system where you are shooting just six
35:56Rays for each probe into all the
35:58directions and just basically clamping
36:01your bounding Box by that this of course
36:04can have some artifacts if you have
36:05something uh very small in the middle of
36:07the probe and there is nothing U around
36:10that because the Ray would hit such an
36:12object and would report that there is
36:14actually uh so so you would decrease the
36:16bounding box too much and would actually
36:18miss quite a lot of space but we are not
36:21using these probes uh to um to light up
36:24fog and other uh effects directly and
36:28when sampling these we are jittering the
36:29position inside of each probe quite a
36:31lot so this is actually not a
36:34problem uh so as I said uh before uh we
36:38are reusing all our rays which we have
36:40from the frost them and World cash
36:41probes we take around 10% of that and
36:44make use of the information that we have
36:46a start and end point in uh the
36:48threedimensional space and do a bounding
36:51box intersection with that uh so this is
36:54again not uh from us it's um a r
36:57marching algorithm which is linked below
37:00and it's very fast
37:02actually there is still one problem
37:04though that we cannot just simply blend
37:06in the Rays because all of our probes
37:08are on the top of a surface uh and in
37:11this particular case you see a frosten
37:13probe on the surface it's having um it's
37:16it's uh hitting a array on the bottom
37:18where there is let's say grass therefore
37:20the contribution would be green and the
37:22ambient probe is colored green on the
37:25other hand we have actually a sky ray
37:28which hit the sky and uh got blue color
37:30so we color all the uh ambient probes
37:33which intersect this Ray with
37:35blue and this is actually how it would
37:37really look like in game then you would
37:39uh you would start running around and
37:41the ground would be green and everything
37:43above one meter around would start to be
37:46just uh just blue so this is not the way
37:48to go what we actually do is we query
37:51the other side of the probe as we have
37:53the uh the distances saved and extend
37:56the start position of the ray
37:58and therefore we are able to apply the
38:00full range of green and blue to the
38:02whole
38:04area so now that we have
38:08um accumulated our Ray how we do that is
38:11again into an octahedral uh encoded uh
38:14buffer because one uh probe could have
38:18thousands of rays intersecting it and
38:20when we are accumulating it with an
38:21interlocked at that would be quite a
38:23bottleneck therefore we are splitting it
38:25into 4x4 and uh sorting by Direction and
38:29at some point which is right now we have
38:31to again group them together so we
38:33calculating one mean value out of all of
38:35these Ray and then update one moving
38:37average estimator per probe uh to make
38:40the probes very reactive to light
38:42changes and still quite stable but not
38:46stable enough when we were flying around
38:48the world very fast uh it was still
38:50flickering if you intercepted some very
38:53emissive surfaces so we needed another
38:55indirection and that's our fr volume so
38:58it is a 32x 32 uh grid of the screen
39:02which is then again subdivided into the
39:05depth with 384 slices for Ultra 256 for
39:09Mid and 128 for low-end devices the
39:12distribution is not linear is actually
39:15taken from Doom CRA uh and is very fast
39:19because you can pre-compute most of it
39:22on the
39:23CPU uh from here on we finally sampled
39:26the ambient Pro and have something to uh
39:30to sample for the
39:31fog and also yeah very important uh this
39:34is very fast because before uh we were
39:37sampling eight probes again like with
39:39the aradian uh texture before plus
39:41occlusion which is very expensive and
39:43since we have Dynamic weather which can
39:45be 7 to 10 kilm away from you we need
39:49quite some samples so that the weather
39:51still looks good therefore we needed
39:53speed and the foxal volume also gives us
39:56this because we are sampling just one
39:57un32 instead of these eight propes plus
40:01occlusion of course it is temporarily
40:03reprojected and the hisis is based on
40:05relative foxal movement speed to the
40:07camera this is how the fril would look
40:09like uh in the middle there's a cross
40:12representing the color and you see that
40:13the spacing is getting a little bit
40:15bigger over distance in this debug
40:17visualization there are actually
40:19occlusion Cults just so that I'm able to
40:21fly through them but they are actually
40:23not occlusion cult in um in the the real
40:27game because we found out that behind
40:30geometry uh behind vegetation like trees
40:33uh we would get some flickering I know
40:35there are some techniques how we can
40:36tackle that but since the whole pass on
40:38lowend devices costs around 50 microc
40:41this is quite
40:43fine so now that we have generated our
40:46uh fro of volume we are finally able to
40:49sample it with frock a
40:53Fu so this is how it looks like when you
40:56are flying very flying very fast this is
40:59actually double the speed of any the
41:02fastest glider we have in the game so
41:04this is nothing a player could achieve
41:05and it's still a rock stable and still
41:08has locality so we are not bleeding the
41:10red part into the blue part and so
41:16on all right so let's get into
41:19optimizations it looks like I'm on time
41:21actually this is great so um first of
41:24all uh GI is very bent with heavy so try
41:28to reduce
41:29bandwidth everywhere where you can the
41:32steam deck was especially uh complaining
41:35about this so we had to uh do quite some
41:38stuff here we are encoding most stuff in
41:41E5 RGB 9 this is a very nice format
41:44because it keeps Grail scale values and
41:46for the IR Radiance and reflected
41:47Radiance values as you could see they
41:48were mostly whitish so uh keeping Grail
41:51scale is very important it is also
41:54possible to directly load it into LDS
41:56and just decompress after you had them
41:59accumulated in local data store this is
42:02especially important on Steam deck
42:04because it has very low uh shared memory
42:07and therefore it couldn't start too many
42:10waves and groups if we did not do that
42:15then we are R sorting by Road position
42:16and Direction and reusing the cascaded
42:19cache for uh for this we also sort our
42:23head points into Sky terrain and
42:25building there's also designs field
42:27models which we have which I talked
42:29about earlier they are currently uh not
42:32uh written here because this is actually
42:34a buck which we have right now they are
42:36actually sorted into the terrain by
42:38accident but this is something I will
42:39change on Monday
42:42so so we have actually a fixed Ray
42:45budget as I said and a fixed buffer
42:48sizes for everything and of course
42:51adjustable settings for every system
42:53which we have that means uh you can very
42:55easily just change
42:57a lot of settings by just editing a
43:00couple of lines of
43:02code so conclusion it is very much
43:05possible to write your own solid Dynamic
43:07GI system nowadays I think there is a
43:09lot of resources out there a lot of Open
43:11Source Code which you can uh pick and
43:14and read through and just fuse them all
43:16into your custom solution because we
43:18think that Global illumination is very
43:20worth it and it gives your game some
43:24more Dynamic and and and live I hope
43:28that you uh could maybe learn something
43:30about techniques which we are using
43:32because they are very important not only
43:33for computer Graphics but also for other
43:35areas so we have our SP uh spatial
43:39cascaded cache which is a very good
43:41which has a very good spatial coverage
43:43and a very small memory footprint then
43:46we have our octal Direction coding which
43:48converts a threedimensional vector into
43:50a 2d space and back very efficiently and
43:52it can be scaled very easily then we
43:55have important sampling uh which gives
43:57you more suitable samples you just need
43:58to know by what you want to wait and the
44:01chish shf inequality which is an
44:03occlusion test that can be easily
44:05filtered then the moving average
44:07estimator is very great for stabilizing
44:09uh stabilizing your signal uh and
44:11keeping reactivity to light changes and
44:14last but not least the E5 RGB 9 format
44:17which is a simple and efficient way to
44:19store an RGB color into one un 32 and
44:23still keeping the Grail scale values so
44:26we are looking actually for a junior
44:28programmer so if you feel compelled and
44:30you like ined then please um just grab
44:35us after the talk and uh we can we can
44:39have a talk so if there are any
44:41questions then this is the
44:45[Applause]
44:53time so thanks jaob uh we are definitely
44:55on time we have 5 minutes for question
44:57questions and I be running around and
44:58getting them
45:08perfect okay I'll try not to take up too
45:11much of your time as I do have a ly of
45:13questions um so uh one of the things
45:15that you mentioned was that you guys
45:17have a fixed Ray budget and towards the
45:19start you also mentioned that you have a
45:21metric for assessing quote probe quality
45:23unquote for determining this Ray budget
45:25do you have any more details to share on
45:27that I'm sorry I didn't understand the
45:29question can you speak a little bit
45:30louder okay so you mentioned that you
45:32guys have a fixed Ray budget um and
45:35towards the start you mentioned that you
45:36have a system for determining probe
45:38quality to kind of allocate the array
45:40budget could you share a few more words
45:42about that what metric yeah sure so our
45:44aray budget start with 100,000 Rays for
45:46the low end then goes about
45:49250,000 for mid-range and actually half
45:51a million for uh high-end devices and
45:54the probe quality is very simple you
45:56start with zero and every frame you add
45:58a small portion this is um adjustable
46:01value which we just played around with
46:03to see um where is a good threshold
46:07basically so there is no no mathematical
46:10formula behind that okay thank you um on
46:13the topic of calling actually you
46:15mentioned you also have a hard 16k limit
46:17on uh World cash probes how do you
46:19determine which ones to call out and
46:21which ones to keep this is uh basically
46:24a push through system that means that
46:27probes that are actually too old uh will
46:29just get push out of that and new probes
46:32will um actually replace them so uh if
46:36we are running out of space we are
46:38actually increasing the rate by which we
46:40are uh making the probes
46:44um like uh where when when we are
46:47decreasing the probe quality so if for
46:49example a world cash probe wouldn't get
46:52um activated for a couple of frames then
46:55you would each frame decreas its quality
46:58and if we are running out of budget we
47:00are basically cutting it right
47:02away um one last more kind of open-ended
47:05questions you guys mentioned a bany of
47:08techniques it's a whole
47:09Kaleidoscope um and you guys also do a
47:12lot of spatial reuse a lot of temporal
47:14reuse did you guys consider using
47:16anything uh inspired by rester in your
47:18systems yes there is actually something
47:21we are currently discussing because this
47:23technique has its limitations and is
47:25also as you could see we have a 2 met
47:27spacing therefore there might be some
47:30light leaking around uh or not really
47:32leaking but the the specular
47:34contributions and light contributions
47:35will be a little bit bigger than you
47:37would expect and therefore we want to
47:39tackle that maybe with rester in the
47:41future okay thank
47:44you any more
47:46questions here
47:51oh hey uh great talk um I was curious um
47:56you're using U um uh SDF um cascaded SDF
48:00U but then when you actually get your
48:02rayit where are you getting your
48:04material properties from is that also
48:05encoded in a voxal grid somehow or is
48:08exactly it's encoded in a voxal world
48:10and we have some very optimized way to
48:13store that I was not involved in that so
48:15I cannot give you any details
48:16unfortunately but yes we are querying
48:18the voxel world and then out of that we
48:21get uh right now the voxul are saving
48:23albo uh the normal we can get from the
48:26surface
48:27uh then roughness specularity is also
48:29there and emiss of course okay and for
48:33the important sampling you say you're
48:36doing a a a linear search through in
48:38order to to find the um you like the the
48:41correct bin from your for you from your
48:44um your random sampling um have you
48:47considered you like basically making a
48:49like a a sued area table and then doing
48:51kind of binary search or something like
48:52that not yet good idea thank you okay
48:57you're
48:59welcome I guess on Tuesday then right
49:02right any more questions
49:08yeah uh hello I would like to ask how
49:12you manage uh your sparse geometry I
49:15noticed some trees in a trailer
49:17alongside this SDF do you make your voel
49:20semi-transparent if it hits the tree or
49:23something else very good question you
49:25are very sharp we actually do not have
49:27anything for trees yet so this is a big
49:30hole in our system so if you are
49:31standing beneath some trees your your uh
49:35whole armor will quite be be quite shiny
49:39because actually the tree above will not
49:41be represented this is something we are
49:43working on right now and we will give
49:46give you a little bit of information so
49:48for the trunk I think we will be just
49:49using voxels but for the leaves we'll be
49:52do doing some
49:54volumetric thing that's just decreases
49:57the radiance of the ray but that's all
50:00in work I don't have a solution a final
50:03solution yet for
50:05that all right one more
50:11question none okay and thanks for the
50:13speaker
50:14again thanks for listening

About this transcript

This page contains the full transcript of Realtime Global Illumination in Enshrouded by Graphics Programming Conference, generated from the public captions YouTube serves with the video. The transcript has 8,819 words across 1,236 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.