YT YouTube2Text

YouTube2Text → Transcripts → Rendering Tiny Glades With Entirely Too Much Ray Marching

Rendering Tiny Glades With Entirely Too Much Ray Marching — Transcript

by Graphics Programming Conference · 12,132 words · 1,665 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00[Applause]
0:01hello uh Welcome to The Strange Talk
0:03where we solve all of life's problems
0:05using uh only Ray marching and stupid
0:08rendering tricks um I'm Tom and half of
0:11a new new uh IND the game studio called
0:13poun light and we have recently released
0:14our first game called tiny Glade uh in
0:16case you haven't seen it uh here's our
0:18[Music]
0:21trailer so it's a little cozy dama
0:24doodling game where you just uh build
0:26things for your own enjoyment uh making
0:28uh Co cages romantic ruins uh and uh
0:32lovely
0:33castles there's no goals uh there's no
0:35stress uh you just uh s k back and uh
0:39enjoy yourself making something
0:43[Music]
0:47pretty and one of the premises of the
0:50game is that uh whatever you do uh the
0:52game will make beautiful it's rendering
0:55and protig generation
1:02[Music]
1:10[Music]
1:23[Applause]
1:29thank you but we're here the graphics so
1:30from the graphics nerd angle uh the game
1:33essentially does doodling so like Ms
1:35Spain style but with pres generation uh
1:37we have hundreds of thousands of
1:39individual objects with modable geometry
1:41and dynamic time of day uh but the uh
1:44audience is casual therefore the
1:46hardware is like often 10 years old so
1:48we have to run kind of potato devices uh
1:50and to pull this off we have crafted a
1:52custom engine on top of Vulcan rust and
1:54HL uh and it is mostly written from
1:57scratch but using tons and tons of Open
1:59Source software and some of our own uh
2:01secret sauce uh but nobody actually
2:03works on fulltime fulltime on rendering
2:05in our game uh besides me doing kind of
2:07long like night shifts and not knowing
2:09the separation between uh work and life
2:11uh so it's a it's going to be a lot of
2:13Jank warning and things kind of like if
2:15you think I'm saying something wrong
2:16it's probably it is wrong because we
2:17don't really have time to properly
2:18research things uh because the team is
2:20super tiny it's it's Anar and I and we
2:22have collaborated with um UD and Martin
2:24from KNN audio for the games
2:26audio so you might be wondering why did
2:29such a tiny Studio decided to make a
2:30custom engine and the story of that is
2:32kind of connected to Where it All Began
2:34and it started as a side project by Anna
2:37uh and it was a hobby project for about
2:38a year before it turned to into a game
2:41uh and only joined after that one year
2:43and within that time it was a u a hubby
2:46thing to learn rendering so Anna was
2:48quite vers in proced procedual
2:49generation offline but she wanted to uh
2:52learn rendering and actually tried to do
2:53procedure in real time as well using
2:54computer shaders and all the stuff that
2:56we all love uh and the whole thing
2:58started with just one wall generat that
3:00was placing bricks procedurally then the
3:02whole thing turned into multiple walls
3:03archers terrain and so on and after a
3:06few months it kind of got this moniker a
3:08tiny building game and a lot of people
3:11got uh enroled by that uh uh and the
3:14things they fall fell in love with was
3:16the kind of very gradual or granular
3:18procedure generation and and the organic
3:20nature of it so that time some of the
3:22elements of that were already set in
3:24stone that uh we had this like tons of
3:26individual bricks and so on and every
3:28tble and plank were unique
3:30and uh that was fine initially because
3:33this is the the first announce trailer
3:34we had for the game uh as you can see it
3:36was not much complexity just a few
3:37buildings uh and maybe a thousand bricks
3:40and but it had to scale it will scale a
3:42lot because uh the more we streamline ux
3:45and the more we optimize things U people
3:47just kept building crazier and crazier
3:49things uh some people just don't accept
3:51the tiny part of tiny Glade uh and some
3:54of them are just fine working for like
3:5510 FPS all the time so builds get crazy
3:58uh this is just a few samples of what
4:00the community has created and this is
4:02still on the same side uh just like lots
4:04of beautiful
4:06things different kinds of Aesthetics
4:08some people just decide to enti build
4:10entire towns or even fragments and
4:11cities in the
4:13game just uh yeah comparing to the
4:16initial shots this is this is completely
4:22Bonkers uh oh yeah people some people
4:24just recreate parts of Lord Rings
4:25because why not and running into the the
4:28limits of all the instances we have in
4:29the
4:30game all of those are individual Bricks
4:32by the way just me to render separately
4:34and so people just like like okay cool
4:36it's the dueling app going to make
4:38something
4:39funky and like just making things we
4:41never anticipated and and and by the way
4:43we don't have don't we don't have
4:44indoors in the games all Outdoors people
4:46just have figured out how to still make
4:48indoors work somehow so it's ugc game
4:51for you so how to deal with it uh well
4:54we have tons of geometry to somehow make
4:56it appear uh so we employ driven Runing
5:00for that uh so the reason is that just
5:03like at the start our generators don't
5:05synthesize meeses we don't deform
5:06geometry any fcy skinning which generate
5:08tons and tons of individual Briggs and
5:10planks like hundreds of thousands of
5:11them uh and similar for uh tiles and
5:15Pebbles and P up plants and window
5:17shutters and so on everything just comes
5:18out those little bits uh and well gpus
5:20are way better dealing with with the
5:22crazy amounts of stuff therefore we do
5:24it on the GPU but before you create a
5:26cool runer system you have to name it
5:28right so everyone does that days uh but
5:31n night was already taken so ours is
5:32called
5:34N9 and what n does is performs GPU Dr
5:38calling sorting LOD and Dr generation uh
5:41we don't use clusters of any of the new
5:42fancy stuff because our meshes are way
5:44too simple for that like our plank or
5:46brick would just like one cluster
5:47probably and we don't use M shaders
5:49because our Min spec doesn't support
5:50them um but we do have one pretty simple
5:52scheme that we just uh like put all the
5:55geometry data for that for single
5:57material into the same vertx buffer and
5:58we can render any mesh from that using
6:00just Sub sub slicing from that index
6:02buffer uh and for example changing to
6:05different discrete LOD just becomes a
6:07plus one to the um index in the mesh
6:09array or in the
6:12atlas and in terms of calling uh It Is
6:15by now pretty standard stuff that was
6:16already covered by at the conference
6:17today few days ago uh which is the two
6:21plus two pass occlusion calling uh
6:24introduced I think by Seb Alon and kind
6:26of popularized by nanite uh we render a
6:29first pass where we draw everything that
6:30was visible in in the previous frame uh
6:32that we can know du to stable mesh IDs
6:34between frames uh then we build a depth
6:36pyramid and then we detect what we had
6:38missed render those objects and then
6:41rins and
6:42repeat so that is pretty standard uh and
6:45then in the simplest path once we have
6:46determined what is visible uh the uh we
6:50can just potentially just add every
6:51object to a separate draw list U and uh
6:55the simple path essentially has one
6:57drist per uh material and mesh type uh
7:01we just add increment integer we uh draw
7:04to write to an IND direction map uh and
7:07then we can draw everything using sing
7:08Single draw index indirect and because
7:10uh uh all the meshes for a single
7:13material are packed into the same mesh
7:16uh or like verx buffer we can draw
7:18essentially all of the uh meshes of a
7:20single material using a single draw
7:21index indirect so the draw count number
7:24of draws is the number of meshes we have
7:26so we can use the correct uh sub spice
7:28of the index buffer and then the number
7:30of instances in each of the Gen draws is
7:32just the number of visible instances so
7:34pretty simple stuff here and that works
7:36well for bricks because for them we need
7:38something with very low overhead since
7:40we have up to a million of them and
7:41we're going to be extending to up to two
7:43millions soon uh but it's still flexible
7:46enough that you can actually do some
7:47pretty cool stuff like for the last LOD
7:49for bricks uh we actually have a model
7:51which uses just six triangles uh which
7:52is like half a cube that we flip the
7:55vertices to actually make them always
7:56front front facing and that's uh the Pix
7:59sh essentially becomes just a dispatch
8:01mechanism for Ray marching or in this
8:03case analytical intersection with a
8:05campert box so we can synthesize little
8:07bevels around the edges and make sure
8:09that even far away we can catch lights
8:10on those kind of fin lines and get a
8:13similar look to what we have with the
8:14full mashing
8:17top but not everything is so simple uh
8:19for example our trees they use a scheme
8:22of expanding Billboards from geometry uh
8:24and that scheme introduces a bunch of
8:27overdraw and on top of that we also have
8:29dep writing in order to break up the the
8:32detail actually make it appear like we
8:34have individual leaves uh which means
8:36that the whole pass becomes more overdo
8:37heavy and we need to do something more
8:39and uh if you have something like this
8:41you really want to sort things front to
8:42back but even with a simple scheme uh I
8:45mean with with our system it's pretty
8:46simple to do that so uh we use
8:48approximate sorting but just binning
8:50things into a number of buckets we use
8:52at24 and essentially do a single pass
8:55radic um over the the visible meshes or
8:58a counter Counting SW if you will uh so
9:00we uh insert uh the meshes or the
9:03visible instances in the correct place
9:05place based on the the uh on the depth
9:08slice uh then we end up with a single
9:10mesh single list which is approximately
9:12sorted uh and once we have that list we
9:14can generate DRS from from it but
9:16because it is uh sorted roughly from
9:19front to back it means that something
9:21have a mhes meshes which are different
9:22so you have to switch the which Mass
9:24you're drawing in between uh and to uh
9:26to strike a balance between sort and
9:30number of draws we what we generate from
9:31it uh we process the uh the instances in
9:34in groups and in each group we generate
9:37a many histogram of which M meshes are
9:39actually visible which means that if you
9:41happen to have uh the same mesh and just
9:44say 32 in of of them uh you you should
9:46just one draw from that but if you have
9:48mixure you you do as many uh draws as
9:50you have matches within the group uh and
9:53you can kind of fine tune just how much
9:54sorting you get and how much how many
9:56draws you generate uh by just changing
9:58the group size in that process and then
10:00uh it also means that uh just like with
10:02the previous scheme you can we can draw
10:03essentially everything using a single uh
10:06in this case draw index indir count
10:08which means on the CPU side it is still
10:10very
10:11cheap and in terms of the the wind this
10:14gets us uh for the sort approach on the
10:16for the trees on the chunky GPU is
10:17roughly 65% of the Rend time compared to
10:20the unsorted and in general it seems to
10:22be a bit faster on on AMD at least but
10:26we're still kind of looking into it and
10:27figuring out what works best because we
10:28discovered that that's uh like pre-
10:30touring Nvidia cards actually have a
10:32pretty bad tax on the indirect count
10:34because you have to provide the maximum
10:36number you can provide and it seems to
10:38just Spam the command processor of the
10:40GPU uh and you end up paying a pretty
10:41hefty tax and you look at the GPU
10:43profiler and just like sitting there and
10:45sleeping so still figuring out with the
10:47best approach
10:48here and in terms of deferring stuff uh
10:51this is not as neat as something like a
10:53V buffer approach like n nuse uh and
10:56probably wouldn't scale thousands of
10:57materials but we don't have thousands of
10:58materials we have a just a handful of
11:01hand handcrafted materials uh and it
11:03ends up being good enough for us
11:05partially because uh we can kind of
11:07roughly know at the material stage in
11:09which order things should be drown like
11:10we know that grass will usually not
11:12include buildings and trees will then to
11:14include buildings and so on so we can
11:15kind of just do a classic pre- sorting
11:17Bas of materials and it just happens
11:19that our G buffer is very thin so we can
11:22suffer some
11:25overdraw and speaking of Jers and light
11:27we they usually plug into lighting so
11:29it's like about lighting or starting
11:31with the lack thereof which is Shadows
11:33uh the most important light source we
11:34have is the sun uh and we knew from the
11:36start that it will be soft so we look at
11:39the most uh U attractive entry into it
11:42at the start which is nor Shadow mapping
11:44and after some initial tests uh we have
11:47determined that the game seems to be a
11:49worst case generator for moment Shadow
11:51Maps if you use vssm you might have seen
11:53some like that we just get this kind of
11:54leaking whenever there's multiple depth
11:55layering and MSM is supposed to fix it
11:57but uh we have found that uh
11:59because of the free form nature of the
12:01game and the ugc of it people will just
12:03create scenes that will that create
12:05those kind of artifacts so we can really
12:07go with with MSM as as nice as it would
12:09would have been uh so we have to go with
12:11PCF that we all love and I guess know
12:14and hate I guess uh it's it does work in
12:17general but requires lots of bias
12:19tweaking but we can mostly get away with
12:21the the normal off the bias and just
12:22lots of hand tuning of the whole thing
12:25uh and then uh we just throw out pretty
12:26standard pcss uh content sharpening on
12:29of
12:31it however if you implement Shadows uh
12:34with even syn softness and you do your
12:36all of your biasing well you still end
12:37up with something like this and in our
12:40game we have continuous time of day and
12:41you can also change the time of day as
12:43you like and you end up this kind of hor
12:45aliasing so unless you have enough
12:47Shadow M up resolution and you rely do
12:50you will end up with this artifact
12:52however if you uh if we see something
12:54like this in the primary visibility we
12:56typically throw it at anti-aliasing
12:58right so why don't we do it for Shadows
13:00uh and ta specifically is really good at
13:02dealing with kind of temporal aing so uh
13:06typically in in ta you would take the
13:08new input you have just rendered uh you
13:10combine it with with history that you
13:12Rectify modify or reject in some way and
13:16you do that by by estimating some some
13:18kind of Statistics from a local
13:19neighborhood of the new input uh and
13:21then clamp to those statistics for
13:23example but if you do it in screen space
13:25it will not do anything because the the
13:26the features that are flickering are uh
13:29larger than that tiny 3x3 kernel you
13:30would throw at it so what you do instead
13:32is you derive those statistics from the
13:34the actual space in which the the
13:36content changes and in this case it is
13:38the shadow kernel in the shadow m space
13:41uh that is the exact resolution or the
13:43frequency of which you need to to offset
13:45and find your neighboring
13:46samples and so nice thing about it is
13:49that you're already likely fetching all
13:50of this data for your PCF kernel so you
13:52actually don't do anything special uh
13:54typically you when calculating self
13:56Shadows you are essentially doing a mean
13:58estimator which is the first statistical
14:00moment if you also track the second
14:01statistical moment you can calculate
14:02variance from that and then borrowing
14:04from
14:05Marv's variant based TAA you can
14:07calculate a bounding box for clipping
14:10the history and thus you can essentially
14:12apply TAA to history as well to the
14:16Shadows uh you end up with a slight
14:19artifact in that's in areas of high
14:22variance you can have ghosting but
14:23that's also pretty easily fixable
14:25because the largest areas of variant
14:26will be in VAR s shadows and those are
14:28less on to aliasing so you can just
14:30shring that box and you can control the
14:32size of the bounding box based on on the
14:34shadow softness and if you combine all
14:36of that together you can end up with
14:38shadows something like this from the
14:39same resolution of uh of Shadow M
14:43data so you can still see a little bit
14:45of flickering but it's uh generally much
14:47more stable even some like subance some
14:49of the motion of like the flag waving
14:51and own
14:54own but this is still not sufficient
14:56resolution and unless you have the
14:57budget you cannot really uh crank up
15:00your uh shut up resolution to the point
15:02where you get all the fine
15:04details and that is something that's uh
15:06uh has been fixed in post in recent
15:08years uh days back I think all the way
15:10to crisis 2 uh but the idea General is
15:12that you rearch towards the um this the
15:15light source and then uh in the screen
15:18space you can detect whether you
15:18actually intersect something and then uh
15:20claim out the be Shadow as
15:23well and uh there is uh yeah in our case
15:27we just do a simple Ray March uh using
15:29um just like a Shader we use essentially
15:31for all all of array marching but there
15:33also a solution uh that has recently
15:36been quite popular from Ben Studio we
15:38tried it as well I have a comparison but
15:40we also found that our approach was
15:42slightly better fit and just for
15:45comparison this is uh what we get with
15:47no contact shadows in a pretty extreme
15:49uh case where the sound is just like
15:50grazing building that typically you just
15:52get look just blurryness and nothing
15:54cool and if you just add three tabs of
15:57Ray marching uh suddenly all the pops
15:59and looks like oh you've actually just
16:00added geometry so that was kind of fun
16:02to see like we actually had to Frink the
16:04bricks a little bit because they're
16:05sticking out too much because we're over
16:06compensating previously and then
16:09comparing that to the B Studio Version
16:11uh you actually get slightly better
16:12detail resolution with their approach
16:15but we found that we can get the same if
16:16we just increase the number of of tabs
16:18in R marcher just a little bit uh and
16:21having the control over how many uh tabs
16:23we have and having also ability to to
16:25potentially move them shoot them at an
16:27angle to get soft Shadows was a win for
16:30us also nice thing is that compared to
16:33the Ben cud version ours is fully stable
16:37as well if you use them in and out
16:38because it doesn't have that dispatch
16:41restrictions but other pressure works
16:43for you then that's
16:44cool uh but if you try impant R marching
16:47you will quickly run into this kind of
16:48issue which is uh just uncontrollable
16:51acne
16:52uh that just doesn't look good today if
16:56you don't apply any bias you're just
16:58going to have a bad time if you apply
17:00some bias you'll probably still have it
17:01if you app a lot of bias you remove all
17:03of the detail and still end up with uh
17:06kind of we started with and then if you
17:08look at the same thing from different
17:09angle it turns out you still haven't
17:11remove bias and you end up fighting the
17:13same War as as you had with PCF self
17:16Shadows but um what if I told you
17:18there's a thing you can do to uh remove
17:20pretty much all the acne and uh and keep
17:23all of the detail so this is comparing
17:25to the nobias version and even behaves
17:28itself smoothly in temporal
17:35cases so it's going to be weird uh
17:37because um so you know your parents told
17:40you not to sample depth maps using
17:42linear Samplers well listen to them but
17:45also don't listen to them um let me
17:47explain so if you Point sample Shad them
17:50UPS you end up with this kind of like
17:51Duplo legol of the micro scale where the
17:54stair steps between samples create this
17:57kind of sharp edges and you you end up
17:59hitting them accidentally and this is
18:00what the acne comes from and biasing
18:02essentially just offsets the ray from
18:04the geometry which also removes
18:05interesting detail but if you use a
18:07linear sampler you can reconstruct the
18:08underlying
18:10geometry but so why don't we just use
18:12linear Samplers for our depth uh well
18:14because not everything is continuous and
18:16the the linear sampler would pretend
18:18that everything is going to shrink
18:19wrapped and everything is a continuous
18:20surface and you're going to have false
18:21kits which create own artifacts but uh
18:24what if we um use
18:27both I know it's weird but uh if you
18:30patch both the linear and and and Point
18:33sample data and then for the for the
18:35intersection test use the Fus of the dev
18:37values and for the thickness check you
18:39use a closest to the two values it just
18:41happens to
18:42work can really I can only hand waving
18:45explain it because I just I was messing
18:47with things and managed to stumble upon
18:48it and it just happen to to work too
18:50well I was kind of misbelieving so maybe
18:52there's a reason Works maybe it doesn't
18:54uh I don't have time to dig into it but
18:56the kind of handwave explanation is that
18:58the uh false HS the quantization or the
19:00third stepping are gone because you're
19:01essentially cutting up the corners by
19:02thinking the fusion of those two and
19:05then the discontin are gone as well
19:07because you're reintroducing the
19:08vertical lines as well in in the
19:10function you're intersecting and those
19:11vertical lines allow the ray marcher to
19:14detect misses uh which is further helped
19:16by using the uh the larger of the
19:18estimates for the U for the fix fix
19:21calculations but honestly I don't really
19:23get it so if you want to grab it we are
19:25sharing the source code on on gist so
19:26you can try it yourself we use it for a
19:28very Mar in the game for all sorts of
19:30effects uh one of the things we use it
19:32for is localizes as well uh we haven't
19:35implemented anything fence here just
19:36essentially for the same R marcher at
19:37them uh and so because it is quite
19:40flexible you can uh just do a classic
19:43thing of for Loop goes B and uh
19:46intersect all the local lights rain
19:48March towards them and we have uh
19:50Shadows from local lights uh and it's
19:52something so we're still investigating
19:54because of the ugc nature of the game we
19:55end up having thousands of lights or
19:57hundreds of thousands of Lights
19:58potentially so I'll need to look into
19:59restore di for for now it's kind of
20:01simplistic but uh but gives you idea
20:03that you can get local stuff as well uh
20:06the other most important light source
20:08that we have is the sky and we didn't go
20:10for any fancy atmospheric scattering
20:12model because we wanted to have all the
20:13control uh our game is not meant to be
20:15photo realistic so we just threw a bunch
20:17of lurs together uh handcrafted some
20:19timelines and for each of the uh levels
20:21that we have uh we made Skies that just
20:24look nice from artistic point of view
20:26and then our SP our clouds are just a
20:28bunch of Sprites scattered on a cylinder
20:30uh then we have fre Bas and lighting
20:31using a bunch of spheres together and
20:33then painted over a little bit uh does
20:35on in Houdini and then we blend on with
20:37the different textures depending on the
20:39some of the angles of the light
20:40depending how how it hits the the Sprite
20:42so I have a few versions for came from
20:44few different angles it's like a
20:45standard trick used in lighting of of
20:47particles in games uh but the cool thing
20:50about the sky is that it becomes a light
20:52source for our Global elimination so
20:54let's talk about that H sorry let's talk
20:56about global elimination
20:59uh when we started the tiny Glade was uh
21:01going to be quite tiny and uh flat uh so
21:04we didn't have any of the fany shaped
21:06layering we didn't really have buildings
21:08we just had some walls and the first
21:10implementation of of a GI was using
21:12technique from like 20 year one years
21:14ago now from Moto GP uh where you
21:17essentially render a top down map of
21:18lighting and then you uh sample that for
21:20every pixel and you have approximation
21:21to lighting uh we had a slighty fencer
21:23twist in that which was uh shooting Rays
21:26from a cone and kind of approximating
21:28where it would intersect that that the
21:30hype map and it was giving us something
21:32it was not completely terrible was a
21:34little bit of fill light I guess but um
21:38uh when we actually had a bit more
21:39layering of shapes uh it became clear I
21:42wasn't going to cut it it was kind of
21:43just making those cold scenes that
21:45didn't really make sense and it will be
21:46a while until we figure out exactly what
21:48was missing from it and so to do it well
21:51in general like in global Elation
21:53especially uh you have to start with
21:55reference mode uh because even if you
21:58cannot eventually match the reference
22:00mode it's very important to have it on
22:02hand and know whether when you're
22:03tweaking something whether you're
22:04actually making it better or making it
22:05worse like should we have more AO should
22:07we have less AO should we tweak this
22:09filter that way or this other way uh and
22:11sometimes very very hard to tell whether
22:13it's all fine or whether you're
22:15completely off with it just like you saw
22:16with the picture earlier so ideally you
22:19would just uh add a PA Tracer straight
22:21to the game uh and ideally then verified
22:24against something that well known and
22:25good like mitsuba and then Carry On from
22:27there
22:29uh and sometimes screen space is enough
22:31potentially on the game but sometimes it
22:33is not uh in our case it was not going
22:35to be sufficient because of the layering
22:37of shapes and some people doing indoors
22:39uh so we considered a few different
22:40options um sdfs or voxels might work but
22:43uh at the time we were actually already
22:45generating proxy geometry for collisions
22:48so we figured well why do just R trce
22:49those uh so we built uh some bves and we
22:52sh race uh against them uh there was a
22:55cool project by Yan F Burgen uh called
22:58cuda path Tracer which implements a wide
23:00bvh Tracer uh so I borrowed code from
23:04that one and to build the bvh I use
23:06Embry which was very simple uh we
23:08eventually switched to a rust crate that
23:11I uh C collaborate with a little bit
23:13with Griffin onto OB VHS great well all
23:16the building of the Fus Griffins I just
23:19helped a little bit but essentially we
23:20now have a rust native solution building
23:22building the BBH was which is pretty
23:24cool oh yeah and and and and and we of
23:27course we went for uh software R tracing
23:30because uh our Hardware is potato so we
23:32cannot support Hardware tracing and we
23:33want to have the same look on all
23:37platforms uh but uh this is the proy we
23:39actually have they don't even have roofs
23:41but it's kind of surprising that uh even
23:43with this you can get a pretty good-
23:45looking uh GI if you use it well enough
23:48if you combine that information with
23:49screen
23:50space but so now we have some tracing
23:53you can obviously just ship a b Tracer
23:54using that because it's going to be way
23:55too slow uh so we can try some
23:57approaches initially we tried DGI
24:00because that was kind of popular at the
24:02time and I was curious to try it uh
24:04where you have a regular grid of probes
24:06scattered around and then R Trace from
24:08them and try to apply that as kind of
24:10like a mini mini light source of the
24:11scene uh initially sounded like a pretty
24:13good idea it meant that so we could
24:15lazily update all those probes where
24:17geometry changed and they were stable in
24:19word space so you wouldn't have this
24:21typical repr projective trailing behind
24:23moving objects but it turned out that we
24:25just never got enough prob density and
24:27density out of it because GGI has a lot
24:29of uh uh angular distribution uh you
24:32cannot really have much spatial density
24:34between those probes and we also ran
24:36into essentially aliasing issues where
24:38if you had a small shape you just
24:40sometimes didn't end up with a probe
24:41inside it sometimes you did and then uh
24:43visibility between them was also
24:44probiotic where you have to really
24:46constrain that like I don't I don't want
24:47the light from the inside to to leak the
24:49outside and it turns out they were still
24:51ghosting anyway because uh if you had a
24:53a building and then you resize it
24:55shrinking for example turns out the part
24:57which was inside is now outside and it's
24:59darkening your outside for some reason
25:01because it's just lagging with the
25:02update and because of our time of day
25:04everything was updating anyway so the
25:05whole lazy updating went kind of out the
25:08window anyway so we look into another
25:11thing which was popular at the time
25:12which was moving the probes to screen
25:13space kind of like Lumen or GI 1.0 uh
25:17and the cool thing is that uh you end up
25:19with probes which are well always sort
25:20of well distributed in in in screen
25:22space and they allow really cheap
25:24spatial filters uh because of the low
25:26resolution appr in Spring space
25:28uh and you end up with very little noise
25:30in the output uh you can also use them
25:32for rough specular uh on the other hand
25:35they were actually very finicky too this
25:36approach was veryy to actually get to
25:38look good so the initial results were
25:40quite promising but to get it to
25:42shippable state would have been a bit
25:43work so this was the initial protype we
25:46did actually audio didn't expect that or
25:50something else never
25:51mind oh
25:53yeah so you can see it's kind of
25:56working and it's uh suff responsive
25:59even but if I spin around you can see
26:02that it's kind of flattish everywhere
26:04and that's uh again connected to the low
26:06spatial resolution just like we have the
26:08the GI uh we couldn't have quite as much
26:11resolution as as Lumen or GI 1 1.0 has
26:14uh so we had to rely on bit more
26:16aggressive filtering which meant that we
26:18would somehow need to synthesize
26:19addition detail on top of
26:21it and we still had a problem of probs
26:24getting stuck in strange places so if
26:25you look at the tower on the left side
26:26there's like this kind of strange dark
26:28fles and that happens when you actually
26:29spawn a probe in like a strange crease
26:31in between objects so you have to do
26:33some kind of adaptive placement or
26:35trying to figure out all this probe is
26:36in bad place Let's uh try to to get get
26:38it out of there uh there's also the
26:40issue of projection error into his H and
26:42the the estimate being uh blurry and
26:45potentially more blurry than you you
26:46would like to be and it might look like
26:48they're both fine uh but if you actually
26:50flip between them uh this is a reference
26:52and then this is the prob base solution
26:55it kind of looks odd like old lighting
26:57slightly different
26:58it should be and it kind of gets this
27:00strange metallic Sheen so I think if you
27:02spend enough time on this kind of
27:03solution it's going to be great but we
27:05didn't have time to uh to polish too
27:07much out of it and I had experience with
27:09different BR before called rester uh so
27:11I threw away the probes and tried rester
27:14instead in this case you instead of
27:16feeding probes you feed individual
27:17reservoirs in the screen and you get all
27:19the spatial resolution that you might
27:20potentially want and then it's just a
27:21matter of filtering it well and the REO
27:24algorithm gives you really good like
27:27very introduction and filtering via a
27:29complex chain of temporal and spatial
27:30filters and can be very reactive as well
27:32as you can see in the in the video here
27:35however it turned out to be complete
27:35overkill for us because it turns out
27:38that our mostly outdoor scenes are
27:40fairly low in variance and we actually
27:41when implementing that uh we could
27:43actually get away with just using the
27:44Temple part of rester uh and we didn't
27:47really need this fancy sample validation
27:48either because the lights in our game
27:50don't really flicker so they don't to
27:52reactive uh and turns out if you use rer
27:55without the the s for the spatial part
27:57is actually less sample efficient than
27:59using Brute Force because you end up
28:01generating some some samples then if
28:02they don't win the uh the reservoir
28:04battle they end up not contributing at
28:06all and therefore we went to just full
28:09potato GI mode where we just R trace a
28:12bunch of RS in the hemisphere and throw
28:14it at the spatial spatial temporal
28:15filter to clean up and it's a very round
28:17out by the way to arrive into something
28:19that's actually quite simple uh but it
28:20just wasn't very obvious from the start
28:22that the simple thing would work uh and
28:24so it it is there is some devil in the
28:27details in how the spatial temp actually
28:29has to work in order to extract all the
28:31all the detail and stability out of
28:33various par traces so in terms of what
28:35you actually do not just the journey to
28:38it uh We R Trace at uh one quarter the
28:40resolution so we have one Ray per 16
28:42pixels in the screen uh we start by Ray
28:44marching and only when that Ray March
28:46fails when it goes behind geometry or
28:48off screen we switch to Ray tracing and
28:50even at the end of that if it hits
28:52something on screen uh we use the
28:54radiance on the screen so we can kind of
28:55hide the the fact that we have very
28:57potato proxy
28:59geometry uh then once we have calculated
29:02that quas uh R data we project it to
29:06spal harmonics uh the the S2 s four
29:09component per Channel version of it uh
29:12and then uh we do that in a spatial
29:14filter which uses eight of the quot
29:16resolution Trace um outputs uh in a
29:19which already prefilter simple data and
29:20kind of reconstructed at the higher
29:22resolution ready then we do in spal
29:25harmonic space a little bit like Metro
29:27does uh and then uh we evaluated spical
29:30harmonics it's in screen spice by once
29:32again using a small spatial filter in a
29:34cross bilateral uh to sample the uh s s
29:37data and for that we use the
29:39hallucinated Zone harmonics uh to to
29:42actually calculate the the final
29:43Radiance given spe
29:47normals and the dooing uh is actually
29:51quite simple it's based on the fast
29:53denoising with st stabilizing recurrent
29:55blurs from demitry Jan done I'm not sure
29:58how to pronounce but uh uh essentially
30:01it's it's a filter which kind of feeds
30:04into itself and fine tunes itself to to
30:07use as little of a spatial filter as
30:09possible which is kind
30:11of uh similar to I for name uh the the
30:16whole pipeline is made of three passes
30:17in the first pass we uh generate new uh
30:20new R data new samples uh new radians
30:24sorry uh then in the second pass we uh
30:27combined New generat Radiance with uh
30:29previous generat Radiance in a
30:31reprojection pass uh if the reprojection
30:33fails we do a gap fill uh if the Gap
30:36full fails we actually embrace race
30:38conditions and sample from the same
30:39texture that we otherwise be outputting
30:41uh and doesn't really matter in this
30:42case and so after that we just do a
30:45small spatial filter on top of it uh
30:47which with actually very small kernel
30:49because then when you recurse the whole
30:51thing into itself uh that kernel grows
30:53in a kind of conical shape and still
30:54gives you very very good ding power and
30:56the uh the St style stab stabilizing
30:59part of the thing means that you
31:01actually count how many samples you have
31:02accumulated and then if you the sample
31:04count is low use a large spatial filter
31:06and if it increases you you decreas and
31:08therefore tightening that the GI uh and
31:11as an additional twist we also use SSO
31:13as a cross bilateral filter uh crossb
31:17filter weight in the D user and to
31:20illustrate that uh we start witho gener
31:22by Intel's a some x gtao uh
31:25kernel and this is what you get if you
31:27just Supply vanilla noising and if you
31:30use SSO as a as a kind of filter guide
31:33you can sharpen the the noising in in
31:35the corners where you kind of know that
31:37you don't want to to blend uh samples
31:39because they they would be rapidly
31:41changing and then we apply additional
31:44the SS on top as well and then we
31:46calculate the final
31:48image and in terms of why why do we
31:51combine this additionally with SSO uh
31:53well we found with with like comparing
31:55to reference kind of how I mentioned
31:56before uh you want to have a a reference
31:58mode that you can see whether you're
32:00what you're tweaking is is moving in the
32:01right direction and it turns out that's
32:03we are just missing a little bit of that
32:04additional detail but it's like SSO
32:06times like 0.2 or something so it's very
32:08tiny uh and in this case our reference
32:10mode is actually not really ground trof
32:12because it also just uses Ray marging in
32:14screen space we don't have like full RTX
32:16mode in the game uh but it is still uh
32:19better than nothing and quite sufficient
32:20for what we need because what we're
32:21after is creating the same kind of
32:23general impression in the lighting and
32:24not matching it bit by bit and I think a
32:26lot of the uh error is actually due to
32:28the S projection rather than other
32:30sources it's good enough for us at least
32:33uh but now that we have R tracing it'll
32:35be a shame to only use it for GI so
32:37let's do it for other stuff um but um
32:39sparingly uh one of the things we do is
32:42everyone's favorites which is uh
32:43Reflections Reflections on water uh and
32:46as a water uh the basic setup is that we
32:49uh run it after the F lighting so the F
32:51rendering the for lighting and then we
32:52apply water in a sort of forward lit way
32:55uh currently our water is fully Flatland
32:58but we have plans to Extended so we
32:59didn't go with something like plan
33:00Reflections because we don't have the
33:02flexibility therefore R marching and R
33:04tracing uh we have some waves which are
33:06normal only that are based on some uh
33:08pre-cal pre-calculated textures and
33:11ripples for our floaty creatures uh and
33:14then for water we have the major
33:15components which is reflection
33:17refraction and and Co stics and starting
33:20reflection uh we start with SSR U and as
33:24this is a r resolution image so we can
33:25see the artifacts uh but as everyone
33:27also SSR kind of fails quite often and
33:30gives you something ugly uh so we detect
33:32the ugly and in those cases we do a rate
33:35tracing
33:37instead and to do it fairly efficiently
33:40uh we start with a fairly cheap uh Ray
33:43march with just eight steps of linear
33:45marching com combined with three
33:47bisection steps and one second operation
33:50uh uh and then we can in theory just in
33:53that case if we have a Miss We R Trace
33:55in we could R Trace in the uh in the
33:56same Shader but that would throw all of
33:59the occupancy of the GPU out the window
34:00because you can end up with a random
34:02collection race which missed and which
34:04succeeded uh and we also want to R Trace
34:06at half resolution as opposed to the ray
34:08March which is full resolution so we
34:10write out the the misses separate buffer
34:12uh we essentially compact them and do a
34:15separate Dispatch over just those misses
34:16to to R Trace at half us and similar to
34:20GI was Sample the screen space if the
34:22hit from R tracing happens to be on the
34:24screen there's a little B bit of caveat
34:26with that is is that due to the random
34:29nature of Ray marching uh and the uh and
34:32the waves you end up with this kind of
34:33like random peppers and salt noise where
34:36uh the SSR has missed uh and it's kind
34:39of annoying because you you can kind of
34:40just guess that will be similar and
34:42that's we apply our little hack and we
34:46hallucinate additional hits by just uh
34:48looking at Neighbors in quads and if any
34:51at least at least one array in a quote
34:53has hit something we just assume I'm
34:55guessing like you're going to hit at the
34:56same distance so so we kind of
34:58synthesize the hit point uh using those
35:00which also gives a nice uh benefit in
35:02that whenever misses happen they
35:04actually happen for the entire quad
35:05which means that we can rate Trace at uh
35:07Health resolution which is what we
35:08wanted from the
35:10start and speaking of the rate tracing
35:12fallback uh as I shown before the rate
35:14tracing geometry is kind of potato and
35:17uh doesn't look very good uh and we
35:19found that actually if you uh use the
35:22the full lighting for it in that R
35:23tracing fallbacks it can look a little
35:24bit jarring especially that the uh
35:26Lighting on that for subsequent boun
35:28bounces uh can look look a bit off
35:31because we don't have the same kind of
35:32uh uh feedback loop that we have for uh
35:35for SSR which uh or screen space Gi
35:37which um can kind of feed multiple
35:39bounces just from the screen space uh so
35:41the r tracing path has to kind of make
35:43up something and some looks bit too
35:45bright a little bit too dark and we
35:46actually found it's better to just fall
35:47back to Blackness because uh what you're
35:50very often after is just blocking the
35:52reflection as opposed to actually
35:53generating like proper uh results out of
35:55it maybe for a game was uh a theme park
35:58it will be a different story but in our
35:59case uh stuff can be dark and looks fine
36:01and then the waves actually break up the
36:03geometry sufficiently that you really
36:04don't notice that everything is just
36:05looks dup
36:08land and then in terms of refraction uh
36:10that's another area where we hack and
36:12and cheat we actually tried like doing
36:14proper real refraction by bending the
36:16Rays but we actually just didn't like it
36:18because the ray for shortening meant
36:20that the pawns and and little streams
36:22looked a bit too shallow uh and it also
36:25was more expensive and meant that
36:28sometimes you want to sample information
36:29which was not on the screen because if
36:30the ray bends a lot you may end up
36:32sampling in places where you don't have
36:33geometry so we uh cheat and we just
36:36distort the ray slightly by the waves
36:39and and ripples and uh just Remar the
36:41distorted Ray which also means that we
36:43can get away with like a very simple uh
36:45rain march with just three St tabs and
36:47two B section uh and we sample from a a
36:50copy of the dep Target that contains
36:52just the underwater stuff that we uh
36:54captured during the uh early rendering
36:58and we also have some cosics which are
36:59complete fakery as well uh by just doing
37:02some random thresholds on top of the uh
37:04uh wave textures uh and then we project
37:06those uh this strange thresholder thing
37:08onto the water using the the sun angle
37:11and just gives us something that looks
37:12good
37:14enough uh speaking of water let's talk
37:17about solid part and that is our Ice uh
37:21ice is a very similar setup to to the
37:23liquid water also runs after all the
37:26Deferred rendering uh but now it's a bit
37:28chunkier a bit more involved because the
37:30reflection and refraction need to be
37:31rough now uh so the U uh the surface
37:34path additionally also need to calculate
37:37uh diffuse lighting and diffuse shadows
37:40and we have seen before Shadows can
37:41create this kind of Blocky uh temporal
37:44aling uh so this pass also internally
37:46calculates the shadow TAA which means
37:48that like one of the outputs has to
37:49actually be just the shadow map itself
37:52uh and then in addition to that also
37:54generates a bunch of data for uh the
37:55deferral of uh reflection and for the
37:58deferral of filtering of of refraction
38:00and we try to defer all the things so we
38:02can actually once again reuse all the
38:03samples and synthesize detail from from
38:06very few uh rays and then we follow the
38:08whole thing of spal tempal noising the
38:10reflections and refractions starting
38:12with reflection uh so we uh rearch again
38:16in the theer Pass Health resolution
38:18against heal resolution dep depth uh we
38:21using spherical cap ggx sampling with a
38:23heavy bias of like 70% I think so we
38:25kind of cut off the edges of the the G
38:28GDX cone uh we use a fairly uh a bit
38:32more detailed frame March this time
38:33because we found out like we just want a
38:35little bit of extra definition by like a
38:37little read sticking out of the Ice uh
38:39then once again we compact the misses we
38:41dispatch a r Trace over just the misses
38:43uh but then instead of just directly
38:45outputting gradients we do this uh uh
38:47dance kind of similar to the stochastic
38:49SSR technique I presented previously
38:51where we generate the uh we gather all
38:53the hit data and then we uh kind of
38:55synthesize how the the hit KS look like
38:58if all the Rays uh if all the samples CH
39:00multiple Rays um and we tried using uh
39:03the the fancy ratio estimator but find
39:06out in in this specific case a simple
39:08cross bilateral over roughness and hit
39:10distance was
39:11sufficient and uh one thing which really
39:13helps a lot is uh using the kernel
39:15radius for this filter based on the
39:17projection of the of the lob of the brdf
39:19which means I can blur at adjust the
39:20right scale that's the you would
39:22actually get the kind of blurring you
39:24get from the stochastic
39:25process and a refraction we seem to
39:27really hate the proper refraction so uh
39:29we hack it here again we just take the
39:32BR of lobe for reflection we flip that
39:34and essentially do reflection upside
39:36down uh which turned out to be good
39:38enough for us again and once again it's
39:40a very puny rain March which is very
39:41cheap uh and then we throw that at the
39:44deiser once again here the refraction is
39:47at Full Resolution but we still want to
39:48Denise spatially and this is also once
39:51again using the projection of the BR
39:53brdf lobe uh which is why it's helpful
39:56to actually just use simple uh uh
40:00reflected reflection because then the
40:03that Lo is simpler than for
40:05refraction and uh we throw that at a
40:07temporal filter together with the
40:09reflection data uh and that one's pretty
40:12simple we actually uh get a bit of
40:14ghosting here but doesn't really matter
40:15because everything under the ice on top
40:16of the eyes is a bit blurry and this one
40:18specific case is fine so this also the
40:21spatial resolve ends up serving as a
40:22temporal filter of everything that's
40:24mostly it for ice but we also have this
40:26cool feature that's under the eyes
40:28that's cracks and it looks more than
40:30than actually is it's just a bunch of
40:31meshes with uh some fakery in the
40:33normals kind of Point them at the light
40:35source make them all catch light always
40:37and then if you throw that at our uh
40:39blurry refraction they they look pretty
40:41cool together especially with the uh
40:43beber absorption that happens with
40:46it and speaking of blurring things uh
40:49depth of field so that's something we
40:50have invested quite a lot of effort into
40:53uh but we've started quite simple
40:54actually we started with not really
40:56knowing what we wanted with do and we
40:58just plugged in this uh cool single pass
41:00Shader from danis gustavson uh which is
41:03awesome for its Simplicity uh but also
41:05has this kind of PE artifact that
41:06sometimes you get features that are a
41:07little bit too sharp in the foreground
41:09and uh it doesn't really make much sense
41:11but we actually consider that to be a
41:13bit of a feature initially because we
41:15didn't really know whether we're going
41:16for like Optical effects or kind of
41:17painterly effects uh we generally tried
41:20to treat the the frame as a as a
41:22painting we we're not aiming for photo
41:24realism and in this case we're kind of
41:25inspired by uh Studio gibl and how an
41:28artist might just use cor or brow
41:30Strokes in the background and
41:31essentially have this sameish amount of
41:33detail in the forr background so we kind
41:35of like this simplification it was doing
41:38uh so for some time we stuck with that
41:39but eventually like a proper Optical
41:42depto field one over uh because of one
41:44important feature that was tilt shift uh
41:47because uh we use a little bit of a
41:49called squeed driven development uh
41:51where you try to maximize cute
41:53aggression and in terms of if if you
41:55have this like cute little diaram and
41:57then you turn on a tilt shift off and
42:00everything looks even tinier you just
42:02have this like reaction and uh that just
42:05was sufficient to conflict the win over
42:07and move move to a proper Optical effect
42:10but for that we need to fix the
42:11artifacts and I have to admit that I I
42:13don't really actually understand do at
42:16least the way it's presented usually in
42:17in graphics literature I kind of get the
42:20the path Tracer of optical part of it
42:22but this kind of diagram is what comes
42:24to my mind when I think of do from uh
42:26hor
42:28uh and I just I like the diagram I think
42:31it's cool and I can understand how each
42:32point kind of increases into a little
42:35dis based on its circular confusion but
42:37I don't really understand how they
42:38should be occluding each other or why
42:40just like changing the slope slightly
42:41means that they blend against each other
42:42in different way uh and I I the the
42:45original paper suggested that you can do
42:47BR Force by sorting all the samples so I
42:49tried kind of Trea it as an oit problem
42:51throwing a bunch of O algorithms at it
42:53but I couldn't really get anything cool
42:55out of it and I still really don't don't
42:57don't buy that it works but there's a
42:59classic thing which does work and that
43:01is you just jutter the image from
43:04multiple viewpoints and you use an
43:05accumulation buffer to uh to generate a
43:08single picture out of like hundreds
43:13potentially and now we can also do it uh
43:16a bit quicker by synthesizing those
43:18additional viewpoints by just um doing a
43:20paralle distortion of M of the imag you
43:22have generated and then filling in the
43:24detail however we can only generate one
43:27image so uh our options are limited we
43:29have to do pretty fensive reconstruction
43:31to to do that uh but I have tried a
43:34whole lot of things I tried like I think
43:35have a giant do hael file which had like
43:3810 different algorithm implementations
43:41and this one's specifically about trying
43:42to synphase the Views I found the whole
43:44feeling to be really difficult to
43:45actually achieve and uh in general like
43:47nothing really worked well and uh it
43:49doesn't seem like there's any algor that
43:51handle foreground defocus well and
43:53actually run in real time so uh I kind
43:56of came back came back to the drawing
43:57board and I figure out like okay is it
43:59even possible to extract something cool
44:01out of screen space and so what would a
44:04path Trace version of do look like but
44:06using Ray marching instead because
44:08that's how we solve all the problems
44:09here
44:11so uh I uh simply use the ray marer by
44:15tracing tons and tons of rays in the
44:16same way that's a path Tracer would so
44:18you sample additional points on top of
44:20your sensor aperture and then you kind
44:22of mirror them around the focal point
44:24and you can actually get an image like
44:25this so it turns out that there is a lot
44:27of information in screen space that you
44:29could use to get pretty good forr
44:31defocus however this was super slow was
44:33freshing all the caches and costing like
44:35a 100 milliseconds on a chunky GPU so I
44:37need it to be spit up uh but if you if
44:40we look at from it like from the front
44:42and consider just the different arcs of
44:45the radial uh Dove kernel and if you see
44:48what kind of race we Trace within just
44:50that one slice they all kind of Trace
44:52similar
44:53information they end up sampling the
44:56same points over and over again which
44:58means we can uh prefetch a bunch of data
45:00and reuse information between
45:03them so what we do in in the shter is
45:06that we working those 1D radial slices
45:08of the the kernel we prefet all of the
45:10COC depth well it can pref deep depth
45:14values if that's what you work with we
45:15pref those CC values into local data
45:17share uh and then uh we calculate all
45:21the intersection so for each of the uh
45:24Rays um and each of the slopes uh we
45:27essentially find o n time M
45:29intersections uh the math ends up being
45:32very similar because the uh within a
45:33single slice is just the slope of the
45:35ray changing uh so it's just a little
45:38bit of like extra division per
45:40intersection uh and we store per each
45:42Ray has it intersected with the surface
45:44yet for each of the steps that we take
45:46the Y Ray marching and we start as a Ray
45:48mask uh as a bit mask for each of those
45:51Ray slopes in vgps and then finding the
45:53actual intersection points is just like
45:56first bit first bit High uh and then you
45:58can find out which is essentially the
45:59first
46:00intersection uh and then we can also
46:02refine that using a second test so now
46:04we have a way of do bulk rate
46:07marching and it's essentially still the
46:09full rate marcher just optimized so we
46:11can do a fun thing of uh visualizing how
46:13individual sub rays of that work so you
46:15can actually see that this is quite
46:16similar to the accumulation buffer
46:18approach but all of those views are just
46:20done by Ray marching and so we get
46:22enough Precision out of that little uh
46:24test and the second intersection so the
46:26camera is is not moving here I'm just uh
46:28nudging which Ray is being visualized
46:30and if you combine them all you get Del
46:33essentially so uh I guess we're done
46:35here and uh this
46:37works
46:40right and then uh we get
46:44this it's like what is even going on
46:47here so I thought I had something cool
46:49and it will work but then artifacts
46:51appeared yet again U and this Edge case
46:54was initially a bit difficult to find
46:56but appears because that's that's a
46:58pretty basic back background
47:00reconstruction when Ray travels uh we uh
47:03just use the last hit point that
47:05actually s if it travels under the
47:06surface and use that as the uh uh as the
47:09color so on the left side you can see
47:11that this essentially because it's kind
47:12of trailing artifact behind objects
47:14where you don't have information which
47:15most of the time is actually fine uh but
47:18uh in this specific Edge case when you
47:19have multiple Rays coming at different
47:21angle so different of those ond slices
47:23going behind the surface they all end up
47:25having the same point they as all use
47:27the same fullback and that's you end up
47:28just kind of sharpening just next to uh
47:31to an edge and uh you can see it by just
47:33showing different angles with ond slice
47:36you can kind of see like something is
47:37rotating around those the singular
47:39points that previously were uh sticking
47:41out and the sad thing is that I actually
47:44don't know how to fix it yet properly at
47:46least not in a way there will be
47:47performance and we have a sort of hack
47:49so what I found works sufficiently well
47:51for us is that when you actually find
47:53that intersection Point uh you can see
47:55whether it's actually close to what we
47:56trying to initially intersect so you're
47:58trying to find intersection close to the
48:00uh focal point and if that uh point you
48:02have actually found is somewhere far
48:04away from the focal point it means the
48:05rate has traveled under the surface and
48:06your intersection is actually not
48:08accurate in that case I assume that well
48:10it will be a blurry something uh
48:13therefore I fall back to a uh just a
48:15pre- blurred version of the background
48:16that the Shader does uh just alongside
48:19of all the calculations uh and I haven't
48:21really had time to figure anything
48:23smarter but this is sufficient at least
48:25for us because this artifact is still
48:26pretty rare
48:27and you can still get pretty good
48:28results out of it without any of the
48:29typical uh like aliasing issues or like
48:32seeing where each each layer of the do
48:35splits up from the others and you can
48:36get something quite
48:42creamy and you can also throw typical
48:45optimization sets such as analyzing
48:47content of the image so we detect for
48:49example which U of the tiles can be done
48:52Health resolution because of the low U
48:55Co variations within there uh which
48:57parts we need to be doing with with full
48:59detail and which uh can be simplified by
49:01just doing a simple like background GA
49:03kind of like the uh unreal five
49:04presentation and we also going to use
49:06myips to pre filter some of the data and
49:07and get rid of some of the sharp like
49:10hot spots and
49:11noise uh but it still is not free it's
49:13not as sample efficient as some of the
49:15other approaches uh works for us because
49:17um uh our dynamic range is fairly low so
49:19we don't have any like super hot
49:21highlights in the background like some
49:22sci-fi games might have uh so for us is
49:25fine uh but in practice if you try to
49:28use it for a game with like very intense
49:29HDR it might you might find it a bit
49:31under sampled so it might be bit of a
49:33Sit situational thing uh and in practice
49:36for us we just throw the resulting noise
49:37at tea and at least in our case it works
49:40but uh in other cases it might give you
49:42the typical uh ghosting uh and for us it
49:44runs in roughly 1.3 to 1.5 milliseconds
49:47on a 208 at 1440p so it's sufficient at
49:51least and uh but in photo mode we also
49:53like crank up the settings because when
49:54you actually have the full tilt shift
49:56you need to few more samples to properly
49:58resolve
49:59it uh and but to illustrate what we can
50:01get out of that this is the the version
50:03that was costing 100 milliseconds and
50:04this is the version costing 1.3
50:06milliseconds uh which actually looks
50:08slightly better on some of the edge
50:10cases because you can see the pinching
50:11is gone but otherwise same
50:14thing in terms of code uh I was going to
50:17prefer like a nice version you can plug
50:19in into your code and it might come soon
50:21at some at some point but I just like
50:23essentially copy pasted one of our shers
50:24and dumped it into gist so if you grab
50:27it you it will not work Straight Out of
50:29the Box same you need to do some
50:30massaging but at least like some of the
50:31ideas are there you can just use it as a
50:33Bas as a reference and doesn't have any
50:35of the fancy multi resolution stuff so
50:36it's going to be slower than the version
50:37we have right now because that requires
50:39quite a complex uh pipeline multiple
50:41comput shaders analyzing all the
50:44content uh and since we're moving down
50:47the the graphics pipeline or to the end
50:49of the graic pipeline uh let's talk
50:51about image formation so you have
50:53generated a bunch of RGS and Bs using
50:55all fancy light lighting fancy post
50:57processing and so on but how do they
50:59actually form an image like there's a
51:01bit of a discussion or debate about that
51:04do you model human perception and try to
51:06kind of synthesize something that human
51:07would see when looking at the picture uh
51:09do you try to use go after photography
51:11and film uh but in practice it rendering
51:14is its own art medium you can do
51:16whatever the heck you want it's
51:17eventually all about uh uh evoking some
51:20kind of um U perception in in in a human
51:23being uh just make sure to use conscious
51:27choices and not fall into traditional
51:28traps and one of the traps you may fall
51:30into is this notorious six problem where
51:34uh if you apply a pair Channel toil
51:36mapping curve or display mapping curve
51:38to your image uh you will most likely
51:40see those six colors popping up
51:42everywhere and uh at the bottom you can
51:45see just clamping the colors you can
51:46apply see applying Reinhard and the
51:48typical acis curve and the right side of
51:50it where exposure increases is always
51:52the same and the only thing which is
51:54special about the colors is they are
51:55corners of the RGB Cube so if you send
51:58see that in your pictures it's not
52:00something you actually made as a choice
52:02it's just something that is an
52:03engineering choice of the color space
52:05that you're
52:06using to deal with that we use a display
52:09mapper that's a this is a Shameless plug
52:11but I I made this play M for kind of the
52:14same timeos starting on Tiny Glades
52:17called a tiny map map face and it design
52:19specifically to avoid this uh this issue
52:22so instead of kind of allowing the
52:23current the the colors to saturate the
52:25extremes of the SB Cube instead it
52:27shifts towards White using a path uh to
52:31White which I consider to be uh fairly
52:33neutral but the whole thing is is
52:34generally kind of opinionated and might
52:37not work for you but in terms of how you
52:40actually would plug that in it's
52:40actually very simple 3D lot that you
52:42apply after rainhard uh and it's just
52:44kind of can fix your colors potentially
52:47there are some caveats in that it only
52:49works with srgb color it doesn't do HDR
52:51output of the the boook uh and it is
52:54subjective to tune to my specific
52:56preferences uh but if you want something
52:59bit more bit more neutral and with
53:00similar characteristics you might try
53:02want to try agx by choice bka which has
53:05some of similar
53:07characteristics and uh just to
53:09illustrate some of the uh some of that
53:11in pictures this is what happens if you
53:13just take the H values the TR stimulus
53:15and you clamp it you end up with this uh
53:17lovely yellow color that has been uh dou
53:20the rat piss
53:23uh which shouldn't be there because if
53:25you just reduce the the the radiance
53:27values to the to levels where they don't
53:29clamp it's actually doesn't have this
53:31yellow but if you apply everyone's
53:33favorite ases you have this yellow as
53:35well and even if you use something
53:37fairly gentle like the Reinhard uh
53:40operator uh you still end up with this
53:42kind of like shift towards this kind of
53:44sickly unsaturated strange colors and if
53:46you apply Tony mcmap face you actually
53:49get something that's kind of resembles
53:50those original colors at least to some
53:53human perceptional
53:54systems and uh that's would be it uh I
53:57think we have like maybe three minutes
53:59for questions and so thank you so
54:11much thank you and and if we don't have
54:15enough time for questions just catch me
54:16afterwards and uh I I'll answer
54:18everything okay please raise your hand
54:21and I'll come
54:24by thank you for the present was very
54:27interesting um so you've worked with
54:29rust right like so uh what made you
54:31decide using rust over C++ and how was
54:33your like experience working with it uh
54:36so for our game I don't really imagine
54:38using anything other than rust uh
54:40because it is simple at the surface but
54:42under the hood there's uh lots of very
54:44complex chains of operations uh we
54:46support things like infinite under read
54:48with animations and all of those
54:50generators uh have just dozens upon
54:53dozens of systems that would be kind of
54:55a nightmare to keep track of
54:56all of the generat are also fully
54:58deterministic which means that we need
54:59to have this kind of precision that rust
55:01is really famous for and so this kind of
55:03harsh control so some people might see
55:05using rust as an impediment iteration we
55:07actually kind of see it as a
55:08prerequisite and uh we haven't had a
55:11single memory corruption related crash
55:13in the entire development of the game
55:15and that's not something I'm familiar
55:17with uh from Plus+ uh so in our PR in
55:20practice for us using R has been
55:22absolute blast and we also found that so
55:24you can get stuff rolling very very
55:26quick quickly by using the the Cates
55:28ecosystem for example uh we use the the
55:30ECS from the bavy engine uh to like run
55:33all over our uh gameplay logic and
55:35there's a whole bunch of crates for all
55:36of our pliance processing backgrounds
55:39and window interactions and so on so uh
55:42it's it's kind of really a superpower
55:44like I'm used to just writing your from
55:45scratch but from R you just like collect
55:47crates quickly move on then rewrite
55:48what's important and it's really a
55:51refreshing
55:52experience all right any other questions
55:56please raise your
56:02hand uh first off great presentation the
56:05passion comes off immensely and you're
56:07quite good at explaining Concepts um I
56:10uh was kind of curious about your
56:13experiences with reer you mentioned that
56:14you felt that GI rester GI was a bit
56:16overkill for your applications U first
56:19I'm curious why that's the case second
56:21you mentioned you were doing some
56:22experimentation with restar di how's
56:24that been going uh so research GI
56:27specifically it's overkill for our case
56:29because the game is still Mo mostly
56:31Outdoors uh some people are trying to
56:33kind of hack it to be indoors as well
56:34but the simply because of the softness
56:36of the of the sky and the Sun Well sun
56:39is not very soft but because of the this
56:41presence of plenty of light Outdoors if
56:43you don't have lots of local lights your
56:45variance on the on the input on the Mard
56:47proc is actually just very low and you
56:49don't need a lot of denoising Machinery
56:51to to deal with that and when you're
56:53doing Mont caros style GI calculations
56:55uh what you're you're really doing is
56:57just dealing with variant so you have
56:59some kind of way to sample the the
57:00radiance and some way to filter it and
57:02rester was just too much for our needs
57:04and so it comes with some uh fixed costs
57:07that we need to run to to do the
57:09reservoir exchanges and there those are
57:11just like a little bit too much memory
57:13traffic for the potato devices that we
57:15try to run the game on so in the end
57:17just wasn't a win it would have been
57:18fine but but uh we didn't need it that's
57:21that's it in terms of Resto di I really
57:23only just started tinkering with it uh
57:25because we Curr they have a silly limit
57:26of like you only get 32 light sources if
57:28you place additional ones they don't
57:30cast light and that's a just a silly
57:32hack that we had to have to actually sh
57:34game uh but I'm looking into um using
57:38something like a light tree to to
57:39accelerate the sampling of the lights
57:41and then to to throw that at rester di
57:43to actually do the U allow pixels to
57:46talk to figure out which samples
57:48actually relevant to their uh lighting I
57:50have done some early tasks this a little
57:52bit too noisy for my taste uh so I'll
57:54just be t with it it's a
57:56mildly promising I suppose but still
57:59still I end up having a little bit too
58:01much variance from the initial test and
58:03then was anticipating so it hasn't moved
58:05on very quickly yet all right we have
58:07time for one more
58:17question uh first of all I just wanted
58:20to say it's a really great game and a
58:22really great presentation uh I'm just
58:24curious when you're doing putt racing
58:26how many uh bounces do you do and do you
58:29also do maybe a Radiance cach so for uh
58:33path tracing the only real path tracing
58:34we use in the game is for the reference
58:36mode and then we just have something
58:38like three bounces because because again
58:40once again because of the Alder nature
58:41we don't need more don't need more uh
58:44but for the game itself it's not path
58:45tracing per se it's more like U I guess
58:48single pass of final Gathering is that
58:51we just sample we just should race out
58:53sample radians and feed them back but by
58:55the temporal compon which means that you
58:57essentially get like infinite bounces
58:59from it as well so that's that's that's
59:01that's essentially that it's not really
59:03the path
59:05tracing all right thanks to much thank
59:08you

About this transcript

This page contains the full transcript of Rendering Tiny Glades With Entirely Too Much Ray Marching by Graphics Programming Conference, generated from the public captions YouTube serves with the video. The transcript has 12,132 words across 1,665 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.