YT YouTube2Text

YouTube2Text → Transcripts → Bird's Eye View_ Software and Human Perception - Part 2

Bird's Eye View_ Software and Human Perception - Part 2 — Transcript

by NPTEL-NOC IITM · 2,181 words · 332 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01[Music]
0:17So indeed what we think is around us
0:21even in the real world is only what our
0:23senses can sense and what our brain can
0:26interpret. So we are unable to see or
0:29feel anything else. So similarly the
0:32modern VR experiences are simply an
0:34illusion to our senses. Right? And
0:36that's why it's so important to
0:37understand how actually how does it
0:39actually happen and perception
0:41psychology will help us understand how
0:43our senses and brain work together to
0:46experience things around us including
0:48virtual reality. So here's some typical
0:51questions that arise in virtual reality
0:54that perceptual psychology will help us
0:57answer. For example, how much resolution
1:00is enough so that you don't perceive an
1:03individual pixel on the screen but you
1:06see a picture as continue?
1:08Or how many frames per second is enough
1:11so that we perceive motion as continuous
1:14and real?
1:16or how do we perceive the source of
1:18audio signal, right? Like if someone
1:20calls you, usually you know which way to
1:23turn, right? So how do we do that in VR?
1:26So these are the examples that we know
1:29the answers to and they're related to
1:30these neural structures in our brains
1:32and perceptual psychology helps us with
1:35understanding this. So here are some
1:38questions that we don't quite know the
1:40answers to. For example, is my height
1:44right in virtual reality? So why do I
1:47feel like the floor is wrong at the
1:50wrong distance? Or why is that user too
1:52short or too tall?
1:55Now does this carton of milk appear to
1:58be right size or the correct distance?
2:01Because we know how well sorry we know
2:04here in Europe I guess you you have
2:06cartons of milk or something cartons of
2:08something that you really know the size
2:10of in VR how to make it look exactly how
2:13it should be.
2:16That's a really perceptual question.
2:19Now, is one tracking method better than
2:21another?
2:23So, at Oculus, for example, we
2:24implemented tracking methods for which
2:26all of the engineering numbers, so
2:29tracking errors such as worst case
2:30arrow, best case error, average error,
2:33they were all better, but perceptually
2:36methods appeared worse. Similarly for
2:39lenses the distortion parameters they
2:42had to be perceptually tuned just like
2:44you do it in in your homework one before
2:47at Oculus we released them as consumer
2:49product. So why why is that and how do
2:52you test these methods in VR then if
2:54they are so individual if if they if
2:57they depend on individual perception so
3:01we don't quite know yet for VR all of
3:03these questions are still somewhat open.
3:06So what or at least this is a re these
3:08are researched topics. So what causes
3:11nausea? What causes fatigue?
3:14Big questions we don't know. We have
3:17researchers in our lab who are trying to
3:19figure out what causes nausea in VR. Is
3:22it really vion or something else? So do
3:25I really feel immersed in this
3:26experience or not? So all of this has to
3:29do with sensation and perception. And to
3:32start answering this questions and more
3:34questions, we need to learn some basic
3:37physiology of the human body like
3:40including our sense organs like eyes and
3:42ears and neural pathways. And we must
3:45learn uh known results from experimental
3:47perceptual psychology that are rel
3:50relevant or related to VR. But most
3:53importantly, we really need to
3:54understand what is this interface of
3:56these VR systems with our senses and
3:59perception. So this artificial stimuli
4:01that coming from these engineered VR
4:04devices or uh displays they confuse our
4:07brains and sometimes our brains don't
4:09mind but sometimes they do and that's
4:12why you feel uncomfortable. So we need
4:14to understand the reasons for this in
4:16general we really really need to
4:17understand the reasons from the
4:19physiological point of view.
4:23Okay.
4:25A proven VR system without any motion
4:27sickness.
4:30I don't know such thing. No, I don't
4:32know. I think any VR system can produce
4:34some environments where you'll feel
4:36motion sick.
4:39There are very comfortable systems out
4:41there, but I don't know anything that
4:43would never cause any discomfort to
4:45everybody. Like for example, our own
4:48Steve Laval who is a co-instructor in
4:50this course. He is always uncomfortable
4:53in any VR systems in any demos. So there
4:56you go.
4:58Can't we input our height weight and
5:00create our avatar? You think that it's
5:02so simple, right? No, it's not because
5:05it's all relative like how you you first
5:08of all you don't even have your hands in
5:10many experiences and you can't have them
5:13with the technology that we have. So
5:15that all messes up with the rest of the
5:17perception.
5:19Okay. So let's move on. Let's see.
5:23One relevant to VR question that is um
5:27important to your relevant to what you
5:29just asked is how to perceive depth and
5:32that's very related to perceiving size.
5:35So something like your height or how far
5:37the floor is. So how and you now you'll
5:40see why the answer is not that simple
5:43that there is a lot of processes that
5:44are going on into figuring out the
5:47depths of objects or the distances to
5:50objects. So this has been studied by
5:52perceptual psychologists for quite a bit
5:55of time and there are some answers to
5:57this question. So depth perception for
5:59example is something that uh is studied
6:02really well and what it does is it
6:05allows us to depth perception in our
6:07brains is something that allows us to
6:09see objects in 3D and also something
6:14that allows us to estimate distances to
6:16objects. So why is this relevant to VR?
6:19I think you already know because you
6:20just asked that question. So um in
6:24engineering
6:27uh we computer sciences and engineering
6:30engineers we overemphasize
6:32the importance of real realism or
6:35especially stereo uh what we can have in
6:38these devices. So how is it important?
6:40How much is it important to see the
6:43world through two eyes?
6:45And um the truth is that moninocular
6:48cues so something that you see even with
6:51your eyes one eye closed or in
6:54panoramas. So those are moninocular cues
6:57that provide a lot of information and
6:59there is no stereo.
7:01Uh so for many applications this is
7:04quite enough to just have that and not
7:06have binocular cues. So using panoramas
7:08instead of 3D models for example could
7:10reduce the cost and simplify the
7:13engineering design as well
7:14significantly. That's why I want to see
7:16it in our projects I hope. But of course
7:19realism is also very important. St is
7:22also very important in many applications
7:24such as virtual surgery or virtual
7:27prototyping training program programs of
7:29of sorts. However, we need to learn when
7:33stereo is absolutely necessary and when
7:35it is not. and panoramas might be good
7:38enough for some applications and maybe
7:41not good for others. So it's important
7:42to understand what are those. So uh
7:45let's look briefly at how the brain
7:48processes uh this depth information. So
7:52how does our brain know when something
7:54is close or far away? So we call it
7:57cues. The cues that help the brain to
8:00make interpretations or make conclusions
8:02about distance to the objects. So there
8:04are two types of cues.
8:06um cues that come from one eye only,
8:09they're called moninocular. And cues
8:10that come from uh two eyes, binocular
8:13cues. So not all depth cues have to be
8:16binocular. Uh just like panoramas. So if
8:19you close one eye, you can still infer
8:21depth to many objects and those are
8:24moninocular cues that your brain will
8:25use. So for example, if you're in the
8:27mountains or in an airplane, also a lot
8:29of uh binocular cues are missing, but
8:32you still can in perceive depth infer
8:35distances to objects.
8:37Uh okay, so let's talk about that
8:39binocular cues first.
8:42Um so one binocular Q is convergence and
8:47that means that depending on how much
8:49your eyes needed to converge on an
8:51object the bra when looking at it the
8:53brain will remember that effort how much
8:55effort went into that and will infer the
8:58distance to that object from that. So
9:01this is called convergence cube. And we
9:03turn our eyes only on objects that are
9:05within kind of four to 6 m away, closer
9:10than 6 m away, not further than that.
9:13And this angle and how much the eyeballs
9:15needed to turn is what the brain will
9:17remember. So notice that the objects
9:19that are further away, this cue is
9:21absent, but we still can infer distances
9:24based on other cues. So another
9:27binocular cue is binocular disparity
9:31and that's something that depending on
9:33how different the two objects appear on
9:36your retinas
9:38and this is also true only for objects
9:4010 to 6 m away from us not further than
9:44that. So this cue is very very important
9:47because this is what allows us to
9:49perceive objects as three-dimensional or
9:52stereo. So, two eyes will see completely
9:56different objects, right? And the brain
9:58combines them into this
9:59three-dimensional image. And this what
10:01makes HMDs or um uh cardboards so
10:06powerful because they allow for so much
10:08immersion because they have this
10:10binocular disparity cue in them. And
10:14they these visual displays, they're
10:15quite different from other displays that
10:17we had before like TVs, computer screens
10:20or phones. they they didn't provide
10:22binocular disparities so they were less
10:24immersive.
10:26So now let's look at all of the
10:28moninocular cues that also used by the
10:31brain to estimate distances even with
10:33one eye closed. So uh there are many of
10:37them many studied moninocular cues and
10:41each of them gives provides a lot of
10:43information to the brain and I will not
10:45go through them now because we will go
10:47through a special lecture on binocular
10:50on monoccular and binocular cues on
10:52depth perception. But if you understand
10:55these cues well enough then you can
10:57create illusions and also uh answering
11:00this question about why is it not simple
11:02to just set up the height of your avatar
11:04because all of this the brain will still
11:06analyze all of this. It will use a lot
11:09of other information from your
11:11environment to create this coherent
11:14picture of the world and all of this
11:16should be right and I don't know how
11:19it's related exactly because again I I'm
11:22not a researcher in that area. Uh I just
11:24know it's not that simple. If you just
11:26set up your avatar with a particular
11:28height, you'll see that it will not look
11:30the same because maybe your shoulders
11:32are not the right uh width and that will
11:35give you an illusion of different
11:37height. So all of this play together in
11:40some way and the brain makes
11:42conclusions. So see how easy it is to
11:44make an illusion.
11:47Um like and and this is just to
11:49summarize all of these monarch cues. Our
11:52brain automatically will make certain
11:55assumptions using all of these cues
11:57which will sometimes be wrong if other
12:00information is meas. So these illusions
12:03demonstrate how much work our brains uh
12:08do all the time together with the sense
12:10organs. So they are constantly doing a
12:13lot of work to fill in missing details
12:16uh or make interpretations about the
12:18world. So, um, let me create some kind
12:22of illusions back here. I think I wanted
12:25to show this here. How easy it is create
12:28illusions. So, let's turn on the pen and
12:30I'm going to give a stick to this panda
12:33and give the same size stick. So, think
12:35about it. It's the same size. Well,
12:37roughly the same size, right? So, it
12:39will occupy the same amount of photo
12:41receptors on your retina. However, it
12:44appears that this one is smaller. So
12:48here I just gave it the same height like
12:50in Unity I could have modeled it in some
12:52way. So what what went wrong? It's the
12:54same size, right? Because um well it
12:57turns out that if the brain will again
13:00use all of these cues to analyze
13:02distances and it will infer that this
13:05might be a closer stick than the other
13:08one and therefore the sizes will be
13:10inferred as well. So size perception and
13:12depth perception are very closely
13:15linked.
13:16So um this means that whenever we are
13:20putting the head headset headsets on or
13:25viewing virtual experiences through
13:27cardboards we are interfering with this
13:30complex processes
13:33that are constantly going on in our
13:35brains and we better understand what it
13:37is that we're interfering with right to
13:39create
13:41control illusions.
13:44So if some of the depth information is
13:46missing
13:48uh then the brain will just make stuff
13:49up like in this example. Let me click on
13:52this illusion on this link.
13:57So it's a very famous illusion. I want
13:59to ask you
14:02do you see the dancer spinning clockwise
14:04or counterclockwise
14:07or both? People can flip it. So what is
14:12your experience?
14:17Let me show it again. So I clearly see
14:20it as clockwise.
14:26[Music]

About this transcript

This page contains the full transcript of Bird's Eye View_ Software and Human Perception - Part 2 by NPTEL-NOC IITM, generated from the public captions YouTube serves with the video. The transcript has 2,181 words across 332 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.