YT YouTube2Text

YouTube2Text → Transcripts → Bird's Eye View_ Software and Human Perception - Part 3

Bird's Eye View_ Software and Human Perception - Part 3 — Transcript

by NPTEL-NOC IITM · 2,892 words · 465 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:01[Music]
0:16So that's good. Now I have a question.
0:18So could you try to think and uh figure
0:22out how to explain the solution? So what
0:26information depth information in this
0:29case is missing that the brain makes
0:32incorrect conclusions or inconsistent
0:34conclusions in this case.
0:47Okay. So
0:50many different ideas. Very good.
0:53Leg is not on the ground. So some
0:55information is missing and there's a lot
0:58like if you add lighting that would help
1:00but I can add just one cue. So again
1:02from the perceptual psychology point of
1:05view or from the this monocular cues
1:08that we will learn in details later one
1:11of the cues is interposition.
1:15So look if I just create these two
1:18lines. Oh this is a bad line.
1:21Oh let me this is not good. Let me let
1:24me try again.
1:26So, which leg is in front? Is it the
1:31right leg?
1:35Is it this way? Right. The leg this way.
1:37This is the right leg and this is the
1:39right leg. But is it is it the which leg
1:42is in front? The one that the ballerina
1:44is standing on or the the one that it's
1:46pointing with. Right. So just that or
1:49the face like if I could see whether the
1:52face is this way with eyes
1:56or it's the hair here right so there you
1:58go that would solve the illusion so all
2:01we're missing is this interposition
2:03which leg is in front of which of course
2:06you could do some uh something with
2:07shading or u demonstrating the floor but
2:10this is the simplest of the cues that's
2:12missing. There are many other that's
2:14missing but this is the simplest that
2:16could be used to um explain the
2:20illusion. Similar here interpositions
2:22right which object is in front. So the
2:23brain will take into account any T
2:28um T shapes between the edges of the
2:32objects. If it sees a T on the retina
2:36that means the object here is in front
2:38of the objects behind and that's it.
2:42So,
2:44but we'll learn more about it. So, now
2:47uh it's a bird's eye view. So, now let's
2:49look at uh human senses that are
2:51relevant to VR because we'll talk a
2:53little bit about the physiology.
2:56So, what are the senses that are
2:58relevant? It's the eyes, the ears, and
3:00the skin for touch. And what are these
3:04structures within these organs that
3:06actually do the sensing and how they do
3:09it? So, that's what we want to talk
3:10about. And that's related to physiology
3:12of the human bodies. So let's recall
3:15again that a sensor or sense converts
3:17the real world energy into a signal of
3:21some sort. In the case of our bodies,
3:22the signal is the neural impulses that
3:25go to the brain. So each sense organ
3:28receives a different type of energy. And
3:30for the eyes, it's light waves. For
3:33ears, it's the uh air pressure waves. Of
3:36course, there's also the vestibular
3:37sense inside of the ear, but we don't
3:40talk about that. And for the skin, it's
3:43some kind of feedback force, not thermal
3:45energy. Again, we talk don't talk about
3:47that for the purposes of virtual
3:49reality. So, now let's look at a formal
3:52table of classifications of senses. And
3:55here you'll see why it's not so simple
3:58to answer the question, how many senses
4:01does the human body have? So you'll see
4:04that I'm not even mentioning
4:06propriception here as another sense but
4:08it will be in another slide. So let's
4:10see this is a more appropriate
4:12description of our senses and their
4:14classifications. So for each sense there
4:18is a sense organ and there are
4:20receptors. So these are actually this
4:22tiny sensors cells that are doing the
4:26sensing. So in each eye there is over
4:29100 million photo receptors that receive
4:31the electromagnetic energy
4:34uh and converted into neural impulse. So
4:36this electromagnetic energy that we can
4:38send so that our photo receptors can
4:40sense it's just this small uh frequency
4:43range that is called visible light. The
4:45others we are we we we're blind to other
4:48forms of electromagnetic energy
4:51for audio and touch touch and balance.
4:54So audio, touch and balance these are um
4:59they so they involve sensing motions,
5:02vibrations, gravitational force. So
5:04therefore if you need to sense uh forces
5:08the receptors that are called that are
5:10doing it are called mechano receptors.
5:13So they are sensing these forces in some
5:16way. And note that for our vestibular
5:18sense or balance sense here it it helps
5:23us to know which way our head is
5:25oriented including the direction of up.
5:29So very similar to I'm used that we
5:31talked about during the last lecture. So
5:33in fact inside of our ear there is this
5:36physiological structure that closely
5:39resembles accelerometers and gyroscopes
5:42the ones that we design.
5:45So that's really fascinating. And then
5:47for the sense of touch, there's also
5:50thermal receptors and they can detect
5:52the temperature. And finally for this uh
5:55sense of taste and smell, these are
5:58called chemo receptors. So they provide
6:00signals based on the chemical reactions
6:03that are happening inside of our mouths
6:05or inside of our noses.
6:08And then the brain analyzes all of the
6:11signals coming from all of the sensors.
6:14So it does a lot of work, a lot of anal
6:17analysis. And we are mostly not paying
6:19attention to what our brain is
6:21constantly doing. But if you stop for a
6:24bit and think, this is a very very
6:26complex task because all of the senses
6:28they come in, they need to be
6:30interpreted interpreted together with
6:33many other things that are going on. So
6:35the brain also takes into account
6:37attention, current mood, memories,
6:40biases, language. So all of these are
6:43used together by the brain and fused
6:46together into this uh into one coherent
6:49picture. So what is really interesting
6:52is that these processes they're going on
6:55continuously and uh so the brain is
6:58doing perception all the time. Uh but
7:01it's effortless, right? So we don't even
7:03notice. So maybe only on some intense
7:06days we may feel like a headache or we
7:09need to sleep a little more to reboot
7:11our brain. Um but still we are not
7:14consciously aware of all of this
7:16perception process that are going in in
7:18our brains. But it's becomes very
7:21important for us computer sciences and
7:23engineers to become aware of it and
7:25study it a little bit as if we were
7:27perception psychologists because we need
7:30to understand how to create this
7:32engineered illusions.
7:35Also something to remember is that when
7:37the brake made mistakes and sometimes
7:39the brain makes mistakes that's what uh
7:42then illusions uh are based on that.
7:46Okay. So let's talk about uh the
7:50perception process a little more closely
7:53with respect to neural structures in the
7:55brain. So the perception happens after
7:58the sense organs send signal to the
8:00brain v via neurons and human bodies
8:04contain around 86 billion neurons
8:08and 21 billion neurons are located in
8:11this what's called cerebral cortex. So
8:14it's the outer part of the brain that
8:17handles highlevel functions such as
8:19attention, memory, language, emotion but
8:22also the large portion of it is devoted
8:25to vision interpretation. So 30% of the
8:29cortex uh is uh cerebral cortex is um
8:33devoted to vision. So we are largely
8:36visual animals and we can compare that
8:40to 8% of devoted from cerebral cortex
8:44devoted to touch and only 3% to hearing.
8:48So you can uh go to this Wikipedia page
8:51and it's listing uh a lot of organisms
8:55different animals by number of neurons
8:58that they have in their whole bodies and
9:00in the cerebral cortex. So there are
9:02animals that have more neurons than
9:05humans both in their bodies and in the
9:07cerebral cortex. So for example,
9:09elephants have 250 billion neurons in
9:12their bodies while some dolphins and
9:15whales have almost twice as many neurons
9:18in their cerebral cortices. So how does
9:22that reflect intelligence is not very
9:24clear of course, but we know how to
9:27count neurons at least. So another
9:30important factor in how we perceive
9:32things or our overall cognitive ability
9:36is um the interconnections between
9:40neurons. So if you're a computer
9:42scientist, you um you can think of the
9:45brain as a large graph, a very very
9:48large graph. And then the nodes are
9:50neurons and the lines or connections
9:53between nodes are axons and connections
9:56in our brains are formed as our brains
9:59develop. So you can think of some
10:01organisms who have more nodes. So they
10:04have more neurons and and then um these
10:08are but maybe less connections. These
10:10are more sparse graphs or maybe some
10:13organisms who have less neurons but more
10:15connections and these are then dense
10:17graphs right so you can think of it like
10:19that and of course there is a debate
10:21whether that affects uh intelligence as
10:24well for example females and males they
10:26have different number of neurons but
10:28also different number of
10:30interconnections
10:32so then how this processes happen so as
10:35I said the biggest part of the cerebral
10:38cortex in the humans in humans is
10:41devoted to vision which means that there
10:43are a couple of billion of neurons that
10:46are only responsible for vision. So
10:49compare that to um
10:52computer vision algorithms that we have
10:54or neural nets that we build right. So
10:57so far any of the algorithms for
11:00computer vision cannot compete with the
11:02human brain. So how does the brain do
11:04this? So the brain has to fuse a lot of
11:08visual stimulas. So when the light hits
11:11retina, the signal is sent to the brain.
11:14But there are already neurons on the
11:15retina on human eye retina which will be
11:19doing a lot of information already. So
11:21some low-level information. So some
11:23low-level processes will happen already
11:26on the retina and that will be uh simple
11:30edge detection, line detection, corner
11:32detection,
11:33motion detection probably not but still
11:35not proven that it's not happening in
11:37retina. So these are already analyzed on
11:40the retina itself and then this
11:42information is passed down the optic
11:44nerve
11:46and uh to the cerebral cortex and then
11:50there is a hierarchy. So each next level
11:53of neurons combine the information from
11:55neurons before. So that means it's
11:58responsible for larger larger amount of
12:00pixels on the retina and they will
12:03recognize more complex patterns such as
12:05shapes then objects then eventually
12:08recognizing faces, places,
12:11uh people. So later on this information
12:14is fused together with memory, attention
12:16and other senses. So there are different
12:19regions in the brain responsible for
12:21each of these hierarchical levels of
12:23recognition and all of them are in the
12:26cerebral cortex. So this is how uh it's
12:30believed that we are able to see the
12:32global picture of the world through this
12:34hierarchical uh processing.
12:40Uh animals having more neurons than
12:42humans do they perceive? We we don't
12:44know. But yes, I think there's more and
12:46more speculations that they they highly
12:48intelligent. But uh more neurons doesn't
12:51mean that's very correct. More neurons
12:53doesn't necessarily mean highly
12:55intelligent, but yet it's not really
12:57clear. So I don't think anybody can say
13:00definitely yes or no to this.
13:04Yeah, that's interesting. Smell
13:05illusions if you through vision. Yeah,
13:07that's very very cool. A lot of
13:09illusions. I'll show a lot of illusions
13:11in this class as well if I have time.
13:13Okay. Actually, yeah, we have very
13:15little time left. Okay. So, um here's an
13:20example of how such hierarchical
13:23processing
13:25can lead to an illusion. So, let's look
13:39[Music]
13:41So can you explain this illusion? So
13:43where does the brain make mistake in
13:46processing of this visual information?
13:58Angles. Very good. It's related to
14:00angles. Be more specific. Maybe angles.
14:05Direction.
14:07Yeah, it's related to depth information.
14:08Very good. It's depth. But still, you
14:10can be even more specific.
14:15So, yeah, the angles are wrong,
14:20but the view is still not. So, it turns
14:22out it's bias
14:24bias towards some shapes. So, let's look
14:27at the explanation.
14:31[Music]
14:36[Music]
14:40So
14:41[Music]
14:45the explanation is that
14:48many objects in the real world
14:50especially faces are convex. So our
14:54visual information our visual system is
14:57biased towards convexity
15:00and that's it. Right? So if there is no
15:02cue to the contrary, we really really
15:04perceive it as it should be canvas but
15:07something weird happens with shadows,
15:09right? And try seeing this uh dinosaur
15:14dragon the way it really is, right? It's
15:16really really hard like I cannot unsee
15:19what I see. It's just weird all the time
15:20and I've been showing this illusion for
15:22many years. uh so even having the
15:25conscious knowledge will not undo the
15:28lifetime
15:30long bias and also evolutionary bias. So
15:34even babies can recognize faces. So some
15:37of these neural pathways they are
15:39genetically predefined. So it's very
15:41difficult to see a face as being
15:44concave.
15:46Okay. So, um,
15:49what makes understanding of perception
15:53even more difficult is something that's
15:56demonstrated in this video. So, let's
15:59watch it. Another illusion,
16:02the monkey business illusion.
16:06Count how many times the players wearing
16:08white pass the ball.
16:42The correct answer is 16 passes.
16:47Did you spot the gorilla?
16:49So, let's go. Did you see the gorilla?
16:58So, many of you may have heard of this
17:00experiment already. And if you've seen
17:02it, then you've seen the gorilla.
17:04Usually 60% of people don't see the
17:06gorilla.
17:07But the thing is that the professor,
17:09he's actually our friend who made this
17:11um illusion. He updated this video. So I
17:14have another question. Did you notice
17:17that a curtain changed the color? So he
17:20updated this video so that people who
17:21know about the gorilla Yeah, there you
17:24go.
17:25So yeah, the the curtain did change the
17:27color
17:29from at some point from yellow to
17:32something else. But um okay, let let me
17:37show you
17:39the monkey business.
17:41Let's start it again.
17:48Just watch the curtain
17:51and um
17:59the gorilla
18:01and the curtain changing color.
18:05So it was red, it became yellow. There
18:09you go. Orange.
18:11The correct answer is 16 passes.
18:15So uh that's very interesting, right?
18:22So, does the dinosaur illusion work with
18:24the outside with the moninocular or
18:27binocular? I think it's moninocular. You
18:29can make it in panorama. It will be
18:30still.
18:32Oh, yeah. Very good. Because there's
18:34parallax actually. You know what? I take
18:36my answer back. Something to very good
18:39question.
18:41Would it work in panorama? Because
18:43there's no pan parallax.
18:48But I think if you have a video
18:49panoramic video then it will still work.
18:51So it depends how you implement the
18:52panorama. Yeah.
18:56Okay. Genai integrated in VR can create
18:58the matrix moving real.
19:01Maybe.
19:03Okay. Okay. Maybe. Yeah. Okay. Very
19:06good. So let's move on. Propriception.
19:09Something that I really want to cover.
19:10We still have 15 minutes. We might be
19:12okay on time.
19:14So propriception. That's something that
19:16I didn't put in the classification
19:19of senses
19:22uh because it's something different but
19:24it's considered to be a can be
19:26considered to be another sense. So in
19:28addition to information from our sensing
19:31organs memory attention we also use
19:34propriception
19:36which is so what is propriception is our
19:38ability to know where we are in space
19:41even when you're blindfold. So if I
19:44close my eyes and I put my arm around in
19:47front, I'll know exactly where my arm
19:49is, right? Even without looking. So this
19:52information is so important to our brain
19:55that there are even special neurons in
19:57our muscles and tendons and joints
20:00designated just to send signals called
20:03eference copies to the brain about what
20:06motion we just executed. So
20:09propriception can be considered uh
20:12another kind of sense. It is effectively
20:15a kind of sense because proprioception
20:18is this collection of all of these
20:19neurons that are responsible for this
20:21memory of where we just put our body. So
20:26uh propriception is extremely important
20:28in VR. If you move your hand or body and
20:31you don't see them at all or you don't
20:33see them moving appropriately
20:36like in cardboards or like with uh in in
20:39high-end HMDs, you don't have perfect
20:41bodies there, right? They don't move
20:43appropriately. So your brain then will
20:46have to analyze this confusing and
20:48contradictory stimulus coming from
20:50different senses, right? So
20:52propriceptive neurons will signal
20:54movement of your body while the dominant
20:57vision sense will say there will be no
20:59movement or different kind of movement.
21:01So how does it affect the brain right?
21:04So how does it affect comfort but also
21:06the brain our neural connections?
21:09So we know that propriception can be
21:11trained. So these neurons can be
21:13retrained and there's even special
21:16physical therapy that can help you
21:18improve your reaction rates or your
21:21balance, right? So what does it mean for
21:24VR? Completely open question, by the
21:26way. So here's the situation. Let's
21:28watch the video.
21:33It's
21:33too
21:34sweaty.
21:36[Music]
21:38So
21:40I'll have to go back. I don't want to
21:46remember that in VR these people will
21:47not see their bodies or will we'll see
21:49avatars that are not necessarily perfect
21:52bodies their height will be wrong.
22:06Oh my god. Oh my god.
22:14Oh,
22:15[Music]
22:17heat.
22:22[Music]

About this transcript

This page contains the full transcript of Bird's Eye View_ Software and Human Perception - Part 3 by NPTEL-NOC IITM, generated from the public captions YouTube serves with the video. The transcript has 2,892 words across 465 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.