YT YouTube2Text

YouTube2Text → Transcripts → Bird's Eye View_ Hardware - Part 3

Bird's Eye View_ Hardware - Part 3 — Transcript

by NPTEL-NOC IITM · 2,989 words · 477 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:04[music]
0:16So, inertial measurement unit is correct
0:18answer
0:20if you know a little bit of mus but I'll
0:22cover it right now. So, don't worry if
0:24you don't know yet. But what is not
0:26usually a part of an MU? Yeah, correct.
0:28Cameras are not part of EMU. Usually the
0:30standard part of IMU is accelerometer,
0:32gyroscope and magnetometer. So let's go
0:34through them. So let's talk about the
0:37components and one of the most useful
0:39components is IMU which is inertial
0:42measurement unit. So IMUs have been
0:44around for a long time. They had a lot
0:46time a lot of time to evolve from these
0:49large devices to something small like
0:52this like on this picture just like
0:54screens evolved from large um tube TVs
0:59to phone screens that you carry in your
1:01pockets. You know the similar evolution
1:04happened to IMUs. So the purpose of IMU
1:07is to provide orientation only three
1:10degrees of freedom tracking and it often
1:14the the board IMU board consists of
1:16gyroscope, accelerometer and
1:18magnetometer. So here's a picture.
1:20They're pretty small. Um I forget which
1:24one of them is which but it doesn't
1:25matter. These three uh boxes here are uh
1:28the the the accelerometer, gyroscope and
1:31magnetometer.
1:33So, IMUs were designed for navigation
1:35originally. So, especially in aircraft,
1:37spacecrafts, military, torpedoes,
1:39rockets.
1:42So, if you flew, if you ever flew in an
1:44airplane, there was an enemy on board
1:46for sure. And it would be a large
1:48mechanical device because they have to
1:50be accurate. So, they're very expensive.
1:53They're really accurate. They're very
1:54well calibrated. However, other devices
1:58uh on the market go from the cost of
2:00just a dollar
2:03uh on on you can buy it on Amazon or
2:07wherever
2:08up to thousands of dollars or tens of
2:10thousands of dollars depending on their
2:12calibration. So this cheap um cheap IMUs
2:18um they um they appeared with the
2:22advances of the MEMS technology. So
2:24that's why we have to be grateful for
2:26this technology because it made it
2:28possible to have such small IMUs
2:30attached to inside of the smartphones
2:33but also attached to the headsets.
2:37Okay. So let's see
2:41here's a mechanical
2:43uh gyroscope. Let's think how they work
2:45so that we can see how we can use them
2:47for tracking.
2:50So this is an IMU.
2:53Oh, sorry. [music] This is a mechanical.
2:54This is a gyroscope. It consists of a
2:57metal wheel mounted on an axle. The
2:59wheel and axle spin freely secured in a
3:01metal frame. This simple device can
3:04behave in the most unexpected ways.
3:08The components of this simple mechanism
3:10are obvious and it doesn't seem to have
3:12any special capabilities.
3:15Try to stand it on end and it falls
3:17over.
3:20Try to suspend it like this and it
3:22drops.
3:24Apparently, it can't defy gravity.
3:28But spin the wheel and all that changes.
3:32A string provides a simple method to
3:34spin the wheel.
3:43The first thing I notice about the
3:45spinning gyroscope is that it resists
3:47attempts to change its position. A
3:50resistive force appears when I attempt
3:52to tilt or rotate the gyro.
3:55Now, when I stand it on the tabletop, it
3:57stays vertical. The spinning wheel is
4:00creating a force that holds the
4:01gyroscope upright.
4:04When the wheel stops, the force
4:06disappears and the gyroscope falls.
4:15So that's
4:15incredibly this force will support the
4:17gyroscope like this.
4:20It appears to be levitating.
4:24So now uh how they used in airplanes is
4:28right here.
4:30The gyroscopes will maintain a
4:32particular orientation in space is very
4:34useful in modern aircraft. An inertial
4:38guidance system uses spinning gyroscopes
4:40to monitor and control the orientation
4:42of the aircraft.
4:45The gyroscope is suspended in a special
4:47cage that allows it to maintain its
4:49orientation independent of the
4:51aircraft's position. [music]
4:53If the aircraft rolls, electric sensors
4:56and contacts connected to the gyro send
4:58information to the pilot about the
5:00aircraft's orientation.
5:13Okay. [music] So that's the principle,
5:15right? And here I have another video of
5:18a M's gyroscope and how it works. So the
5:20principle is very similar. Microructure
5:23of the gyroscope looks something like
5:25this. a mass that is constantly moving
5:28or oscillating and when an external
5:31angular rate will be applied a flexible
5:33part of the mass would move and make the
5:35perpendicular displacement.
5:38Okay. So what do gyroscopes measure?
5:43Uh gyroscopes measure 3D angular
5:46velocity in radians per second. So what
5:52can these measurements provide with
5:53respect to tracking? So if you know an
5:56instantaneous angular velocity, so how
5:59fast um the speed at which an object
6:02rotates, then you can integrate it from
6:05the last orientation. So say you're at
6:07orientation theta 1, you just multiply
6:09that instant velocity by the time over
6:12which you want to integrate to get the
6:14nest orientation. So this is process is
6:17called integration, right?
6:20And if you do this several times over
6:22and over, you can obtain very reliable
6:24estimations for a couple of seconds.
6:28If you integrate over very small
6:30periods, tiny periods of time for that
6:32instantaneous velocity.
6:34But then after this period of
6:36integration, drift happens. So drift
6:39happens due to noise, calibration
6:42problems. these cheap devices are not
6:44necessarily perfectly calibrated and
6:46accuracy with which you do the
6:47computations in the on the PC. So drift
6:52is when these little arrows after each
6:54iteration they add up to a larger area a
6:58larger error after many iterations. So
7:01if drift happens then you may lose
7:07you you you may not know anymore where
7:10straight ahead is or where straight up
7:12is. So the horizon may feel like it's
7:14tilted or even worse sometimes when you
7:18have this drift in your tracking system
7:20you may feel like your head is then
7:22spinning a roller coaster out of
7:23control. [snorts] So drift may feel
7:27extremely uncomfort uncomfortable and it
7:29should a be avoided at all costs.
7:33Fortunately there are other sensors to
7:36correct for the drift and that's where
7:37accelerometer and magnetometer will help
7:39us and cameras if you have cameras. But
7:42remember in our cardboards we don't have
7:45an external camera or any camera that we
7:47can use for correcting for this drift.
7:49So only accelerometer magnetometer can
7:52be used. However, what you what I want
7:55you to think of a gyro is that it's a
7:57sensor that is does the most heavy
8:00lifting out of all the sensors. So, it's
8:03used most frequently in uh tracking
8:06systems. And the reason is that it
8:07provides pretty accurate measurements
8:09for at up to a thousand so usually
8:12around,000 hertz and you can do this
8:15integration every time you get a
8:17measurement at such high frequency. Then
8:20you have to correct for track uh for
8:22drift. But you do this measurements all
8:24all the time. We don't use
8:26accelerometers
8:28as much or as reliably or magnetometers
8:31or cameras. For example, cameras they
8:33provide um measure pictures at a
8:37frequency of 60 Hz much slower than um
8:42gyroscopes. So gyroscopes are extremely
8:44useful but we have this drift problem.
8:46So let's see how we can avoid that.
8:49So there let's look at the another part
8:52of IMU which is accelerometer.
8:54So when you think of an accelerometer
8:56think of a weight on the string like
8:59this. So a mechanical accelerometer
9:03measures this deviation from this mass
9:06from this weight from the normal
9:08position
9:10uh where where it would be in a relaxed
9:12state. So if this acting force in in
9:15this direction then the spring contracts
9:17and if it's in the opposite direction
9:19the spring will expand right and this
9:21deviation will be proportional to the
9:23force and therefore acceleration
9:27okay so digital acceler accelerometers
9:30are not much different from mechanical
9:32accelerometers let's look at the video
9:34modern MEMS's chip let's take a look
9:36under the microscope
9:38this is a tiny resistor the lighter
9:41colored material is actually
9:43electrically conductive silicon. And
9:45this darker area that's been etched away
9:47doesn't conduct. This long winding
9:49electrical path forms a resistor very
9:52similar to how a long piece of wire
9:54would also have a significant
9:55resistance. So if you made an electrical
9:58connection between these two points,
9:59you'd have a microscopic resistor. Now,
10:02in order to understand how an
10:04accelerometer works, let's look at a
10:05MEMS's capacitor. It doesn't look much
10:08like a capacitor, does it? Well,
10:10remember that all a capacitor really is
10:12is two conductive plates that are
10:14electrically separated. Here are the two
10:17terminals of the capacitor. Over here,
10:19we have what's called a combed finger
10:21arrangement. The two structures are very
10:24close to each other, but they aren't
10:26quite touching. Let me highlight it for
10:28you. Now, it should be more obvious that
10:30you have parallel surfaces which form a
10:32capacitor. But this is no ordinary
10:35capacitor. It's a physical structure
10:37that can move. This thing over here is
10:39basically a tiny weight made out of
10:41silicon. And it's kind of like a
10:43suspended mass on the end of a spring.
10:45Movement, vibrations, and even gravity
10:47can cause this little mass to move
10:49around. And when it does, it shifts the
10:52entire combed finger structure. When the
10:54fingers move, the distance between the
10:56fingers changes. And when the distance
10:58between the fingers changes, you get a
11:00change in capacitance. So now we have an
11:03electromechanical system that can sense
11:05movement and turn it into a changing
11:07capacitance value. The next step would
11:10be to design circuitry that can sense
11:12the change in capacitance and convert it
11:14into useful voltages or serial data. But
11:17that's beyond the scope of this
11:18tutorial. A modern Okay. So,
11:24so now how would we use these
11:27accelerometers or these micro springs to
11:31measure the acceleration of an object
11:33due to its motion. So then we could um
11:37we could estimate position using the
11:40accelerometer position of the headset
11:42and therefore do positional tracking and
11:44then we would do positional tracking
11:46with accelerometer and orientation
11:47tracking with gyroscope.
11:50But that's impossible. So here are two
11:53problems with that. So one is that the
11:55accelerometer measures the vector sum of
11:58all accelerations including due to
12:01gravity. So it's very difficult to
12:04separate true linear acceleration of
12:06motion with respect to fixed earth from
12:10gravity away.
12:12So let's think about it. Imagine I have
12:14this mass on the spine, right? And let
12:17me swing it like this, right?
12:20the the the uh the curve that the mass
12:23would uh trace would be quite different
12:26if I spin it in different orientations
12:28because there's also gravity. So, it's
12:30not only the motion, it's not only the
12:32centrial forces, but it's also the
12:34gravity that pulls the object down at
12:37all times.
12:39Therefore,
12:41it's hard it's hard to separate them. At
12:43certain
12:45um times when there is not much motion,
12:48you would be able to separate the
12:49gravity from true acceleration due to
12:52motion. And those are very useful times.
12:55And that's exactly how the accelerometer
12:57is used for drift correction for the
13:00gyroscope.
13:02At those times when we can really grab
13:04the gravity away from motion, we can use
13:07that gravity vector to uh correct for
13:10the drift from the gyroscope and that
13:13way we could reconstruct where true up
13:16is because that's opposite from gravity.
13:18So that's how accelerometer is used.
13:22But there's another problem due to
13:25double integration. So even if we are
13:27able to separate at certain times
13:29gravity from true acceleration. Now if
13:32you take that true acceleration due to
13:33motion we would have to double integrate
13:36it to get the position double
13:38integration
13:40in has drift that is higher much higher
13:43than single integration of the gyro
13:46because it grows quadratically. So we
13:48can only integrate it double integrate
13:51it for a very very very few um amount of
13:55times and after that we would need
13:57another sensor to correct for that drift
14:00but we don't have any others we don't
14:02have anything else we would need a
14:05camera to correct for the drift from
14:07double integration of accelerometer and
14:09we don't have it in cardboards so once
14:11you introduce camera then you can do
14:13positional tracking however if you don't
14:16have another sensor answer for this
14:18drift correction due to double
14:20integration. Then you can only have
14:23reliable orientation orientation
14:25tracking because you can use a
14:28solarometer then to correct for the
14:30drift of where up is and you can use
14:33magnetometer
14:34for it's kind of you can extract this um
14:38horizontal uh axis of the magnetometer
14:41from the from the measurements that
14:43magnetometer gives you to correct for
14:45where straight ahead is. So that's
14:48that's two ways you could use
14:50magnetometer accelerometer to correct
14:53for drift accumulated from gyro
14:55integration and have a reliable 3° of
14:58freedom uh tracking system.
15:01Recent HMDs introduced by Meta and Apple
15:03are not much successful. No, they're
15:05not.
15:07[laughter]
15:08Well, we're waiting for 2027. That's
15:10what they say that they will be really
15:12cool this glasses kind of AR devices
15:15that we'll see that use this advanced
15:18optics and advanced displays that um
15:22hopefully will be as cheap as what we
15:24have now and um as good. I don't know. I
15:28I really cannot cannot comment on this
15:31because we don't know
15:34uh Flickr. Okay. So let me I have just a
15:38couple more slides and hopefully we can
15:39finish with that today. So cameras
15:41cameras are used for let's see
15:46for positional tracking because then
15:48they can be used for um drift correction
15:53from accelerometers. Why they can't be
15:56used in every frame for positional
15:58tracking? Because they're very slow. So
16:00they they are they give you image every
16:0216 milliseconds. So that's not good
16:04enough. uh that's quite a bit of a
16:06delay. So you cannot just use camera for
16:08positional tracking. You need to use
16:10accelerometer that is much more frequent
16:12measurements and then correct for drift
16:14every 60 milliseconds or with frequency
16:17of 60 Hz from using camera readings. So
16:22um that's what they used for and there's
16:25outside in tracking that was uh done in
16:28early versions of Oculus uh HMDs when
16:32you had an external camera somewhere on
16:35the desk and you had this infrared uh
16:38LEDs on your uh display that were easy
16:41to track. But now recently all of these
16:45um Oculus Quest all of the modern HMDs
16:48um they have this cameras on the headset
16:50itself and then they do natural features
16:53tracking from of the world around. So
16:56it's much more difficult computer vision
16:58problem but it's done successfully and
17:00we have quite good positional tracking
17:03degrees of freedom tracking on the
17:05devices that have cameras. But again
17:08remember that with cardboards we do not
17:10have cameras.
17:12So I have a question for you from the
17:15point of view of positional tracking
17:18in Quest 2 that do have cameras. What
17:21happens if there is no light for a
17:22camera? So if the lights are turned off
17:25in the room and some of you have Quest
17:27two so you may know what happens. So
17:30what happens if you turn off the light?
17:33It will still work. But does it do
17:36positional tracking?
17:39Okay. Then no infrared light. [laughter]
17:42What if there's no infrared lights in
17:43the room as well? So all lights are off,
17:46even infrared.
17:48Yeah, it it will limit to three degrees
17:50of freedom. So usually what I say, thank
17:52you. So some of you said it would fall
17:54back to 3° of freedom. So yeah, that's
17:56correct. So the thing is I don't know
17:58how particular applications or Quest 2
18:01implements it particularly now because
18:03there may be some updates um software
18:06updates and maybe it will change it
18:08different differently from uh update to
18:10update but I know what should be done.
18:13So there is always two choices either to
18:15black out and say okay camera doesn't
18:18detect any features so please turn on
18:20the lights so that there's nothing or
18:23another choice is to fall back to 3D
18:25degrees of freedom tracking just using
18:27IMUs and then you know integrate your
18:30gyro correct for up using accelerometer
18:32correct for straight ahead using
18:34magnetometer you have 3° of freedom
18:36tracking just like cardboard and that's
18:38okay and whatever meta does now is uh is
18:42one of these choice uses, but I don't
18:44know what it is. Now, what happens
18:48with positional and rotational tracking
18:51if you use your Quest 2 in a moving car
18:53or a flying airplane? Do you know? So,
18:56that's another question we have.
18:59We always have fun with this question
19:01because some students actually know they
19:02tried and some nothing and no something
19:06happens. Something happens.
19:10It's stable.
19:13No,
19:16I'm use work. Yeah, but what about
19:18cameras? So, the thing is since you're
19:21flying then your accelerometer will
19:24detect that and the integration of
19:26accelerometer will actually
19:31they'll it'll know that you're flying.
19:33So, it shouldn't be it shouldn't be um
19:36working in the airplanes because you'll
19:38be actually flying in your virtual
19:40reality experience unless it it unless
19:42it has been fixed somehow which I think
19:45is really difficult to fix because there
19:46is a standard tracking techniques.
19:49So, any of you tried it
19:53in an airplane? Because I had students
19:55who really tried it in a car
19:58and tried it in airplanes.
20:05Oh, maybe if you set if you if you
20:07disable positional tracking then maybe
20:10it would it would have no change but
20:13it's supposed to not work in the car but
20:16try it again and we can discuss it next
20:18time. Um okay
20:22now what happens in a if you use VR
20:24cardboard in a moving car flying
20:27airplane. So this is pos rotational
20:30tracking only and yes it will not be
20:34affected. This is no change correct. So
20:36this is correct for uh cardboards but
20:39for quest two
20:42unless something has been changed
20:45um it shouldn't work in when you're
20:47flying or driving in a car.
20:52Okay.

About this transcript

This page contains the full transcript of Bird's Eye View_ Hardware - Part 3 by NPTEL-NOC IITM, generated from the public captions YouTube serves with the video. The transcript has 2,989 words across 477 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.