Bird's Eye View_ Software and Human Perception - Part 4 — Transcript

Full transcript

1. 0:01[Music]
2. 0:16So um so in this video right you saw
3. 0:20that all everything was almost realistic
4. 0:23but there was still a lot of mismatch.
5. 0:25There are other mismatches that we'll
6. 0:26learn about uh in VR but also the
7. 0:29mismatch between your propriception and
8. 0:31what you see your body is where it is in
9. 0:34space. So some researchers actually
10. 0:37think that after using VR we might
11. 0:40change the structure of our brains
12. 0:42especially related to this
13. 0:44propriception.
14. 0:45So some say we'll improve propriception
15. 0:48and they're quite optimistic like it's
16. 0:50going to be a positive change. we can
17. 0:52increase reaction rates or improve
18. 0:53memory or something. Um,
19. 0:57but it's not 100% right. So, so let's be
20. 1:00optimistic. But please watch this video
21. 1:02if you want to learn more about
22. 1:04propreception related um research.
23. 1:09So that's a big question. Will VR
24. 1:11improve propriception or change it? I
25. 1:14want to say damage it but you know.
26. 1:17Okay. So let's move on. Um now what
27. 1:21let's talk about other sensory
28. 1:22conflicts. So modern virtual reality
29. 1:24devices such as Oculus Quest and even
30. 1:27more so the cardboards that we have in
31. 1:29class they interfere with some senses
32. 1:32but not the others. Right? We completely
33. 1:34ignore other senses.
34. 1:36Uh so and this causes mismatch between
35. 1:39multiple senses and this contradictory
36. 1:42information needs to be analyzed by the
37. 1:44brain and the brain may react in a
38. 1:47variety of ways. So it could be the case
39. 1:50that you will not be consciously aware
40. 1:52of the conflict but then you may feel
41. 1:55fatigue right or develop a headache or
42. 1:57even worse have dizziness and c and
43. 2:00nausea right or like really or feel
44. 2:03tired for for days after the experience.
45. 2:06So that's not good. In other cases the
46. 2:09brain might react by making us so
47. 2:11consciously aware of the conflict that
48. 2:13we immediately understand that the
49. 2:15experience is not real artificial. So
50. 2:17this would result in loss of immers
51. 2:20immersiveness in uh immersion in VR
52. 2:23experiences.
53. 2:25So uh to therefore to make a comfortable
54. 2:29and effective VR experience, how would
55. 2:31we do that? You need to do trials with
56. 2:34human subjects and it's essential to
57. 2:36understanding how the brains will react
58. 2:38across population. So particularly if
59. 2:42you develop your VR experiences for the
60. 2:44class projects. So always ask other
61. 2:47people to try it before releasing it,
62. 2:49especially people who didn't do VR at
63. 2:51all. So you really don't want the last
64. 2:54thing you want to do is to make people
65. 2:56sick with your uh final projects, right?
66. 2:59So one of the most important examples of
67. 3:02bad sensory conflict in in VR is
68. 3:06vection. So vection is the illusion of
69. 3:09self motion. And this conflict arises
70. 3:12when your vision sense reports that uh
71. 3:15you're moving, you're accelerating, but
72. 3:17your balance sense say no I'm
73. 3:19motionless. So that's happens when you
74. 3:21move your avatar in VR with a
75. 3:23controller. So your body remains still,
76. 3:26but visually you feel like you're
77. 3:28accelerating. So how to eliminate the
78. 3:31nausea? That is often related or
79. 3:35vaction often leads to nausea. So how to
80. 3:37eliminate this uh nausea is an ongoing
81. 3:41research in VR. So here's an example of
82. 3:44vection not in VR but in real world. Uh
83. 3:47so vection is an illusion of motion.
84. 3:49These people feel like they're moving
85. 3:51but they're not. So this bench is
86. 3:54stationary. It's the walls that are
87. 3:56moving. So this is a very unusual
88. 3:59scenario, right? There are some
89. 4:00amusement parts like that. So we're not
90. 4:03used to seeing room spinning around us,
91. 4:05right? So this is uh then the people
92. 4:08these people feel illusion of self
93. 4:10motion and many of them get sick because
94. 4:12of that. So because of this large scale
95. 4:15motion happens.
96. 4:18Okay.
97. 4:20So calibration. Yeah, I saw request
98. 4:22calibration. Let's see if they
99. 4:25Oh, yeah, they replaced it. This would
100. 4:28be cool. How is paral error corrected in
101. 4:31Cardboard VR? Oh, there is no parallax
102. 4:34in Cardboard VR uh for panoramas,
103. 4:38but there is parallax with rotation
104. 4:42only. tracking in your cardboard
105. 4:44experiences. So, it's very important to
106. 4:46uh differentiate between panoramas and
107. 4:50unity based or 3D model experiences in
108. 4:53in cardboards. They're quite different.
109. 4:56There's always three degrees of freedom
110. 4:57tracking, but there is parallels in one
111. 5:00and no parallels in another. I think
112. 5:02we'll have a lecture to just look into
113. 5:03it in more details. But now in homework
114. 5:062, you can you can experience them
115. 5:08differently. So, if there's panoramas,
116. 5:10then you're right. There is no parallax.
117. 5:13But um like shadows of truth see if when
118. 5:16you rotate there is there's some at
119. 5:19least there's binocular disparities
120. 5:21right so there is some kind of
121. 5:22differences between due to the distance
122. 5:25between the eyes view but yeah there's
123. 5:28uh only paral due to that we'll we'll
124. 5:31talk maybe in more details the math
125. 5:33related to that so they they're
126. 5:35different experiences in VR but
127. 5:38um that's an open question So
128. 5:44consciousness I don't know conscious
129. 5:46nobody knows the answer unity is free
130. 5:48for students okay so maybe need ideas to
131. 5:52implement locomotions in cardboard very
132. 5:54good question so if you want to do that
133. 5:57jump into homework 4 we give you a good
134. 5:59idea I think
135. 6:02okay so let me go back to our slides
136. 6:06another conflict that is not between
137. 6:08senses which is not uncomfortable
138. 6:10But really, really interesting.
139. 6:16Have a look at this.
140. 6:18What do you hear?
141. 6:21Ba ba
142. 6:24ba ba ba?
143. 6:27But look what happens when we change the
144. 6:30picture.
145. 6:32Ba ba
146. 6:35ba ba ba
147. 6:38ba. And yet the sound hasn't changed. In
148. 6:43every clip, you are only ever hearing
149. 6:45bar with a B.
150. 6:51It's an illusion known as the Mgherk
151. 6:54effect.
152. 6:55Take another look.
153. 6:58Concentrate first on the right of the
154. 7:00screen.
155. 7:02Now to the left of the screen.
156. 7:06The illusion occurs because what you are
157. 7:08seeing clashes with what you are hearing
158. 7:12in the illusion um what we see overrides
159. 7:15the mgherk effect that's the name of
160. 7:19this illusion it's not uncomfortable so
161. 7:21the brain is totally fine with this
162. 7:22contradiction between audio and visual
163. 7:25information but there's a conflict and
164. 7:27it will have to pick vision because we
165. 7:29are largely visual animals we have more
166. 7:32neurons devoted to that so we trust
167. 7:34vision more than hearing.
168. 7:37Okay. So, another very bad problem
169. 7:42is sensory adaptation. It's it's not a
170. 7:44problem. It's great for us. We adapt to
171. 7:46all of these uh unnecessary sensory
172. 7:50stimulus that the brain will just ignore
173. 7:52after a while. So, this rel this comes
174. 7:56to this is true about different senses
175. 7:59like loudness. We adapt to that. after a
176. 8:02while we live in the loud city we stop
177. 8:04noticing the noise right or in the
178. 8:06airplane you don't notice after a couple
179. 8:08of minutes uh in case of vision I don't
180. 8:11show vision but we adapt to darkness we
181. 8:13adapt to lightness it takes up to like
182. 8:16sometimes seconds to minutes smells
183. 8:19temperature we adapt to those right uh
184. 8:22the touch sense of touch so this this
185. 8:25clothes and uh jewelry that we put on in
186. 8:28the morning we don't notice them right
187. 8:29the brain after a while it just ignores
188. 8:32the sensory stimulation. So maybe in the
189. 8:34end of the day uh you get tired, you
190. 8:36feel tired and you want to take them off
191. 8:37because the brain was doing a lot of
192. 8:39work ignoring it. Nevertheless, that's
193. 8:42what's called adaptation.
194. 8:44So I want to ask you if we have this
195. 8:47adaptation related to many senses and
196. 8:49many stimuli um what would be one
197. 8:52example that is relevant to v virtual
198. 8:55reality? Adaptation to what? Can you
199. 8:58guess?
200. 9:05Yeah, roller coasters, right?
201. 9:08Adaptation to low resolution. Yeah,
202. 9:11after a while probably like, "Oh, this
203. 9:12is the greatest thing."
204. 9:15A lot of things, right? So, very good.
205. 9:17The weight of the headset. Yeah, I guess
206. 9:19you It's really heavy though. I cannot
207. 9:22adapt to that to be honest, but maybe if
208. 9:24you're really engaged in some
209. 9:26environments, online classes. Yeah. So
210. 9:28what
211. 9:31so um
212. 9:34what does it mean for VR? Yeah, those
213. 9:36are good examples. But I think I was
214. 9:38asking I was kind of hoping to get
215. 9:40vection. Vection is an adaptation. So
216. 9:42let's let's talk about it. So after long
217. 9:46periods of time, the sensory mismatches
218. 9:50that we have in virtual reality,
219. 9:51including what leads to vections,
220. 9:54illusion of motion. So the sensory
221. 9:56mismatches may lead to adaptation as
222. 9:58well. Uh so think about it like
223. 10:01astronauts they train themselves to
224. 10:04through this regular exposure to get
225. 10:06adapted to high G forces. Similarly you
226. 10:09guys those who spend hours or days
227. 10:12playing video games or those people who
228. 10:15do car racing. So they develop
229. 10:17adaptation to vetin and vetin is the
230. 10:20biggest what theory is the biggest cause
231. 10:23of uh nausea and discomfort. So then
232. 10:27people who are adapted through video
233. 10:29games they experience less fction in VR
234. 10:32too. So this becomes really critical for
235. 10:36VR developers because you're constantly
236. 10:38exposed to this VR while developing the
237. 10:41environments. But and if you even test
238. 10:44your um your experiences on your game of
239. 10:47friends, they mo may also report that
240. 10:50your experiences are comfortable. But
241. 10:52you really really need to test your
242. 10:54experiences on people who are newcomers
243. 10:56to virtual reality, but also maybe
244. 10:59newcomers to video games and or racing
245. 11:02cars.
246. 11:03So um however, it's not so bad.
247. 11:06Adaptation is not so bad. Even though
248. 11:07you're adapted uh you can through series
249. 11:11of exercises you may also in VR develop
250. 11:14perceptual training. So then you can
251. 11:16detect all these things like latency can
252. 11:18be detected through specific exercises
253. 11:21also. Um
254. 11:24curvature or like distortion parameters
255. 11:26they can you can improve your perception
256. 11:28of distortion and improve your ability
257. 11:31to set up parameters by specific
258. 11:33exercises and exposure. So that's all
259. 11:36good.
260. 11:38I have a question for you and I have to
261. 11:40I think I have to finish because we have
262. 11:43still 15 slides. I think that would be
263. 11:47too long to take
264. 11:50take um but we'll we'll we'll see this.
265. 11:53How long would it take to adapt to the
266. 11:55upside world? Uh so we'll we'll start
267. 11:59next time with this. Imagine you have
268. 12:01upside down goggles and everything seems
269. 12:03upside down. Would you be able to adapt
270. 12:05to that? Would it be comfortable? Maybe
271. 12:08not at first, but if if you can adapt,
272. 12:11how long?
273. 12:13So, the correct answer is at least on
274. 12:16one experiment that I was going to show
275. 12:18is 7 days. And we'll see that experiment
276. 12:21starting next time.
277. 12:24[Music]