VR Definition, History - Part 1 — Transcript

Full transcript

1. 0:01[Music]
2. 0:16Welcome to the intense course. So, let's
3. 0:18start uh talking uh about virtual
4. 0:21reality. This is chapter 1. Oh, let me
5. 0:24go back to my um this slide. I didn't um
6. 0:29talk about uh Steve Lavell's book. He is
7. 0:32one of our co-instructors. He wrote this
8. 0:34virtual reality book that you can access
9. 0:36free online. It's also published by
10. 0:39Cambridge Press. So, you can have a hard
11. 0:42copy if you want, but um there's a free
12. 0:44online version. We will start on chapter
13. 0:471 right now. And by the end of today's
14. 0:50lecture, hopefully we'll also take start
15. 0:53a little bit on chapter 2 as well. And
16. 0:55then next week will be over with uh
17. 0:59first two introductory chapters and
18. 1:00we'll jump into geometry virtual worlds
19. 1:04which is a more serious kind of intense
20. 1:06material. Okay. So let's go back to
21. 1:09definition of virtual reality. So it
22. 1:11seems like we'll be talking a lot about
23. 1:14virtual reality. We'll be using the
24. 1:15term. So it's probably a good idea to
25. 1:17define what it is. Um so we call virtual
26. 1:21reality experience an experience that is
27. 1:25um that induces targeted behavior in an
28. 1:29organism uh by using artificial sensory
29. 1:32stimulation while the organism has
30. 1:35little or no awareness of the
31. 1:37interference. So that's uh the
32. 1:40definition.
33. 1:43Okay. So here's an example of a system
34. 1:46and I want to ask you does it match our
35. 1:49definition or not.
36. 1:52So let's see.
37. 2:00[Music]
38. 2:10[Music]
39. 2:12So, do you hear the sound of the video?
40. 2:14By the way, just to make sure that we
41. 2:16have sound of the Yeah.
42. 2:47So the question is is this VR? So what
43. 2:50is the organism? What is the behavior
44. 2:53engineered uh stimulation to which
45. 2:55senses right? So this is called omi one
46. 2:58a very immersive VR system uh and uh
47. 3:01there's a lot of interference to
48. 3:02different senses right so there is VR
49. 3:04headset for vision headphones for sound
50. 3:07controllers for hands have haptic suit
51. 3:09vibration maybe in the for the sense of
52. 3:12touch in the controllers so a lot of
53. 3:14interference so this is clearly VR right
54. 3:16so um let's look at a different organism
55. 3:20on a different treadmill
56. 3:22and um in this case we have a mouse
57. 3:25running on a spherical treadmill also
58. 3:27exploring a virtual environment. And the
59. 3:30mouse here is suspended by its head that
60. 3:32is attached to a brain imaging device
61. 3:34right here somewhere and the scientists
62. 3:36can literally the scientists can
63. 3:39literally see uh what is going in the
64. 3:41brain. Right? So they can literally look
65. 3:44in the brain while the mouse is
66. 3:46exploring this environment and the body
67. 3:49of the mouse is tracked by this caveike
68. 3:51system. So as the mouse moves the uh
69. 3:55visual stimulation is updated
70. 3:57accordingly as if the mouse is really
71. 3:59exploring that environment. So is this
72. 4:01VR right? So we have the organism we
73. 4:04have engineered stimulation to many
74. 4:06senses of this mouse and awareness
75. 4:08something that we didn't define yet but
76. 4:12the mouse seems completely engaged in
77. 4:13this right experiment. So it's
78. 4:16participating maybe in the end of the
79. 4:18tunnel it get some kind of reward like
80. 4:20food or something like that. So is this
81. 4:22VR according to our definition.
82. 4:24Uh so this work was published in nature
83. 4:27in 2009. Uh nature is the top journal
84. 4:30for uh bi biology related uh stuff. So
85. 4:34it's uh it was very important work. Even
86. 4:37more bizarre setting with a fruitfly
87. 4:40also in a treadmill. So the treadmill is
88. 4:42very little here 2 mm in diameter and
89. 4:46it's almost frictionless suspended by
90. 4:48air. So the um so let's see what's going
91. 4:51on here in this experiment. So the
92. 4:53fruitly is also tethered by its head and
93. 4:56the brain is also exposed to a
94. 4:58microscope and what the scientists
95. 5:01observe when the uh fly moves is this
96. 5:03neural activity of single neuron in the
97. 5:06brain that is injected with the flores
98. 5:09and die. So the scientists can see the
99. 5:12neuron activity and this neuron uh
100. 5:15responses to lateral uh movement in the
101. 5:18visual field of the fly. As the fly
102. 5:21moves, it's trapped again and this
103. 5:24environment responds or being trapped
104. 5:27responds to the uh fly movements and it
105. 5:29moves and whenever there's mo uh
106. 5:31movement lateral movement then the
107. 5:34neuron activates. So this work was
108. 5:37published in 2010 in nature methods and
109. 5:40also it was popularized by wired
110. 5:43magazine
111. 5:44at some point. Uh so the only thing that
112. 5:47is left unclear about this definition is
113. 5:51awareness. So how would you define
114. 5:53awareness? What is awareness? Is it like
115. 5:56immersion? Is it presence? All of those
116. 5:59are different terms. So how would we
117. 6:01know if the organism has awareness of
118. 6:04interference or not? Right? We always
119. 6:06remember putting on the headset on our
120. 6:09eyes. Right? So what is awareness? How
121. 6:12would you define awareness? So I'll ask
122. 6:14you that question first. So
123. 6:18what is awareness in this definition of
124. 6:20virtual reality?
125. 6:25So when an organism
126. 6:27uh has little or no awareness of
127. 6:30interference to the senses
128. 6:35knowing the surroundings, right? H so
129. 6:38think about uh that human on omnione
130. 6:42treadmill.
131. 6:45Um what is that VR system according to
132. 6:49our definition? Does the human have
133. 6:51awareness of interference to senses or
134. 6:53is he is is he so immersed that kind of
135. 6:56forgets for a moment right
136. 6:59your known surroundings consciousness
137. 7:02fully focused yeah so that's so what is
138. 7:05it right so what is it so I ask you just
139. 7:08to start thinking about this but we
140. 7:10actually have definition of awareness or
141. 7:13at least we're close to defining
142. 7:15awareness
143. 7:17so um and this is how It turns out that
144. 7:20something fascinating happened lately in
145. 7:23science. Uh virtual reality became
146. 7:26extremely popular in neuroscience,
147. 7:28neurobiology and perceptual psychology.
148. 7:31Uh because the scientists in this field
149. 7:33they can use VR extensively for their
150. 7:36research and the reason is that virtual
151. 7:39reality allows to study the neural
152. 7:40processes of organisms. So just like you
153. 7:43saw in the video uh in this videos with
154. 7:46mouse and fruitfly, right? So the
155. 7:48scientists can present control stimuli
156. 7:50to the organisms and by inserting
157. 7:52electrodes in their brain or using
158. 7:54microscope or whatever imaging
159. 7:56techniques they can literally see the
160. 7:58neural activity inside of the brain
161. 8:02during the experiment while controlling
162. 8:03the organism behavior. So essentially
163. 8:06they use VR exactly according to our
164. 8:08definition except that uh the motivation
165. 8:11for this work is completely different
166. 8:13right. So the scientists in those fields
167. 8:15they want to reverse engineer the brain
168. 8:18of these organisms while in our class
169. 8:22mostly and in mainstream use of VR we
170. 8:25just want to entertain the organism
171. 8:28right so quite different purposes
172. 8:30however for the purposes of our course
173. 8:33this recent groundbreaking results from
174. 8:36neuroscience
175. 8:37can help us define awareness so how do
176. 8:40we know if the organism is being fooled
177. 8:42by the experience
178. 8:44Um and um it turns out that the
179. 8:47definition of awareness can be something
180. 8:49concrete because it can be defined by
181. 8:52the new neural structures that many of
182. 8:56um that are created during the
183. 8:58experience. So these neural structures
184. 9:00many organisms including humans have and
185. 9:04they are called place cells and grit
186. 9:05cells. So there's a special neurons in
187. 9:08our brains and in the brains of rodents
188. 9:10and mice that are responsible for new
189. 9:13environments. So let's see how it works.
190. 9:17So the following so this experiment will
191. 9:20demonstrate how this uh neural
192. 9:22structures work and how it's connected
193. 9:24to defining of awareness. So the
194. 9:27following discovery was awarded the
195. 9:29Nobel Prize in physiology or medicine in
196. 9:322014
197. 9:34and there is something called place
198. 9:37cells in our brains and in rat and minds
199. 9:40where a particular neuron in the brain
200. 9:43fires only when you are in a particular
201. 9:47place in the environment. So let's see
202. 9:49how it works on this video. So here a
203. 9:53rat in a labyrinth.
204. 9:56I hope you hear the video of the sound,
205. 9:57the sound of the video. This clicks,
206. 10:01that's when the neuron fires, the play
207. 10:03cell uh fires. Um, so a rat here is in a
208. 10:08labyrinth and it's attached to a single
209. 10:12unit recording device. So that means
210. 10:15it's an electrode that's inserted in the
211. 10:18brain of the mouse. And um that device
212. 10:23measures the firing of a single neuron
213. 10:26only. So these left boxes on this on the
214. 10:28right on the left. So you can see that
215. 10:31that particular neuron different color
216. 10:34coded fires only when rat the rat is
217. 10:38roughly in the same place in the
218. 10:39environment. So this kind of experiments
219. 10:41were repeated multiple times on various
220. 10:44organisms and on various environments
221. 10:47including virtual. So then this place
222. 10:50cells is what could be the definition of
223. 10:53being fooled be right. So if your brain
224. 10:56decides that the space is new and real
225. 10:59enough then it designates a special
226. 11:02neuron the play cell for that place. So
227. 11:04if there are no new play cells that
228. 11:07means the experience wasn't immersive or
229. 11:09realistic or like real didn't behave
230. 11:12like expected in the real world. Right?
231. 11:17So um so that's how we could define
232. 11:20virtual reality uh awareness in virtual
233. 11:23reality. Um so there's also um grid
234. 11:27cells that are um those are special
235. 11:30neurons that are designated to grit
236. 11:32spaces in the in the uh in the
237. 11:36environments. So also awarded Nobel
238. 11:39Prize for the discovery. Let's see what
239. 11:43does it mean right. Um, so here are a
240. 11:47couple of warnings for VR developers.
241. 11:50When neuroscientists work with mice or
242. 11:53rats or other low form organisms,
243. 11:56there's a clear distinction between who
244. 11:59is the scientist and who is the lab rat.
245. 12:01And this is quite different uh from the
246. 12:04setup that we have in engineering,
247. 12:06right? So when a VR developer uh creates
248. 12:10a virtual reality experience for humans,
249. 12:12the developer is both the scientist and
250. 12:14the lab rat. Right? So think about it.
251. 12:16This is quite dangerous because
252. 12:19the creator is trying to fool himself
253. 12:23while developing the creation. Right? So
254. 12:26this can result in what's called
255. 12:28developer bias. So for example, if
256. 12:31you're developing your final project,
257. 12:33you may be just simply biased by the
258. 12:36desire for the word for the project to
259. 12:39succeed, especially if it's right before
260. 12:41the deadline. Right? So another um
261. 12:44another bias is that developers usually
262. 12:47know what the experiment is trying to
263. 12:48accomplish while fresh unaware subjects
264. 12:51may not have that information and have a
265. 12:54completely different experience or
266. 12:56impression from the experience. So for
267. 12:58example, try scaring yourself or
268. 13:00tickling yourself or surprising
269. 13:02yourself. Right? So so those kind of
270. 13:05biases is something that we need to be
271. 13:07aware of when we develop VR experiences.
272. 13:11So so just to rephrase the creator might
273. 13:14adapt to the flaws and even
274. 13:16unconsciously avoid the flaws of the
275. 13:19system. Um so the flaws might
276. 13:22effectively become invisible to the
277. 13:24creator. So the conclusion is that it is
278. 13:27very important to test VR experiences on
279. 13:29other people who didn't try it before or
280. 13:31maybe even on people who have never been
281. 13:33exposed to VR at all. So they or even
282. 13:37those who never played first person
283. 13:38shooter game or those who do get motion
284. 13:42sick because motion sickness is related
285. 13:45to uh simulator sickness that we have in
286. 13:47virtual reality.
287. 13:49So if your experiences make them
288. 13:51uncomfortable like your grandmother may
289. 13:54feel sick for the rest of the day after
290. 13:56you're trying your project that's not
291. 13:57good right so you better test your
292. 13:59experiences on this kind of subjects
293. 14:03and in this course we'll start
294. 14:04addressing the issues of comfort right
295. 14:06away so in your homework number two you
296. 14:09will study first you will first study
297. 14:12the Oculus VR best practices guide uh I
298. 14:15think it's uh now called meta it's
299. 14:17called something else but It's the
300. 14:18official meta guide to developers about
301. 14:21how to create best possible VR
302. 14:23experiences and um and we'll study that
303. 14:26in homework 2. So you can start on it
304. 14:28right now. Read through the best
305. 14:29practices guide and ask answer our
306. 14:32questions in homework too.
307. 14:36Okay.
308. 14:37Okay. I hope my voice is better now.
309. 14:44Yeah. So is the user we are aware of
310. 14:47this equipment and yet we interact with
311. 14:49this virtual world. So if we could test
312. 14:52if there are new neural cells neural uh
313. 14:54structures um that appeared after this
314. 14:58experience then um then that means the
315. 15:02virtual reality
316. 15:03uh experience behaved similarly to how a
317. 15:06real world would behave or at least the
318. 15:09brain reaction to it was similar how it
319. 15:11would be in the real world. So according
320. 15:12to our definition it's virtual reality.
321. 15:19Uh so there's different colors there's a
322. 15:21different neurons each color is one
323. 15:24neuron responsible for one place.
324. 15:27So as we explore new environments for
325. 15:31each new place we'll have a neuron or a
326. 15:34new neural structure.
327. 15:36Those are place cells which is really
328. 15:38surprising discovery. That's why it um
329. 15:41earned a Nobel Prize.
330. 15:45So are place cells unique cells
331. 15:47activated for some situation or the
332. 15:49particular state of any no they are new
333. 15:52neurons. So they are new cells
334. 15:55designated for that place in the world.
335. 15:59So as you explore more more and more of
336. 16:01neurons will be designated to new places
337. 16:03in the world. Is the mouse awareness
338. 16:06through the neurons comparable to that
339. 16:07of humans? We have similar structures uh
340. 16:10as uh mice and uh durables and many
341. 16:13other organisms. So uh many of these
342. 16:16organisms including humans have place
343. 16:18cells
344. 16:20place cells. So they are responsible for
345. 16:23places new places.
346. 16:26Okay.
347. 16:28Uh so is consciousness and awareness are
348. 16:30related? uh no because awareness can be
349. 16:35can be linked to this place cells while
350. 16:37consciousness is not a really well-
351. 16:39definfined concept. So consciousness
352. 16:41there's no definition of it and uh
353. 16:43scientists still struggle with what is
354. 16:45it really. So this is something still
355. 16:47metaphysical while awareness can be now
356. 16:50very physical due to this uh neural
357. 16:52structures.
358. 16:54How do we assert in the level of
359. 16:56stimulus in humans? How do we know the
360. 16:58neurons are getting activated like this
361. 16:59in the case of rats? Um so I am not uh
362. 17:03sure 100% is it still a theory or has it
363. 17:06been demonstrated for sure. I think it
364. 17:09has been demonstrated for sure but I
365. 17:10cannot give you reference right now. Um
366. 17:13however there are many humans that would
367. 17:16dedicate or that would donate their
368. 17:18brains to signs like prisoners or people
369. 17:21who are um gravely ill. And therefore
370. 17:25it's possible to conduct similar
371. 17:26experiments that you've seen on rats uh
372. 17:29on humans. And I think scientists
373. 17:32believe that yes we do have the same
374. 17:34neural structures right now but I don't
375. 17:36remember exactly like how and why
376. 17:39can it uh we have different experience
377. 17:42for all. Sorry I don't understand this
378. 17:44question.
379. 17:46Motion sickness related to V experiences
380. 17:49yes very good question. So let's go
381. 17:50further and we'll try to answer more
382. 17:52questions later.
383. 17:56Yes. Yes. So this is if you experience
384. 17:58something new and it's real enough like
385. 18:00it was being the real world then if I
386. 18:02have new self brain cell or designating
387. 18:06your new brain cell to that experience
388. 18:08would would um define awareness here. So
389. 18:12it would say okay the human was unaware
390. 18:14that there was interference because this
391. 18:16the brain reacted in the way as it would
392. 18:19react to the real world stimulation.
393. 18:22Okay.
394. 18:25[Music]