[CS61C FA20] Lecture 37.1 - Data Centers, Cloud Computing (WSC): Eras of Computing Hardware — Transcript

Full transcript

1. 0:00and welcome back this is our final
2. 0:03lecture on the parallelism topic and
3. 0:05we're going to be talking about data
4. 0:06centers and cloud computing
5. 0:08how do you actually run that mapreduce
6. 0:10and spark job
7. 0:12well on a data center or a cloud
8. 0:13computing model so let's think about
9. 0:15that we'll first start with a little
10. 0:16history
11. 0:17the eras of computer hardware so
12. 0:20let's first start uh with the review
13. 0:22where are we
14. 0:23six great ideas of computer architecture
15. 0:25we've hit the layers of representation
16. 0:27of
17. 0:28interpretation see abstract is an
18. 0:30abstraction above an assembly level
19. 0:32which
20. 0:32there's abstraction above machine code
21. 0:34which is an abstraction above
22. 0:36the hardware that it takes to run it
23. 0:38each particular piece of hardware is an
24. 0:39abstraction about
25. 0:40how that's built all the way down to the
26. 0:42gate level all the way down to the
27. 0:43abstraction above
28. 0:44electrons moving around and filling
29. 0:46holes pretty neat
30. 0:48we've seen moore's law moore's law
31. 0:52has been what has fueled the growth of
32. 0:54the computer
33. 0:57manufacturing industry for many years
34. 0:59for decades
35. 1:00um it's a trillion dollar industry it's
36. 1:02very exciting we've seen that moore's
37. 1:03law is
38. 1:04continuing unabated so far um fingers
39. 1:07crossed here but that some things are
40. 1:09changing 2005 we saw
41. 1:11a flattening of the curve of clock
42. 1:13frequency
43. 1:14and the single thread performance but
44. 1:17then
45. 1:18a resilience of the industry was we're
46. 1:20going to go multi-core and so you saw
47. 1:21the number of
48. 1:23cores per per machine continue to
49. 1:25increase
50. 1:26we've seen the principle of locality
51. 1:27memory hierarchy
52. 1:29caches virtual memory disk tape
53. 1:32cloud computing is even on that
54. 1:33conversation somewhere down in the lower
55. 1:35levels as well
56. 1:36right very interesting and we've we've
57. 1:38seen some parallelism we've seen thread
58. 1:39level parallelism data level parallelism
59. 1:41many different five different layers
60. 1:43we've seen all five of those layers
61. 1:45um one of the big ones is data level
62. 1:47parallelism how do
63. 1:48how do we actually work at a level um
64. 1:51how does spark and map produce actually
65. 1:52work well works on a cloud system
66. 1:54but let's say we're still going back in
67. 1:56the 1950s and 60s before i was born
68. 1:59these were the machines people were
69. 2:01using to compute
70. 2:02they didn't exist everywhere they
71. 2:03existed big companies and at big
72. 2:05research labs and big universities
73. 2:07you might have one computer that the
74. 2:09whole campus used back in the early
75. 2:11early days
76. 2:12um if you look at this it's pretty
77. 2:13incredible that's the cpu
78. 2:16and that's the person by the way there's
79. 2:17just just you know the first for scale
80. 2:19here
81. 2:20these are people okay these aren't just
82. 2:21these aren't just uh these aren't
83. 2:23they might be actors and they might play
84. 2:24one on tv but in terms of scale it was
85. 2:26several size of a human for the central
86. 2:29processing unit
87. 2:29and you see all these other kind of
88. 2:31person size boxes what are they
89. 2:33all i o all input output magnetic tape
90. 2:36reading
91. 2:37control console card punch system
92. 2:40card reader control console high speed
93. 2:43printers disk storage tape controller
94. 2:44many
95. 2:45magnetic tape units all i mean just
96. 2:48incredible how big things were i mean if
97. 2:49you think about all the engineering has
98. 2:50gone to taking that
99. 2:52and making that as small as you know
100. 2:54today's today's
101. 2:55laptop today's cell phones today's
102. 2:57watches it's incredible what we've done
103. 2:59ibm was leading this ibm uh univac
104. 3:02um digital they were companies that put
105. 3:06together one million dollar computer's
106. 3:07business cobalt was being used as a
107. 3:09language fortran
108. 3:10as another programming language and a
109. 3:12time sharing art remember time's sharing
110. 3:13a little slice of time to everybody
111. 3:15os for all their users in the 70s
112. 3:19when i was kind of uh coming up i was
113. 3:21still young at my time but
114. 3:22they had digital hp prime
115. 3:25were out there ibm was still out there
116. 3:28as well putting out
117. 3:29bigger computers there's about ten
118. 3:30thousand dollars computers so they
119. 3:31almost a million dollar computers
120. 3:33um these were labs universities they
121. 3:35were running c and unix so things that
122. 3:36you're doing
123. 3:37they were doing the 70s that were pretty
124. 3:38pretty nice
125. 3:40then the 80s hit mid 80s to the
126. 3:41mid-2000s was
127. 3:43the pc era and there was a huge boom for
128. 3:46individuals owning computers pretty
129. 3:47pretty nice that was exciting um i
130. 3:49bought my first computer in college that
131. 3:50was really fun which i had one when i
132. 3:52was in high school
133. 3:53many of you have had it in high school
134. 3:54as well it says about a 1k computer i've
135. 3:57noticed that computers cost about two
136. 3:59and a half thousand dollars
137. 4:00uh continually so that's kind of a funny
138. 4:02thing that they've been
139. 4:03they've been uh that's not the same
140. 4:06price forever
141. 4:07my first computer was like two thousand
142. 4:09and this back in 1980 dollars two
143. 4:11thousand five hundred dollars
144. 4:12it's still two thousand five hundred
145. 4:13dollars ish uh two to three thousand now
146. 4:14you can get cheaper ones like that but
147. 4:16a higher end computer that's what you're
148. 4:17probably paying now for one person
149. 4:19you're running basic you're running java
150. 4:20you wanna windows os mac os
151. 4:23uh nice we call it the next phase is the
152. 4:26post pc era
153. 4:28this is the late 2000s um this was the
154. 4:30era of a smart device a personal mobile
155. 4:33device relying on
156. 4:34wireless networking to to make it work
157. 4:36apple nokia
158. 4:37google made lots of these about 500
159. 4:39smartphones although the higher ones now
160. 4:40are above a thousand
161. 4:42um and the program languages we use to
162. 4:43program those objective c
163. 4:45swift java android os and ios for the
164. 4:47underlying operating systems
165. 4:49served by drum roll please cloud
166. 4:52computing
167. 4:54so the idea of local area networks
168. 4:57uh amazon google facebook they built
169. 4:59massive warehouse scale computers
170. 5:02with 100 000 servers in them um serving
171. 5:05the data on your personal mobile devices
172. 5:07so if i go to my
173. 5:08watch let's try it here um what's the
174. 5:10weather today
175. 5:12let's see here how long it takes come on
176. 5:14give me some love it's currently partly
177. 5:16cloudy and 61 degrees
178. 5:18okay that's pretty amazing how did that
179. 5:21happen
180. 5:21personal mobile device the cloud was
181. 5:23serving that data pretty incredible
182. 5:25how do you program those devices
183. 5:27mapreduce and
184. 5:28ruby on rails so
185. 5:32that's the that's the first part of this
186. 5:34lecture is a little history lesson
187. 5:36and the next lecture we'll talk about
188. 5:38warehouse scale computing we'll see you
189. 5:39there