[CS61C FA20] Lecture 34.1 - Thread-Level Parallelism II: Parallel Programming Languages — Transcript

Full transcript

1. 0:00and welcome back this is the
2. 0:03part of lecture entitled thread level
3. 0:06parallelism part two
4. 0:07and the meta goal here is to understand
5. 0:10from the software point of view
6. 0:12how can we how can we make it easier for
7. 0:14the programmer to deal with all these
8. 0:15threads
9. 0:16how do we do this we've got a machine
10. 0:17that's multi-core many logical cpu is
11. 0:20available
12. 0:20how do i write software to be able to
13. 0:22make use of all of that if i have a
14. 0:23really big problem that needs that
15. 0:25certainly i could run if i had let's say
16. 0:27eight logical cpus i can run eight
17. 0:28different programs and then i feel good
18. 0:29about that eighth improvement i got a
19. 0:31quick time and i got a web browser and i
20. 0:32got a
21. 0:33that's fine terminal running a dot out
22. 0:36but what if i have one program that
23. 0:37wants to use all of them what can i do
24. 0:39so part of that
25. 0:40part of this goal this series of
26. 0:41lectures to understand what software
27. 0:42support we have
28. 0:43we're going to do that so let's let's
29. 0:44step back we've got choice of
30. 0:46programming languages so let's talk
31. 0:47about parallel programming languages
32. 0:49and what's out there in this in the
33. 0:50space
34. 0:52these are the languages supporting
35. 0:53parallel programming
36. 0:55it's a lot it's a ton
37. 0:59um and uh i will point out that the one
38. 1:03language that folks are telling me is
39. 1:05just maybe the one to do it right a lot
40. 1:06of people different different
41. 1:08attempts at this go go is a really nice
42. 1:11language
43. 1:12i haven't programmed a little a little
44. 1:14baby toy program but i haven't forgotten
45. 1:15a lot in that
46. 1:16but i've heard that go people who are in
47. 1:18this space think that go might be the
48. 1:19future
49. 1:20to be able to a really clean way so play
50. 1:22with go and explore that if you want to
51. 1:23explore this is more
52. 1:24but there's a lot of things here for
53. 1:25different reasons uh cuda is how you
54. 1:27drive your gpu lots of different things
55. 1:29are here
56. 1:30um we're saying to this with current
57. 1:32with currently with with c so how which
58. 1:34one to pick
59. 1:34let's talk about this well why do we
60. 1:36have so many let's step back why do we
61. 1:38have so many well first of all why do we
62. 1:39have intrinsics
63. 1:41the reason we have intrinsics is because
64. 1:43um the compilers don't support it we
65. 1:45intrinsics are a way of dropping code
66. 1:46directly in the assembly level because
67. 1:48it doesn't know how to
68. 1:49generate that automatically so that's a
69. 1:51little bit annoying and so we're trying
70. 1:52to try to encourage intel
71. 1:53and other folks to fix your compilers uh
72. 1:55so that'll do it automatically for us
73. 1:57um cimd features are continually
74. 2:01being added compilers is really nice we
75. 2:03appreciate that it is certainly an area
76. 2:05of research
77. 2:06people who are in and professor kathy
78. 2:08yellick in our department
79. 2:10is kind of at the intersection of both
80. 2:12of these pieces she's a programming
81. 2:13language person
82. 2:14who cares about parallelism she's
83. 2:15exactly at the space the intersection of
84. 2:17how do you get
85. 2:18programming languages to wake up and
86. 2:19realize we got to make it easier for
87. 2:21people to write parallel programs
88. 2:23um so there's been a you know 20 plus
89. 2:25years of thinking about how to take old
90. 2:27school c
91. 2:28and auto paralyze it to become really
92. 2:31fast assembly
93. 2:32um so first of all sorry 20 plus years
94. 2:35of taking c
95. 2:36and making fast assembly without
96. 2:38parallelization so there's that
97. 2:40how long will it take them to figure out
98. 2:41how to take c and
99. 2:43auto paralyze it if i can automatically
100. 2:45generate assembly and really be clever
101. 2:47about optimizations boy you should see
102. 2:48all the optimizations that the new
103. 2:49assemblers
104. 2:50have i mean the new compilers have which
105. 2:51eventually becomes the assemblies but
106. 2:53the compilers do to become assembly
107. 2:54language
108. 2:55that that first compile level is
109. 2:57remarkable
110. 2:58um so it's really compiler technology
111. 3:00how can we do the same thing for
112. 3:02parallelization i've got a boring c
113. 3:04code and i can really write pretty good
114. 3:05assembly that's if it's not parallel but
115. 3:07if it is a parallel it's a still harder
116. 3:08problem
117. 3:10we think it was an open problem still we
118. 3:12have you know great cases and very
119. 3:14simple things there is
120. 3:15there is automatically handled loops in
121. 3:17an interesting way um that's coming up
122. 3:19but it's it
123. 3:20it's requires a lot of work here's your
124. 3:22opportunity to become
125. 3:23famous if you decide if you have some
126. 3:25breakthrough talk to personaelic and
127. 3:27relative
128. 3:27related people in our department if
129. 3:29you're interested in that space
130. 3:31so the number of choices is an
131. 3:33indication that there is no universal
132. 3:35solution there isn't just this easy
133. 3:36answer that somebody
134. 3:37owned somebody just owns that space and
135. 3:38you know there's there's a couple
136. 3:40you know there there there there there
137. 3:42isn't really a clear sense so everyone
138. 3:44says well
139. 3:44it's not solved yet i'll make my own and
140. 3:46now i'll make my own two and
141. 3:47me too me too me three and you have all
142. 3:50these people clamoring because nobody's
143. 3:51really one
144. 3:52um and the needs are very specific you
145. 3:54have people operating at very large
146. 3:56scale levels with massive data
147. 3:57you work in a very small micro level
148. 3:59with the multiple core space
149. 4:01you know i huge data a million machines
150. 4:04computers
151. 4:05all parts of the world in the
152. 4:06distributed computing space and i have
153. 4:08also
154. 4:08i want to be able to work on just a very
155. 4:10small problem but i want to be able to
156. 4:11have
157. 4:12um you know eight cores all be very fast
158. 4:14one computer so that's a different set
159. 4:16at a different scale and so these
160. 4:17languages are kind of optimized for
161. 4:18different things
162. 4:20um for example scientific computing
163. 4:22matrix multiply
164. 4:23uh machine machine learning we've got
165. 4:25that i've got a web server handling
166. 4:27multiple requests that's usually
167. 4:28distributed computing space
168. 4:29io is happening simultaneously anybody
169. 4:31any web server anybody running a web
170. 4:33server knows that
171. 4:34how do you make that work um languages
172. 4:36are special specialized for different
173. 4:38tasks we've seen that before
174. 4:39um here's a problem not as particularly
175. 4:41easy to use of all these
176. 4:43um and what we're gonna teach you in
177. 4:44621c is parallel
178. 4:46language examples for high performance
179. 4:48computing we're gonna teach you openmp
180. 4:50in the next couple of lectures and after
181. 4:51that we'll teach you mapreduce so
182. 4:52two things that are really powerful and
183. 4:54high level and spark and all those
184. 4:56things will teach you that idea as well
185. 4:57so that's what we're going to do in 61c
186. 5:00and we'll next lecture we'll teach you
187. 5:02opennp
188. 5:02we'll see you there