[CS61C FA20] Lecture 33.4 - Thread-Level Parallelism I: Multithreading — Transcript

Full transcript

1. 0:00and welcome back up till now all we
2. 0:03understand
3. 0:03is you've got a core you've got a single
4. 0:05hardware thread on that that's all you
5. 0:06know
6. 0:08let's actually talk about the idea of
7. 0:10what multi-threading is and maybe you
8. 0:11could have more than one
9. 0:13thread per core how would that even work
10. 0:15let's think about that
11. 0:17so typical scenario active thread you're
12. 0:19working and you've got a cache miss
13. 0:21oh we're going to sacramento you got to
14. 0:23wait a thousand cycles to go to dram
15. 0:25so we know this already we'll switch
16. 0:26this out and different thread is going
17. 0:27to now run
18. 0:28until the data is available we've seen
19. 0:29that already but what do we have we got
20. 0:31to save the current thread state and
21. 0:33load a new thread state
22. 0:34pc registers could be a lot the avx
23. 0:37could be a lot of registers to save that
24. 0:38and you got to perform you have to
25. 0:39perform that switch in a thousand cycles
26. 0:41you've got to be able to perform that
27. 0:42switch in a thousand cycles how does
28. 0:43that even work
29. 0:44can hardware help can hardware somehow
30. 0:47help
31. 0:48the fact that i've got to switch threads
32. 0:50in less than a thousand cycles when the
33. 0:51when the previous memory request came in
34. 0:53going to sacramento to do a load or
35. 0:55store word
36. 0:57here it is hardware assisted software
37. 1:01multi-threading that's that's the title
38. 1:02here and here's the big idea there's a
39. 1:04really big idea a very clever idea
40. 1:07you got one core remember one core i'm
41. 1:09fine right one core
42. 1:12two threads two hardware threads and one
43. 1:15core
44. 1:16how would you make this happen i'll take
45. 1:17a look
46. 1:19i could have two different pcs
47. 1:22two different pcs two separate registers
48. 1:26one aou look at that transistors are
49. 1:30cheap
50. 1:30transistors are cheap so i two copies of
51. 1:33my pc and registers inside
52. 1:35this looks identical here's the key
53. 1:38from the software point of view this
54. 1:41looks like this looks like
55. 1:42two different hardware threads it's one
56. 1:46core
57. 1:46this is one core it looks like two
58. 1:49different hardware threads
59. 1:51okay pretty neat
60. 1:55we call this or actually intel came up
61. 1:56with this name intel called it
62. 1:58hyper threading okay because both
63. 2:01threads can be active simultaneously and
64. 2:03again we still have one memory
65. 2:05but i have two threads in there
66. 2:09okay two hardware threads on one core
67. 2:13here's a slide from an is an intel pr uh
68. 2:16pr video explaining how this worked in
69. 2:19this picture blue is idle
70. 2:21just here blue all these things that are
71. 2:22blue if you're light blue or all these
72. 2:23things here blue
73. 2:24are idle so nothing's happening there
74. 2:26and i've got in the olden days
75. 2:29logically in terms of physical
76. 2:31processors i still have one physical
77. 2:33core
78. 2:33processor one physical core okay logical
79. 2:37processors visible to the os
80. 2:38if i just have one and i have two
81. 2:40threads a red thread here
82. 2:42and a green thread then this is the red
83. 2:45one and this is the green one
84. 2:47if you're having trouble with the color
85. 2:49here is the red one being loaded in
86. 2:51and here oh look these resources maybe
87. 2:53you have some resource can be used
88. 2:54two of them at the same time we've seen
89. 2:56that before okay now i've got this and
90. 2:57i'll be able to able to
91. 2:58compute the red guy first all that
92. 3:00thread is to finish before the green guy
93. 3:01finishes
94. 3:03here's the idea could you actually run
95. 3:06both at the same time if i have multiple
96. 3:08resources maybe one's doing a load store
97. 3:10maybe one's doing the alu
98. 3:12that's two different parts of the same
99. 3:13cpu couldn't you somehow run both of
100. 3:15them at the same time
101. 3:16so that's the idea i'm going to have
102. 3:18logically visible to the os
103. 3:20there are two of them and they're both
104. 3:22being active at the same time and
105. 3:24sometimes the resources are able to
106. 3:26happen at the same time
107. 3:27so all this blue this is all the blue
108. 3:29area that's idle notice there's less all
109. 3:31that time you're basically the green
110. 3:33guys filling in some of those spots
111. 3:35that's really really neat and what you
112. 3:37see the throughput is much much
113. 3:39higher in terms of throughput so in this
114. 3:43model
115. 3:43of simultaneous multi-threading
116. 3:46multi-threading means
117. 3:47at the same time two threads are working
118. 3:49on one core
119. 3:51you have the number of logical cpus
120. 3:53greater than the number of physical cpus
121. 3:56cpu means core here so i've got one core
122. 3:59one
123. 4:00physical cpu maybe two logical cpus
124. 4:03done okay so run multiple threads at the
125. 4:06same time per core
126. 4:08each thread has its own state pc
127. 4:10registers etc
128. 4:11and we can share some resources cash
129. 4:14instructional unit execution students
130. 4:15and there's a whole group at university
131. 4:17of washington that has
132. 4:18talked about simultaneous
133. 4:20multi-threading that's the smt
134. 4:22group there so check that out so here is
135. 4:25multi-threading logical threads
136. 4:27a little bit more hardware most of the
137. 4:29same hardware right the alu is the same
138. 4:30aou
139. 4:31well add some registers add a pc that's
140. 4:33very little and all of a sudden i can
141. 4:35have
142. 4:35on one core two different hardware
143. 4:37threads pretty powerful
144. 4:39arguably ten percent or more better
145. 4:40performance because sometimes like you
146. 4:42know i'm kind of filling in the gaps
147. 4:43when this guy's idle or do one resource
148. 4:45i can fit another guy in there
149. 4:46a lot of clever engineering has gone
150. 4:48into it to figure out how to make these
151. 4:50as efficient as possible
152. 4:51so separate registers we saw that but
153. 4:53i'm sharing the data path i'm sharing
154. 4:54the alus and sharing the caches
155. 4:58but from the point of view of software i
156. 4:59don't care about that i still i just see
157. 5:01two different i see two different uh
158. 5:04logical cpus we call them logical now
159. 5:05the distinguishment physical cpus
160. 5:07so multi-cores duplicate processors 50
161. 5:10more so versus
162. 5:11that's the logical threads multi-core
163. 5:13means now that's in one cp what if i had
164. 5:15multiple cores there
165. 5:16well that's every time i have core i
166. 5:18have to i'm sharing l3 i'm sharing
167. 5:20memory but i'm distributing l1 and l2
168. 5:23differently and my
169. 5:23and and the and the alus are different
170. 5:25as well so duplicate processors maybe
171. 5:28double
172. 5:28performance but not we're certainly not
173. 5:30going to get to double but only
174. 5:31the logical threads the hyper threading
175. 5:33gives me 10 better performance
176. 5:35multicore gives me arguably up to two
177. 5:37times better performance
178. 5:38and modern machines do both modern intel
179. 5:41pieces of hardware
180. 5:42intel architectures do both multiple
181. 5:44cores with multiple threads per core
182. 5:47so here's my laptop you go to the laptop
183. 5:50you say syscontrol
184. 5:51hw and you get this list and if you grep
185. 5:54on
186. 5:55these two lines you see it says hardware
187. 5:57dot physical cpu
188. 5:59four physically i got four cores
189. 6:02hardware logical cpu
190. 6:04eight nice and if you bring up activity
191. 6:08monitor and i encourage you to do this
192. 6:09on a mac bring up opportunity monitor
193. 6:10you will see
194. 6:11eight bars that tell you whether you're
195. 6:14processing well in fact let me do this
196. 6:16now i'm gonna go go rogue and try this
197. 6:17and i'll probably have to edit this here
198. 6:18out here but let me go and see if i can
199. 6:20bring up activity monitor
200. 6:22activity monitor here and let's see what
201. 6:25happens
202. 6:25this is a floating window from my
203. 6:27activity monitor which shows
204. 6:30eight different bars those eight bars
205. 6:33are
206. 6:35the eight logical cpus so even though i
207. 6:38only have four cores in my machine
208. 6:40i have eight bars of my activity monitor
209. 6:42all up and down and all as i you know
210. 6:44were to run a big quick time and maybe
211. 6:46process this video using premiere pro
212. 6:48all of them are going to be kind of
213. 6:49pinned but it's really fun to watch this
214. 6:51to make sure
215. 6:52to see what the status of the eight
216. 6:55logical cpus you have
217. 6:56so again four cores but eight hardware
218. 6:59threads
219. 6:59total neat okay
220. 7:03let's now take a look at the intel
221. 7:05highest end
222. 7:06as of this printing as of this recording
223. 7:09which is the fall of 2020
224. 7:10the intel w3275m
225. 7:14processor this is a very expensive
226. 7:16device this is a several multi-thousands
227. 7:18of dollars you buy this thing
228. 7:20it's 2 000 2 000
229. 7:23to buy this upgraded cpu what are the
230. 7:26things that we're going to look at on
231. 7:27this
232. 7:28number of cores 28 cores
233. 7:32number of threads 56 there's hyper
234. 7:36threading
235. 7:36in action and what's our
236. 7:39thermal design power how much power does
237. 7:41this use
238. 7:43200 watts this is this and what's the
239. 7:45description the power displays when
240. 7:47um under an intel defined high
241. 7:49complexity workload so
242. 7:50you're pounding on this with some big
243. 7:52quick time or
244. 7:54video processing something where all 56
245. 7:58threads are all kicking in
246. 8:01and you're using 200 watts imagine this
247. 8:03little guy this little teeny guy using
248. 8:05200 watts how to cool that
249. 8:07is certainly a design challenge for the
250. 8:08intel team neat
251. 8:11so here's an example here's six cores 24
252. 8:14logical threads
253. 8:14each of those with hyper threads imagine
254. 8:16a core with
255. 8:18four times hyper threading there's no
256. 8:19reason you couldn't have a design like
257. 8:20that
258. 8:21and so four logical threads per core
259. 8:23this would be presented to the user as
260. 8:2524 logical threads that you could
261. 8:28actually process with
262. 8:29pretty neat so
263. 8:32we're almost done with this lecture
264. 8:33definitions a thread is a sequence of
265. 8:35instructions with its own program
266. 8:36counter
267. 8:37and processor state registered files and
268. 8:39maybe a lot more registers than just the
269. 8:40standard 32 we have
270. 8:42within the space of multi-core a
271. 8:45physical cpu
272. 8:46is at the early days is a one at a time
273. 8:48one threaded time in cpu
274. 8:50and the software is going to be
275. 8:51multiplexing bringing that back bringing
276. 8:52that back in typically result
277. 8:54to an i o event a stall some kind of
278. 8:56blocking you pulled it in
279. 8:57a logical cpu says you can now have
280. 9:00perhaps
281. 9:01more logical cpus than physical cpus
282. 9:03with the idea of this
283. 9:05and the bullet below says hyper
284. 9:07threading model this hyper threading
285. 9:08we saw for two for intel you know you
286. 9:10couldn't have four or more if you're
287. 9:11very clever in your engineering
288. 9:13to have simultaneous multi-threading
289. 9:15that means multiple threads
290. 9:17multiple hardware threads on one core
291. 9:19running at the same time pretty
292. 9:20remarkable uh architectural feat to make
293. 9:22that happen
294. 9:24in conclusion sequential software
295. 9:28execution speed is limited
296. 9:292005 it basically flattened they're not
297. 9:32turning the clock speed up anymore
298. 9:33they're putting more transistors on the
299. 9:35chip but nothing's helping me in my
300. 9:36sequential app performance line so what
301. 9:39do i got to do
302. 9:39i better parallel pedal isn't the only
303. 9:41other path to higher performance
304. 9:43we saw simdee high performance cpus all
305. 9:46have cmd you're all going to see that
306. 9:48you're going to be able to have
307. 9:49much wider vectors that you're operating
308. 9:51on this floating point
309. 9:52units um partially supported by
310. 9:54compilers this is something
311. 9:55we try to like get the compiler folks to
312. 9:57get around to it um and it doubles the
313. 9:59width roughly every three to four years
314. 10:00so that's great
315. 10:02so for the computational science folks
316. 10:03they're like yes go simdi
317. 10:05mimdi is the idea of thread level
318. 10:08parallelism which we're talking about in
319. 10:09these set of lectures
320. 10:10multi-core processors it is supported by
321. 10:12the os very cleanly
322. 10:14unlike the simdi work as well because
323. 10:15each of those is like a different
324. 10:16particular
325. 10:17technique to do it but mimdi is more
326. 10:19more traditional
327. 10:20um it needs processor
328. 10:24programmer intervention as simdee did
329. 10:26you gotta you jump in there with pragmas
330. 10:27for the
331. 10:28for the chimney you gotta figure out how
332. 10:29to do this with mindy how to do this
333. 10:30cleverly
334. 10:31and roughly you're seeing a jump of
335. 10:33about two cores every every two years
336. 10:34which is quite interesting
337. 10:35and i mentioned the intel w3275 has 28
338. 10:39cores and 56 threads
339. 10:40amazing and by the way we do both of
340. 10:43them we turn our clock as much
341. 10:45and by the way just fyi normally if you
342. 10:48look at the
343. 10:48scale of the intel recent intel xeon
344. 10:51processors
345. 10:52when you go to the 28 core model they
346. 10:55turn their clock speed down to 2.5
347. 10:57gigahertz
348. 10:58if you go to a 20 core model it's a
349. 11:00higher gigahertz if you go to an eight
350. 11:01core model it's even higher so they're
351. 11:02trading off
352. 11:04fewer cores with higher clock speed but
353. 11:06once you get to 28 cores it's the lowest
354. 11:08clock speed of them all
355. 11:09partly because you can't get the heat
356. 11:10off of that you can't have high clock
357. 11:12speed with 28 cores all chunky at the
358. 11:14same time
359. 11:14you got to give in a little bit and so
360. 11:16as you have more cores you bring the
361. 11:17clock speed down so if you have if
362. 11:19you're on that if you're on a mac and
363. 11:20you're only running a single core ever
364. 11:21a single thread ever well you want to
365. 11:23run different programs certainly but if
366. 11:24you're never going to
367. 11:25make use of that parallelism it might
368. 11:27actually make sense for you to have a
369. 11:28fewer cores
370. 11:29but a higher clock speed for the
371. 11:30particular applications you're doing if
372. 11:32you're
373. 11:32living large in the parallel space then
374. 11:35go crazy with the 28 core machine with a
375. 11:37lower overall clock speed which is very
376. 11:38interesting thing
377. 11:39okay so here's the key idea before we
378. 11:42take you home with this last lecture
379. 11:43the challenge is how do you craft
380. 11:45parallel programs with high performance
381. 11:47on multi-processors
382. 11:49as the number of processors increase
383. 11:50what software help can i have
384. 11:52boy if i only had a lecture that would
385. 11:54teach me how to be able to make use of
386. 11:56all this is all about hardware this
387. 11:56lecture not about software
388. 11:58but how do i and from the software
389. 12:00programming from c
390. 12:01let's go back to the first lecture
391. 12:02second lecture in c can you help me dan
392. 12:05can you help me learn how to program
393. 12:07these multiple core machines how do you
394. 12:08even deal with how to control threads
395. 12:10myself
396. 12:11and the answer is thankfully yes there's
397. 12:12some wonderful libraries that are coming
398. 12:14around to make that easy
399. 12:15and we're going to teach that in the
400. 12:15next series of lectures and we'll see
401. 12:17you there