YouTube2Text

[CS61C FA20] Lecture 35.1 - Thread-Level Parallelism III: Hardware Synchronization — Transcript

by CS 61C Departmental · 4,873 words · 758 segments · language en · Watch on YouTube

Full transcript

  1. 0:00and welcome back in this last series of
  2. 0:02lectures we're going to talk about a
  3. 0:04couple of details
  4. 0:05uh we titled this thread level parallels
  5. 0:08in part three
  6. 0:09hardware synchronization how do we deal
  7. 0:11with race conditions
  8. 0:12that's essentially that first part of it
  9. 0:14what's deadlocked there's some more
  10. 0:15details of openmp we'll have some fun
  11. 0:17here so
  12. 0:17let's jump right in hardware
  13. 0:20synchronization
  14. 0:24as a review openmp has a beautiful way
  15. 0:27to
  16. 0:28abuse beautiful abstraction to be able
  17. 0:30to add parallelism to your c code
  18. 0:32you have a couple of lines of code maybe
  19. 0:34there's a header file and you say i want
  20. 0:36a parallel
  21. 0:36a pragma that says i want to parallel
  22. 0:38this for loop and all of a sudden this
  23. 0:39beautiful for loop
  24. 0:41that used to be a full serial thing is
  25. 0:42now paralyzed across all the threads and
  26. 0:44just works and it works in a really nice
  27. 0:45clever way
  28. 0:47um what this means in some sense is it's
  29. 0:50doing what we talked about
  30. 0:51a couple of lectures ago it takes a for
  31. 0:53loop and it says all right for i equals
  32. 0:540 to some max value here is max
  33. 0:57it says let me just chop that that up so
  34. 0:58maybe if it's two threads it's gonna say
  35. 1:00well zero to
  36. 1:01half of it so if it's a hundred zero to
  37. 1:0349 is one of them and 50 to 99 is the
  38. 1:05second one
  39. 1:05so it kind of keeps each of the each of
  40. 1:07the threads
  41. 1:09each of the cores is going to be working
  42. 1:10on a contiguous part of memory rather
  43. 1:12than well one from here and one from
  44. 1:14there you could do it that way but it's
  45. 1:15a really bad way to do that for your
  46. 1:16caches
  47. 1:17so you have a contiguous area of memory
  48. 1:18that each of the cores is going to be
  49. 1:20able to process
  50. 1:22you should also have so this is the
  51. 1:24third point says you should have a
  52. 1:25simple shape
  53. 1:26um to be able to paralyze something you
  54. 1:28have to sometimes even even a doubly
  55. 1:29nested loop doesn't paralyze very well
  56. 1:31it makes more sense to
  57. 1:32kind of see if you can unwrap that to be
  58. 1:34a single top level
  59. 1:35parallelization rather than two loops
  60. 1:37inside of it it gets complicated so make
  61. 1:39sure you
  62. 1:40you you practice it you'll see it you'll
  63. 1:41see this as you practice with some of
  64. 1:43this maybe for your project
  65. 1:44as well you're not allowed to have a
  66. 1:46premature exit in any of these so if you
  67. 1:47have this
  68. 1:48special code there like a break a return
  69. 1:50and exit to go to is
  70. 1:51you don't put this uh you know don't
  71. 1:53don't don't jump outside of any pragma
  72. 1:55within that
  73. 1:56you can move around inside there but
  74. 1:57don't jump outside of the pragma you can
  75. 1:59mess with things that's not that's not
  76. 2:00appropriate
  77. 2:02we talked about this if you have two
  78. 2:04memory access of a shared memory system
  79. 2:06and two memory accesses form can form a
  80. 2:08data race you can have two memory
  81. 2:10accesses
  82. 2:10both trying to read and update a
  83. 2:12particular variable it doesn't even
  84. 2:14necessarily
  85. 2:14in c code doesn't necessarily look like
  86. 2:16that variable is going to be
  87. 2:18uh in memory but that that variable a
  88. 2:20sum
  89. 2:21we've tried to sum up pi you can't uh
  90. 2:23sometimes
  91. 2:24depending on how you write your code you
  92. 2:26cannot have people
  93. 2:27affecting a shared variable that that
  94. 2:29could be trouble and that's a data race
  95. 2:30and that's a race condition you want to
  96. 2:32try to prevent that with some kind of
  97. 2:33synchronization
  98. 2:34so you want to you want to do this you
  99. 2:36want to you want to be able to
  100. 2:38not have to serialize all of it could
  101. 2:40you actually have
  102. 2:41um everyone contribute to some shared
  103. 2:44space as they're all at the end of a
  104. 2:45thousand different software cores
  105. 2:47software course software threads and you
  106. 2:49want to be able to have them all
  107. 2:50add their contribution to an eventual a
  108. 2:53sum that's going to be pi hopefully very
  109. 2:55close approximation
  110. 2:56how do you do that without having to do
  111. 2:57it in a serial way because you could
  112. 2:59imagine this
  113. 2:59this being something with now i have a
  114. 3:01million different software threads
  115. 3:03you know it'd be nice if you didn't have
  116. 3:05to serialize that
  117. 3:06whole part of it and make it in the
  118. 3:08serial part how could you paralyze right
  119. 3:09to that so that's important
  120. 3:11we're going to see you can't do it at c
  121. 3:13you can't write this in c you've got to
  122. 3:14have lower level support
  123. 3:15at the hardware level to make this
  124. 3:17happen hardware synchronization is the
  125. 3:19secret to this
  126. 3:21so the secret and you're going to see
  127. 3:23this in every solution of any any any
  128. 3:25particular hardware device that has this
  129. 3:27problem
  130. 3:27it happens at the upper level language
  131. 3:29level you need to have hardware support
  132. 3:31to fix it
  133. 3:32and the support the solution is
  134. 3:34something called atomic read and write
  135. 3:36this means you can read and write in a
  136. 3:38single instruction
  137. 3:39and nobody else is permitted to to have
  138. 3:42no other access to that is permitted
  139. 3:44between that read and write um
  140. 3:46and the idea is this is in a shared
  141. 3:48memory space so this is in a shared
  142. 3:50memory space so
  143. 3:51a common implementation here's how we do
  144. 3:53this in a very regular way
  145. 3:54um the this atomic is a swap between
  146. 3:57registers and memory
  147. 3:58um so in some sense you're you're you
  148. 4:00then link
  149. 4:01a read and a write with this uh and the
  150. 4:04right would fail the memorization has
  151. 4:05been tampered with as it says here in a
  152. 4:07slide
  153. 4:07um and risk five has has variations of
  154. 4:10both we'll talk about that so
  155. 4:11we're gonna call these uh atomic memory
  156. 4:13operations
  157. 4:15or amos and the idea is
  158. 4:18you perform an operation on an operand
  159. 4:20in memory
  160. 4:21and set a destination register to the
  161. 4:23original memory value you remember back
  162. 4:25in the day and
  163. 4:26here's a picture of what this looks like
  164. 4:27there's a couple of instructions that
  165. 4:28are that are
  166. 4:29support this uh and you can see them
  167. 4:31here add and
  168. 4:32swap is the one we're gonna see in a
  169. 4:34second this is what it looks like if you
  170. 4:36if you look at the the way it's broken
  171. 4:37down
  172. 4:38uh in risk five and here's what's doing
  173. 4:42you remember i'll just i'll give you a
  174. 4:43summary of this slide um
  175. 4:45you remember if i want to add something
  176. 4:48to a memory location so a memory
  177. 4:50location has a value there's a memory
  178. 4:51location that you know
  179. 4:52a of 5 there's a value there and i want
  180. 4:54to add something i want to add 10 to
  181. 4:55that value
  182. 4:56i can't do that there's no operation to
  183. 4:58add 10 to that value
  184. 5:00the only way i can do this is to bring
  185. 5:01that value into a register
  186. 5:03add something to that register and put
  187. 5:04it back it's a three-step process
  188. 5:06what i'm telling you about this atomic
  189. 5:08memory operation is you're allowed to do
  190. 5:09that you now they're providing you a
  191. 5:11hardware support to do all three of
  192. 5:12those
  193. 5:13and now we can think about how you might
  194. 5:14even build the controlling data path to
  195. 5:15make it work
  196. 5:16all three of those happen at once and
  197. 5:19nothing can be interrupted in this
  198. 5:21process
  199. 5:21so what it means is here's an example
  200. 5:24for the ad
  201. 5:25i'm here's an example of amo ad so let's
  202. 5:27take a look at that real quick and
  203. 5:28what what that looks like here and this
  204. 5:31amo ad
  205. 5:33here we go ammo add rd
  206. 5:37and rs1 here's what's going to do
  207. 5:41i have a value in rs1 and i want to add
  208. 5:44it
  209. 5:45to the memory locate to the value at the
  210. 5:48memory location
  211. 5:49pointed to sorry i have a value in rs2
  212. 5:52and i want to add it
  213. 5:53to the value in the memory location in
  214. 5:56rs1
  215. 5:58and i'd also be nice if i
  216. 6:01then read the old value of rs1 and
  217. 6:04stuffed it in rd so it's kind of a
  218. 6:06two-step process
  219. 6:07think about this so this is what's
  220. 6:08happening so let's let's look at this
  221. 6:10so first i read in this is a pointer
  222. 6:13okay this is a pointer this is star p
  223. 6:14this is a pointer p
  224. 6:16i'm going to read this says read in this
  225. 6:19is a load word into t
  226. 6:20into a local variable t i'm now
  227. 6:24storing that value the old value in that
  228. 6:26in that register
  229. 6:28the old value in the memory location
  230. 6:29pointed to by rs1
  231. 6:31and i write it into rd that's what this
  232. 6:33says
  233. 6:34x of r d is t so i read i read the old
  234. 6:36value put it into t
  235. 6:38and then i update that value here's the
  236. 6:41here's the x of rs2
  237. 6:45which is what's the value i want to add
  238. 6:46to i want to add 10 to the thing to a of
  239. 6:485.
  240. 6:49this so this is my star p it's basically
  241. 6:52like 10
  242. 6:52plus star p this stores my 10
  243. 6:56and i'm going to say this says rd
  244. 7:00gets star p that's what's happening with
  245. 7:02this line
  246. 7:03okay x of rd gets star p
  247. 7:06and then i say star p equals star p
  248. 7:09plus 10 and 10 is stored here
  249. 7:13this says star p equals star p
  250. 7:16which is the t see t is star p plus
  251. 7:19there's my ten
  252. 7:20okay so it's both so in one operation
  253. 7:23it's doing read the old value then do an
  254. 7:27add
  255. 7:27on what the old value was and then put
  256. 7:29it back as the new value that's
  257. 7:31in one operation as an atomic value and
  258. 7:34swap
  259. 7:34is this swap idea so let's do let's look
  260. 7:37at this how we do if we do a swap
  261. 7:38what happens here okay the lock is going
  262. 7:41to be a register
  263. 7:42stored the lock is going to be in memory
  264. 7:44location stored in register a0 so a0 is
  265. 7:46a pointer
  266. 7:47to where my lock is remember that okay
  267. 7:48so a0 is kind of the
  268. 7:50address the p here got that right
  269. 7:52remember setting is
  270. 7:531 and unset is is is 0. free is 0.
  271. 7:57so first i'm gonna set t zero to one
  272. 7:59okay t
  273. 8:00zero equals one now what does this do
  274. 8:04amo swap aq stands for acquire rl stands
  275. 8:07for release
  276. 8:08this is i'm gonna acquire the lock what
  277. 8:10this says this says
  278. 8:12get t1 gets the old lock value okay so
  279. 8:15t1 is going to get remember
  280. 8:16this is the older this if you remember
  281. 8:18the add this guy gets the old value
  282. 8:21and this is the new value i'm going to
  283. 8:22try to put in there okay that's the idea
  284. 8:24here so
  285. 8:25a0 is going to now get that memory
  286. 8:27location is going to get my t0
  287. 8:29and t1 is going to get the old value of
  288. 8:31it okay
  289. 8:32the old value of whatever that lock was
  290. 8:34a star this star
  291. 8:35star a0 if you think about it okay
  292. 8:39now i'm going to spin i'm going to spin
  293. 8:40on this branch not equal to zero so
  294. 8:43if if t one meaning what i got what i
  295. 8:46what was there
  296. 8:47is already a one branch not equal to
  297. 8:49zero
  298. 8:50so if it's a one meaning it was already
  299. 8:52grabbed oh it's already busy
  300. 8:53i just go back here i branch back up to
  301. 8:55there and i spin on this i spin on here
  302. 8:58okay i spin weight on that but the key
  303. 9:00is the amo swap that makes this work
  304. 9:04here we go if i fall through the branch
  305. 9:07not equal to zero that means it was zero
  306. 9:08so now
  307. 9:09i can do my thing and here's the key
  308. 9:11it's not like remember this is the whole
  309. 9:13thing
  310. 9:14if i fell through it it meant that i
  311. 9:16grabbed it this is the whole beauty of
  312. 9:17this ammo swap
  313. 9:19only it's not like well we both could
  314. 9:21read before if
  315. 9:23i follow through the branch not equal to
  316. 9:24zero
  317. 9:26this says that i have it now because i
  318. 9:29wrote the one
  319. 9:30nobody else wrote the one this is this
  320. 9:31is atomic operation
  321. 9:33so only i was writing the one if there
  322. 9:34are two competing guys and it falls
  323. 9:36through only i'm gonna be the one that
  324. 9:38falls through
  325. 9:39the bench not equal to zero i'd be the
  326. 9:40one that actually writes the value
  327. 9:42so now i'll follow through if if i don't
  328. 9:45take the branch that means
  329. 9:46it was zero okay this takes the branch
  330. 9:49if it's
  331. 9:49one which means if it's zero that means
  332. 9:51it was free and i just grabbed it
  333. 9:53that's the idea this swap stuffed the
  334. 9:55one in there not
  335. 9:56anybody's won my one only so now it's
  336. 9:59mine
  337. 10:00and now i go through now the critical
  338. 10:01side now i own the lock that's the key
  339. 10:04if it wasn't if i was doing this if i
  340. 10:06somehow break the ammo swap if you
  341. 10:08remember the bmw swap had like
  342. 10:09four three things you were doing the
  343. 10:11critical idea is only
  344. 10:13if um how do i say it because it
  345. 10:15happened atomically it happened without
  346. 10:16any interruption
  347. 10:17that's the reason this succeeds now if i
  348. 10:19get to that critical section line i know
  349. 10:21that number one was written to by me and
  350. 10:22me alone
  351. 10:23that was the problem before two people
  352. 10:25wrote the one thinking they both had it
  353. 10:27only i had it in this case so now i do
  354. 10:30my critical section in red i do it
  355. 10:32and then i want to release it how do i
  356. 10:34release it i release it by swapping out
  357. 10:36x0 i don't care when i do the swap i
  358. 10:38don't care what the old value is i know
  359. 10:40it was a 1. i know that star a0 is a 1
  360. 10:42because i owned it
  361. 10:43this value the old value is going to go
  362. 10:45into x0 that's ignored nothing happens
  363. 10:47there
  364. 10:48and i basically take my x0 which is zero
  365. 10:50and i stuff this into there and now i
  366. 10:52reset it to zero
  367. 10:53and now it's free i release the lock i'm
  368. 10:56good this is it
  369. 10:57the critical part about it was that amo
  370. 11:00was an atomic operation
  371. 11:02i can read and write at the same time
  372. 11:05so the old way the broken
  373. 11:06synchronization is this wild lock
  374. 11:08and we're both spinning and two people
  375. 11:09could be in a while lock and they both
  376. 11:10say oh great the lock is free boom and
  377. 11:12they both right there one thinking
  378. 11:14i own it can't happen this can't happen
  379. 11:17now with this ammo swap
  380. 11:19this is the same idea so this is the
  381. 11:20same idea by the way if you try to
  382. 11:21translate
  383. 11:22one to one if i just take this by this
  384. 11:24is broken okay
  385. 11:25if i take this one to one and write this
  386. 11:27in risk five it's not going to work
  387. 11:29same the same problem happens risk five
  388. 11:30without this you need the amo swat to
  389. 11:32make this work
  390. 11:34so this is in a way my piece of
  391. 11:37keep trying until i get it and once i
  392. 11:39get it i know it's his mind now i'm
  393. 11:40guaranteed this is mine i've got the
  394. 11:42lock i've got this
  395. 11:43and i unlock with the simple there so
  396. 11:45all this kind of process through so all
  397. 11:47this
  398. 11:47this is this and this is this but done
  399. 11:50correctly
  400. 11:51this is broken this works okay that's
  401. 11:53the idea
  402. 11:54pretty powerful stuff how does this work
  403. 11:57in openmp do i have to
  404. 11:59now do wait dan i can't use c anymore i
  405. 12:01have to now write risk five
  406. 12:02no you can do this in openmp2 here's how
  407. 12:04it's done openmp let's take a look
  408. 12:06so first we're going to declare a
  409. 12:09abstract data type called a lock
  410. 12:11omp lock sub t who knows what that is i
  411. 12:14don't have any is it a number
  412. 12:15is it a whole struct i don't care i just
  413. 12:17have to reserve one of them i have a
  414. 12:18lock
  415. 12:20here is my parallel section i say get
  416. 12:22thread numbers ids thread numbers in my
  417. 12:24parallel section and now i want to have
  418. 12:25a piece
  419. 12:26that only i do the sequential section so
  420. 12:29i first say
  421. 12:30omp set lock and i say address of lock
  422. 12:33and now
  423. 12:34only when i get it do i proceed into
  424. 12:36that section knowing it's only me who's
  425. 12:38in that
  426. 12:38sequential section by the way all a
  427. 12:40thousand threads are all saying the same
  428. 12:41thing
  429. 12:42if they get to that at the same time and
  430. 12:43only one of them is going to grab it and
  431. 12:45the second one will grab it then the
  432. 12:46third one will grab it
  433. 12:48i print some id and then i end the
  434. 12:50sequential section by say
  435. 12:51omp unset the lock and i have to put
  436. 12:54address of lock so i pass it in
  437. 12:56the pointer to that lock both times okay
  438. 12:59and then when i'm all done i don't know
  439. 13:01whether that that required this
  440. 13:02omp the lock there was space there i'm
  441. 13:05gonna have to free it
  442. 13:06so uh i had go back to my parallels i
  443. 13:07had a parallel section before and a
  444. 13:08parallel section afterwards and i'm
  445. 13:10pretty good and at the end i'm gonna
  446. 13:11destroy that lock
  447. 13:12then i'm all done pretty clean pretty
  448. 13:15nice and pretty clean
  449. 13:17so that hardware synchronization was key
  450. 13:19that amo was key to make this work
  451. 13:22normally you have um libraries that
  452. 13:25this is true anytime you're above the
  453. 13:27lowest level normally you have to be
  454. 13:29able to
  455. 13:30well in c that's how you do it because
  456. 13:32openmp does any other language
  457. 13:34has to have a way every language has to
  458. 13:35have a way to synchronize these parallel
  459. 13:37things to say
  460. 13:38nobody only one person can own these as
  461. 13:40i said locked or semi-four at a time
  462. 13:42you have to support that so this is
  463. 13:44gonna be supported in almost every
  464. 13:45language that i know of that supports
  465. 13:46parallel programming there's an idea of
  466. 13:48a semaphore of idea of a lock
  467. 13:49that's built in the system and by the
  468. 13:51way the way it builds it in is by going
  469. 13:53down to and actually making call to the
  470. 13:54amo levels below that it'll actually
  471. 13:56compile or interpret down to that that's
  472. 13:58the key here
  473. 13:59oh openmp also has other pregnant for
  474. 14:02other things critical cases atomic
  475. 14:04barrier ordered there are other things
  476. 14:06please read the manuals in the bottom
  477. 14:07there's many more features private
  478. 14:08variables reductions a lot of stuff in
  479. 14:10there
  480. 14:11if you're going to really dive deep into
  481. 14:13openmp
  482. 14:14there's a link that uh at the openmp.org
  483. 14:17site that has
  484. 14:18uh there and there's a nice hands-on
  485. 14:20documentation which is useful to play
  486. 14:21with there's a tutorial there
  487. 14:23so here's an example of a critical
  488. 14:24section here's an example of another way
  489. 14:26to do this
  490. 14:27mutual exclusive set mutual exclusion
  491. 14:29says that only one thread at a time can
  492. 14:31be in the critical section
  493. 14:32and they're going to wait their term
  494. 14:33into that it make their wait their turn
  495. 14:35so let's go back into our
  496. 14:36you know adding up uh trying to
  497. 14:38approximate the value of pi
  498. 14:40i can just say here look at this look
  499. 14:41how clean and beautiful this is i can do
  500. 14:43the lock i certainly can do the lock and
  501. 14:44that would be fine i can
  502. 14:45work with that but it's a lot more piece
  503. 14:47i have to make a lock and reserve it and
  504. 14:49or i can just say omp critical very
  505. 14:52beautiful very clean
  506. 14:53so we've got omp parallel here's the
  507. 14:55open and close here omp parallel open
  508. 14:57actually open and close here this is
  509. 15:00important
  510. 15:01this is an open and close here for that
  511. 15:03o p parallel and within this little
  512. 15:05range is open p
  513. 15:06critical this this is the loop that
  514. 15:08closes here
  515. 15:09so this is the line there and right
  516. 15:11inside the parallel i'll say i'm saying
  517. 15:13this is a critical section within the
  518. 15:14parallel section saying this guy better
  519. 15:16be only one thread at a time and there
  520. 15:18it is
  521. 15:18pi plus equals some id and it works so
  522. 15:21we started
  523. 15:22we started by if you remember by the way
  524. 15:24there's a thousand threads here software
  525. 15:25threads
  526. 15:26we started by saying well let's just
  527. 15:27have a couple threads four would be fine
  528. 15:29then we said well why don't we have a
  529. 15:30thousand threads well then it's kind of
  530. 15:32annoying at the end i have to wait till
  531. 15:33the thousand things
  532. 15:35they're all done to be able to add them
  533. 15:36up that's a little annoying
  534. 15:38it's not really a big deal for a
  535. 15:39thousand but if i had that number of
  536. 15:40threads is a lot bigger that'd be it may
  537. 15:42be a problem
  538. 15:42how can i synchronize that well let's
  539. 15:44just let's just put that pi plus equals
  540. 15:45and then we introduced we introduced
  541. 15:47explicitly a race condition
  542. 15:48just to teach what race conditions were
  543. 15:50and we said well how can we get back and
  544. 15:51so now i'm at the point of
  545. 15:52like right paren i'm now closing the
  546. 15:54thought the thread
  547. 15:56the conversation about uh how to do this
  548. 15:58and so i introduced a problem to then
  549. 15:59fix it to be able to teach you what this
  550. 16:01critical section was
  551. 16:02and how you can do this with hardware
  552. 16:03synchronization and that's that's the
  553. 16:05key piece here
  554. 16:06now now that i bring up this idea of
  555. 16:09locks i now have to bring up the second
  556. 16:11kind of problem that you have
  557. 16:12introduced with with parallel code
  558. 16:16which is deadlock we talked about race
  559. 16:17conditions and how to deal with race
  560. 16:19conditions with these locks but once you
  561. 16:20introduce
  562. 16:21lock or simmer for you now have the
  563. 16:23possibility of deadlock
  564. 16:25what is deadlock the deadlock is a
  565. 16:27situation where
  566. 16:30multiple actors are each waiting for
  567. 16:32each other and
  568. 16:33we're frozen livelock by the way is the
  569. 16:36same idea
  570. 16:37but people are moving there's motion but
  571. 16:39still you're kind of stuck
  572. 16:40um so deadlock is up you're lurched and
  573. 16:42i'm waiting for you waiting for me to
  574. 16:43know when nothing moves
  575. 16:44um here's a beautiful picture just to
  576. 16:46show you
  577. 16:47of deadlock and i believe this is uh a
  578. 16:50traffic jam
  579. 16:51i think this is in china i'm not sure
  580. 16:53where this is uh within that but
  581. 16:55somebody took a picture of
  582. 16:56a beautiful case of deadlock that
  583. 16:58computer scientists all around the world
  584. 16:59said ah
  585. 17:00deadlocked and we all grabbed that
  586. 17:01photograph and are using it in our in
  587. 17:02our slides
  588. 17:03to teach deadlock i mean that's amazing
  589. 17:05all it takes is
  590. 17:06is one car if just like one car could
  591. 17:09could l i mean look at this
  592. 17:10thing no one can move they're all the
  593. 17:13problem is
  594. 17:13you know you get in the autumn just back
  595. 17:14up well people behind you now nobody can
  596. 17:16move at all because if this if you're
  597. 17:17actually
  598. 17:18packing in sonoma moves you're literally
  599. 17:19stuck no way to do this
  600. 17:21um if you just judge a if this guy can
  601. 17:23just move here move here then these guys
  602. 17:25can get through and then it all frees up
  603. 17:26but you can have this kind of situation
  604. 17:28if you don't have the traffic light set
  605. 17:29up right
  606. 17:31the most famous deadlock by the way is
  607. 17:33called the dining philosopher's problem
  608. 17:34and here's the problem you have these
  609. 17:36philosophers uh around the table
  610. 17:38um and each of them in parallel this is
  611. 17:41a parallel system
  612. 17:42thinks a little bit because philosophers
  613. 17:44they think about something and then
  614. 17:46they grab a left fork it's available if
  615. 17:47it is pick it up and then
  616. 17:49there's a fork on both sides by the way
  617. 17:51five people and five forks is the idea
  618. 17:54you can do this again with like two
  619. 17:55chopsticks one on each side but let's
  620. 17:56just do this one okay
  621. 17:58think until the left fork is available
  622. 17:59if one is pick it up think until the
  623. 18:00right fork is available when it is pick
  624. 18:02it up
  625. 18:03when both forks are held i don't know
  626. 18:05who eats with two forks but
  627. 18:07eat for a fixed amount of time so two
  628. 18:08forks and you're like maybe you're
  629. 18:09pulling apart some meat i don't know
  630. 18:10putting apart some piece of tofu or
  631. 18:11something okay
  632. 18:12and then when you're done put the right
  633. 18:14fork down put the left fork down repeat
  634. 18:15from the beginning
  635. 18:16well what can happen is everybody goes
  636. 18:20and picks up the left fork
  637. 18:22and everyone is now spin waiting on the
  638. 18:25right fork but everyone picked up a left
  639. 18:26fork
  640. 18:27and so this looks actually works better
  641. 18:28with chopsticks to be honest so
  642. 18:30everyone goes to the right one and
  643. 18:32there's no right one so all five
  644. 18:33are stuck in this deadlock scenario
  645. 18:36where there's no
  646. 18:37and just like the parking situation no
  647. 18:40no no traffic jam
  648. 18:41no one can grab their right fork so
  649. 18:43nothing happens
  650. 18:44here's an example of live lock you walk
  651. 18:47past somebody in the hallway
  652. 18:48and you say oh i'm so sorry you're like
  653. 18:49this but that person also walks like
  654. 18:51that you're
  655. 18:51sorry and you walk like this and you do
  656. 18:53this and the person follows your mirrors
  657. 18:55trying to
  658. 18:55do this and if you if these are kind of
  659. 18:57robots you can imagine a scenario where
  660. 18:59each robot pauses for
  661. 19:00the same amount of time and then moves
  662. 19:02to the right and then pauses for the
  663. 19:03same amount of time and move to the left
  664. 19:04and never pass each other ever you can
  665. 19:07imagine a little simulation where they
  666. 19:08just do this wiggle back and forth that
  667. 19:10would be a problem
  668. 19:12so all this is deadlock we have to think
  669. 19:14about how to how to prevent that what
  670. 19:15are some solutions to think about
  671. 19:16preventing deadlock we'll let you think
  672. 19:18about that but that is something certain
  673. 19:19we need to think about
  674. 19:22we also want to talk about timing we
  675. 19:23want to be able to think
  676. 19:25how do we prove that this wonderful how
  677. 19:28do i how do i adjust the parameters to
  678. 19:29make this parallel
  679. 19:31program work faster normally if i were
  680. 19:34looking at 621b 61a in an algorithm i'd
  681. 19:36do
  682. 19:37algorithm analysis and count the number
  683. 19:38of basic primitive operations and i have
  684. 19:39what's called a running time which is
  685. 19:41the number of primitive steps
  686. 19:42it's not time by the way running time is
  687. 19:43not time you learn this in cs10
  688. 19:45629 and 610b it's not time it's
  689. 19:47primitive operations account it's a
  690. 19:49count really and it's a count
  691. 19:51so it's a you know how does how do
  692. 19:52things grow as the size of the input
  693. 19:54grows that's what running time is
  694. 19:55we said don't use the clock don't use
  695. 19:57wall clock time or stop watch time
  696. 20:00well when you're running parallel code
  697. 20:02often you do use wall clock time because
  698. 20:04you have
  699. 20:05a thousand things you might you care
  700. 20:07less about how many steps each of these
  701. 20:09threads does
  702. 20:09but how much faster is this
  703. 20:11parallelization compared to before
  704. 20:12so in some sense you are using wall
  705. 20:14clock time so let's go back to
  706. 20:16let me undo the idea that you never use
  707. 20:17wall clock time to actually do that
  708. 20:20omp or openmp provides uh some support
  709. 20:23some software support to be able to help
  710. 20:24you with that what they do is they
  711. 20:25provide something called
  712. 20:26open omp get w time which is a void
  713. 20:29uh returns a double doesn't take any
  714. 20:31arguments the idea is it returns the
  715. 20:34elapsed wall clock time in seconds
  716. 20:37from some other time in the past and the
  717. 20:39way you can
  718. 20:40then figure this out is you have two
  719. 20:43calls you
  720. 20:44assign at the time maybe it's like
  721. 20:45seconds since 1900 who knows second
  722. 20:47since
  723. 20:48the year five who knows what that is
  724. 20:51it's some random
  725. 20:51at some random it's some value it's some
  726. 20:54some value
  727. 20:55of the number of uh of some time in the
  728. 20:58past
  729. 20:58boom and now i then run my code
  730. 21:02i then split it 14 ways i join i split
  731. 21:04and fork and join
  732. 21:05and i come back and i stop it and now i
  733. 21:07can say and some there are some
  734. 21:08parameters to this so maybe it was the
  735. 21:10number of threads i have or how i do
  736. 21:11something or maybe the algorithm
  737. 21:12whatever i'm doing i'm doing something i
  738. 21:14want to kind of measure this better
  739. 21:16than this as the number of threads goes
  740. 21:18up say or this is i run this on
  741. 21:19different machines
  742. 21:20or machines that have a different number
  743. 21:22of logical or
  744. 21:23or or physical cpus so then you have a
  745. 21:27second
  746. 21:28you have the end time and then you
  747. 21:29subtract these two so this is the number
  748. 21:32of seconds since say 1900
  749. 21:33this is the number of seconds in the
  750. 21:35future and if i subtract these two
  751. 21:37then the only thing i'm left with is the
  752. 21:38difference in time between start and end
  753. 21:40so that's how you use omp get w time as
  754. 21:42a way to measure wall clock time
  755. 21:44to see how some parallel analysis works
  756. 21:46okay
  757. 21:48that's the end of this mini lecture
  758. 21:49we'll see the next one thanks so much

About this transcript

This page contains the full transcript of [CS61C FA20] Lecture 35.1 - Thread-Level Parallelism III: Hardware Synchronization by CS 61C Departmental, generated from the public captions YouTube serves with the video. The transcript has 4,873 words across 758 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.