YouTube2Text

[CS61C FA20] Lecture 21.3 - Pipelining I: Energy Efficiency — Transcript

by CS 61C Departmental · 2,368 words · 419 segments · language en · Watch on YouTube

Full transcript

  1. 0:10Hello and welcome back.
  2. 0:11Let's talk a bit about energy
  3. 0:13efficiency. Energy efficiency
  4. 0:16is something that relates processor
  5. 0:18power with the processor performance.
  6. 0:21Processor power dissipation has become
  7. 0:24much more important over the past couple
  8. 0:27of decades. Back in the '80s, architects
  9. 0:30and designers had a very different
  10. 0:34design objectives.
  11. 0:36Back then, the only goal was to increase
  12. 0:39the performance as fast as possible.
  13. 0:44So, they did everything they had. They
  14. 0:47used all the tools that they had under
  15. 0:49their belt
  16. 0:50to
  17. 0:51do that, to increase the performance.
  18. 0:54So, they shortened the logic depth. They
  19. 0:56used the maximum
  20. 0:58possible supply voltage to crank up the
  21. 1:01frequency of processors. And as a
  22. 1:03result, processor performance was
  23. 1:05doubling every other year.
  24. 1:07But the price in power was high.
  25. 1:10The power was doubling every
  26. 1:13approximately 3 years.
  27. 1:16That was not that big of an issue in
  28. 1:17early '80s when the processors were
  29. 1:19relatively simple
  30. 1:21and uh
  31. 1:24the power
  32. 1:25consumption was relatively low.
  33. 1:28But over time, the power consumption hit
  34. 1:30the limits. And essentially in across
  35. 1:33all the power domains, the power became
  36. 1:37a primary limiting factor sometime in
  37. 1:40the early 2000s.
  38. 1:42And nowadays, power is limited in every
  39. 1:45compute domain. Power is limited for
  40. 1:49um cell phones, for desktop processors,
  41. 1:53and for um um, warehouse scale computers
  42. 1:57in data centers.
  43. 1:59The nature of these power limitations is
  44. 2:00different.
  45. 2:02For example, in cell phones, the power
  46. 2:04is limited by the size of the battery
  47. 2:07that they're we are willing to carry on
  48. 2:09ourselves.
  49. 2:11In desktops, the power is limited by the
  50. 2:14amount of heat that, um, relatively
  51. 2:16inexpensive fan can remove from a
  52. 2:19processor.
  53. 2:20And in data centers, it's both the heat
  54. 2:23and also the cost of energy that it
  55. 2:25takes us to run that data center.
  56. 2:30So, the goal of designs nowadays and in
  57. 2:34foreseeable future is to maximize the
  58. 2:39performance under power constraints. So,
  59. 2:42it's not anymore to just get the maximum
  60. 2:44performance, but to deliver available
  61. 2:47perform performance under power and
  62. 2:49energy constraints.
  63. 2:51In order to get a bit better insight
  64. 2:54into energy efficiency, let's first
  65. 2:55understand
  66. 2:56where does the energy go in CMOS. All
  67. 2:59our logic gates are built in CMOS
  68. 3:01technologies nowadays.
  69. 3:03Um, so, let's examine the very simple
  70. 3:07inverter here. Here is the logic
  71. 3:11symbol of an inverter that we have seen
  72. 3:13many times. Um, there are three
  73. 3:15additions to it. It got the power supply
  74. 3:19and the ground, so supply voltage and
  75. 3:21the ground,
  76. 3:22because it needs, uh, supply and a
  77. 3:25ground in order to the logic levels
  78. 3:28from zero to one. And then we see
  79. 3:31capacitance at its output.
  80. 3:35So, there is a symbol of an inverter on
  81. 3:37the left-hand side and a schematic of an
  82. 3:39inverter on the right-hand side. Um,
  83. 3:42they represent exactly the same circuit,
  84. 3:44just two different level of abstraction.
  85. 3:47The capacitances happen because the
  86. 3:49nature of transistors. Every transistor
  87. 3:52that we use in an inverter has its input
  88. 3:56and output capacitances.
  89. 3:58So, whenever we are trying to switch the
  90. 4:01output level
  91. 4:03of an inverter from zero to one, we have
  92. 4:05to charge a capacitor. That capacitor
  93. 4:09consists of capacitances that are at the
  94. 4:10output of this inverter, but also at
  95. 4:15the inputs of any of the inverters that
  96. 4:19are downstream that are being driven by
  97. 4:22this inverter.
  98. 4:25How much energy does it take us to
  99. 4:27charge a capacitor
  100. 4:28from zero to one, from zero to V? Well,
  101. 4:31the amount of energy is equal to CV
  102. 4:33squared.
  103. 4:34If you remember
  104. 4:36uh from high school physics, if you
  105. 4:38haven't taken any e-classes yet, a half
  106. 4:40of that energy gets stored in a
  107. 4:41capacitor,
  108. 4:43and the other half gets dissipated
  109. 4:46during the
  110. 4:48charging period. So, you in the end we
  111. 4:50end up with a half of a CV squared on
  112. 4:54this capacitor, um
  113. 4:56and a half of CV squared was dissipated
  114. 4:59on this transistor. Uh eventually, when
  115. 5:01we switch back from one to zero, we dump
  116. 5:05that half CV squared into ground.
  117. 5:10That is the dominant part of um
  118. 5:14of power dissipation. This is our energy
  119. 5:17dissipation in in uh CMOS technologies.
  120. 5:20The other one is the leakage.
  121. 5:24Leakage stems from the fact that uh
  122. 5:26transistors are not ideal switches
  123. 5:29anymore. They are much more ideal ideal
  124. 5:31back in the '80s and '90s. Um
  125. 5:35you know, when we would turn off the
  126. 5:36transistor, we would set this its
  127. 5:38voltage input voltage to zero, uh it
  128. 5:41would really turn off. There would be
  129. 5:43very low
  130. 5:44current that would be trickling through
  131. 5:46a transistor. Nowadays, they're more
  132. 5:48like dimmers if you prefer an
  133. 5:51equivalence to a light switch. So, we
  134. 5:53are just able to dim the current that is
  135. 5:55going through the transistor. So, when
  136. 5:57transistor is supposed to be off, it is
  137. 5:59still leaking some amount of current.
  138. 6:01And one thing that you may find somewhat
  139. 6:04paradoxical that the balance between the
  140. 6:07switching energy that goes into charging
  141. 6:09capacitance
  142. 6:10and uh leakage energy is somewhat
  143. 6:13surprising.
  144. 6:15We spend about 70% on charging and
  145. 6:18discharging capacitances, which is our
  146. 6:20useful use of the energy, and 30% may go
  147. 6:24into leakage. That may be a bit of a
  148. 6:26surprise.
  149. 6:28It comes from the basic things that are
  150. 6:32associated with the technology with the
  151. 6:34CMOS technology, and we're not going to
  152. 6:35dive into that. That is covered in other
  153. 6:37classes like
  154. 6:40um 151 or 130.
  155. 6:44But, let's pop back up at a bit of a
  156. 6:48higher level.
  157. 6:53How much energy does it take us to
  158. 6:56complete a task? And let's say that our
  159. 6:58task is to execute a program, for
  160. 6:59example, to compress a video.
  161. 7:04So,
  162. 7:05that's again a complex thing to evaluate
  163. 7:07and there are many parameters, so it is
  164. 7:09a good idea to break it down. And in
  165. 7:11this case, it's uh a good idea to break
  166. 7:13it down into two components. So, energy
  167. 7:16per program, which may be executing our
  168. 7:18tasks, can be broken down into the
  169. 7:21number of instructions that we have in a
  170. 7:23program and the energy per instruction.
  171. 7:28So, um the number of instructions per
  172. 7:31program we have already encountered when
  173. 7:33we're trying to measure the the
  174. 7:35performance. And in this case, you know,
  175. 7:37the shorter the program, the less energy
  176. 7:38it is going to to use.
  177. 7:40Um
  178. 7:41um
  179. 7:42I don't need to say much more about it.
  180. 7:44That is usually outside of designers,
  181. 7:47um,
  182. 7:48um,
  183. 7:49range of, um, of things that they can
  184. 7:52influence.
  185. 7:53Um,
  186. 7:54but it does affect the architectures and
  187. 7:57and general, um,
  188. 8:00computer design.
  189. 8:02On the other hand, what we can much more
  190. 8:04influence is the second part, which is
  191. 8:06the energy per instruction. And the
  192. 8:08dominant part of that is CV squared.
  193. 8:10There is some leakage that goes on, but
  194. 8:12I'm going to ignore the leakage at the
  195. 8:14moment. So, there is some energy that is
  196. 8:19associated with computing that is
  197. 8:21proportional to the capacitance times
  198. 8:23the voltage squared.
  199. 8:26And capacitance, like we've had in a
  200. 8:28simple case of an inverter, is now a
  201. 8:31total equivalent capacitance of all the
  202. 8:34gates that are switching, say in a
  203. 8:36processor core.
  204. 8:38Um, that capacitance is,
  205. 8:42um,
  206. 8:43is the capacitance that we need to
  207. 8:45switch in every instruction to move, uh,
  208. 8:48certain amount of wires from zero to
  209. 8:51one. It depends on the technology that
  210. 8:53we are using and also on the
  211. 8:55architecture of a processor and its
  212. 8:57features.
  213. 8:58On the other hand, the supply voltage is
  214. 9:03a very strong parameter here in trading
  215. 9:07off
  216. 9:08power for performance or energy for for
  217. 9:10performance. Supply voltage nominally is
  218. 9:14around 1 volt, more like 0.8 volts or
  219. 9:160.7 volts in the recent single digit
  220. 9:20nanometer technologies,
  221. 9:23but the energy depends quadratically
  222. 9:26on the
  223. 9:28on the supply voltage. So, if we lower
  224. 9:31the supply voltage by 10%, we save 20%
  225. 9:35of the energy, 21 to be
  226. 9:38um
  227. 9:38more
  228. 9:40um
  229. 9:4219 to be more precise.
  230. 9:46All right.
  231. 9:48Um
  232. 9:49So, the basic goal here of a designer is
  233. 9:51to try to reduce
  234. 9:53the capacitance and the voltage to
  235. 9:56reduce the energy per task.
  236. 9:59And
  237. 10:00voltage is a more powerful knob. One
  238. 10:03thing to keep in mind, lowering the
  239. 10:06voltage also affects the performance
  240. 10:09because transistors are slower when they
  241. 10:13run at a lower supply voltage.
  242. 10:19So, trading off
  243. 10:21ch- changing su- supply voltage trade
  244. 10:24off trades off
  245. 10:25energy savings for loss in performance.
  246. 10:29And we are going to see in different
  247. 10:31compute domains, we're going to be
  248. 10:34seeing processors operating at a bit of
  249. 10:36a higher voltage versus a little bit
  250. 10:38less
  251. 10:39of a voltage to achieve different power
  252. 10:42performance trade off point.
  253. 10:45Okay, talking a little bit more about
  254. 10:47energy trade off examples,
  255. 10:49one thing that happens in every when we
  256. 10:52scale the technology node and implement
  257. 10:54a new processor, we get some natural
  258. 10:57energy savings.
  259. 10:59So,
  260. 11:01back in the '80s and '90s, every time
  261. 11:03and up to um probably about 2010, every
  262. 11:06time we scale the technology, we go into
  263. 11:09the next node in Moore's law,
  264. 11:11the capacitance would shrink by about
  265. 11:1430%. We would save 30% of a capacitance.
  266. 11:19Now,
  267. 11:20those capacitance savings are more like
  268. 11:2415%
  269. 11:26uh from node to node. They're still, you
  270. 11:28know, don't throw away 15%. 15% discount
  271. 11:31is huge.
  272. 11:33Supply has recently been also scaling by
  273. 11:36about 15% and that has been
  274. 11:38traditionally going on since like mid
  275. 11:41'90s or so.
  276. 11:42Um early on the supply reduction again,
  277. 11:46supply is supposed to be reduced by 30%
  278. 11:49to match what we get from the um
  279. 11:52capacitance scaling and that is
  280. 11:54something that is associated with the
  281. 11:56Nard's law.
  282. 11:57But it has been more like 15% because
  283. 12:01designers were greedy and they wanted to
  284. 12:03maximize the performance and they were
  285. 12:06just
  286. 12:07trying to keep that supply as high as
  287. 12:10possible. Now supply reductions are
  288. 12:13limited
  289. 12:14by this leakage because it affects if we
  290. 12:18really want to drop the supply voltage,
  291. 12:20the leakage is going to go up.
  292. 12:22So as a result when we multiply C and V
  293. 12:25squared now from a node to node, we get
  294. 12:27almost uh 40% discount in energy.
  295. 12:31That allows us
  296. 12:33to do things with that. I mean when we
  297. 12:35build a new product, if you keep exactly
  298. 12:37the same architecture and same number of
  299. 12:38cores,
  300. 12:39it naturally becomes 40%
  301. 12:43um
  302. 12:44more energy efficient or has 40% less
  303. 12:47energy.
  304. 12:48Uh or we can pack 40% more stuff in
  305. 12:52there if we have the fixed energy
  306. 12:53budget. So that's kind of convenient.
  307. 12:56That's exactly what has been happening
  308. 12:58with scaling lately.
  309. 13:01So in uh a brief summary,
  310. 13:04the energy efficiency has been improving
  311. 13:06because of Moore's law and reduced
  312. 13:08supply voltage and architects have been
  313. 13:10using that to pack more features and
  314. 13:12improve performance from that.
  315. 13:15Um a little bit about
  316. 13:17power and performance trends over time.
  317. 13:21Um this is a plot that we have shown in
  318. 13:23the first lecture. It is showing years
  319. 13:25of introduction of
  320. 13:27a whole lot of process of processors
  321. 13:30over the past 48 years in Intel's 40
  322. 13:334040
  323. 13:35in early 70s.
  324. 13:37And then
  325. 13:39we see the number of transistors, which
  326. 13:42is Moore's law,
  327. 13:43steadily increasing with a little bit of
  328. 13:46a
  329. 13:47slow down up here. Then we are seeing
  330. 13:51frequency
  331. 13:53and single thread performance power and
  332. 13:56then
  333. 13:57number of cores.
  334. 13:59So, what we have seen is we frequency
  335. 14:02was the primary means for gaining the
  336. 14:06performance in the 80s and 90s and
  337. 14:11architects and designers work together
  338. 14:13to crank it up. But it hit the limit in
  339. 14:15early 2000s and it has been flat since.
  340. 14:18The reason for that is because the power
  341. 14:20had to to stay flat. So, architects
  342. 14:23worked hard and they had to continue
  343. 14:26working on trying to improve the single
  344. 14:27thread performance, which is what we
  345. 14:29have seen here, but also were utilizing
  346. 14:32parallelism by adding more cores to add
  347. 14:36the performance to add to end
  348. 14:39applications.
  349. 14:42How are things going to change over time
  350. 14:45with the end of Moore's law?
  351. 14:48It's not easy to predict it exactly, but
  352. 14:51we got some sense.
  353. 14:53So, first these capacitance savings are
  354. 14:56not going to come that
  355. 14:57easy.
  356. 14:59Voltage savings are not going to happen
  357. 15:03easily either.
  358. 15:05Um
  359. 15:06Simply, as I mentioned before, we can't
  360. 15:08reduce the voltage, otherwise things are
  361. 15:10going to leak.
  362. 15:11Um
  363. 15:12In order to
  364. 15:15So, capacitance savings are also not
  365. 15:17going to happen for free because we are
  366. 15:20slowing down the shrinkage of
  367. 15:22transistors themselves. Lots of savings
  368. 15:24in the capacitance are likely coming
  369. 15:27from packing things in the third
  370. 15:28dimension. So, when things are spread
  371. 15:31apart on a board in two dimensions, we
  372. 15:34get a lot of system capacitance
  373. 15:36basically over there. But, if we can
  374. 15:37pack things on top of each other,
  375. 15:40we're going to save quite a bit of
  376. 15:43of energy there because we'll have less
  377. 15:44capacitance.
  378. 15:46Um
  379. 15:47and
  380. 15:50um that is going to try to
  381. 15:52help us work around this power wall.
  382. 15:58Important takeaway here is a basic
  383. 16:01relationship between the performance and
  384. 16:04power.
  385. 16:07And it is shown here as the energy iron
  386. 16:09law law.
  387. 16:11Energy efficiency, which is managed uh
  388. 16:13which is expressed as the number of
  389. 16:16tasks that we can
  390. 16:18um
  391. 16:19perform per joule. Is what relates the
  392. 16:22performance and the power. So,
  393. 16:24performance is the number of tasks that
  394. 16:26we can execute per second. Power is
  395. 16:28joules per second.
  396. 16:30Energy efficiency is task per joule.
  397. 16:34So, energy efficiency now becomes really
  398. 16:38the key metric of our designs.
  399. 16:41Um and it is something that tells us
  400. 16:44actually how much performance we can get
  401. 16:48out of a power limited design. If the
  402. 16:50power limited design is a cell phone,
  403. 16:52the power on the average is 600 mW. I
  404. 16:55mean, that's what we can drain out of
  405. 16:57the of the battery on the average what
  406. 17:01is specked out. The max power of cell
  407. 17:04phones is limited to a few watts, but
  408. 17:08for a very short burst of time. On the
  409. 17:10other hand, if you're working with a
  410. 17:12Google scale data center,
  411. 17:15um that may have a power envelope of 20
  412. 17:19megawatts that tells us that the energy
  413. 17:21efficiency of that entire data center,
  414. 17:23tells us what kind of a performance we
  415. 17:26can get out of a data center.
  416. 17:29And that's basically it.
  417. 17:32We're going to look into pipelining
  418. 17:36as a way to improve performance after a
  419. 17:40break. See you then.

About this transcript

This page contains the full transcript of [CS61C FA20] Lecture 21.3 - Pipelining I: Energy Efficiency by CS 61C Departmental, generated from the public captions YouTube serves with the video. The transcript has 2,368 words across 419 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.