[CS61C FA20] Lecture 21.3 - Pipelining I: Energy Efficiency — Transcript
Full transcript
- 0:10Hello and welcome back.
- 0:11Let's talk a bit about energy
- 0:13efficiency. Energy efficiency
- 0:16is something that relates processor
- 0:18power with the processor performance.
- 0:21Processor power dissipation has become
- 0:24much more important over the past couple
- 0:27of decades. Back in the '80s, architects
- 0:30and designers had a very different
- 0:34design objectives.
- 0:36Back then, the only goal was to increase
- 0:39the performance as fast as possible.
- 0:44So, they did everything they had. They
- 0:47used all the tools that they had under
- 0:49their belt
- 0:50to
- 0:51do that, to increase the performance.
- 0:54So, they shortened the logic depth. They
- 0:56used the maximum
- 0:58possible supply voltage to crank up the
- 1:01frequency of processors. And as a
- 1:03result, processor performance was
- 1:05doubling every other year.
- 1:07But the price in power was high.
- 1:10The power was doubling every
- 1:13approximately 3 years.
- 1:16That was not that big of an issue in
- 1:17early '80s when the processors were
- 1:19relatively simple
- 1:21and uh
- 1:24the power
- 1:25consumption was relatively low.
- 1:28But over time, the power consumption hit
- 1:30the limits. And essentially in across
- 1:33all the power domains, the power became
- 1:37a primary limiting factor sometime in
- 1:40the early 2000s.
- 1:42And nowadays, power is limited in every
- 1:45compute domain. Power is limited for
- 1:49um cell phones, for desktop processors,
- 1:53and for um um, warehouse scale computers
- 1:57in data centers.
- 1:59The nature of these power limitations is
- 2:00different.
- 2:02For example, in cell phones, the power
- 2:04is limited by the size of the battery
- 2:07that they're we are willing to carry on
- 2:09ourselves.
- 2:11In desktops, the power is limited by the
- 2:14amount of heat that, um, relatively
- 2:16inexpensive fan can remove from a
- 2:19processor.
- 2:20And in data centers, it's both the heat
- 2:23and also the cost of energy that it
- 2:25takes us to run that data center.
- 2:30So, the goal of designs nowadays and in
- 2:34foreseeable future is to maximize the
- 2:39performance under power constraints. So,
- 2:42it's not anymore to just get the maximum
- 2:44performance, but to deliver available
- 2:47perform performance under power and
- 2:49energy constraints.
- 2:51In order to get a bit better insight
- 2:54into energy efficiency, let's first
- 2:55understand
- 2:56where does the energy go in CMOS. All
- 2:59our logic gates are built in CMOS
- 3:01technologies nowadays.
- 3:03Um, so, let's examine the very simple
- 3:07inverter here. Here is the logic
- 3:11symbol of an inverter that we have seen
- 3:13many times. Um, there are three
- 3:15additions to it. It got the power supply
- 3:19and the ground, so supply voltage and
- 3:21the ground,
- 3:22because it needs, uh, supply and a
- 3:25ground in order to the logic levels
- 3:28from zero to one. And then we see
- 3:31capacitance at its output.
- 3:35So, there is a symbol of an inverter on
- 3:37the left-hand side and a schematic of an
- 3:39inverter on the right-hand side. Um,
- 3:42they represent exactly the same circuit,
- 3:44just two different level of abstraction.
- 3:47The capacitances happen because the
- 3:49nature of transistors. Every transistor
- 3:52that we use in an inverter has its input
- 3:56and output capacitances.
- 3:58So, whenever we are trying to switch the
- 4:01output level
- 4:03of an inverter from zero to one, we have
- 4:05to charge a capacitor. That capacitor
- 4:09consists of capacitances that are at the
- 4:10output of this inverter, but also at
- 4:15the inputs of any of the inverters that
- 4:19are downstream that are being driven by
- 4:22this inverter.
- 4:25How much energy does it take us to
- 4:27charge a capacitor
- 4:28from zero to one, from zero to V? Well,
- 4:31the amount of energy is equal to CV
- 4:33squared.
- 4:34If you remember
- 4:36uh from high school physics, if you
- 4:38haven't taken any e-classes yet, a half
- 4:40of that energy gets stored in a
- 4:41capacitor,
- 4:43and the other half gets dissipated
- 4:46during the
- 4:48charging period. So, you in the end we
- 4:50end up with a half of a CV squared on
- 4:54this capacitor, um
- 4:56and a half of CV squared was dissipated
- 4:59on this transistor. Uh eventually, when
- 5:01we switch back from one to zero, we dump
- 5:05that half CV squared into ground.
- 5:10That is the dominant part of um
- 5:14of power dissipation. This is our energy
- 5:17dissipation in in uh CMOS technologies.
- 5:20The other one is the leakage.
- 5:24Leakage stems from the fact that uh
- 5:26transistors are not ideal switches
- 5:29anymore. They are much more ideal ideal
- 5:31back in the '80s and '90s. Um
- 5:35you know, when we would turn off the
- 5:36transistor, we would set this its
- 5:38voltage input voltage to zero, uh it
- 5:41would really turn off. There would be
- 5:43very low
- 5:44current that would be trickling through
- 5:46a transistor. Nowadays, they're more
- 5:48like dimmers if you prefer an
- 5:51equivalence to a light switch. So, we
- 5:53are just able to dim the current that is
- 5:55going through the transistor. So, when
- 5:57transistor is supposed to be off, it is
- 5:59still leaking some amount of current.
- 6:01And one thing that you may find somewhat
- 6:04paradoxical that the balance between the
- 6:07switching energy that goes into charging
- 6:09capacitance
- 6:10and uh leakage energy is somewhat
- 6:13surprising.
- 6:15We spend about 70% on charging and
- 6:18discharging capacitances, which is our
- 6:20useful use of the energy, and 30% may go
- 6:24into leakage. That may be a bit of a
- 6:26surprise.
- 6:28It comes from the basic things that are
- 6:32associated with the technology with the
- 6:34CMOS technology, and we're not going to
- 6:35dive into that. That is covered in other
- 6:37classes like
- 6:40um 151 or 130.
- 6:44But, let's pop back up at a bit of a
- 6:48higher level.
- 6:53How much energy does it take us to
- 6:56complete a task? And let's say that our
- 6:58task is to execute a program, for
- 6:59example, to compress a video.
- 7:04So,
- 7:05that's again a complex thing to evaluate
- 7:07and there are many parameters, so it is
- 7:09a good idea to break it down. And in
- 7:11this case, it's uh a good idea to break
- 7:13it down into two components. So, energy
- 7:16per program, which may be executing our
- 7:18tasks, can be broken down into the
- 7:21number of instructions that we have in a
- 7:23program and the energy per instruction.
- 7:28So, um the number of instructions per
- 7:31program we have already encountered when
- 7:33we're trying to measure the the
- 7:35performance. And in this case, you know,
- 7:37the shorter the program, the less energy
- 7:38it is going to to use.
- 7:40Um
- 7:41um
- 7:42I don't need to say much more about it.
- 7:44That is usually outside of designers,
- 7:47um,
- 7:48um,
- 7:49range of, um, of things that they can
- 7:52influence.
- 7:53Um,
- 7:54but it does affect the architectures and
- 7:57and general, um,
- 8:00computer design.
- 8:02On the other hand, what we can much more
- 8:04influence is the second part, which is
- 8:06the energy per instruction. And the
- 8:08dominant part of that is CV squared.
- 8:10There is some leakage that goes on, but
- 8:12I'm going to ignore the leakage at the
- 8:14moment. So, there is some energy that is
- 8:19associated with computing that is
- 8:21proportional to the capacitance times
- 8:23the voltage squared.
- 8:26And capacitance, like we've had in a
- 8:28simple case of an inverter, is now a
- 8:31total equivalent capacitance of all the
- 8:34gates that are switching, say in a
- 8:36processor core.
- 8:38Um, that capacitance is,
- 8:42um,
- 8:43is the capacitance that we need to
- 8:45switch in every instruction to move, uh,
- 8:48certain amount of wires from zero to
- 8:51one. It depends on the technology that
- 8:53we are using and also on the
- 8:55architecture of a processor and its
- 8:57features.
- 8:58On the other hand, the supply voltage is
- 9:03a very strong parameter here in trading
- 9:07off
- 9:08power for performance or energy for for
- 9:10performance. Supply voltage nominally is
- 9:14around 1 volt, more like 0.8 volts or
- 9:160.7 volts in the recent single digit
- 9:20nanometer technologies,
- 9:23but the energy depends quadratically
- 9:26on the
- 9:28on the supply voltage. So, if we lower
- 9:31the supply voltage by 10%, we save 20%
- 9:35of the energy, 21 to be
- 9:38um
- 9:38more
- 9:40um
- 9:4219 to be more precise.
- 9:46All right.
- 9:48Um
- 9:49So, the basic goal here of a designer is
- 9:51to try to reduce
- 9:53the capacitance and the voltage to
- 9:56reduce the energy per task.
- 9:59And
- 10:00voltage is a more powerful knob. One
- 10:03thing to keep in mind, lowering the
- 10:06voltage also affects the performance
- 10:09because transistors are slower when they
- 10:13run at a lower supply voltage.
- 10:19So, trading off
- 10:21ch- changing su- supply voltage trade
- 10:24off trades off
- 10:25energy savings for loss in performance.
- 10:29And we are going to see in different
- 10:31compute domains, we're going to be
- 10:34seeing processors operating at a bit of
- 10:36a higher voltage versus a little bit
- 10:38less
- 10:39of a voltage to achieve different power
- 10:42performance trade off point.
- 10:45Okay, talking a little bit more about
- 10:47energy trade off examples,
- 10:49one thing that happens in every when we
- 10:52scale the technology node and implement
- 10:54a new processor, we get some natural
- 10:57energy savings.
- 10:59So,
- 11:01back in the '80s and '90s, every time
- 11:03and up to um probably about 2010, every
- 11:06time we scale the technology, we go into
- 11:09the next node in Moore's law,
- 11:11the capacitance would shrink by about
- 11:1430%. We would save 30% of a capacitance.
- 11:19Now,
- 11:20those capacitance savings are more like
- 11:2415%
- 11:26uh from node to node. They're still, you
- 11:28know, don't throw away 15%. 15% discount
- 11:31is huge.
- 11:33Supply has recently been also scaling by
- 11:36about 15% and that has been
- 11:38traditionally going on since like mid
- 11:41'90s or so.
- 11:42Um early on the supply reduction again,
- 11:46supply is supposed to be reduced by 30%
- 11:49to match what we get from the um
- 11:52capacitance scaling and that is
- 11:54something that is associated with the
- 11:56Nard's law.
- 11:57But it has been more like 15% because
- 12:01designers were greedy and they wanted to
- 12:03maximize the performance and they were
- 12:06just
- 12:07trying to keep that supply as high as
- 12:10possible. Now supply reductions are
- 12:13limited
- 12:14by this leakage because it affects if we
- 12:18really want to drop the supply voltage,
- 12:20the leakage is going to go up.
- 12:22So as a result when we multiply C and V
- 12:25squared now from a node to node, we get
- 12:27almost uh 40% discount in energy.
- 12:31That allows us
- 12:33to do things with that. I mean when we
- 12:35build a new product, if you keep exactly
- 12:37the same architecture and same number of
- 12:38cores,
- 12:39it naturally becomes 40%
- 12:43um
- 12:44more energy efficient or has 40% less
- 12:47energy.
- 12:48Uh or we can pack 40% more stuff in
- 12:52there if we have the fixed energy
- 12:53budget. So that's kind of convenient.
- 12:56That's exactly what has been happening
- 12:58with scaling lately.
- 13:01So in uh a brief summary,
- 13:04the energy efficiency has been improving
- 13:06because of Moore's law and reduced
- 13:08supply voltage and architects have been
- 13:10using that to pack more features and
- 13:12improve performance from that.
- 13:15Um a little bit about
- 13:17power and performance trends over time.
- 13:21Um this is a plot that we have shown in
- 13:23the first lecture. It is showing years
- 13:25of introduction of
- 13:27a whole lot of process of processors
- 13:30over the past 48 years in Intel's 40
- 13:334040
- 13:35in early 70s.
- 13:37And then
- 13:39we see the number of transistors, which
- 13:42is Moore's law,
- 13:43steadily increasing with a little bit of
- 13:46a
- 13:47slow down up here. Then we are seeing
- 13:51frequency
- 13:53and single thread performance power and
- 13:56then
- 13:57number of cores.
- 13:59So, what we have seen is we frequency
- 14:02was the primary means for gaining the
- 14:06performance in the 80s and 90s and
- 14:11architects and designers work together
- 14:13to crank it up. But it hit the limit in
- 14:15early 2000s and it has been flat since.
- 14:18The reason for that is because the power
- 14:20had to to stay flat. So, architects
- 14:23worked hard and they had to continue
- 14:26working on trying to improve the single
- 14:27thread performance, which is what we
- 14:29have seen here, but also were utilizing
- 14:32parallelism by adding more cores to add
- 14:36the performance to add to end
- 14:39applications.
- 14:42How are things going to change over time
- 14:45with the end of Moore's law?
- 14:48It's not easy to predict it exactly, but
- 14:51we got some sense.
- 14:53So, first these capacitance savings are
- 14:56not going to come that
- 14:57easy.
- 14:59Voltage savings are not going to happen
- 15:03easily either.
- 15:05Um
- 15:06Simply, as I mentioned before, we can't
- 15:08reduce the voltage, otherwise things are
- 15:10going to leak.
- 15:11Um
- 15:12In order to
- 15:15So, capacitance savings are also not
- 15:17going to happen for free because we are
- 15:20slowing down the shrinkage of
- 15:22transistors themselves. Lots of savings
- 15:24in the capacitance are likely coming
- 15:27from packing things in the third
- 15:28dimension. So, when things are spread
- 15:31apart on a board in two dimensions, we
- 15:34get a lot of system capacitance
- 15:36basically over there. But, if we can
- 15:37pack things on top of each other,
- 15:40we're going to save quite a bit of
- 15:43of energy there because we'll have less
- 15:44capacitance.
- 15:46Um
- 15:47and
- 15:50um that is going to try to
- 15:52help us work around this power wall.
- 15:58Important takeaway here is a basic
- 16:01relationship between the performance and
- 16:04power.
- 16:07And it is shown here as the energy iron
- 16:09law law.
- 16:11Energy efficiency, which is managed uh
- 16:13which is expressed as the number of
- 16:16tasks that we can
- 16:18um
- 16:19perform per joule. Is what relates the
- 16:22performance and the power. So,
- 16:24performance is the number of tasks that
- 16:26we can execute per second. Power is
- 16:28joules per second.
- 16:30Energy efficiency is task per joule.
- 16:34So, energy efficiency now becomes really
- 16:38the key metric of our designs.
- 16:41Um and it is something that tells us
- 16:44actually how much performance we can get
- 16:48out of a power limited design. If the
- 16:50power limited design is a cell phone,
- 16:52the power on the average is 600 mW. I
- 16:55mean, that's what we can drain out of
- 16:57the of the battery on the average what
- 17:01is specked out. The max power of cell
- 17:04phones is limited to a few watts, but
- 17:08for a very short burst of time. On the
- 17:10other hand, if you're working with a
- 17:12Google scale data center,
- 17:15um that may have a power envelope of 20
- 17:19megawatts that tells us that the energy
- 17:21efficiency of that entire data center,
- 17:23tells us what kind of a performance we
- 17:26can get out of a data center.
- 17:29And that's basically it.
- 17:32We're going to look into pipelining
- 17:36as a way to improve performance after a
- 17:40break. See you then.
About this transcript
This page contains the full transcript of [CS61C FA20] Lecture 21.3 - Pipelining I: Energy Efficiency by CS 61C Departmental, generated from the public captions YouTube serves with the video. The transcript has 2,368 words across 419 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.