[CS61C FA20] Lecture 23.3 - Pipelining III: Superscalar — Transcript
Full transcript
- 0:12Hi, welcome back to our pipelining
- 0:13module.
- 0:16We're almost done.
- 0:18We have learned how to design a pipeline
- 0:21processor. We actually learned the
- 0:23principles of designing a pipeline
- 0:24processor.
- 0:27You will get the true feeling for how to
- 0:30design one
- 0:31when doing a project and actually going
- 0:34through design of a pipeline and
- 0:37resolution of all the hazards.
- 0:39That's really fun, actually.
- 0:43But for now, we understood all the
- 0:46principles that we need to deal with in
- 0:48order to design a good processor.
- 0:53There is one more advanced concept that
- 0:55is out there. That's the concept of
- 0:56so-called superscalar processors.
- 1:00You might have heard about that. Many of
- 1:01the modern, very high-performance
- 1:04processors are of superscalar type.
- 1:08Before we get to we get to those, let's
- 1:11see how can we increase processor
- 1:15performance further from what we
- 1:18um know so far.
- 1:21So, we have designed a five-stage
- 1:23pipeline. And what is kind of
- 1:24interesting, this five-stage pipeline is
- 1:27the industry workhorse nowadays.
- 1:29Probably 80% of processors that are out
- 1:32there, all around us, are five-stage
- 1:34pipelines.
- 1:36But they don't have the highest possible
- 1:38performance. Uh
- 1:40there are many designs
- 1:43that um
- 1:45are
- 1:46different. They're higher performance.
- 1:49And when I say majority of of processors
- 1:51out there are five-stage pipelines,
- 1:53you'll find them in all kinds of
- 1:55applications. cars, and
- 1:58um various appliances, and so on.
- 2:01They're really common. They're like
- 2:03that. They're in They're these invisible
- 2:04computers that are all over us, all
- 2:06around us. But, those that we'll find in
- 2:09laptops or in desktops or even in cell
- 2:11phones,
- 2:13there
- 2:14are usually a few higher performance
- 2:18processors, and a lot of these
- 2:20five-stage pipelines as well.
- 2:23So, how do we crank up the performance
- 2:25of a processor?
- 2:26We have a few options. One, we can try
- 2:28to crank up the clock frequency.
- 2:31But, the clock frequency is limited by
- 2:33two things. Um how fast transistors will
- 2:36switch, and that is set by the
- 2:37technology.
- 2:39And uh the other one is the the power
- 2:42dissipation. We have mentioned that
- 2:44before. Um
- 2:45you know, cranking up the clock
- 2:47frequency was our primary
- 2:49means of adding performance in the '90s
- 2:52up to early 2000s, but then we hit the
- 2:55power ceiling, and
- 2:58we discovered or designers discovered
- 3:00that they can design processors that
- 3:01will
- 3:02run
- 3:03very fast at very high clock
- 3:05frequencies,
- 3:06but there was no practical means of
- 3:09cooling them.
- 3:11So, the clock frequency had to be held
- 3:14constant to allow for reasonable
- 3:16cooling.
- 3:18Then, the second option that's often
- 3:20works in concert with clock rate
- 3:24increases is the concept of deeper
- 3:27pipelining.
- 3:28Um so, we have seen our five-stage
- 3:30pipeline, but we can break up the
- 3:33execution into smaller and smaller
- 3:35chunks. So, people have built pipelines
- 3:38that are 10 stages deep or 15 stages
- 3:41deep. Uh Intel processors, I think, went
- 3:44up to 20 stages deep at some point.
- 3:48So, the idea here is to do let less work
- 3:52in each cycle to have less logic depth
- 3:54to go through less logic gates, so we
- 3:56can have a shorter clock cycle.
- 3:59And that enables us to either run at a
- 4:01higher frequency or if you're limited
- 4:04with power to drop the supply voltage
- 4:06and run at a same clock frequency but
- 4:09with lower power.
- 4:12However, the deeper the pipeline,
- 4:15the higher the chances that we are going
- 4:17to run into hazards, so our CPI is
- 4:20naturally going to
- 4:22get higher to be higher than one.
- 4:25So, what came as an alternative to that
- 4:29are so-called multi-issue or superscalar
- 4:31processors. Those are the ones that
- 4:34theoretically can have CPI that is much
- 4:37less than one.
- 4:39So, how do those work? In a superscalar
- 4:41processor, we have multiple issues. What
- 4:44does it mean? We have multiple execution
- 4:47units and everything else that helps
- 4:50feed those execution units.
- 4:52So, each execution unit is actually a
- 4:54pipeline on its own that is capable of
- 4:57executing either integer instructions or
- 5:00floating-point instructions, or we may
- 5:02have dedicated load and store
- 5:06pipelines.
- 5:08So, since we have multiple execution
- 5:10units,
- 5:11we can fetch multiple instructions per
- 5:13clock cycle
- 5:15and so-called issue them to different
- 5:18execution pipelines.
- 5:21Um so, this enables us to
- 5:25drop our CPI below one. So, we are going
- 5:29to be completing multiple instructions
- 5:32per clock cycle.
- 5:35Often people will use a concept IPC such
- 5:38that
- 5:39um it's easier to to understand things
- 5:41to compare things that are greater than
- 5:43one than one.
- 5:46For example, if you have a 4 GHz
- 5:49four-way multiple issue processor, it is
- 5:52capable of executing 16 billions of
- 5:55instructions per second
- 5:57uh with it it would have a peak CPI of
- 6:010.25
- 6:02and peak IPC would be one over that,
- 6:05which is four.
- 6:07Peak means that if there are no
- 6:10conflicts, uh there are no dependencies,
- 6:13no hazards, we can get to that peak
- 6:17CPI or IPC, but in practice, that number
- 6:20is not going to be so great.
- 6:26Often, these processor employ something
- 6:28is called out-of-order execution.
- 6:31So, in order to deal with the
- 6:33dependencies and hazards, there'll be a
- 6:35hardware unit that'll be picking and
- 6:38figuring out which instructions have
- 6:39dependencies and trying to execute them
- 6:43in an order where they don't depend on
- 6:44each other. And then there'll be a
- 6:46reorder unit um
- 6:48at the at the end of the pipeline that
- 6:50will put the results back in order
- 6:53um such that whoever run that program
- 6:56gets meaningful results.
- 6:59And that is all done perfectly.
- 7:02This we definitely are not going to
- 7:04cover in 61C, but
- 7:07CS 152 goes into a great depth of that.
- 7:11What we'll we will do is um a quick
- 7:14overview of what you will find in a
- 7:17superscalar processor. So, generally,
- 7:19there will be an instruction fetch and
- 7:21decode that will be fetching and
- 7:24decoding multiple instructions per
- 7:26cycle.
- 7:28And then it would go and issue them um
- 7:32in different execution pipelines.
- 7:36Um
- 7:37it will recognize what kind of an
- 7:38instruction it is and it would send it
- 7:41down the integer pipelines or floating
- 7:43point pipelines or load store pipelines.
- 7:47It would use some kind of a reservation
- 7:49station that would be there to reserve
- 7:53the resources the reserve to reserve
- 7:55this functional unit um for future use
- 7:58because they the the processor will try
- 8:01to keep them all filled as much as
- 8:03possible. And finally,
- 8:06the instructions will be committed or
- 8:08retired in order. So, they'll be
- 8:10arriving out of order but coming out of
- 8:12this commit unit in order.
- 8:17Here is an example of um
- 8:19an
- 8:20um
- 8:21out of order superscalar processor which
- 8:23is Intel i7, a relatively modern
- 8:27processor that um
- 8:28you'll find in laptops like the one that
- 8:31I'm using now for this presentation.
- 8:35It is what is shown here is the CPI
- 8:40for um
- 8:41number of benchmarks here. And these are
- 8:43different kinds of benchmarks. You don't
- 8:45really need to know to do uh
- 8:47need need to know which kinds are those
- 8:49but this has something to do with some
- 8:51quantum mechanics simulations. H.264
- 8:55is uh video compression. Um
- 8:58Bzip is a zip um
- 9:01type um of a program that is compressing
- 9:04some files.
- 9:05Um you'll find a Go game and then GCC
- 9:09that is compiling some kind of a target
- 9:11code.
- 9:12And then they measure what is the CPI
- 9:16that um
- 9:19is achieved for these benchmarks. And
- 9:21what you'll find out here is that uh
- 9:24these CPIs
- 9:26in most cases will drop below one. So,
- 9:29there will be less than one
- 9:32and in some cases maybe more than one.
- 9:37There is another interesting thing to
- 9:38notice on this graph. There are
- 9:41There is a ideal CPI that is sitting
- 9:43around 0.25 down here.
- 9:49And then there is realistic CPI that
- 9:52rarely gets below 0.5. So, the the ideal
- 9:56CPI is 0.25 or about a quarter, and
- 9:59realistic one is 0.44 and so on. This
- 10:02realistic CPI is due to real CPI is due
- 10:06to stalls,
- 10:08mis-speculation, and handling hazards.
- 10:13By the way, how is this CPI number
- 10:15actually achieved or or measured? Um
- 10:19it's not something magic. It's just the
- 10:21reverse of our iron law of processor
- 10:23performance. So, what we have there in
- 10:26there we had our CPI, which stands for
- 10:28cycles per instruction.
- 10:30And the CPI
- 10:34can be evaluated by knowing all the
- 10:36other elements of this equation. So, we
- 10:39can time the program,
- 10:41how long does it take us to execute the
- 10:43program, then we can
- 10:45count the number of instructions that
- 10:46that benchmark has, and finally
- 10:50we can look up
- 10:51how long would
- 10:53does it take to execute a cycle
- 10:56as the frequency of a processor. That
- 10:58give us gives us the number of cycles
- 11:00per instruction,
- 11:01and that is straight time to execute a
- 11:04program divided by instructions per
- 11:07program times the time
- 11:09um
- 11:10per cycle.
- 11:14So, that's it. This is the way how these
- 11:17CPI numbers, average CPI numbers, have
- 11:20been
- 11:22measured for i7 or many other
- 11:26processors.
- 11:30Few notes about the design of ISA and
- 11:33which ISAs are good for for pipelining.
- 11:36So, RISC-V is a
- 11:38a type of a RISC
- 11:40ISA that is designed with pipelining in
- 11:43mind. And there are several features
- 11:46that you will recognize that are in
- 11:47there
- 11:49that are very helpful for designing a
- 11:51pipeline.
- 11:52For example,
- 11:54in our version, 32-bit version, all um
- 11:59instructions and registers are 32-bits.
- 12:02But in every case, in every
- 12:05variant of RISC-V, all instructions are
- 12:0732-bits.
- 12:10So, all these instructions are easy
- 12:14to decode in one cycle.
- 12:18Compare that to x86,
- 12:21their instructions can be
- 12:24one or
- 12:26anything up to 15 bytes. So, decoding is
- 12:31really complicated there. So, you know,
- 12:32take a look at one instruction, you may
- 12:34be able to decode what is what is in
- 12:37there, what are you supposed to do, but
- 12:39more likely you have to go and
- 12:42oh, give me a few more bytes. Oh, can I
- 12:44decode? Not yet.
- 12:47A few more bytes and finally after a 15
- 12:49of them, the most complex instructions
- 12:51can be decoded.
- 12:54RISC-V has a small number, six different
- 12:57instruction formats. They're very easy
- 13:00to decode and read registers in one
- 13:03step. So, in one stage, we can do both.
- 13:07Many more complex ISAs require us to do
- 13:09that in multiple steps.
- 13:12Um load-store addressing can be done in
- 13:16third stage by using the ALU and access
- 13:20the memory in the fourth stage. And then
- 13:22memory operands are all all aligned and
- 13:24take only one cycle.
- 13:26Um that is something that is very
- 13:29convenient.
- 13:31And we have taken advantage of all of
- 13:32this when designing our pipeline.
- 13:37And that's it. So,
- 13:40we
- 13:41we have done it. We got our pipeline. We
- 13:43got our working processor that is
- 13:45capable of executing all RV32I
- 13:49instructions.
- 13:51First in one cycle and then we figured
- 13:54out how how to pipeline it.
- 13:56We identified these five
- 13:59phases of execution that we associated
- 14:01with five stages of a pipeline.
- 14:04And we have designed this controller
- 14:06first for a single
- 14:07stage pipeline, but then we have
- 14:10outlined the principles for a single
- 14:11stage execution, I'm sorry. And then we
- 14:14have I identify how we need to modify it
- 14:18to support
- 14:19resolution of hazards in a pipeline.
- 14:24Pipelining improves the performance
- 14:27and opens an avenue to building
- 14:30something that is really really
- 14:32powerful.
- 14:34We're going to wrap up this module now.
- 14:38In the next module, we are going to
- 14:40continue building better and better
- 14:42computer.
- 14:45See you after a bit of a longer break.
About this transcript
This page contains the full transcript of [CS61C FA20] Lecture 23.3 - Pipelining III: Superscalar by CS 61C Departmental, generated from the public captions YouTube serves with the video. The transcript has 1,911 words across 341 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.