YT YouTube2Text

YouTube2Text → Transcripts → [CS61C FA20] Lecture 24.3 - Caches I: Memory Hierarchy

[CS61C FA20] Lecture 24.3 - Caches I: Memory Hierarchy — Transcript

by CS 61C Departmental · 3,539 words · 567 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00and welcome back in this series of
0:02slides
0:04we're going to take a look at what we
0:06understood the memory hierarchy to be
0:07which before was just i got registers in
0:10memory
0:10are there more elements that we were
0:12hiding from you in terms that we were so
0:13we're going to reveal
0:14kind of some of the layers of the onion
0:16to you in this set of lectures
0:18so again let's go back to our library
0:19analogy i'm writing a report
0:21using the library books i'm going to the
0:22stacks it's uh
0:24some english class it's i'm freshmen and
0:27i've got to report the works of jd
0:28salinger
0:30so i could go to the library what i
0:32probably do by the way what i probably
0:33do
0:34is i go to the library i look up the
0:35books i fetch
0:37all of the books from the stacks i have
0:39a stack you know everybody sees that the
0:40picture of people walking back after the
0:42collector all the books they're supposed
0:43to get
0:44and then i'll find a desk in the library
0:46near enough that if i need to go back
0:47it's not too far
0:49and then what i'll do is i'll open all
0:51the books and i'll have them all open on
0:53my table and i'll be able to reference
0:54this book
0:55you've probably seen this you've
0:56probably seen your classmates have that
0:57well you have the books all around the
0:58table
0:59and you're kind of up you're writing
1:00you're writing a report on jd salinger
1:02yep
1:03i remember writing that last year yep
1:04thanks so much that kind of thing you're
1:05having no conversations with your
1:06friends
1:08now if you need more you go check them
1:10out now you check them out until
1:12your desk kind of gets full and maybe
1:14they say you know sorry you can't stack
1:16the books or something so
1:17maybe there's a limit so then you have
1:18to put some of those books back and you
1:19have
1:20kind of a working set in a way like you
1:22have a working set of
1:23books that are all laid out there if
1:25they don't let you because they're
1:26damaged to the books that means you
1:28can't stack the books up so you have a
1:30certain fixed limit as for a certain
1:32size on your desk
1:33but you're able to really save a lot of
1:35time imagine
1:36if i were to make much smaller desks how
1:38much more painful that would be for you
1:40to only have one or two books and i have
1:41to
1:42keep running back and forth to the
1:43stacks so actually we like the idea of
1:44having a really large desk
1:46i can open more of these things i
1:47actually like the idea that they can let
1:48me stack them but let's say i couldn't
1:50do that
1:50so this idea this analogy of
1:54maybe that the 10 books that i have on
1:56the desk are enough
1:57to cover most of what i need in the rare
2:00case i need an 11th or 12th book i'll be
2:02able to do that
2:03but but for the most part my working set
2:05of books that i have working set of data
2:07that i have
2:08is able to be very very efficient and
2:09the time to reach it is
2:11got it that's how fast it took me to
2:12grab something rather than get up go
2:14look it up again go find it walk
2:16walk the stacks come back get it and
2:18bring it back and do the same
2:19that would be painful so this distance
2:21the time ready one two three
2:23grabbing it and that was about a second
2:25to grab the book and open it up
2:26that was great we like that we like the
2:28idea of having stuff that we're going to
2:29use often
2:30close to us and that's a great analogy
2:32it's close to us and that close to us
2:34physically but close to us in time as
2:35well
2:36it's great stuff that's the idea
2:39memory caching is exactly the same idea
2:41that
2:42that desk is in a way a local
2:46it's a very fast to access but smaller
2:50certainly not smaller than all the books
2:51in the library smaller
2:53collection of the data that i'm going to
2:55be using and so
2:57based on the idea that these the you
2:59know the cpu versus dram speed curve
3:02just got larger and larger that was
3:03crazy we don't want that
3:05we're going to introduce the idea of a
3:07memory cache and this is by the way
3:09a big idea in computer science if you
3:11talk about computational thinking
3:13and in fact i'm on some national
3:15committees to talk about computational
3:16thinking
3:18and my perspective what computational
3:19thinking is it's thinking like a
3:21computer scientist it's thinking
3:22of the world in a lens in the glasses
3:25that computer scientists
3:26all walk around with all the time and
3:29caching is one of those big ideas
3:32i have it on my phone you have a
3:33favorites list you have a recently
3:35called list favorites is where you get
3:36to specify what's it what's in your
3:38faster access but the recently called
3:40list if it only shows 10
3:42that is in a way recently called it's
3:44exactly the idea of what we're talking
3:46about here it's like the recently used
3:47books that i had
3:49so typically the way we implement we're
3:52going to build these caches are going to
3:54be done with the same ic technology the
3:56same processing technology as the cpus
3:58and even sometimes on the same chip
4:00that's great
4:01same chip really the farther even
4:04physically the farther you go from the
4:05cpu from the registers
4:07the farther you go physically and
4:08distance the slower it's going to be to
4:10get there
4:10probably the bigger you're going to have
4:11but the slower it's going to be to get
4:12there i've got my hard drive
4:14with a long cable across the room i got
4:15a network of computers
4:17in the cloud all those things farther
4:19away larger but
4:20really small and fast so you want to
4:23have
4:24the caches be on chip if you can if you
4:27can do that you get great speed
4:29the important idea boy we put it in
4:31yellow here and i can't emphasize it
4:32enough
4:34a cache and this is by the way with the
4:36library analogy falls down
4:38i grabbed the single copy of the book
4:39and now it's on my table so it wasn't
4:40there but in digital versions
4:42it's not like one version of those bits
4:44so i the cache is always
4:47a copy a subset of what memory had
4:49memory has
4:50in a way the reference copy the
4:53reference values
4:54and the cash is always a copy it's
4:57always a copy in fact for the first
4:58couple of lectures we're going to talk
4:59about
4:59cash reads where i'm only reading values
5:02only when you have storage do you have
5:03conversations of
5:04keeping the copy consistent so for now
5:06just think they're always consistent
5:07until i tell you otherwise
5:09and i'm only ever reading from a cache
5:10just to just for now let's make it easy
5:12okay
5:13important so say this to yourself cache
5:15is a subset and it's a copy of what was
5:18in memory that's the important idea
5:20and by the way this is also something
5:22we're going to see when i show you
5:23pictures of the cpu later
5:25most processors actually separate caches
5:28for instructions and data
5:30you remember in the place where code
5:32lived instructions were down here
5:34and data was up here right you know
5:36whether the data is static
5:37you know growing up from the heap or
5:39going down from the stack
5:40that's still a different place than the
5:42instructions
5:44so rather than kind of corrupt them
5:47and put them all in one place it
5:48actually makes a little bit more sense
5:49to have separate caches
5:50for only instructions which are all the
5:52stuff down here and the data which
5:54all could be a large group of you know
5:56data could come from a large
5:58range of memory addresses where the
6:00restrictions probably come from smaller
6:02but separating them actually keeps it
6:03more clean it's kind of interesting so
6:04think about having two caches at least
6:06an instruction and a data cache
6:08pretty cool this is the picture this is
6:11the picture memory hierarchy that you've
6:13seen
6:13up till now you got a cpu the core is
6:15there the fastest
6:17smallest fastest and most expensive
6:20storage there are registers they're
6:22amazingly fast blisteringly speeds
6:25but they're really small not that many
6:26of them and they're really close to the
6:28cpu so i can get to them an easily one
6:30clock cycle
6:31fractions usually have a clock cycle
6:33farther away and down in this triangle
6:36is physical memory the dram chip when
6:39you say i want to upgrade the memory
6:40you're talking about that the dram is
6:42the dynamic ram
6:43and dynamic means you have to refresh it
6:45uh that's the dynamic means
6:46you have to actually keep pinging it hey
6:48remember that was a one remember it was
6:49a zero you got to keep refreshing it
6:51so that's why it's called dynamic ram
6:53versus static range but you don't want
6:54to do that
6:55and reasonably fast um
6:59but price more reasonably then boy every
7:01bit for registers cost a lot
7:03um those are farther away you know i go
7:05to fry's electronics if that still
7:07exists anymore and i buy up
7:08you know some dims and i plug them in um
7:12dual input memory module so you could
7:14have
7:15um a larger space because they're
7:17external i can now swap them in and out
7:19which is great that's nice sometimes
7:20they
7:20burn on the board and you can't move
7:22them but most of the time you can you
7:23know most desktop computers you can swap
7:24a minimum out
7:25and increase by the way if you want to
7:27make your computer run faster
7:28you're going to learn this when you
7:29learn about vm a virtual memory
7:32buy more memory so talking about
7:34swapping it in and out
7:36are you ready for this revealing the
7:37layer of the onion the layer of the
7:39onion
7:40we're going to put caches in the middle
7:43and they're typically a different
7:44process they're not going to be
7:45sometimes it could be on the cpu which
7:46is great
7:47uh on the on the same core on the same
7:49chip
7:50as the cpu um they are
7:53as they're shown in this triangle it's a
7:54beautiful triangle it's consistent and
7:56by the way
7:56if ever you come with a technology that
7:58is kind of out of place
8:00it will the market will sort it out
8:02meaning well what if i had this if this
8:04could be bigger than memory what if this
8:05could actually be bigger than memory
8:07without it without being slowing it down
8:08well then it would move down there and
8:10memory would be above it like it
8:11actually will the market will move
8:12things around to
8:13automate that it's really interesting um
8:15so it is a three
8:16of three of the three parameters of this
8:18it is faster than memory but slower than
8:21registers
8:22it is more expensive cheaper it is more
8:25expensive than memory
8:26but less expensive than registers same
8:29thing
8:30and it's larger it's smaller
8:33remember the triangle it's smaller than
8:35memory uh but
8:37larger than the registers above it so
8:39those three parameters
8:41it fits right in the middle of all those
8:42three parameters
8:45jim gray was a berkeley alumnus he got
8:48his bachelor's in phd at berkeley
8:51he came up with this beautiful analogy
8:54thinking about
8:55if you had a fact or a piece of paper
8:57with like a secret thing on it like a
8:59password or something you need to get it
9:01how long would it take and making
9:02analogy so people could again that was a
9:04library analogy now what if you actually
9:06had
9:06the same idea but in terms of relative
9:09speed to
9:10how long it would take to get to
9:11something so registers the
9:13model that you've seen before is
9:16registers
9:17i can get to you know insta as fast as i
9:19can
9:20about a minute if you say register is a
9:21minute even though it's like one
9:22nanosecond one clock cycle
9:24um for one gigahertz clock is one
9:26nanosecond so imagine
9:28if the analogy in a physical piece paper
9:30is about a minute to get something
9:31i'm thinking what was that number again
9:33okay i got it okay that's about a minute
9:35now memory the equivalent time
9:39for how long it would take you to get
9:40something from memory
9:42we're going to talk about something in
9:43the hundreds to maybe even a thousand
9:46times
9:47that speed usually it's a several
9:48hundreds times that so what's several
9:50hundred
9:51minutes well we're talking about a trip
9:54to sacramento
9:54so that means you left a piece of paper
9:56how annoying would that be
9:58in sacramento and you gotta go grab that
9:59that would be just terrible
10:01and that maybe it's a book you're
10:02talking about maybe it's a piece of
10:03paper with a password on it
10:05caches are like in the middle right in
10:08the middle of that
10:08triangle that memory hierarchy so the
10:11analogy there is
10:12it's like it's in this room somewhere oh
10:13it's in the room okay i'll find it
10:15did i leave it under that no it's not
10:16under that under the plate knows
10:18but searching the room isn't too bad
10:20it's about two minutes so we love
10:22caches they are bigger than registers
10:24remember register's only a few of them
10:26now they're bigger than registers but
10:28they're about the same speed as
10:30registers you know they're not that far
10:31away in terms of distance at least
10:32the closest cache we're gonna learn
10:33there's more than one cache but just for
10:35now
10:36that's pretty cool to be able to have
10:37something that's much larger than
10:38registers but
10:39just about as fast as them a little
10:41slower but not that bad maybe one or two
10:43clock cycles that's great
10:45so we love this we love this analogy and
10:47we're going to actually see the things
10:48way above sacramento as we expect to go
10:50to disk
10:51how far is disk away you're going to see
10:53be surprised how far disc away
10:55disc is away in this analogy in terms of
10:57how long it takes for
10:58how many clock cycles would it take to
11:00go to disk
11:01so again i think i mentioned before in
11:03my other memory hierarchy but as we look
11:05at this triangle here are some of the
11:06parameters about this i think i might
11:08have said this
11:09as you get farther away as you get
11:11farther away from the processor
11:13you end up having increased access time
11:16okay
11:16so you're going to be slower it's just
11:19slower to go farther away from the cpu
11:21okay and what i mean is it's not just
11:23like it's the same speed but farther
11:24kind of
11:25like in the library analogy just farther
11:27distance no you're going to different
11:28technologies so it actually
11:29kind of it's like walking through
11:30quicksand and then walking through
11:31quicksand with
11:32with a big weight behind you and then
11:33walking through so it gets slower and
11:35slower the farther you go
11:36you get to different processes and you
11:37end up having different speeds
11:39as you get there that's not just the
11:41same speed but farther
11:43also the relative size of memory each
11:45level gets larger so as i move away from
11:47the processor i get bigger and bigger
11:49so i have caches is smaller than memory
11:51uh and we call memory and secondary
11:53memory here but
11:54so cache remember secondary memory which
11:56might be disks so second remember we
11:58often call disks
11:59which might be flash based or might be
12:01of spinning spinning
12:02uh cylinders however we decide that's
12:05what we call secondary memory and
12:07if we include that in this conversation
12:09the farther you get away from the cpu
12:10it's
12:10still consistent it is slower access
12:13time
12:14larger but we didn't mention here it's
12:17cheaper another factor here it's much
12:20cheaper the farther you go away
12:21so the price for a single bit on disk
12:24man
12:25nothing price for a single bit in memory
12:28still pretty good
12:29price for a single bit in cash whoo
12:30price for a single bit on the register
12:32boy that's pretty expensive expensive so
12:34i mean even in today's dollars that's
12:35pretty expensive thinking about what a
12:36single bit costs
12:37relative to a single bit on a disk drive
12:39interesting okay
12:41the other important thing is always it's
12:42the inclusivity it's always a subset the
12:45smaller is
12:46always a copy of the larger and every
12:48single one of the every single layer
12:49from the
12:50from the from the processor working on
12:52something right now it's
12:53computing something right now to the
12:54registers which is more set but the
12:56factory processors i had some data that
12:58flowing under the path i'm working on
12:59two numbers
13:00okay that's really okay smaller um
13:03and a copy of what was in the registers
13:05a copy of what's in my cache
13:07copy what's in memory and a copy in some
13:09sense of what's in disk
13:10remember what the loader does the loader
13:12grabs remember that
13:14it grabs from disk it loads into memory
13:17remembers then a copy of what was on
13:18disk
13:18same idea pretty cool right okay
13:25so the trick here's the whole the whole
13:28beauty it's a little longer
13:29video but this is the beauty of this
13:30idea
13:32the beauty is this abstraction
13:35abstraction is the key idea in this
13:37whole course it's the abstraction
13:40that you're living at forget the cost
13:41for a moment okay but you're living
13:43at the speed of the smaller guy but at
13:47the size of the bigger guy
13:48that's it that's the big idea it's like
13:51having
13:52imagine like having i don't know the
13:54disk available to you
13:55at the speed of registers that's the
13:57idea and the idea of caches is what if
13:59you had all of main memory
14:01which was big but at the speed of a
14:04process that
14:05in some sense the registers but you know
14:06something faster than that the speed of
14:08it's really the speed of the cache but
14:09what if you had access to all the memory
14:11at the speed of the cache it's not
14:12registered but it's pretty close that's
14:14the idea so every
14:15this whole beautiful idea look at the
14:17numbers here so
14:18the speed in cycles i think i want to
14:20circle some of these things so you know
14:22how fast is it to get to the reg file uh
14:24you know
14:25half half a half of a clock cycle
14:29well how about to get to the you know
14:31the caches about ones you know the
14:33single digits
14:34of clock cycles to get to the cache well
14:36how about main memory you know we talked
14:37about this you know about a thousand ish
14:39that's sacramento how about disc boy you
14:42don't even know how bad how far away
14:44this is okay and this is the size you
14:46know the highest is the highest cost in
14:48the small guy and the lowest cost here
14:50and the size is roughly
14:51like that and determine well how big do
14:53you mean this triangle what's the width
14:54of that triangle
14:55you know in the order of a hundreds you
14:58know they had 32
14:59registers something small in the order
15:01of like 10 000 ish
15:03for caches you know memory you know what
15:05size memory is memory is in the gibby
15:07usually some several gibby and disc is
15:09typical in the tubby so
15:10it's interesting to see what the width
15:12is typically for
15:13for computers and technology uh today
15:16and that number is changing every year
15:20so those are the on-chip components the
15:21things that are on ship are
15:23the cache that's the idea so some part
15:24of the cache is on chip and that's just
15:26that's just great
15:28so summary of the whole idea if the
15:30level is close to the process the higher
15:32up you get in that triangle the memory
15:33hierarchy it is smaller
15:35faster and more expensive per bit that's
15:37clear and also the important thing is a
15:39subset of the lower levels i think i
15:40said that four times now i want to make
15:42sure that you don't
15:43get that question wrong on the exam it's
15:45always a subset of copy of the lower
15:46levels
15:47um and the idea of the memory harkey is
15:50the illusion it's the
15:53and illusion woody allen has a line that
15:54says uh i put some backlights but
15:56i have backlights here in my studio i
15:58put some backlights behind me to give me
16:00to give me the illusion of three
16:01dimensions
16:02so same idea the idea here is
16:06in this hierarchy i have the illusion of
16:09the size of the lower levels at the
16:11speed of the upper levels that's it
16:12that's it that's the big idea we'll talk
16:14about how to actually make this work the
16:15next lectures we'll see you there

About this transcript

This page contains the full transcript of [CS61C FA20] Lecture 24.3 - Caches I: Memory Hierarchy by CS 61C Departmental, generated from the public captions YouTube serves with the video. The transcript has 3,539 words across 567 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to translate the transcript into 100+ languages.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, translation to 100+ languages, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.