YT YouTube2Text

YouTube2Text → Transcripts → Scale an App to Millions of Users - System Design

Scale an App to Millions of Users - System Design — Transcript

by Caleb Curry · 7,730 words · 1,095 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:00hey welcome in this video we're going to
0:02go through an exercise of talking about
0:04everything you need to know to scale a
0:06web app from a few users to millions of
0:09users this is likely a pretty common
0:12system design interview question but
0:14it's also a good thing to understand if
0:16you're building out your own apps so if
0:19you're going to end up being a backend
0:20engineer or if you are a software
0:23architect these are the kinds of things
0:25you're going to want to understand and
0:27we'll start with just the small little
0:30tiny app that might have a few users
0:33nothing too crazy I mean look at it it's
0:35so cute it's not going to be able to
0:37support millions of users though so
0:39we're going to continue to add new
0:41things to improve the overall capacity
0:44of our app so we can support more and
0:46more users before we get started a few
0:48things I wanted to mention first off the
0:50system design interview book pretty good
0:52book if you want some practical
0:53exercises on system design so I've been
0:56reading through this might be a good
0:58additional resource for those of you who
1:00are looking to do system design
1:02interviews the next thing is I have a
1:04backend Engineers mind map and this is
1:06going to categorize a substantial number
1:09of different softwares and Technologies
1:11so you have a good framework for
1:13learning as well as a road map at the
1:15bottom giving you a step-by-step guide
1:17on which Technologies to learn in which
1:19order now if you're learning the things
1:21to become a software engineer but you
1:22need some additional support I have a
1:24mentorship program where you can get
1:26one-on-one support and accountability to
1:29help you learn all of the different
1:31components of building backend services
1:33and websites perhaps you're already in
1:35Tech and you're looking to advance your
1:37career or you want to transition into
1:40your first software engineering role
1:42whatever it might be that mentorship
1:44program can help you get there so I'll
1:46have a link to that and the mindmap down
1:48below so we've talked about scalability
1:50a little bit in the previous system
1:52design videos but it's not really a
1:54prerequisite to watch this video so if
1:55you're just jumping in that's fine but
1:57we're going to review real quickly what
1:59scalability is in general scalability is
2:03the ability to support more users so if
2:05your app grows your app doesn't collapse
2:08because you've hit maximum capacity so
2:11pretend this is your web server and you
2:14might be utilizing say 50% of This
2:17Server maybe even a little bit more but
2:19as you start to consume all of the
2:22resources of This Server you might have
2:24some issues so you definitely don't want
2:26to get too close to using all of the
2:28computer's resource ources now that
2:30exact number whether you utilize 50% or
2:3370% that really depends on how much
2:37variation in the computation is needed
2:40so if you're at a constant 70% all the
2:44time that's probably fine but the
2:46reality is typically the usage will go
2:49up and down and that usage can vary on
2:53time of day or a certain season so you
2:56might typically say only be using 60% of
3:00the total computation power of the
3:03server but let's say your application is
3:05a little bit more popular on the
3:07weekends and now you start getting up
3:09here where you're at 90% well now you
3:12have to really start considering how can
3:14we scale so we can support more users so
3:17you'll need to concern yourself about
3:18scalability from ups and downs but also
3:21the growth of the app over time perhaps
3:25when you start out you might be right
3:27here in terms of the grow both of your
3:30application and your business so this
3:32might potentially work but next year you
3:35might have twice the number of users so
3:38not only do you have to think about the
3:40ups and downs either dayto day week by
3:43week season by season but also the
3:45growth over time so that's the idea of
3:49scalability we need to structure the
3:52backend to be able to support more users
3:55if you're dealing with just daytoday ups
3:58and downs then you need to think about
4:00having enough capacity to support the
4:03changes in traffic and this can often be
4:06as simple as getting a stronger server
4:09or adding more servers but when you
4:11start thinking about growth over time
4:13you might need to introduce some
4:15different systems or structures to your
4:17back end to make it more reasonable so
4:20we'll talk about both of those things in
4:22this video so let's make sure we are on
4:24the same page with vertical and
4:26horizontal scaling so I have a video
4:29that should be coming out soon which
4:31we'll talk about the ups and downs of
4:34these in much more detail but as a
4:37general idea vertical scaling is
4:39improving the power of a single system
4:42often called scaling up where horizontal
4:45scaling is just adding more servers so a
4:49visual difference would be going from
4:54this to this and now I have a much
4:58beefier system that we can support more
5:00users with comparing that to just going
5:03from this
5:05to now say three smaller notes so both
5:10of these can achieve similar results
5:13it's just a matter of which one you want
5:14to do and most likely it's going to be a
5:17combination of both but throughout this
5:20you'll need to understand that it's
5:21often the case we can be working with
5:24multiple servers that are working
5:27together as a single unit that's really
5:30important to understand because one of
5:31the easiest ways to increase the scale
5:33of our app is just to add more servers
5:36but you'll have to set up your back end
5:38in such a way that this can work such as
5:39using a load balancer and the ease of
5:43doing this really depends on the type of
5:46thing you're trying to
5:47scale so doing this is a lot easier with
5:51a stateless backend API for example but
5:54more complicated to do with a database
5:57now it's often the case you can also
5:58have some automation so we'll just say
6:02autoscaling and this addresses the first
6:05thing we were talking about which is the
6:07ups and downs in usage from day to day
6:11or week to week The Simple Solution for
6:14this is an autoscaling setup where if
6:17you have more users come in you just add
6:21another node and I didn't draw it good
6:24here with room but you can add more
6:26nodes to support those users as needed
6:30then when the UPS are over you no longer
6:34need that additional computing power you
6:36can remove those nodes and go back to
6:38your standard three while people often
6:41use three you don't have to use three so
6:44don't think there's any magical number
6:46here in terms of the number of nodes
6:48that you use when you're using
6:50horizontal scaling for our visuals here
6:53though I'll often draw as three nodes
6:56now a node is just another name for a
6:58server so you probably got that figured
7:00out by now but as we add nodes it
7:03basically just means adding a server to
7:06a larger cluster or unit so autoscaling
7:11is a term to describe elasticity and
7:15this is a little bit different than
7:19scalability which this deals with the
7:22growth of users being able to support
7:24them whereas elasticity refers to going
7:27up and down in computer hardware to be
7:31able to dynamically adjust all right so
7:34we understand the types of scaling now
7:36there's one additional thing I want to
7:38talk about and that is a typical app
7:41structure so we'll go through that
7:43architecture real quick and then we'll
7:44go through the exercise of starting with
7:46a few users and going to Millions so an
7:49app architecture could depend a lot on
7:51the type of app you're building for this
7:54exercise I'm really thinking of a web
7:56application or even like a mobile
7:58application with a backend API and a
8:02database so this is the typical three
8:04tier architecture which we did a
8:06dedicated video on but that's what we're
8:09going to be going through in this video
8:11as well so I won't get into all of the
8:13details of this because it's not really
8:15the point of the video but the general
8:17structure will look like this you'll
8:19have a database layer then you'll have a
8:21backend layer and then you will have a
8:24frontend layer this front end could be a
8:27website it could be a desktop
8:28application it could be a mobile
8:30application it could be a video game it
8:33doesn't really matter the main thing you
8:35need to understand is that it connects
8:37to some centralized backend which is our
8:39job to design and a database to store
8:43information this is the basic structure
8:46and then we'll kind of adapt this as we
8:48go I wanted to draw these out separately
8:51at first so we have the DB we have the
8:54back end and we have the front end
8:57because the way you SC scale these might
9:01differ so the database scaling I'm going
9:04to have a different approach than the
9:06backend scaling and same idea the front
9:09end might be a little bit different as
9:11well so we'll go through some of those
9:13differences throughout this video but
9:14we'll primarily be focusing first on
9:17this back end as there's a lot of things
9:19that we can do to make this back end
9:21better and scaling a database is more
9:24complex there's a lot of things we can
9:26do at this level to red U the
9:29computation at the database level so by
9:33working here first we can simplify this
9:37so a very simple example of what I mean
9:39there would be a caching layer we can
9:41cache data so we don't have to access
9:43the database as much so that's just an
9:45example we'll get into the details of
9:47what that means and now with all of that
9:49Foundation out of the way we should be
9:50able to get started by going back to our
9:53little tiny square box and scaling from
9:56a few users to Millions so let's start
9:58with a single server I'm going to make
10:00it just a little bit bigger than what we
10:02started with so I have some room to draw
10:05in here with this structure we're going
10:06to have the database the back end and
10:10the front end all served from a single
10:13server so this would be the most simple
10:16architecture but it's also going to be
10:18the most
10:19limited So within the server we have a
10:23database server so basically the
10:25database is in the same computer this
10:28would be kind kind of like if you're on
10:30your local computer and you download
10:33postgress or some database you can run
10:35your backend from that computer and you
10:37can connect to the database that's also
10:39on that same computer in this
10:41environment we would also be hosting the
10:42web server so we will have something
10:44like engine X or Apache which will allow
10:47us to take requests and process those
10:49requests on the back end getting the
10:52data from the database and ultimately
10:54giving HTML or css or whatever the front
10:56end is to the client so so it will look
11:00something like this let's say the client
11:01is over here and they're accessing from
11:03a computer let's just say we're building
11:05a
11:06website they make a request to our
11:09server we have a single application in
11:12code which is going to probably have
11:17some om or database models to work with
11:21the database it will probably have some
11:23API so we'll just
11:26say the back end and it probably has
11:30some way to generate the front end so
11:33with it being just a single monolithic
11:36application you might be using some
11:39template engine such as Ginger 2 razor
11:42pug there's a bunch of different
11:44template engines and basically what
11:46those will do is they will allow you to
11:49take some backend code let's say it's
11:51python give you an HTML structure and
11:56you can write that python in the HT HML
12:00it will process it on the back end
12:02ultimately giving you out just HTML
12:05which can then be sent to the client so
12:08we're kind of getting into the weeds
12:09here of how to actually design the code
12:12which I don't want to get too distracted
12:14from scaling apps but with this
12:17structure everything is being done
12:19within a single application we've talked
12:22about the MVC design pattern and that
12:24might be something you implement for
12:26this monolithic application where the
12:29request from the user hits a controller
12:32that model will do the database stuff
12:34getting data from the database we'll
12:36embed it in that view which is this
12:39templating stuff and then that is the
12:41response that ultimately makes it back
12:43to the end user so you can kind of
12:46ignore all of the details here the main
12:48thing you need to understand is that we
12:50are running from a single computer and
12:53that's very limiting for a couple of
12:54reasons so let's talk about the pros and
12:57cons of this setup the pros it's super
13:00simple for beginners if you're working
13:03on your first web app you can do this in
13:06no time additionally it is simple to
13:09scale now I don't really want to say
13:11that it's easy because there are some
13:15complexities but it's simple because all
13:19you have to do is add computer hardware
13:21the limitation is you can't be specific
13:24about what parts of the app you want to
13:26scale so it's simple to scale because
13:29all you would do is vertically scale if
13:31you need more RAM you just add more RAM
13:33to the server if you need more CPU you
13:36just add CPU to the server but this is
13:39directly related to the con which is you
13:41cannot scale sections independently
13:45second major con is there is no
13:49separation of
13:51concerns and this is actually focusing
13:54more at the code level everything is
13:57just in one app on one server meaning
14:01everything's tightly coupled it's going
14:03to be hard to work on that application
14:06with a larger team so if it's just you
14:09hey maybe this will work but as you
14:11start to add new members it's going to
14:13be more challenging to scale not just
14:16the actual computer hardware but
14:18actually build code that's easy to work
14:21with in a team environment so you may be
14:23limited to just the general scalability
14:26of the code and the maintenance of that
14:28code so that might not be a direct
14:30performance concern but it is a concern
14:34in terms of going from where you are now
14:36to millions and millions of users if
14:38you're doing that all within just a
14:40single project it might be difficult to
14:42achieve the end result of having
14:44millions of users if you have a
14:46limitation in the code maintainability
14:49and you cannot easily add code or keep
14:51it organized working in a team
14:53environment next major con is this is a
14:55single point of failure you have one
14:58server if something goes wrong with that
14:59server literally everything is broken so
15:02this is the biggest red flag in my book
15:05here you don't want to have a single
15:07point of failure and having everything
15:09on one server is asking for problems
15:11there's also concerns such as dsing so
15:15if somebody wanted to get your app to
15:18stop working they could just send a ton
15:20of traffic to this one server maxing out
15:23its resources and causing your app to
15:25crash the other thing is this is limited
15:27to Vertical scaling
15:30which likely has diminishing returns as
15:32you go to a larger and larger system so
15:34we want to think of ways we can
15:36introduce horizontal scaling to this
15:38setup so this would be the pros and cons
15:40of this scenario which is the most
15:42extreme example of a monolithic
15:44application probably not something
15:46you're going to be doing in the real
15:48world as a professional developer but if
15:51you just have a small project of your
15:52own that really doesn't have a lot of
15:55users this architecture definitely could
15:57work it's simplest you just need to set
15:59up one virtual machine on the cloud you
16:02can run everything from that VM and
16:04you're good to go but if you need any
16:05kinds of reliability failover or scale
16:08you're going to immediately hit all of
16:09these problems so we're going to talk
16:11about the next logical move which is to
16:13go to that three-tier architecture where
16:16we split out the database the back end
16:17and the front end so now with the three-
16:20tier architecture we're going to have a
16:22dedicated server for the database so you
16:25can think of our database inside of this
16:28this computer that looks really terrible
16:31I'm going to have to redraw that so now
16:33we have a dedicated database server we
16:36have a dedicated server for the back end
16:40and we can have if needed a dedicated
16:43server for the front end this split
16:47wouldn't necessarily be required but
16:49it's a very logical split and it allows
16:51us to independently scale so going from
16:54that previous all-in-one monolithic
16:56thing I would probably just jump to the
16:58three tier architecture and just split
17:00it out to have the back
17:02end the front
17:04end and the database I would also split
17:08it out this way just because of my
17:10experience with Cloud infrastructure
17:13very easy to do this so for example in
17:16AWS it might look like this we have
17:21RDS
17:24ec2 and then S3 for our frontend static
17:28file hosting the exact Technologies you
17:30use for cloud providers might vary if
17:32you're going to Azure or Google Cloud
17:34platform and if you're self-hosting it's
17:36going to be a little bit different but
17:38these are the things I'm familiar with
17:40and doing it in this structure is very
17:42easy where you have a dedicated
17:44front-end application using something
17:47like
17:48react or another frontend Library so
17:52we're no longer using the template
17:53engines embedded in the back end one way
17:56to achieve scale easily is to use
17:59serverless services so in this scenario
18:02you pay a cloud provider for the usage
18:05of your application instead of for a
18:07certain size of server so that's pretty
18:11common with the front end so going back
18:13to what I just had with S3 it is a
18:16serverless thing where you don't have to
18:18provision you know a medium server with
18:21this much RAM and this much CPU you just
18:23host stuff on S3 and get charged for
18:27doing so so the easiest Next Step would
18:29be to make this a serverless thing and
18:33we're just going to jump right to that
18:35so running the front end from some
18:36serverless service is going to be the
18:39ideal setup since you're just working
18:41with static files it's very cheap very
18:43simple and you don't have to be serving
18:45those from the back end they would just
18:48have
18:51HTML CSS and JavaScript and that
18:54JavaScript would then make a request to
18:57your backend API so that's going to be
19:00your typical architecture the exact
19:02details of how you host the front end is
19:04not a huge concern because this is most
19:07likely not going to be the bottleneck
19:09since you're just working with static
19:10files you're not doing any computation
19:12there is computation being done in
19:14JavaScript but that's all done client
19:16side so that's given to the user and the
19:19JavaScript will execute on their machine
19:22our biggest concern is probably going to
19:23be here where we have to do computation
19:27and doing computation for for each
19:29request and doing that for potentially
19:31millions of users can get pretty
19:32expensive so we're going to prioritize
19:35scaling this layer first we're not even
19:37going to worry about this layer and then
19:39we're going to look at that layer later
19:41so let's just kind of zoom in on this
19:44section and talk about things we can add
19:46or change to make it better before we go
19:48in and change a bunch of stuff with our
19:50backend architecture let's just do a
19:52quick check in so this would be the next
19:55logical step going from just a monolith
19:57to now having this three tier
20:00architecture with basically three
20:02sections and you can now scale each one
20:05of these independently so if your
20:07original bottleneck was just the entire
20:09server it might have been a little bit
20:11more challenging because you'd have to
20:12go in there and figure out what's using
20:14what resources but now if you have a
20:16dedicated database server you can just
20:18see how much resources is on that server
20:21and is it enough to run that database
20:24server so you can independently scale up
20:27or down same thing for the back end and
20:31the front end so that is the biggest
20:33upgrade here is independently scaling
20:36these things having these split out to
20:38independently scale can potentially help
20:40us find any bottlenecks and just improve
20:42the overall performance of our
20:44application so let's talk about the pros
20:46and cons of this setup the first one
20:48it's really
20:50common which means you're not doing some
20:53crazy setup nobody's done before if you
20:55run into issues you'll probably be able
20:56to find plenty of articles TOS or people
20:59to support you to get those issues
21:01figured out next up we can independently
21:04scale the different
21:05sections which is a major improvement
21:08over the previous architecture where
21:10everything was ran from a single server
21:12the
21:13cons from a code perspective you will
21:16now have to have an API whereas before
21:19you were possibly just using some kind
21:21of template engine you didn't have to
21:23design your API layer so I don't even
21:26know if that's a con it's just something
21:27to be aware of perhaps you now have to
21:31code more interfaces between different
21:34components it also adds complexity with
21:38cores so cross origin requests from the
21:42front end to the back end you'll have to
21:44figure that all out as well as
21:47networking between the backend and the
21:50database so you have to learn a little
21:53bit about permissions and how to
21:56actually get those to communicate now
21:57when it comes to scaling you can be more
21:59granular growing each individual thing
22:01up or down but you have to be able to
22:04know how to identify what parts aren't
22:07working well and you can't just grow one
22:10server which is really simple and we're
22:12still limited to Vertical scaling in
22:15this
22:17setup with the exception of the front
22:19end which is probably serverless but
22:22we're really thinking about the backend
22:24here we didn't talk about how to
22:25actually set up horizontal scaling and
22:27that's going to be our next logical
22:29progression so overall this is better
22:32than the first setup but still has some
22:34limitations all right so we've made it
22:36through the extreme monolith the three-
22:38tier architecture and now we're going to
22:40zoom into to the backend layer of that
22:42three- tier architecture and talk about
22:44scaling the back end most likely this is
22:47going to be the first bottleneck you
22:49encounter because the front end is
22:51extremely easy to scale and the database
22:54layer you'll most likely just vertically
22:55scale for a while so we'll zoom into
22:58that back end layer and the very first
23:00thing we're going to want to do is do
23:02horizontal
23:03scaling this is going to allow us to
23:06remove the single point of failure if
23:09you have just one backend server and all
23:13of the requests go to this same server
23:15and for some reason this server goes
23:17down your entire application is now
23:20broken so if we introduce horizontal
23:22scaling we'll now have multiple servers
23:25which means we have some redundancy if
23:27one server goes down we can redirect
23:29traffic to a server that is working so
23:32we can just introduce a couple of
23:34additional servers these don't have to
23:37be more powerful servers simply by
23:40adding more you can achieve more
23:42throughput so we've pretty much 3x the
23:46number of users we can support and if
23:49for some reason one of these goes
23:51down we can just redirect this traffic
23:55to another node instead of this one so
23:58all in all our system is still
24:01operational so far so good the challenge
24:05is how do you know which of these
24:07servers to connect to and how do you do
24:10that from a frontend application if you
24:13have multiple end points and what if one
24:15of them goes down well this is all going
24:18to be done for you by a load balancing
24:21layer so you can think of the load
24:25balancer as a service which will take
24:27requests and direct them to the correct
24:29location so it might look like this
24:31where blue is a request and then this
24:34will direct that to the appropriate
24:38server so we only have one end point and
24:41this would be where the front end
24:43connects to all of this splitting out
24:45the traffic to the individual servers is
24:49going to be abstracted away from the
24:50front end handled all by the load
24:53balancer so the load balancer would be
24:55the one responsible to adjust the
24:58traffic if one of these servers goes
25:00down so it might look something like
25:02this and the load balancer will also do
25:05its best to balance the traffic so
25:08instead of This Server just getting
25:10double the traffic it would now just
25:12split 50/50 across these two nodes so
25:17that's what a load balancer does and
25:18when I draw these out I'll often just
25:20draw it as like a circle or a square
25:22around the servers so it might look
25:25something like this where we have the
25:27three servers and they're all wrapped by
25:30a load balancer so now this is what our
25:33back end would look like where this
25:35would be the front end and that would be
25:36the database and you can imagine having
25:39a domain such as api. mysite.com and
25:43this is going to point to the IP address
25:46of that load balancer so far so good
25:49let's go through the pros and cons of
25:50this so for the pros we removed the
25:54single point of failure in our back end
25:57we're able to to achieve High
26:00availability because we have failover if
26:03one server goes down the traffic's
26:05redirected to another server so the
26:07overall functionality of our application
26:09is not disrupted we are able to
26:11horizontally scale which in theory is
26:15cheaper than vertically scaling due to
26:17diminished returns of vertical scaling
26:20plus apis are typically stateless they
26:22don't have to store any information like
26:25a database so there's no major downside
26:29to using horizontal scaling over
26:31vertical scaling another Pro is that
26:34this would support Auto scaling so we
26:36can increase and decrease Computing
26:38Hardware as needed for different spikes
26:41in our traffic cons is you have to add a
26:44load balancing service not a huge deal
26:47but you have to be aware of that and
26:48know how to do that and then I would say
26:50the other major con is no
26:54caching so every request is seen as a
26:57fresh fresh request coming from the
27:00front-end client hitting the API making
27:02a request to the database then returning
27:05that ultimately back to the client even
27:07if 80% of the requests are the same
27:09thing it's still seen as a completely
27:11new request every time so the next
27:13logical move will be to talk about
27:15things we can do to Cache information
27:18you can think of a cache as something
27:20that stores information temporarily so
27:22say we have a database
27:26here and this database has has some data
27:29let's just put some
27:32numbers and then we have a cach layer
27:36here and then we have a client make a
27:39request and the request is to retrieve
27:43the value five so five is going to be
27:46stored in the cash and then the next
27:48time a client makes a request to get
27:50five it just retrieves the cached five
27:53not having to make another request to
27:55the database so you can see there's only
27:57one one request to the database but two
28:00clients requesting five so this means we
28:02can drastically reduce the amount of
28:05requests to the database using that cach
28:07layer say a client requests something
28:10that's not in the cach say it wants s it
28:13would basically look in the cache
28:15realize it's not
28:17there retrieve it from the
28:21database and then that would be stored
28:23in the cache as well and then seven
28:26would be given back to this client so
28:29that first client might take a little
28:30bit extra time cuz it has to go to the
28:33database but then for future requests
28:35say this client wants seven as well it's
28:38going to be a lot faster because it can
28:40just grab the cash version so that's the
28:42idea of a Cash There's also another
28:44related thing which is a CDN and this is
28:47a Content delivery network but it
28:50functions pretty much like a cache so
28:52say you are over here and you want to
28:55make a request to my server
28:58down here the amount of distance you
29:00have to travel is quite far and when you
29:04make a request to some web server that
29:06request ultimately does have to make its
29:08way through the internet and travel some
29:12distance so that's a far distance to
29:14travel and that's going to make the
29:17application slower so what a CDN will do
29:20is it will basically put your
29:22application casted results in a bunch of
29:26different locations so so it might look
29:28something like
29:31this and now when you make a request you
29:33get some information from the server the
29:35request will actually go to the closest
29:39node to you drastically reducing the
29:42time it takes to get the response
29:45because you don't have to go all the way
29:47back to the main server so a CDN is
29:49going to introduce two major benefits
29:51the first is it's going to be an
29:53increase in speed for the end user or
29:56you can think of it as reducing lat laty
29:58and it's going to reduce computational
30:01effort on the end server because that
30:05request does not have to make it to the
30:07server so introducing a CDN to our
30:11structure is going to be the next
30:13logical step now what people typically
30:16use cdns for is for caching static
30:19content CSS HTML JavaScript pictures
30:23videos and this is ultimately to
30:25increase the speed of the application so
30:29the end user's experience is much better
30:31while you can potentially cache certain
30:34API responses that don't change very
30:36frequently you're most likely going to
30:39use cdns for this purpose and use a
30:42separate caching layer such as redus to
30:45improve the compute aspect of the back
30:48end that being said if you are trying to
30:51build an app that millions of people are
30:52going to use you will definitely want it
30:55to be fast and performant and one of the
30:58best ways to do that is to get the files
31:00as close to the user as possible so one
31:03of the most popular CDN is cloud flare
31:06and you could add that to your overall
31:09architecture so let's go through the
31:11pros and cons of a CDN the biggest Pro
31:15is going to be a reduction in latency so
31:18from the time the user requests your web
31:21page to when it receives a response it's
31:24going to be much faster with the CDN the
31:27biggest con is it increases the
31:31complexity because now you have to know
31:33how a CDN works you have to worry about
31:37the cach time so for example if data
31:40does change how long until that cash
31:43expires and the new static files are
31:46pulled for the CDN or you'll have to go
31:49in there and manually expire stuff so
31:52there's a lot of complexities introduced
31:54with caching layers so under
31:56complexities we have the time to live
32:00which is the cash time we were just
32:02referring to you also have to worry
32:04about things like cash hit
32:06ratios freaking
32:08fly he terrorizing me this one goes back
32:12to the TTL but you have a risk of
32:14serving outdated data this is also the
32:16easiest way to bring an app global
32:18pretty similar to latency but if you
32:20zoom in on that idea a little bit you
32:22could have servers in different
32:23continents or you can cheat and just use
32:25a CDN to kind of make your app Global
32:28not the perfect solution especially if
32:30you have to make requests to the backend
32:32and the back end is in a different
32:33continent but this is a great first step
32:36because it's relatively simple to set up
32:38it's affordable and it will reduce
32:39latency for users significantly it can
32:42improve security so often cdns have
32:46things like dos protection and other
32:48services to protect your app and if set
32:50up correctly cdns could be good for SEO
32:53so you can read more about that if
32:55you're into SEO but just some that might
32:58be worth knowing about the other one is
33:01it's great for images videos downloads
33:04if your app utilizes a lot of images
33:06videos and downloads you may benefit
33:08from a CDN not only for performance but
33:11also for cost savings so you may need to
33:14look into that in more detail but worth
33:16knowing that this exists if you are
33:18building something like this you can do
33:20some additional Research into cdns so
33:23here is the flow of a request from an
33:26end user's computer they're going to
33:28make a request to our front end now when
33:32they do this they'll just put in a URL
33:34into their browser this is going to
33:37utilize a DNS record so this is kind of
33:40a separate thing but just kind of wanted
33:42to mention it here and that DNS record
33:44is going to associate that URL to some
33:48IP address typically the IP address of
33:51the front end application that they're
33:52trying to access when you're utilizing a
33:54CDN it might be a little bit more
33:56complicated because it might direct them
33:58to the IP address of an edge node
34:00instead of directly to our frontend
34:07server so I'll just write the CDN here
34:11so they'll either get the front end from
34:14one of the edge nodes or directly from
34:16the server and that'll make a request to
34:19the load balancer which is going to
34:21manage our back end and in here we might
34:24have say three
34:25servers and it will manage the traffic
34:28between those and then I'll just say
34:30back end here so it might look something
34:33like
34:34that you can visualize it however you
34:36need but basically the load balancer
34:39manages the three backend servers in
34:41this scenario and we just have a single
34:43end point for the front end to connect
34:46to the back end and
34:48then each of these will connect to the
34:51same database server so it looks
34:55something like that so far so good now
34:58the next thing that we can introduce is
35:00a caching layer which instead of caching
35:04here we'll cash here and this can reduce
35:08the number of requests to the database
35:10because we might be retrieving the same
35:11stuff over and over and over again so we
35:13get to store it in this cach layer in
35:15memory which will increase the overall
35:18speed but also reduce the computation
35:21needed at the database level so as
35:23mentioned the next thing you can do to
35:24improve both the performance and the
35:26scale of your system is to add in
35:29caching between the API backend server
35:32and the database so it might look
35:34something like this where you have the
35:38database and then you have a cash
35:42layer and this will act as an
35:44intermediary between the server and the
35:46database so a request to the database
35:49will first check the cash for that value
35:52so the biggest cache that you're
35:53probably going to come across is reddis
35:56there's also M cach and probably some
35:58others but this is an inmemory thing so
36:01this will be in
36:05Ram but you don't need a lot of ram to
36:09cach the majority of requests it's often
36:13the case that there is some natural data
36:16that gets retrieved a lot more
36:18frequently than other data so it might
36:21be the case that 80% of the requests are
36:25retrieving 20% of the dat
36:28so in that situation we could put the
36:3120% in Ram which is substantially
36:35faster and reduce the total of request
36:39counts to the database by 80% so not
36:43only do you get a lot more speed but you
36:45massively reduce the amount of computing
36:47power that the database layer needs now
36:50we talked about how caches work we said
36:53that when some value is retrieved from
36:55the database the first time it will pass
36:58through the cache grab the data and then
37:01on the way back store that data in the
37:03cache the other thing you need to be
37:05aware of is if you update this data
37:08let's say we updated it to a six we
37:10would need to write our application code
37:12so that when we make that update we also
37:15update the cach or clear the cach so
37:19that the cach doesn't have outdated data
37:23so again an extra complexity you need to
37:25be concerned with that's an easy way of
37:27doing it is just whenever you update the
37:28data in the database also update the
37:30cache or clear the
37:32cach otherwise there could be out of
37:35sync data here which in some
37:36applications that's not a huge deal
37:38especially if the time that the data
37:40stays in the cache is not forever right
37:43if it expires every so often then this
37:46will basically just lag behind the
37:48database a little bit but just updating
37:51both at once is a really good way to
37:53keep those in sync now an in-memory
37:55cache does store data but it's all
37:57volatile because it's in Ram meaning if
38:00the system shuts down and comes back
38:03online the data is going to be gone it's
38:06not stored on disk that's usually fine
38:08because we can just get the data from
38:10the database again it would just be a
38:12temporary performance hit because nobody
38:15would be hitting the cached data they
38:17would have to make the request to the
38:19database you can scale the cash
38:21horizontally and have less of concerns
38:24with data issues but you can look into
38:27into all of the details on reddis
38:29clustering and the different
38:30configurations you can do but I would
38:32horizontally scale the cache before I
38:35horizontally scaled the database layer
38:37unless you're working with a database
38:39that is very good at horizontal scaling
38:42which we'll talk about here in just a
38:44moment after we get through the cach
38:46Section but you can scale this cach as
38:49needed to support the number of requests
38:51from the API so this does add an
38:54additional layer and you might think oh
38:56that would cost more but based on the
38:58way the cash works it's probably going
39:00to end up being cheaper and
39:02substantially faster to use a cash
39:05versus just making a request straight to
39:07the database every time and you can also
39:09really consider situations where perhaps
39:12there are some complex joins or some
39:14things going on behind the scenes for a
39:16particular query and caching the result
39:19of that is going to save a lot of
39:21computation because you're not going to
39:23have to make those complex joints over
39:25and over again so let's talk about the
39:27pros and cons the pros it is going to
39:31reduce the amount of compute needed for
39:34the
39:35database it's going to improve the
39:38overall speed of your application or
39:41reduce latency it's probably going to
39:43end up being cheaper the major cons is
39:46an increase in complexity so all of the
39:49issues that we were talking about with
39:50the CDN is going to apply here but it's
39:53worse because it actually requires us to
39:56do different stuff in our code such as
39:58write to the cach or read from the cache
40:01if we update something in the database
40:03we might need to update the cach or
40:04clear the cache all of this stuff has to
40:06be done in code where the CDN is a bit
40:08more automatic so it's going to increase
40:11the development
40:13efforts which could actually cost money
40:17because if you're paying for development
40:20time and they have to write more code or
40:22the system is more confusing or complex
40:25that's going to take more time that's a
40:26pretty good summary of the pros and cons
40:28of a caching layer and now let's talk
40:31about the database layer up until this
40:33point you've done a lot of things to
40:35scale your app and you're probably in a
40:37good
40:38scenario you usually don't have to do
40:41too much with the database Beyond
40:42vertical scaling basically just creating
40:45a really big database server but you
40:47might be concerned about the high
40:49availability aspect and for that you can
40:52have a standin
40:54database so if this is your database
40:56server you can have one that is just
40:59there in the background as a backup if
41:01something goes wrong with your main
41:03database that is one option as an
41:05alternative to having multimaster
41:08clustering which can be fairly complex
41:11you can also clone to read nodes and I'm
41:14kind of glossing over a lot of the
41:16details here because I'm going to have
41:18multiple videos on database systems
41:21coming up so I would just encourage you
41:24to check those videos out for more
41:26details but I just want wanted to
41:27include the basics here to
41:29comprehensively cover scaling and app
41:32these could be read
41:34nodes and it's often the case that an
41:37application is going to do primarily
41:39reads so you might
41:41do just one big write or Master node
41:45server and then have a bunch of
41:46independent databases that are for
41:49reading so you can scale reads and wrs
41:51independently and probably scale reads
41:54higher than the rights because because
41:57you probably have more reads than writes
42:00this may be the setup you use if you're
42:01using a relational database hosted in
42:03RDS for example but you can also use
42:06larger cluster services for relational
42:08databases and you can also look into
42:11multimaster databases especially if you
42:14go into the nosql world that's pretty
42:17common but typically that previous setup
42:19is adequate however if you do get into
42:21multimaster it might be something like
42:24this where you have multiple nodes
42:29and the data is either split up across
42:31these nodes or replicated to each
42:36other so there's various setups for
42:39cluster databases again we're going to
42:41get into that in more detail in upcoming
42:43videos so to summarize the database
42:45layer I would pretty much just scale
42:47vertically as much as humanly possible
42:50before having to scale horizontally
42:53unless you're working with a database
42:54service that does scale horizontally
42:57very nicely and then as for high
42:58availability you could have a
43:00synchronized standin node which could
43:02fill in if the main server goes down
43:04that's just one approach there are
43:06probably many different approaches to
43:08the data layer but most of the time the
43:11issues are going to be at the API level
43:14the caching level or even the CDN static
43:18content and front-end level another
43:20thing you can do with the database layer
43:21is instead of storing everything in one
43:23massive database you could split that
43:25out into smaller more Focus fed
43:27databases so for example you might have
43:29a database just for user authentication
43:31same idea with the backend you can split
43:33out your services into smaller more
43:35focused services and start moving
43:38towards a microservice approach which
43:40you can then more independently scale it
43:43does always introduce complexities
43:44because you'll have to worry about
43:45communication between the microservices
43:48but as a general rule of thumb if a
43:51service starts to get too many users you
43:54can split it out horizontally or grow
43:56vertically or split the service into
43:58smaller microservices and scale those
44:02independently sort of how we did that
44:04from a monolith to a three tier you
44:07could do the same idea just more
44:08granular so let me just do a quick
44:10overview of what our full architecture
44:12will be kind of something like this so
44:14we have the end user making a request
44:18and this will get the front end or some
44:22cached version from the
44:25CDN this will make request to the load
44:30balancer which will hit the API servers
44:34and then you don't really go through a
44:37cache to get to the database it's more
44:39kind of like
44:42this where we have the cach and then we
44:46have the
44:49database because you can think of the
44:50cach is a data store kind of like a
44:52database it's just that it uses memory
44:55instead of the disc
44:58so in code you would check the
45:00cache and then you would go to the
45:03database if it wasn't found so there's a
45:06million different ways of visualizing it
45:07so apologies if I'm changing it but I
45:10don't want to give you the impression
45:11that there's only one visual correct way
45:15of representing this information I also
45:17forgot an e so there we go so that is an
45:20overview of scaling and app we're going
45:22to cover a lot more system design topics
45:24and get into some of the information we
45:26talked about here here in much more
45:27detail so stay tuned for those upcoming
45:30lessons again as a reminder if you want
45:32the backend mind map or you're looking
45:35for more involved mentorship check out
45:37the links down below that will get you
45:39started to get into a software
45:41engineering role and get some more
45:43practical guidance on things like this
45:46so stay tuned for upcoming content I'll
45:47see you in the next one

About this transcript

This page contains the full transcript of Scale an App to Millions of Users - System Design by Caleb Curry, generated from the public captions YouTube serves with the video. The transcript has 7,730 words across 1,095 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.