YT YouTube2Text

YouTube2Text → Transcripts → CS75 (Summer 2012) Lecture 9 Scalability Harvard Web Development David Malan

CS75 (Summer 2012) Lecture 9 Scalability Harvard Web Development David Malan — Transcript

by archn · 19,464 words · 2,766 segments · language en · Watch on YouTube

Open in transcriber Summarize with AI Download SRT/VTT

Full transcript

0:09welcome back to computer science s75
0:11this is lecture nine our very last it
0:13has been a pleasure having everyone in
0:15the course this semester um so tonight
0:17we talk about scalability so we try to
0:19revisit some of the topics that we
0:20looked at earlier in the semester and
0:22think about how we can deploy
0:24applications not just on say a virtual
0:26machine on your laptop or desktop as
0:28we've been doing with the appliance but
0:30you can scale to servers on the internet
0:32and indeed multiple servers on the
0:33internet so that you can handle hundreds
0:35or thousands or tens of thousands or
0:37even more than that in theory so some of
0:40the issues that we'll inevitably
0:42encounter is how to go about doing this
0:44so when it comes time to put something
0:46on the Internet recall from lecture zero
0:47that we talked about web host so this is
0:49by no means a list of recommendations
0:51per se it's just some representative
0:53ones that we happen to recommend um if
0:55only because the teaching fellows and I
0:57have had prior experiences with these
0:59particular vendors um but you can Google
1:01around and you can see that there's many
1:02many many different options these days
1:04however among the takeaways hopefully
1:06from the summer thus far has been what
1:09kinds of featur should you be looking
1:10for or expecting minimally in any web
1:13hosting company that you might
1:16choose and in fact not all of these even
1:19have those features
1:21necessarily
1:25accessibility example I have
1:28Aver Daddy my access isos Ser to have s
1:34to the post interesting okay good so if
1:37you're if you're country or your work or
1:39really any network that you happen to be
1:41on or people that you know happen to be
1:42on block access to certain IP ranges
1:45among them for instance God daddies in
1:47this case YouTube is a popular thing to
1:49block Facebook is a popular thing to
1:50block um that can be a sticking point so
1:53doing a bit of do diligence or testing
1:55first can be a good thing what else
1:58should you look for in a hosting company
2:01company Isaac SFTP good SFTP in contrast
2:06with what FTP FTP and why because ft
2:10SFTP is secure okay good so the S
2:13literally stands for secure and what
2:14that means is that all of your traffic
2:16is encrypted and this maybe isn't a big
2:18deal for the files that you're
2:19transferring because after all if you're
2:21uploading like gifs and jpegs and video
2:23files that are meant to be downloaded by
2:25people on the web well then who really
2:26cares if you encrypt those between your
2:28machine and the server but it is
2:30important to have what data
2:32encrypted Jack usernames passwords
2:35anything of exactly usernames and
2:37passwords right I mean that's one of the
2:39biggest failings of something like FTP
2:41which granted is a fairly dated or older
2:43protocol is that it sends also your
2:45username and password in the clear which
2:47means anyone sniffing wirelessly around
2:49you anyone sniffing the wi Network
2:51between point A and B can see in the
2:53clear what your username and password
2:54are yeah we looked at some hosting
2:57companies that offer unlimited
2:59everything for a really low price that
3:01could be due to um um virtual hosting
3:06and if you want to implement A system
3:08that can grow by itself you maybe don't
3:10want to share the same computers the
3:12same the same server with many others
3:15good so dream host in particular I think
3:17we pulled up their feature list and it
3:19was ridiculous how many features they
3:21give you unlimited bandwidth unlimited
3:23storage space unlimited Ram or something
3:25like that and that just can't possibly
3:27be uh real if you're only paying like
3:31$9.95 a month so there's some catch and
3:33in general the catch is that they're
3:35making that offer to you and to 100 to
3:37hundreds of other customers all of whom
3:40might be on that same machine so you're
3:42contending now for resources and the
3:43reality is they're probably banking in
3:45the fact that 90 something per of their
3:46customers don't need that many resources
3:49but for the one person or two persons
3:50that do probably going the way of a
3:52shared host is not necessarily in your
3:54best interest certainly not something to
3:55try to build a bigger business on and so
3:58there's alternatives to things like web
3:59hosting companies there's vpss a virtual
4:02private server that you can essentially
4:05rent for yourself and what's the
4:07fundamental distinction between a VPS
4:09and a shared web
4:11post Axel well VPS is like our Linux
4:16it's a it's it's virtual own machine
4:18running on a box but it's it's
4:20completely its own system okay good well
4:22and so be to be clear the operating
4:24system is largely irrelevant since dream
4:26host and shared web host could also be
4:27running Fedora or any operating system
4:29but what's key is that you get your own
4:31copy of Fedora or Ubuntu or whatever
4:34operating system they happen to be
4:35running because in the world of VPS is
4:37what they do is they take generally a
4:38super fast server with lots of ram lots
4:40of CPU lots of dis space and they chop
4:43it up into the illusion of multiple
4:45servers using something called a
4:46hypervisor something like a product from
4:48VMware or Citrix or other companies and
4:52even open source providers have
4:53platforms that allow you to do this run
4:56multiple uh virtual machines on a
4:58physical machine and so in this world
5:00you're still sharing resources but
5:02different typ uh in a different way
5:04you're instead getting some slice of
5:07Hardware to yourself and no one else has
5:10user accounts on that particular virtual
5:12machine now with that said the system
5:14administrators the owners of the VPS
5:17company they themselves depending on
5:19what hypervisor they're using they might
5:21actually have access to your virtual
5:22machine and to your files and frankly if
5:24you have physical access to the machine
5:26you undoubtedly have access to your
5:28files because they can always reboot the
5:30virtual machine for instance in what's
5:32called single user mode or diagnostic
5:34mode and at that point it's not they're
5:35not even prompted for a root password so
5:37realize that even when you're using a
5:39VPS your data might be more secure more
5:42private from other customers but
5:44definitely not the web hosting company
5:46itself if you want even more privacy
5:48than that you're probably going to have
5:49to operate your own servers that only
5:51you or your colleagues have physical
5:53access to so here's just a list of some
5:55of the popular VPS companies there is
5:57one catch in order to get these
5:59additional features or these uh
6:01properties in a VPS you generally pay
6:04more so instead of $10 a month $20 a
6:06month you're probably starting it $50 a
6:08month or something like that maybe even
6:10in the hundreds depending on how many
6:12how much you want in the way of um uh
6:15resources and toward the end of today
6:17we'll talk about one particular vendor
6:20of vpss um namely Amazon uh web services
6:24Amazon ec2 their elastic compute Cloud
6:27that essentially lets you self-service
6:29and spawn as many virtual machines as
6:31you want so long as you're willing to
6:33pay some number of cents per minute to
6:35have that virtual machine up and running
6:37and it's actually a wonderful way to uh
6:39plan for uh unexpected growth because
6:43you can even automate the process of
6:44spawning more web servers more database
6:46servers if you happen to suddenly get
6:48popular even overnight or because you've
6:50been slash dotted or posted on Reddit or
6:52the like and then you can have those
6:54machines automatically power off when
6:56interest in your product or website has
6:58started to subside all right so how
7:01suppose you are the fortunate sufferer
7:04of a good problem which is that your
7:05website all of a sudden is super super
7:07popular and this website has maybe some
7:09static web content HTML files gifs and
7:12the like Dynamic content like PHP code
7:15uh maybe even some database stuff maybe
7:18some uh database stuff like MySQL well
7:22how do you go about scaling your site so
7:24that it can handle more users well the
7:28most straightforward approach which is
7:29generally what's called vertical scaling
7:31vertical in the sense that well if
7:33you're running low on Ram or you're kind
7:35of exhausting your available CPU Cycles
7:38or you're running low on disk space
7:40what's the easiest most obvious solution
7:42to that
7:43problem AEL get a better processor and
7:46more RAM good get more RAM more
7:48processor more disk space and just throw
7:50resources or equivalently money at the
7:52problem unfortunately there is a catch
7:54here there's sort of a a seiling on what
7:57you can do why why is vertical scaling
8:00not necessarily a full solution well you
8:04can only upgrade one machine so much
8:06after a while you can't upgrade yeah
8:08exactly there's some real world
8:09constraints here where you can only buy
8:11a machine that's you know maybe 3
8:13gigahertz these days and maybe only has
8:16a few a handful of maybe a couple dozen
8:18CPUs or cores but at some point you're
8:22either going to exhaust your own
8:23financial resources or just the
8:25state-of-the-art in technology because
8:27just the world hasn't made a machine
8:29that has as many resources as you need
8:31so you need to get a little smarter but
8:33at least within here you have some
8:34discretion so in terms of CPUs these
8:37days most servers have at least two CPUs
8:41sometimes three or four or more and in
8:43turn each of those CPUs typically has
8:45multiple cores in fact most of the
8:46laptops you guys have here these days
8:48are generally at least dual core
8:51sometimes even quad core which means
8:53that you effectively have the equivalent
8:54of four CPUs or four brains inside of
8:57your computer even though they're all
8:58inside of the same chip essentially what
9:01does that mean concretely it means if
9:03you have a quad core machine you can
9:05your computer can literally do four
9:07things at once whereas in yesterday year
9:09when you had single core single CPU
9:11machines they could only do one thing at
9:13a time and even though we humans seem to
9:16think that you're simultaneously
9:18printing something you're pulling up a
9:19Google map you're getting email and all
9:21this stuff seems to be happening
9:22simultaneously the reality is the
9:24operating system is scheduling each of
9:26those programs to get just a Split
9:28Second of CPU time before giving another
9:31program then another program a split
9:33second of CPU time and we humans are
9:35just so slow relative to today's uh
9:37processors that we don't even notice
9:39that things are actually happening
9:40serially as opposed to uh in parallel
9:43but when you actually have quad core
9:45especially in a server that means
9:47whereas in yesterday year with a single
9:49core machine you could handle one web
9:51request at a time now for instance you
9:53could handle at least four at a time
9:55truly in parallel and even then a server
9:58will typically spawn what are called
9:59multiple processes or multiple threads
10:01so in reality you can at least give the
10:04impression that you're handling many
10:06more than even four requests per second
10:08so in short machines these days have
10:10gotten more and more and more CPUs as
10:13well as more cores and yet the funny
10:15thing here is that we humans also aren't
10:17very good at figuring out what to do
10:19with all of this available Hardware
10:21right most of you even if you have a
10:22dual core machine you don't really need
10:24a dual core machine to check your mail
10:26or to like write an essay for school
10:28right you were able to do that 5 10
10:29years ago with far fewer computational
10:31resources now In fairness there's bloat
10:34and software and Mac OS and windows and
10:36office are just getting bigger and
10:37bigger so we're using those resources
10:39but one of the really nice results of
10:43this trend toward more and more
10:44computational resources is that the
10:46world has been able all the more easily
10:48to start chopping up bigger servers into
10:51smaller vpss and indeed that's how
10:53Amazon and other Cloud providers so to
10:55speak uh are able to provide people with
10:57the self-service capability ility as
10:59we'll discuss a bit later so within
11:01these things there are few things you
11:03have discretion over uh if you've ever
11:05built a computer you might be familiar
11:07with uh parallel ATA or IDE or SATA or
11:12SAS anyone Axel what are these refer to
11:15yeah SATA has to do with hard drives
11:17okay good so SATA has to do with hard
11:19drives in fact all three of those have
11:20to do with hard drives years ago
11:22parallel ATA or IDE hard drives were
11:25very much in Vogue they you might still
11:27have them in older computers these days
11:29desktop computers uh pretty much but you
11:31wouldn't buy a new uh parallel ATA Drive
11:34these days instead you'd most likely get
11:35a SATA drive whether 3.5 in for a
11:38desktop or 2 and 1 12 in for a laptop
11:41and if you have servers or you have lots
11:44of money and a fancy desktop computer
11:46you can go with the SAS Drive um SAS is
11:49serial attached scuzzy and this really
11:52just boils down to faster for instance
11:54whereas parallel ATA and SATA drives
11:56typically ran at 7200 RPMs per minute uh
12:01revolutions per minute um SAS drives
12:03anyone know what they typically spin
12:06at and for those unfamiliar inside of a
12:09mechanical hard drive there is a one or
12:11more metal platters that literally spins
12:13much like an old school record player
12:16where the bits are now stored what what
12:19speed does a SAS Drive spin it's more
12:21than 7200 RPM Excell is it 15,000 uh
12:25yeah so 15,000 is where um they would
12:27typically perform sometimes 10,000 but
12:2915,000 so just twice as fast so that
12:31alone gives you a bit of a speed bump of
12:33course it comes at a price literally
12:35more money but that's one way of
12:37speeding things up so in oftentimes what
12:39people will do is for a given website
12:42that they're creating if it has a
12:43database databases tend to write to disk
12:46quite a bit right every Facebook update
12:48requires writing to disk and then
12:50reading back out some number of times so
12:52really where you might be touching disc
12:54a whole lot people can throw things like
12:56SAS drives um in their database so that
12:59their data can be read or written even
13:01more quickly and what's even faster than
13:04um mechanical drives these days Axel
13:07solid state drives yeah solid state
13:09drives ssds which have no moving parts
13:11and as a result electrically perform
13:13much better than mechanical drives but
13:15those two cost more money and they tend
13:17to be smaller in size so whereas you can
13:19buy like a 4 terabyte uh SATA drive
13:23these days 3 and 1/2 in for your desktop
13:25you can buy maximally a 768 G
13:29SSD these days for a lot more money
13:32typically all right
13:35so let's skip uh let's skip raid for now
13:41so horizontal scaling so this is in
13:43contrast with what we just discussed
13:44throwing money and throwing more of
13:46everything at a problem horizontal
13:48scaling is sort of accepting the fact
13:50that there's going to be the ceiling
13:51eventually so why don't we instead
13:53architect our system in such a way that
13:55we're not going to hit that rather we
13:57can kind of stay below loow it by even
14:00using not state-of-the-art hardware and
14:01the F most expensive stuff we can buy
14:03but cheaper Hardware servers that might
14:05be a few years old or at least are not
14:08uh the topof the line so that they'll be
14:10less expensive so rather than get few or
14:13one really good machine why don't we get
14:15a bunch of slower or at least cheaper
14:18machines instead plural number of
14:20machines so this is just a picture of a
14:22data center which is just meant to
14:23conure up the idea of scaling
14:25horizontally and actually using multiple
14:27servers to build out your topology but
14:30what does this actually mean well if you
14:34have a whole bunch of servers now
14:36instead of just one how what's the
14:39relationship now with lecture zero where
14:41we talked about HTTP and DNS right the
14:45world was very simple a few weeks ago
14:47when you had a server and it had an IP
14:49address and that IP address might have a
14:51domain name or host name associated with
14:53it and we told that story of what
14:54happens when you type in something.com
14:56enter on your laptop and you get back
14:58the pages on that single server but now
15:00we have a problem if we have a whole
15:02Isles worth of web servers Axel well
15:05you're going to have to have a
15:09way
15:13okayest okay good so now if we get an
15:15inbound HTTP request we somehow want to
15:18distribute that request over all of the
15:21various web servers that we might have
15:23whether it's two web servers or 200 the
15:25problem really is still the same if it's
15:27more than one we have to somehow figure
15:29that out so let me put up a fairly
15:32generic picture here let me flip fast
15:34those to this guy here so if we have a
15:37whole bunch of servers on the bottom
15:39here server one two dot dot dot and on
15:41the top we have some number of clients
15:43just random people on the internet we
15:45need to interpose now some kind of black
15:47box that's generally called a load
15:49balancer depicted here as a rectangle so
15:52that the traffic coming from people on
15:54the Internet is somehow distributed or
15:56balanced across our various backend
15:59servers so to speak so it might still be
16:02the case that server one and two and so
16:04forth have unique IP addresses but now
16:08when a user types in something.com and
16:10hit hits
16:12enter what IP address should we
16:15return how do we go about achieving the
16:18equivalent of this man in the middle who
16:20can somehow balance load across all in
16:23servers well let
16:26the return the IP addresses the load
16:29balancer send the request to the load
16:31balancer and then let the load balancer
16:34handle which computer send it to okay
16:36good so instead of in DNS returning the
16:39IP address of server one or server 2 or
16:41server dot dot dot you could instead
16:43return the equivalent of the IP address
16:44of this black box the load balancer and
16:47then let the load balancer figure out
16:49how to actually route data to those
16:52backend servers so that's actually
16:53pretty clean so now if the load balancer
16:55has a public IP address the backend
16:57server is now technically you don't even
16:59need public IP addresses they can
17:00instead have private addresses and what
17:03was the key distinction between public
17:04and private IPS back in lecture
17:09zero any anyone over here yeah Le the
17:13rest of the world see private exactly so
17:17the rest of the World by definition of
17:18private can't see private IP addresses
17:21then so that seems to have some nice uh
17:24privacy properties and that if no one
17:25else can see them they can't address
17:27them so those servers just by nature of
17:29this privacy can't be contacted at least
17:32directly by random adversaries bad guys
17:34on the internet so that's a plus
17:36moreover the world has been running out
17:38of uh version 4 IP addresses the 32-bit
17:41IP address has come into scarcity for
17:43some time now so it's just hard or
17:46sometimes expensive to get enough IP
17:48addresses to handle the various servers
17:50that you might buy so this alleviates
17:51that pressure now we need one IP and not
17:54multiple for our servers because we can
17:56give these backend servers like a a
17:58number like 192.168 which most of you
18:00have in your home routers probably or
18:0210. something or 17216 something all of
18:05those to Mark the start of a private IP
18:08address so that works all right so the
18:10load balancer has its own IP address now
18:12it gets a request from some client on
18:14the internet how using jargon from
18:17lecture zero onward can the load
18:19balancer decide or get that data to one
18:23of the backend servers how could we go
18:26about implementing that actually
18:28well you would probably first want to
18:30figure out which server to send it to so
18:32you want to check um if somebody has
18:35available CPU cycles that they're not
18:38using okay so and once you see that
18:40there's one server with enough CPU
18:42Cycles to handle that request s a local
18:44request the same request but locally
18:46inside your server Network okay to that
18:49machine um get back whatever it is that
18:52the client requested and then the load
18:54balance s excellent so this request
18:57arrives then at the load balance
18:58balancer the load balancer decides to
19:00whom he wants to send this packet server
19:02one or two or dot dot dot and you can
19:04make that decision based on any number
19:06of factor so Axel propose doing it based
19:08on load like who is the busiest versus
19:10the least busy odds are I should send my
19:13request to the least busy server in the
19:15interest of uh optimizing performance
19:18all around so let's assume that there's
19:20some way as Demar demarcated by those
19:22black arrows of like talking to those
19:24backend servers and saying hey how busy
19:25are you let me know so now the load
19:27balancer figures out it wants to send
19:29this particular request to server one so
19:31it sends that request to server one
19:33using similar mechanisms tcpip much like
19:36the packet how it traveled to the load
19:38balancer in the first place the server
19:39then gets the packet does its thing and
19:42says oh they want some HTML file here it
19:44is the response goes to the load
19:46balancer the load balancer then responds
19:48to the client and voila so that works
19:51what are some alternatives to load
19:53balancing based on the
19:56actual load on these server so load in
20:00general refers to how busy a server is
20:02so what's an even simpler approach than
20:04that because that frankly that sounds a
20:05little complex we've not talked at all
20:07about how one device can query another
20:09for uh characteristics like how busy are
20:11you even though it's possible
20:13Axel first let me point out inite that
20:16you need to have every file um every
20:18file the website has on every server so
20:21if you would instead uh say have one
20:23server containing all the HTML and one
20:26running and containing all the P you
20:29would then see well oh okay bad example
20:32one example one server with all the
20:34images and one with all the HTML client
20:36requests an image it sends it to the
20:38image server okay and if it's an HTML
20:41request to the HTML server okay good so
20:43the implication of the previous story
20:44that Axel told is that under this model
20:46server one two and so forth all need to
20:48be identical have the same content which
20:51is nice and that then it doesn't matter
20:53to whom you send the request the
20:54downside is now you're using n times
20:57this much dis space as you might
20:59otherwise need to but that's perhaps the
21:01price you pay for having this redundancy
21:03or to having this uh horizontal
21:05scalability or instead you could have
21:07dedicated servers these are for HTML
21:09these are for gifs these are for movie
21:11files and the like and you could do that
21:13just by having different URLs different
21:15host names this is for instance images.
21:18something.com this is videos.
21:20something.com and then the load balancer
21:22could take into account the host HTTP
21:25header to decide which direction it
21:27should go in so could work for us all
21:29right so what's an even simpler fistic
21:32than asking a backend server how busy
21:34are you right now like if you have no
21:36idea how to do that how instead could we
21:38balance load across an arbitrary number
21:40of
21:42servers think again back to lecture zero
21:45we can do all of this with only lecture
21:48zero under our
21:53belt so let's quickly tell the story I
21:55type in something.com into a browser I
21:57hit enter what
22:01happens Jack we create a packet to send
22:04okay packet to send to whom do we send
22:06it we send it to uh some place that will
22:09determine the IP address where we're
22:11sending it okay good something that will
22:12determine the IP address of where we're
22:14sending it to what's that thing called
22:16uh Isaac what's that called router uh
22:19not router routers get involved
22:22but DNS the DNS server domain name
22:25system server so that server in the
22:26world a whole bunch of of them that
22:29whose purpose in life is to translate
22:30host names to IPS and vice versa so I'm
22:34going to pause the story there that
22:36seems to be an opportunity now for us to
22:39return something yeah could you do some
22:43DNS tricks and return different IP
22:46addresses based on what the user
22:48requested good so this black box this
22:50load balance or maybe it's just a fancy
22:52DNS setup whereby instead of returning
22:54the IP address of the load balancer
22:56itself maybe in instead the DNS server
22:59just Returns the IP address of server
23:01one the first time someone asks for
23:03something.com and the next time someone
23:05requests something.com it Returns the IP
23:07address of server two followed by server
23:09three followed by dot dot dot and then
23:11wrapping around eventually to server one
23:13again so this is actually generally
23:14called round robin and you can do this
23:17fairly easily this is just a snippet of
23:19a popular DNS server called bind berley
23:22internet name uh name demon I believe is
23:26the D and this just suggests that if you
23:28want to have multiple IP addresses for a
23:30host name called dubdub duub you mention
23:33a which is denotes a record in just
23:35refers to an inp pointer here but a is
23:38the same as in lecture zero and then you
23:40just enumerate the IP address one after
23:42the other and by default this particular
23:45uh DNS server very popular one
23:48bind will return a different IP address
23:52for each request so that's nice it's
23:54simple again uses only some knowledge
23:56from lecture zero even though granted
23:58you have to know how to configure the
23:59DNS server but you don't need any fancy
24:01bidirectional communication with the
24:03backend servers in this model so that's
24:04nice but there's a price we pay for this
24:08Simplicity if we only do round robin
24:11where again we just spit out a different
24:13IP address each time and let me make
24:15this more concrete just so this isn't
24:17quite as abstract let me open up a
24:19terminal program here and do NS lookup
24:22for name server lookup of google.com
24:25this is exactly what Google does at
24:27least part their their load balancing
24:30solution is more sophisticated than this
24:32list suggests but indeed Google's DNS
24:35server returns multiple IP addresses
24:36each time so
24:39if this is so simple to implement what's
24:43the
24:45catch uh Axel well it's not a very smart
24:48solution
24:49because just there could be the case
24:52could be that one server gets all the
24:55really tough and hard requests that a
24:57lot processing power and the other ones
24:59just get the Fe the HTML files and
25:01there's no way to know that good so just
25:03by bad luck one of the servers could
25:05just get a really a real power user
25:08someone who's really doing something
25:10computationally difficult like I don't
25:12know uh what's a good examp sending lots
25:16of mail whereas someone else is just
25:17kind of logging in and poking around at
25:19a much slower rate and you we could come
25:21up with even more sophisticated examples
25:22than that but over time server one might
25:25just happen to get more heavyweight
25:28users than other servers so what's the
25:30implication well round robin is still
25:33going to keep sending more and more
25:35users to that server one nth of the time
25:38just by nature of round robin so that's
25:41not so good um what
25:44else causes
25:47problems here or what else breaks
25:53potentially so back to the lecture zero
25:56story I type something.com I hit enter
25:58my browser sends a request to the DNS
26:00server or my operating system sends a
26:02request to the DNS server gets the IP
26:04address and in this model it's the IP
26:06address of one of these servers then I
26:09send my packet as Jack proposed to that
26:11particular server get back a response
26:14stories ends but then a few seconds
26:16later I visit another link on
26:18something.com and hit enter which part
26:20of the story now
26:22changes Jack is it now going to send to
26:25the same server request do it send to a
26:28new server good
26:31question where it's coming from so what
26:34how does the story change and that'll
26:36give us the answer here Axel well it has
26:38to send it to
26:40Aver oh ideally yes so if you want a
26:43truly uniform distribution across all n
26:45servers then the DNS server has to
26:47return another response and I'd argue
26:48the DNS server will return a different
26:51response the next time it is queried but
26:54oh oh it's not queried why because it's
26:58saved or no but
27:01it's IP address
27:03isor Open Session it's it's really
27:06useless to query the DNS server for the
27:08same thing over and over again good so
27:09recall these caches back in lecture Z we
27:12talked about the implications of the
27:14good parts of caching whereby as ael's
27:16proposing there's no reason for Chrome
27:18or IE to send the same DNS request every
27:21single time you click a link on
27:23something.com that would just be a waste
27:24of time um you're going to lose some
27:26number of milliseconds every time time
27:27that happens or worse yet a second or
27:29two so instead your operating system
27:31typically caches these responses your
27:33browser typically caches these responses
27:36as well and so you just don't need to do
27:38those lookups so if you do happen to be
27:40that power user who's doing a heck of a
27:42lot of work of whatever sort on server
27:45one the next guy is going to be sent to
27:47server two not you with your subsequent
27:50requests so caching too can contribute
27:53to a disproportionate amount of load on
27:55certain servers largely and due to to
27:57bad luck indeed in DNS we didn't spend
28:00much time on this particular detail but
28:02there's typically expiration times ttls
28:04time to live values associated with an
28:07answer from a DNS server and that's
28:09typically an hour or five minutes or a
28:12day it totally depends on who controls
28:14the DNS server what that value is but
28:16that suggests too that if you are this
28:18power user on server one it might be a
28:20few minutes or hours or even days until
28:23you get assigned to some other server
28:26simply because you're TTL has expired by
28:29then so it's nice and simple we can do
28:31it with a simple configuration change
28:32but it doesn't necessarily solve all of
28:34our problems so in fact the approach
28:36Axel proposed first is actually pretty
28:39good whereby you don't use dns-based
28:41round robin rather a more sophisticated
28:44approach would be to let the load
28:45balancer decide to whom to send you in
28:48the back end and the load balancer can
28:49make that decision using any number of
28:51her istics it could even use round robin
28:54or Randomness because at that point you
28:56don't have to worry about caching issues
28:58because the DNS server has only returned
29:00one IP so but that still leads you to
29:02the risk that you'll be putting too much
29:04load on some server so we could take for
29:06instance server load into account at
29:08that point but there is something else
29:10that breaks if we fast forward
29:11mid-semester to when we started talking
29:13about cookies and
29:16HTTP and sessions in
29:19PHP to spark discussion I propose that
29:23sessions have just broken in PHP if our
29:26backend servers are PHP based
29:30websites and they are using the session
29:33super Global load balancing seems to
29:36break this model why Jack because now
29:39the different different servers have
29:41different people sessions so Al one
29:43might have my session if I then am
29:46redirected from server one to server two
29:49server two might not have my session
29:51exactly so sessions recall tend to be
29:54specific to the given machine we saw
29:56examples involving sltm which is a
29:58temporary directory on a Linux system
30:00where sessions are typically saved as uh
30:02text files serialized text files so that
30:05means though that your session might be
30:06sitting on the hard drive of server one
30:08and yet if by a random chance you are
30:10sent via round robin into server two or
30:12server three instead of server one in
30:15the worst case you're going to see the
30:16same website but you're going to be told
30:18to log in again for instance because
30:20that Server doesn't know that you've
30:21logged in more okay fine you kind of
30:23bite your tongue and you type in your
30:25username and password again and hit
30:26enter and suppose you're a really good
30:29sport and you do this for all end
30:30servers you have no idea why
30:32something.com keeps prompting you to log
30:34in but eventually you will have a
30:35session cookie on all of those servers
30:37the catch then though is that if
30:39something.com is an e-commerce site and
30:41you're adding things to your shopping
30:42cart now you literally have put a book
30:44in your cart over here a different book
30:46in this card a different book in this
30:48card and when you check out you can't
30:49check out the Aggregates so this is a
30:52very non-trivial problem now Axel um but
30:56this wouldn't happen if you
30:57if you had dedicated machines
30:59Distributing dedicated files one machine
31:01running PHP and then one maching all the
31:04images very true so if we have
31:06horizontally scaled in the sense that we
31:08factored out disparate services this is
31:10our PHP server this is our uh GIF server
31:13this is our video server then indeed
31:15this problem would not arise because
31:16presumably all the the PHP traffic would
31:19get routed to the PHP server but an
31:21obvious push back to that solution is
31:24what you're Isaac well if one of them
31:27crashes you lose all images okay good so
31:31there's no redundancy which is not good
31:33for uptime if anything breaks Axel and
31:35also well at some point in time if you
31:39get popular enough that one PHP server
31:41is not going to be able to handle
31:42everything
31:43good then the story is the same as soon
31:45as you get popular you have too much
31:47load for a single PHP server then we
31:49have to solve this problem anyway so how
31:52do we go about solving this problem this
31:54seems to be a real pain this one
31:58and to be clear the problem now is that
31:59in as much as sessions are typically
32:02implemented per server in the form of
32:04like a text file like we saw in sltm
32:08then you can't really use round robin
32:10you can't really use loow true load
32:13balancing taking into account each
32:14server's load because you need to make
32:16sure that Alice if she's initially sent
32:18to server one subsequently gets sent to
32:20server one again and again and again for
32:23at least you know an hour or a day or
32:25some amount of time so that her is
32:27useful
32:34Jack started there and excellent yes so
32:37absolutely we could just continue this
32:39idea of factorization and factor out not
32:42the various types of files but a service
32:45like session States so if we instead had
32:47a file server you know like a big
32:49external hard drive so to speak that is
32:52connected to all of the servers one and
32:54two and three so that anytime they store
32:57session data they store it there instead
33:00of on their own hard drive then this way
33:02we could share state so indeed that
33:04could be a solution here
33:06Axel I I don't know if a load balancer
33:09has that function but maybe the instead
33:11of having an extra server that all the
33:13other servers need query load balancer
33:16because all traffic goes through that
33:17anyhow but if the load balancer session
33:20okay so that's not bad at all so we
33:22already have a man in the middle here
33:23it's a black box but there's no reason
33:25it couldn't be a server with hard dis
33:27so want to put the sessions on the load
33:29balancer that could absolutely work so
33:31let me be difficult then and whether we
33:33put the sessions in the load balancer or
33:35whatever that it's no longer a load
33:36balancer then it's obviously doing more
33:38it's more of a a server that happens to
33:40be balancing load and storing sessions
33:42whether we put sessions there in that
33:44black box or elsewhere in a new box on
33:46the screen we seem to have introduced uh
33:51weakness now in our Network topology
33:54because what if that machine breaks
33:57it would seem to be the case that even
33:59though we have n servers which in theory
34:01those guys are never all going to die at
34:03once assuming that it's not the power
34:05electricity or something stupid like
34:06that that's somehow related to all of
34:08them but odds are they're not all just
34:10going to up and die simultaneously so we
34:12have really good redundancy in our
34:14server model right now but as soon as we
34:15introduce just a database or file server
34:18for our sessions if that guy dies then
34:21what was the point of spending all this
34:23money on all these backend servers our
34:25whole site goes down because we have no
34:27ability to remember that people are
34:28logged in if we can't remember they're
34:29logged in no one can buy anything so how
34:31do we fix that problem so we've solved
34:33one problem but if you think of that
34:35sort of old uh visual where you have
34:38like a garden hose with lots of leaks in
34:39it and you plug one of them with one
34:41hand all of a sudden a new leak Springs
34:43up elsewhere that's kind of what's
34:44happened here we've solved the problem
34:46of shared state but now we've sacrificed
34:50some uh some robustness some redundancy
34:54how do we now fix the
34:55ladder
35:03Exel probably not
35:14the okay good so we could just use a
35:17sort of different approach to storing
35:19our data and rather than just store it
35:21on the hard disk as usual we could use
35:23something called raid so actually this
35:24is a good way to tie in the thing we
35:26skipped over a moment ago let me just
35:29pull up something to write on
35:31here
35:35uh
35:37so redundant array of independent discs
35:40is a technology more succinctly known as
35:43raid raid can actually be used in
35:44desktop computers these days um even
35:47though it's not all that common some
35:48companies like Dell and apple make it
35:51relatively easy to use Raid on your
35:53system and what does this mean well raid
35:55can come in a few different forms
35:56there's something called raid zero
35:58there's something called raid 1 there's
36:00something called raid five there's
36:02something called raid six there's
36:03something called RAID 10 and there's
36:05even more but these are some of the
36:07simplest ones to talk about so all of
36:09these techn all of these variants of
36:12raid assume that you have multiple hard
36:14drives in your computer for different
36:16purposes potentially so in the world of
36:18raid one of raid zero you typically have
36:21uh two hard drives that are of identical
36:23size terabyte 2 terabytes 512 me G
36:27whatever it is two identical hard drives
36:30and you do what's called stripe data
36:32across them whereby every time the
36:35operating system wants to save a file
36:36especially big files it will first write
36:39to this drive a bit then to this one
36:41then to this one then to this one the
36:43motivation being these hard drives are
36:44typically large and mechanical with
36:46spinning platters like we discussed
36:48earlier and so it might take this guy a
36:50little while to write out some number of
36:53bits now it's going to be split second
36:54in reality but that's a split second we
36:56don't really have when we're trying to
36:57service lots of users so striping allows
37:00me to write some data here then here
37:02then here then here then here then here
37:04effectively doubling the speed at which
37:06I can write files especially large ones
37:07to diss so raid zero is nice for
37:10performance however raid one gives you a
37:12very different property with raid one
37:14you still have two hard drives but you
37:16mirror data so to speak across them so
37:19that anytime you write out a file you
37:21store it both places simultaneously
37:24there's a bit of performance overhead to
37:25writing it in two places albe it in
37:27parallel but the upside now is that
37:30either of these drives can die and your
37:32data is still perfectly intact and it's
37:34actually an amazing technology because
37:37if even if you just have this in your
37:38desktop computer you have two drives one
37:40of them dies just because of bad luck it
37:42was there was a defect or it's multiple
37:44years old and it just upped and died so
37:46long as the other one is still working
37:47the theory behind raid is that you can
37:49then run to the store buy another hard
37:51drive that's at the same size or bigger
37:54plug it into your computer boot back up
37:57and most typically automatically The
38:00Raid array will rebuild itself whereby
38:03all of the data that's on the remaining
38:05drive will copy itself automatically
38:07over to the new one and after a few
38:08minutes or hours you're back to a safer
38:11place whereby now even the other Drive
38:13can up and die sometimes you have to run
38:15a command or choose a menu option to
38:16induce that but typically it's automatic
38:18you can do it even sometimes in some
38:20machines while the computer's still on
38:22so you don't even have to suffer any
38:23downtime so that's great RAID 10 is a
38:26essentially the combination of those two
38:27you typically use four drives and you
38:29have both striping and redundancy so you
38:32sort of get the best of both worlds but
38:33costs you twice as much because you need
38:35twice as many hard diss raid five and
38:37raid six are kind of nice middle grounds
38:40with raid one uh or nice variants of
38:42raid one whereby raid one's kind of
38:43pricey right rather than buy two one
38:46hard drive I literally have to spend
38:48twice as much and get two hard drives
38:50raid five is a little more versatile
38:53whereby I typically have say three
38:55drives four drives five drives but only
38:58one of them is used for redundancy so if
39:00I get five 1 tbte drives I have four
39:04terabytes of usable space so I'm only
39:07sacrificing oneth in that case of my
39:10available disc capacity whereas in raid
39:12one you're sacrificing one one over one
39:14half so 50% of it so raid five you just
39:17get better economy of scale whereby you
39:19can grow bigger and you still have some
39:22redundancy so in raid five if you have
39:24three or four or five hard drives in in
39:26the array one of them can die any of
39:28them you run to the store put in a new
39:30one and you haven't lost any data raid
39:33six is even better what does raid six do
39:36do you think Axel I think two drives can
39:38die exactly in raid six any two drives
39:41can die you still won't have lost any
39:43data and so long as you run to the store
39:44fast enough and put in one or both
39:46drives again you'll be good to go of
39:48course the price you pay with rate six
39:50is literally another hard drive but at
39:51least now you can maybe sleep a bit
39:53better at night knowing that man two of
39:55my hard drives has to die before I have
39:57to really worry about this so these are
40:00really nice Technologies and so as Axel
40:02proposes here the upside of using
40:04something like that in whatever uh file
40:07server we're storing our shared sessions
40:09is we can at least decrease the
40:10probability of downtime at least related
40:12to hard diss unfortunately it still has
40:16a power cord that someone could trip
40:17over or the power supply could die it
40:19still has Ram that could go on the fritz
40:21a motherboard that could die any number
40:23of things could still happen but at
40:25least we can throw redundant inside of
40:27the confines of a single server and this
40:29can definitely help with our uptime and
40:31with our robustness and indeed with
40:33actual servers that you would buy for a
40:35data center not so much the home it's
40:38very common for computers to have not
40:40only multiple hard drives and lots of uh
40:43multiple Banks of RAM and they would
40:45often have multiple power supplies as
40:48well and it's actually a really cool
40:49technology there too if one of your
40:50power supplies dies you can literally
40:52pull it out the machine keeps running
40:54you put in a new one and then it spreads
40:56the amperage across two power supplies
40:59once both are back up and running all
41:01very hot swappable amazing technology
41:03these days and as an aside if you still
41:04own a desktop computer there is no
41:06reason you shouldn't use Raid these days
41:08it is just very good practice since it
41:10will uh allow you to avoid downtime and
41:13data loss with higher
41:15probability okay but someone tripped
41:18over the power cord someone tripped over
41:20both power cords in the case of
41:22redundant power suppli so axle solution
41:24and even mine with redundant power
41:25supply hasn't solved the problem of
41:28shared storage becoming all of a sudden
41:31a single point of failure so what else
41:33could we do to still get the property of
41:37shared state so it doesn't matter what
41:39backend server I end up on but I instead
41:42get I still get the ability to suffer
41:46some downtime well shared storage is can
41:48to come in a bunch of different ways so
41:50we talked really about things as a file
41:51server but this can be incarnated with
41:53very specific Technologies and just to
41:55Rattle them off even though we won't
41:56talk about them in much technical detail
41:58fiber channel FC is a very fast very
42:01expensive technology that you can use in
42:04offices and data centers not so much the
42:06home to provide very fast shared storage
42:08across servers so that's just one type
42:10of file server if you will um I scuzzy
42:13is another technology that uses uh IP
42:16Internet Protocol and uses ethernet
42:18cables to exchange data with servers so
42:20that's a nice somewhat cheaper way of
42:22exchange of having a shared file server
42:25that um uh that can be used by multiple
42:29actually in the case of iuz you
42:30typically use it with single servers um
42:33so let me retract that that is not a
42:34solution to our current cookie problem
42:36um but what about MySQL right we use
42:38that for a couple weeks MySQL seems to
42:42be a nice candidate because it's already
42:43a separate server potentially could not
42:46the backend servers just write their
42:48session objects to a database they
42:51definitely could so just because you're
42:53storing things in a just because we
42:56usually store things like user data and
42:58user generated data in a database
43:00doesn't mean we can't store metadata
43:01like our cookie information as well or
43:04that too comes from users though NFS
43:06Network file system this is just a
43:07protocol that you can use to implement
43:10the idea that Axel proposed of a shared
43:12file system it just means you've got one
43:14server and you're exposing your hard
43:16disk to multiple other computers but
43:18again we haven't really solved the
43:20problem of share of downtime so what's
43:24the most obvious way of mitigating the
43:27risk that your single file server will
43:29go
43:31down Axel keep the cop of session loc
43:35good right if it's not if you don't have
43:38um if you're worried about the one file
43:40server going down well the obvious
43:41solution even though money um and some
43:44technical complexity will just get two
43:46now somehow you have to figure out how
43:47to sync the two so that one has a copy
43:51of the other's data and vice versa so
43:53let's actually come back to that issue
43:55it's generally known as rep application
43:56but it is something we can potentially
43:59achieve but before we segue to um
44:05distribution of things let's finish out
44:08this load balancer question so how do
44:10you go about implementing this black box
44:12well these days you actually have a
44:13bunch of options um in software you can
44:16do things relatively easily with a
44:17browser pointing and clicking using
44:20things like Amazon's elastic load
44:21balancer a scenario for which we'll talk
44:23about a bit later ha proxy High
44:26availability proxy is free open source
44:28software that you can run on a server
44:30that can do load balancing as well using
44:31any either of the heris stics we
44:33discussed earlier round robin or
44:35actually taking load into account
44:36somehow um Linux virtual server LVS is
44:39another free software a piece of
44:41software you can use and then in the
44:42world of Hardware people have made big
44:44business out of load balancers Barracuda
44:47Cisco Citrix F5 are some of the most
44:49popular vendors here um most of whom are
44:51atrociously overpriced for what they do
44:54so case in point like Citrix is a
44:57popular company that sells load
44:58balancers take a guess as to what a load
45:01balancer might cost you these days it's
45:02a highly variable range but there's
45:04different models but take a guess how
45:07much did that black bux cost Isaac in
45:09the thousands definitely in the
45:11thousands
45:14indeed in fact we have a small one so to
45:17speak on a small one relatively speaking
45:19on campus that was
45:22$220,000 and guess what that one's cheap
45:27so you can literally spend on these
45:28kinds of things granted not this is this
45:30is not what the costs that await you
45:31right after the semester ends today um
45:34but $100,000 for a load balancer or even
45:37generally a pair of load balancers so
45:39that either of them can die and the
45:41other one can stay alive so in the world
45:43of Enterprise Hardware these ideas we're
45:45talking about are ridiculously priced
45:47typically because of support contracts
45:49and the like so just realize software is
45:51number one on the list because there are
45:53other ways to achieve this much more
45:55inexpensive indeed for years um one of
45:57the courses I teach we used ha proxy to
46:00balance load because it was so
46:02relatively easy to set up and 100% free
46:04so realize these same ideas can be both
46:06bought and set up uh on your own quite
46:10readily these days all right um let's
46:15pause here and when we come back we'll
46:16take a look at some issues of like
46:18caching of replication in databases and
46:20also how we can speed up PHP a bit let's
46:22take our five minute break
46:24here
46:27all right we are back and I almost
46:29forgot we have one other solution to
46:31this problem of the need for sticky
46:33sessions sticky sessions meaning that
46:36when you visit a website multiple times
46:39your session is somehow preserved even
46:42if there are multiple backend servers so
46:45shared storage was the idea we really
46:47vetted quite a bit and we didn't quite
46:49get to a perfect solution since even
46:52though we factored out the storage and
46:55and put everyone's cookies or session
46:57objects on the same server we feels like
46:59we need some redundancy but we'll come
47:01back to that in the context of MySQL in
47:03just a bit but what about cookies I
47:05propose that cookies themselves could
47:09offer a solution to the problem of
47:12sticky sessions and again Sticky session
47:14means even if you visit a website
47:17multiple times you're still going to end
47:19up with the same session object or more
47:21specifically you're still going to end
47:22up on the same backend server
47:33Axel which server you want to go to in a
47:36cie
47:38okay store everything in cookies but
47:40that's that's store okay yeah so storing
47:43everything in cookie is probably bad
47:45because one then it's really starting to
47:47violate privacy because rather than
47:48store a big key you're going to store
47:50like the isbns of all of the books in
47:52your shopping cart and that might be
47:53fine but feels like your roommates and
47:55family members don't don't need to know
47:56what is in your cookies moreover cookies
47:58typically have a finite size of a few
48:01kilobytes so there's definitely going to
48:02be circumstances in which you just can't
48:03fit everything you want to in the cookie
48:05so you know an interesting idea but
48:07probably not the best so you could store
48:09the ID of the server in a cookie so that
48:12the user the second and third and fourth
48:14times they visit your website as by
48:16following links or coming back some
48:19other time they are going to present the
48:22equivalent of a hand stamp saying hey I
48:24was on backend server one send me there
48:27again so that's a pretty nice idea there
48:29is one at least one downside here what
48:32do you like what do you not like
48:34potentially about this idea of storing
48:35in a cookie that gets put on the user's
48:38browser that they subsequently transmit
48:40back to you the number of or the ID of
48:42the server to which they should be
48:47sent maxel well for one expiration
48:50expiration in what sense well cookie
48:53expires Okay so eventually cookie is
48:56going to expire though we saw a couple
48:58lectures ago we could make it expire in
49:0110 years if we really wanted to and
49:03frankly we're never going to avoid that
49:05even if we had a single server cookies
49:06could eventually expire so at least
49:08that's not a new problem so I'm not too
49:09worried about expiration now because
49:12that's not a problem new to us simply
49:15because of load
49:18balancing does anything not feel right
49:21about storing the ID of the server in
49:23the
49:24cookie
49:32yeah so if we just put like the backend
49:34IP so the private IP address in the
49:36cookie you know what if the IP changes
49:38what if that uh what if the IP changes
49:40so that's a little problematic and it's
49:42also one of these principled things like
49:44you don't really need to reveal to the
49:46world what your IP address scheme is
49:48it's not necessarily something they
49:50could exploit but it's just the whole
49:52world doesn't need to know that moreover
49:54we can implement the same idea by still
49:56storing a cookie on the user's computer
49:57but why don't we take the PHP approach
49:59of let's just store a big random number
50:01and then have the load balancer remember
50:03that that big random number belongs to
50:05server one and this other big random
50:07number belongs to server two and so
50:10forth so a little more work on the load
50:11balancer but in this way then we're
50:13really not putting any state that might
50:17change or might be a little privacy
50:19revealing on the actual user's computer
50:21moreover we also take away the ability
50:23for them to spoof that cookie
50:26just to get access to some other server
50:28now whether or not they could do
50:29anything with that uh trick is unclear
50:31but at least we take the bill away that
50:33ability Al together so there's no
50:35surprises all right so cookies indeed
50:37are something that these black boxes of
50:39load balancers tend to do whereby you
50:41can configure them to insert a cookie
50:43themselves it doesn't just have to be a
50:45back-end web server that generates
50:46cookies the load balancer similarly
50:49could be inserting a cookie with the set
50:51cookie header that the end user then
50:54subsequently sends back so that we can
50:56remember what backend server to send the
50:58user to now if the user has cookies
51:00disabled well then this whole system
51:02breaks down but again so does a lot of
51:04functionality we've discussed thus far
51:06this semester but there are sometimes
51:08some workarounds so a word on PHP PHP
51:12and interpreted languages in general
51:13tend to get a bad WAP for performance
51:15because they tend not to be as high
51:17performing as a compiled language like
51:18C++ or C or the like however there are
51:22some ways to mitigate this there's this
51:23notion of PHP acceleration
51:26whereby you can run a PHP program The
51:29Source Code Through php.exe The
51:32Interpreter on the system and it turns
51:34out that PHP does typically compile that
51:37file in a sense down to something that's
51:39more efficiently executed much like Java
51:41compile something down to something
51:42called bik code but you typically PHP
51:46throws the results of that compilation
51:49away whereby it does it again and again
51:52for every subsequent request however
51:54with very simple
51:56uh with relatively straightforward and
51:58freely available software you can
52:00install a PHP accelerator here are just
52:03four possibilities that essentially
52:05eliminate that discarding of the PHP op
52:09codes and instead keep them around so
52:11another words the first time someone
52:12visits your site the PHP file is going
52:14to be interpreted and some op codes for
52:16performance generated but they're not
52:19going to be thrown away because the next
52:21time you or someone else visits the site
52:23that PHP file is not going to have to be
52:24rep first and reinterpreted the op codes
52:27are just going to be executed so you get
52:28the benefit of some added performance
52:30now the only gotcha is if you ever
52:32change any of your. PHP files you have
52:35to uh throw away the cached op codes but
52:40uh these various tools typically do that
52:42for you python has a similar mechanism
52:44where you'll get py files or your source
52:46code files but pyc files are the
52:49compiled versions that can be executed
52:51more quickly so the same idea as at play
52:54here so this is one of the these things
52:55that is relatively easy and free to
52:58enable and gives you all the more
53:00performance specifically the ability to
53:02handle all the more requests per second
53:04in the context of a PHP based website so
53:07what about caching too caching in
53:09general is a great thing it solved some
53:10of our DNS concerns early on but it
53:12introduced it others because caching can
53:15be a bad thing if some uh value has
53:17changed but you have the old one but
53:19caching can be implemented in the
53:21context of dynamic websites in a few
53:23different ways so I propose that through
53:25HTML through MySQL and through something
53:27called MCD we can achieve some caching
53:30benefits here so this is a screenshot of
53:33one of the most 1990s websites out there
53:36and this was not even taken in the 1990s
53:38this was taken a couple years ago and I
53:40just visited out of curiosity Craigslist
53:41today still looks the same so what's
53:44interesting about Craigslist though is
53:46that it is a dynamic website and that
53:48you can fill out a form and post a for
53:50sale ad or a roommate ad or the like and
53:53the website does actually change but
53:55it's if we zoom in on this and it's
53:57going to be a little blurry because of
53:58the screenshot the URL that's up there
54:01is actually do HTML even though it's
54:04barely readable at this resolution which
54:07is to suggests that Craigslist is
54:09apparently accepting user input through
54:11forms for instance whoever uh wrote up
54:14this uh job advertisement some time ago
54:17but then Craigslist is spitting it out
54:19as a HTML file as opposed to storing it
54:24where or in
54:27what
54:34Axel yeah so this is in stark contrast
54:36to what we've done for project zero
54:39project one using PHP as the back end
54:41whereby you store data like this like
54:43server side in maybe an XML file or more
54:46realistically in an in a mySQL database
54:48or similar and then you generate a page
54:50like this dynamically so why is
54:53Craigslist doing this apparently
54:56could just be they're stuck in the 90s
54:58but there's a compelling reason too AEL
55:01well if they store the actual HTML file
55:03then they don't have to regenerate it
55:06every time it's visited yeah exactly if
55:08they're storing the HTML file they just
55:09don't have to regenerate it every time
55:11it's Revisited so this itself is caching
55:14it's not caching in any particularly
55:15fancy way you're just generating the
55:17HTML and saving it something called like
55:19something. HTML and storing it on dis
55:22and the upside of this is that web
55:23servers like Apache are really really
55:25really good and fast at just spitting
55:27out bits just spitting out raw static
55:30content like a gif a JPEG an HTML file
55:33right the performance optimizations
55:34these days generally relate to the
55:36languages like PHP and Python and Ruby
55:39where you're trying to fine-tune
55:40Performance but if all you have to do is
55:42respond to a tcpi tcpip request with a
55:45bunch of bites from dis that's
55:46relatively straightforward these days
55:48and so they're taking advantage of the
55:50performance presumably of serving up
55:52static content but this comes at a cost
55:55what's the downside of this file based
55:58caching
56:02approach someone
56:05else nothing we've done thus far is sort
56:08of a complete win there's always a
56:10gotcha Lou space okay so space so we're
56:14storing it on disk and you know if
56:16you've ever posted on Craigslist they're
56:18also storing it somewhere in a database
56:20because they do let you go back and edit
56:22it it's just Craigslist is one of these
56:25where reads are probably much more
56:27common than writes indeed when people
56:29visit Craigslist they're probably
56:30flipping through Reading Pages as
56:32opposed to posting lots and lots and
56:33lots of ads all at the same time so
56:37there's some redundancy there that's
56:38unfortunate
56:39AEL
56:41yeah just build upon that you
56:45willing of code that you use on every
56:48page again plus it's not a very elegant
56:50solution to have a big folder on your on
56:52your server containing 10,000 CLS okay
56:54okay good so there's redundancy well
56:56actually with all these thousands of
56:58files there's redundancy too just in the
56:59basic stuff like you have the same HTML
57:01tag the same body tag the same link tag
57:04the same script tag in every single page
57:07if they're indeed just static HTML files
57:09so whereas you get some benefits of
57:11using something like PHP and recall our
57:13MVC discussion where we factored out
57:15template code like the header and the
57:17footer so that we started it one place
57:19and not thousands of places Craigslist
57:21is kind of sacrificing that feature and
57:24going with this instead so in the end
57:26it's probably a calculated tradeoff you
57:27get much better performance presumably
57:30from just sving up the static content
57:32but the price you pay is more disk space
57:35but at the same time you know for a few
57:36hundred dollars you can typically get
57:37even bigger hard drives these days so
57:39maybe that's actually the lesser of the
57:41evils but there is one got there's
57:43another big gotcha here if you've
57:45generated tens of thousands of
57:46Craigslist pages that look like this
57:49what's the implication now and maybe why
57:51are they stuck in the 90s well I decid
57:54to add we change the background color
57:57good change the design entirely which is
57:59necessary I can't do that without
58:01editing each one of those um 10,000
58:04files and that's that could be done
58:06automatically but it's much harder than
58:08just uh editing a generic template that
58:10generates it exactly if you want to
58:12change the Aesthetics of the page and
58:14add a background color change the CSS or
58:16make it the font something other than
58:17Times New Roman it's non-trivial now
58:20because assuming this is a fully intact
58:22HTML file with no server include
58:25mechanism no like require mechanism like
58:27you have in PHP you have to now change
58:30the background color in tens of
58:32thousands of files and less maybe you at
58:34least put it in a CSS file but even then
58:37if it's a less trivial change than color
58:39suppose you want to restructure the HTML
58:41of the page then you really have to do a
58:42massive find and replace or more
58:45realistically probably regenerate all
58:4710,000 plus pages and I'm we latched on
58:50to 10,000 arbitrarily but it's a lot of
58:53pages in this case so upsides and
58:56downsides they're one of the few people
58:57on the internet who do this particular
58:59approach um but it does have some value
59:01and I think the last time I read up on
59:03statistics they get by with relatively
59:04little Hardware as a result which is
59:06definitely compelling so myle query
59:09cache this is a mechanism that we didn't
59:10use but it's so easily enabled on a
59:14typical server with mySQL there's a file
59:16called my.cnf for your configuration
59:19file and you can simply add a directive
59:21like query cach type equals one and then
59:24restart the server to enable the query
59:26cache which pretty much does what it
59:28says the ne if you execute a command
59:30like select Fu from bar where baz equals
59:33123 that will be could be slow the first
59:36time you execute it if you don't have an
59:38index or if you have a really huge table
59:42but the next time you execute it if the
59:44query cache is on and that row hasn't
59:46changed the response is going to come
59:47back much more quickly so MySQL provides
59:50this kind of caching for identically
59:52executed queries which might certainly
59:54happen a lot if a user is navigating
59:56your website going forward and back
59:58quite a bit mcash is an even more
1:00:01powerful U mechanism um Facebook has
1:00:04made great use of this over the years
1:00:06especially initially mcash is a memory
1:00:08cache so it is a piece of software a
1:00:11server that you run on a server it can
1:00:12be on the same server as your web server
1:00:15it can be on a different box altogether
1:00:17but it essentially is a mechanism that
1:00:19stores whatever you want in Ram and it
1:00:23does this with in the PHP context with
1:00:25code like this so this mcash can be used
1:00:28by all sorts of languages here is php's
1:00:31own interface to it and you use mcash as
1:00:34follows you first connect to the mcash
1:00:36server using mcash connect which is very
1:00:38similar in spirit to mySQL connect which
1:00:40you might recall from a few lectures
1:00:42back then we try in this example to get
1:00:45a user so the context here is it's
1:00:47pretty expensive to do select star from
1:00:49users on my database table because I've
1:00:52got millions of users in this table and
1:00:54I'd really rather not execute that query
1:00:56more often than I have to I'd rather
1:00:58execute it once save the results in Ram
1:01:01and the next time I need that user go
1:01:03into the ram go into cache to get that
1:01:05user rather than touching the database
1:01:07so there's this sort of tier of
1:01:10performance objectives disc is slow
1:01:12right spinning discs especially slow but
1:01:15it's fast to serve up so generally you
1:01:18might want to store something instead of
1:01:19on disk you want to store it in a table
1:01:21that has indexes so that you can search
1:01:24it more quickly for instance think back
1:01:26to project zero the XML file it's
1:01:28relatively small but at the same time
1:01:30anytime you wanted to search it you had
1:01:31to load it from disc build up a Dom
1:01:33thanks to the simple XML API then search
1:01:36it kind of annoying it'd be nice if we
1:01:38could skip the disk step so that things
1:01:40would just be faster and thus was born
1:01:42MySQL in Project one MySQL is a server
1:01:45which means it's always running it's
1:01:46using some Ram so in that case you have
1:01:48the ability to execute queries on data
1:01:51that's hopefully in Ram but even if it
1:01:53isn't you at least have the opportunity
1:01:54to Define indexes primary Keys unique
1:01:57keys index Fields so that at least you
1:01:59can search that data more readily than
1:02:01you can with say XPath in XML so the
1:02:04next step is not even to use a database
1:02:06because database queries can be
1:02:08expensive relative to just a cash which
1:02:10is just a key value store I want to give
1:02:13you x equals y and the next time I ask
1:02:15for X you give me Y and I want it quick
1:02:17much faster than a database would return
1:02:19it so here we've gone and connected to
1:02:21the memory cache demon the server in the
1:02:23second line
1:02:25I am trying to get something from the
1:02:27cache the arguments to mem cach get take
1:02:30the first argument is a uh a reference
1:02:34to the cache that you want to grab
1:02:35something from and then dollar sign ID
1:02:38just represents something like one two3
1:02:39the ID of the user that I want to get if
1:02:42the user is null what's the implication
1:02:45apparently Isaac well you do the quy all
1:02:49over again okay good but in what case
1:02:51would uh user be null do you think um
1:02:54when they don't exist good when they're
1:02:56not in the cache when user One Two Three
1:02:58or whoever I'm looking for is not in the
1:02:59cache that variable is going to be null
1:03:01and so we do this if condition as Isaac
1:03:03says and here there's some somewhat
1:03:05familiar code PDO which relates to my
1:03:07SQL in our case um we connect to the
1:03:11database uh using that user and pass we
1:03:13then execute the query function in PDO
1:03:16in this case select star from users
1:03:18where ID equals ID I'm not using my I'm
1:03:21not um escaping ID because in this case
1:03:24I'm assuming that I know it's an integer
1:03:25so it's not a dangerous string just to
1:03:27be clear then I'm calling fetch to get
1:03:29back an associative array of my data the
1:03:32user's name email address ID whatever
1:03:35else I've got in my database but then
1:03:37the last thing I do before apparently
1:03:40nothing else because this is out of
1:03:41context before actually using that user
1:03:44for anything what am I doing with him
1:03:46Axel you're storing it in the exactly
1:03:50I'm storing a key value pair in the
1:03:52cache whereby the key is the the user's
1:03:54ID so this implies that there's an ID
1:03:57field in the user object that came back
1:03:59from the database from this line here
1:04:01and the value is the user object itself
1:04:04so again a memcache in this case is a
1:04:06key value storage mechanism and the next
1:04:09time I want to look up this user I want
1:04:10to look him up by his ID and case in
1:04:14point that's what I did in line two up
1:04:16top now caches are finite because Ram is
1:04:21finite and even disk is finite so what
1:04:24could happen eventually with my
1:04:26cash just by nature of that those
1:04:29constraints Axel it gets so big can't on
1:04:33Ma good So eventually the cash could get
1:04:35so big you can't keep it on the machine
1:04:36so what would be a reasonable thing to
1:04:39do at that point when you've run out of
1:04:41Ram or dis space for your
1:04:46cash you're the person implementing
1:04:48mcash itself now what do you do in that
1:04:50case you could just kind of quit
1:04:52unexpected error but that would kind of
1:04:56be bad and completely
1:05:00unnecessary what could you
1:05:03do is it
1:05:06sorry yeah so some kind of garbage
1:05:08collection and what which things would
1:05:10you collect what things would you uh
1:05:12remove from
1:05:18memory
1:05:19good so we can essentially expire
1:05:22objects based on when they were put in
1:05:24so if I put in user 123 yesterday and I
1:05:28haven't touched him since or needed him
1:05:30since and I need more space well out
1:05:32goes user 123 and I can reuse that space
1:05:35that memory for user 456 if user 456 is
1:05:39the next person I'm trying to insert
1:05:41into the cache so indeed this is a very
1:05:43common mechanism whereby the first one
1:05:46in is the first one out if that object
1:05:49has not been needed since by contrast if
1:05:52123 is just one of these power users who
1:05:54logging in quite a bit and he he he or
1:05:56she is logging in again and again and
1:05:58again well I should remember every time
1:06:00and we don't see it in the code here but
1:06:02every time I get a cach hit and I
1:06:05actually find user 123 in the cache I
1:06:08could somehow execute another mcash
1:06:10function that just touches the user
1:06:13object so to speak thereby updating his
1:06:15timestamp to be this moment in time so
1:06:17that you remember that he was just
1:06:20selected and hopefully mcash get itself
1:06:22would do that for us and indeed it does
1:06:24I don't need to do this manually the
1:06:25cache software would remember oh you
1:06:28asked for user 1 two 3 I should move
1:06:30probably move him back to the front of
1:06:31the line so that the person at the end
1:06:32of the line is the first one to get
1:06:34evicted next time around so it's a
1:06:37wonderfully useful mechanism and
1:06:38Facebook is very read heavy or very
1:06:40right
1:06:42heavy if you're a
1:06:47user it's kind of both these days you
1:06:50know early on it was much more read
1:06:52heavy than write heavy because there
1:06:54were no like status updates and you
1:06:56would just have your profile and that
1:06:57was about it so these days there's
1:06:59definitely more rights but I'm going to
1:07:00guess that reads are still more common
1:07:03than not right when you log if you're a
1:07:04Facebook user and you log into your uh
1:07:07account and you see your newsfeed you
1:07:09know you might have 10 20 whatever
1:07:10friends show up in that Newsfeed that's
1:07:12potentially like 10 or 20 queries of
1:07:15some sort and yet you're probably not
1:07:17going to update your status 30 times at
1:07:20in that same unit of time so odds are
1:07:22Facebook is still a little more read
1:07:24heavy which makes cashes all the more
1:07:26compelling because if your own profile
1:07:28isn't changing all that often at least
1:07:31you might get 10 page views 100 page
1:07:33views by friends or random strangers
1:07:35before you actually update your status
1:07:36or your profile again that's an
1:07:38opportunity for optimization so early on
1:07:40and this day Facebook uses things like
1:07:42mcash quite a bit so that they're not
1:07:44hitting their various databases just to
1:07:46generate your profile they're instead
1:07:48just getting the results of some
1:07:50previous lookup unless it has since
1:07:53expired
1:07:55well on to mySQL optimization so that
1:07:58you can squeeze all the more performance
1:07:59out of your setup so this table is a
1:08:01little more overwhelming right now than
1:08:02it needs to be but recall our discussion
1:08:04of MySQL storage engine some time ago
1:08:07and we talked briefly about my Isam and
1:08:09INB does anyone remember at least one of
1:08:12the distinguishing characteristics of
1:08:14those two storage engines and again a
1:08:16storage engine was just like the
1:08:17underlying format that was used to store
1:08:20your database data um I think in
1:08:24support transaction good so INB which is
1:08:28the default these days so you haven't
1:08:30really needed to think about this much
1:08:32since project one um in ODB supports
1:08:34transactions whereas my Isam does not my
1:08:37Isam uses locks which are full table
1:08:39locks but T does tend to have some other
1:08:42properties and these this list here is a
1:08:45very long list of the various
1:08:47distinctions among these several storage
1:08:49engines um transactions is one of them
1:08:52um but there's a few other storage
1:08:54engines here that I thought I would just
1:08:55draw our attention to so
1:08:59one you have a memory engine uh
1:09:02otherwise known as a heap engine this is
1:09:05a table that's intentionally only stored
1:09:07in Ram which means if you lose power
1:09:09server dies or whatnot the entire
1:09:11contents of these memory tables will be
1:09:13lost but still kind of a nice feature if
1:09:16you yourself want to implement a cache
1:09:18relatively easily by writing keys and
1:09:21values into your database into two
1:09:22columns Maybe you yourself can Implement
1:09:25some kind of cash to avoid having to
1:09:27touch maybe much larger tables that you
1:09:29yourself have so that's an option to you
1:09:32archive storage engine haven't had to
1:09:34use this but take a guess as to what it
1:09:37does besides archiving
1:09:41something what does this engine do for
1:09:43you do you
1:09:52think
1:09:56and what what was the last sentence the
1:09:57last part of your
1:10:02comments oh it doesn't store anything in
1:10:04cash you have to query it all the time
1:10:05not quite so the property you're
1:10:07actually getting and you can kind of see
1:10:10it here um but there's some footnotes on
1:10:13the other storage engines is it's
1:10:15compressed by default so archive tables
1:10:18are actually slower to query but they're
1:10:21automatically compressed for you so they
1:10:23take up much much less space so a common
1:10:25case uh common use case for archive
1:10:28tables might be log files where you want
1:10:31to keep the data around and you want to
1:10:32write out a whole bunch of key of values
1:10:35in a row every time someone hits your
1:10:37web server anytime something buys
1:10:39someone but suppose you rarely query
1:10:41that data you're keeping it for
1:10:42posterity for research purposes for
1:10:44Diagnostic purposes but you're not going
1:10:46to do any selects on it any time soon so
1:10:48it would just be a waste to keep use
1:10:50more dis space than you need to so
1:10:52you're willing to sacrifice some future
1:10:54performance when you do need to query it
1:10:56for some long-term dis saving so the
1:10:58archive format allows you to do that NDB
1:11:01is a network uh storage engine which is
1:11:03used for clustering so that actually
1:11:05there is a way of addressing the issue
1:11:08of single points of failure that we
1:11:09discussed earlier with shared storage
1:11:11but we'll see a simpler approach in just
1:11:13a
1:11:14moment so in the world of datab bases
1:11:18like MySQL they typically offer this
1:11:20replication feature that I mentioned
1:11:22earlier so replication is all about
1:11:24making automatic copies of something and
1:11:26the terminology generally goes as
1:11:27follows you generally have a master
1:11:29database which is where you read data
1:11:32from and write data to but just for good
1:11:35measure that Master has one or more
1:11:37slave databases attached to it via
1:11:39network connection and their purpose in
1:11:42life is to get a copy of every row
1:11:45that's in the master database you can
1:11:47think of it rather simply as anytime a
1:11:49query is executed on the master that
1:11:51same query is copied down to one or more
1:11:53slaves and they do the exact same thing
1:11:55so that in theory the master and all of
1:11:58these slaves are identical to one
1:11:59another so what's the upside now of
1:12:02having databases one two three and four
1:12:05all of which are copies of one another
1:12:07apparently what problems does this solve
1:12:09for us if
1:12:11any AEL
1:12:17datab good so if something if database
1:12:19one dies because of human error you trip
1:12:21over the court hard drive dies r Fizzles
1:12:24out whatever the case may be you have
1:12:26literally three backups that are
1:12:28identical so there's no tapes involved
1:12:30there's no backup server I mean these
1:12:31are full-fledged databases and in the
1:12:33simplest case you could just unplug the
1:12:36master plug in the slave and voila you
1:12:38now have a new master and you might have
1:12:40to do a bit of reconfiguration in the
1:12:42databases to make him to promote him to
1:12:44master so to speak and then leave
1:12:46servers three and four as the New Slaves
1:12:48while you fix server number one but that
1:12:51would be one approach here so so you
1:12:53have some redundancy even though you
1:12:54might have a little bit of downtime at
1:12:56least you can get back up and running
1:12:57quickly and indeed you could automate
1:12:59this process if you notice that the
1:13:00master is down you could uh take him
1:13:02offline completely promote the slave and
1:13:05reconfigure them all just by writing a
1:13:07script what
1:13:09else how else could we take advantage of
1:13:11this
1:13:20topology let me ask a more leading
1:13:22question
1:13:24in the context of Facebook especially
1:13:26early days how might they in particular
1:13:30have made good use of this
1:13:36topology AEL well maybe if you get a lot
1:13:40of a lot of queries uhhuh you can
1:13:43Outsource them to different slaves okay
1:13:45so if you're getting a lot of queries
1:13:47you could you know maybe you could just
1:13:48load balance across database servers and
1:13:51absolutely you could the load balancers
1:13:52don't have to be used for HTTP alone you
1:13:54could use it for MySQL traffic but why
1:13:56do I say Facebook in particular early on
1:13:59they didn't get that many queries but
1:14:01this was still a good Paradigm for
1:14:04them why yeah why is this good
1:14:15perhaps so back to my hypothesis that
1:14:18they're more read heavy than write Heavy
1:14:21How can you
1:14:24adapt that that reality to this
1:14:27particular topology effectively or put
1:14:30another way why is this a good topology
1:14:33for a website that is very read heavy
1:14:35and less write heavy
1:14:44Ben
1:14:47okay okay good so reading can be
1:14:49expedited so if we kind of combine Ben
1:14:51and Axel's proposals here for a read
1:14:53heavy website like Facebook certainly in
1:14:55the early days you could just write your
1:14:57code in such a way that any select
1:14:58statements go to databases two three or
1:15:02four and any inserts updates or deletes
1:15:04have to go apparently to server one
1:15:08which even though that query then has to
1:15:10propagate to servers 2 three and four it
1:15:12is less common and that happens
1:15:14automatically so the code wise you don't
1:15:16have to worry about it too much and if
1:15:18you're suffering a bit of performance
1:15:19there well you can just throw more
1:15:20servers at it and have even more read
1:15:22servers to lighten the load for further
1:15:23so this approach of having slaves that
1:15:26can typically be used either for
1:15:27redundancy so you just have a hot spare
1:15:30ready to go or so that you can balance
1:15:33read requests across them is a very nice
1:15:36solution but of course every time we
1:15:39solve one problem we've introduced
1:15:41another or at least we haven't fixed yet
1:15:42another here what is a fa what is a
1:15:45fault in this layout
1:15:49still be
1:15:52paranoid
1:15:56kind of talked about it earlier but like
1:15:58what if one dies right unless this is
1:16:01there's got to be some blip on the radar
1:16:04here because we have to like promote a
1:16:06slave and right so you still have a
1:16:08single point of failure here at least
1:16:10for rights we could keep Facebook Alive
1:16:12by letting people browse profiles and
1:16:14read profiles but status updates for
1:16:16instance could be offline for as long as
1:16:18it takes us to promote a slave to a
1:16:20Master feels like it'd be nicer or at
1:16:22least our prob ility would be better of
1:16:24up time if we instead had not just a
1:16:26single Master but again let's just throw
1:16:28Hardware at the problem so another
1:16:30common Paradigm is actually to have a
1:16:32master master set up whereby as the
1:16:35labels imply and as the arrows suggest
1:16:38this time you could write to either
1:16:40server one or two and if you happen to
1:16:42write to server one that query gets
1:16:44replicated on server two and vice versa
1:16:48so now you could keep it simple you
1:16:50could always write to one but then the
1:16:52quer Theory goes to uh number two
1:16:55automatically or you could write to
1:16:57either thereby load balancing across the
1:16:58two and they'll propagate between each
1:17:01other but in this case here if you've
1:17:04laid out your network connections
1:17:06properly either one or two can go down
1:17:08and you still have a master that you
1:17:10could read from and you could even
1:17:11implement this in code recall very
1:17:13simply we had the MySQL connect function
1:17:15weeks ago or even the PDO Constructor
1:17:18function which tries to connect to a
1:17:19database you could implement this in PHP
1:17:22code if MySQL connect fails when
1:17:24connecting to server one then just try
1:17:26server two so you yourself could build
1:17:28in some redundancy so that now we could
1:17:30lose server one or two and not have to
1:17:32intervene as humans just yet because we
1:17:34at least still have a second master that
1:17:36we can continue writing to even though
1:17:39server one is Now offline for some
1:17:41amount of
1:17:42time all right but we still have to
1:17:44Route traffic there so in pursuit of
1:17:47this idea of load balancing here's a
1:17:49more complex picture that starts to
1:17:51unite some of our web ideas and some of
1:17:53our database ideas so at the top there
1:17:55we have some kind of network we have a
1:17:57load balancer in between and then we
1:17:59have this frontend tier so web servers
1:18:01are typically called a tier a service
1:18:04tier um this is a multi-tiered
1:18:06architecture would be the jargon here
1:18:08and those web servers now apparently are
1:18:10routing their requests through what in
1:18:13order to reach some MySQL slaves for
1:18:19reads who's the man in the middle here
1:18:22yeah AEL uh for reads it would be the
1:18:25load balancer okay so for reads yeah we
1:18:28have a second load balancer depicted
1:18:30here frankly in reality they could be
1:18:32one and the same they could be the same
1:18:33device but just listening for different
1:18:35types of connections but for now they're
1:18:37drawn more simply as separate now we
1:18:39have one MySQL master so we also have
1:18:41wires or arrows pointing from the web
1:18:43servers to the master and the master
1:18:45meanwhile has some kind of connection to
1:18:47the slaves so not bad no frankly this is
1:18:50starting to hurt my brain because now
1:18:52what was a very very simple class where
1:18:54you have a nice uh self-contained
1:18:56Appliance on your machine on your laptop
1:18:58does everything web database caching
1:19:00anything you wanted to do my God look at
1:19:02all the things we have to wire up now
1:19:04and it's still not perfect what could
1:19:05die here where what are our single
1:19:07points of
1:19:08failure Jack oh Isaac
1:19:13sure my master okay so the MySQL Master
1:19:16we haven't really solved that problem so
1:19:18kind of be nice to steal part of the
1:19:20previous picture and maybe uh it into
1:19:23here
1:19:25Jack same thing Exel load load balancers
1:19:29right so single point of failure is very
1:19:32well defined like single point of
1:19:34failure just look for any um any
1:19:37bottlenecks here whereby things point in
1:19:40and then go out load balancers is one
1:19:42here load balancers one there so it
1:19:44turns out with load balancers for your
1:19:46$100,000 you can get two of them
1:19:48typically in the package and what they
1:19:50tend to do is operate also in what's
1:19:52called um similar in spirit to master
1:19:55master mode but in the context of load
1:19:56balancers it's typically called active
1:19:58active as opposed to active passive and
1:20:01the idea here is with active active you
1:20:04have a pair of load balancers that are
1:20:06constantly listening for connections
1:20:08either one of which can receive uh
1:20:11packets from the outside world and then
1:20:13relay them to backend servers and what
1:20:16they typically do is they send
1:20:18heartbeats from left to right and right
1:20:19to left so that if this guy ever stops
1:20:22hearing a heartbeat from this guy so to
1:20:24speak and a heartbeat is just like a
1:20:25packet that gets sent every second or
1:20:27something like that if this guy stops
1:20:30hearing a heartbeat from this guy he
1:20:31automatically assumes that this guy must
1:20:33have gone offline so he's completely in
1:20:35charge now and he continues to send
1:20:38traffic uh from the outside world in or
1:20:41if you instead have active passive mode
1:20:43if this is the active guy at the moment
1:20:46rather if this is the active guy at the
1:20:47moment and he dies this guy similarly
1:20:50detects oo no more heartbeat and what
1:20:52the passive guy will do is promote
1:20:53himself to active which essentially just
1:20:55means he takes over the other guy's IP
1:20:58address so that all traffic now comes to
1:21:00him so in short we definitely need
1:21:02another load balancer in the picture how
1:21:04it's implemented is um is uh not as
1:21:08important to us right now but having a
1:21:10single load balancer is probably a bad
1:21:12thing right and this is the tragedy you
1:21:14can throw money at you can throw a lot
1:21:16of brainpower at various tiers here but
1:21:18if you have a lot of web servers a lot
1:21:20of MySQL servers but you have one look
1:21:22balancer just cuz it was really
1:21:24expensive or you didn't know how to
1:21:25configure it properly like the rest of
1:21:27it is pretty much for not because you
1:21:29still have things things that can die
1:21:32and take down your entire website so
1:21:35let's make this more complex still so
1:21:37let's now introduce two load balancers
1:21:40and let's actually introduce an idea of
1:21:42partitioning and this was actually
1:21:44something that Facebook coincidentally
1:21:45did make good use of early on back in
1:21:47the day there was harvard. harvard. thee
1:21:50face.com there was mit. the facebook.com
1:21:54and the earliest partitioning that they
1:21:55used was to essentially have a different
1:21:58server as best Outsiders could tell for
1:22:00each school so they literally just like
1:22:02copied the database copied the files
1:22:04over to another server and then voila
1:22:06thus was born MIT mit's copy of Facebook
1:22:10but this is actually even though this
1:22:12would get kind of messy for 800 million
1:22:13users and thousands and thousands of
1:22:15universities and uh networks it's pretty
1:22:18clean early on because it's just
1:22:21leverages this idea of part partitioning
1:22:23right it's kind of we didn't have a
1:22:24Facebook didn't have a big enough server
1:22:26to handle Harvard and MIT so why not
1:22:28just get two and say Harvard users go
1:22:30here MIT users go here and now we've
1:22:33kind of avoided that problem now
1:22:35unfortunately when bu comes on we need a
1:22:37third server but at least we can scale
1:22:39horizontally now there is a catch with
1:22:41partitioning as soon as you wanted to be
1:22:43able to poke someone at MIT or vice
1:22:45versa you had to somehow cross that
1:22:47Harvard MIT boundary at which point it's
1:22:50kind of a bad thing that they're all in
1:22:51separate databases so early on there
1:22:53were some features that you could only
1:22:54do within your own network um not until
1:22:57there was more shared State could you
1:22:59send messages and the like so
1:23:01partitioning though could be used even
1:23:02more simply suppose that you just had a
1:23:05whole bunch of users well you need to
1:23:07scale your architecture horizontally why
1:23:10don't I just put users whose last name
1:23:11start with a to M on half of my servers
1:23:14and then n through Z on the others right
1:23:17and when they log in I just send them to
1:23:18one or the other server based on that so
1:23:20in general partitioning is not a bad
1:23:23idea it's very common in databases
1:23:26because you can still have redundancy a
1:23:27whole bunch of slaves in this case here
1:23:29whole bunch of slaves over here but you
1:23:31can balance load based on some highlevel
1:23:33user information not based on load not
1:23:35round robin you can actually take into
1:23:38account what someone's name is and then
1:23:39send them to this particular server so
1:23:42partitioning is a very common Paradigm
1:23:45and then lastly just to slap a word on
1:23:47It High availability refers to what we
1:23:50described in the context of load
1:23:51balancers but it can apply to databases
1:23:53as well whereby High availability or ha
1:23:56is the buzz word simply refers to some
1:23:59kind of relationship between a pair or
1:24:01more of servers that are somehow
1:24:03checking each other's heartbeats so that
1:24:05if one of them dies the other takes on
1:24:07the entire burden of the uh of the
1:24:10service that's being provided whether a
1:24:12database or whether a load
1:24:16balancer so even though we finally got
1:24:19the iPad working it's a little small to
1:24:20draw on so what I wanted to do
1:24:23as our final example here is let me
1:24:28raise the screen
1:24:31here however one of these buttons will
1:24:33do it
1:24:37uh all right we're going to old school
1:24:40now our first and last piece of chalk in
1:24:42the
1:24:43class
1:24:44um let's
1:24:49see uh start with the middle one
1:24:52[Music]
1:24:54all right let's build ourselves a
1:24:55network here so we have a need for uh
1:25:00one or more web servers one or more
1:25:02databases maybe some load balancers but
1:25:04we're also going to try to tie together
1:25:06last week's conversation about security
1:25:08so we'll have to think about firewalling
1:25:09things out now so in very simple form we
1:25:13have here a web server which I'll draw
1:25:17is www all right so that's our web
1:25:20server and now my website's doing so
1:25:21well that I need a second web server so
1:25:23I'm going to draw it like this and now
1:25:26we need to revisit the issue of
1:25:28balancing load so what felt like one of
1:25:31our best options here how do I still
1:25:35have the internet which I'll draw as a
1:25:38cloud
1:25:39here connected to both of these servers
1:25:42somehow but I want the property of
1:25:45sticky sessions so what are my options
1:25:47or what was my best
1:25:51option
1:25:53how do I Implement sticky
1:25:56sessions AEL use load balanc keep all
1:26:00the sessions in one place okay good so
1:26:02use a load balancer and store all the
1:26:04sessions in one place and that's not
1:26:07okay so we can actually do one but not
1:26:09necessarily both of those let me
1:26:11interpose now the thing we started
1:26:13calling a black box so this is some kind
1:26:15of load balancer now I still have my
1:26:17backend servers and here's another one
1:26:21here okay and now this is connected here
1:26:24but I still want sticky sessions but you
1:26:26know what shared states that sounded
1:26:29expensive fiber channel ice scy sounded
1:26:31complicated there's a simpler way how do
1:26:33I ensure that I get sticky sessions
1:26:34using only a load balancer and no shared
1:26:37state
1:26:44yet how can I ensure that when Alice
1:26:47comes in and she's sent to This Server
1:26:49the first time that the next time she
1:26:51comes in she sent to the same
1:26:59oneel okay good so why don't we have the
1:27:02load balancer listen at the HTTP level
1:27:05and when the response comes back from
1:27:07the first web server let's give them
1:27:09numbers so let's call this one and this
1:27:11guy two the load balancer can insert
1:27:13some kind of cookie that allows it to
1:27:16remember that this user belongs on
1:27:19server one and how it does that I don't
1:27:20know it's a big random number and it's
1:27:21got a table like um PHP does for its
1:27:25sessions and figures out which server to
1:27:27send her to in this case all right so
1:27:29now I have a database the easiest way I
1:27:31know how to set up a database is to put
1:27:33it on the web server itself so much like
1:27:35the cs50 appliance you have a database
1:27:37in the same box as a web server if I now
1:27:41have a database here and here on the
1:27:43same boxes as our web servers what's the
1:27:47most obvious problem that now
1:27:50arises yeah Alice shopping cart is oh
1:27:54well if she does something to her
1:27:56profile or whatever it's just going to
1:27:58be on server one and not on server two
1:28:00since she just visited server one
1:28:02exactly so if Alice just happens to end
1:28:04up on server one and she updates her
1:28:06profile or credit card information or
1:28:07something persistent not the shopping
1:28:09cart thing because that involves the
1:28:10session but she does something
1:28:11persistent it's going to persist on this
1:28:13database and that's fine because sticky
1:28:16sessions are solving all my problems now
1:28:17but she comes back in a week or she logs
1:28:20in from a different computer cookie
1:28:22expires whatever and she ends up here
1:28:24what happened to my credit card
1:28:25information what happened to my profile
1:28:27I now have no profile because I'm on a
1:28:29different database so clearly this is
1:28:31not going to fly unless we partition our
1:28:35users and have the load balance or
1:28:37actually take into account who is this
1:28:39user and then send the user based on
1:28:41Alice's last name always to the same
1:28:43server so that could be one approach but
1:28:45for now let's instead factor out the
1:28:47database and say that it's not on the
1:28:50web servers it's separate and it's it's
1:28:52got some kind of internet connection
1:28:53here of course we've solved one problem
1:28:56but introduced a new one which is
1:28:59what
1:29:01ISAC single point of failure yeah so
1:29:05single point of failure again so how can
1:29:07we mitigate this well we can do a couple
1:29:09of things we can attach slave databases
1:29:12off of this and that's kind of nice but
1:29:15it then involves somehow promoting a
1:29:17slave to a Master in the event one dies
1:29:19so maybe the cleanest approach would be
1:29:21something like
1:29:22two Master
1:29:24databases so we'll call this DB1 this
1:29:27will be db2 and now how do I want to do
1:29:30this connect these like
1:29:35this Isaac you shook your
1:29:37head well you could connect the two
1:29:40databases to each other okay so we
1:29:42should probably do this for master
1:29:44master replication so that's good but
1:29:47what about these lines good
1:29:50bad
1:29:53answer's
1:29:54bad why bad Jack well you can connect
1:29:57each one to both of the right so the
1:30:01problem we just identified was database
1:30:03on same server as web server bad because
1:30:05then it's talking only to it and if
1:30:07Alice ends up on the other server she
1:30:08has no data that you're expecting well
1:30:10functionally this is equivalent I've
1:30:12just drawn a line but they're still only
1:30:13connected to each other and if the tri
1:30:16traffic should probably not go like that
1:30:18so we at least need to have some kind of
1:30:19cross connect so okay so I can do
1:30:23this but now what do I do so now my load
1:30:27balancing has to be done in code if
1:30:29those are the only uh components in my
1:30:32system right now the line suggests that
1:30:33dubdub du1 has a network connection to
1:30:35DB1 and db2 but that means now I have to
1:30:38do something like an if condition only
1:30:40in my PHP code to say if this database
1:30:42is up right here else if this database
1:30:45is up right there and that's not bad but
1:30:47now your developers have to know
1:30:48something about the topology if you ever
1:30:50introduce a third master or something
1:30:52like that although my SQL wouldn't play
1:30:54nicely with that now you have to change
1:30:56your code this is not a nice layer of
1:30:58abstraction so how else could we solve
1:31:00this
1:31:02Axel I don't like the idea of connecting
1:31:04each of my web servers to the database
1:31:07because frankly you know what this is
1:31:08going to get really ugly if it starts
1:31:10looking like this right very quickly
1:31:13this degrades into a mess yeah get
1:31:15another machine load Balan will okay
1:31:18good all of that so connect your um
1:31:22your dub du dub servers to below
1:31:24balancer and then let it handle requests
1:31:26and then you can Implement all kinds of
1:31:28of features like say um they are both
1:31:32Masters right they have the same data
1:31:34but what if only users with u last name
1:31:37that starts with end logs in well you're
1:31:39going to have a load on one particular
1:31:41server but then the load balancer can
1:31:43take all those features we talked about
1:31:44good CPU cycles and all that and
1:31:47actually distribute uh my SQL queries
1:31:49across databas okay good so we insert a
1:31:52load balancer here which is connected to
1:31:54both the dub machines and also the
1:31:55database servers and then he can be
1:31:58responsible for load balancing across
1:31:59the two masters um it's actually harder
1:32:02for the database to do any kind of
1:32:03intelligence load balancing based on
1:32:05last names at this point since the MySQL
1:32:08traffic is going to operate with binary
1:32:10messages not with HTTP style textual
1:32:13messages um load balancer up here can
1:32:15look at HTP headers and make Intelligent
1:32:17Decisions it's harder and maybe not
1:32:19impossible but it wouldn't be very
1:32:20common to do load balancing based on
1:32:23application layer intelligence here you
1:32:25would probably push that to the PHP code
1:32:27again in that case but this isn't bad
1:32:30but Isaac doesn't like this picture now
1:32:32because of what it still fails at one
1:32:34point yeah so we still have the single
1:32:36point of failure so you just cost me
1:32:38even more money or more complexity even
1:32:41if I'm using free software this just
1:32:42takes more time now so now we have load
1:32:45balancer one load balancer two I need to
1:32:49do something like
1:32:50this
1:32:53um and even though this looks a little
1:32:54ridiculous um oh actually it's a little
1:32:56elegant that's pretty sexy so you would
1:32:59do this with switches or some kind of
1:33:01ethernet cables all going into some
1:33:03Central source so suppose instead we
1:33:06actually did that if you've ever plugged
1:33:07in a computer into a network Jack which
1:33:09most of you probably have even if you
1:33:10have a laptop you don't connect these
1:33:12computers all to themselves you instead
1:33:14connect them to like a big switch that
1:33:16has lots of ethernet ports that you can
1:33:18plug into but now Isaac what what do you
1:33:20not like about this idea if I'm plugging
1:33:22everything into a
1:33:24switch still fails yeah so welcome to
1:33:27the world of like Network redundancy so
1:33:30really the right way to do this is to
1:33:32have two switches so almost every one of
1:33:34your servers database and web alike as
1:33:36well as your load balancers would
1:33:38typically have at least two ethernet
1:33:40Jacks in them and one cable would go to
1:33:42one switch another cable would go to the
1:33:43other switch you have to be super
1:33:45careful not to create Loops of some sort
1:33:47so switches have to be somewhat
1:33:49intelligent typically so that you don't
1:33:50create this crazy mess where traffic is
1:33:52just bouncing and bouncing around in
1:33:54your internal Network and nothing's
1:33:55getting in or out so there's an uh
1:33:58there's some care that has to be taken
1:33:59but in general this is really the theme
1:34:02in ensuring that you have not only
1:34:03scalability but redundancy and higher
1:34:06probabilities of uptime and resilience
1:34:07against failure you really do start
1:34:09cross-connecting many different things
1:34:12but let's push harder Isaac what suppose
1:34:14I fix the switch issue suppose I also
1:34:16make this two load balancers and fix
1:34:18that
1:34:19issue what's something else could fail
1:34:32now can't do this on an iPad very well
1:34:35um so this is your data center here's
1:34:38the door to your data
1:34:39center Jack the building burns
1:34:44down that's good more extreme than I had
1:34:46in mind I was thinking the power goes
1:34:47out but that works too um so the
1:34:49building itself burns down or goes
1:34:52offline you have some kind of network
1:34:54disconnect between you and your ISP the
1:34:55whole building or um the power indeed
1:34:58does go out and this has happened in
1:35:00fact one of the things that happens
1:35:01every time Amazon goes out is the whole
1:35:03world starts to think that clouds cloud
1:35:06computing so to speak is a a bad thing
1:35:09because oh my god look you can't keep
1:35:10the cloud up but the tragedy here is in
1:35:12this perception that cloud computing
1:35:14really just refers to Outsourcing of
1:35:16services and sharing resources like
1:35:18power and networking and and security
1:35:20and so forth across multiple customers
1:35:23so Amazon Services ec2 elastic compute
1:35:25cloud is kind of this picture here
1:35:27whereby you don't own the servers but
1:35:29you do rent space on them because they
1:35:31give you vpss that happen to be housed
1:35:33inside of this building Amazon offers
1:35:35things called availability zones whereby
1:35:37this might be an availability Zone
1:35:38called us East one so this is a building
1:35:40in Virginia in that particular case and
1:35:43what they offer though is Us East 2 and
1:35:45three and four they call them A and B
1:35:47and C and D and what that simply means
1:35:50in theory is that there's another
1:35:52building like this drawn over there that
1:35:54does not share the same power source
1:35:56does not share the same networking
1:35:58cables and so even if something goes
1:36:00wrong in one building in theory the
1:36:02other shouldn't be affected however
1:36:04Amazon has suffered outages in multiple
1:36:06availability zones multiple data centers
1:36:09so in addition to having servers in
1:36:10Virginia guess where else they have
1:36:15servers anywhere else okay anywhere else
1:36:17yeah that's actually a pretty hard
1:36:18question the world's a big place so the
1:36:20west coast and in Asia and in South
1:36:22America and in Europe as well they have
1:36:24different regions as they call them
1:36:26inside of which are different data
1:36:27centers or availability zones but this
1:36:29just means that you can really drive
1:36:31yourself crazy thinking through all the
1:36:33possible failure scenarios because even
1:36:34though Jack's building burning down is a
1:36:37little extreme things like that do
1:36:39happen right if you have a massive storm
1:36:41like a tornado or hurricane that just
1:36:43knocks out power absolutely could a
1:36:45whole building goes down go down so what
1:36:47do you do in that case well we have to
1:36:49have a second data center availability
1:36:51zone I'll draw it much smaller this time
1:36:53even though it might be this physically
1:36:54the same so here's another one suppose
1:36:56that inside of this building is exactly
1:36:58that same topology so now really what we
1:37:01have is the internet outside these boxes
1:37:04connecting to both buildings so so
1:37:08internet is no longer inside the
1:37:10building so once you have two data
1:37:12centers how do you now distribute your
1:37:15load across to Data Centers Axel well
1:37:18then you can use the DNS trause yeah so
1:37:21we we didn't really spend much time on
1:37:23it but recall that you can do load
1:37:24balancing at the DNS level and this is
1:37:26indeed how you can do uh geography based
1:37:29geoloc geography based load balancing
1:37:32whereby now when someone on the internet
1:37:34requests the IP address of something.com
1:37:37they might get the IP address really of
1:37:40this building or more specifically of
1:37:41the load balancer in this building or
1:37:44they might get the IP address of the
1:37:45load balancer in this building right
1:37:47when we did the NS lookup on Google we
1:37:49got a whole bunch of results that's not
1:37:51because they have one building with lots
1:37:53of load balancers inside of it that's
1:37:54because they probably have lots of
1:37:56separate buildings or data centers
1:37:58different countries even that themselves
1:38:00have different entry points different IP
1:38:02addresses so you have Global load
1:38:05balancing as it's typically called then
1:38:06the request comes into a building and
1:38:08you still have the issue of somehow
1:38:09making sure that subsequent traffic gets
1:38:11to the same place because odds are
1:38:13Google is not sharing your session
1:38:14across entirely entirely different
1:38:17continents even though they could be but
1:38:19that would probably be expensive or slow
1:38:21do so odds are you're going to stay in
1:38:22that building for some amount of time
1:38:24but again these ideas we've been talking
1:38:26about just get magnified the bigger and
1:38:28bigger you start to think and even then
1:38:30you have potential downtime because if a
1:38:32whole building goes offline and your
1:38:34browser or your computer happens to have
1:38:36cashed the IP address of that building
1:38:40that data center could take some minutes
1:38:42or some hours for your TTL to expire at
1:38:46which point you get rerouted to
1:38:48something else not too long ago just a
1:38:50few weeks I think or was offline for
1:38:52several hours one night cuz they use
1:38:53Amazon um a bunch of other popular
1:38:55websites too who use Amazon Services
1:38:58were down alt together because they were
1:39:00in a building or a set of buildings that
1:39:02suffered this kind of downtime and it's
1:39:05hard like if you are having the
1:39:07fortunate problem of way too many users
1:39:09and lots of Revenue um it gets harder
1:39:11and harder to actually scale things out
1:39:13globally so typically people do what
1:39:15they can but as Isaac has gotten very
1:39:16good at pointing out you can at least
1:39:18avoid as best as possible these kinds of
1:39:20single point points of
1:39:23failure
1:39:25questions so a word on security then
1:39:27let's focus only on this picture not so
1:39:29much on the buildings what kind of
1:39:30traffic now needs to be allowed in and
1:39:33out of the building so let me go ahead
1:39:35and just give myself some internet here
1:39:38connecting to the load balancer somehow
1:39:41what type of internet traffic should be
1:39:43coming from the outside world in if I'm
1:39:46hosting a website with a lamp based
1:39:48website yeah well you would want to fire
1:39:51that allows only for 80 connections okay
1:39:54good so I want TCP recall um which is
1:39:57one of the transport protocols 80 on the
1:40:00way in that's good but you just
1:40:02compromised my ability to have certain
1:40:05security why you're now blocking a very
1:40:08useful type of traffic oh yeah ENC good
1:40:10so we also want 443 which is the default
1:40:14Port that's used for SSL for https based
1:40:17URLs so that's good this means now that
1:40:19the only traffic allowed into my data
1:40:21center is TCP 80 and 443 now those
1:40:24familiar with SSH you've also just
1:40:26locked yourself out of your data center
1:40:27because you cannot now SSH into your
1:40:29data center so you might want to allow
1:40:30something like Port 22 for SSH or you
1:40:34might want to have an ssl-based VPN so
1:40:36that you can connect somehow to your
1:40:37data center remotely and again it
1:40:39doesn't have to be a data center this
1:40:40can just be some web hosting comp or
1:40:42some VPS hosting company that you're
1:40:44using and okay so we might need one or
1:40:47more other ports for our VPN but for now
1:40:48that's pretty good how about the load
1:40:50balance bancers what kind of traffic
1:40:52needs to go from the load balancer to my
1:40:54web
1:41:00servers Axel well it's really a mess to
1:41:03keep it encrypted because once it's
1:41:05inside the data center nobody else is
1:41:07going to listen than the people inside
1:41:09the so you would want to drop the
1:41:11encryption and do 80 good and that's
1:41:14actually very common to offload your SSL
1:41:17to the load balancer or some special
1:41:19device and then keep everything anything
1:41:21else unencrypted because if you control
1:41:23this it's at least safer not 100%
1:41:25because if someone compromises this now
1:41:27they're going to see your traffic
1:41:28unencrypted but if you're okay with that
1:41:30doing the SSL termination here so
1:41:33everything's encrypted from the internet
1:41:34down to here but then everything else
1:41:36goes over normal unencrypted HTTP the
1:41:39upside of that is remember the whole
1:41:41certificate thing you don't need to put
1:41:42your SSL certificate on all of your web
1:41:44servers you can just put it in the load
1:41:46balancer or the load balancers you can
1:41:48get expensive load balancers to handle
1:41:50the cryptog graphy and the the
1:41:51computational cost thereof and you can
1:41:53get cheaper web servers because they
1:41:54don't need to worry as much about that
1:41:56kind of overhead so that's one option so
1:41:58CCP 80 here and here how about the
1:42:00traffic between the web server and the
1:42:03databases perhaps through these load
1:42:07balancers this is a more of a trivia
1:42:10question but what kind of what kind of
1:42:12traffic is it even if you don't know the
1:42:13port number
1:42:15yeah it's
1:42:18query yeah uh query ex or more specific
1:42:20it's the SQL queries like select and
1:42:22insert and delete and so forth so this
1:42:26is generally TCP
1:42:29336 which is the port number that my SQL
1:42:33uses by default so what does this mean
1:42:35um if you do have firewalling
1:42:37capabilities and we haven't drawn any
1:42:39firewalls per se so we do need to insert
1:42:41some Hardware into this picture that
1:42:43would allow us to actually make these
1:42:45kinds of configuration changes but if we
1:42:47assume we have that ability in large
1:42:48part because all of these things are
1:42:50plugged in in as we said to some kind of
1:42:51switch well the switch could be a
1:42:53firewall itself and we could make these
1:42:55configuration changes we can further
1:42:57lock things down why I mean everything
1:43:00just works if I don't firewall
1:43:02things why would I want to bother
1:43:05tightening things so that only 8 and 443
1:43:07are allowed here and 3306 is allowed
1:43:10here and in fact notice there's no line
1:43:12between these
1:43:14guys well it would be really stupid to
1:43:17keep for example 3306 open in the
1:43:20because then people they might not be
1:43:22able to because of other security
1:43:24measures but in theory they they are
1:43:26allowed by the firewall to quer your
1:43:29database and do
1:43:30SQL commands good exactly there's just
1:43:33no need for people to be able to
1:43:35potentially even execute SQL querries
1:43:37coming in or make MySQL connections and
1:43:39even if you're not even listening for
1:43:40MySQL connections it again is sort of
1:43:42the principle of the thing you should
1:43:44really have the uh the principle of
1:43:47least privilege whereby you only open
1:43:49those doors that that people actually
1:43:51have to go through otherwise you're just
1:43:53inviting unexpected Behavior because you
1:43:56left the door a jar so to speak you left
1:43:57the port open and it's not clear whether
1:44:00someone might in fact take advantage of
1:44:02that case in point if somehow you screw
1:44:04up or Apache screws up or PHP screws up
1:44:06and this server is compromised it'd be
1:44:09kind of nice if the only thing this
1:44:11server can do is talk via MySQL to this
1:44:14server and cannot for instance suddenly
1:44:16SSH to this server or poke around or
1:44:19execute any commands on your Network
1:44:21other than MySQL so at least if the bad
1:44:24guy takes this machine over he really
1:44:25can't leave this uh rectangle here that
1:44:30I've drawn so again beyond the scope of
1:44:32things we've done in the class and even
1:44:34though the appliance itself actually
1:44:36does have a firewall that allows certain
1:44:37ports in and out um all of the ones you
1:44:40need we haven't had to fine-tune it um
1:44:42for any of the projects realize that you
1:44:44that even on something like a Linux
1:44:46based operating system so in short as
1:44:48soon as you have the happy problem of
1:44:50having way too many users for your own
1:44:52good lots of new problems arise even
1:44:54though thus far we focused pretty much
1:44:56entirely on the software side of things
1:44:58so that is it for computer science s75
1:45:01I'll Stick Around for questions
1:45:02oneon-one um we still have a final
1:45:04section tonight um for those of you who
1:45:06would like to dive into some related
1:45:08topics um otherwise realize that the
1:45:10final project two um is its deadline is
1:45:14coming up you should have gotten
1:45:15feedback by from your TFS about project
1:45:17one if not just drop him or her a note
1:45:19um or me otherwise it's been a pleasure
1:45:21having one in the class we will see you
1:45:23online after tonight oh that's
1:45:28okay thanks I wasn't trying to build up
1:45:31to that there
1:45:38so

About this transcript

This page contains the full transcript of CS75 (Summer 2012) Lecture 9 Scalability Harvard Web Development David Malan by archn, generated from the public captions YouTube serves with the video. The transcript has 19,464 words across 2,766 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.