AWS Storage Explained | Amazon EBS, EFS, FSx & Snow Family | AWS CLF-C02 Day 11 — Transcript

Full transcript

1. 0:01Welcome back cloud builders. We are
2. 0:03officially halfway through the AWS
3. 0:05builder challenge on day 11. Yesterday
4. 0:07we mastered Amazon S3. But today we are
5. 0:10tackling the storage you interact with
6. 0:12every single day without even realizing
7. 0:14it. We are talking about the hard drives
8. 0:17powering your virtual machines. How to
9. 0:19save files across a fleet of servers and
10. 0:21my personal favorite literal 45T trucks
11. 0:24filled with hard drives. I know if you
12. 0:27are a beginner storage terminology can
13. 0:29feel like alphabet soup. EBS, EFS, FSX,
14. 0:33IOPS. It's a lot. But don't stress.
15. 0:37Today I'm going to explain exactly how
16. 0:39this stuff works in the real world
17. 0:41because this domain makes up to 34% of
18. 0:44your cloud practitioner exam. So let's
19. 0:46get into it.
20. 0:48Here is our game plan for today. First,
21. 0:51we are going to decode EBS volume types.
22. 0:53You will learn exactly which virtual
23. 0:56hard drive to pick based on what you are
24. 0:57trying to build. Then we will solve a
25. 1:00really common cloud problem sharing
26. 1:02files across multiple servers at the
27. 1:04exact same time. We will look at EFS and
28. 1:08FSX and I will give you the exact
29. 1:10trigger words you need to spot on the
30. 1:12exam. Finally, we are talking about
31. 1:14offline data migration with the AWS is
32. 1:18no family because believe it or not,
33. 1:20sometime the absolute fastest way to get
34. 1:22your data into the cloud isn't through
35. 1:25an internet connection. It's by mailing
36. 1:27a rugged suitcase to an AWS data center.
37. 1:30Yeah, you heard it right.
38. 1:32All right, let's start with EBS or
39. 1:35elastic block store. When you spin up a
40. 1:39virtual computer in AWS, that is an EC2
41. 1:41instance, it needs a hard drive to
42. 1:43install the operating system. That hard
43. 1:46drive is an EBS volume. But AWS doesn't
44. 1:49just give you an one sizefits-all hard
45. 1:51drive. Look at this chart. They give you
46. 1:54four main categories split between fast
47. 1:57SSDs, that is solid state drives on the
48. 2:00top, and the traditional spinning hard
49. 2:02drives, hard disk drive on the bottom.
50. 2:05Before we even look at the specific
51. 2:07types, write this golden rule down for
52. 2:09the exam. You cannot use a cheap
53. 2:12spinning SDDD as a boot volume. If you
54. 2:15want to install Windows or Linux, you
55. 2:17have to use an SSD.
56. 2:19SDS that is hard disk drives are
57. 2:21strictly for holding massive amount of
58. 2:23data, not running operating systems.
59. 2:26Now, let's break down exactly which
60. 2:28drive you should choose in the real
61. 2:30world.
62. 2:31If you are just spinning up a standard
63. 2:33web server for a blog or you just need a
64. 2:35normal boot drive, you are going to use
65. 2:37GP3, general purpose SSD,
66. 2:41it's the default one. Think of it as
67. 2:43your daily driver car. Reliable, cost
68. 2:45effective, and it gives you a solid
69. 2:47baseline of speed right out of the box.
70. 2:50But what if you are a massive bank
71. 2:52processing thousands of credit card
72. 2:53swipes every single second? That GP3
73. 2:56drive isn't going to cut it. You need
74. 2:59the absolute fastest drive AWS offers.
75. 3:02That is IO2 block express. This is your
76. 3:05Formula 1 race car built specifically
77. 3:08for mission critical databases where
78. 3:10even a millisecond of lag cost money. It
79. 3:13is incredibly fast and accordingly the
80. 3:15most expensive as well.
81. 3:18Now what if you don't need a blistering
82. 3:20speed but you have a mountain of data.
83. 3:22Let's say you are processing 500 TBs
84. 3:24that terabytes of daily website logs.
85. 3:27That's where you switch to a spinning
86. 3:29disc that is SD1 throughput SDDDS.
87. 3:32It's your heavyduty fright train. It's
88. 3:35perfect for sequential big data
89. 3:37workloads like CFKA and it's way cheaper
90. 3:39than an SSD. And finally, if you have an
91. 3:42old employee records or compliance data
92. 3:44that you only look at maybe once a year
93. 3:46during an audit and you just want the
94. 3:48cheapest solution possible, cheapest
95. 3:50storage solution possible, that's SC1,
96. 3:54also called SDDD. So it's basically a
97. 3:57digital storage locker.
98. 4:00So EBS is great, but it has a major
99. 4:02limitation. It's a onetoone
100. 4:04relationship.
101. 4:06You generally can't plug a single EBS
102. 4:08hard drive into 10 different web servers
103. 4:10at once. If you have a fleet of Linux
104. 4:13servers that all need to read and write
105. 4:15to the same exact folder at the same
106. 4:17time, maybe they all are serving the
107. 4:19same users profile,
108. 4:22then you want to use EFS, that is
109. 4:24elastic file system. It's natively built
110. 4:27for Linux. And the coolest part is that
111. 4:29you don't even have to provision a size.
112. 4:31It just automatically grows and shrinks
113. 4:33as you add or delete files. But here's a
114. 4:36catch. EFS that is elastic file system
115. 4:40doesn't work for Windows.
116. 4:43Now let's say if your corporate office
117. 4:45is moving to the cloud and they want to
118. 4:47keep their classic shared S drive where
119. 4:49all the employees save their word
120. 4:51documents using active directory you
121. 4:54can't use EFS
122. 4:56that is you can't use elastic file
123. 4:58system you have to use FSX for Windows
124. 5:01file server if you see keywords on the
125. 5:04exam like SMB protocol NTFS or active
126. 5:08directory your answer is FSX for Windows
127. 5:11AWS has also Alo another flavor called
128. 5:14FSX for luster. And this luster
129. 5:16literally comes from the words Linux and
130. 5:20cluster.
131. 5:22It is built strictly for high
132. 5:23performance supercomputing. If a
133. 5:25university is running insane weather
134. 5:27simulations or feeding pabytes of data
135. 5:30into a hungry AI machine learning model,
136. 5:32they use FSX for luster.
137. 5:35And another we have Amazon FSX for
138. 5:38NetApp on tap. This enables you to lift
139. 5:40and shift your on premises NetApp
140. 5:42environments directly into a fully
141. 5:44managed AWS services that is actually
142. 5:46like allows you to move your enterprise
143. 5:48storage to the cloud without any
144. 5:51application refactoring, code changes or
145. 5:52altering how you manage your data.
146. 5:56Okay, moving on to my favorite topic,
147. 5:59the snow family.
148. 6:02Imagine you are a hospital with 100 TBS
149. 6:04of old patient X-ray data and you want
150. 6:06to move them into AWS. If you try to
151. 6:09upload that over your standard corporate
152. 6:11internet connection, it could literally
153. 6:12take 92 days over 3 months. You can't
154. 6:16wait that long. So AWS created
155. 6:19sneakeret.
156. 6:20They will literally mail you a physical
157. 6:22hard drive and you plug it into your
158. 6:24local network, copy your files and hand
159. 6:26it back to the shipping company. And for
160. 6:29small jobs or extreme environments like
161. 6:31a research drone out in a desert with
162. 6:33zero internet, they have the snow cone.
163. 6:36It holds about 8 to 14 terabytes and it
164. 6:40is so small that it can fit in a
165. 6:41backpack.
166. 6:43For medium standard jobs, you order a
167. 6:45snowball edge. It looks like a heavyduty
168. 6:48piece of a military luggage and holds
169. 6:50anywhere from 80 to 210 terabytes.
170. 6:55And finally, what if you are shutting
171. 6:56down an entire enterprise data center
172. 6:58with xabytes of data? AWS will literally
173. 7:02drive a 45 foot 18-wheeler truck to your
174. 7:05office. That is called snow mobile. Now
175. 7:09let's look at how the exam is actually
176. 7:11going to test you on this.
177. 7:14They love to test your ability to pick
178. 7:16the right device based on capacity.
179. 7:19Look at exam pattern number one on the
180. 7:21screen. If they ask you how to move a
181. 7:23500 terbte data set over a slow internet
182. 7:26connection, you can't use a snow cone.
183. 7:28It's too small.
184. 7:30and 500 TV isn't nearly big enough to
185. 7:32justify a massive snowmo truck. So the
186. 7:36answer is to order multiple snowball
187. 7:38edge devices. Also pay attention to the
188. 7:41word edge for the exam. It means it
189. 7:43actually has a compute power built-in.
190. 7:46You can run EC2 instances directly on
191. 7:48this suitcase while it sits on your
192. 7:50desk.
193. 7:51For example, pattern number two. If you
194. 7:53see the words entire data center
195. 7:55migration or exabyte scale, stop reading
196. 7:58and immediately choose snowmo.
197. 8:01Also remember the capacity lead letter.
198. 8:03This snow cone, snowball edge and we
199. 8:06have snow mobile. How much capacity uh
200. 8:09do they actually are related with?
201. 8:13So now it's time to put everything into
202. 8:15practice. Today's lab is going to teach
203. 8:17you a really important physical rule
204. 8:19about how the cloud works. Availability
205. 8:22zones aren't just virtual concepts. They
206. 8:24are physical buildings miles apart. If
207. 8:26your EC2 instance is running in a
208. 8:28building A, its EBS hard drive is
209. 8:30physically bolted in a server rack
210. 8:32inside that specific building. You can't
211. 8:35just magically drag and drop that hard
212. 8:37drive over to a server in a building B.
213. 8:39So, how do we move our data to a
214. 8:42different availability zones?
215. 8:44We use snapshots.
216. 8:47What is a snapshots? A snapshot is
217. 8:49basically a backup photograph of your
218. 8:51hard drive.
219. 8:53So let's do it together right now.
220. 8:56I will go to my AWS console and simply
221. 9:00search for EC2.
222. 9:03Now I will create one instance for me.
223. 9:07What I will do is click on launch
224. 9:09instances and let me name it something
225. 9:12like um my
226. 9:16SBG
227. 9:18server
228. 9:20and let me keep it default like Amazon
229. 9:23Linux machine AMI will be same and uh I
230. 9:28will select T3 micro for the instance
231. 9:30type.
232. 9:32Let me generate a key pair as well. Um
233. 9:35so let me keep this name as SBG key
234. 9:38pair. I will create a key pair and yeah
235. 9:42it is downloaded
236. 9:44and let this network security setting be
237. 9:48same or I can do this allow https allow
238. 9:51http traffic from the internet. Keep
239. 9:53them checked and uh store 8b storage and
240. 9:57GP3. All right you can configure this
241. 10:00volume as well. And um okay let that
242. 10:05other all setting be same. So now my
243. 10:08instance is being created and once your
244. 10:10instance is created you can simply check
245. 10:13whether it this instance is in running
246. 10:15state or not and you can say it may be
247. 10:18saying pending state for you because it
248. 10:20takes time to start this machine to
249. 10:23launch this machine.
250. 10:25So what I will do is uh go on the left
251. 10:28menu click on volumes
252. 10:30and find the hard drive. You can simply
253. 10:33refresh and you can find this hard drive
254. 10:36attached to instance. Look at the
255. 10:38availability zone column. Mine says AP
256. 10:41South 1B.
257. 10:44To back this up, select the volume,
258. 10:47click on the accents button and uh hit
259. 10:51create snapshot.
260. 10:53Now you can see you can give a quick
261. 10:56description about it. Let me say SBG
262. 11:00snap sort
263. 11:02and when you click on create AWS takes a
264. 11:05perfect block level copy of your data
265. 11:07and safely stores it over in S3. You can
266. 11:11see the notification successfully
267. 11:13created snapshot and you can manage fast
268. 11:17snapshot restore from over here. So now
269. 11:20for the magic trick, click on snapshots
270. 11:23in the left menu.
271. 11:25Once your snapshot says completed,
272. 11:28select uh it and click on accents and
273. 11:31you can create volume from snapshot.
274. 11:35Now you are basically telling AWS to 3D
275. 11:37print a brand new hard drive based on
276. 11:39your backup photograph.
277. 11:41But look at the availability zone
278. 11:43dropdown. You can change it. If your
279. 11:45original was in A1, you can change a new
280. 11:48one to 1B as uh it was in 1B I guess. So
281. 11:52I will uh keep that in 1A for now. Now
282. 11:56simply hit create volume and just like
283. 11:58that you have successfully migrated your
284. 12:00block storage across physical data
285. 12:02centers. So now let me go to the volumes
286. 12:05tab back and you can see I have two
287. 12:08volumes created over here. Now now you
288. 12:11can attach your new volume that is
289. 12:12created in another availability zone. uh
290. 12:14to your EC2 instance as well. Um so now
291. 12:18uh just remember to clean this up
292. 12:19because uh we don't want to let my AWS
293. 12:23bill grow up. Now you can simply select
294. 12:26the volume which you don't want. Uh I
295. 12:29don't want any of the volume for now.
296. 12:35Done. And for another volume
297. 12:38also I will click on accents. uh you can
298. 12:42delete first you need to detach volume
299. 12:44if you it is attached to any of the EC2
300. 12:46instance simply what I will do go to the
301. 12:48instances I will simply delete that
302. 12:50instance once your instance is stopped
303. 12:54uh you can simply delete the instance
304. 12:58again go to instance state and terminate
305. 13:01instance that is deleting your instance
306. 13:04now if I go to the volumes tab you can
307. 13:06see the volume that was attached to my
308. 13:08EC2 instance is automatically deleted
309. 13:13So always remember to stop your billing
310. 13:15meter when you finish a lab. That is
311. 13:17deleting your instances, deleting your
312. 13:19created resources.
313. 13:21So always remember to stop your billing
314. 13:23meter when you finish a lab. That is
315. 13:24deleting your instances, deleting your
316. 13:27created resources. And that's a wrap on
317. 13:30the content for day 11. Let's do a
318. 13:32really quick rapid fire recap using this
319. 13:34cheat sheet on the screen. So let's say
320. 13:37default boot drive for the normal
321. 13:39server. What do you use? GP3
322. 13:42crazy fast database performance
323. 13:46that is IO2 EBS IO2
324. 13:50big data and sequential logs
325. 13:53ST1
326. 13:54that is HDD
327. 13:56sharing Linux file across multiple
328. 13:59servers
329. 14:00EFS
330. 14:03and sharing Windows file with active
331. 14:05directory that is FSX for Windows
332. 14:09and if the Internet is way too slow. You
333. 14:12order a snowball or a snow mobile.
334. 14:15Don't forget this. So your homework for
335. 14:18today's is to jump into the practice
336. 14:19portal and knock down 10 questions. When
337. 14:22you read a question, just ask yourself,
338. 14:24is this workload random or sequential?
339. 14:27Are they running Windows or Linux? Those
340. 14:29two question will almost always reveal
341. 14:32the right answer. Tomorrow on day 12, we
342. 14:34are tackling the biggest one that is VPC
343. 14:36networking. This is the absolute heart
344. 14:39of AWS infrastructure and it's the
345. 14:40heaviest weighted topic on the exam. So
346. 14:43do not miss it and yes keep building. I
347. 14:46will see you in day 12.