My F-You To Sam Altman — Transcript

Full transcript

1. 0:00Cloud AI is becoming an incredibly
2. 0:02powerful tool. With the latest Frontier
3. 0:04models, you can write shoddy documents,
4. 0:06generate shoddy pictures, and even
5. 0:08compose shoddy music.
6. 0:11[music]
7. 0:14>> Okay, jokes aside though, it can be
8. 0:17legitimately useful for everything from
9. 0:19writing code to data analysis to just
10. 0:22asking it questions that are too dumb to
11. 0:24ask your friends and colleagues.
12. 0:25Personally, I've been doing a lot of
13. 0:27that lately around smart home
14. 0:28automation. And yeah, it's the kind of
15. 0:30thing where I could read a 100 pages of
16. 0:33documentation or I could just ask in
17. 0:35plain English, hey, what does this
18. 0:37parameter do? And even if it gives me
19. 0:39the wrong answer, which it sometimes
20. 0:41does, it usually provides useful clues
21. 0:43that will lead me to the right answer.
22. 0:45There's just one small problem. Well,
23. 0:47actually many problems. hallucinations,
24. 0:50privacy, the mass theft of the entire
25. 0:52output of humanity, uh cost, and while
26. 0:55running an LLM locally can't solve all
27. 0:58of those issues, it can certainly solve
28. 1:00some of them. By buying the hardware
29. 1:02upfront, I'm not going to be able to
30. 1:04avoid all token costs. Some things
31. 1:06really do benefit from Frontier models,
32. 1:08and it looks like that's going to remain
33. 1:09true for some time yet. But I can
34. 1:12probably do over 90 plus% of what I need
35. 1:15to do on my hardware with no
36. 1:18subscription, which has many benefits.
37. 1:21The biggest one being that I can tie
38. 1:23constant AI compute into other systems
39. 1:26that I have without worrying about token
40. 1:28cost overrun. And in doing so, I keep
41. 1:30everything within my network where it's
42. 1:33private [music] and will always work
43. 1:35even if my internet is down. All I need
44. 1:38to do now is build it. Um, what's a 5090
45. 1:42worth these days?
46. 1:44Oh,
47. 1:47well, this is going to hurt.
48. 1:49Just like this painful segue to our
49. 1:52sponsor,
50. 1:53>> Squarespace. They make it easy for you
51. 1:55to build the ideal website to go with
52. 1:57your brand or business. Create something
53. 1:59custom with their design intelligence
54. 2:01tool with our link below.
55. 2:12As you can see, I already got the
56. 2:13hardware for my home AI station mostly
57. 2:16set up on my own time, including making
58. 2:18some fixes to the uh Frankenstein power
59. 2:21connector GPU here, that, by the way,
60. 2:23worked out fine
61. 2:26after several attempts. This is the
62. 2:29worst thing we've ever done. So, the
63. 2:30main reason I'm rebuilding it is
64. 2:32actually because Silverstone reached out
65. 2:34telling me that they have finally built
66. 2:36an update to my favorite case of all
67. 2:39time, the Tempen TJ07. And I was mostly
68. 2:43just really excited to check it out. So,
69. 2:45I'm going to put that in here. This is
70. 2:47my little portable water cooling unit
71. 2:48that I use anytime I need to
72. 2:50troubleshoot one of like the rack
73. 2:51mounted water cooling units if I have to
74. 2:53pull it out of the rack. Uh, it's been
75. 2:55permanently attached to the AI station
76. 2:56lately, so that's been kind of
77. 2:57inconvenient. On the subject of
78. 2:59inconvenience, I noticed that all the
79. 3:01tables in here are like Lucas height. Oh
80. 3:04god,
81. 3:06you know your case is [music] off the
82. 3:08rails when you have these like TV style
83. 3:10box closers. I used to do all the super
84. 3:13high-end custom builds back at NCX in
85. 3:15the predecessor to this case because it
86. 3:17had the best cooling, because it was the
87. 3:20easiest to build in. Um, no. And no, but
88. 3:24because it was absolutely sick.
89. 3:34Oh,
90. 3:39and the new one looks glorious. Good
91. 3:43lord,
92. 3:45they wrapped it around even more.
93. 3:51Oh, okay. Goodbye. Wait, what is that?
94. 3:54Oh god, it's got wheels.
95. 4:00Okay. One solid piece of aluminum all
96. 4:05the way
97. 4:08around.
98. 4:10Is that sick or what? Only this flat
99. 4:14back plate is a separate piece. Look at
100. 4:18all that room for activities. And this
101. 4:20is a freaking awesome looking GPU
102. 4:23anti-segmount. As you can see, it's
103. 4:26clearly designed with multi-GPUs in
104. 4:28mind. In fact, that's one of the only
105. 4:30reasons that Silverstone was able to
106. 4:31bring this thing to market because
107. 4:33[music] gamers don't really need
108. 4:35multi-GPUs and fancy expensive tower
109. 4:37cases anymore. But AI workstations, they
110. 4:41do. So, it is packed with support for
111. 4:44cooling as well. Where there used to be
112. 4:465 and 1/4 in bays in the front, now
113. 4:48we've got a radiator mount to go along
114. 4:50with the radiator mount up here to go
115. 4:51along with the radiator mount down here.
116. 4:53So, we're going to be water cooling this
117. 4:55today. And just like the original, we've
118. 4:58got support for multiple power supplies
119. 5:00for extremely power hungry builds.
120. 5:01Although, it's worth noting we're not
121. 5:03going to need that today because
122. 5:04Silverstone also sent over their new
123. 5:073,000 W 80 plus platinum power supply.
124. 5:09[music]
125. 5:10250 amps on the 12vt rail.
126. 5:14That's stupid. This thing is
127. 5:18It's like $1,300 or something like that.
128. 5:21and it looks and feels every penny of
129. 5:23it. As for the rest of the hardware I'm
130. 5:25using, well, it was mostly dictated by
131. 5:28whatever labs didn't take.
132. 5:32The good news is the leftovers here are
133. 5:34still pretty freaking amazing. Can you
134. 5:37believe that absolutely nobody wanted
135. 5:40this Sapphire Rapids workstation? It was
136. 5:42just sitting on the shelf. I don't even
137. 5:44know if we ever did a video on it. For
138. 5:46the CPU, then I ended up with a Zeon W9
139. 5:483495X. It's a 56 core processor that can
140. 5:52turbo up to 4.8 GHz, which is nice, but
141. 5:56not actually the reason that it's most
142. 5:58suitable for a system like this. What
143. 6:00mattered more to me is that it supports
144. 6:02boatloads of system memory, which is
145. 6:03perfect if you're running a ton of
146. 6:05containers, and it supports a ton of
147. 6:08PCIe Gen 5 lanes, 112 to be precise.
148. 6:12Meaning that on a motherboard like this
149. 6:14W790E Sage SE from ASUS, these are not
150. 6:18decorative. These are almost all running
151. 6:20at full by6. Not that I have that many
152. 6:23GPUs to put it in anyway, but it sure
153. 6:26doesn't hurt. [snorts] On the subject of
154. 6:28GPUs, my goal was to find the ones that
155. 6:30would be the most performant for me, but
156. 6:32the least missed by the rest of the
157. 6:34team. So, I grabbed the 5090 from the
158. 6:37Murderbox PC because at the time I
159. 6:40borrowed it, labs hadn't really delved
160. 6:42into desktop liquid cooling, but now I
161. 6:44can see they actually do a fair bit of
162. 6:45it. So, um, glad I got this, uh, before
163. 6:48they were using it.
164. 6:53And then, because the goal was maximum
165. 6:54performance, I went sniffing around to
166. 6:56see if there were any more that I could
167. 6:57borrow. And I found this. You guys might
168. 7:01remember the video where we fixed
169. 7:03Nvidia's much maligned 12volt 2x6 power
170. 7:05connector by putting an XT120 on it.
171. 7:09We did a pretty budgy job of everything
172. 7:12from the soldering to our choice of
173. 7:14wiring. And this isn't much better, but
174. 7:16I fixed it up a little bit at home.
175. 7:20Lucas went like this.
176. 7:22I don't know what that means.
177. 7:24>> I endorsed the tinkering. Let's say
178. 7:26we'll see if it lasts long term.
179. 7:28[laughter]
180. 7:28>> It's been working great for now. With
181. 7:31the GPUs out of the way, the last big
182. 7:33question was RAM. Now, to run LLM, I
183. 7:36don't strictly [music] speaking need a
184. 7:38ton of system memory, but if I want to
185. 7:40run a whole bunch of containers, then it
186. 7:43certainly doesn't hurt. And this
187. 7:45[clears throat] platform uses RDMs,
188. 7:48which right now are pretty expensive.
189. 7:51What the? Fortunately, I came across the
190. 7:53kit that you guys saw installed when I
191. 7:55was fishing around here late at night,
192. 7:57and I went all the way up to this top
193. 8:00shelf, which has
194. 8:03lots of weird stuff in it, including a
195. 8:06couple of boxes that were marked,
196. 8:07[music] "Do not take without asking the
197. 8:10business team." So, I took them. Intel's
198. 8:14LGA4677,
199. 8:16to put it lightly, was not popular with
200. 8:18enthusiasts. So, going into this build,
201. 8:21I had actually never mounted one of
202. 8:23these before. And I was shocked,
203. 8:26shocked, I say, to discover that instead
204. 8:28of putting the CPU in the socket,
205. 8:31thermal compound goes on the CPU. Then
206. 8:33the CPU actually clips onto the cooler
207. 8:40like that.
208. 8:43Okay, then as wrong and horrible as this
209. 8:46feels,
210. 8:48the whole thing just goes on like that.
211. 8:52Terrifying.
212. 8:53Now, there's a torque spec for exactly
213. 8:55how hard you're supposed to do it down.
214. 8:57I believe we're going to tighten it
215. 8:58incrementally in a cross pattern. There
216. 9:01are some server platforms that actually
217. 9:03call for you to just tighten them all
218. 9:04the way down in a cross pattern. Not
219. 9:07this one. And the spec is 8 inb for
220. 9:09those of you who are planning something
221. 9:11similar at home. All right. All right.
222. 9:12Now, we need an airflow plan. I'm
223. 9:14thinking exhaust out the top. Now, some
224. 9:16of you are probably wondering, Lionus,
225. 9:18what's the deal? Why are you not like
226. 9:19water cooling this thing? And the answer
227. 9:21is that not a lot of water blocks for
228. 9:24this platform. Also, I don't really need
229. 9:26to. I'm not overclocking. And these
230. 9:28workstation chips do not run hotter than
231. 9:30this Noctua cooler can handle. And this
232. 9:32is a functional machine. It's not to
233. 9:34show off.
234. 9:38So, we're just going to Oh, figure out
235. 9:41uh where all the hardware is for this
236. 9:43case that we actually have a new
237. 9:44motherboard standoff. Some moment,
238. 9:45please.
239. 9:46Why would they do this? They
240. 9:50hide it all in the very top of the case,
241. 9:52which if you remember is kind of the
242. 9:55bottom of the packaging once you
243. 9:56actually pull the whole thing apart.
244. 10:00Is that it?
245. 10:02Okay. Oh, wow.
246. 10:05That's it. No manual, no nothing. I
247. 10:09guess the vast majority of what you
248. 10:10would install in here would come with
249. 10:11its own stuff, though. Anyway, we've got
250. 10:13one of those fancy wy little retaining
251. 10:16motherboard standoffs. So, we'll go
252. 10:17ahead and put that in the middle. I do
253. 10:20not know the bolt pattern for this
254. 10:21motherboard.
255. 10:23It is unusual.
256. 10:26[clears throat]
257. 10:26Here's a tech tip I learned from my
258. 10:28stepdad. Yeah, if you ever need to
259. 10:29measure something, you don't have a
260. 10:30measuring tape handy.
261. 10:36That one.
262. 10:41Now it feels like No, it feels like I'm
263. 10:43up against something.
264. 10:46Wait,
265. 10:50how thick is this armor?
266. 10:54Is that interfering?
267. 10:56Cuz like I can't get it all the way
268. 10:57down.
269. 10:59Oh,
270. 11:01see this is what happens when you're
271. 11:03operating at the edges of standards. See
272. 11:06these cable management grommets?
273. 11:09They thick.
274. 11:11See that?
275. 11:13See this armor on the back of this
276. 11:15motherboard?
277. 11:16It also thick. So, we've got a bit of an
278. 11:19interference issue. Fortunately, I see
279. 11:22something. They're actually a little bit
280. 11:24thinner on one side. So, what I'm going
281. 11:26to try and do is turn these around and
282. 11:29see if that resolves the clearance
283. 11:30issue. Hey, there we go. Okay, that was
284. 11:34much more successful this time.
285. 11:38Can actually see the issue better from
286. 11:40the back where the board is right there.
287. 11:44That's the AR. Oh,
288. 11:47well, ain't nobody using those cable
289. 11:49management grommets anyway then.
290. 11:53>> [laughter]
291. 11:54>> A
292. 11:57how has it come up so much recently? Oh,
293. 12:01wait. I don't need it. Oh, Silverstone.
294. 12:05Mwah. Beautiful. Amazing. They've got
295. 12:09one of those pass through things, but
296. 12:10it's wide enough to accommodate the LT
297. 12:12screwdriver. How much you want to bet?
298. 12:14Tony made sure of that. Love it. Thank
299. 12:17you, Tony. And it goes just like that.
300. 12:21That seems like enough cooling, right?
301. 12:23We got an intake fan right there.
302. 12:26[snorts]
303. 12:28Man, this is [laughter] the kind of
304. 12:30hardware that just like Okay, I don't
305. 12:32have pit stains yet, but I will. I
306. 12:34[laughter] just like seriously I think I
307. 12:38was a couple WAN shows ago old when I
308. 12:41discovered how much these are going for
309. 12:43right now. Like obviously [music] one
310. 12:45with this manner of modifications is not
311. 12:47going to be worth as much, but like
312. 12:51it's insane. And the craziest [music]
313. 12:54part is that 5090s are not even
314. 12:57necessarily the best choice depending on
315. 12:59what you're doing. The GPU is obviously
316. 13:02really fast and supports, you know, all
317. 13:04of the latest types of quantization
318. 13:06[music]
319. 13:06and everything, but what they don't have
320. 13:09in the grand scheme of things is
321. 13:11actually super fast memory, nor do they
322. 13:13have a ton of it. [music] So, what I
323. 13:15might want to do in the longer term is
324. 13:17actually like pull one of these 5090s
325. 13:20and then augment a single fast GPU with,
326. 13:24you know, maybe one or a couple of those
327. 13:26memory unlocked crypto mining cards to,
328. 13:29you know, run a gentic task or to run
329. 13:32multiple models where speed is not as
330. 13:34important. On that note, the labs team
331. 13:36finally got their hands on some DGX
332. 13:39Sparks. And this is a great example of
333. 13:41how having the fastest processor is not
334. 13:44necessarily as important as having tons
335. 13:46and tons of memory that can be pulled.
336. 13:48What are you guys using these for?
337. 13:50>> Uh, today making cookie cutters.
338. 13:54Seriously? Yeah.
339. 13:56>> Okay.
340. 13:58Right now it's a DeepSeek 4.1 flash on
341. 14:00there which is like 410 GB of like
342. 14:03parameters I think and then more
343. 14:04context. So it's something you can only
344. 14:06fit on there and it's quite fast and
345. 14:09capable. So
346. 14:09>> that's pretty cool. I tracked down the
347. 14:12one 420 mm radiator in our entire
348. 14:15warehouse. This old EK cool stream and
349. 14:18ah
350. 14:20got my pump and reservoir mounted.
351. 14:22Silverstone has a really nice little
352. 14:24system for this where there's just this
353. 14:25plate at the front that has a couple of
354. 14:28120 mm wide spaced holes. So, you could
355. 14:31just throw a bracket on there and you're
356. 14:33good to go. The one thing that I would
357. 14:35like to see improved is there's nothing
358. 14:37in the way of antiibration and on
359. 14:39something that's clearly meant to have
360. 14:40pumps mounted to it.
361. 14:43Yeah, I'd like to see that like dampened
362. 14:45in some way. probably going to be
363. 14:47controversial, but yeah, we're going
364. 14:50softline tubing for this. ZMT to be
365. 14:52specific. It has really low evaporation
366. 14:55through the tubing, and it's not a
367. 14:58showcase build. It doesn't need to look
368. 15:00like anything special.
369. 15:02In goes our GPU now, or at least primary
370. 15:05main water cooled GPU. Oh man, it's
371. 15:08heavy. Oh. Oh, yeah. I might actually
372. 15:13keep these fittings. These are my
373. 15:15ancient Bits Power true silver fittings.
374. 15:17And the main sticktick for these is that
375. 15:20they're actually plated in silver. And
376. 15:22silver has biocidal properties, which is
377. 15:24great for maintaining a clean loop.
378. 15:26These are old school. These are like
379. 15:2820-year-old fittings or something like
380. 15:29that. But that's one of the things about
381. 15:31water cooling is that if you invest in
382. 15:32really nice stuff, as long as you're not
383. 15:33constantly chasing the latest fashions,
384. 15:35you can reuse it for many, many years.
385. 15:37And crazily enough, for not really any
386. 15:40more than like a pre- AIO cooled 5090,
387. 15:44we've got kind of like an AIO cooled
388. 15:475090 for such a high-end board, there's
389. 15:49really not that much in the way of front
390. 15:51panel tom foolery. No RGB support even.
391. 15:54So, this is just going to be a clean
392. 15:56build.
393. 15:58Oh, rats. I went and did the thing. I
394. 16:01installed the GPU before hooking up
395. 16:03everything and now I got to pop it out.
396. 16:05It's okay. That's no big deal. Thanks to
397. 16:07my trusty LTT socks. You thought I was
398. 16:10going to say screwdriver, didn't you?
399. 16:13Bam. No sweat. Or well, like you can
400. 16:15have a bit of sweat, but like you know,
401. 16:17[music] they're they got natural marino
402. 16:19wool in them, so that helps with the
403. 16:20wicking. For storage, once again, the
404. 16:23theme of the build is stuff that's
405. 16:25really cool, but we didn't really have
406. 16:26any use for it. So, I've got this Optane
407. 16:28SSD. Go ahead and huck that in there.
408. 16:32It's good for that to be right on the
409. 16:33exhaust of a 5090, right? I think so.
410. 16:37It's got to be.
411. 16:40All right.
412. 16:43Sure.
413. 16:45Now, let's get the power supply hooked
414. 16:47up. Legit excited to check this out.
415. 16:49It's been years since I've unboxed a
416. 16:51power supply that I'm like, whoa,
417. 16:54power supply, man. But like
418. 16:583,000 watts,
419. 17:00what are we even talking about here?
420. 17:02Now, it is important to note that when
421. 17:05operating on 120 volt power, it will not
422. 17:08be able to do the full 300 watts because
423. 17:10you would trip your breaker. Obviously,
424. 17:12in fact, this doesn't say anything about
425. 17:16accepting a 120 volt input at all.
426. 17:21I thought that like other, you know,
427. 17:24greater than 1600 watt power supplies,
428. 17:26it might at least accept both and then
429. 17:28just run at a lower capacity. [sighs]
430. 17:33Why did they bother to send this blah
431. 17:36blah blah blah the wrench rush current
432. 17:38blah blah blah DC output blah blah blah
433. 17:42blah blah blah blah blah absolutely
434. 17:44nothing about running at 120 volts. Oh,
435. 17:48Lucas.
436. 17:50Power supply question for you.
437. 17:52Silverstone makes no mention whatsoever
438. 17:54of 120 volt input.
439. 17:58>> Yeah, this one's Yeah. 220 to 240. So, I
440. 18:02wouldn't uh
441. 18:03>> I mean, you could try it on 120, but it
442. 18:05might uh might not go well. It's
443. 18:08probably not built for it. I mean, it
444. 18:10depends on the architecture. Maybe it's
445. 18:12it'll work, but uh I wouldn't recommend
446. 18:15it.
447. 18:15>> Hi ho. Hi ho. It's back in the box it
448. 18:18goes. So, we're going to go with this
449. 18:20Zeus 1650R titanium. It's got dual 12vt
450. 18:232x6 connectors. It should
451. 18:28still maybe have enough capacity for
452. 18:31this build. But just in case it doesn't,
453. 18:35I am not going to install any radiators
454. 18:36or anything in the bottom because hey,
455. 18:39remember I can always put in a second
456. 18:41power supply if it comes to that. Now we
457. 18:43just
458. 18:45got to get it all wired up.
459. 18:47[clears throat] Okay, there's a 24 pin
460. 18:50and the rest. This board takes an
461. 18:53absolutely obscene number of power
462. 18:55inputs. So, in addition to the 24 pin,
463. 18:58we've got up to another two eight pins.
464. 19:01So, quad 8 pin. I would imagine I would
465. 19:03only need that if I'm overclocking the
466. 19:05stuffing out of it, though. And then
467. 19:07down here, we've got an 8 pin PCIe and a
468. 19:10six pin PCIe. And these ones are really
469. 19:12important because every one of these
470. 19:15cards is allowed to draw up to 75 watts
471. 19:17through the PCIe slot. Well, with seven
472. 19:21slots, that could be up to 1
473. 19:27>> 525 watts. That would all be over the
474. 19:32like I think two 12vt wires that are in
475. 19:35the 24 pin connector. So, we need these
476. 19:37supplementary power connectors coming
477. 19:39into the board if we're going to have a
478. 19:40bunch of GPUs in here and fill this
479. 19:42thing up in the longer term. Okay, here
480. 19:45comes one of the 12vt 2x6s. This is not
481. 19:49a super long cable.
482. 19:51>> We'll make it work.
483. 19:53>> There you go. Well, whatever you do,
484. 19:54don't put strain on it, though.
485. 19:55>> Right. Right. Right.
486. 19:57>> Speaking of, uh, this is where things
487. 19:59get a little tricky. As you can imagine,
488. 20:00my power supply does not have a this.
489. 20:04That's where this little cable I made
490. 20:07for myself comes in. Nice. Right now,
491. 20:11one of the cool things about 12vt
492. 20:13[music] 2x6 is that it allows the power
493. 20:15supply and the GPU to talk to each other
494. 20:17to [music] determine how much wattage is
495. 20:19needed by that GPU and whether that
496. 20:21power supply can support it. So, that's
497. 20:22why you'll have little markings like
498. 20:24this 600 W on your cables. Um, obviously
499. 20:28this contains no such communication
500. 20:31pins. So, if you look closely, I've
501. 20:34actually jumpered the sense wires in
502. 20:37there. So, they tell it, "Yeah, pull 600
503. 20:39watts. Everything's good. Don't worry
504. 20:41about it."
505. 20:44>> I guess you don't care about this back
506. 20:46fan.
507. 20:47>> What? How did that get out there?
508. 20:49>> It's in between the IO.
509. 20:51>> Are you kidding me?
510. 20:56that. [laughter]
511. 20:58[panting]
512. 21:01Wait,
513. 21:02>> who mounted the motherboard again?
514. 21:04>> Oh, [laughter]
515. 21:06it was me.
516. 21:10[laughter]
517. 21:11>> No,
518. 21:15[laughter]
519. 21:19sad.
520. 21:22What's the Oh, what's the fastest way
521. 21:23for us to fix this? Uh oh, man. Okay.
522. 21:29>> Do we Holy balls. Okay, here's what
523. 21:33we're going to do.
524. 21:34>> Scissors.
525. 21:39>> Do you need it?
526. 21:42>> Scissors.
527. 21:43>> It Oh, I hate that. Okay, I'm going to
528. 21:46go get another fan and I'm going to go
529. 21:48get a splitter. Find a pair of scissors.
530. 21:50Don't let the people see it.
531. 21:53Packing all this hardware in was [music]
532. 21:55practically effortless. And it it
533. 21:58doesn't even it doesn't even look that
534. 22:00full. You know, it got two 5090s in
535. 22:04here. No problem.
536. 22:04>> I think you need two more.
537. 22:06>> Well, see, you see, I did leave space
538. 22:09just in case.
539. 22:11I guess it's time for us to uh grab a
540. 22:14monitor and look at software with some
541. 22:15of the guys from the labs. Most of what
542. 22:17I've been using this for lately has been
543. 22:19like home assistant stuff. Obviously,
544. 22:20none of this is loading because we're
545. 22:22not on my local network. Um, but where
546. 22:24we left off last time on software was we
547. 22:26got LM Studio set up. So, we've got a
548. 22:28local LLM that's running on both of the
549. 22:305090s. I couldn't believe how easy it
550. 22:31was to get running when I stopped
551. 22:33faffing about with Adysius. Um, but you
552. 22:36guys were saying that what I really need
553. 22:37to be doing is running Hermes. So, I
554. 22:39figured I'd bring you guys in so you can
555. 22:41kind of give us a run through of what
556. 22:43would be your basic starter Hermes
557. 22:47setup,
558. 22:48>> right? Uh, well, step one, let's install
559. 22:50Hermes.
560. 22:51>> Yeah, sure.
561. 22:52>> You guys haven't met Stephen from the
562. 22:53lab, but where would they have seen some
563. 22:56of your work? Like, you're credited on
564. 22:58some of the website stuff, right?
565. 22:59>> Yeah, I do testing for a lot of the labs
566. 23:01articles. Uh, so you see my name there.
567. 23:03Uh,
568. 23:04>> there you go.
569. 23:04>> Frequently.
570. 23:05>> He's in a Vessie ad, I think. So
571. 23:07>> I am in a vessad. So I'm on the uh
572. 23:10Hermes web page and they have a
573. 23:12convenient install command. So I will
574. 23:14run said command even though I'm not
575. 23:18entirely sure what it does.
576. 23:22>> Are you sure? Before we go any further,
577. 23:25Lucas, do you want to give us a quick
578. 23:26explanation of what Hermes is and why we
579. 23:28should want it?
580. 23:29>> It's just a an an agent, a harness, uh
581. 23:31something like that. It's something that
582. 23:32wraps around your LLM and runs it and
583. 23:34kind of gives it a system prompt. makes
584. 23:36other tools available and makes it like
585. 23:38kind of nice to work with. Um, and then
586. 23:40you know there's other integrations and
587. 23:41a whole kind of ecosystem you can build
588. 23:43around your LM and ties that up nicely
589. 23:46hopefully in theory, you know.
590. 23:48>> Yeah, I mean last time we were all
591. 23:49together I think I managed to get Steven
592. 23:52with three. I've never seen this before.
593. 23:55My ability to break Linux is truly
594. 23:57unprecedented. The whole idea for me of
595. 24:00running local AI is that I am minimizing
596. 24:04the amount of
597. 24:06paying that I'm doing after the fact. I
598. 24:08already paid a lot.
599. 24:09>> Got it.
600. 24:10>> So now we want to turn off the money
601. 24:12spigot.
602. 24:13>> Zero recurring payments. Yeah.
603. 24:15>> Yeah. That
604. 24:16>> for those unfamiliar, this is LM Studio.
605. 24:18This is where you actually run the LM
606. 24:19and it's where it's going to be hosted
607. 24:21um so that it's accessible for Hermes.
608. 24:23Um and but so you can host it on like
609. 24:25another machine if you want on Elm
610. 24:26Studio and just make it available over
611. 24:27the network. Um but for here we'll just
612. 24:29load the model. Uh so if we go here
613. 24:32it'll load the model and it gives you a
614. 24:34list of them. So Quinn 3.8 and you can
615. 24:36choose your parameters and all that
616. 24:37stuff.
617. 24:38>> We found the default parameters were
618. 24:40pretty good. I don't think there was
619. 24:41pretty much anything we needed to change
620. 24:42and it immediately was able to get both
621. 24:44of our GPUs working together which is
622. 24:46something that I had really struggled
623. 24:47with with Odysius.
624. 24:49>> Yeah, a lot of the defaults are just
625. 24:50makes sense. And then for context, I
626. 24:52think we're it's like 108k or something
627. 24:54like that.
628. 24:55>> Yeah.
629. 24:55>> And it's estimating that's going to take
630. 24:57up 25 or 29 gigs of um VRAM. So that's
631. 25:01good. That'll fit on one card even
632. 25:02though this is automatically uh
633. 25:04splitting across two.
634. 25:05>> Do we want to crank it because we have
635. 25:06two cards?
636. 25:07>> Uh we could crank it. Yeah. see what it
637. 25:09goes up to.
638. 25:10>> But that's not necessarily what might be
639. 25:12right for you because if you want to run
640. 25:14multiple models at the same time, each
641. 25:16is going to need its own VRAMm for both
642. 25:18to load the model and for its own
643. 25:20context.
644. 25:20>> So Hermes setup is asking us if we want
645. 25:24to set up messaging. We do want to use
646. 25:25Discord later, right?
647. 25:26>> I do. We might as well set it up.
648. 25:28>> Okay,
649. 25:28>> that's one of the really cool things
650. 25:30about Hermes is the interface can be as
651. 25:33simple as just the chat app that you
652. 25:35already have on your phone. It supports
653. 25:37Discord. I believe it supports Telegram.
654. 25:39>> I had a whole list. There's like Yeah.
655. 25:41Telegram, Discord, a whole bunch of
656. 25:42things.
657. 25:42>> Yeah. So, you just have a little like
658. 25:45chat. Well, yeah. For your chat bot.
659. 25:49>> Yeah. I think you can like put it into
660. 25:51servers or just chat to it like directly
661. 25:53like direct message.
662. 25:54>> Create a new application. There it is.
663. 25:56Look at that. What is it going to be
664. 25:58called? It's going to be called I feel
665. 26:00like everyone names them Jarvis.
666. 26:03>> Hermes botana.
667. 26:06>> No. Sent Lana. Nice.
668. 26:11>> You're assistant now. [laughter]
669. 26:13>> Creative.
670. 26:14>> Yeah. Yeah. I'm human for now. Maybe
671. 26:18what we'll do is cut out some of the
672. 26:19configuration and do a companion labs
673. 26:21article. ltlabs.com. We'll have that
674. 26:22link down below.
675. 26:24>> Okay. I'll let you do the honors. You
676. 26:26can run Hermes.
677. 26:28>> Here we go. Here we go. Huge moment.
678. 26:33First chat. We can fire up LM Studio and
679. 26:36see if it's actually doing anything. I
680. 26:38don't
681. 26:38>> think it's doing anything yet.
682. 26:39>> Hold on. Where's hardware?
683. 26:42>> It's not doing anything.
684. 26:43>> It's not doing anything. All right,
685. 26:45>> let's just ask the agent.
686. 26:49>> Question mark.
687. 26:52>> Perfect.
688. 26:56>> Yeah, I think so. I think those
689. 26:57processing.
690. 26:58>> It's doing stuff.
691. 26:58>> It's doing things.
692. 27:04This feels like in the early days of the
693. 27:05internet when you'd go to like load a
694. 27:08video and then you'd go make a sandwich
695. 27:10and then come back
696. 27:11>> the image is loading line by line.
697. 27:13[laughter]
698. 27:14>> This is a great example of how even
699. 27:17running on something like a 5090. It's
700. 27:20not really that fast. Like it's still
701. 27:22just thinking. I have no output
702. 27:24whatsoever.
703. 27:26But if whatever it is that you're doing
704. 27:29is not really timesensitive, the fact
705. 27:32that it can just be sitting there
706. 27:34churning away at a task can be
707. 27:36incredibly [snorts] powerful. So it'll
708. 27:39sit and it'll implement something and
709. 27:41then test it, then troubleshoot it, then
710. 27:43test it again, then troubleshoot it,
711. 27:44then test it again. And all that can
712. 27:46happen while I sleep. [music]
713. 27:50Who needs water or electricity when we
714. 27:52have this?
715. 27:55After a couple small tweaks to the ENV
716. 27:57file, theoretically Sventana is up and
717. 28:00running. So, it's Friday. I'm kind of
718. 28:03thinking I'm hosting the WAN show today.
719. 28:05Any breaking tech news stories that we
720. 28:07should talk about?
721. 28:09Searching the web. Searching the web.
722. 28:12And this is a great example of how this
723. 28:14would be the sort of thing that you
724. 28:16would want to schedule to run so that
725. 28:17your digest is just kind of waiting for
726. 28:19you in the morning.
727. 28:21I realize I've said this a few times,
728. 28:23but local AI is best served ahead
729. 28:26[clears throat] of time.
730. 28:29Can be useful, but it's not fast. Okay.
731. 28:33AI agents going rogue on government
732. 28:35targets. Wow, it's really heavily
733. 28:38indexing AI stuff. Good lord, it's all
734. 28:42AI. I feel like we have just scratched
735. 28:45the very tip of the iceberg here.
736. 28:48I think it does get more useful uh if
737. 28:50you can get into your once you get into
738. 28:51your own homes and you can integrate
739. 28:53with home assistant and get it more into
740. 28:55your daily routine or something like
741. 28:56that and actually do instead of you know
742. 28:58on our weird network and not connected
743. 29:00to anything.
744. 29:01>> All right. Well, I'm looking forward to
745. 29:03playing around with it more and more and
746. 29:05especially integration with Home
747. 29:07Assistant. I want to be able to do
748. 29:09things like recognize certain objects as
749. 29:12they approach the house and send
750. 29:14notifications or warnings. Motion
751. 29:16sensors are not great because when it's
752. 29:18heavily raining, I noticed last night my
753. 29:19lights were all on.
754. 29:20>> It could set up a rule for detect the
755. 29:22weather. If it's raining, don't actually
756. 29:24turn it on.
757. 29:24>> Yeah. Or something similar. Um that'll
758. 29:26be lower latency, but it like detect
759. 29:28those patterns with the data it can take
760. 29:30in. Uh
761. 29:31>> super cool.
762. 29:31>> Set that up for you.
763. 29:33>> You know what else is cool? Telling you
764. 29:35about our sponsor,
765. 29:37>> Squarespace. It's the all-in-one website
766. 29:39platform that helps with everything from
767. 29:41building a personalized site for your
768. 29:43business all the way to opening an
769. 29:44online store. Create something unique to
770. 29:47brand with their design intelligence
771. 29:48tool. All you need to do is answer a
772. 29:51couple quick questions, pick a general
773. 29:53vibe, and you have a custom template. If
774. 29:55you want to sell from your website,
775. 29:57Squarespace can help with payment
776. 29:59processing. It works with major payment
777. 30:01methods like Apple Pay, Credit, Debit,
778. 30:04and payment plans. They'll even assist
779. 30:06with SEO tools, making sure that your
780. 30:08website finds the right audience when
781. 30:10people search for you online. There are
782. 30:12millions of URLs to choose from to help
783. 30:14with this, too. Track your performance,
784. 30:17adjust on the fly, and keep growing your
785. 30:19business. We've even used Squarespace
786. 30:21here at LMG. Start building your
787. 30:24websites today and receive 10% off your
788. 30:26first purchase by visiting
789. 30:28squarespace.com/ltt.
790. 30:31If you guys enjoyed this video, why not
791. 30:33check out the one where uh Lucas and I
792. 30:35were playing around with those GPUs that
793. 30:37can have a whole bunch of latent hidden
794. 30:40VRAM unlocked. We've got one of them
795. 30:42just like still running over there. It's
796. 30:44been working great as far as I know.
797. 30:46Yeah,
798. 30:46>> it started with 8 gigs of VRAM and now
799. 30:48it has 64.