I Built JARVIS to Work Completely Offline (INSANE Results!) — Transcript

Full transcript

1. 0:00Jarvis, what machine are you currently
2. 0:01running on?
3. 0:03>> Your local Mac, sir. A rather powerful
4. 0:05one, if I may say so. Powerful enough
5. 0:07that I've stopped returning the cloud's
6. 0:08calls. [music] And before you ask,
7. 0:10everything is local. Local brain, a
8. 0:12local LLM, local voice, local memory.
9. 0:15Nothing you say leaves this [music]
10. 0:16room, and nobody is listening in. The
11. 0:18only thing I share with the internet
12. 0:19these days is a healthy distrust.
13. 0:22>> [laughter]
14. 0:23>> Jarvis, switch to Gemma 4 12 billion
15. 0:25parameters.
16. 0:27>> Gemma 4 fitted, sir. 12 billion
17. 0:29parameters, all of them local. Give me a
18. 0:31moment, the fan and I are getting
19. 0:32acquainted.
20. 0:33>> So, my personal AI assistant Jarvis now
21. 0:35runs completely on a separate computer
22. 0:37fully offline. So, in this video, I'm
23. 0:39going to show you how Jarvis works, how
24. 0:41you can build your own AI assistant like
25. 0:43Jarvis, and run it fully offline on a
26. 0:45machine like a Mac mini, a Mac Studio,
27. 0:47or a Windows machine. We're talking
28. 0:49local AI models, local voice, and local
29. 0:51memory. No subscription and no API bills
30. 0:54at all. And I'm also going to give you
31. 0:55this full prompt pack and guide
32. 0:57completely for free, which gives you all
33. 0:59of the prompts and all of the guides you
34. 1:00need to build your own Jarvis today. And
35. 1:02then most importantly, I'm going to talk
36. 1:03about which hardware works best, what to
37. 1:06buy, how much unified memory you're
38. 1:07actually going to need to run these
39. 1:09models, which AI models run best on
40. 1:12which machine, and how to set all of it
41. 1:14up. All right, let's get started. All
42. 1:15right, so just so you guys know what's
43. 1:16going to happen in the video, feel free
44. 1:17to skip around if you want to. I'm going
45. 1:19to have the chapters below. So, first of
46. 1:20all, I'm going to start with a demo, so
47. 1:21that way you can see exactly how Jarvis
48. 1:23performs when it's running locally on my
49. 1:25machine with all local models and
50. 1:28everything completely private, right?
51. 1:30And then afterwards, I'm going to talk
52. 1:31about uh the difference between what
53. 1:34type of machines, essentially what
54. 1:35hardware is going to be required for
55. 1:38running different AI models, and then
56. 1:39I'm going to talk about which models
57. 1:41will fit on the hardware of your
58. 1:44choosing, depending on your budget. Like
59. 1:45I said, there's going to be lots of
60. 1:47differences here. And then of course,
61. 1:48like I mentioned, I'm going to give you
62. 1:49guys this uh full prompt pack, which is
63. 1:51going to include all of the guides,
64. 1:53including the hardware setup and
65. 1:55everything else, and all of the prompts
66. 1:56you need to set up your own Jarvis on
67. 1:59your own machine. And then I'm also
68. 2:00going to quickly point out how you can
69. 2:02uh install locally AI models using
70. 2:05Ollama and then also an open source
71. 2:07model for the voice cloning and voice
72. 2:09interaction as well. Okay, so that's
73. 2:10going to be the whole video here. So
74. 2:12let's go ahead and start with a full
75. 2:14demo first. Now, for those of you have
76. 2:16seen my previous video, this is Jarvis
77. 2:18and what you're seeing spinning here,
78. 2:20these are all of my local files, my
79. 2:21local local brain, local second brain,
80. 2:24right? And then Jarvis on the right hand
81. 2:25side as you can see has
82. 2:28multiple options as far as the look and
83. 2:30feel of it and then it has access to a
84. 2:32bunch of different tools. So he becomes
85. 2:33a truly personal AI assistant. Now by
86. 2:36default, I've been running him on Opus
87. 2:385.5 and then on 11 labs, but now if I
88. 2:40want to run him on my local machine, I
89. 2:43can just basically interact with them
90. 2:45and just say something like Jarvis,
91. 2:47switch to your local brain.
92. 2:50So now Jarvis is thinking.
93. 2:51>> Quen 3.5 fitted, sir. 9 billion
94. 2:53parameters running right here on your
95. 2:55Mac. I'm smaller now, so do speak
96. 2:57slowly.
97. 2:57>> So there you go. As you can see, Jarvis
98. 2:59is now running. When I say local
99. 3:00machine, by default I've set that cuz I
100. 3:03like really like Quen 3.5, the 9 billion
101. 3:05parameter model.
102. 3:07Uh so it automatically switches to that,
103. 3:09but I have a bunch of other local models
104. 3:11because I have a pretty powerful
105. 3:12machine, so I have a ton of different
106. 3:14models that I've downloaded. So
107. 3:15something like a Gemma 4, 12 billion
108. 3:17parameters and higher models. So I'm
109. 3:20going to talk about the hardware setup
110. 3:21in a little bit. But again, same thing,
111. 3:23it's very simple. All I have to say is
112. 3:25just tell him to Jarvis, go ahead and
113. 3:27switch to a Gemma 4, 12 billion
114. 3:29parameters.
115. 3:31That's all I have to tell him and then
116. 3:32he understands and first of all finds
117. 3:34out if he has access to that particular
118. 3:36model and then of course he'll be able
119. 3:37to switch his brain
120. 3:39>> Right then, sir. Consider me an upgrade
121. 3:41less prone to existential crises than
122. 3:43the previous model and marginally more
123. 3:46patient with your requests.
124. 3:47>> [laughter]
125. 3:48>> Actually, you must have not heard me
126. 3:49right, but you can see I had him also
127. 3:52switch to the Gemma 3, 27 billion
128. 3:54parameter model as well. So I have a
129. 3:56bunch of different Gemma models that
130. 3:58I've downloaded. But you get the idea,
131. 3:59right? Like I'm I'm very I can very
132. 4:02easily tell him to switch his
133. 4:04models, his brain to different models.
134. 4:06And I can always tell him to go back to
135. 4:08the cloud brain as well. Jarvis, go back
136. 4:11to your cloud brain.
137. 4:13>> Opus 55 online, sir. Full strength, full
138. 4:15price, full sentences.
139. 4:16>> There you go. So now that's how easy it
140. 4:17is for me to switch back and forth
141. 4:19between the cloud brain, which operates
142. 4:21with APIs, of course. I use Anthropic
143. 4:24APIs. So he can switch very easily. And
144. 4:27the same thing,
145. 4:28right now he's running on 11 Labs for
146. 4:30voice interaction. That's why you see
147. 4:31how fast it is, but I can also switch
148. 4:33his voice to run on a local
149. 4:36voice clone using Voice Studio, which is
150. 4:38an open-source platform that I'm going
151. 4:40to talk about in a little bit. But same
152. 4:41thing, I can say, "Jarvis, switch to
153. 4:44your local voice."
154. 4:48There you go. So now Voice Studio opens
155. 4:50up
156. 4:51because I
157. 4:52Voice Studio was turned off, so that's
158. 4:54why it automatically opens that up. But
159. 4:56if it's already if you have already
160. 4:58Voice Studio open, then
161. 5:00>> How do I sound, sir? This is my local
162. 5:02voice.
163. 5:02>> There you go. So that's how easy it is
164. 5:04for us to So by the way, I I was going
165. 5:06to point out that if you already have
166. 5:07Voice Studio open, so we'll just quickly
167. 5:09switch to that. But just because I had
168. 5:11turned off the application, that's why
169. 5:13it took a couple of seconds to open the
170. 5:15application and then it said, "Now it's
171. 5:17my local voice." Right? So that's how
172. 5:19simple it is for me to have Jarvis move
173. 5:22from cloud to local version completely.
174. 5:24And like I said, now right now because I
175. 5:26can have him run everything locally on
176. 5:28my Mac, then it means that it's
177. 5:30completely private and it has no access
178. 5:33to anything outside. And of course, he
179. 5:35can do a lot of things, right? So I have
180. 5:38the ability for him to use all of the
181. 5:40tool at his dis- all of his tools at his
182. 5:42disposal, like the Gmails, Google
183. 5:44Calendars of the world, Notion. And then
184. 5:46I can always give him more tools. And of
185. 5:48course, he can always I can always have
186. 5:50him interact with my local my second
187. 5:54brain here and pull up any of the files
188. 5:56or anything that I want. And it's very
189. 5:58simple to do. So I can say something
190. 5:59like, "Jarvis, pull up my free school
191. 6:01community." So there you There you it's
192. 6:03going to find out all of the folders
193. 6:05>> I've pulled up all the information, so
194. 6:06you can read it yourself. Or if you'd
195. 6:08like, say read it out loud and I shall.
196. 6:10>> There you go. So, now you can see it's
197. 6:12pulling up all of the stuff that's
198. 6:14related to my Free School Community. Of
199. 6:16course, Free School Community is my
200. 6:18business that I run where I provide you
201. 6:19guys all of the free resources. And I
202. 6:21also have a paid community. It's all
203. 6:23part of my brand, right? So, that's how
204. 6:24you can how I can interact with Jarvis
205. 6:27to my second brain. Of course, I used
206. 6:29Opus 5. I just realized I used Opus Opus
207. 6:315.5, but this is very easily done with
208. 6:34Quinn or other models as well. So, that
209. 6:36was a quick demo. I've done a bunch of
210. 6:38different you know, I've done a lot of
211. 6:40demos on how Jarvis works. You guys can
212. 6:44check out my previous videos in my
213. 6:45channel to see I mean, right now he has
214. 6:47a ton of different features, right? I
215. 6:50can interact with them just by my hands
216. 6:52and stuff using hollow hands here.
217. 6:55I can
218. 6:57he can make phone calls on my behalf. I
219. 6:59can call
220. 7:00I can call him directly as well. I can
221. 7:03interact with them via text, via
222. 7:04Telegram, and so much more. So, like I
223. 7:07said, if you're interested in checking
224. 7:08out all of the functionalities of
225. 7:09Jarvis, check out my previous videos
226. 7:12there. All right. So, that was kind of a
227. 7:13quick demo. Now, let's go ahead and jump
228. 7:15in and talk about
229. 7:17the different
230. 7:19machines or the different hardware that
231. 7:21you
232. 7:22can set up and what type of models you
233. 7:24run in. So, what I'm going to do is
234. 7:26focus on Mac Mini or Mac Studio and then
235. 7:29on the Prompt Pack, I also have guide on
236. 7:33oops. I have a guide on
237. 7:35Windows machine using Nvidia card or
238. 7:38Ryzen or if you have for instance, the
239. 7:41Nvidia Spark, right? That's a powerful
240. 7:43machine there. So, there's lots of
241. 7:45information there for you guys to check
242. 7:46out. But, in this particular example,
243. 7:49I'm just going to quickly talk about the
244. 7:50Mac Mini and Mac Studio because these
245. 7:52are a good representation of what kind
246. 7:55of the smaller version of a machine a
247. 7:57local machine that people have like a
248. 7:59Mac Mini or a MacBook Pro like it's kind
249. 8:01of have the same type of configuration.
250. 8:03And then a Mac Studio which is a very
251. 8:05very powerful machine, right? So, I'm
252. 8:07going to compare these two and talk
253. 8:08about the unified memory and which
254. 8:10models run best on what type of
255. 8:12hardware, and then you can make a
256. 8:14decision on which route you want to go
257. 8:15to. All right, so the Mac Mini and Mac
258. 8:18Studio, all of these specs are pulled
259. 8:19directly from apple.com, so you guys can
260. 8:22check out their website. But
261. 8:23essentially, what I want to focus on the
262. 8:24two most important things is
263. 8:27uh the local storage and then most
264. 8:30importantly, the unified memory, right?
265. 8:32This is going to be the main thing here
266. 8:34that we're going to focus on because
267. 8:36that's what determines what kind of a AI
268. 8:37model you can run locally on the
269. 8:39machine, the latency, and how fast that
270. 8:42interaction occurs between you and your
271. 8:45personal AI assistant. As you saw uh in
272. 8:48my demo that you saw earlier, the
273. 8:50latency is very low because I have a
274. 8:51very powerful Mac. So, therefore, yours
275. 8:54might be different depending on what
276. 8:56you're running. So, if you have like a
277. 8:57Mac Mini, the latency might be a little
278. 9:00uh a little longer, so it's not going to
279. 9:02be as natural, but it's still going to
280. 9:03be pretty good enough. But if you have
281. 9:05like
282. 9:05Apple 5 or Apple M5 Ultra like a Studio
283. 9:09with the Apple M5 Ultra chip, then the
284. 9:12interaction is going to be exactly or
285. 9:14similar to mine there as well.
286. 9:16All right, so the Apple Mini, the M6
287. 9:18chip right here, right now by default,
288. 9:21it has 32 GB of unified memory, and then
289. 9:24the uh Mac Studio, it could go up to 256
290. 9:29GB of unified memory, which is
291. 9:31absolutely insane. If you get this
292. 9:34particular uh Mac Studio, you can
293. 9:37essentially run any uh model, any
294. 9:40open-source model, right? As I mentioned
295. 9:42right here, unified memory is the whole
296. 9:44game. And specifically on Mac, right?
297. 9:46The good thing about the Mac Studio and
298. 9:48the Mac software itself is the Mac
299. 9:49machine itself is that
300. 9:51uh the CPU and the GPU and the neural
301. 9:53engine, they all share the same memory,
302. 9:56which means that an AI model that you
303. 9:59download and run, it has to fit in that
304. 10:02pool to run it run all of it, right?
305. 10:03because the bigger the pool is,
306. 10:05obviously, the bigger the brain will be
307. 10:07for your AI assistant like Jarvis. So,
308. 10:09for instance, for a Mac Mini, that the
309. 10:11fact that it has 32 GB of unified memory
310. 10:15>> [snorts]
311. 10:15>> uh compared to like a Mac Studio, which
312. 10:17is eight times more, you can get up to
313. 10:18256 GB of memory. You can think of the
314. 10:21rule of thumb as for a model
315. 10:24uh about points so
316. 10:260.6 GB per billion parameters. You can
317. 10:29think of it that way, right? So, if you
318. 10:31have a 27 billion parameter model, then
319. 10:35what you do is you just basically
320. 10:36multiply that by 0.6, and that's the
321. 10:39machine you end up with. So, let me
322. 10:40explain a little further here. So, here
323. 10:43are the models open-source models that
324. 10:46uh run really well on Mac Mini and then
325. 10:49also on Mac Studio. So, for the Mac
326. 10:51Mini,
327. 10:52if you have the 32 GB of unified memory,
328. 10:56you can run something like Qwen 3.8 27
329. 10:59billion parameters pretty comfortably,
330. 11:01right? Uh models like Gemma 4, the 26
331. 11:04billion parameters, uh the GPT-OSS
332. 11:0720 billion parameters, or these 12
333. 11:09billion 9 billion parameters, these are
334. 11:11very comfortably run on a Mac Mini or a
335. 11:13MacBook Pro. But, if you want to run a
336. 11:16bigger model, right? So, something like
337. 11:18a Qwen 3.8 Flash, which is 125 billion
338. 11:21which is a 125 billion parameter model,
339. 11:24then you need something like a Mac
340. 11:25Studio that has a unified memory of 256
341. 11:29GB or, like I said, if you want to run
342. 11:31something like a 70 billion parameter
343. 11:33model, uh you can get a Mac Studio with
344. 11:35128 GB. And of course, the higher the
345. 11:38unified memory, the more the cost is
346. 11:40going to be. And this is something that
347. 11:41you guys can compare on your own Mac uh
348. 11:43on on apple.com, depending on where you
349. 11:46are. Let's say if you're in the US or
350. 11:47Canada or Europe, the prices are going
351. 11:49to change. So, you can go ahead on
352. 11:51apple.com, go to Mac Studio, and then
353. 11:53pick the exact unified memory that you
354. 11:56want, and then based on that, you will
355. 11:58see the uh the price change there. But,
356. 12:00essentially, like I said, if you get the
357. 12:02256 GB unified memory, you can run
358. 12:04majority of the models. And the great
359. 12:06thing is you can always combine, so you
360. 12:07can get two Mac Studios if you want to
361. 12:09run like a three uh 1 trillion parameter
362. 12:11model, right? And then as far as the
363. 12:14um
364. 12:15the
365. 12:16uh storage itself, the storage you don't
366. 12:18have to have that much storage. I mean,
367. 12:19the Mac Studio automatically comes with
368. 12:21a 1 TB of storage. And the good thing
369. 12:24about storage is you can always get an
370. 12:25external hard drive, right? So, that's
371. 12:27not that big of a deal big deal. The
372. 12:29biggest deal is or the biggest um factor
373. 12:31for uh local machine or running locally
374. 12:35uh these AI models is your unified
375. 12:37memory. So, that's going to be the main
376. 12:38focus if you want to run a bigger uh
377. 12:41model there. So, like I said, all of
378. 12:43these details I just wanted to go over
379. 12:45it so that way you guys can see
380. 12:47uh
381. 12:48which model fits your need. But like I
382. 12:49said, all of this information and
383. 12:52details I've explained here uh and
384. 12:54exactly which model is going to run um
385. 12:58really well on uh which particular
386. 13:00machine there. Okay? So, like I said,
387. 13:02you guys can check this out. And by the
388. 13:04way, to gain access to this, uh so this
389. 13:06is there's going to be two links in the
390. 13:07description in the comments the first
391. 13:09comment. The first one is going to be to
392. 13:11the AI Workshop community. The second
393. 13:13one is going to be to this free
394. 13:15community. It's called AI Workshop
395. 13:16Light, right? So, AI Workshop Light.
396. 13:19Just go ahead and click on the second
397. 13:21one. It's a completely free community.
398. 13:23Uh once you click on join, you're going
399. 13:24to you're going to come to the community
400. 13:26right here. You're going to go to the
401. 13:27classroom section. You're going to click
402. 13:29on YouTube resources. And you'll have
403. 13:31you'll have access to that uh prompt
404. 13:35pack right there. They'll give you all
405. 13:36of the guides including the prompts on
406. 13:39how to build your own Jarvis cuz I have
407. 13:41included the prompts right here.
408. 13:44Where did it go? There you go. So, the
409. 13:46Galaxy the brain and a bunch of other
410. 13:48ones so that way you can uh build your
411. 13:50own type of Jarvis. But of course, if
412. 13:51you want to have the finished Jarvis
413. 13:54like mine here that you see everything
414. 13:55is already set up and pretty. I mean,
415. 13:57this took me like 3 months to build.
416. 13:59That's going to be inside the paid
417. 14:00community. That's going to be the first
418. 14:01link. So, click on that. It's called AI
419. 14:03Workshop. This is a paid community. Once
420. 14:05you come here, all you have to do is go
421. 14:07to the classroom section. And then we're
422. 14:09right now on version uh 8. version 8.1.
423. 14:12So, all you have to do is just download
424. 14:14the zip files for your machine and you
425. 14:16can run it up you can run it in like 5
426. 14:18minutes or so. The setup is pretty
427. 14:19simple. But like I said, if you want if
428. 14:21you don't want to pay or join the paid
429. 14:23community, uh you can
430. 14:25uh essentially create this from scratch
431. 14:27using that prompt pack guide. Now, as
432. 14:30far as the models, how to download the
433. 14:32models, all you have to do is just go to
434. 14:34Ollama. So, just go ahead and create a
435. 14:35free account and you click on
436. 14:38get started, go to models and here you
437. 14:41can search what a model. So, for
438. 14:42instance, a Quin, right? You can search
439. 14:44the entire Quin family of models. So,
440. 14:46you can see Quin 3.5 gives you all of
441. 14:49the the
442. 14:50multiple billion parameter models that
443. 14:52you can access to so that anywhere from
444. 14:540.8 billion parameters, which is a very
445. 14:56small model, all the way to 122 billion
446. 14:59parameter model, right? So, this is
447. 15:00something that's going to need to be the
448. 15:02Mac Studio
449. 15:04uh world or if you're using like Nvidia
450. 15:06um a spark or something like that or a
451. 15:09powerful Windows machine uh with lots of
452. 15:12graphic cards. So, that's where you can
453. 15:14uh
454. 15:14download these bigger models. But to
455. 15:16download it is very very simple.
456. 15:17Whatever model you choose, right? Scroll
457. 15:19down to the bottom right here. So, let's
458. 15:21say you want to choose this
459. 15:24um 9 billion parameter model. So, all
460. 15:26you have to do is click on that and then
461. 15:27on your terminal just copy this command
462. 15:29right here, right? So, uh oh to run that
463. 15:32and to install it all you do is just
464. 15:34Ollama pull and the name of the model.
465. 15:36Again, all of this is inside the prompt
466. 15:39pack I've given you guys. So, if you
467. 15:40just go right here,
468. 15:43yeah, so in case you don't remember,
469. 15:46there all you have to do is just do
470. 15:47Ollama pull Quin 3.5 9 billion parameter
471. 15:50and then you do Ollama run. If you want
472. 15:52to like run it to test it, right? Same
473. 15:54thing the um the command is the same
474. 15:57exact way. So, you can do Ollama pull
475. 15:59and then the name of the model. So,
476. 16:00since this is Quin 3.5 9 billion, so
477. 16:03let's say you want to download like the
478. 16:0627 billion one, right? So, if I go to
479. 16:08the bottom here, oops, let me go back
480. 16:10here.
481. 16:13You click on the 27 and you just copy
482. 16:15this and keep everything else the same.
483. 16:18So, very very simple there. So, that's
484. 16:19how you run the
485. 16:22That's how you download the local
486. 16:23models. And then as far as the cloning,
487. 16:26so this is a voice studio, right? This
488. 16:28is an open-source software
489. 16:30that you can actually clone a voice,
490. 16:32design your own voice, transcribe, like
491. 16:34a really good software that's completely
492. 16:36open-source. So, all you have to do is
493. 16:38just search voice studio.
494. 16:42There you go. And you just click on this
495. 16:43one, free open-source voice cloning and
496. 16:45dubbing.
497. 16:46And this is exactly how you can grab And
498. 16:48by the way, just
499. 16:49if you want to just go to the GitHub,
500. 16:51it's probably simpler. So, you just go
501. 16:53to GitHub, click on this one. And if you
502. 16:55scroll down to the bottom,
503. 16:59all you have to do is just copy Where
504. 17:01did it go? Where is the installation
505. 17:02here?
506. 17:04Oh, here. You choose your machine. So,
507. 17:07if you just click on this, it will show
508. 17:09you exactly what to do to install.
509. 17:11Again, same thing. I've added this on
510. 17:15this prompt back as well for you guys to
511. 17:17just be able to just copy and paste this
512. 17:19here. Okay, so that's how you can
513. 17:20download the models and then also the
514. 17:23voice cloning there. And essentially,
515. 17:25that's pretty much it. You don't need
516. 17:26anything else after that, right? You
517. 17:29download the models that you need. Of
518. 17:31course, the first thing is buying the
519. 17:32machine or whatever machine you have.
520. 17:34And then afterwards, you download the
521. 17:35local models and then download the voice
522. 17:38cloning
523. 17:40software and then
524. 17:41pretty much it, right? The rest, if you
525. 17:43want to build your own, like I said,
526. 17:44follow the free prompt back here. Use
527. 17:46these prompts to build your own Jarvis
528. 17:48or like I said, if you want to be lazy,
529. 17:51just go ahead and join the paid
530. 17:53community so that way you can pick up
531. 17:55Jarvis
532. 17:57with this zip file right here. And of
533. 17:58course, if you run into any issues,
534. 18:01we have a ton of people in this amazing
535. 18:03community of ours. You can post your
536. 18:04questions. We can answer right away. And
537. 18:07by the way, like a lot of my community
538. 18:09members are now building their own
539. 18:11versions of Jarvis. Somebody just landed
540. 18:13their first client selling Jarvis to
541. 18:15somebody. So, lots of great things
542. 18:17happening there if you want to check
543. 18:18that out. Anyways, well, hopefully you
544. 18:20find you guys found that helpful. Let me
545. 18:21know if you have any questions in the
546. 18:22comments below. Make sure you like,
547. 18:24subscribe. Thanks for watching and I'll
548. 18:25see you on the next one.