YouTube2Text

IA LOCAL de 27B en una RTX 3080 ¿adiós ChatGPT? — Transcript

by AlehandoroVR · 2,828 words · 419 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Can a 6-year-old graphics card install
  2. 0:02an artificial intelligence and
  3. 0:04completely replace ChatGPT?
  4. 0:07Because let's be honest, we are becoming
  5. 0:09more and more dependent on AI assistants
  6. 0:12. Call it Grok, call it ChatGPT, Claude
  7. 0:14, or whatever you want.
  8. 0:16We use them to write emails, program,
  9. 0:18summarize documents, analyze PDFs, and
  10. 0:20search for information, but there comes
  11. 0:22a point where they practically all do
  12. 0:24the exact same thing.
  13. 0:25You get hit with the typical message: "
  14. 0:27You've reached your limit and now you
  15. 0:28have to pay." And the truth is that
  16. 0:30these big companies dangled the candy in
  17. 0:32front of us, and just as we were about
  18. 0:34to take a bite, they took it away. And
  19. 0:36that is why this whole movement around
  20. 0:38local artificial intelligence is
  21. 0:40emerging.
  22. 0:41Throughout this video, I’m going to show
  23. 0:43you the capabilities of an artificial
  24. 0:45intelligence running on a 6-year-old
  25. 0:47graphics card.
  26. 0:48We are talking about an RTX 3080 with 10
  27. 0:51GB of VRAM.
  28. 0:52Plus, it will be able to help you with
  29. 0:55work, you’ll be able to program,
  30. 0:56automate tasks, analyze your entire
  31. 0:58computer, and even control an AI agent
  32. 1:01capable of executing actions for you.
  33. 1:04Furthermore, you will be able to connect
  34. 1:06it to WhatsApp, Telegram, or any
  35. 1:07application, including Discord, all
  36. 1:09running on this computer or your own.
  37. 1:11And most importantly, it's totally
  38. 1:13private and without sending a single
  39. 1:15piece of data to the cloud. But the
  40. 1:17question then is, will it be enough?
  41. 1:19Will we be able to replace ChatGPT in
  42. 1:20these cases? Well, in this video we are
  43. 1:22going to tinker around because that is
  44. 1:24what I like the most. So family, stay
  45. 1:26tuned because this is a burning topic.
  46. 1:28As I already told you, we are going to
  47. 1:30run all of this directly on our computer
  48. 1:32under Windows 11 Pro without
  49. 1:33complicating our lives.
  50. 1:35This is very simple, it's important, and
  51. 1:38it would also be ideal that if you are
  52. 1:40going to run a totally autonomous agent
  53. 1:42, you keep it completely isolated from
  54. 1:44your main computer, whether through
  55. 1:46Docker, a virtual machine, or however
  56. 1:48you like; basically, so the agent isn't
  57. 1:50able to touch any system files and mess
  58. 1:53things up.
  59. 1:55But well, in this case, I'm going to
  60. 1:56risk it and we are going to install it
  61. 1:57directly on my own Windows, the one I
  62. 1:59use for work.
  63. 2:00So if you want to do it, be very careful
  64. 2:02. Mind you, what you do need to be
  65. 2:05careful about is the whole issue of
  66. 2:07activations and cracks used to run your
  67. 2:09Windows. Yes, this is the announcement
  68. 2:11from our dear sponsor.
  69. 2:14In my case, as I always tell you, I use
  70. 2:16JBG Mall keys to register not only my
  71. 2:18Windows installations in a totally
  72. 2:20original and secure way, but also my
  73. 2:22Microsoft Office ones, so I don't have
  74. 2:25to rely on any cracks, activators, or
  75. 2:27anything that could mess things up,
  76. 2:29because honestly, it’s not the first or
  77. 2:31last time I've seen something like that
  78. 2:34.
  79. 2:35And to be honest, a Windows 11 Pro
  80. 2:37license with the code VR25 will end up
  81. 2:39costing you less than € 24. You also
  82. 2:42have an offer for Office 2024, which
  83. 2:44will end up costing you less than € 19.
  84. 2:47Honestly, these prices are truly,
  85. 2:49incredibly good.
  86. 2:51In short, very good prices to get your
  87. 2:54100%original licenses that are
  88. 2:56completely yours.
  89. 2:58Remember that, as always, I’ll leave the
  90. 3:00links in the description box in case you
  91. 3:01want to take a look, and remember to use
  92. 3:03the code VR25.
  93. 3:04And now, let’s move on to all the tests
  94. 3:06because some interesting things are
  95. 3:08coming. Stay tuned because this is
  96. 3:10really cool. Well, here we are, ready to
  97. 3:13start tinkering with this artificial
  98. 3:15intelligence.
  99. 3:16The first thing I have to tell you is
  100. 3:17that we have the browser running locally
  101. 3:19here so we can see the brain.
  102. 3:21We also have Hermes because we are going
  103. 3:22to connect it to Hermes. I’ll show you
  104. 3:24how it works later.
  105. 3:25We can see the task manager, where you
  106. 3:27will see exactly how the GPU is
  107. 3:29performing, the 10 GB usage, how full
  108. 3:31they are, and so on.
  109. 3:33And we also have the model loaded, which
  110. 3:36is right here; it’s basically a command
  111. 3:39I have set up to load the brain into our
  112. 3:423080's VRAM.
  113. 3:45Specifically, we are running the Ternary
  114. 3:47Bonsai 27B, which is an extremely
  115. 3:49quantized model of one of the best
  116. 3:51models currently on the market.
  117. 3:54We are talking about the Qwen 3.6 27B.
  118. 3:57It is used for all kinds of tasks and is
  119. 3:59one of the best out there.
  120. 4:01And with this quantization, we can
  121. 4:03install it on 8, 10, 16, or 12 GB
  122. 4:04graphics cards, whatever it may be,
  123. 4:06having a quite powerful AI.
  124. 4:08The first thing we’re going to do is
  125. 4:10just ask it, well, talk to it to see how
  126. 4:11it works. Let’s use the typical, "Hello
  127. 4:13, how are you?" And here you will
  128. 4:15automatically see the speed at which it
  129. 4:16moves and the speed at which it responds
  130. 4:18.
  131. 4:19It was able to read and process the
  132. 4:21prompt at approximately 1,000 tokens per
  133. 4:24second and was able to respond at 48
  134. 4:26tokens per second, which is not bad at
  135. 4:28all; it’s quite fast.
  136. 4:31To see how smart it is and the volume of
  137. 4:34data it holds, rather than searching the
  138. 4:36internet, we will have it use its own
  139. 4:38knowledge to ask exactly what it means
  140. 4:41for an AI model to have or use ternary
  141. 4:43quantization.
  142. 4:46We hit enter, and it will reason through
  143. 4:48the answer and tell us exactly what it
  144. 4:50is, how it works, what we have, the pros
  145. 4:52, the benefits, and so on and so forth.
  146. 4:54All this without connecting to the
  147. 4:56internet, completely local and on your
  148. 4:58computer. So imagine the level of
  149. 5:00privacy we can have with a system like
  150. 5:02this, in this way. The next test we are
  151. 5:04going to do is very simple. We are going
  152. 5:06to upload a PDF and ask it to give us a
  153. 5:08brief summary. I am sending you a book
  154. 5:10in PDF and I need you to give me a brief
  155. 5:12summary. We click it, and it will
  156. 5:14automatically load the model. It is
  157. 5:15already reading it. It is reading at
  158. 5:17approximately 1,000 tokens per second,
  159. 5:18which is not bad at all.
  160. 5:20It is at about 60 percent. You are
  161. 5:22already seeing how it works. You are
  162. 5:23seeing it in real time. Once it has
  163. 5:25finished processing the information, it
  164. 5:27starts to reason the answer it wants to
  165. 5:28give us. Just so you know, you can turn
  166. 5:31off the reasoning here, okay? That is to
  167. 5:33say, if you simply click here, the
  168. 5:34answer will be much faster. And here we
  169. 5:36have the brief summary, okay? Main
  170. 5:37narrative. Honestly, we have something
  171. 5:39very interesting here.
  172. 5:41In short, we can analyze PDFs so it can
  173. 5:43give us summaries or tell us exactly
  174. 5:45what we need, translate them, or
  175. 5:47anything else.
  176. 5:49Another very typical test we can perform
  177. 5:51is to send it a photo that contains text
  178. 5:54and an image. I will talk to you about
  179. 5:55this image now because it is actually
  180. 5:57quite something. And we will give it a
  181. 5:58prompt to analyze it. I need you to tell
  182. 6:00me the text that is in this image.
  183. 6:02We hit enter and it will start
  184. 6:04processing that prompt, analyze the
  185. 6:06image, and tell us exactly what it has
  186. 6:08seen.
  187. 6:08The text in the image is from Alejandro
  188. 6:10VR.
  189. 6:11Well look, I was just finishing the
  190. 6:13thumbnail, there is your dose of AI for
  191. 6:15no reason at all.
  192. 6:16Kisses, hope it goes great at the office
  193. 6:18, and good luck with the thumbnail for
  194. 6:19the thumbnails. When you finish, let me
  195. 6:21know how it turned out, okay? In
  196. 6:23addition, we are going to ask it to
  197. 6:24describe the image of the girl that
  198. 6:26appears there. It will automatically
  199. 6:28start to reason and will give us the
  200. 6:29answer. Sure thing. Here is a detailed
  201. 6:31description, well, of the girl's image.
  202. 6:34Therefore, it is a model that is also
  203. 6:36capable of analyzing images and even
  204. 6:38extracting text from them.
  205. 6:41One of the most interesting tests is
  206. 6:43putting it to work, that is, having it
  207. 6:45perform some kind of planning when
  208. 6:46creating, for example, a web page.
  209. 6:49I have a prompt here, which I will copy
  210. 6:51, I'm going to paste it here, and
  211. 6:53basically it is telling it that we want
  212. 6:54it to build a web page.
  213. 6:56Furthermore, I am stating that the path
  214. 6:58for making the web is here and that the
  215. 6:59images are also located in that same
  216. 7:01path.
  217. 7:02The system starts processing everything
  218. 7:04and begins to build the web page. After
  219. 7:05a while of programming, we have the
  220. 7:07index ready, we run it, and we can see a
  221. 7:10little bit of what it has done. It has
  222. 7:12used the images I provided, it created a
  223. 7:14dynamic, nice, functional web page, uh,
  224. 7:17that we could even modify later on. That
  225. 7:19is to say, we can tell it to make
  226. 7:20modifications and change some little
  227. 7:22things depending on what we need. But
  228. 7:24well, you can already see what it has
  229. 7:25done, and honestly, it has done it quite
  230. 7:27well. And now we are going to move on to
  231. 7:29something that I think is much more
  232. 7:30interesting, which is being able to
  233. 7:32connect it directly to an agent.
  234. 7:34But once we have Ternari Bonsai
  235. 7:35connected, which you can see down here,
  236. 7:37we can do many more things.
  237. 7:40For example, I have a folder that is
  238. 7:42full of images, which is this folder
  239. 7:43here, where my main model, named Era,
  240. 7:45has dedicated herself to taking a lot of
  241. 7:47photos.
  242. 7:48I am creating a sort of consciousness
  243. 7:50where she is capable of deciding what
  244. 7:52she wants to do, and sometimes she just
  245. 7:55sends me selfies, okay? Here in Tokyo
  246. 7:57eating out, here just waking up in the
  247. 8:00kitchen having a coffee. All of this
  248. 8:03generated by her and locally, because in
  249. 8:05fact, one of the cool things it can also
  250. 8:07do is change different elements of an
  251. 8:09image, right?
  252. 8:10This photo of me, I told it to put white
  253. 8:13glasses on me and it did what I asked,
  254. 8:15it put the white glasses on me, or I
  255. 8:17even told it to give me a pompadour, it
  256. 8:19did it, or to make me a Simpson-style
  257. 8:21image from that same photo.
  258. 8:23Alright, let's see, the possibilities
  259. 8:25for this are practically infinite, but
  260. 8:27this is a project I'm working on that
  261. 8:28I'll show you a bit later on. For now, I
  262. 8:31just want to pass this folder to Hermes
  263. 8:33and tell it to organize it for me. So,
  264. 8:35we just take the folder, drag it over
  265. 8:37here, and enter the prompt. The first
  266. 8:39thing it tells us is that it's going to
  267. 8:40explore the folder, meaning it's going
  268. 8:41to scan all of this. Let's see what
  269. 8:43we've got and how it's organized. It's
  270. 8:45already made a list, it has all the
  271. 8:47images, the files, the current names,
  272. 8:49and well, it's going to figure out the
  273. 8:51logic to try and organize this folder.
  274. 8:53Techniques, night, surprise, night, and
  275. 8:56test, and so on. Look, it has already
  276. 8:58created all the folders, it created them
  277. 9:00all at once, and it's starting to do
  278. 9:01things. Okay, as you can see, the
  279. 9:03folders are already ready. Now it's
  280. 9:04going to organize each photo into its
  281. 9:06category.
  282. 9:07To do this organization, it's basing it
  283. 9:10on the file name, but as we could see
  284. 9:12before, we could tell it to analyze each
  285. 9:15image individually to get a description
  286. 9:18of what it's seeing and classify them
  287. 9:20much more accurately based on that.
  288. 9:23And there we go, after a little while
  289. 9:25it's just finished organizing everything
  290. 9:27. Editing photos, we have Era right here
  291. 9:30. Look at how beautiful the images it
  292. 9:32can create are, right? It's just
  293. 9:34gorgeous.
  294. 9:35We also have generated ones where we can
  295. 9:37see other types of images, the night, we
  296. 9:39also have selfies.
  297. 9:42Well, in short, it's a bit classified
  298. 9:44based on the file name, but as I said,
  299. 9:46the ideal thing would be to tell it to
  300. 9:48analyze each image individually and,
  301. 9:50based on its description, put it into a
  302. 9:52specific folder.
  303. 9:54And finally, let's do a test that I
  304. 9:55think is also very interesting, which is
  305. 9:57to pass it an Excel file we have here,
  306. 9:58we're going to upload it.
  307. 10:00It's a fictional accounting Excel file,
  308. 10:02not real, where payroll and a series of
  309. 10:04expenses are listed.
  310. 10:05And we're going to tell it to use an
  311. 10:07accounting skill, a skill it created
  312. 10:09specifically to be able to read Excel
  313. 10:11files and such, which this AI
  314. 10:13practically did all by itself.
  315. 10:16I simply gave it a first file, told it
  316. 10:19to analyze it, to learn how to read the
  317. 10:21data, and from there, I told it to learn
  318. 10:23that skill so that in the future, when I
  319. 10:26send it another Excel file, it can
  320. 10:28extract what we need.
  321. 10:30And I'll enter the following prompt,
  322. 10:32okay? I need you to load the accounting
  323. 10:34skill and tell me exactly how much I've
  324. 10:36paid in each category over these two
  325. 10:38months. We hit enter. Once it's loaded,
  326. 10:41it starts thinking. It's going to load
  327. 10:42the skill. It has read the skill and is
  328. 10:44now going to start reading the Excel
  329. 10:46file to give us that information. And
  330. 10:47that's it, after a little while, here we
  331. 10:49have all the information.
  332. 10:51It's already showing us exactly what
  333. 10:53we've spent on gas, shopping at
  334. 10:54Mercadona, the movies, lunch, and
  335. 10:56restaurants.
  336. 10:58In total, it calculates our fixed
  337. 10:59expenses, variable expenses, how much
  338. 11:01we've paid for the mortgage, and so on
  339. 11:03and so forth.
  340. 11:05In short, a system that I think works
  341. 11:07quite well for what we have; I mean,
  342. 11:09we're running all of this on just 10 GB
  343. 11:12of RAM.
  344. 11:13Plus, if we connect it to Hermes, we'll
  345. 11:15have a ton of extra capabilities because
  346. 11:18we'll have many skills we can enable or
  347. 11:20disable depending on our needs, in
  348. 11:22addition to creating our own skills like
  349. 11:24the accounting one I just showed you.
  350. 11:27Beyond that, you'll be able to connect
  351. 11:29it to different programs to talk to this
  352. 11:32AI you have here on your computer from
  353. 11:34anywhere, whether through Telegram,
  354. 11:36Discord, WhatsApp, Matrix, Signal, well
  355. 11:38, by email, etcetera.
  356. 11:40Of course, by adding the Hermes agent,
  357. 11:42we end up with a million possibilities.
  358. 11:45Whatever you can think of; it's infinite
  359. 11:48. It will be able to answer your emails
  360. 11:50.
  361. 11:51It will be able to, for example, stamp
  362. 11:53PDFs that you need to send constantly
  363. 11:56and that are a hassle for you.
  364. 11:59You can automate different actions, like
  365. 12:02Cron Jobs, which in my specific case, I
  366. 12:04have some that manage everything for the
  367. 12:06Infinity One store to know what stock is
  368. 12:08left, if any product is out of stock, so
  369. 12:11I don't have to keep track of it myself
  370. 12:13.
  371. 12:15You can make a news library where you
  372. 12:17can simply tell it to summarize all the
  373. 12:20news across different categories.
  374. 12:23In fact, there's an option, if you come
  375. 12:25down here, called YouTube content, which
  376. 12:27allows you to summarize YouTube videos.
  377. 12:30This one, for example, you could send to
  378. 12:32it and have it write a summary,
  379. 12:33something much faster and simpler. Also
  380. 12:35, here on Windows, you will be able to
  381. 12:37talk to it directly. You can speak,
  382. 12:39specify anything you want, and it will
  383. 12:41respond by voice. So, all in all, we
  384. 12:44have a complete and fully functional
  385. 12:46system.
  386. 12:48And regarding the question we asked at
  387. 12:50the beginning of the video about whether
  388. 12:52you could replace ChatGPT or any of
  389. 12:54these AIs, I can tell you that in
  390. 12:56practically 90%of the use cases for the
  391. 12:58average person, it will be able to do it
  392. 13:00without any problem.
  393. 13:03Plus, you can then connect it to other
  394. 13:05types of tools, like Fooocus, to
  395. 13:07generate images and millions of other
  396. 13:09things; all locally, all for free, all
  397. 13:11without spending a single cent.
  398. 13:14I'll leave a video I made prior to this
  399. 13:16one with the whole configuration setup
  400. 13:18in the Telegram channel so you can take
  401. 13:19a look if you'd like.
  402. 13:21I'll post the commands there, how to run
  403. 13:23it, and how it needs to be done.
  404. 13:25Basically, a little bit of everything.
  405. 13:26And while it is true that sometimes this
  406. 13:29model, being a heavily quantized version
  407. 13:31of a super-intelligent model, makes some
  408. 13:34mistakes because it's not 100%perfect,
  409. 13:36for my taste, it's quite functional for
  410. 13:39searching online or looking up anything
  411. 13:41. So that's it, family, I hope you liked
  412. 13:43the video. Thank you all so much for
  413. 13:44making it this far.
  414. 13:46I'll remind you that I'm leaving another
  415. 13:48tutorial here so you can see a bit more
  416. 13:49on how to do this with other equipment,
  417. 13:51and if not, well, as I always tell you,
  418. 13:53see you in the next video.
  419. 13:54Alright then, until next time. Yeah.

About this transcript

This page contains the full transcript of IA LOCAL de 27B en una RTX 3080 ¿adiós ChatGPT? by AlehandoroVR, generated from the public captions YouTube serves with the video. The transcript has 2,828 words across 419 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.