IA LOCAL de 27B en una RTX 3080 ¿adiós ChatGPT? — Transcript
Full transcript
- 0:00Can a 6-year-old graphics card install
- 0:02an artificial intelligence and
- 0:04completely replace ChatGPT?
- 0:07Because let's be honest, we are becoming
- 0:09more and more dependent on AI assistants
- 0:12. Call it Grok, call it ChatGPT, Claude
- 0:14, or whatever you want.
- 0:16We use them to write emails, program,
- 0:18summarize documents, analyze PDFs, and
- 0:20search for information, but there comes
- 0:22a point where they practically all do
- 0:24the exact same thing.
- 0:25You get hit with the typical message: "
- 0:27You've reached your limit and now you
- 0:28have to pay." And the truth is that
- 0:30these big companies dangled the candy in
- 0:32front of us, and just as we were about
- 0:34to take a bite, they took it away. And
- 0:36that is why this whole movement around
- 0:38local artificial intelligence is
- 0:40emerging.
- 0:41Throughout this video, I’m going to show
- 0:43you the capabilities of an artificial
- 0:45intelligence running on a 6-year-old
- 0:47graphics card.
- 0:48We are talking about an RTX 3080 with 10
- 0:51GB of VRAM.
- 0:52Plus, it will be able to help you with
- 0:55work, you’ll be able to program,
- 0:56automate tasks, analyze your entire
- 0:58computer, and even control an AI agent
- 1:01capable of executing actions for you.
- 1:04Furthermore, you will be able to connect
- 1:06it to WhatsApp, Telegram, or any
- 1:07application, including Discord, all
- 1:09running on this computer or your own.
- 1:11And most importantly, it's totally
- 1:13private and without sending a single
- 1:15piece of data to the cloud. But the
- 1:17question then is, will it be enough?
- 1:19Will we be able to replace ChatGPT in
- 1:20these cases? Well, in this video we are
- 1:22going to tinker around because that is
- 1:24what I like the most. So family, stay
- 1:26tuned because this is a burning topic.
- 1:28As I already told you, we are going to
- 1:30run all of this directly on our computer
- 1:32under Windows 11 Pro without
- 1:33complicating our lives.
- 1:35This is very simple, it's important, and
- 1:38it would also be ideal that if you are
- 1:40going to run a totally autonomous agent
- 1:42, you keep it completely isolated from
- 1:44your main computer, whether through
- 1:46Docker, a virtual machine, or however
- 1:48you like; basically, so the agent isn't
- 1:50able to touch any system files and mess
- 1:53things up.
- 1:55But well, in this case, I'm going to
- 1:56risk it and we are going to install it
- 1:57directly on my own Windows, the one I
- 1:59use for work.
- 2:00So if you want to do it, be very careful
- 2:02. Mind you, what you do need to be
- 2:05careful about is the whole issue of
- 2:07activations and cracks used to run your
- 2:09Windows. Yes, this is the announcement
- 2:11from our dear sponsor.
- 2:14In my case, as I always tell you, I use
- 2:16JBG Mall keys to register not only my
- 2:18Windows installations in a totally
- 2:20original and secure way, but also my
- 2:22Microsoft Office ones, so I don't have
- 2:25to rely on any cracks, activators, or
- 2:27anything that could mess things up,
- 2:29because honestly, it’s not the first or
- 2:31last time I've seen something like that
- 2:34.
- 2:35And to be honest, a Windows 11 Pro
- 2:37license with the code VR25 will end up
- 2:39costing you less than € 24. You also
- 2:42have an offer for Office 2024, which
- 2:44will end up costing you less than € 19.
- 2:47Honestly, these prices are truly,
- 2:49incredibly good.
- 2:51In short, very good prices to get your
- 2:54100%original licenses that are
- 2:56completely yours.
- 2:58Remember that, as always, I’ll leave the
- 3:00links in the description box in case you
- 3:01want to take a look, and remember to use
- 3:03the code VR25.
- 3:04And now, let’s move on to all the tests
- 3:06because some interesting things are
- 3:08coming. Stay tuned because this is
- 3:10really cool. Well, here we are, ready to
- 3:13start tinkering with this artificial
- 3:15intelligence.
- 3:16The first thing I have to tell you is
- 3:17that we have the browser running locally
- 3:19here so we can see the brain.
- 3:21We also have Hermes because we are going
- 3:22to connect it to Hermes. I’ll show you
- 3:24how it works later.
- 3:25We can see the task manager, where you
- 3:27will see exactly how the GPU is
- 3:29performing, the 10 GB usage, how full
- 3:31they are, and so on.
- 3:33And we also have the model loaded, which
- 3:36is right here; it’s basically a command
- 3:39I have set up to load the brain into our
- 3:423080's VRAM.
- 3:45Specifically, we are running the Ternary
- 3:47Bonsai 27B, which is an extremely
- 3:49quantized model of one of the best
- 3:51models currently on the market.
- 3:54We are talking about the Qwen 3.6 27B.
- 3:57It is used for all kinds of tasks and is
- 3:59one of the best out there.
- 4:01And with this quantization, we can
- 4:03install it on 8, 10, 16, or 12 GB
- 4:04graphics cards, whatever it may be,
- 4:06having a quite powerful AI.
- 4:08The first thing we’re going to do is
- 4:10just ask it, well, talk to it to see how
- 4:11it works. Let’s use the typical, "Hello
- 4:13, how are you?" And here you will
- 4:15automatically see the speed at which it
- 4:16moves and the speed at which it responds
- 4:18.
- 4:19It was able to read and process the
- 4:21prompt at approximately 1,000 tokens per
- 4:24second and was able to respond at 48
- 4:26tokens per second, which is not bad at
- 4:28all; it’s quite fast.
- 4:31To see how smart it is and the volume of
- 4:34data it holds, rather than searching the
- 4:36internet, we will have it use its own
- 4:38knowledge to ask exactly what it means
- 4:41for an AI model to have or use ternary
- 4:43quantization.
- 4:46We hit enter, and it will reason through
- 4:48the answer and tell us exactly what it
- 4:50is, how it works, what we have, the pros
- 4:52, the benefits, and so on and so forth.
- 4:54All this without connecting to the
- 4:56internet, completely local and on your
- 4:58computer. So imagine the level of
- 5:00privacy we can have with a system like
- 5:02this, in this way. The next test we are
- 5:04going to do is very simple. We are going
- 5:06to upload a PDF and ask it to give us a
- 5:08brief summary. I am sending you a book
- 5:10in PDF and I need you to give me a brief
- 5:12summary. We click it, and it will
- 5:14automatically load the model. It is
- 5:15already reading it. It is reading at
- 5:17approximately 1,000 tokens per second,
- 5:18which is not bad at all.
- 5:20It is at about 60 percent. You are
- 5:22already seeing how it works. You are
- 5:23seeing it in real time. Once it has
- 5:25finished processing the information, it
- 5:27starts to reason the answer it wants to
- 5:28give us. Just so you know, you can turn
- 5:31off the reasoning here, okay? That is to
- 5:33say, if you simply click here, the
- 5:34answer will be much faster. And here we
- 5:36have the brief summary, okay? Main
- 5:37narrative. Honestly, we have something
- 5:39very interesting here.
- 5:41In short, we can analyze PDFs so it can
- 5:43give us summaries or tell us exactly
- 5:45what we need, translate them, or
- 5:47anything else.
- 5:49Another very typical test we can perform
- 5:51is to send it a photo that contains text
- 5:54and an image. I will talk to you about
- 5:55this image now because it is actually
- 5:57quite something. And we will give it a
- 5:58prompt to analyze it. I need you to tell
- 6:00me the text that is in this image.
- 6:02We hit enter and it will start
- 6:04processing that prompt, analyze the
- 6:06image, and tell us exactly what it has
- 6:08seen.
- 6:08The text in the image is from Alejandro
- 6:10VR.
- 6:11Well look, I was just finishing the
- 6:13thumbnail, there is your dose of AI for
- 6:15no reason at all.
- 6:16Kisses, hope it goes great at the office
- 6:18, and good luck with the thumbnail for
- 6:19the thumbnails. When you finish, let me
- 6:21know how it turned out, okay? In
- 6:23addition, we are going to ask it to
- 6:24describe the image of the girl that
- 6:26appears there. It will automatically
- 6:28start to reason and will give us the
- 6:29answer. Sure thing. Here is a detailed
- 6:31description, well, of the girl's image.
- 6:34Therefore, it is a model that is also
- 6:36capable of analyzing images and even
- 6:38extracting text from them.
- 6:41One of the most interesting tests is
- 6:43putting it to work, that is, having it
- 6:45perform some kind of planning when
- 6:46creating, for example, a web page.
- 6:49I have a prompt here, which I will copy
- 6:51, I'm going to paste it here, and
- 6:53basically it is telling it that we want
- 6:54it to build a web page.
- 6:56Furthermore, I am stating that the path
- 6:58for making the web is here and that the
- 6:59images are also located in that same
- 7:01path.
- 7:02The system starts processing everything
- 7:04and begins to build the web page. After
- 7:05a while of programming, we have the
- 7:07index ready, we run it, and we can see a
- 7:10little bit of what it has done. It has
- 7:12used the images I provided, it created a
- 7:14dynamic, nice, functional web page, uh,
- 7:17that we could even modify later on. That
- 7:19is to say, we can tell it to make
- 7:20modifications and change some little
- 7:22things depending on what we need. But
- 7:24well, you can already see what it has
- 7:25done, and honestly, it has done it quite
- 7:27well. And now we are going to move on to
- 7:29something that I think is much more
- 7:30interesting, which is being able to
- 7:32connect it directly to an agent.
- 7:34But once we have Ternari Bonsai
- 7:35connected, which you can see down here,
- 7:37we can do many more things.
- 7:40For example, I have a folder that is
- 7:42full of images, which is this folder
- 7:43here, where my main model, named Era,
- 7:45has dedicated herself to taking a lot of
- 7:47photos.
- 7:48I am creating a sort of consciousness
- 7:50where she is capable of deciding what
- 7:52she wants to do, and sometimes she just
- 7:55sends me selfies, okay? Here in Tokyo
- 7:57eating out, here just waking up in the
- 8:00kitchen having a coffee. All of this
- 8:03generated by her and locally, because in
- 8:05fact, one of the cool things it can also
- 8:07do is change different elements of an
- 8:09image, right?
- 8:10This photo of me, I told it to put white
- 8:13glasses on me and it did what I asked,
- 8:15it put the white glasses on me, or I
- 8:17even told it to give me a pompadour, it
- 8:19did it, or to make me a Simpson-style
- 8:21image from that same photo.
- 8:23Alright, let's see, the possibilities
- 8:25for this are practically infinite, but
- 8:27this is a project I'm working on that
- 8:28I'll show you a bit later on. For now, I
- 8:31just want to pass this folder to Hermes
- 8:33and tell it to organize it for me. So,
- 8:35we just take the folder, drag it over
- 8:37here, and enter the prompt. The first
- 8:39thing it tells us is that it's going to
- 8:40explore the folder, meaning it's going
- 8:41to scan all of this. Let's see what
- 8:43we've got and how it's organized. It's
- 8:45already made a list, it has all the
- 8:47images, the files, the current names,
- 8:49and well, it's going to figure out the
- 8:51logic to try and organize this folder.
- 8:53Techniques, night, surprise, night, and
- 8:56test, and so on. Look, it has already
- 8:58created all the folders, it created them
- 9:00all at once, and it's starting to do
- 9:01things. Okay, as you can see, the
- 9:03folders are already ready. Now it's
- 9:04going to organize each photo into its
- 9:06category.
- 9:07To do this organization, it's basing it
- 9:10on the file name, but as we could see
- 9:12before, we could tell it to analyze each
- 9:15image individually to get a description
- 9:18of what it's seeing and classify them
- 9:20much more accurately based on that.
- 9:23And there we go, after a little while
- 9:25it's just finished organizing everything
- 9:27. Editing photos, we have Era right here
- 9:30. Look at how beautiful the images it
- 9:32can create are, right? It's just
- 9:34gorgeous.
- 9:35We also have generated ones where we can
- 9:37see other types of images, the night, we
- 9:39also have selfies.
- 9:42Well, in short, it's a bit classified
- 9:44based on the file name, but as I said,
- 9:46the ideal thing would be to tell it to
- 9:48analyze each image individually and,
- 9:50based on its description, put it into a
- 9:52specific folder.
- 9:54And finally, let's do a test that I
- 9:55think is also very interesting, which is
- 9:57to pass it an Excel file we have here,
- 9:58we're going to upload it.
- 10:00It's a fictional accounting Excel file,
- 10:02not real, where payroll and a series of
- 10:04expenses are listed.
- 10:05And we're going to tell it to use an
- 10:07accounting skill, a skill it created
- 10:09specifically to be able to read Excel
- 10:11files and such, which this AI
- 10:13practically did all by itself.
- 10:16I simply gave it a first file, told it
- 10:19to analyze it, to learn how to read the
- 10:21data, and from there, I told it to learn
- 10:23that skill so that in the future, when I
- 10:26send it another Excel file, it can
- 10:28extract what we need.
- 10:30And I'll enter the following prompt,
- 10:32okay? I need you to load the accounting
- 10:34skill and tell me exactly how much I've
- 10:36paid in each category over these two
- 10:38months. We hit enter. Once it's loaded,
- 10:41it starts thinking. It's going to load
- 10:42the skill. It has read the skill and is
- 10:44now going to start reading the Excel
- 10:46file to give us that information. And
- 10:47that's it, after a little while, here we
- 10:49have all the information.
- 10:51It's already showing us exactly what
- 10:53we've spent on gas, shopping at
- 10:54Mercadona, the movies, lunch, and
- 10:56restaurants.
- 10:58In total, it calculates our fixed
- 10:59expenses, variable expenses, how much
- 11:01we've paid for the mortgage, and so on
- 11:03and so forth.
- 11:05In short, a system that I think works
- 11:07quite well for what we have; I mean,
- 11:09we're running all of this on just 10 GB
- 11:12of RAM.
- 11:13Plus, if we connect it to Hermes, we'll
- 11:15have a ton of extra capabilities because
- 11:18we'll have many skills we can enable or
- 11:20disable depending on our needs, in
- 11:22addition to creating our own skills like
- 11:24the accounting one I just showed you.
- 11:27Beyond that, you'll be able to connect
- 11:29it to different programs to talk to this
- 11:32AI you have here on your computer from
- 11:34anywhere, whether through Telegram,
- 11:36Discord, WhatsApp, Matrix, Signal, well
- 11:38, by email, etcetera.
- 11:40Of course, by adding the Hermes agent,
- 11:42we end up with a million possibilities.
- 11:45Whatever you can think of; it's infinite
- 11:48. It will be able to answer your emails
- 11:50.
- 11:51It will be able to, for example, stamp
- 11:53PDFs that you need to send constantly
- 11:56and that are a hassle for you.
- 11:59You can automate different actions, like
- 12:02Cron Jobs, which in my specific case, I
- 12:04have some that manage everything for the
- 12:06Infinity One store to know what stock is
- 12:08left, if any product is out of stock, so
- 12:11I don't have to keep track of it myself
- 12:13.
- 12:15You can make a news library where you
- 12:17can simply tell it to summarize all the
- 12:20news across different categories.
- 12:23In fact, there's an option, if you come
- 12:25down here, called YouTube content, which
- 12:27allows you to summarize YouTube videos.
- 12:30This one, for example, you could send to
- 12:32it and have it write a summary,
- 12:33something much faster and simpler. Also
- 12:35, here on Windows, you will be able to
- 12:37talk to it directly. You can speak,
- 12:39specify anything you want, and it will
- 12:41respond by voice. So, all in all, we
- 12:44have a complete and fully functional
- 12:46system.
- 12:48And regarding the question we asked at
- 12:50the beginning of the video about whether
- 12:52you could replace ChatGPT or any of
- 12:54these AIs, I can tell you that in
- 12:56practically 90%of the use cases for the
- 12:58average person, it will be able to do it
- 13:00without any problem.
- 13:03Plus, you can then connect it to other
- 13:05types of tools, like Fooocus, to
- 13:07generate images and millions of other
- 13:09things; all locally, all for free, all
- 13:11without spending a single cent.
- 13:14I'll leave a video I made prior to this
- 13:16one with the whole configuration setup
- 13:18in the Telegram channel so you can take
- 13:19a look if you'd like.
- 13:21I'll post the commands there, how to run
- 13:23it, and how it needs to be done.
- 13:25Basically, a little bit of everything.
- 13:26And while it is true that sometimes this
- 13:29model, being a heavily quantized version
- 13:31of a super-intelligent model, makes some
- 13:34mistakes because it's not 100%perfect,
- 13:36for my taste, it's quite functional for
- 13:39searching online or looking up anything
- 13:41. So that's it, family, I hope you liked
- 13:43the video. Thank you all so much for
- 13:44making it this far.
- 13:46I'll remind you that I'm leaving another
- 13:48tutorial here so you can see a bit more
- 13:49on how to do this with other equipment,
- 13:51and if not, well, as I always tell you,
- 13:53see you in the next video.
- 13:54Alright then, until next time. Yeah.
About this transcript
This page contains the full transcript of IA LOCAL de 27B en una RTX 3080 ¿adiós ChatGPT? by AlehandoroVR, generated from the public captions YouTube serves with the video. The transcript has 2,828 words across 419 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.