YouTube2Text

Ollama + Claude Code = 99% CHEAPER — Transcript

by Nate Herk | AI Automation · 6,096 words · 836 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Today, I'm going to be showing you guys
  2. 0:01two different ways that we can run
  3. 0:02Claude code for free. The first way is
  4. 0:04going to be by running the LLM locally
  5. 0:06on our own machine, and then the second
  6. 0:08way is using OpenRouter. Both of these
  7. 0:10ways are extremely easy, a lot easier
  8. 0:12than you would think, and I'm going to
  9. 0:13walk you through step-by-step both of
  10. 0:15them. And my goal is to cover everything
  11. 0:16that you need to know in order to run
  12. 0:17this and answer any questions that you
  13. 0:19may have. So, let's not waste any time
  14. 0:20and just get straight into the video.
  15. 0:22All right, so before we get into the
  16. 0:23tutorials of the two methods, there is
  17. 0:24some important stuff that we need to
  18. 0:25cover. The first one is about what
  19. 0:28actually is Claude code. And when I say,
  20. 0:29"How do you use Claude code for free?"
  21. 0:31what does that mean?
  22. 0:33So, this is the analogy that any AI
  23. 0:34model would pretty much give you, and it
  24. 0:36is the fact that Claude code is the car
  25. 0:38and the chat model, the AI model, is the
  26. 0:40engine. So, when we use Claude code
  27. 0:43natively, we are using basically a
  28. 0:45harness that is wrapped around Opus or
  29. 0:48Sonnet or one of Claude's models that's
  30. 0:50really, really smart. So, Claude code
  31. 0:52tells the model how to organize its
  32. 0:54folders and how to use tools and how to,
  33. 0:57you know, execute plans and build
  34. 0:58projects for you and stuff like that.
  35. 1:00And by default, we're using Opus,
  36. 1:02Sonnet, and Haiku. So, when you're
  37. 1:04thinking about your context limit or
  38. 1:05when you're hitting your token limit,
  39. 1:06things like that, that is because you're
  40. 1:08paying Anthropic for access to the
  41. 1:11Claude Opus model or the Sonnet model or
  42. 1:13the Haiku model. Now, what I'm going to
  43. 1:15show you guys how to do is we basically
  44. 1:16just open up the hood and we switch out
  45. 1:19the engine. So, we can use the
  46. 1:20OpenRouter models in their cloud, which
  47. 1:23are free, or we can use self-hosted
  48. 1:26local models, which live on our
  49. 1:27desktops, on our laptops, whatever it
  50. 1:29is, and those are also going to be free.
  51. 1:31And that means you don't run into any
  52. 1:33limits, or at least the way that the
  53. 1:34limits work are much different. So, now
  54. 1:36that we understand that, the question
  55. 1:37is, okay, why can't you just run Opus
  56. 1:39locally for free? Well, that is because
  57. 1:41there's a difference between open-source
  58. 1:43models and closed-source models. An
  59. 1:45open-source model basically means that
  60. 1:46the model is published somewhere so that
  61. 1:48anyone can download it. It is open. It's
  62. 1:51It's open-source. We can see inside of
  63. 1:53it. We can edit it. We can run it. We
  64. 1:54can modify it. And a closed-source model
  65. 1:56is closed. So, Sonnet and GPT Gemini
  66. 1:59models, those are kind of the top-tier
  67. 2:02leading closed-source models, and the
  68. 2:04only way that we can access them is
  69. 2:05through the company's API, which means
  70. 2:07in order to hit their API, we have to
  71. 2:08pay them for tokens. So, if you think
  72. 2:10back to the car analogy, when we have an
  73. 2:12open-source model, that means, you know,
  74. 2:13the car lets us open the hood and we can
  75. 2:16actually like look at the engine and
  76. 2:17change things. But with a closed-source
  77. 2:19model, the hood is locked shut and we
  78. 2:21can't get in there and we can't do
  79. 2:22anything to the engine. So, the question
  80. 2:23is, why are the majority of the people
  81. 2:25in the world using closed-source models?
  82. 2:27That's because they're typically just
  83. 2:29better. So, here's the chart. You can
  84. 2:30see in blue we have closed-source
  85. 2:32models. Up here we have Opus 4.6 and
  86. 2:34Sonnet 4.6. And then in the green we
  87. 2:36have open-source models. So, this green
  88. 2:38one right here is a Qwen model. So,
  89. 2:40what's really cool is there's always
  90. 2:42been a gap between the performance of
  91. 2:43closed-source models and the performance
  92. 2:45of open-source models, but that gap is
  93. 2:46just shrinking and shrinking. And I'm
  94. 2:48sure we're going to get to the point
  95. 2:49where they are just as competitive as
  96. 2:51closed-source models. And by the way,
  97. 2:52this chart is based on SWE bench
  98. 2:55verified scores, which is a, you know,
  99. 2:56kind of computer engineering programming
  100. 2:59sort of benchmark for AI models. Now,
  101. 3:01what's really cool is if we look at
  102. 3:02this, but we also overlay Sonnet 3.7 on
  103. 3:05top of this, we can see that Sonnet 3.7
  104. 3:07in the orange here is not even in the
  105. 3:08top five, which means some of the
  106. 3:11open-weight models that we can access
  107. 3:12today are better than Claude Sonnet 3.7.
  108. 3:15And if you remember, when that model
  109. 3:16dropped, everyone was freaking out. It
  110. 3:18was the best model at one point. So,
  111. 3:20just to put into perspective that the
  112. 3:21open-source models are really good, but
  113. 3:23still right now, if you want to do like
  114. 3:25heavy, heavy work and you've got
  115. 3:26something that you can't really mess up
  116. 3:27on, then Opus 4.6 is probably going to
  117. 3:30be the best or maybe GPT 5.4 with Codex.
  118. 3:33And because of that, there's a few
  119. 3:34reasons why open-source models in Claude
  120. 3:36code might seem to misbehave. They might
  121. 3:38have been not trained on Claude code's
  122. 3:40tools. They might have a context window
  123. 3:41that's too small for Claude's system
  124. 3:43prompt. And they might not follow the
  125. 3:45exact same JSON protocol that Claude
  126. 3:47code expects. So, once again, the car
  127. 3:49and the engine analogy, it might be like
  128. 3:51trying to put a motorcycle engine into a
  129. 3:53truck or maybe putting the wrong type of
  130. 3:55gas into your vehicle. So, sometimes you
  131. 3:57do have to play around with the models
  132. 3:58to see if they can do the proper tool
  133. 4:00calling that Claude code or your car
  134. 4:02actually expects. And you know what
  135. 4:04inspired me to make this video? It was
  136. 4:05the release of Google's new open-source
  137. 4:08model, Gemma 4. And what's really cool
  138. 4:10to see is this is a comparison of some
  139. 4:13other open-source models that are pretty
  140. 4:15good. On the x-axis, we have the size of
  141. 4:18these open-source models, and on the
  142. 4:19y-axis, we have their Elo score. And
  143. 4:21what you can see is that these Gemma 4
  144. 4:23models are the highest Elo score and the
  145. 4:27smallest size. And in this context, a
  146. 4:29small size better because that means
  147. 4:31that we can actually host it on our
  148. 4:32machine and our machine can handle it
  149. 4:34because it's basically like, how many
  150. 4:36gigabytes is the model? How many
  151. 4:37parameters does it have? Um, you need
  152. 4:39more hardware in order to run larger
  153. 4:42models. All right, so now you should
  154. 4:44have a good understanding of open-source
  155. 4:46models, closed-source models, and how
  156. 4:47this is going to work. The final thing
  157. 4:49is the question that people might be
  158. 4:50asking, is this against Anthropic's
  159. 4:52terms of service? And the answer to that
  160. 4:54is no. We're using their agent harness.
  161. 4:56We're just plugging in a different local
  162. 4:58model or open-source model. So, with all
  163. 5:00that out of the way, let's get into our
  164. 5:01first method, which is going to be
  165. 5:03running a local model. We're going to be
  166. 5:05using Ollama, and then we're going to
  167. 5:06hook that up into Claude code and see
  168. 5:09how it works. All right, so step one
  169. 5:10here is to go to ollama.com. Now, once
  170. 5:13you get here, what you're going to do is
  171. 5:14you're going to download Ollama, and
  172. 5:15you're going to do it for your operating
  173. 5:17system. So, here on Windows, I would
  174. 5:19download for Windows. Once everything is
  175. 5:21downloaded, we're going to have to pull
  176. 5:22in a model or download a model. So,
  177. 5:24you're going to click on models up in
  178. 5:25the top left, and now we just have to
  179. 5:26choose which model do we actually want
  180. 5:28to pull in and download. So, if we go
  181. 5:31over to OpenRouter, and we're not
  182. 5:32jumping to step two yet or to method two
  183. 5:34yet, OpenRouter just has this really
  184. 5:35cool way where we can actually look at
  185. 5:37rankings between different AI models.
  186. 5:39This page can be very, very slow.
  187. 5:41There's a lot of data here, so just keep
  188. 5:43that in mind. But you can see that for
  189. 5:44programming category, there's a lot of
  190. 5:47models here that are open-source. So,
  191. 5:49right here we can see there's a Qwen 3.6
  192. 5:50that's pretty good. There's a Mimo.
  193. 5:52There's a MiniMax. So, let's hop over to
  194. 5:54Ollama and see what we can do. So, let's
  195. 5:56go ahead and choose this Qwen 3.5 model.
  196. 5:59If I open this up, there's a few things
  197. 6:00that we can see, and we can see all of
  198. 6:02these different models. So, these are
  199. 6:03different sizes. We can also scroll down
  200. 6:05and we get some highlights. So, we can
  201. 6:07see like some benchmarks that they have,
  202. 6:08and these benchmarks will compare them
  203. 6:10to both other open-source, but also some
  204. 6:12other closed-source models, which is
  205. 6:13nice to be able to look at. But what
  206. 6:15we're concerned about right now is how
  207. 6:16we actually download this. So, you'll
  208. 6:18see that we have different model names,
  209. 6:20and we have different sizes. We also
  210. 6:21have different context windows and
  211. 6:23different inputs. Now, I'm not like a
  212. 6:24hardware expert, but what you want to
  213. 6:26think about is how much RAM you have on
  214. 6:28your computer. There's some other things
  215. 6:29to look at, you know, your CPU and your
  216. 6:31GPU. If I was you, I would just go to
  217. 6:33Claude code and say, "Hey, here are my
  218. 6:34specs of my hardware. Which model sizes
  219. 6:37should I be looking to download?" In
  220. 6:38this case, what I'm going to do is I'm
  221. 6:39going to go ahead and download this Qwen
  222. 6:413.5 9 billion parameter model, which is
  223. 6:446.6 gigabytes. Now, obviously, this is
  224. 6:46not going to be the best model ever.
  225. 6:48This one would be much better, but this
  226. 6:50is the demo, so I'm just going to show
  227. 6:51you how this will work. So, what we need
  228. 6:53to do is we're going to take this model
  229. 6:54name, and you can see up here it says
  230. 6:56that you can run Ollama run Qwen 3.5.
  231. 6:59But what we need to do first is download
  232. 7:00the model cuz this basically just lets
  233. 7:01us talk to it. So, we're going to do
  234. 7:03Ollama pull and then the model name.
  235. 7:05Now, you could just open up your search
  236. 7:07bar, search for terminal, and then do it
  237. 7:09right there. But what I'm going to do is
  238. 7:10I'm going to do this in the environment
  239. 7:11where we're actually going to be using
  240. 7:13this model in Claude code. So, in VS
  241. 7:15code, where I like to use it, I'm going
  242. 7:17to open up my terminal, and I'm just
  243. 7:18going to go ahead and do that command.
  244. 7:19So, Ollama
  245. 7:21pull
  246. 7:22and then the model that I want to pull.
  247. 7:24And when I hit enter, this is going to
  248. 7:26pull in this
  249. 7:27um, model. So, right now, it's actually
  250. 7:29downloading this locally on my machine.
  251. 7:31The bigger the model, the longer it will
  252. 7:32take to download, and you can see over
  253. 7:34here on the right, it kind of gives me
  254. 7:35this 1 minute 30 second estimation of
  255. 7:37how long it'll take. And by the way, all
  256. 7:39of these little indicators that you'll
  257. 7:40see next to a model just tells you about
  258. 7:42it. So, this one has vision, tools,
  259. 7:44thinking. You can also run it from the
  260. 7:46cloud, Ollama's cloud, and then it has
  261. 7:48these different sizes available.
  262. 7:50Whereas, if we go back to some other
  263. 7:51models, you can see that some of them
  264. 7:52might only be available in the cloud.
  265. 7:54Like MiniMax here, it has tools, it has
  266. 7:56thinking, but it's only a cloud use. So,
  267. 7:59you can see here, there's not nearly as
  268. 8:01many models, and we don't see the size
  269. 8:03because we can't download them locally,
  270. 8:05at least not from Ollama. But we could
  271. 8:06still launch them in Claude code, no
  272. 8:08problem, and it would still be free. All
  273. 8:10right, so now you can see that this
  274. 8:11model has been pulled in. Now, I can do
  275. 8:13Ollama run and then that same model
  276. 8:16name, and when I do that, this is now me
  277. 8:18talking to Qwen 3.5 on my machine. So, I
  278. 8:21can say hi. It's going to think. It's
  279. 8:23going to respond to me. And this is
  280. 8:24completely free because the model is
  281. 8:25running right down there on my desktop.
  282. 8:27Now, if I had better specs, this would
  283. 8:29be faster. If it was a bigger model, it
  284. 8:30would be smarter. But that's kind of the
  285. 8:32way that this works. All of this talking
  286. 8:34I'm doing, all of this brainstorming,
  287. 8:36all of this formatting of text I could
  288. 8:37be doing, is completely free. You can
  289. 8:39see this is a pretty robust answer and
  290. 8:41and reasoning process just for the word
  291. 8:43hi. But hey, we got the job done. All
  292. 8:46right, so the next step now, I'm going
  293. 8:47to exit out of that chat. Oops, it's
  294. 8:49control D, I guess. Now, the next step
  295. 8:51is to open up Claude code with our local
  296. 8:54model. So, if I go back into the
  297. 8:57Ollama here, and actually, I could just
  298. 8:59open up my app that it should install.
  299. 9:01And if you click on launch, you can see
  300. 9:03that there's this command that says
  301. 9:04Ollama launch Claude. And if you do
  302. 9:06that, it'll just basically pull up
  303. 9:07Claude. So, if I click on copy and I go
  304. 9:10back into our VS code where we just
  305. 9:11were, and all the way at the bottom I
  306. 9:13paste that, it's going to launch up
  307. 9:15Claude, but then it's going to let you
  308. 9:16choose which model you want to run
  309. 9:17Claude with. So, these are the three
  310. 9:19Gemmas that I had installed, or I could
  311. 9:21do the new one, Qwen 3.5
  312. 9:239 billion parameter. So, now it's going
  313. 9:24to launch Claude code. You can see that
  314. 9:26we are in the same directory here. We
  315. 9:28are on this model. It's API usage
  316. 9:30billing, and this should be free now.
  317. 9:33So, I know some people are going to ask
  318. 9:34about this, right? So, I just logged out
  319. 9:35of my account, and if I come in here and
  320. 9:37I launch Claude as if I was doing it
  321. 9:38with the Ollama thing, it goes through
  322. 9:40this onboarding. So, I would choose dark
  323. 9:41mode, and then I have to choose how to
  324. 9:43sign in. So, I can either use a
  325. 9:44subscription, which is like 20 bucks a
  326. 9:46month, and it gets more expensive, or
  327. 9:48you can use an API key. So, if I open up
  328. 9:50that method, it basically takes me to
  329. 9:52this place where I have to authorize
  330. 9:53myself into my Anthropic account, and
  331. 9:56then I do have to buy $5 of credits just
  332. 9:58to get started. But, those $5 you'll
  333. 10:00never actually consume because you're
  334. 10:02not going to pay here, you're going to
  335. 10:04pay nothing because you're using free
  336. 10:06open source models instead. So, now that
  337. 10:08that payment went through, you open up
  338. 10:10Claude Code, and you can now use Claude
  339. 10:12Code in here because you have, you know,
  340. 10:14set up that API connection. But, the
  341. 10:16thing is, if you start talking to it
  342. 10:17right now, like I just said, "Hey." That
  343. 10:19would actually charge my account. You
  344. 10:21can see the funds are even too low, so
  345. 10:22it won't charge you. But, this is where
  346. 10:24we would come in here and we'd switch
  347. 10:25this account to using our local model or
  348. 10:28our open source model, and then we'd be
  349. 10:29fine. So, I'll even show you here, I'll
  350. 10:31do Ollama launch Claude. I will choose
  351. 10:33the model that I want to use. So, it's
  352. 10:34saying that there's a conflict because
  353. 10:36of the off. So, I might have to add the
  354. 10:38actual API key directly, but you can see
  355. 10:40that this is hitting the Min Max. So,
  356. 10:42yes, I'm sorry, you do have to pay $5,
  357. 10:44but your usage ongoing is going to be
  358. 10:46unlimited with these open source models,
  359. 10:47but I just I had to address that.
  360. 10:49Otherwise, I had a feeling that I was
  361. 10:51going to get in trouble. So, if I say,
  362. 10:53"Hey Claude Code, what do you know about
  363. 10:54me and my project?" I can shoot that
  364. 10:56off, and this is now using Claude Code,
  365. 10:59using this harness, you looking at my
  366. 11:00project and my skills and my agents, but
  367. 11:02it is running through a model on my
  368. 11:04desktop. But, remember, this is going to
  369. 11:06feel significantly slower than if you
  370. 11:08were sending off this request to
  371. 11:10Anthropic's server. But, at this point,
  372. 11:12because everything's completely local,
  373. 11:13it's completely private as well. And the
  374. 11:16other thing you might notice is you
  375. 11:17don't see in your actual chat the way
  376. 11:20that it's calling tools the same way
  377. 11:21that you do when you have a Claude model
  378. 11:23in there. But, yeah, this just took 4
  379. 11:25minutes to basically look at my project
  380. 11:27and answer this question. Now, keep in
  381. 11:30mind, there's a ton of context in here,
  382. 11:31a bunch of folders and files that it
  383. 11:33basically just looked through. But, the
  384. 11:34other thing is, we don't have visibility
  385. 11:36into exactly what it actually did
  386. 11:39because of the fact that it basically
  387. 11:40just spins forever until it responds,
  388. 11:43rather than we getting to see like each
  389. 11:44step that it's doing. So, that was a
  390. 11:46read tool. Let's just make sure that it
  391. 11:47can actually do things. I'm telling it
  392. 11:49to create a file at the root level of
  393. 11:51this project and just call it Qwen 3.5.
  394. 11:54It misspelled Qwen.
  395. 11:55Um and then I said, "Inside that file,
  396. 11:57write a joke." So, we'll see how long
  397. 11:58this takes, and we'll see if it can
  398. 11:59actually do, sort of like, you know,
  399. 12:01write and edit and create actions. So,
  400. 12:04there's a few things going on here. The
  401. 12:05first thing is that we didn't get full
  402. 12:06visibility when it was creating the file
  403. 12:08because it's a local model, and it
  404. 12:09wasn't showing us those tool calls. The
  405. 12:11second thing is that it didn't seem to
  406. 12:12have state of what was going on here. I
  407. 12:15think that was because it exceeded the
  408. 12:16context window limit because in Ollama,
  409. 12:19sometimes they by default, even though
  410. 12:21down here it might say 200k, not
  411. 12:23actually 200k.
  412. 12:24So, back in the terminal, I'm going to
  413. 12:26run this. Which is going to basically
  414. 12:27let us change how much context is given
  415. 12:30to this model. And we can see we have
  416. 12:32now created this. So, if I try to do
  417. 12:36Ollama launch
  418. 12:38Claude.
  419. 12:39We should now see that we have this
  420. 12:40different model which we made. So, Qwen
  421. 12:423.5 9B 64,000 context. So, if I open
  422. 12:46this one up, we now should be able to
  423. 12:47launch Claude, and this is a custom
  424. 12:49model that we sort of configured off of
  425. 12:51the back end of Qwen 3.5. And by the
  426. 12:53way, the way you get that command is
  427. 12:55just go to Claude and say, "Hey, I'm
  428. 12:56pulling in a model from Ollama, and I
  429. 12:58want to increase the context window. Can
  430. 13:00you help me run these commands?" And
  431. 13:02it's like literally one command. The
  432. 13:03only context you might need to give it
  433. 13:05is if you're using like PowerShell or
  434. 13:06command prompt or what your operating
  435. 13:08system is. Okay, this is awesome. This
  436. 13:10time we were able to see the actual tool
  437. 13:12call. So, it right here called write. It
  438. 13:15wrote seven lines, and it put it right
  439. 13:16there. So, if I scroll down over here,
  440. 13:18we can see right here. Nice, we have
  441. 13:20that text file, and it has a joke. Why
  442. 13:22did the AI go to therapy? Because it had
  443. 13:24too many tokens to process and needed to
  444. 13:25talk it through. Bonus, I asked the
  445. 13:27model if it's okay, and it said, "I
  446. 13:29don't have feelings, but my loss
  447. 13:31function is definitely in the red." So,
  448. 13:33you can see how us giving Qwen a little
  449. 13:35bit more context was helpful. Now, what
  450. 13:37if we wanted to do a
  451. 13:39model from Ollama that was in the cloud?
  452. 13:42So, for example, let's say we wanted to
  453. 13:43use Min Max M 2.7. What we would do is
  454. 13:47we would just launch this command. So,
  455. 13:49we don't have to run this or download
  456. 13:50it. We would just copy this command, and
  457. 13:52go back into my VS Code, and I would
  458. 13:54paste this in at the bottom down here.
  459. 13:56And then when I run this, it's prompting
  460. 13:58me to sign in at ollama.com. And now
  461. 14:00it's saying that we need to connect our
  462. 14:01device here. And now that that has been
  463. 14:03connected, I can close this window. So,
  464. 14:05if I open back up over here, we can now
  465. 14:07see that we have our Claude Code
  466. 14:09running, and we have Min Max M 2.7 on
  467. 14:12the cloud. But, because of the fact that
  468. 14:13you are not running this locally
  469. 14:15anymore, you do have a pricing barrier
  470. 14:17at some point. So, when you're just
  471. 14:19getting started, you can definitely play
  472. 14:20around with those for a while, and you
  473. 14:21will be able to use them, but you do
  474. 14:23need to upgrade at some point if you
  475. 14:25need to run more than one model at a
  476. 14:27time, and if you need more usage, and
  477. 14:29then same thing with the Max. So, that's
  478. 14:31the thing that you have to realize is
  479. 14:32that there's like there's really no such
  480. 14:34thing as free because if you want to run
  481. 14:35a really good model locally, then you
  482. 14:37need the hardware to support it. If you
  483. 14:39want to run it on the cloud, you either
  484. 14:41have to pay for a subscription like
  485. 14:43Ollama, or you could pay for a VPS
  486. 14:46that's big enough in order to host that
  487. 14:48model there. So, that's the trade-off.
  488. 14:49Um there's really no such thing as free.
  489. 14:51You got to find the balance between
  490. 14:52quality and price, and also thinking
  491. 14:54about like where are you making your
  492. 14:56investment, and are you going to max out
  493. 14:58your usage, or are you going to pay per
  494. 14:59token, all these kind of things. How can
  495. 15:01you optimize your processes and also
  496. 15:04your price at the same time. And we can
  497. 15:06now use this model, which is probably
  498. 15:07going to be much faster and better than
  499. 15:10this previous one up here with this
  500. 15:12local version of Qwen 3.5, which was
  501. 15:14only 9 billion parameters. So, I'm
  502. 15:16asking it that same question where I'm
  503. 15:17testing to see how quick and how good it
  504. 15:19is at reading through everything in
  505. 15:21here. And you can see that came back
  506. 15:22way, way quicker. So, now I'm going to
  507. 15:24ask it to do something else. I'm going
  508. 15:25to see if it can run my morning coffee
  509. 15:27skill. And look at this, we can actually
  510. 15:29see that it is indeed running the skill,
  511. 15:31and we get a little bit more visibility
  512. 15:33because I'm assuming that this Min Max
  513. 15:34model is much, much bigger than 9
  514. 15:37billion parameters. So, the point I'm
  515. 15:38trying to make here is if you don't have
  516. 15:40hardware that can support larger models,
  517. 15:42then you probably want to find a way to
  518. 15:44run larger models somewhere else because
  519. 15:46this is just such a better experience.
  520. 15:48This almost feels like I'm actually
  521. 15:49using Sonnet in Claude Code right here.
  522. 15:52I mean, this literally spun up four
  523. 15:53different local agents, and it's
  524. 15:55actually running pretty fast. The only
  525. 15:57thing to think about here is this is no
  526. 15:58longer 100% local because it's now going
  527. 16:01to Ollama's cloud, which is why you can
  528. 16:03understand now why it's such a big deal
  529. 16:05that Google Gemma 4 is so much smaller
  530. 16:08than all of the models that are its same
  531. 16:10quality. All right, so before we get on
  532. 16:12to the method number two, which is using
  533. 16:14Open Router, when would you actually
  534. 16:16want to use these open source models?
  535. 16:18There's a few different scenarios that I
  536. 16:19want to talk through. The first thing is
  537. 16:21if you have low stakes or high volume
  538. 16:23tasks. So, things like reading and
  539. 16:24summarizing files before passing to a
  540. 16:26smarter model, or grepping through a
  541. 16:28code base to find relevant files and
  542. 16:29functions, so doing some searches, or
  543. 16:31generating repetitive code scaffolding.
  544. 16:34The things where you really don't need
  545. 16:35the most top-tier model to be doing. We
  546. 16:37also have things like research and
  547. 16:39information gathering. So, web searches,
  548. 16:41summarizing emails, pulling docs, things
  549. 16:43like that. Organizing and classifying
  550. 16:45things, categorizing tasks, triaging
  551. 16:47emails, organizing local files, or even
  552. 16:49your CRMs. And then if you want to go on
  553. 16:51like the actual coding side, it can
  554. 16:53definitely be some simple tests, or
  555. 16:55maybe improving some bugs or doing some
  556. 16:56reviews. But, I personally would just be
  557. 16:58careful, and then always make sure that
  558. 16:59if you don't really know what's going
  559. 17:01on, you would be checking later with a
  560. 17:03more powerful model. Now, of course, the
  561. 17:04other times when you might want to do
  562. 17:05this is if Claude is down. So, if you go
  563. 17:08to status.claude.com, and you're having
  564. 17:10issues with Opus or Sonnet, then maybe
  565. 17:12you switch to a local model for a little
  566. 17:13bit just so you don't feel unproductive
  567. 17:15for the next 2 hours. Or, of course, if
  568. 17:17you just hit your session limit, and you
  569. 17:18have to wait another few hours, then
  570. 17:20maybe you open up your local version and
  571. 17:21just play around with that. Okay, so
  572. 17:23that was the local method. Now, let's
  573. 17:25talk about running it through Open
  574. 17:27Router. So, on Open Router's docs, they
  575. 17:29even have an integration guide for
  576. 17:30basically how this works. And all we
  577. 17:32have to do is we just have to change a
  578. 17:33few things in our environment variables,
  579. 17:35which override the default permissions.
  580. 17:37So, we're basically just telling Claude
  581. 17:38Code, "Hey, instead of pointing to
  582. 17:39Anthropic's API, you're going to point
  583. 17:41to Open Router, and I'm going to give
  584. 17:43you this key, and then you're just going
  585. 17:44to basically talk to these models
  586. 17:46instead of Anthropic's models." So, if
  587. 17:47you don't have an Open Router account,
  588. 17:49go ahead and go to openrouter.ai and
  589. 17:50sign up. And what you're going to have
  590. 17:52to do is, in order to use these models,
  591. 17:55you're going to have to load your
  592. 17:55account with like $5 or $10. And then
  593. 17:58after you do that, you're free to use
  594. 18:00all of these free models, and it won't
  595. 18:01actually eat into that 10 bucks that you
  596. 18:03put into Open Router. And basically, the
  597. 18:05reason why is because of rate limits.
  598. 18:07So, if you don't have anything on your
  599. 18:09account, then you will get limited to 50
  600. 18:12requests per day with your free models.
  601. 18:14But, if you put $10 in, then you will
  602. 18:15get 1,000 requests per day. And by the
  603. 18:17way, this whole doc with all the setup
  604. 18:18information will be attached in my free
  605. 18:21school community. It's completely free.
  606. 18:22The link for that is down in the
  607. 18:23description. Claude Code, and we
  608. 18:24basically launch up Claude. We will see
  609. 18:26that it says, "Okay, cool, we're using
  610. 18:28Opus, and we're using my Claude Max
  611. 18:30plan." So, we need to change this. And
  612. 18:32the way we do that is in our project
  613. 18:34settings. So, in the dot Claude, in the
  614. 18:36settings.local.json,
  615. 18:37you can see that this is what I have.
  616. 18:39I've got some permissions and stuff like
  617. 18:40that. I've got an environment variable,
  618. 18:42but we need to add a few more
  619. 18:43environment variables. All right, so I
  620. 18:45switched into a different project just
  621. 18:46so I could keep this clean, but I have
  622. 18:48this other JSON file right here called
  623. 18:49free router. So, what I'm going to do is
  624. 18:51I'm just going to take all of this and
  625. 18:52copy it,
  626. 18:54and then I'm going to open up Claude,
  627. 18:55and I'm just going to say,
  628. 18:58"Hey, can you please add this
  629. 18:59configuration to our
  630. 19:00settings.local.json?"
  631. 19:02And I would go ahead and paste that
  632. 19:04right in here. Now, in my case, there my
  633. 19:06API key is in there, so
  634. 19:08just be aware that if you do this,
  635. 19:10Anthropic is reading that. But, the
  636. 19:12reason I'm doing this is because right
  637. 19:13after this video, I'm just going to
  638. 19:14delete this anyways. So, now let's talk
  639. 19:16about what this is doing. So, the first
  640. 19:18thing is it's pointing to Open Router
  641. 19:19instead of Anthropic. And then in the
  642. 19:21Anthropic off token, this is where you
  643. 19:23put your Open Router API key. So, I know
  644. 19:25it says Anthropic, but this is where you
  645. 19:26put your Open Router one. So, you'd go
  646. 19:28into your Open Router account, you would
  647. 19:29go over here to credits,
  648. 19:32and then you're going to go into your
  649. 19:33API key section, and you're going to
  650. 19:34create a new key. So, give it a name,
  651. 19:37hit create, and then when it gives you
  652. 19:38that value, you're going to copy that,
  653. 19:40and then you're going to put that into
  654. 19:41the configuration that I just showed you
  655. 19:43right here, Anthropic off token. And
  656. 19:45then the Anthropic API key, you're going
  657. 19:47to leave completely blank. The Anthropic
  658. 19:49model and all of these other models,
  659. 19:50you're going to put in a free model from
  660. 19:52OpenRouter.
  661. 19:54Now, the reason you have to do all of
  662. 19:55them and not just this one, like the
  663. 19:57OpenRouter documentation suggests, is
  664. 19:59because if you do it like that, it's
  665. 20:00actually going to, by default, call on
  666. 20:04Anthropic paid models for Haiku and
  667. 20:06Sonnet. So, here in my logs, you can see
  668. 20:08I was testing this out and I did the
  669. 20:10free configuration, but then I noticed
  670. 20:12in my logs I was kept getting charged by
  671. 20:14by Haiku, and I was very confused.
  672. 20:16Because whenever it was doing little
  673. 20:17tool calls or doing little file
  674. 20:19searches, it was using Haiku instead of
  675. 20:21using the free models that I had
  676. 20:22installed it to use. So, if you were to
  677. 20:24have just put in this configuration, and
  678. 20:27you would have left out these variables
  679. 20:28down here, then it would have, by
  680. 20:30default, used Sonnet or Haiku for
  681. 20:33basically all of these things, and it
  682. 20:34would have charged you without you even
  683. 20:35knowing. So, make sure you put these
  684. 20:37ones in there as well. And then you can
  685. 20:38see, after I figured that out, it was
  686. 20:40able to use all of these for tool calls
  687. 20:42and everything instead of using the paid
  688. 20:44models. The only reason these were paid
  689. 20:46is because I did that intentionally. But
  690. 20:47yeah, this is what OpenRouter says, so
  691. 20:49just do it the way that I did it. So,
  692. 20:50now that we have this set up, if I
  693. 20:52actually open up a terminal, and I type
  694. 20:54in Claude, what do we see here? We see
  695. 20:57that we are using OpenRouter free
  696. 21:00API billing usage rather than using our
  697. 21:03Max plan or our Team plan. So, what is
  698. 21:05OpenRouter free? Okay, well, this is
  699. 21:07very similar to Ollama. You've got all
  700. 21:08these different models you can look at.
  701. 21:10When you click on a model, it'll show
  702. 21:11you the price per input tokens and the
  703. 21:13price per output tokens. But what you
  704. 21:15can do is if you come up here and you
  705. 21:16just search free, you can see that
  706. 21:18there's a ton of models right here
  707. 21:19available that are free. And if you go
  708. 21:21down to the actual OpenRouter one, which
  709. 21:24is right here, the free models router,
  710. 21:26this one is completely free, but it
  711. 21:28basically routes you to whichever model
  712. 21:29is like the most available at that
  713. 21:31moment. So, it kind of helps you from
  714. 21:32getting rate limited and things like
  715. 21:34that. Now, I'll show you how this works,
  716. 21:35but I'm also going to show you why I
  717. 21:37wouldn't recommend using this because
  718. 21:38you just don't have any control over
  719. 21:40what model is getting called. So, here
  720. 21:42I'm asking it to make a file in this
  721. 21:43folder called OpenRouter test. You can
  722. 21:45see that we are able to get the
  723. 21:46visibility as far as what tools it's
  724. 21:48using, which is great. We have over here
  725. 21:50the OpenRouter test file, and there's
  726. 21:51nothing in it, but it did create that,
  727. 21:53and it didn't take that long, either.
  728. 21:55And now in my logs, you can see this is
  729. 21:56what we just did, April 3rd, 5:15. It
  730. 21:58called the OpenRouter free model three
  731. 22:01different times, and all of these times
  732. 22:03costed us zero dollars, as you can see.
  733. 22:05Now, let's say instead I want to use
  734. 22:06this awesome Qwen 3.6 model with a
  735. 22:08million context and completely free,
  736. 22:11once again. Well, what I would do is I
  737. 22:13would copy this model name, so Qwen Qwen
  738. 22:153.6 plus colon free. Back in Claude
  739. 22:18Code, I would just close out of this
  740. 22:19terminal real quick. I'm going to open
  741. 22:20up the settings.local.json,
  742. 22:22and I will basically just come in here,
  743. 22:24and I will paste that new model name
  744. 22:26into all of the instances where we said
  745. 22:28OpenRouter free. And now, after I save
  746. 22:31this and I go ahead and open up a new
  747. 22:33terminal
  748. 22:34and launch Claude, we should see that we
  749. 22:36have in here this is running now with
  750. 22:37that new Qwen model, and I can say
  751. 22:40hello, and it should be able to respond
  752. 22:42to us fairly quickly as Qwen.
  753. 22:44There we go. Hey, what can I help you
  754. 22:46with today? If I go back into my logs
  755. 22:47and I refresh, we should see that this
  756. 22:49most recent call will be with Qwen.
  757. 22:53There we go. Those two came through as
  758. 22:54Qwen, and you can see that once again,
  759. 22:56it costed us nothing. So, let's see what
  760. 22:59else we can do with our open source
  761. 23:00models. Because if you remember, I
  762. 23:02talked about some things where
  763. 23:04Anthropic has trained its Claude models
  764. 23:06to act a certain way with Claude Code,
  765. 23:08but not the all the other models have
  766. 23:09done that. So,
  767. 23:11can you go ahead and search the web and
  768. 23:12do some research on Google's new model
  769. 23:14Gemma 4. So, what you'll notice when I
  770. 23:16asked it to search the web, it's doing
  771. 23:18web searches, and it's knowing how to
  772. 23:20use those tools, but it's running into
  773. 23:21an issue where the current model seems
  774. 23:23to not have web search access. So, it's
  775. 23:25trying a different approach where it's
  776. 23:27basically just doing a fetch directly.
  777. 23:29So, what happened here is it tried to do
  778. 23:30fetches, and it looks like that worked,
  779. 23:32but I guess maybe it didn't because it
  780. 23:34said that it couldn't do any web
  781. 23:35lookups, but it does have some
  782. 23:37information about the um actual model
  783. 23:40based on
  784. 23:41training data. So, what that means is
  785. 23:43you can still do web search with tools
  786. 23:45like, you know, Brave Search or
  787. 23:46Perplexity or Tavily, but it's a little
  788. 23:48bit harder because it now might not
  789. 23:50natively have the web search ability. If
  790. 23:52you need to like look up API
  791. 23:53documentation or things like that, you
  792. 23:55just might have to give it more
  793. 23:56information manually or let it use more
  794. 23:59Perplexity credits or something like
  795. 24:00that. So, just keep in mind, you want to
  796. 24:02be monitoring this kind of stuff because
  797. 24:04you do have certain limits on your
  798. 24:06account as far as how many calls you can
  799. 24:08make to free models per day, as well as
  800. 24:10how many calls you can make to free
  801. 24:12models every single minute. So,
  802. 24:14something else you can keep in mind,
  803. 24:15though, is the ones where you won't be
  804. 24:17limited as much are models that are
  805. 24:19really, really cheap.
  806. 24:21It's not going to work right now in
  807. 24:23Claude Code, but if you were using Gemma
  808. 24:244, 31 billion parameters, this is 14
  809. 24:28cents per million input tokens and 40
  810. 24:30cents on the output. And if I go ahead
  811. 24:32and compare this with Opus 4.6, you can
  812. 24:36see that the cost is just a fraction.
  813. 24:38Down here, we have five bucks and 25,
  814. 24:40and we have 14 cents and 40 cents. So,
  815. 24:43you could very well just use this
  816. 24:44OpenRouter combination with Claude Code
  817. 24:46to basically get yourself Claude Code
  818. 24:48for like 50 to 100x cheaper rather than
  819. 24:51just being completely free, and I think
  820. 24:53that's still a huge win. So, you
  821. 24:54understand these two different methods
  822. 24:55of using Claude Code for free, whether
  823. 24:57that is with a local model or whether
  824. 24:58that's through Ollama Cloud or
  825. 25:00OpenRouter, but what's still really
  826. 25:02important is managing context and tokens
  827. 25:05because these open source models don't
  828. 25:07have as big of a context window as
  829. 25:09something like Opus does with a million,
  830. 25:11for the most part. So, what you need to
  831. 25:12watch next is my 18 token management
  832. 25:15hacks, which I will tag right up here,
  833. 25:17and hopefully I see you guys over there.
  834. 25:18Appreciate making it to the end of the
  835. 25:19video.
  836. 25:21See you in the next one.

About this transcript

This page contains the full transcript of Ollama + Claude Code = 99% CHEAPER by Nate Herk | AI Automation, generated from the public captions YouTube serves with the video. The transcript has 6,096 words across 836 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.