YouTube2Text

Which is The Best Qwen3.8-27B? — Transcript

by Sam Witteveen · 4,030 words · 568 segments · language en · Watch on YouTube

Full transcript

  1. 0:00Okay, so this is the same question going
  2. 0:03into two versions of the same model.
  3. 0:06Same weights underneath, similar answer
  4. 0:08at the end, but one of them has used a
  5. 0:10lot more thinking tokens to get there,
  6. 0:12and the other has used hardly any in
  7. 0:14comparison to get there. So on my
  8. 0:16machine, that's a difference between 18
  9. 0:18seconds and 90 seconds. And if you think
  10. 0:20about it, that's sort of the problem
  11. 0:22with reasoning models now, right? They
  12. 0:24get the answer, but they take a long
  13. 0:26time to get there. It's almost like one
  14. 0:27model flies there to get the answer as
  15. 0:30fast as possible and the other one
  16. 0:32drives around the coast to take a scenic
  17. 0:33route. And you'll see this in ways that
  18. 0:35are often not totally bad. They'll do
  19. 0:38things like they restate the question,
  20. 0:40they check. They check again. They say
  21. 0:42to themselves, "Wait, let's reconsider."
  22. 0:45And every one of those tokens is time
  23. 0:47that you're spent waiting for this
  24. 0:49answer. And of course, money that you're
  25. 0:52paying for the inference for this. Now,
  26. 0:54in the past, I've talked about Open AI
  27. 0:56focusing on reducing the number of
  28. 0:59tokens between the 5.0 model and the 5.5
  29. 1:03model. Well, they're not the only ones
  30. 1:05that have been focused on this, right?
  31. 1:07Most of the major labs have been looking
  32. 1:09at how do they take these long chains of
  33. 1:11thought and reduce them but still keep
  34. 1:13the accuracy. And the cool thing is that
  35. 1:15there are also teams trying to do this
  36. 1:17with open-source models. So, in the
  37. 1:19past, I covered the thinking cap model
  38. 1:20for the Quen 3.627B 627B. And no
  39. 1:24surprise, I'm going to cover their new
  40. 1:25model in this video, but also I want to
  41. 1:27look at some of the other teams that
  42. 1:29have taken on the Quen 3.827B
  43. 1:33and fine-tuned it to think less. So on
  44. 1:36top of thinking cap, we've also got
  45. 1:38Quenpi, which is sort of like a
  46. 1:40specialized version with less thinking
  47. 1:42for coding agents. Got Swift 1.5. They
  48. 1:46all claim fewer tokens for about the
  49. 1:48same accuracy, but it seems they all go
  50. 1:51about it in really different ways. And
  51. 1:53it also seems that they're probably
  52. 1:55really good at different things. So, in
  53. 1:56this video, I want to look at how each
  54. 1:58one was made, then I'll run all of them
  55. 2:00against the base model and see if the
  56. 2:02claims actually hold up. All right, so
  57. 2:04before we do the fine tunes and look at
  58. 2:06those, you need to know a bit about the
  59. 2:09sort of thinking and reasoning on Quen
  60. 2:113.8 that it already gives you out of the
  61. 2:14box. and they call it reasoning effort.
  62. 2:16It has three settings, low, medium, and
  63. 2:19X high. And amazingly, out of the box, X
  64. 2:21high is actually the default. So, if
  65. 2:23you're serving with VLM, you actually
  66. 2:25set this in the chat template. And you
  67. 2:28can see that when we set that reasoning
  68. 2:30effort, that's going to affect how long
  69. 2:32the chains of thought are that's coming
  70. 2:33back. Now, I talked about this when I
  71. 2:35made the video about serving Quen
  72. 2:373.827B,
  73. 2:39and I got to say that model is still one
  74. 2:41of the best open models that you can
  75. 2:43get. and serve on reasonably small
  76. 2:45hardware. For some uses, the 3.8 next is
  77. 2:48probably better, but that's going to
  78. 2:50move you into the realm of needing a
  79. 2:52bigger GPU or needing multiple units in
  80. 2:54there. Now, back to the 3.827B.
  81. 2:57The whole reasoning effort there is you
  82. 3:00are basically telling the model to try
  83. 3:02less hard. So, yes, you can get less
  84. 3:05tokens out of it, but when you actually
  85. 3:07turn that reasoning down, it's often
  86. 3:09going to cost you inaccuracy. And if
  87. 3:11you're running this locally on 1GP,
  88. 3:13you're really going to be decode bound.
  89. 3:15So the thinking or reasoning tokens
  90. 3:17there match pretty much directly to
  91. 3:20seconds for you getting your final
  92. 3:21result out. If you can have those
  93. 3:23tokens, you're roughly having the weight
  94. 3:26that you're going to have to wait for
  95. 3:27your final answer out. Secondly, these
  96. 3:29models have multi-token prediction heads
  97. 3:31for speculative decoding. And that
  98. 3:34actually works better on shorter traces
  99. 3:36than longer traces. And we actually see
  100. 3:39this in some of the reported results.
  101. 3:40So, thinking cap's model card actually
  102. 3:43reports that speculative decoding goes
  103. 3:45from about 2.6 tokens per step at X high
  104. 3:49to over three tokens per step at medium
  105. 3:52and low. So, you can actually think
  106. 3:54about this as thinking less doesn't just
  107. 3:56mean fewer tokens. Each token comes out
  108. 3:59faster as well. All right. So, the big
  109. 4:00question here is what are these fine
  110. 4:02tunes actually trying to do? And are
  111. 4:04they going to get you the savings
  112. 4:06without any sort of accuracy hit here?
  113. 4:08All right. So here's the family tree.
  114. 4:10Everything starts from Quen 3.827B.
  115. 4:14On one branch of the tree, you've got
  116. 4:16Thinking Cap from Bottlecap AI. Their
  117. 4:19first thinking cap model was the one
  118. 4:20that I covered a couple of months back
  119. 4:21and that was built on the Quen 3.6 model
  120. 4:24then. And you could say related to this
  121. 4:26was the UKIS AI swift model. So when
  122. 4:30they built their first version of Swift,
  123. 4:32part of that recipe was a transfer
  124. 4:34component from the thinking cap model.
  125. 4:36So you can sort of think of Swift of
  126. 4:38having some of the thinking cap DNA in
  127. 4:40it and being related. Now on a
  128. 4:43completely separate branch here, you've
  129. 4:44got Quen Pi which is playing a different
  130. 4:47game totally. I'll get to that as we go
  131. 4:49through each of the models, but let's
  132. 4:50start with Thinking Cap. So Bodcap AI is
  133. 4:53a lab in Prague and one of the
  134. 4:55co-founders there is Thomas Michelov. So
  135. 4:57at the time that I recorded the original
  136. 4:59video about the first Thinking Cap, I
  137. 5:01didn't actually know that until one of
  138. 5:02you pointed it out in the comments. And
  139. 5:05so for those of you who don't know who
  140. 5:06Thomas Mikolof is, he was actually the
  141. 5:09first author on a very famous paper word
  142. 5:11tovec. So this is one of the key papers
  143. 5:13that really changed natural language
  144. 5:16processing. The whole idea of having
  145. 5:18vectors that could represent words and
  146. 5:21then could be used for things like
  147. 5:22classification like a whole bunch of
  148. 5:24these things. At the time Michelof was
  149. 5:26working at Google from memory he was
  150. 5:28also working on a lot of the ideas for
  151. 5:30seek to seek which all of these were
  152. 5:32very early precursors to the transformer
  153. 5:34idea of sequence in sequence out. Now
  154. 5:37after one of you had pointed that out. I
  155. 5:39actually reached out to him to send the
  156. 5:40video and message him where they're
  157. 5:42going to do one for 3.8. And I found out
  158. 5:45they're working on an interesting
  159. 5:46project that hopefully I can cover in
  160. 5:47the near future. But clearly I wasn't
  161. 5:49the only one giving feedback that the
  162. 5:51community wanted a 3.8 eight version of
  163. 5:54the thinking cap models. So I think they
  164. 5:56reallocated some of the compute that
  165. 5:58they were using for their other project
  166. 6:00to this. And just a very quick side note
  167. 6:02here, I would say that we really should
  168. 6:04be grateful for startups like this. His
  169. 6:06startup is not some billion-dollar
  170. 6:07startup in San Francisco that's training
  171. 6:09their own frontier models to compete
  172. 6:11with OpenAI or Anthropic, etc. Bottlecap
  173. 6:15is a startup that's raised a tiny
  174. 6:16fraction of that money compared to some
  175. 6:18of these bigger ones and most likely has
  176. 6:21a fraction of the compute that those
  177. 6:22more famous startups actually have. So
  178. 6:25the fact that they're allocating compute
  179. 6:26to do something like this for the
  180. 6:28community, I think is very cool. All
  181. 6:30right, so how was this actually made?
  182. 6:32Well, Bodcap didn't publish the actual
  183. 6:34training recipe, but they're clear about
  184. 6:36the objective here. The target was one
  185. 6:38thing, how many thinking tokens it takes
  186. 6:40to reach the same answer. And they
  187. 6:43mentioned that the aim was deliberately
  188. 6:44conservative here. They didn't try to
  189. 6:46make the model smarter, to teach it
  190. 6:48anything new, to change how it talks.
  191. 6:51They haven't tried to add any knowledge
  192. 6:52in here or even improve the instruction
  193. 6:55following or safety or any of those
  194. 6:57things. Those are meant to come through
  195. 6:59totally untouched. So stream to X high
  196. 7:01and this time they put more focus on
  197. 7:03much harder sort of benchmarks and
  198. 7:05tasks. So things like math, reasoning,
  199. 7:07long context, agentic tasks, etc. You
  200. 7:10can kind of think of this is teaching
  201. 7:12the model when it's done rather than
  202. 7:14teaching it to be brief. So the headline
  203. 7:17here is 37% fewer tokens on average
  204. 7:20across 12 benchmarks with the average
  205. 7:23accuracy going from 86.6 to sort of
  206. 7:2685.8. So it's just a tiny percentage
  207. 7:29less accuracy there. So, for example, on
  208. 7:32the GPQA diamond, this goes from an
  209. 7:34average of 12,800 thinking tokens on the
  210. 7:37original version down to 7,300 on the
  211. 7:40thinking cap one. And interesting on
  212. 7:42some of them, like the long context
  213. 7:44result is a really interesting one. On
  214. 7:46their long context retrieval benchmark,
  215. 7:48it uses 39% less thinking, but the
  216. 7:51accuracy actually goes up a couple of
  217. 7:53points there. The chart I think that is
  218. 7:55most useful in their blog post about
  219. 7:58this is this one because it ties
  220. 8:00straight back to where we set the
  221. 8:01reasoning effort. If you take the base
  222. 8:04model and just drop it to medium, you
  223. 8:06remove about half the thinking, but you
  224. 8:08give up about nine points of accuracy.
  225. 8:11Thinking cap at X high removes 37% for
  226. 8:15under a point of accuracy. So suddenly
  227. 8:18that X high level gets really useful
  228. 8:19there. All right. So what's this model
  229. 8:22actually good for? It's basically just a
  230. 8:24drop-in replacement for the Quen version
  231. 8:27at X High. So anything that needs
  232. 8:29general reasoning, knowledge questions,
  233. 8:31long documents, etc. All of those
  234. 8:33things, you can basically just drop this
  235. 8:35model in. Perhaps the only downside I
  236. 8:37would say here would be the agentic
  237. 8:39traces. And not because they're not
  238. 8:41shorter. They're shorter, but it seems
  239. 8:43that the reasoning only drops by about
  240. 8:4511% there. Now, interestingly, the other
  241. 8:47thing here is that this version is made
  242. 8:49for X high. If you want a version that's
  243. 8:52made for the medium or lowinking and
  244. 8:55optimized for that, that's something
  245. 8:56that they're actually selling to
  246. 8:58enterprises. And along with that, the
  247. 9:00other change here is the actual license.
  248. 9:03So, it's gone from being Apache 2 for
  249. 9:05the 3.6 version to this one being a
  250. 9:08small business plus personal use kind of
  251. 9:10license, which totally makes sense if
  252. 9:12they're trying to make some money to
  253. 9:13actually cover their compute costs, etc.
  254. 9:16All right. So, next up is Swift 1.5 from
  255. 9:20Yucus AI. And this one's quite a lot
  256. 9:23more aggressive. So, this one is going
  257. 9:25for up to 58.5%
  258. 9:28fewer thinking tokens on the Quen
  259. 9:313.827B.
  260. 9:33They built it in two steps. So, for
  261. 9:35Swift 1.0, they were really looking for
  262. 9:37specific tokens in Quen's reasoning that
  263. 9:40seem to trigger overthinking. And you
  264. 9:42could think of some examples of those
  265. 9:44are where things like it says wait let
  266. 9:46me ask what the user wants hm and then
  267. 9:49you know asks itself something let me
  268. 9:51reconsider and then they fine-tuned the
  269. 9:53model to penalize those tokens rather
  270. 9:56than just penalizing the length and then
  271. 9:58they added that to the original version
  272. 10:00of the thinking cap model for 1.5 though
  273. 10:02they took the model and scaled up the
  274. 10:04reinforcement learning and on policy
  275. 10:06distillation with their main focus being
  276. 10:09on long horizon agentic and coding ing
  277. 10:12tasks. Now, the cool thing here is that
  278. 10:14not only have they published the models,
  279. 10:16the training data is also on
  280. 10:17HuggingFace, which is really good to see
  281. 10:20here. You can see it in the coding
  282. 10:21numbers here on their model card. Live
  283. 10:23Codebench goes from just under 77% for
  284. 10:26the base model with around 11,200 tokens
  285. 10:30out to actually an increase of accuracy
  286. 10:32of 81.7%
  287. 10:34with only 8,400 tokens out. So, the
  288. 10:37other interesting thing here is that
  289. 10:39they also claim that the savings that
  290. 10:40they're getting hold at every effort
  291. 10:43level. At low, it actually scores
  292. 10:45slightly above the base model with about
  293. 10:4729% fewer thinking tokens. Practically
  294. 10:50too, this one has probably the most ways
  295. 10:52for you to run it. There are standard
  296. 10:54Lama CPP GGUFS for this. There are small
  297. 10:57ones to fit on small GPUs. There's MLX.
  298. 11:00There's an NVF4 version and even an AMD
  299. 11:03build here. And they even have like a
  300. 11:05free research API for this with no API
  301. 11:08key if you want to just try this out.
  302. 11:10Now, that headline of the 58% fewer
  303. 11:12thinking tokens probably turns out to be
  304. 11:15a bit less when you start looking at the
  305. 11:17mean thinking reduction across this, but
  306. 11:19it's still very impressive here. Another
  307. 11:22thing that they've focused on is that
  308. 11:23they claim that the base model loops and
  309. 11:25fails early, which they're claiming that
  310. 11:27their version doesn't do. Now, the
  311. 11:29license for this one is their own custom
  312. 11:31license, which is free up to a million
  313. 11:33dollars in revenue. After that, you need
  314. 11:36to basically contact them. All right, so
  315. 11:38that brings me to the third one, which
  316. 11:39is kind of the odd one out here. And
  317. 11:41incidentally, this is the only one of
  318. 11:43the three that actually has an Apache 2
  319. 11:45license. So, this is Quenpi. To
  320. 11:48understand it, you first need to know
  321. 11:49about what PI is. Pime is an open-
  322. 11:51source coding agent that's very
  323. 11:53deliberately minimal. It has a very
  324. 11:56limited set of tools for doing read,
  325. 11:58write, edit, and bash. And last time I
  326. 12:01checked, it even has a system prompt
  327. 12:02that's under a thousand tokens. So Quen
  328. 12:05Pi is Quen 3.827B
  329. 12:08fine-tuned specifically to work inside
  330. 12:10of the Pi harness. And I want to cover
  331. 12:12this one because I think this is
  332. 12:13something we're going to see a lot more
  333. 12:15of where people fine-tune a particular
  334. 12:17model for a very specific harness. And
  335. 12:21it totally makes sense. If you can get
  336. 12:23the model and the harness to work
  337. 12:25together, and the model can be primed
  338. 12:27for what kind of inputs it's going to
  339. 12:28see, what kind of inputs it should
  340. 12:30generate out, you're probably going to
  341. 12:32end up with a model that's a lot more
  342. 12:33suited to a very particular vertical
  343. 12:35harness there. And this idea is
  344. 12:37something that I've been running tests
  345. 12:38on recently with the number of vertical
  346. 12:40harnesses in the co-scientist area. So,
  347. 12:42I feel like this model shows a little
  348. 12:44bit of where that direction is actually
  349. 12:46going. So, how was this made? First,
  350. 12:49it's basically supervised fine-tuning on
  351. 12:51real PI sessions, but only the
  352. 12:53successful ones. So, the model learns
  353. 12:55the whole workflows, how to look at
  354. 12:57code, how to make a change, how to run
  355. 13:00tests, how to fix what's broken, etc.
  356. 13:02Secondly, they make sure in those tasks
  357. 13:04that they actually check for working
  358. 13:06code versus broken code separately
  359. 13:08themselves. Thirdly, they're doing
  360. 13:10reinforcement learning with GPO and a
  361. 13:13custom reward for reasoning efficiencies
  362. 13:15there. So obviously the goal there is to
  363. 13:17basically reduce the long chains of
  364. 13:19thought. Interestingly that reward is
  365. 13:21aimed at the low and medium settings
  366. 13:23while X high is left to focus on getting
  367. 13:25it right. The other interesting thing
  368. 13:26that's mentioned in here is that they
  369. 13:28actually pick the checkpoints based on
  370. 13:31actual agent results not just based on
  371. 13:34training loss. So you could imagine that
  372. 13:36as the model gets better and the RL
  373. 13:39kicks in to reduce that training loss
  374. 13:41that doesn't always lead to the best
  375. 13:43agent runs. So, they're much more
  376. 13:45focused on getting this to actually work
  377. 13:47for agent runs than purely just for some
  378. 13:50LLM training goal. Now, my guess here is
  379. 13:52a big part of why this works is that PI
  380. 13:55is a minimal harness. When it's only got
  381. 13:57a few tools and a really tiny prompt
  382. 13:59like that, every session looks
  383. 14:01reasonably similar. And you can imagine
  384. 14:03that that generates really nice data to
  385. 14:05actually fine-tune on. So, the idea with
  386. 14:08this one is different. Thinking Cap and
  387. 14:09Swift are trying to make your X high
  388. 14:11shorter. Pi is actually trying to make
  389. 14:14the low and medium effort settings
  390. 14:16actually good. On their numbers, Pi's
  391. 14:18medium setting matches the base models X
  392. 14:21high on Terminal Bench with about 41%
  393. 14:24fewer output tokens. They're trying to
  394. 14:26reduce the number of tokens you're using
  395. 14:28by actually getting you to use the lower
  396. 14:30and the medium levels by increasing
  397. 14:32their intelligence up to the high level.
  398. 14:35Now, obviously, the weakness for this
  399. 14:36one is that it's a very narrow use case.
  400. 14:40All the evidence comes from using it
  401. 14:42inside of Pi. From what I've seen,
  402. 14:43nobody has really shown that this
  403. 14:45carries over to other harnesses. If you
  404. 14:47wanted to use open code or you wanted to
  405. 14:49use other things, etc. And on the
  406. 14:51general sort of reasoning tasks like the
  407. 14:53GPQA, the base model still actually wins
  408. 14:56at the medium level. So, in many ways,
  409. 14:58I'd say this is sort of like a community
  410. 15:00testing effort that perhaps isn't
  411. 15:02totally proven yet, but is certainly
  412. 15:04very promising here. So, let's jump in
  413. 15:06and actually take a look at some of the
  414. 15:07outputs that we've got here. Okay, so
  415. 15:09each of these models is running on a
  416. 15:11Dell Pro Max tower here and they're
  417. 15:13running on an RTX Pro 6000 with standard
  418. 15:17VLM here. I haven't really optimized any
  419. 15:19of that in here. So, thank you to Dell
  420. 15:21for sponsoring the compute for this. It
  421. 15:24really makes it possible for me to be
  422. 15:25able to benchmark these models out.
  423. 15:27Okay, so let me set up the demo here.
  424. 15:29What I've basically done is I've gotten
  425. 15:32Claude Fable and Opus to generate a
  426. 15:35bunch of questions or prompts that that
  427. 15:39are going to create a large amount of
  428. 15:40thinking tokens or reasoning tokens etc.
  429. 15:43And then what I've done is I've gone
  430. 15:45through and actually rendered out the
  431. 15:47inference for each of these models and
  432. 15:50saved it with exact timestamps so that
  433. 15:53we can look at all four of them playing
  434. 15:55back at once here. So you can see here
  435. 15:57we've got the original Quen 3.827B.
  436. 16:01This is technically the fine-tuned model
  437. 16:03that they released of their base model,
  438. 16:06but this is sort of the standard version
  439. 16:08of the model. I've got the Quen Pi one,
  440. 16:12I've got the Thinking Cap one, and I've
  441. 16:14got the Swift one in here. So you'll see
  442. 16:16that if I come up and I pick something
  443. 16:18like this, I'm actually playing back at
  444. 16:2116 times the speed that it was recorded.
  445. 16:25And you can see sure enough here this is
  446. 16:27now thinking cap has done. It's
  447. 16:30completed. It's got the correct answer.
  448. 16:33We've now got swift 1.5 done completed
  449. 16:36with again with the correct answer.
  450. 16:38We've got the quen pi model done with
  451. 16:40the correct answer. And finally at some
  452. 16:43point we will get the original 3.827b
  453. 16:48looking at this one also with the
  454. 16:50correct answer there. So in this case,
  455. 16:52each of the models has given us the
  456. 16:54exact same answer back. But we can see
  457. 16:57very clearly in this case that thinking
  458. 16:59cap was way shorter, right? With only
  459. 17:034,000 tokens. You can see that even the
  460. 17:05inference speeds are roughly the same.
  461. 17:07Second is the swift model and third is
  462. 17:11the pi model and then finally is the
  463. 17:14long version. So all three of them are
  464. 17:16doing much better than the original
  465. 17:19version. Let's try it with a different
  466. 17:21prompt. Okay, so again you can see these
  467. 17:24are all off and we can see in this case
  468. 17:27the swift model actually was fastest
  469. 17:30followed by the pi thinking followed by
  470. 17:33the original model and in this case
  471. 17:35thinking cap actually came last. All
  472. 17:37right. Now, I will point out that this
  473. 17:39one is more of a sort of coding task
  474. 17:42with this. And remember that the pi
  475. 17:44model is made for coding tasks and tends
  476. 17:48to specialize in that kind of thing. All
  477. 17:50right. If we look at another one like
  478. 17:52this, we can see that sure enough, in
  479. 17:55this case, the Swift model did really
  480. 17:57well followed by thinking cap. The pi
  481. 17:59model took longer than both of those,
  482. 18:01but still was way shorter than the
  483. 18:03original one. All right. If we give it
  484. 18:05sort of a logic puzzle to go through,
  485. 18:08you see in this case the thinking cap
  486. 18:10one, swift came second followed by the
  487. 18:13base model and then pi. If we give it
  488. 18:15another mathematics one, we can see in
  489. 18:16this one that thinking cap 1, swift came
  490. 18:19second, pi came third and then the
  491. 18:22original model was quite a long way
  492. 18:23behind that. And you see if we translate
  493. 18:25these into time that we're getting huge
  494. 18:29savings with some of these models, the
  495. 18:31thinking cap one is just a lot faster, a
  496. 18:34lot more concise in what it's actually
  497. 18:36producing there. Another example, again,
  498. 18:39thinking cap first followed by Swift.
  499. 18:42And in this case, the base version
  500. 18:43actually beat the PI version. So just to
  501. 18:46finish up, I'll show you one that is
  502. 18:48kind of a bit surprising. In this one,
  503. 18:51you've got the Swift model actually
  504. 18:54doing slightly better. We've then got
  505. 18:57the base model coming second, followed
  506. 18:59by thinking cap, and finally the PI
  507. 19:01model in here. And for me, the
  508. 19:03interesting thing here is that even
  509. 19:05though we've got the same prompt for
  510. 19:07making the SVG, each of them is making a
  511. 19:09slightly different SVG, you know, coming
  512. 19:12out of it. And I would say this is sort
  513. 19:14of the perfect example of where you
  514. 19:16probably should try each of them out for
  515. 19:19your specific use case because just as
  516. 19:21the SVGs change in quality in here, you
  517. 19:25will find that other answers that don't
  518. 19:27necessarily always have an exact answer
  519. 19:30also can change quite a lot from model
  520. 19:32to model in here. While in my test,
  521. 19:34thinking cap seems to be winning most of
  522. 19:36the time, the variance can be really
  523. 19:39different on different kinds of
  524. 19:41questions. And I think that's got a lot
  525. 19:43to do with just how people are doing the
  526. 19:46fine-tunings here. Whatever they make
  527. 19:48the model focus on and use the RL on,
  528. 19:50it's going to get good at that. And I
  529. 19:52think for each of these three fine tunes
  530. 19:54here, they've gone for slightly
  531. 19:56different data curation and perhaps
  532. 19:58curriculum learning in the way that
  533. 20:00they've actually put these together. So
  534. 20:01if you remember back, I also talked
  535. 20:03about that the Quen Pi model was
  536. 20:05actually trained to do better on Medium.
  537. 20:08And the thing I find is that sure
  538. 20:10enough, this model actually is a lot
  539. 20:13faster out on medium than the original
  540. 20:16base model. It doesn't mean though that
  541. 20:18it's always going to be faster for all
  542. 20:20of them. Often the thinking cap one will
  543. 20:23still be faster for some of these. And I
  544. 20:26find that even the thinking cap one
  545. 20:28tends to be a little bit more accurate
  546. 20:30at a lot of the different tasks and
  547. 20:32stuff when you have it actually set to
  548. 20:34medium. In this particular use case
  549. 20:36here, you can see though when we compare
  550. 20:38this that the PI model is actually being
  551. 20:41faster than both of them when we've got
  552. 20:44the setting on medium there. So anyway,
  553. 20:46let me know in the comments if you've
  554. 20:47actually had a play with these, how you
  555. 20:49found them out, which one of the three
  556. 20:51is your favorite. I would say that all
  557. 20:53three of these are actually very good.
  558. 20:55In general, they all tend to do better
  559. 20:58than the base model for a lot of the
  560. 21:00particular tasks. Now, the Pi 1 seems to
  561. 21:03me to be definitely more suited to
  562. 21:04coding kind of tasks and it really
  563. 21:06shines at those. The thinking cap and
  564. 21:08the Swift one are much more sort of
  565. 21:10generalist models. So, anyway, as
  566. 21:12always, if you like the video, click
  567. 21:14like and subscribe and I will talk to
  568. 21:16you in the next video. Bye for now.

About this transcript

This page contains the full transcript of Which is The Best Qwen3.8-27B? by Sam Witteveen, generated from the public captions YouTube serves with the video. The transcript has 4,030 words across 568 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.