YouTube2Text

【ローカルLLM】32GBメモリで今どのモデルを使う?Qwen4を待つのをやめて Flash Next にした話 — Transcript

by ふじもったチャンネル · 1,100 words · 189 segments · language en · Watch on YouTube

Full transcript

  1. 0:01No way, I’ve been so anxious all week,
  2. 0:03but I think everyone’s starting to
  3. 0:06realize it: Qwen 4 isn't coming out, is
  4. 0:08it?
  5. 0:11It's like, "I guess Qwen 4 isn't
  6. 0:13happening."
  7. 0:18I mean, since last week, everyone kept
  8. 0:20saying it was going to drop, kept going
  9. 0:22on and on about it.
  10. 0:26I really thought it would, but the 27-
  11. 0:28billion parameter one didn't show up,
  12. 0:31and I was just, like, totally crushed.
  13. 0:37Just completely gutted.
  14. 0:40Like, uh, Kodak or something.
  15. 0:45You know, it had been kept under wraps
  16. 0:47like some sort of hidden model, but it
  17. 0:50turns out the one everyone thought was
  18. 0:5227 billion was actually just Qwen 4 Max
  19. 0:54.
  20. 0:59And Qwen 4 Max is available now as a
  21. 1:01preview model for, like, two weeks at a
  22. 1:04super cheap price.
  23. 1:08But honestly, every time I use Qwen 4
  24. 1:11Max, I’m just like, "Eh, it's not that
  25. 1:13great," and keep judging it as
  26. 1:15underwhelming.
  27. 1:21Since Qwen 4 didn't drop after all, I’ve
  28. 1:24realized it’s a lost cause, and now I’m
  29. 1:26thinking there’s a real chance it won't
  30. 1:28be released this month.
  31. 1:34It's looking like it might not come out
  32. 1:36this month after all.
  33. 1:39At the very least, I don't think the
  34. 1:40next step will happen until this
  35. 1:42two-week preview ends.
  36. 1:47So, I’ve decided to stop holding my
  37. 1:49breath for Qwen and start looking into
  38. 1:51something new.
  39. 2:00On the other hand, there are actually
  40. 2:01some new movements happening outside of
  41. 2:03Qwen.
  42. 2:07Where was it again?
  43. 2:11There’s talk about a model called Ernie
  44. 2:13, or something like that, from Baidu
  45. 2:15coming out late this month, so things
  46. 2:17outside of Qwen are starting to look
  47. 2:19active again.
  48. 2:23Yeah.
  49. 2:26I think a model based on Qwen 3.8 Flash
  50. 2:28Next that’s been further fine-tuned
  51. 2:30might have even come out today.
  52. 2:34Wait, what was that model called? What
  53. 2:36was it again?
  54. 2:42There are models like the Edge 145B and
  55. 2:44120B, which seem to be built by applying
  56. 2:47the structure of Flash-Next directly, so
  57. 2:50I think this might become a new trend.
  58. 2:59However, when you look at the licensing
  59. 3:01, it’s quite complicated, and, honestly
  60. 3:03, you can't really use them for
  61. 3:05commercial purposes.
  62. 3:08So, when it comes to business use, it’s
  63. 3:10a model that’s a bit hard to choose.
  64. 3:18Right now, I'm focusing on a slightly
  65. 3:20older model, DeepSeek version 4, the V4
  66. 3:23Flash, and I've been wondering if I
  67. 3:25could run it locally.
  68. 3:30Well, not just wondering, I was actually
  69. 3:32thinking about that. The V4 model is
  70. 3:34actually a bit smaller in size. What was
  71. 3:37it again? It was around 280, something
  72. 3:40like that.
  73. 3:42The active parameters are around 12
  74. 3:44billion, so, in other words, since the
  75. 3:47active size isn't that large and the
  76. 3:49total size isn't, you know, 500 or
  77. 3:51something, I thought it might be
  78. 3:53interesting if I could get it running,
  79. 3:55so I tried it out.
  80. 3:59It really is satisfying when it works.
  81. 4:03Surprisingly, the speed was 2.6 tokens
  82. 4:06per second.
  83. 4:08On an M6 Mac.
  84. 4:10"Wow, that's slow," I thought to myself
  85. 4:12. Yeah.
  86. 4:15But even so, the fact that it actually
  87. 4:18runs was a bit fresh for me; it was like
  88. 4:21, "Wow, DeepSeek V4 is actually running
  89. 4:23," so I was quite moved.
  90. 4:30Still, after trying it out, I realized
  91. 4:32it is slow; 2.5 tokens per second is
  92. 4:35just damn slow, and it ended up being
  93. 4:38just a romantic experiment.
  94. 4:42Yeah. I definitely don't think I'll be
  95. 4:44using this for practical work.
  96. 4:47Yeah.
  97. 4:52When you think about it that way, Qwen
  98. 4:543.8 Flash-Next reaching 17 tokens per
  99. 4:57second is really something quite
  100. 4:59impressive.
  101. 5:07I wonder, Qwen is probably strong when
  102. 5:09it comes to smaller models, I guess.
  103. 5:14I'm actually using the 27-billion
  104. 5:16parameter Qwen 3.8 Flash-Next model
  105. 5:19again lately.
  106. 5:23The other day, Anthropic announced a new
  107. 5:25quantization method, maybe it was
  108. 5:27Dynamic 3.1 or something.
  109. 5:32They released something like that, which
  110. 5:34apparently improves the accuracy even
  111. 5:36further.
  112. 5:40So, even with the same quantized model,
  113. 5:43the size goes up just a little bit, but
  114. 5:45the performance improves by about 10%,
  115. 5:47and stuff like that is popping up here
  116. 5:50and there.
  117. 5:54Even when I try playing around with it,
  118. 5:56I feel like Anthropic's model is more
  119. 5:58stable.
  120. 6:03It feels like there's less garbled text
  121. 6:05, so I've been using Anthropic Dynamic's
  122. 6:08Qwen 3.8 Flash Next, the one they call
  123. 6:11the 2-bit model, IQ2XS.
  124. 6:19But in reality, if you add up all the
  125. 6:21quantization settings and divide them,
  126. 6:24it apparently comes out to 3.5
  127. 6:25quantization.
  128. 6:30It’s like 3.50 BPW, so it’s actually
  129. 6:33quite strong.
  130. 6:35Yeah.
  131. 6:38So, the number feels a bit like a lie,
  132. 6:41but that’s why I use that Anthropic Qwen
  133. 6:443.8 Flash Next model quite regularly.
  134. 6:52I run that using the Splash engine, and
  135. 6:54as for the tool, I use an agent tool
  136. 6:57called Hermes Agent to interface with it
  137. 7:00—so, Qwen 3.8 Flash Next via Splash.
  138. 7:07And the interface is the Hermes Agent
  139. 7:09that I have installed on Windows.
  140. 7:12Splash is already at 1.6.1, they say.
  141. 7:17Yeah, Splash has been updated to 1.6.1.
  142. 7:24If you haven't updated yet, please make
  143. 7:26sure you do.
  144. 7:28Memory efficiency has improved
  145. 7:30significantly, so it’s definitely better
  146. 7:32to update.
  147. 7:37It’s gotten a lot better with memory
  148. 7:39efficiency and how it handles SSD
  149. 7:41caching, so it can process things more
  150. 7:44stably.
  151. 7:50Yes. And sorry to keep asking. How much
  152. 7:53speed do you get when running Qwen 3.8
  153. 7:55Flash Next with 32GB of memory? In my
  154. 7:58current setup, it runs at 16.7 tokens
  155. 8:01per second. Yes. About 40 watts.
  156. 8:05That is, the power consumption.
  157. 8:09This fluctuates quite a bit depending on
  158. 8:11what kind of questions or input you
  159. 8:13throw at it.
  160. 8:15>> Yeah.
  161. 8:19>> So, well, during standard agent
  162. 8:22operations, it sometimes hits 20 tokens
  163. 8:24, but occasionally, in turns where it
  164. 8:26needs to think things through, it might
  165. 8:29drop to 11 or 12, so the token speed is
  166. 8:31actually quite volatile.
  167. 8:37Well, I think that kind of fluctuation
  168. 8:39is probably the same for any model.
  169. 8:41Still, you can definitely have plenty of
  170. 8:43fun with it.
  171. 8:46Yeah.
  172. 8:48Yeah.
  173. 8:50Lately, though, it feels like things
  174. 8:53have come full circle, and running AI—
  175. 8:55or rather, running local LLMs—has become
  176. 8:58quite stable with Splash.
  177. 9:01Though, I do sometimes find myself
  178. 9:03wondering what exactly I should be
  179. 9:05running on Splash.
  180. 9:10I go back and forth between running a 27
  181. 9:12-billion parameter 4-bit quantized model
  182. 9:15or something like Flash-Next 3.5, or a
  183. 9:183.5 B model, but when I really need
  184. 9:20processing speed, I know I should stick
  185. 9:23with the 27B.
  186. 9:31However, when I'm unsure about a
  187. 9:33decision or really want it to think
  188. 9:35things through, Flash-Next is clearly
  189. 9:37the stronger choice.

About this transcript

This page contains the full transcript of 【ローカルLLM】32GBメモリで今どのモデルを使う?Qwen4を待つのをやめて Flash Next にした話 by ふじもったチャンネル, generated from the public captions YouTube serves with the video. The transcript has 1,100 words across 189 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.