China Just BROKE America's Biggest AI Advantage? — Transcript

Full transcript

1. 0:00Hello world. I'm an unemployed ex-big
2. 0:02tech software engineer with 25 years of
3. 0:04experience in the tech industry. So,
4. 0:07over the last couple of days, I've been
5. 0:09reading up on the Huawei Mate Pro Max
6. 0:1290. It's their flagship smartphone and
7. 0:15apparently has already sold millions of
8. 0:17units. Now, don't worry. Early
9. 0:20retirement, it hasn't turned me into
10. 0:22some sort of consumer electronics
11. 0:24fanboy, at least not yet. I'm not really
12. 0:27interested in the smartphone for itself.
13. 0:29I mean, it's an okay phone. It's a
14. 0:31pretty decent phone. But, what I'm
15. 0:33really interested in is the chipset
16. 0:35inside of that phone. So, this phone's
17. 0:38chipset, called the Kirin 9050 Pro, it
18. 0:42uses this novel concept called tau
19. 0:45scaling or time scaling. I believe this
20. 0:48technology will play a crucial role in
21. 0:50the outcome of this AI race between
22. 0:53America and China. So, to give a little
23. 0:56bit of context for this AI competition,
24. 0:58AI, even in its current state, it's
25. 1:01already having a huge profound impact on
26. 1:04humanity. So, the country that dominates
27. 1:07this technological ecosystem, it stands
28. 1:11to gain huge political and financial
29. 1:13benefits from it. Now, America obviously
30. 1:17wants to be a top dog in AI, right? But,
31. 1:20China wants the same spot and that's why
32. 1:22we got this competition going. Now, to
33. 1:25be able to train the most powerful
34. 1:27frontier AI models, you generally need
35. 1:30the most powerful AI chips, right? And
36. 1:34the way that chips get powerful is by
37. 1:36packing more and more transistors onto
38. 1:39the same amount of surface area of the
39. 1:41chip. The higher the transistor density
40. 1:44on that chip, the more computations the
41. 1:47chip can do for the same amount of
42. 1:49energy. Now, one way to describe how
43. 1:52densely we can pack transistors onto a
44. 1:55chip is by using the nanometer process
45. 1:58node. Smaller process nodes allows chip
46. 2:02makers to fit more transistors onto the
47. 2:05same space. So, for example, the current
48. 2:08state-of-the-art chip making process
49. 2:10node, it's the 2-nanometer process node
50. 2:14by a company called TSMC, the world's
51. 2:17biggest chip maker. This process node,
52. 2:20it can create a chip with a transistor
53. 2:22density of 330 million transistors per
54. 2:27square millimeter of surface area. So,
55. 2:30given this background context, an easy
56. 2:33way for Captain America to win this AI
57. 2:36race is to cut off China's supply of
58. 2:39advanced AI chips. Now, we have tried to
59. 2:42do this via two-pronged strategy, right?
60. 2:45Firstly, we tried to sanction the export
61. 2:48of Nvidia's most powerful AI chips to
62. 2:51China. And secondly, we have sanctioned
63. 2:54the export of advanced semiconductor
64. 2:56manufacturing equipment to China.
65. 2:58Equipment like the extreme ultraviolet
66. 3:01lithography machines, or EUV machines.
67. 3:04This EUV tool, you can think of it as
68. 3:06like this ultra-powerful laser that can
69. 3:10etch these tiny nanometer scale circuit
70. 3:13patterns onto a silicon wafer. And by
71. 3:16the way, all EUV lithography machines,
72. 3:20it's made by a single company, this
73. 3:22Dutch company called ASML. So, cutting
74. 3:25off these EUV machines, it makes it way
75. 3:28harder for China to manufacture advanced
76. 3:32AI chips for themselves. Now, the best
77. 3:34lithography tools that China has is
78. 3:37based on this older technology called
79. 3:39deep ultraviolet lithography, or DUV.
80. 3:43And when you use these older DUV tools
81. 3:46to make semiconductors, the entire
82. 3:49process becomes more expensive and more
83. 3:51error-prone. And by using these older
84. 3:54tools, you basically max out at around
85. 3:57the 6-nanometer process node. So, that's
86. 4:00like producing chips with a transistor
87. 4:02density of less than 1/3 of the chips
88. 4:05that America has. So, you got Captain
89. 4:08China over there, right? He's doing
90. 4:10everything, every trick in the book,
91. 4:12jumping through hoops to get more
92. 4:14compute. Everything from networking
93. 4:16hundreds of these less powerful older
94. 4:18chips together and dumping a bunch of
95. 4:20power into it to get it to work, to
96. 4:22smuggling older Nvidia AI chips into
97. 4:25China, to covertly leasing GPU capacity
98. 4:29from data centers that are outside of
99. 4:31the country, and so on. But, it's not
100. 4:34nearly enough, right? Captain America,
101. 4:37he's like drowning in top-shelf,
102. 4:40cream-of-the-crop compute. The man's got
103. 4:42to come up for air. He's got so much
104. 4:44compute. Meanwhile, Captain China, he's
105. 4:47got to settle for the sloppy seconds.
106. 4:49China, in total, in the best case, has
107. 4:53around 1/4 of America's available AI
108. 4:56compute. So, what is Captain China to
109. 4:59do? And this is where the Tao scaling
110. 5:02law comes in. One way to visualize this
111. 5:05law is with a sheet of paper, right?
112. 5:08Imagine that this sheet of paper
113. 5:10represents a silicon chip. And these two
114. 5:14dots, they represent transistors, okay?
115. 5:18For computation to happen, information
116. 5:21has to flow from one transistor to the
117. 5:24other. Well, for the information to
118. 5:26traverse that distance, it requires
119. 5:29time, right? Because information
120. 5:31transfer is bounded by the speed of
121. 5:33light. And it also requires energy in
122. 5:36the form of electrical current. Now, due
123. 5:39to the lack of the mighty ASML EUV pen,
124. 5:43I can't draw these two dots any closer
125. 5:46together. I can't squeeze more
126. 5:48transistors into the same area. But,
127. 5:51what if
128. 5:52we're able to fold this piece of paper
129. 5:56so that the two dots touched each other?
130. 5:59Now, the amount of distance, time, and
131. 6:02energy required to move information
132. 6:04between these two points is dramatically
133. 6:07reduced. You have now effectively
134. 6:09doubled the density of transistors on
135. 6:12the same two-dimensional surface area,
136. 6:14right? Greatly increasing the amount of
137. 6:17computation that can be done for the
138. 6:19same amount of energy. That is the Tao
139. 6:22scaling law. It's sort of like the weird
140. 6:25alternate universe twin of Moore's law.
141. 6:27And this law could, at least in theory,
142. 6:31allow China to produce AI chips that are
143. 6:34nearly as good as America's chips, all
144. 6:37without access to the EUV lithography
145. 6:40equipment. But, of course, theory and
146. 6:42practical reality, they're often two
147. 6:45completely different things, right? Now,
148. 6:47that takes us to this chip inside of
149. 6:50Huawei's smartphone, the Kirin 9050 Pro
150. 6:53chip. It's the first real-world
151. 6:56demonstration of the Tao scaling law in
152. 6:59action. And it uses this manufacturing
153. 7:01technique that Huawei calls logic
154. 7:04folding. And it's a very interesting
155. 7:06chip. Now, before we dive into the
156. 7:08technical weeds, I want to first give a
157. 7:11shout-out to this incredible Mandarin
158. 7:13Chinese-speaking YouTuber called Geeker
159. 7:16Wang. So, this guy, he spent a bunch of
160. 7:19his own money to buy the latest flagship
161. 7:22phone, tear out the chip, and did a
162. 7:24teardown video about this chip. And my
163. 7:27Chinese language lessons, it's been
164. 7:30paying off. I was able to understand a
165. 7:32lot of the technical details that he
166. 7:34went over in his teardown video. I'll
167. 7:37link it in the description. So, anyways,
168. 7:40this new Kirin chip, it's actually two
169. 7:43chips, two silicon dies that are
170. 7:46vertically stacked on top of each other.
171. 7:49The top die, it holds hotter running
172. 7:52components, like the CPU, GPU, 5G modem,
173. 7:56and so on. The bottom die holds the less
174. 7:59computationally intensive components,
175. 8:01right? Like the L1, L2, L3 caches, the
176. 8:04IO interfaces, the memory, and so on.
177. 8:07Now, components that are highly
178. 8:10dependent on each other, they're
179. 8:12co-located directly on top of each
180. 8:14other. Like, for example, the CPU logic
181. 8:17cores. Those cores are placed directly
182. 8:20above the L2, L3 caches, right? And that
183. 8:23allows the minimum amount of distance to
184. 8:26be traveled for information to transfer
185. 8:28between these two components. Now,
186. 8:31what's super interesting to me are these
187. 8:33tiny copper through-silicon via spikes,
188. 8:37or TSV spikes. Each one is like just a
189. 8:40couple of microns in length. If we look
190. 8:43at a cross-sectional view of this chip,
191. 8:46there are like 80,000 of these TSV
192. 8:49spikes distributed across the entire
193. 8:51surface area of the chip. What's
194. 8:53happening is that each of these TSV
195. 8:56spikes, it allows energy to flow between
196. 8:59these two silicon dies, and it allows
197. 9:02for rapid data exchange between these
198. 9:05two layers. Now, with all of these
199. 9:07architectural innovations, Huawei is
200. 9:11claiming that their Kirin chip has
201. 9:13achieved a transistor density of 231
202. 9:16million transistors per square
203. 9:18millimeter. This lands the Kirin chip
204. 9:21somewhere in between the past generation
205. 9:24TSMC 3-nanometer process and the current
206. 9:28generation 2-nanometer process. Meaning,
207. 9:31with Tao scaling, China is now able to
208. 9:34manufacture chips that are within one
209. 9:38generation, within one year of the
210. 9:41bleeding edge of semiconductor
211. 9:43development. And interestingly enough,
212. 9:46with the same Tao scaling logic folding
213. 9:49technique, China has essentially gained
214. 9:51complete independence of the
215. 9:53manufacturing of these chips, right?
216. 9:56Because now they control the entire
217. 9:58semiconductor supply chain to be able to
218. 10:01make these chips. Everything from
219. 10:03materials to semiconductor manufacturing
220. 10:06equipment to design software and so on.
221. 10:09Everything is now done in-house, making
222. 10:12this entire operation basically
223. 10:14impossible to sanction. Now, obviously
224. 10:18there are big differences between a
225. 10:20smartphone chip and an AI chip, right?
226. 10:22It's really an apples to oranges
227. 10:24comparison.
228. 10:26But if we think about it, there are at
229. 10:28least a couple of ways in which an AI
230. 10:31data center is actually a more forgiving
231. 10:33environment than a smartphone case,
232. 10:35right? Especially in terms of things
233. 10:37like power and energy management,
234. 10:40thermal management, and physical space.
235. 10:43So, the same Tao scaling techniques used
236. 10:46in the smartphone chip, it can be
237. 10:49adapted to make more powerful AI chips.
238. 10:52And it seems like Huawei is doing
239. 10:54exactly this. They're adopting some of
240. 10:58these Tao scaling 3D stacking techniques
241. 11:01into their Ascend series of AI
242. 11:03accelerator chips, which are by the way
243. 11:06in volume production right now. So, to
244. 11:08me, this is a game changer, right?
245. 11:11Because if we think about it, AI model
246. 11:14development, it's kind of dependent on
247. 11:15these three pillars, right? Algorithm,
248. 11:19data, and compute. And it's pretty
249. 11:21obvious that America and China, they're
250. 11:24roughly comparable in terms of
251. 11:26algorithms and data. It's really in
252. 11:30compute that America had a clear
253. 11:32decisive advantage over China. It's
254. 11:34pretty much the only thing that's
255. 11:37keeping America's closed source AI
256. 11:40models marginally ahead of the open
257. 11:42source open weight Chinese models. And
258. 11:45because of techniques like towel
259. 11:47scaling, this advantage is disappearing.
260. 11:50With access to more and better AI
261. 11:53compute, it is my belief that Chinese AI
262. 11:56models may in the very near term reach
263. 11:59parity with American models. It might
264. 12:02even be slightly better than American
265. 12:04models. Most importantly, with these
266. 12:06models being open source and open
267. 12:08weight, anybody can download these
268. 12:11models and improve upon them. This opens
269. 12:14the door for other countries who don't
270. 12:17have nearly the same amount of compute
271. 12:19as America or China for them to improve
272. 12:22upon these open source models and create
273. 12:25kind of like these good enough versions
274. 12:27of these models for themselves. Through
275. 12:29the narrowing of this compute gap at
276. 12:32this point in the AI race, I think now
277. 12:35it's becoming conclusive that no one no
278. 12:38one country will be able to develop an
279. 12:40AI model that is significantly superior
280. 12:43to anybody else. Why? Because there are
281. 12:46fundamental limits to how much we could
282. 12:49improve large language models by
283. 12:51throwing compute at it. There was this
284. 12:53recent report that studied this
285. 12:55mathematical ceiling in detail. The
286. 12:57simple takeaway is this. The more we
287. 13:00improve an AI model, the more computing
288. 13:03power we have to throw at it to get each
289. 13:06additional unit of performance gain from
290. 13:09that AI model. This increase is
291. 13:11exponential. Today, we're already
292. 13:13spending on the order of two to three
293. 13:15hundred million dollars to train a
294. 13:18single AI model. So, imagine in the near
295. 13:21future, we might need hundreds of
296. 13:23billions of dollars to train a model
297. 13:26that may only be a few percentage points
298. 13:29better in performance versus its
299. 13:31predecessor. At some point, the juice
300. 13:34just wouldn't be worth squeezing. And
301. 13:36according to this report, we could be
302. 13:39hitting the ceiling within just a few
303. 13:41years, or potentially even sooner. And
304. 13:44that's why I believe the AI race won't
305. 13:46end with one country pulling vastly
306. 13:48ahead of everybody else. Instead, I
307. 13:51think the major players, they will spend
308. 13:53more and more money chasing ever smaller
309. 13:57gains until the cost becomes simply
310. 13:59unbearable. At which point, everybody's
311. 14:02frontier models would converge towards
312. 14:04this capability ceiling. And AI will
313. 14:07become a commodity, sort of like
314. 14:09cabbages, where every country has it,
315. 14:12and no one country or company has a
316. 14:14clear advantage. Thus, I believe the AI
317. 14:17race will not end with a clear
318. 14:20[clears throat] winner. It will end in
319. 14:21the sheer exhaustion of all parties
320. 14:23involved. That's how this race ends.
321. 14:27And that's all I have to say about that.
322. 14:28Hope it helps.
323. 14:31Anyways, if you have a morbid curiosity
324. 14:33to join me in this life journey, please
325. 14:35subscribe to my YouTube channel and
326. 14:37subscribe to my Substack newsletter. If
327. 14:39you want to support my vlog creation
328. 14:42efforts, please become a paid member of
329. 14:44the channel and a paid member of my
330. 14:45Substack, or just buy me a coffee. If
331. 14:48you want a one-on-one coaching session
332. 14:50with me, just schedule it. Anyways,
333. 14:52thanks so much for listening. Talk soon.
334. 14:54Bye.