AI内存的技术混战 | 伯恩斯坦研究报告 | AI内存 | HBM | CXL | DRAM | NAND | KV缓存 | 存储器产业 | AI算力底层硬件 | 存储级内存 | 人工智能推理 — Transcript
Full transcript
- 0:00Hello everyone, this is Best Partner,
- 0:02I'm Da Fei. When most people talk about
- 0:04AI memory, the first reaction is HBM,
- 0:07as if HBM is all there is to AI memory,
- 0:09and as long as you stack HBM, all
- 0:11problems are solved. But you might not
- 0:13know that what really bottlenecks AI
- 0:15memory is not the training stage, but
- 0:17the decoding KV cache in inference. The
- 0:20expansion speed far exceeds most
- 0:22people's expectations; new industry
- 0:24terms pop up every day, such as HBF,
- 0:27ZNAND, CXL, CMX, ZM, HBC, and so on.
- 0:30The industry is busy creating both new
- 0:32technologies and abbreviations at a
- 0:33speed that many practitioners can't
- 0:35even keep up with. Recently, four
- 0:37analysts from Bernstein jointly
- 0:39authored the latest introductory report
- 0:41on AI memory. They categorized all
- 0:43these new technologies into two
- 0:45frameworks: one based on system
- 0:47architecture levels, and the other on
- 0:49hardware media levels. So, in today's
- 0:51video, we will follow this framework to
- 0:53clarify the full picture of AI memory.
- 0:55Let's start with the basics: different
- 0:57stages of AI tasks have completely
- 0:59different requirements for memory and
- 1:01cannot be generalized. Many people
- 1:03start by talking only about HBM, which
- 1:04is actually oversimplifying the problem
- 1:06. Although training is the most
- 1:08compute-intensive stage in the entire
- 1:10AI process, memory bandwidth is
- 1:11critical. Memory bandwidth directly
- 1:13determines the model size you can run
- 1:15and the training speed. The core status
- 1:17of HBM in training goes without saying,
- 1:20but training definitely cannot run on
- 1:22HBM alone. Think about it, massive
- 1:24amounts of training data must be
- 1:25prepared in advance and stored in
- 1:27slower, cheaper storage media. During
- 1:29the training process, data must be
- 1:30gradually scheduled through multiple
- 1:32layers of cache before it can finally
- 1:34enter the HBM. Moreover, training can
- 1:35run for months, requiring frequent
- 1:37checkpointing in between to prevent
- 1:38soft or hard hardware failures from
- 1:40causing a restart. Simply put, training
- 1:42requires the entire hierarchy of memory
- 1:44to function, from HBM to system RAM,
- 1:46local SSDs, and network storage;
- 1:48missing any link won't work. Inference,
- 1:50on the other hand, is even more
- 1:51interesting. It is divided into two
- 1:53completely different stages: prefill
- 1:55and decoding. The bottlenecks for the
- 1:56two are completely different. The
- 1:58prefill stage processes the user's
- 1:59input prompt to generate the first
- 2:01token. This stage is dominated by
- 2:02matrix operations with high GPU
- 2:04utilization; it's a typical
- 2:05compute-bound scenario where data
- 2:07movement time is much lower than
- 2:08compute time, making memory pressure
- 2:10relatively smaller and HBM's role more
- 2:11prominent. The core metric for this
- 2:13stage is the time to first token, or
- 2:15TTFT. In a typical chat scenario, under
- 2:170.5 seconds is ideal, one second is
- 2:19generally acceptable, and exceeding two
- 2:21seconds causes users to lose patience.
- 2:23If you are handling heavy tasks like
- 2:25processing hundreds of pages of
- 2:26documents, the tolerance is higher, and
- 2:28with a "thinking" prompt, users can
- 2:29still find it acceptable. However, the
- 2:31logic for the decoding stage is
- 2:33completely different. Tokens are
- 2:34generated one by one, which is
- 2:35academically called autoregressive
- 2:37generation, where each new token
- 2:38requires information from all previous
- 2:40tokens. To avoid redundant calculations
- 2:42, current large models will store
- 2:44previous key-value pairs, which is
- 2:46known as KV cache. The size of the KV
- 2:48cache grows linearly with the number of
- 2:50tokens. There is a key difference here:
- 2:52model weights are read-only and can be
- 2:54shared among all users, but the KV
- 2:56cache is unique to each user, so the
- 2:57more concurrent users there are, the
- 2:59higher the total consumption. Once both
- 3:01are combined, the decoding stage
- 3:03becomes a typical memory-bound scenario
- 3:05. What many people may not realize is
- 3:07that in large-scale deployments, the
- 3:09memory consumed by the KV cache can be
- 3:10even more than the model weights
- 3:12themselves. It directly determines how
- 3:14many users a server can serve
- 3:15simultaneously and how large a context
- 3:17window it can support; in other words,
- 3:19KV cache efficiency directly impacts an
- 3:21AI model's deployment scale and revenue
- 3:23ceiling. The core metric for the
- 3:25decoding stage is the time per output
- 3:27token, or TPOT. For text chat, anything
- 3:29under 50 milliseconds is generally
- 3:31acceptable. For high-demand scenarios
- 3:33like real-time voice or programming
- 3:34agents, this must be pushed under 10
- 3:36milliseconds. Beyond training and
- 3:37inference, the memory requirements for
- 3:39Retrieval-Augmented Generation, or RAG,
- 3:41follow a different logic. RAG connects
- 3:42the model to external databases, using
- 3:44specific databases or the latest
- 3:45information to supplement the model's
- 3:47knowledge, while also reducing
- 3:48hallucinations. The entire process is
- 3:50divided into database generation and
- 3:52searching. The generation stage
- 3:53involves processing massive amounts of
- 3:55unstructured data like PDFs, code, and
- 3:57web pages into a searchable vector
- 3:59database. First, you need
- 4:00large-capacity storage for raw data;
- 4:02SSDs can lower access latency much
- 4:04better than HDDs. Generating vectors
- 4:06and indexing relies mainly on the
- 4:08system, with little actual use of HBM.
- 4:10This work is done offline and is
- 4:12independent of the number of users or
- 4:14tokens; it is a one-time investment.
- 4:16During the retrieval phase, user
- 4:17queries are first converted into
- 4:19vectors. This is processed quickly by
- 4:20HBM, but most of the time is spent
- 4:22searching for the best matches in the
- 4:23vector database. This part relies on
- 4:25system-level retrieval. Once the
- 4:26results are merged into the user prompt
- 4:28, it follows the standard prefill and
- 4:30decoding inference process, with memory
- 4:31requirements consistent with what we
- 4:33discussed earlier. Moving up to agentic
- 4:34workflows, memory pressure reaches an
- 4:36even higher level. It is not a single
- 4:38prompt-to-output cycle, but an
- 4:39autonomous multi-step iteration where
- 4:41the system breaks down goals, calls
- 4:42tools, evaluates results, and performs
- 4:44rollbacks for corrections if errors
- 4:45occur. This process requires additional
- 4:47computation and memory to manage states
- 4:49, store intermediate results, and
- 4:50handle external tool interactions,
- 4:52generating significant demand for
- 4:53traditional CPUs and standard memory.
- 4:55More importantly, the output of one
- 4:57agent becomes the input for the next.
- 4:59When multiple agents are running, the
- 5:00requirements for prefill, decoding, and
- 5:02KV cache stack up, making memory
- 5:04pressure much greater than in
- 5:05single-round inference. After covering
- 5:07the differences in workload
- 5:08requirements, let's look at how memory
- 5:10is layered from a system architecture
- 5:12perspective. We have categorized memory
- 5:14in AI systems into a complete hierarchy
- 5:16from G0 to G4 based on architectural
- 5:17location and function. This
- 5:19classification is not an official
- 5:21standard; it integrates NVIDIA's SIDM
- 5:23definitions while incorporating new
- 5:25technologies like HBM, CXL, Storage,
- 5:27and Next into a comprehensive framework
- 5:29. After understanding this, you will
- 5:31see where each new technology fits in
- 5:33the system and what issues it solves.
- 5:35G0 memory is typically SRAM, featuring
- 5:37extremely low latency and very small
- 5:39capacity, used for immediate
- 5:41computational caching and produced on
- 5:43the same logic die as the accelerator.
- 5:46G1 is HBM, or High Bandwidth Memory,
- 5:48which uses CoWoS advanced packaging to
- 5:51stack DRAM chips together with the
- 5:53accelerator, offering high bandwidth
- 5:55and low latency. Model weights and
- 5:57frequently accessed "hot" data are
- 5:59stored at this level. G2 is system
- 6:01memory, the common DDR DRAM, which has
- 6:04larger capacity but higher latency than
- 6:06HBM and is typically located next to
- 6:08the CPU. Both NVIDIA and AMD GPUs can
- 6:11bypass the CPU to access this memory
- 6:13directly. G 2.5 is Compute Express Link
- 6:15(CXL) memory; this technology was
- 6:17initially driven by Intel and has now
- 6:19gained full industry support. Its core
- 6:21is to integrate physically separated
- 6:23memory into a logical shared pool to
- 6:25avoid resource waste. Through CXL, the
- 6:27system can access accelerator HBM;
- 6:29accelerators can also access
- 6:30CPU-managed system memory, and even
- 6:32PCIe-based memory expanders can be
- 6:34incorporated into the shared pool.
- 6:36Later, CXL expanded support for NAND.
- 6:38The prerequisite is that NAND products
- 6:40must be optimized to perform closer to
- 6:42DRAM characteristics, usually by
- 6:44pairing them with DRAM or SRAM as a
- 6:45cache to mask NAND's high latency.
- 6:47Samsung's CMM-H is a CXL module that
- 6:50blends DRAM and NAND. Next down is G
- 6:532.6, called StorageNext memory. This is
- 6:55an ecosystem project led by NVIDIA, and
- 6:57almost all mainstream storage vendors
- 6:59have participated. It has two core
- 7:01directions: one is shifting memory
- 7:03management from being CPU-centric to
- 7:05GPU-centric; the second is making NAND
- 7:07act more like DRAM, enabling it to
- 7:08enjoy NAND's low cost and high capacity
- 7:10while achieving near-DRAM latency and
- 7:12fine-grained data access. It is
- 7:14positioned between DDR DRAM and local
- 7:16SSDs, with products mostly being
- 7:18hybrids of DRAM and NAND. Many utilize
- 7:20Single-Level Cell (SLC) chips; Kioxia's
- 7:23FL6 series SSD is representative of
- 7:25this type of product, focusing on
- 7:27ultra-high IOPS, or input/output
- 7:29operations per second. Third is the
- 7:31local SSD using PCIe hardware standards
- 7:33and NVMe protocol standards, located at
- 7:35the edge of the motherboard within a
- 7:36single compute node, also known as a
- 7:38computational SSD. This layer extends
- 7:40capacity beyond DRAM, but cannot
- 7:41communicate across nodes; it is usually
- 7:43managed by the CPU, with some
- 7:44accelerators capable of direct access.
- 7:46G 3.5 is CMX memory and the STX
- 7:48architecture. CMX stands for Context
- 7:51Memory, previously called SMS (Serving
- 7:53Memory Storage), a tier added by NVIDIA
- 7:56specifically for KV caching. SSDs
- 7:58compliant with the CMX standard are
- 8:00deployed in data nodes and can be
- 8:02accessed by multiple compute nodes
- 8:04across the entire cluster, suitable for
- 8:06shared dynamic content in multi-agent
- 8:08scenarios. Compute nodes and data nodes
- 8:10can be connected via Ethernet, with
- 8:11forwarding handled by Spectrum-X switch
- 8:13chips in between, and both sides
- 8:15equipped with BlueField-4 DPUs. GPUs
- 8:17can access storage directly without
- 8:19requiring CPU intervention. STX is a
- 8:21reference architecture launched by
- 8:22NVIDIA to integrate solutions from
- 8:24various storage and ODM vendors,
- 8:26ensuring compatibility between products
- 8:27. The bottom layer is G4, shared
- 8:30storage, which can be SSDs or HDDs,
- 8:32connected via PCIe or network to store
- 8:34long-term, non-critical path data. As
- 8:37long as the network and software
- 8:38support it, GPUs can also access this
- 8:40layer of storage directly. With the
- 8:42architectural layers covered, let’s
- 8:43now look at storage innovation from the
- 8:45perspective of hardware media. This is
- 8:46another dimension of classification,
- 8:48based on the storage unit structure and
- 8:49operating principle, overlaid with
- 8:51various packaging and modular
- 8:52technologies. By combining these two
- 8:54dimensions, the industry can tailor
- 8:55solutions that meet different
- 8:57requirements. SRAM is the fastest
- 8:58within the entire hardware hierarchy.
- 9:00Its unit structure consists of six
- 9:02transistors, which occupies more
- 9:03silicon area than the 1T1C structure of
- 9:05DRAM—that is, one transistor plus one
- 9:07capacitor. Both are volatile memories;
- 9:09once the power is cut off, the data is
- 9:11lost. DRAM requires constant refreshing
- 9:12of its capacitors, while SRAM does not
- 9:14require refresh circuits. Most SRAM
- 9:15today is embedded inside logic chips;
- 9:17standalone packaged versions only exist
- 9:19in niche scenarios, corresponding to
- 9:20memory levels in the architecture used
- 9:22for various cache tiers. L1 cache is
- 9:24closest to the computing unit with the
- 9:26smallest capacity, while L2 and L3
- 9:27caches have larger capacities but
- 9:29slightly higher latency. AI has also
- 9:31spurred new uses for SRAM. For example,
- 9:33the LPU (Language Processing Unit) from
- 9:36Groq—the current version is the LPU
- 9:3820—uses on-chip SRAM to satisfy the
- 9:40most latency-sensitive parts of
- 9:42inference. The next-generation LPU40 is
- 9:44rumored to use hybrid bonding
- 9:45technology to stack SRAM chips onto
- 9:47logic processor chips, resulting in
- 9:49only a slight increase in latency but
- 9:50allowing for much larger capacity and
- 9:52better cost control. This usage roughly
- 9:54corresponds to the memory hierarchy in
- 9:56architecture. There is also embedded
- 9:58DRAM, which sits inside logic chips to
- 10:00compete with embedded SRAM for
- 10:01large-capacity cache scenarios, such as
- 10:03L4 cache. IBM’s POWER processors and
- 10:05Sony’s PlayStation use it, but the
- 10:07overall market share is currently small
- 10:09. DRAM is the absolute powerhouse of
- 10:11revenue and profit for the entire
- 10:12storage industry, larger in scale than
- 10:14NAND. HBM is essentially DRAM as well,
- 10:16utilizing the same wafer capacity as
- 10:18traditional DRAM. In CXL memory
- 10:19expansion, which corresponds to the G1
- 10:21and G2 tiers of the architecture, many
- 10:23NAND products are used, often paired
- 10:24with DRAM as a cache to prefetch data
- 10:26and mask NAND's high latency, though
- 10:28the volume is not large. The evolution
- 10:30of DRAM technology still relies mainly
- 10:31on Moore’s Law, which means EUV
- 10:33lithography is essential. Manufacturers
- 10:35without EUV will gradually lose
- 10:37competitiveness in the DRAM field in
- 10:38the long run. Specifically in China,
- 10:40CXMT, as a major domestic DRAM
- 10:42manufacturer, is developing 4 F² DRAM
- 10:44to compete with traditional 6 F² 1T1C
- 10:47DRAM units. The 4 F² design shrinks
- 10:49the unit area by moving peripheral
- 10:51circuits from beside the capacitor to
- 10:53underneath it, or even to another wafer
- 10:55. However, the improvement brought by
- 10:574F squared is a one-time thing, much
- 10:58like the FAT in logic chips. After that
- 11:00, further development will still rely
- 11:02on lateral scaling. Key players are
- 11:04also researching 3D DRAM, similar to
- 11:06the 3D NAND approach, by vertically
- 11:07stacking memory cells. The average cell
- 11:09area can be smaller than 4F squared,
- 11:11with potential for continuous scaling,
- 11:13but the technical difficulty of 3D DRAM
- 11:14is much higher, and it has a long way
- 11:16to go before commercialization.
- 11:18Manufacturers are innovating in
- 11:19different directions around HBM. There
- 11:21are many technical paths here, so let's
- 11:23go through them one by one. Samsung’s
- 11:25ZHBM aims to push packaging from 2.5 D
- 11:27to 3D. Current HBM stacks sit next to
- 11:29the XPU and connect via an interposer.
- 11:31ZHBM wants to place the HBM stack
- 11:33directly on top of the XPU to further
- 11:35shorten communication distances and
- 11:36increase bandwidth. However, the
- 11:38problem is also obvious: the underlying
- 11:40XPU generates a lot of heat, and it is
- 11:41a major question whether the DRAM chip
- 11:43on top can work stably at high
- 11:44temperatures. Samsung intends to use
- 11:46wafer-to-wafer hybrid bonding so there
- 11:48are no gaps between the chips, which
- 11:49should improve heat dissipation. But it
- 11:51is hard to say right now if the yield
- 11:53and cost of hybrid bonding can support
- 11:54large-scale mass production. NVHBM is a
- 11:57customized HBM led by NVIDIA. Its base
- 11:59die is designed by NVIDIA and produced
- 12:02by TSMC. Moving the memory controller
- 12:04from the XPU to the HBM base die can
- 12:05reduce the XPU's chip size, lower costs
- 12:07, or free up space. This allows for
- 12:09more compute units while also
- 12:10increasing bandwidth and reducing power
- 12:12consumption. NVIDIA is pushing NVHBM
- 12:14toward openness. On one hand, it
- 12:16standardizes the implementation,
- 12:17allowing multiple memory manufacturers
- 12:18to......participate in the supply. On
- 12:20the other hand, through NVLink Fusion,
- 12:22other manufacturers 'accelerators can
- 12:23also use it; Amazon's Annapurna Labs
- 12:25has already announced a partnership.
- 12:27Compared to standard HBM, NVHBM will
- 12:28compress the value space for memory
- 12:30manufacturers. The base die originally
- 12:32designed by memory makers is now
- 12:33designed by NVIDIA and manufactured by
- 12:35a foundry, so the differentiation space
- 12:37for memory makers will be constrained.
- 12:39Intel also has two DRAM-related
- 12:41technical paths: XBM and FeRAM. XBM
- 12:43stands for Cross-point memory.
- 12:45According to its patents, it uses
- 12:46back-end-of-line processes,
- 12:48specifically back-end lithography and
- 12:49thin-film transistors, to build DRAM
- 12:51cells. Traditional DRAM cells are made
- 12:53in the front-end process, with the
- 12:54back-end only used for metal
- 12:55interconnects. XBM places memory cells
- 12:57between the back-end metal layers, so
- 12:59DRAM cells and logic circuits can be
- 13:01integrated on the same chip, making it
- 13:03a potential candidate for in-memory
- 13:05computing. There aren't many publicly
- 13:06available details yet, so it is hard to
- 13:08judge the mass production prospects of
- 13:09this technology or who is best suited
- 13:11to produce it. The full name of-Z is
- 13:12Z-angle memory, an innovation at the
- 13:14packaging level that rotates stacked
- 13:15DRAM chips by 90 degrees to arrange
- 13:17them horizontally, with the vertical
- 13:19gaps between chips improving thermal
- 13:20efficiency. Intel's partner, SoftBank's
- 13:22subsidiary SMOR, stated that they still
- 13:24source DRAM chips from memory
- 13:25manufacturers, focusing mainly on
- 13:27unique packaging and substrate design.
- 13:29The physical connection between the
- 13:30chips and the substrate after rotation
- 13:32is highly difficult; achieving data
- 13:34transmission is the core challenge,
- 13:36with practical use expected by 2029.
- 13:37Qualcomm's High Bandwidth Compute (HBC)
- 13:39takes a cost-performance approach,
- 13:41forgoing CoWoS advanced packaging for
- 13:42traditional packaging; while
- 13:44performance is somewhat compromised, it
- 13:45saves the high costs of interposers and
- 13:472.5 D organic substrates, while using
- 13:49LPDDR to reduce power consumption. The
- 13:51first generation is expected to ship in
- 13:532027, with the second generation
- 13:54released in 2028. Beyond chips and
- 13:56packaging, DRAM module form factors are
- 13:58also innovating, with MRDIMM, CAMM2,
- 14:00and LPCAMM2 serving as representatives
- 14:01of this trend. MRDIMM shares the same
- 14:03physical dimensions as traditional
- 14:05server RDIMMs, simply adding more
- 14:06interface chips to the module to
- 14:08increase bandwidth. CAMM2 features more
- 14:10significant changes, with both the
- 14:11module's shape and dimensions differing
- 14:13from traditional standards. It supports
- 14:15LPDDR chips, doubles the data bus width
- 14:17from 64 to 128 bits, and has a lower
- 14:19profile, which reduces power
- 14:20consumption, boosts single-module
- 14:22bandwidth, and improves airflow and
- 14:24cooling inside servers. Adding a side
- 14:26note: the first-generation CAMM was an
- 14:28Nvidia-proprietary experimental version
- 14:30, while CAMM2 has been officially
- 14:31released by JEDEC as an industry
- 14:33standard; it is first being adopted by
- 14:35Intel's CPUs, and AMD's CPUs will
- 14:37follow with support. LPCAMM2 is the
- 14:39corresponding version for AI laptops,
- 14:40also standardized by JEDEC; the early
- 14:42LPCAMM prototype was led by Samsung,
- 14:44and these modules all correspond to the
- 14:46second level of the architecture,
- 14:47alongside Processing-in-Memory (PIM),
- 14:49sometimes called Computing-in-Memory (
- 14:51CIM). The concept is to integrate logic
- 14:53circuits and memory circuits on the
- 14:54same chip, performing calculations
- 14:56directly within the memory to solve the
- 14:58data bottleneck at its source. This
- 15:00approach is the complete opposite of
- 15:01the classic von Neumann architecture.
- 15:03The von Neumann system separates
- 15:05computing and storage, whereas PIM
- 15:07fuses the two together. Storage
- 15:08manufacturers have researched this for
- 15:10many years and proposed various
- 15:12implementation methods; for instance,
- 15:13Samsung added programmable compute
- 15:15units to HBM. However, actual adoption
- 15:17is very limited because it requires a
- 15:18complete redesign of the processor,
- 15:20software, and network systems to fully
- 15:22realize its potential. Furthermore, the
- 15:23current industry chain has established
- 15:25a structure of separate logic and
- 15:27memory manufacturing, each with massive
- 15:28scale and high efficiency, making the
- 15:30cost of a full-chain overhaul too high.
- 15:32Next is Storage Class Memory (SCM). It
- 15:34does not refer to a specific technology
- 15:36, but is a collective term for all
- 15:37technologies with performance between
- 15:39DRAM and NAND, roughly corresponding to
- 15:40the range from the bottom of the G2
- 15:42tier to the high end of the G3 tier in
- 15:43the architecture. One category includes
- 15:46emerging memory technologies with
- 15:48completely different principles, such
- 15:50as MRAM, PCM, and RRAM; the previously
- 15:53launched 3DXP belongs to PCM technology
- 15:55. The storage principles of these
- 15:57technologies are fundamentally
- 15:58different from DRAM and NAND, though
- 16:00their current mass production scale is
- 16:01not large. Another category is enhanced
- 16:04NAND, such as ZNS and XL-Flash. By
- 16:06pairing with DRAM or SRAM caches, or
- 16:08using SLC/MLC particles, the latency of
- 16:10NAND is brought closer to DRAM levels.
- 16:13Here, I will briefly explain the
- 16:14differences between NAND particles. SLC
- 16:16is Single-Level Cell, storing one bit
- 16:18per cell; it is the fastest with the
- 16:20longest lifespan, but has the smallest
- 16:21capacity and highest cost. MLC is
- 16:23Multi-Level Cell, storing two bits per
- 16:25cell, offering a balanced performance.
- 16:27TLC is Triple-Level Cell, storing three
- 16:29bits per cell, currently the mainstream
- 16:31, offering the best balance between
- 16:32capacity and cost. QLC is Quad-Level
- 16:34Cell, storing four bits per cell,
- 16:36providing the largest capacity and
- 16:37lowest cost, but with inferior speed
- 16:39and lifespan. Kioxia's XL-Flash uses
- 16:41SLC and MLC particles, and its FL6
- 16:43series SSD can currently achieve
- 16:45microsecond-level latency. The goal for
- 16:48next year is to reach the nanosecond
- 16:49level. Samsung's Z-NAND is made into
- 16:52Z-SSD products, focusing on ultra-low
- 16:54latency. Other manufacturers use pSLC
- 16:56mode, which treats TLC particles like
- 16:58SLC, sacrificing capacity for higher
- 17:00performance and lifespan. Since TLC is
- 17:03the most mass-produced and
- 17:04cost-effective process, using pSLC mode
- 17:06allows for enjoying economies of scale
- 17:08while obtaining performance close to
- 17:11SLC. Micron's XTR SSD is one such
- 17:13solution. Kioxia itself has a very
- 17:15clear product segmentation, with
- 17:17different NAND technologies
- 17:18corresponding to different AI inference
- 17:20scenarios: XL-Flash corresponds to the
- 17:22Storage-Next G 2.6 tier, TLC particles
- 17:24correspond to the CMx G 3.5 tier, and
- 17:26QLC particles correspond to G4-tier
- 17:28high-capacity storage. In the NAND
- 17:30field, there are two other noteworthy
- 17:31directions. One is High Bandwidth Flash
- 17:34(HBF), led by SanDisk and SK Hynix,
- 17:35aiming to use NAND to achieve bandwidth
- 17:38close to HBM while offering lower costs
- 17:40and higher capacity. If placed within
- 17:42the architectural hierarchy, it can be
- 17:43considered a Tier 1.5 level. NAND is
- 17:45inherently one to two tiers below DRAM,
- 17:47so placing it directly alongside HBM as
- 17:49a supplement involves a significant
- 17:50technical leap and a very high barrier
- 17:52to entry. The other is Z-NAND, a
- 17:54technology Samsung introduced for edge
- 17:56AI. It uses TLC particles to ensure
- 17:57large capacity, and the packaging is
- 17:59similar to HBM: first stacking NAND
- 18:01chips, then using advanced packaging to
- 18:03connect them tightly with the processor
- 18:05. The bottom two levels of the hardware
- 18:06hierarchy are HDDs and magnetic tape.
- 18:08HDDs are suitable for cold data storage
- 18:09—that is, data with large capacity,
- 18:11low cost, and low access frequency—
- 18:12corresponding to the bottom layer of
- 18:14the architecture. With the massive
- 18:15scale of KV caches now, non-critical
- 18:17data is gradually settling downwards,
- 18:18and demand for HDDs is also growing.
- 18:20Magnetic tape was originally used only
- 18:21for data archiving, sitting at the very
- 18:23bottom of the hardware hierarchy. But
- 18:24with the global explosion in data
- 18:26storage demand, the production capacity
- 18:28for both NAND and HDDs cannot keep up,
- 18:29so even the demand for tape has seen an
- 18:31increase. Finally, in this report,
- 18:33Bernstein provided ratings and target
- 18:35prices for major global storage
- 18:36companies. To clarify, in Bernstein's
- 18:38rating system, "Outperform" is defined
- 18:40as an expectation to beat the
- 18:41corresponding market index by more than
- 18:4315 percentage points over the next 12
- 18:45months. "Underperform" means lagging
- 18:47behind the index by more than 15
- 18:49percentage points. Specifically,
- 18:51Samsung Electronics is rated "
- 18:52Outperform" with a target price of
- 18:5440,000 KRW, SK Hynix is also "
- 18:56Outperform" with a target of 330,000
- 18:58KRW, and Micron is "Outperform" with a
- 19:00target of $ 130. Kioxia is rated "
- 19:02Underperform" with a target of 4,000
- 19:04JPY; SanDisk is "Outperform" with a
- 19:06target of $ 300; Seagate is "Outperform
- 19:08" with a target of $ 135; Western
- 19:10Digital is "Outperform" with a target
- 19:12of $ 77. A reminder here: the above are
- 19:14all Bernstein's forecast data, intended
- 19:16for industry research reference only,
- 19:17and do not constitute any investment
- 19:19advice. Overall, the technical battle
- 19:21in the NAND memory field is still
- 19:22ongoing, and new technical paths
- 19:24continue to emerge. From an
- 19:25architectural perspective, the entire
- 19:27storage system is shifting from
- 19:29CPU-centric to GPU-centric, with every
- 19:31layer being redefined. Looking at the
- 19:33hardware media, the boundaries between
- 19:34DRAM, NAND, and SRAM are becoming
- 19:36blurred, as they are all penetrating
- 19:38into the middle ground. For industry
- 19:43insiders, this is perhaps the most
- 19:45exciting time for the storage sector in
- 19:48the last 20 years; it is still too
- 19:50early to tell which path will
- 19:51ultimately emerge as the winner.
About this transcript
This page contains the full transcript of AI内存的技术混战 | 伯恩斯坦研究报告 | AI内存 | HBM | CXL | DRAM | NAND | KV缓存 | 存储器产业 | AI算力底层硬件 | 存储级内存 | 人工智能推理 by 最佳拍档, generated from the public captions YouTube serves with the video. The transcript has 3,817 words across 655 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.
What you can do with it
Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.
Free YouTube transcript tool
YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.