Chips, compute and data centres
High-bandwidth memory
In short
High-bandwidth memory (HBM) is a type of computer memory made by stacking memory chips on top of each other right next to the processor. It feeds data to AI chips much faster than ordinary memory.
- Also called
- HBM, HBM3E, HBM4
- Topic
- Chips, compute and data centres
- Level
- Some background helps
- First described
- SK hynix says it developed the industry's first HBM in 20132.
AI chips can do maths faster than ordinary memory can supply the numbers. HBM tackles that bottleneck. SK hynix, one of its main makers, describes it as memory that vertically connects multiple DRAM chips and greatly increases data processing speed compared with conventional DRAM1.
The chips in a stack are linked by through-silicon vias: thousands of tiny holes drilled through each chip and filled with connections2. A typical DRAM chip has eight data paths; HBM has 1,024, and its footprint is more than ten times smaller than standard DRAM2. The stack sits in the same package as the processor, so signals travel a very short distance.
There have been six generations: HBM, HBM2, HBM2E, HBM3, HBM3E and HBM41. HBM4 doubles the number of connections to 2,048, and SK hynix says it doubles bandwidth over the previous generation1.
Data centre AI chips depend on it. Nvidia's DGX B200 server holds 1,440 GB of HBM3e across its eight GPUs, with 64 TB/s of total memory bandwidth3. In December 2024 the US added new export controls on certain HBM used for advanced computing4.
HBM is not the same as the memory in your laptop. SK hynix calls it a high-value, high-performance product1, and it is packaged permanently beside the chip it serves rather than plugged into a slot2.
Example
Meta's Llama 3 paper lists faulty GPU HBM3 memory as the cause of 72 unexpected training interruptions, about 17% of the total, during a 54-day stretch of training5.
Why it matters
For large AI models, memory speed often limits performance more than raw processing power. That has turned a handful of memory makers into key players in the AI supply chain.
Questions people ask
Why do AI chips need high-bandwidth memory?
Because a model's billions of numbers must be read from memory for every calculation. Ordinary memory has too few data paths. HBM has 1,024 per stack, so it can keep the processor busy2.
Who makes HBM?
SK hynix is a leading maker and says it developed the first HBM in 20132. Each generation follows a standard set by the industry body JEDEC, which the makers then try to beat on speed1.
What is HBM4?
The sixth generation of HBM. It uses 2,048 connections, double the previous generation, to double bandwidth. SK hynix said in September 2025 it was ready for mass production1.
Related terms
Sources
- SK hynix (12 September 2025), SK hynix Completes World's First HBM4 Development and Readies Mass Production
- SK hynix (2 May 2023), Become a Semiconductor Expert with SK hynix: HBM
- Nvidia, DGX B200 product specifications
- US Bureau of Industry and Security, Foreign-Produced Direct Product Rule Additions, and Refinements to Controls for Advanced Computing and Semiconductor Manufacturing Items, 89 FR 96790
- Grattafiori et al. (2024), The Llama 3 Herd of Models, arXiv:2407.21783, table 5