AI cluster

In short

An AI cluster is a large group of AI chips, usually thousands of GPUs or TPUs, wired together with fast networks so they can work as one machine. Big AI models are trained on clusters because no single chip is powerful enough.

Also called: GPU cluster, AI supercomputer, Training cluster
Topic: Chips, compute and data centres
Level: Some background helps
First described: Google's Jeff Dean and colleagues showed in 2012, with their DistBelief system, that clusters of thousands of machines could train neural networks with billions of parameters⁵.

A single GPU cannot hold or train a frontier model. So companies link thousands of them. Inside each server the chips talk over very fast links; between servers they use a dedicated network. Meta's Llama 3 servers each held eight GPUs connected by Nvidia's NVLink¹.

The network matters as much as the chips. In 2024 Meta described two clusters of 24,576 Nvidia H100 GPUs each, one using an Ethernet-based fabric called RoCE and the other Nvidia's InfiniBand, built for Llama 3 training². Google joins its TPUs into pods over its own inter-chip network³.

Storage and scheduling are part of it too. Meta's Llama 3 training drew on 240 petabytes of storage across 7,500 servers¹.

At this size, hardware fails all the time. During a 54-day stretch of Llama 3 405B training on up to 16,000 H100 GPUs, Meta logged 419 unexpected interruptions, about 78% of them from confirmed or suspected hardware faults¹. Clusters need software that saves progress often and restarts quickly.

"AI supercomputer" is often used for the same thing. The UK's Isambard-AI in Bristol, launched in July 2025, uses 5,448 Nvidia GH200 chips and cost £225 million⁴.

Example

Isambard-AI, at the National Composites Centre near Bristol, is part of the UK government's AI Research Resource and gives British researchers access to a cluster of thousands of Nvidia chips⁴.

Why it matters

The size of the cluster a lab can use sets the size of model it can train. Building one now costs billions and needs as much power as a small town, which is why compute is a matter of national policy.

Questions people ask

How many GPUs does it take to train a large AI model?

Thousands. Meta trained Llama 3 405B on up to 16,000 Nvidia H100 GPUs¹, and in 2024 it described clusters of 24,576 GPUs each².

What is the UK's biggest AI cluster?

At launch in July 2025, Isambard-AI in Bristol was described as the UK's most powerful AI supercomputer, with 5,448 Nvidia GH200 chips⁴.

Why do AI clusters break down so often?

With tens of thousands of parts, something is always failing. In Meta's Llama 3 run, faulty GPUs and GPU memory caused nearly half of all unexpected interruptions¹.

Related terms

Tools that use it

Meta Llama modelsMeta's family of downloadable large language models, from Llama 2 to the multimodal Llama 4

Sources

Back to the glossary