Chips, compute and data centres
Interconnect
In short
An interconnect is the link and communication system that lets chips, processors or computers exchange data. In AI hardware, it moves model data between GPUs quickly enough for several processors to work on one job1.
- Also called
- Chip interconnect, GPU interconnect, High-speed interconnect
- Topic
- Chips, compute and data centres
- Level
- Some background helps
An interconnect is the path between computing components. It can be a connection inside a chip, between chips on a board, between GPUs in one server, or between servers in a cluster. Its important properties include bandwidth, latency and how many devices it can connect1.
AI workloads move large amounts of model weights, activations and training updates. A slow link can leave expensive processors waiting for data, so system designers use different networks for different distances. NVIDIA describes NVLink as a high-speed, low-latency link for GPU-to-GPU and GPU-to-CPU communication, while InfiniBand is used between racks and Ethernet handles storage, management and outside connections1.
An interconnect is not a processor and does not make a model more intelligent by itself. It is closer to a road system for data. NVLink can provide shared memory access across GPUs and NVSwitch can connect many GPU links into a fabric, but the result still depends on the software and workload using it1.
You meet interconnects in AI servers, supercomputers and multi-GPU workstations. NVIDIA’s first Pascal NVLink design connected GPUs to peer GPUs or compatible CPUs, and its P100 implementation combined several links for higher theoretical bandwidth2.
Example
A training job split across eight GPUs may need to exchange partial results after each step. The interconnect carries those updates between the GPUs; a faster link can reduce the time they spend waiting1.
Why it matters
AI systems are often limited by moving data, not only by doing calculations. The interconnect helps determine whether several accelerators behave like a useful system or like isolated chips.
Questions people ask
Is an interconnect the same as a network?
They overlap, but interconnect usually means the link or fabric joining computing components. A data-centre network may also carry storage, management and user traffic, while an AI interconnect may be reserved for GPU communication1.
What is a GPU interconnect?
It is a high-speed connection that lets GPUs exchange data directly or through switches. NVIDIA NVLink is one example, designed to improve GPU-to-GPU and GPU-to-CPU communication1.
Is PCIe an interconnect?
Yes. PCIe is a common computer interconnect. NVIDIA describes NVLink as a way to overcome PCIe bandwidth limits in multi-GPU and GPU-to-CPU systems1.