Chips, compute and data centres
CUDA
In short
CUDA is Nvidia's software platform for programming its GPUs to do general calculations, not just graphics. AI frameworks such as PyTorch use it to run on Nvidia chips, which ties much AI software to Nvidia hardware.
- Also called
- Compute Unified Device Architecture, CUDA Toolkit, Nvidia CUDA
- Topic
- Chips, compute and data centres
- Level
- Some background helps
- First described
- Nvidia introduced CUDA in 20062.
CUDA stands for Compute Unified Device Architecture. Nvidia calls it a parallel computing platform and programming model that harnesses the GPU to speed up computing12.
CUDA lets programmers use a GPU without going through graphics interfaces. They write ordinary C++, and now Python, that runs across thousands of GPU threads at once12.
The CUDA Toolkit bundles a compiler, a runtime library, debugging and tuning tools, and GPU-accelerated maths libraries3. Nvidia adds a further suite of AI, data science and maths libraries, called CUDA-X, that frameworks call into3.
Most AI developers never write CUDA directly. They use a framework such as PyTorch, where the torch.cuda module sets up and runs CUDA operations and places data on the chosen GPU4. But because so much tested code, tooling and expertise is built around CUDA, switching to another chip maker takes effort.
Rivals have their own platforms. AMD's is ROCm, which it calls an open software stack for GPU computing5. AMD's HIP tool can convert CUDA calls into HIP calls, though AMD says HIP is not a drop-in replacement and porting still needs manual work6.
Example
When a researcher in Cambridge trains a model in PyTorch on an Nvidia GPU, the line that moves the model "to cuda" is handing the work to Nvidia's CUDA platform4.
Why it matters
Because so much AI software is built on CUDA, it ties customers to Nvidia hardware. That software lock-in is a large part of why Nvidia is hard to dislodge, even when rival chips are fast.
Questions people ask
What is in the CUDA Toolkit?
A C and C++ compiler, a runtime library, debugging and optimisation tools, and GPU-accelerated libraries3. Code built with it runs on Nvidia GPUs.
Does CUDA work on AMD or Apple chips?
No. CUDA is Nvidia's platform for Nvidia GPUs1. AMD offers ROCm, and its HIP tools can translate CUDA code, though AMD says porting still needs manual work56.
Do I need to learn CUDA to use AI?
No. Frameworks like PyTorch use CUDA behind the scenes through their torch.cuda module4. Only engineers writing very fast custom code tend to write CUDA directly.