Chips, compute and data centres
NPU
In short
An NPU, or neural processing unit, is a small part of a phone or laptop chip built only to run AI tasks. It handles things like live translation and image generation using far less battery than the main processor.
- Also called
- Neural processing unit, Neural Engine, AI engine
- Topic
- Chips, compute and data centres
- Level
- Beginner
- First described
- Apple put a "neural engine" into its A11 Bionic chip for the iPhone X, announced on 12 September 20173.
Modern laptop and phone chips combine several kinds of processor: a CPU for general work, a GPU for graphics, and on newer designs an NPU for AI. Microsoft describes the NPU as a specialised chip for AI-heavy jobs such as real-time translation and image generation, working alongside the CPU and GPU1.
The point is efficiency. Microsoft says an NPU processes large amounts of data in parallel and uses energy on AI tasks more efficiently than a CPU or GPU, which means longer battery life1. The operating system decides which processor gets which job1.
NPU speed is usually quoted in TOPS, trillions of operations per second. Microsoft's Copilot+ PC label requires an NPU that can manage more than 40 TOPS1. Apple calls its version the Neural Engine. The one in its M4 chip, announced in May 2024, does up to 38 trillion operations per second2.
There is a catch for developers. Many NPUs only do integer maths in low-precision formats such as INT8, so a model usually has to be shrunk, or quantised, before it will run on one1.
An NPU is not a replacement for the big data centre chips. It is built to run smaller models on your own device, working alongside the CPU and GPU1.
Example
Face ID on an iPhone was one of the first jobs given to Apple's Neural Engine. Apple said in 2017 that its dual-core design did up to 600 billion operations per second3.
Why it matters
NPUs let AI features run on your device without sending data to the cloud. That can mean faster responses, better battery life and more privacy.
Questions people ask
Do I need an NPU?
Only for some features. Many Windows AI features require an NPU of 40 TOPS or more, which is what a Copilot+ PC has1. Most AI chatbots run in the cloud and work on any device.
Is an NPU the same as a GPU?
No. Both do many calculations in parallel, but an NPU is built specifically for AI maths and uses less energy for it. The GPU still handles graphics, and the two work alongside the CPU1.