Diffusion model
In short
A diffusion model is a type of generative AI that creates images, video or other data by starting from random noise and removing it step by step until a clear result appears. Stable Diffusion is a well-known example.
- Also called
- Diffusion, Latent diffusion model, Denoising diffusion model, DDPM
- Topic
- How AI models work
- Level
- Some background helps
- First described
- Sohl-Dickstein, Weiss, Maheswaranathan and Ganguli, March 2015, in "Deep Unsupervised Learning using Nonequilibrium Thermodynamics", inspired by statistical physics1.
Diffusion models learn by destruction and repair. During training, structure in the data is slowly destroyed by adding noise, and the model learns the reverse process that restores it1. To create something new, it starts from pure noise and removes a little at a time. The 2020 paper from Jonathan Ho and colleagues at UC Berkeley that made the approach competitive used 1,000 steps2.
Latent diffusion, from Robin Rombach and colleagues in 2021, runs the denoising in the compressed latent space of an autoencoder rather than on full-size pixels, cutting the computing cost, and adds cross-attention so text can guide the result3. Stable Diffusion is built this way; its makers describe it as "a latent text-to-image diffusion model"4.
The High Court in London examined these models in Getty Images v Stability AI. The judgment describes Stable Diffusion starting from a random noise image, which the trained network, conditioned by the prompt, iteratively de-noises5. The judge accepted expert evidence that diffusion models learn the statistics of patterns in their training data rather than storing it, and held that the model weights were not an infringing copy5.
Diffusion is not limited to still pictures. OpenAI's DALL-E 2 used a diffusion decoder, and its authors found diffusion more efficient and higher quality than an autoregressive alternative for one stage6. Google DeepMind's experimental Gemini Diffusion applies the method to text7.
Example
In the Getty Images case, the experts agreed that Stable Diffusion could generate images showing a distorted version of the Getty watermark, because it had been trained on images carrying that watermark5.
Why it matters
Many AI image tools, including Stable Diffusion, rely on diffusion, so it shapes both what they can do and the copyright arguments around them.
Questions people ask
How does a diffusion model make an image from a prompt?
It starts with random noise and runs many denoising steps, 1,000 in the original DDPM setup2. At each step the network, guided by your prompt, removes some noise, until an image consistent with the prompt is left5.
Does Stable Diffusion store copies of its training images?
The High Court found in 2025 that its model weights do not store the copyright works it was trained on5. The experts agreed, though, that it can sometimes produce near-identical memorised images5.
Is a diffusion model the same as a GAN?
No. A GAN trains two competing networks, while a diffusion model learns to remove noise. They can be combined: the High Court noted that SD XL Turbo mixes aspects of both to make images quickly5.
Related terms
Tools that use it
Sources
- Sohl-Dickstein, Weiss, Maheswaranathan and Ganguli (2015), Deep Unsupervised Learning using Nonequilibrium Thermodynamics, arXiv:1503.03585
- Ho, Jain and Abbeel (2020), Denoising Diffusion Probabilistic Models, arXiv:2006.11239
- Rombach, Blattmann, Lorenz, Esser and Ommer (2021), High-Resolution Image Synthesis with Latent Diffusion Models, arXiv:2112.10752
- CompVis, Stable Diffusion repository README, GitHub
- Getty Images (US) Inc v Stability AI Ltd [2025] EWHC 2863 (Ch), paras 4, 6, 41, 145, 554-558 and 600
- Ramesh et al. (2022), Hierarchical Text-Conditional Image Generation with CLIP Latents (DALL-E 2), arXiv:2204.06125
- Google DeepMind, Gemini Diffusion model page