Deep learning

In short

Deep learning is a kind of machine learning that uses neural networks with many layers. Each layer turns the data into a slightly more abstract form, which lets the system learn complex patterns in images, speech and text by itself.

Also called: Deep neural networks
Topic: The basics
Level: Some background helps
First described: Geoffrey Hinton, Simon Osindero and Yee-Whye Teh, May 2006, "A Fast Learning Algorithm for Deep Belief Nets", which trained deep networks one layer at a time².

The "deep" refers to depth: the number of layers stacked in a neural network. A 2015 review by Yann LeCun, Yoshua Bengio and Geoffrey Hinton describes deep learning as letting models "composed of multiple processing layers" learn "representations of data with multiple levels of abstraction"¹.

Training works by showing the network examples and nudging its internal parameters. The backpropagation algorithm indicates how each parameter should change, layer by layer¹. Repeated over huge data sets, this lets the network discover intricate structure on its own¹.

The same review credits deep learning with large gains in speech recognition, visual object recognition, and fields such as drug discovery and genomics¹. Convolutional networks drove progress on images, video and audio, while recurrent networks handled sequences such as text and speech¹.

The modern wave took off in 2006, when Hinton, Simon Osindero and Yee-Whye Teh showed how to train deep networks one layer at a time². The Nobel committee says this work helped start a new explosion of results³. In 2024 the Nobel Prize in Physics went to John Hopfield and Hinton for discoveries that "enable machine learning with artificial neural networks"⁴.

Deep learning is not the same as human understanding, and its inner workings are hard to read. The UK government notes that trained AI systems infer patterns and connections that are "often not easily discernible to humans"⁵.

Example

Speech-to-text on your phone is a typical deep learning task: the 2015 review lists speech recognition among its biggest gains¹. Early layers pick up simple patterns, and later layers combine them into more abstract ones¹.

Why it matters

Deep learning is the technique behind today's chatbots, image generators and voice assistants, so most AI headlines are about it.

Questions people ask

What is the difference between machine learning and deep learning?

Deep learning is one kind of machine learning. It uses neural networks with many layers that learn their own representations of the data, rather than relying on simpler models¹.

Why is it called deep learning?

Because the neural networks have many stacked layers. Each layer turns the output of the one before into a more abstract representation¹.

Who invented deep learning?

Many researchers contributed. Geoffrey Hinton and colleagues' 2006 method for training deep networks layer by layer helped spark the modern wave²³, and Hinton shared the 2024 Nobel Prize in Physics with John Hopfield⁴.

Related terms

Sources

Back to the glossary