Natural language processing
In short
Natural language processing (NLP) is the branch of AI that lets computers work with human language, written or spoken. It powers translation apps, spam filters, voice assistants, search engines and chatbots.
- Also called
- NLP, Computational linguistics, Language technology
- Topic
- The basics
- Level
- Beginner
- First described
- Warren Weaver's July 1949 memorandum "Translation" set out the goals and methods of machine translation and directly stimulated the first research1.
"Natural" language means the languages people speak, like English or Welsh, as opposed to programming languages. NLP covers reading, understanding, translating and generating them.
The field began with translation. In July 1949 Warren Weaver of the Rockefeller Foundation circulated a memo called "Translation" to some 30 colleagues, proposing that the new digital computers could translate between languages1. The machine translation historian John Hutchins calls it probably the single most influential publication in the early days of machine translation1. It drew on earlier punched-card experiments in England by Richard Richens and Andrew Booth1.
Early systems used hand-written rules, and later ones statistics. In 2016 Google reported that its new neural network translation system cut errors by an average of 60% compared with its older phrase-based system, in tests judged by people2.
In 2018 Google researchers released BERT, a transformer model pre-trained on unlabelled text that set new records on eleven NLP tasks3. A year later Google said BERT would help Search understand about one in ten English searches in the US4.
Today's large language models are the latest stage of NLP. A single model can now translate, summarise, answer questions and write, jobs that once each needed a separate system.
Example
When you type "2019 brazil traveler to usa need a visa", NLP is what lets a search engine grasp that the traveller is from Brazil going to the US, not the other way round. Google used exactly this query to show what BERT changed4.
Why it matters
Almost every time you talk to, type to or get text from a computer, NLP is involved. Chatbots are simply its most visible form.
Questions people ask
Is NLP the same as a large language model?
No. NLP is the whole field of computers working with human language, dating back to 19491. Large language models are one recent, very capable type of NLP system.
What are examples of natural language processing?
Machine translation such as Google Translate2, search engines that understand the meaning of a query4, spam filters, voice assistants, predictive text and AI chatbots.
How does natural language processing work?
Modern NLP splits text into small pieces called tokens, turns them into numbers and uses neural networks trained on huge amounts of text to predict or classify. Google's translation system, for instance, breaks rare words into sub-word units it calls wordpieces2.
Related terms
Tools that use it
- Google TranslateGoogle's free translator for text, signs, menus and spoken conversations in more than 240 languages.
- DeepLA German translator known for natural-sounding text in European languages, with camera and voice modes in its app.
- GrammarlyChecks spelling, grammar and tone in almost every app, on a phone keyboard or a computer.
Sources
- Hutchins, J. (1999), Warren Weaver Memorandum, July 1949, MT News International no. 22, pp. 5-6, 15
- Wu et al. (2016), Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation, arXiv:1609.08144
- Devlin et al. (2018), BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv:1810.04805
- Nayak, P. (2019), Understanding searches better than ever before, Google blog