Precision and recall
In short
Precision and recall measure a search result. Precision is the share of retrieved items that are relevant, and recall is the share of all relevant items that the system retrieved.
- Also called
- precision, recall, precision-recall
- Topic
- Testing and benchmarks
- Level
- Some background helps
In the TREC evaluation used by NIST, precision is the proportion of retrieved documents that are relevant, and recall is the proportion of relevant documents that are retrieved1. Manning, Raghavan and Schütze write the same ratios from a counts table: precision is true positives divided by true positives plus false positives, and recall is true positives divided by true positives plus false negatives2. They note that a retrieval system can be treated as a two-class classifier, labelling documents relevant or not2.
Retrieving more documents usually raises recall and lowers precision1. Recall does not fall as more documents are retrieved, and returning every document forces recall to 1 at a very low precision2. Manning, Raghavan and Schütze say a web searcher often wants high precision on the first page, while a paralegal or an intelligence analyst may accept low precision in order to raise recall2.
The balanced F measure, called F1, is the harmonic mean of precision and recall, equal to 2PR divided by P plus R2. When the two numbers differ, the harmonic mean stays closer to the lower one than an ordinary average does, so a high F1 needs both to be high2.
Example
Manning, Raghavan and Schütze ask about a system that returns 8 relevant documents and 10 that are not, when 20 documents in the collection are relevant. By the ratios above, precision is 8/18 and recall is 8/202.
Why it matters
Precision says how much of what came back was relevant, and recall says how much of the relevant material was found1. The two often move apart, so one of them can look strong while the other is weak12.
Questions people ask
What is the difference between precision and recall?
Precision is the fraction of retrieved items that are relevant. Recall is the fraction of all relevant items that the system retrieved1.
Why do precision and recall often move in opposite directions?
Retrieving more documents usually adds irrelevant ones, so precision falls, while recall rises because more of the relevant documents are included1. Returning every document gives recall of 1 and very low precision2.
What is the F1 score?
F1 is the balanced F measure, the harmonic mean of precision and recall. It equals 2 times precision times recall, divided by precision plus recall. When the two differ, it stays nearer the lower one2.