What is OCR, and when do you actually need it?

The one-sentence version

Optical character recognition reads an image of writing and outputs the characters it contains. Point it at a scan of a letter and you get back the letter's text as data — selectable, searchable, copyable, countable.

The abbreviation is worth spelling out because it explains the scope. Optical: it works from pixels. Character: it produces characters, not meaning. Recognition: it is a probabilistic guess, not a lookup. Every limitation follows from those three words.

How a machine reads a page

Modern OCR runs roughly four stages. First it cleans the image: straightening a page that went through the scanner at an angle, flattening uneven lighting, pushing grey paper towards white and grey ink towards black. This stage does more for accuracy than anything else, which is why a careful scan beats a clever algorithm.

Then it segments — finding the blocks of text, the lines within them, and the individual glyph shapes. Then it classifies each shape, historically by comparing against known letterforms, now almost always with a neural network trained on millions of examples.

Finally it applies language knowledge. This is the stage people underestimate. If the classifier is unsure between "rn" and "m", a model that knows English will pick whichever produces a real word in context. That is why OCR on a normal English paragraph is far more accurate than OCR on a list of serial numbers, where there is no context to lean on.

Where it reliably fails

Knowing the failure modes is more useful than knowing the success rate, because they are predictable.

• Low resolution. Below roughly 200 DPI the glyph shapes stop being distinguishable. 300 DPI is the usual recommendation and it is a real threshold, not a nicety.
• Handwriting. Recognising handwriting is a genuinely different and much harder problem than recognising print. General-purpose OCR handles it badly.
• Strings with no linguistic context. Serial numbers, licence keys and account numbers lose the language model's help, so confusable pairs — 0 and O, 1 and l, 5 and S, 8 and B — stay confused.
• Tables and multi-column layouts. Recognising the characters is the easy part; reconstructing which cell they belonged to is where output turns to soup.
• Low contrast, skew, and JPEG artefacts. Compression noise around glyph edges is read as part of the glyph.

Does your document even need it?

There is a two-second test that settles it. Open the PDF and try to select a line of text with your cursor. If the text highlights, there is real character data in the file and OCR would add nothing. If your cursor draws a selection box over what is clearly an image, the file is a scan and has no text layer at all.

The same test explains a common frustration: searching a PDF and finding nothing, on a document where you can plainly see the word. Search looks at text, and there is none — only a picture of it.

It is also why a scanned PDF is so much larger than a text one. A page of characters is a few kilobytes. A photograph of that page is hundreds.

Getting a better result

Almost all of the accuracy is decided before recognition starts. Scan at 300 DPI or higher, in greyscale rather than colour for plain documents, with the page flat and evenly lit. If you are photographing rather than scanning, get the page square to the camera and avoid a shadow falling across it — perspective distortion and uneven lighting cost more accuracy than a slightly soft focus.

Then expect to proofread anything that matters. OCR output at 98% character accuracy still contains roughly one error per two lines, and the errors cluster exactly where you would least like them: in the numbers.

Frequently asked questions