How OCR Works: Extract Text from Scanned PDFs

What is Optical Character Recognition (OCR)?

Optical Character Recognition is a computer vision technology that converts images of typed, handwritten, or printed text into editable, searchable machine data (such as ASCII or Unicode text characters).

Modern OCR engines (such as Google’s open-source Tesseract engine, which powers FileTools in-browser OCR via WebAssembly) do not simply match letters against static font templates. Instead, they use neural network feature extraction and line-finding algorithms to detect word baselines, segment character strokes, and evaluate lexical language models to predict the most probable word matches.

Adaptive Binarization: Otsu vs Sauvola Thresholding

Before character strokes can be identified, color or grayscale scanned pixels must be converted into binary (pure black text on pure white background). How this binarization occurs determines whether text is preserved or obliterated.

• Global Otsu Thresholding calculates a single luminosity cutoff for the entire page. While fast, if a document has uneven lighting (e.g. a phone shadow in the corner), dark areas turn into solid black blobs while light areas drop out completely.

• Local Adaptive Sauvola Thresholding calculates dynamic local means across rolling pixel windows (typically 15×15 to 30×30 pixels). It effectively isolates character strokes from background yellowing, watermarks, coffee stains, and uneven smartphone camera lighting.

How to Maximize OCR Accuracy: Scan Preparation & Resolution Math

The accuracy of OCR is governed by mathematical optical constraints. Follow these precision guidelines:

1. Target 300 DPI Resolution (x-Height Rule): Neural character classifiers require lowercase letters (such as "e", "x", "o") to have an x-height of at least 20 to 30 pixels. At 72 DPI, an 8pt footnote font has an x-height of only 4 pixels, making recognition impossible. At 300 DPI, the same letter has 24 pixels, ensuring over 99% character confidence.

2. De-Skew Within ±1.5 Degrees: Text line finders track horizontal baselines. A page skew exceeding 2 degrees disrupts inter-word bounding boxes, dropping recognition accuracy by up to 15–20%. Always straighten pages before running OCR.

3. Ensure Even Illumination: Avoid harsh overhead shadows. Shadows distort adaptive binarization windows, leading to broken character loops (turning "e" into "c" or "d" into "cl").

4. Select the Correct Primary Language Dictionary: OCR engines combine visual stroke classification with trie-based lexical language models. Selecting the correct language dictionary resolves ambiguous letter pairs like "rn" versus "m".

Page Segmentation Modes (PSM): Navigating Multi-Column Layouts

Scanned documents rarely consist of a single continuous block of text. Newspapers, magazines, academic papers, and invoices contain complex layouts with columns, sidebars, tables, and headers.

OCR engines use Page Segmentation Modes (PSM) to detect structural boundaries. If a two-column legal contract is analyzed under a single-block assumption, the engine reads horizontally across both columns, merging unrelated clauses into incoherent sentences. Advanced page layout analysis segments columns vertically first before tracking individual line baselines.

Current Technical Limitations of In-Browser OCR

While in-browser WebAssembly OCR is remarkably convenient and private, it has realistic technical limitations:

• Cursive & Unstructured Handwriting: While OCR excels at printed typography and computer fonts, handwritten doctor notes, cursive signatures, and informal script often yield low confidence scores.

• Complex Multi-Column Tabular Data: Highly nested financial tables without clear borders can occasionally be merged horizontally during plain text extraction. Review extracted tables carefully.

• Device Memory Constraints: Because OCR runs entirely inside your browser tab without cloud supercomputers, processing 100-page scanned documents simultaneously can take several minutes. For optimal speed, process scanned files in batches of 10 to 25 pages.