# OCR Lab

> Read the text out of a screenshot, scan or photo **without uploading it**. Detection
> and recognition both run in the browser tab on Baidu's PP-OCRv6 models through
> onnxruntime-web compiled to WebAssembly. Per-line confidence, bounding boxes, a
> diagnostics panel and every export are free and work signed out. An optional metered
> pass repairs reading order and extracts structured fields, and is cross-checked
> against what the recogniser actually produced.

Live at https://ocr-lab.skillsafe.ai/ · API docs at /api.html

## The point of it

There are a dozen web OCR tools and most of them are uploads. That is the whole reason
this one exists: if the screenshot has an invoice number, a customer address, a patient
name or a client dashboard in it, putting it on a stranger's server is the thing the
user is not allowed to do. Here the image never leaves the tab — there is no upload step
and no server anywhere in the recognition path, and once the models are cached the page
works offline.

The second reason is honesty about quality. A tiny model on a noisy UI screenshot is
genuinely mixed, so the app measures it instead of hiding it:

- Every line carries the recogniser's **mean per-character probability**.
- There is a filter for **only the low-confidence lines**, so triage is one click.
- A **diagnostics panel** reports the probability-map range, non-finite pixels, whether
  the recognition head was already softmaxed, and where the binarisation threshold came
  from. No consumer would open it; it is there so the result can be argued with.
- The bundled sample is a **real capture with real errors in it**, not a flattering demo.

## The models

- **PP-OCRv6_tiny** detection and recognition, from
  [PaddlePaddle/PaddleOCR](https://github.com/PaddlePaddle/PaddleOCR), **Apache-2.0**.
- **Redistributed unmodified**, with the upstream `LICENSE-Apache-2.0.txt` and
  `NOTICE.txt` shipped alongside the weights in `/models/`.
- About 6 MB total. They ship with the app, load once when the page opens, and are then
  content-addressed by SHA-256 in Cache Storage, so a second visit downloads nothing.
- You may supply **your own fine-tuned ONNX package** instead, and keep it in your own
  SkillSafe storage.
- The runtime is single-threaded WebAssembly with SIMD, deliberately: the app pins
  `numThreads = 1` and `proxy = false` because the Content-Security-Policy does not allow
  a `blob:` worker, and it hands the `.wasm` in as `wasmBinary` so the runtime never has
  to fetch it. A browser without WebAssembly SIMD cannot run the engine, and the app says
  so in those words rather than blaming the model files.

## What is free, and what is metered

**Free, no account, works signed out:** loading the engine, reading an image, detection
and recognition, per-line confidence and boxes, the box overlay, the low-confidence
filter, the diagnostics panel, the bundled sample, a replayable saved example, and every
export — plain text, CSV (one row per line with confidence, detection score and box),
JSON of the whole run including diagnostics, and Markdown.

**Metered (`gpt-terra`, needs a signed-in account with credits):** the repair pass. It
takes one `task: "clean"` input carrying the recognised lines and returns a document
rather than a list — reading order fixed, flagged character confusions corrected, and,
on the `fields` target, structured data which the page will also flatten to CSV for a
spreadsheet. Targets are `text`, `markdown`, `fields` and `translate`.

There is **no image lane**: the model is never sent a page image, only the lines the
local recogniser produced.

## The accountability contract

This is the part worth quoting. The free lane exists to hold the paid lane to account:

- Every character the model emits that the recogniser never produced is **counted and
  shown**.
- Any recognised line with **no trace** in the model's output is listed back to the user.
- `low_confidence_lines` is passed to the model explicitly, and the prompt tells it to
  repair those from context only where the context genuinely determines the answer, and
  otherwise to leave them alone and say so.

So a repair pass cannot quietly turn a smudged total into a plausible one without that
disagreement appearing on screen.

## Input clipping

A long page is clipped to fit the input budget, and the clipping is not a head slice: the
lowest-confidence lines are kept first, then the head and tail are walked inward, and each
removed span leaves a single `[...]` elision marker. The counts are reported both to the
user in-band and to the model as `clip_note`, and the reconciliation step skips elided
lines so a clipped page cannot look like a dropped one.

## Persistence

Scan history is a **declared collection** (`scans`) on the user's SkillSafe account, with
`localStorage` as a mirror for instant paint and for guests. Guest scans are migrated up
on sign-in. Documents are size-capped with graceful degradation — box geometry is dropped
first, then line detail, then the text is truncated — and records are read back out of
`rec.doc`. Semantic search over past scans uses the collection's `similar()` query,
embedding the title and a text preview, so you can find a scan by what it said rather
than by what the file was called.

Only titles, previews, text, line data and diagnostics are stored. **Images are never
uploaded** — not to the collection either.

## Known limits

- Accuracy is PP-OCRv6_tiny's, not a commercial engine's. Dense small text, heavy skew
  and low contrast all degrade it; the confidence numbers are there to tell you when.
- **There is no searchable-PDF export yet.** Everything needed for one is present (the
  source image, per-line text and pixel-space boxes), but it is not built.
- Recognised lines are **not editable in place**, so a wrong line has to be fixed in
  whatever you paste it into.
- **One image at a time** — there is no batch queue.
- Stopping a metered run stops the page waiting for it; the run has already been sent, so
  it still completes and is still charged. The button says so.

## Not affiliated

Not affiliated with or endorsed by Baidu or the PaddlePaddle/PaddleOCR project. PP-OCRv6
is their work, used under Apache-2.0.