Skip to content

Roadmap

Versioned, contributor-facing. naina ships vertically — each release is a fully usable surface, not a half-built layer.

History. v0.1 was a face and person understanding runtime. It produced the engine naina still runs on: the C ABI, the backend abstraction, and the manifest-driven model loader. v0.2 repurposed that engine for document reading, which is what the name always suited — naina means eyes. The face modules are preserved on the face-stack branch. See the design spec for why.

v0.1 — Engine (shipped)

C ABI, backend abstraction, tensor and arena types, model registry with sha256-verified downloads, Python and Node bindings, CI matrix.

  • [x] core/include/naina/{naina.h,naina.hpp,backend.hpp,tensor.hpp,model_loader.hpp}
  • [x] IBackend / ISession for ONNX Runtime and NCNN, runtime probe + fallback
  • [x] model_loader — manifest parse, HTTP download, sha256 verify, cache
  • [x] Python binding (pybind11 + scikit-build-core)
  • [x] Node binding (cmake-js + N-API, inference off the event loop)
  • [x] CI: Linux gcc + clang, macOS arm64

v0.2 — Text spotting (shipped)

Detect text, recognise it, return lines with geometry and confidences, from Python and Node.

  • [x] Retier the registry by device (tiny / small / medium) instead of licence
  • [x] Registry: PP-OCRv6 det + rec at three tiers, every sha256 verified
  • [x] Mirror all weights into naina's own release; upstream URLs kept as provenance
  • [x] image_ops — detection resize geometry, quad rectification to 48px strips
  • [x] geometry — convex hull, minimum-area rectangle, convex polygon offset
  • [x] db_postprocess — binarize, 8-connected blob borders, box scoring, DBNet decode
  • [x] charset — parse PostProcess.character_dict, per-tier class counts
  • [x] ctc_decode — greedy decode, repeat collapse, blank handling, confidence
  • [x] text_detect + text_recognize module wiring
  • [x] page — pointer-stable storage, markdown + JSON serialisation
  • [x] naina_read end to end, verified against real weights
  • [x] Python + Node OCR surface
  • [x] No OpenCV, no pyclipper, no PaddlePaddle dependency

Known gaps carried into v0.3:

  • FindNCNN.cmake does not locate a Homebrew NCNN install, so only the ONNX Runtime backend is exercised in practice.
  • Recognition runs one strip per session call. Correct, but batching needs a uniform width per batch and would be meaningfully faster on dense pages.

v0.3 — Layout and markdown (mostly shipped)

Turn a bag of lines into a document.

  • [x] tools/paddle2onnx_layout.py — exports PP-DocLayout-S/-M, which PaddleOCR ships only in Paddle format. Byte-deterministic, and verified per-column against the Paddle original (classes exact, scores to 5e-7, box coordinates to 3e-4 px). This was the design's biggest open risk: without it layout would exist only at the 269 MB tier and the 11 MB tier could not describe structure at all.
  • [x] Layout at every tier — 11.1 / 54.5 / 268.0 MB totals
  • [x] layout_detect — region boxes with class labels from each model's own 23-entry label_list, inputs fed BY NAME since the variants disagree on signature
  • [x] doc_assemble — line-to-region assignment, column-aware reading order, structured markdown. Pure logic, 16 tests from hand-built inputs.
  • [x] MCP server (see v1.0 list)
  • [x] Medium tier verified end to end on an A4 academic page: 14 of 14 regions detected and correctly labelled, reading order correct, 33 of 33 lines recognised at 0.99–1.00, markdown with the right heading hierarchy and the running head omitted as furniture.
  • [ ] Golden corpus: committed images with expected markdown
  • [ ] Recognition batching by padded width
  • [ ] Fix FindNCNN.cmake
  • [ ] Cross-class NMS. PaddleDetection runs NMS per class, so one box comes back under several labels and naina keeps every row over threshold independently. Measured: a running head returned text at 0.677 and header at 0.481 for the identical box. Whenever two labels for one box both clear the threshold, that box yields two regions — and the higher-scoring label is not always the right one. Needs a dedup pass keeping the best-scoring class per box group.
  • [ ] Improve layout recall on out-of-distribution pages. PP-DocLayout is trained on papers and reports and is excellent there (see above), but degrades sharply outside that shape. On a synthetic wide-spaced report page it labelled a body paragraph doc_title and both section headings text, producing structurally wrong markdown; small tier at 480x480 found only 4 of 7 regions on the same page. The correct label was usually present in the raw rows at a lower score, which suggests cross-class NMS above may recover some of this on its own — do that first, then reassess the threshold.

  • [x] Devanagari support. Hindi, Marathi, Nepali and Sanskrit, via a language axis orthogonal to tier. Verified end to end: the page that previously returned 3rarearanlus Tarafaaa: f: at 0.758 confidence now returns अयोध्याकाण्डे नवनवतितम: सग्गः, 129 lines at 0.90–0.99 native and 2731 Devanagari characters in a real browser.

    Detection and layout are shared, not duplicated — they are script-agnostic, which is why this was a registry change rather than engine work. config.version 3 appends a language string; existing offsets are unchanged (40 → 48 bytes) so v1 and v2 configs still work, asserted by test. An unknown language returns NAINA_E_UNSUPPORTED rather than falling back to Latin.

  • [x] Ten alphabets. Default (Latin, Chinese, Japanese) plus arabic, cyrillic, devanagari, el, eslav, korean, ta, te, th — every script upstream ships as ONNX. Each was verified against naina's recognition path before being added ([N,3,48,W], CTCLabelDecode, num_classes == dict + 2); all nine matched, none needed a code change. Mirrored into naina's release and sha256-verified by re-download. Registry entries are generated from the files rather than hand-written, so a hash cannot be mistyped. Verified in a real browser: Greek returned Ελληνικά κείμενο 2026 and Cyrillic Русский текст 2026, both exact.

  • [x] Script auto-detection in the web app. Reads with the default, and if mean confidence is under 0.95 re-reads a 900px copy with each other alphabet, keeping the best if it clears the default by 0.03. Verified on four scripts: Latin correctly not switched (and free — the gate never trips, 3.6s), Hindi → devanagari, Greek → el, Cyrillic → cyrillic.

    The thresholds come from measurement, not guesswork. Best-alphabet margins over the default: Hindi +0.426, Cyrillic +0.104, Greek +0.066, Latin +0.006 (where arabic scored highest, since every alphabet contains Latin — which is why a plain argmax is wrong and a margin is required). 0.03 has 2x headroom under the tightest true positive and 5x over the Latin tie. An early exit at +0.25 cut the Hindi case from 130s to 28s.

    Found and fixed a real WASM binding bug in the process: the JS bridge was installed on globalThis, so a second Reader overwrote the first one's and an older Reader read tensor descriptors from the wrong WASM heap. It is now per-module.

  • [ ] Auto-detection in the libraries. Needs every alphabet's weights present (~70 MB), which defeats an 11 MB tier, and needs the context's session and charset caches re-keyed by language. Probably belongs as an opt-in taking an explicit candidate list rather than as a default.

v0.4 — Browser (binding shipped)

  • [x] bindings/wasm via Emscripten. 143 KB brotli for naina.wasm plus 24 KB for the JS glue, against a 5 MB budget — the no-OpenCV rule is what makes that possible, since OpenCV alone would have exceeded it.
  • [x] wasmjs_backend — the only thing crossing into JS is ISession::run, i.e. "execute this graph on these tensors". All of naina's own arithmetic (preprocessing, DB decode, rectification, CTC, layout, doc_assemble) runs the same C++ compiled to WASM. Uses ASYNCIFY because ort-web's run() is a Promise while ISession::run is synchronous; the alternative (SharedArrayBuffer + Atomics.wait) needs COOP/COEP headers that GitHub Pages cannot set.
  • [x] Model staging: JS fetches through the Cache API, writes into Emscripten's virtual FS, and the shared C++ core sha256-verifies every file — so the browser gets the same integrity check as every other platform, not a weaker one. The staging plan (URLs and cache paths) is computed by C++ via stagingPlan(), so the cache layout has one definition.
  • [x] Verified reading a real A4 page: 33 lines, mean confidence 0.99, correct #/## structure, deterministic across runs.
  • [x] Production web app at jvoltci.github.io/naina/ — client-side only, no upload, no account, no page limit. PDFs via pdf.js, multi-page batches, OCR in a Web Worker so the tab never freezes, offline after first visit. Verified end to end in real Chrome via Playwright (app/test/e2e.mjs), which is the only test that can reach OffscreenCanvas, createImageBitmap, module workers and ASYNCIFY-in-a-worker.
  • [x] mkdocs-material documentation at jvoltci.github.io/naina/doc/
  • [x] Weights are served same-origin, not from the GitHub release. Release assets 302 to release-assets.githubusercontent.com and neither hop sends Access-Control-Allow-Origin, so a browser cannot fetch them at all — measured against the live release. Every other binding is unaffected. The deploy stages tiny + small (65.9 MB) into the Pages artifact, reading the file list from the core via stagingPlan() so it cannot drift. sha256 verification still happens in the C++ core.
  • [ ] WebGPU execution provider. Currently off by default because it silently breaks layout, which is worse than crashing. Chrome 141 on an M3 initialises ORT's JSEP provider and then fails a MatMul kernel; ORT recovers node by node, so text still returned 33 lines at 0.99 confidence and looked correct, while layout regions went from 9 to 0 and the markdown lost all structure. Needs real numbers before it becomes a default.
  • [ ] Medium tier in the browser. ppdoclayout_l.onnx is 129 MB and GitHub Pages caps a single file at 100 MB, so the app offers tiny and small only.

v0.5 — Rust (built)

  • [x] bindings/rust over the C ABI. Safe wrapper with the unsafe surface confined to src/ffi.rs and the marshalling in read_rgb. 13 tests pass against a real libnaina, including reading a page end to end and asserting naina_config is 48 bytes with 8-byte alignment — matching the C compiler, because a mismatch there would write tier into the wrong slot and silently load a different model.
  • [x] Reader is Send but not Sync: the native context is not internally synchronised and concurrent reads would race on its session cache.
  • [x] Vendors the C++ core for a self-contained crate (vendor.sh), but deliberately not ONNX Runtime — ~17 MB per platform with its own floors, and four bundled copies would inherit all of that silently.
  • [ ] Published to crates.io. Needs cargo publish with a token.
  • [ ] Golden corpus passes from Rust

v1.0 — Guarantees

  • [ ] Cross-binding parity enforced in CI. Python, Node and Rust must produce byte-identical output for a fixed (backend, device, tier) on the golden corpus — they run the same compiled core against the same kernels, so anything less is a bug.

    WASM is explicitly outside that set, and this is measured, not assumed. onnxruntime-web is a different build of ONNX Runtime — WASM SIMD kernels rather than native NEON/AVX — so its probability maps differ in the last few float bits. On the A4 fixture at tiny tier that moved one marginal blob across DBNet's 0.3 binarize threshold: native macOS arm64 produced 35 text lines, WASM 33, with 33 character-identical. Because a split fragment takes its own reading-order slot, word order can shift with it too. An earlier draft of this file listed WASM in the byte-identical group; that was wrong. What WASM does guarantee is determinism within itself (same input, same output — asserted in bindings/wasm/test/read.test.mjs) and the same algorithms, since it runs the same C++. - [ ] Full benchmark matrix: accuracy and latency per device, harness in-repo, reproducible from a clean clone - [x] MCP server — mcp/, two tools (read_document, read_document_detailed), verified end to end over stdio - [ ] Move the MCP server to spec revision 2026-07-28, whose stateless request/response model suits naina exactly. Blocked on the SDK: @modelcontextprotocol/sdk@1.30.0 tops out at 2025-11-25, and a client asking for 2026-07-28 negotiates down. The server already holds no session state, so this should be a dependency bump, not a rewrite. - [ ] Prebuilt binaries: macOS arm64/x64, Linux x64/arm64, Windows x64, cp39–cp313 - [ ] Vendored ONNX Runtime in published wheels (delocate / auditwheel)

Non-goals

  • Training or fine-tuning. naina is inference only.
  • Autoregressive VLM parsing (PaddleOCR-VL, DeepSeek-OCR). These need a tokenizer, KV cache and sampling loop — a different engine, not a module. Revisit only as an explicit v2 decision, never as drift.
  • Handwriting. PP-OCRv6 is weak at it; claiming support would be dishonest.
  • Chart and formula semantics. Regions get detected and labelled; their contents are not interpreted.
  • Face and person understanding. Parked on face-stack, not deleted.
  • Vector stores, dashboards, UI frameworks.
  • Crime prediction, risk scoring, government-ID matching.