naina: Architecture

A high-performance, embeddable document-reading runtime. C++ core, thin bindings, runs on edge → server, one API.

naina began as a face and person understanding runtime. The engine described below is that work; v0.2 repurposed it for OCR, which the name always suited (naina means eyes). The face modules live on the face-stack branch.

North star

• Plug-and-play: one C ABI, bindings everywhere (Py, Node, Rust, Swift, Kotlin, WASM).
• Edge-first: must run on a Raspberry Pi 5 and a Jetson Nano. Server is a bonus tier.
• SOTA, swappable: model weights are not hardcoded. New SOTA paper drops → new manifest entry, no code change.
• Honest benchmarks: every model in the registry ships with reproducible accuracy + latency numbers per target. README shows both default (commercial-OK) and research (max accuracy) columns.
• Open source first-class: Apache-2.0 code, permissive default weights, opt-in research weights. Adoption > bragging rights, but we get both.

Layered design

┌─────────────────────────────────────────────────────────────┐
│                    Language Bindings                         │
│   Python │ Node │ Rust │ Swift │ Kotlin │ C/C++ │ WASM      │
└────────────────────────┬────────────────────────────────────┘
                         │  Stable C ABI  (naina.h)
┌────────────────────────▼────────────────────────────────────┐
│                    naina-core  (C++20)                       │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  Pipeline  (zero-copy, async, batched DAG)           │   │
│  │  Source → Decode → Preprocess → Infer → Postprocess  │   │
│  └──────────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  Task modules  (independently shippable)             │   │
│  │  TextDetect · TextRectify · TextRecognize            │   │
│  │  LayoutDetect · DocAssemble                          │   │
│  └──────────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  Backend abstraction  (IBackend / ISession)          │   │
│  │  ┌────────┬─────────┬─────┬─────┬───────┬──────────┐ │   │
│  │  │ ONNXrt │ OpenVINO│ NCNN│ MNN │CoreML │ TensorRT │ │   │
│  │  └────────┴─────────┴─────┴─────┴───────┴──────────┘ │   │
│  └──────────────────────────────────────────────────────┘   │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  HAL, CPU SIMD │ GPU │ NPU │ Hailo │ Coral │ ANE │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

The four hard decisions, locked

Decision	Choice	Reason
API contract	C ABI (naina.h)	Everything else (Py/Rust/Swift/WASM) is glue. llama.cpp-grade portability.
Model exchange	ONNX as source of truth	Convert to NCNN/CoreML/TRT at install or build time. Single graph, many deployments.
Backend selection	Runtime, not compile-time	Probe what's available, pick best, fall back. Build flags toggle availability, not use.
Model loading	Manifest-driven	YAML registry: URL, hash, preprocessing, postprocessing, license, benchmarks. New SOTA = new YAML entry.

Module modularity, why v1 isn't vaporware

Each task module is independently shippable:

v0.2  Text spotting:  detect + rectify + recognise        (shipped)
v0.3  Structure:      layout + reading order + markdown
v0.4  Browser:        WASM target + client-side PWA
v1.0  Guarantees:     cross-binding parity, benchmarks, MCP

Same API. Modules light up over time. No big-bang release.

Two of the five modules, TextRectify and DocAssemble, touch no model at all. They are deterministic pure functions. That is deliberate: it makes the cross-binding identity guarantee provable with plain equality assertions rather than float-tolerance comparisons.

The model registry pattern

Weights are tiered by device, not by licence. Every model naina ships is Apache-2.0, so a permissive-vs-research split would carry no information; what actually varies is size and the hardware it suits.

- id: text_recognize.tiny
  task: text_recognize
  tier: tiny              # ~6 MB with det, browser, phone, Pi Zero
  arch: pp_ocrv6_rec
  license: Apache-2.0
  files:
    onnx:
      url:        "${release_base}/ppocrv6_tiny_rec.onnx"
      source_url: "${hf}/PP-OCRv6_tiny_rec_onnx/resolve/main/inference.onnx"
      sha256:     "9ef676d6ed3c88256a2d92c640c44f25b0c40947e111b14b8be8f594091563e6"
      bytes:      4462639
  output:
    type: ctc_logits
    postprocess: { blank_index: 0, num_classes: 6906 }

Three properties of this that matter:

url points at naina's own release, not upstream. The sha256 already meant an upstream swap failed closed rather than corrupting output, but it would still have broken. Mirroring removes the third-party runtime dependency entirely. source_url records provenance and is never fetched.

A request for a tier that lacks a task falls back to a larger one rather than failing. layout_detect exists only at medium today, so tiny and small layout requests resolve there.

Per-tier postprocessing is data, not code. num_classes is 6906 for tiny and 18710 for small/medium, because their charsets genuinely differ. Hardcoding one value shifts every decoded character.

Hot-path discipline (the engineering moat)

This is where naina earns the "super fast" claim, these are non-negotiable for the C++ implementation phase:

1. Zero allocations per frame in steady state. Arena allocator per pipeline.
2. No exceptions across ABI, status codes only. Internal C++ may throw but never cross the C boundary.
3. Zero-copy pixels end-to-end: V4L2 / GStreamer / DMA-BUF → GPU/NPU directly. Host roundtrips are the enemy.
4. Dynamic batching across streams. One camera = single-frame inference; eight cameras = batched.
5. Async pipeline: detection on frame N runs concurrent with embedding on frame N-1.
6. Quantization-aware: every model has FP32 / FP16 / INT8 variants in the registry. INT8 is default on edge.
7. SIMD everywhere image ops live: NEON on ARM, AVX2/AVX-512 on x86, via a thin HAL.

Deployment matrix

Target	Primary backend	Fallback
Raspberry Pi 5 (ARM Cortex-A76)	NCNN (INT8)	ONNX Runtime CPU
Jetson Orin Nano	TensorRT (FP16)	ONNX Runtime CUDA
Intel NUC / industrial PC	OpenVINO	ONNX Runtime CPU
Apple Silicon / iPhone	CoreML (ANE)	ONNX Runtime CoreML EP
Android phone	NCNN Vulkan	ONNX Runtime NNAPI
Hailo-8 / Coral Edge TPU	vendor SDK	ONNX Runtime CPU
x86 server w/ NVIDIA GPU	TensorRT	ONNX Runtime CUDA
Browser (stretch)	ONNX Runtime Web (WASM SIMD)

Out of scope (by design, not by accident)

• Identity database / vector store, return embeddings, integrators choose Faiss/Milvus/hnswlib.
• UI / dashboards, naina is a runtime, not an app.
• Authentication / authorization, concerns above the library.
• Government ID linkage / "crime prediction", biometric ID against state databases is regulated (EU AI Act high-risk, India DPDP). Library stays identity-agnostic; that's an integrator concern with its own legal review.

Open questions to answer before v1.0

• Training pipeline. Wrap pretrained for v1, train own weights for v2?
• Backend priority. Which 2 backends ship first? My pick: ONNX Runtime (portability) + NCNN (edge perf).
• Tracking lib. Adapt ByteTrack ourselves, or wrap an existing C++ port?
• CI matrix. Which targets do we gate releases on?