Embedded user-docs size budget

Where this stands

User docs (docs/user/**/*.md) ship inside the lattice binary, generated by crates/lattice-help/build.rs and embedded deflate-compressed. :help works offline with no filesystem layout assumption beyond the binary itself — the property that made embedding worth it — without the binary paying raw markdown volume.

Bodies inflate on first open and cache in a OnceLock, so a session that never opens :help never decompresses anything.

As of 2026-07-29: 73 topics, 495 KB raw → 197 KB embedded (2.5×). Budget is 384 KB compressed.

History

The docs were embedded uncompressed (include_str!) until the doc set grew to 498 KB raw against a 512 KB budget — 14 KB of headroom, with 33 more mode pages queued. That triggered this document's own rule ("do not just bump the number") and the compressed-embed work landed.

The prediction here was ~5× compression. Measured: 2.5×. Each doc is deflated independently, and at ~7 KB per file the window barely warms up — a shared dictionary across the corpus would do better but would mean inflating everything to read one topic, which costs the laziness. 2.5× on a 384 KB budget still allows ~960 KB of raw markdown, roughly double today's set.

The budget

384 KB of compressed bytes, asserted by embedded_user_docs_stay_under_size_budget.

It measures the embedded size now, not the raw markdown. Raw size stopped being the cost the moment compression landed; measuring it would fire the alarm on volume the binary never pays for.

Two further tests guard the scheme itself: embedded_bodies_are_actually_compressed (a regression that embedded bodies raw would otherwise slip under the budget while the binary grew) and every_embedded_topic_decompresses_to_its_original_length.

Cost, measured

crates/lattice-help/benches/topics.rs (Apple Silicon; the rest of benchmarks.md is measured on the Ryzen box — not comparable):

Bench	Median	What it is
`help_registry_boot_ns`	~17.8 µs	`builtin_topics()` at editor boot. Decompresses nothing — if this grows with the doc set, laziness has been lost.
`help_topic_first_open_ns`	~66.6 µs	Inflate + cache fill for the largest topic (`modal-editing`). The one-time cost per topic per session.
`help_topic_cached_open_ns`	~562 ns	Every subsequent open — a clone of the cached string.

All three sit on the dispatch path of an explicit user action, never per-keystroke or per-frame, so the bar is "imperceptible within a command" rather than the frame budget.

What to do when the budget fires again

Do not bump the number. Compression is already spent. The next lever is moving docs out of the binary into a runtime directory.

That lever used to carry a second justification — it was also read as the only model that lets plugins ship :help pages. That is no longer true, and the plugin half has been decided the other way (2026-08-22): a plugin's markdown is include_str!'d into its own .wasm and registered through a help WIT seam against a runtime-writable registry handle. Docs travel with the artefact that owns them, and a plugin's pages never enter this budget at all. See ../architecture/contributable-registries.md §4.

So what follows is now a builtin-volume lever only, and it is the weaker for having lost its other reason. If the budget fires, weigh it against simply writing less, or writing more densely, first.

The runtime-directory model

Every editor in this class does the same thing, and the convention is worth following rather than reinventing:

Editor	Location	Plugin docs
Vim / Neovim	`$VIMRUNTIME/doc/` + generated `tags`	every plugin ships `doc/` on `runtimepath`; `:helptags` indexes it
Helix	`runtime/`, found via repo-sibling → user config → `$HELIX_RUNTIME` → compile-time `HELIX_DEFAULT_RUNTIME` → exe-relative	—
Kakoune	`../share/kak/doc/*.asciidoc`, exe-relative	`:doc` also searches the autoload dir
Zed, VS Code	a website	n/a — no offline manual

Zed's answer is out for lattice: the TUI is a first-class peer for headless / SSH, so :help has to work with no browser.

Proposed shape when this lands (decided 2026-07-29, deferred):

resolution order
  1. $LATTICE_RUNTIME/doc
  2. ~/.config/lattice/runtime/doc
  3. LATTICE_DEFAULT_RUNTIME/doc     (compile-time, for packagers)
  4. <exe>/../share/lattice/doc
  5. the embedded set                (always present — the floor)

user      ~/.config/lattice/doc/*.md

The plugins <plugin>/doc/*.md, overlaid at load line this block used to carry is struck — plugin docs ship inside the component now (§ above).

Scoped docs-only but named for growth: build runtime/doc/ and the resolution chain, wire only docs through it, and leave the path shape free for runtime/themes/ and runtime/queries/ later without a second migration.

Keeping the embedded set as the floor matters because of how lattice is installed: cargo install and a scp'd static binary both produce a binary with no runtime directory anywhere, and those are first-class paths for a Rust TUI editor. Helix lives with the equivalent gap, but there a missing runtime also kills syntax highlighting — loud, and self-correcting. A missing docs directory is quiet, and it fails the user least equipped to diagnose it.

Rejected: remote fetch

Embed names + summaries, serve bodies from a URL. Breaks the offline-works property unconditionally.

Anchors

Build script: crates/lattice-help/build.rs
Registry + tests: crates/lattice-help/src/topics.rs
Bench: crates/lattice-help/benches/topics.rs
Slice plan: slice-plans/archive/help-docs.md