Design Document: A Modal, GPU-Accelerated, Plugin-First Editor
Status: Draft v0.6 Codename:
lattice(placeholder -- rename freely) Author: TBD Last updated: 2026-07-15
Status legend (v0.6). This revision reconciled the spec against the implemented code (main branch, through Phase 7). Where a subsection describes design that has not yet shipped, it is fenced inline:
- ✅ built — implemented and exercised (the default; unmarked prose is built).
- 🟡 partial — some of the described surface is built; the rest is planned.
- 📝 planned — designed but not yet implemented; kept here as the forward spec (design.md is authoritative for what to build, not only what exists).
The companion ledger
../operations/implementation.mdis the authoritative per-feature current-state record; this doc is the design. When they disagree on what exists today, the ledger + the code win.
Changes from v0.5
This revision is a reconciliation pass: the spec had drifted from the implementation across many sections (the design predates most of Phases 4–7). v0.6 corrects the load-bearing divergences, replaces superseded designs with what actually landed, and fences not-yet-built design as 📝 planned rather than deleting it. Headline changes:
- §3 / §4 corrected. "Major and minor modes are themselves implemented as plugins" was false — modes are a native
lattice-modetrait crate (WASM is the extension substrate, not the mode substrate). The threading model dropped the unusedrayonpool (off-UI work isspawn_blocking) and now names the dedicatedcurrent_threadeditor actor. The tech stack fencestaffy/cosmic-text/parley/wgpu(none integrated) and adds the shippedratatui/crossterm(TUI),imara-diff,fancy-regex,gpui. - §5.6 Rendering rewritten. The per-buffer
Renderertrait withEditorRenderer/DocumentRenderer, four selectable GPU fast paths, a taffyDocumentRenderer, and a v1 sprite atlas never shipped. What landed: an off-thread cell-grid /DisplayMatrix(lattice-cells) consumed by thelattice-ui-tui(ratatui) andlattice-ui-gpui(GPUI) peers behind a marker-styleRenderer { Theme, PaneRenderRegistry }trait. The old design is demoted to a superseded-alternative note; the sprite atlas is fenced 📝. - §5.1 / §5.2 corrected. The published
Documentstruct was fiction (the real struct is lean; syntax / diagnostics / modes live in sibling crates); undo is a linearUndoStack, not a persisted branching tree. Grammarexecuteis synchronous (thePendingenvelope is the runtime handle layer); the actor mailbox is unbounded (CommandError::Busywas deliberately removed). - §5.4 corrected. LSP attach moved to
LspMode::on_activate(the retiredattach_driver/build_lsp_subsystemare gone) under the mode-ownership rule. - §5.9 / §5.10 / §5.12 reconciled. UI structs updated to the shipped shapes (binary
PaneNode,TabSlot,ModelineRegistry, buffer-backed popups); the event bus is a concretestructinlattice-runtime; config is the v0.5OptionDecl/trait OptionTyperegistry overlattice.toml. Rich minibuffer, notifications, scrollbars/minimap, before-event veto,:autocmd, and theinit.rs-as-WASM config path are fenced 📝 planned. - §6 / §9 corrected. The
Commandenum and cross-process MessagePack framing are retired (in-process typed mailbox); theEventcatalog matcheslattice-protocol. The WIT §9.4 rewritten to the shippedlattice:plugin-hostpackage: guest-exported capability worlds (picker-source,completion-source,decorations) driven by the host, plus a grammar callback trampoline — not the old import-return-id / monolithicuimodel. - §8 / §14 / §15 refreshed. Perf numbers updated to
benchmarks.md; the Phase-7 plugin-host rows filled in (grammar round-trip ~340 ns, CI-gated) and the WASM-overhead risks downgraded to mitigated; resolved open questions struck (Replace mode, narrow-to-region, per-plugin async task, built-in snippets). - §11 Project Layout rewritten to the real 32-crate workspace (the
lattice-render*trisection,lattice-modes,lattice-config-api, andlattice-headlessnever existed;wit/andplugins/corrected). - Appendix B fenced. Only interactive arg specs (B.1) and
:describe-buffer(B.10) are built; the rest marked 📝 / 🟡.
Changes from v0.4
This revision narrows the §5.8 mode design to a high-level anchor and moves the load-bearing detail to the new companion doc docs/mode-architecture.md, in the same shape as keymap-architecture.md for §5.2.
- §5.8.3 corrected. The "Major and minor modes are implemented as WASM plugins. No privileged built-in path" paragraph was a maximalist over-commitment. Modes are now framed as a trait with three implementation paths: built-in (compiled Rust), bundled plugin (WASM, ships with editor), third-party plugin (WASM, user-installed). The host treats all three identically downstream of registration. See
mode-architecture.md§2 for rationale. - §5.8 anchored to the companion doc. Replaces the former three subsections with one §5.8.1 listing the high-level commitments design.md depends on. Trait surface, lifecycle events, typed option registry, group resolution, customize surface, migration plan all live in
mode-architecture.md. - §5.8.2 added -- mode ownership elevated to a named commitment. "Modes own their full surface; the host is a thin substrate" is now a first-class §5.8.1 commitment with its own §5.8.2 subsection (the acid test, the half-migration failure mode, the substrate-vs-helper rule). Previously this load-bearing rule lived only in
mode-architecture.md§5.2 / §5.3 andCLAUDE.md; surfacing it in design.md makes the single deepest Zed differentiator (comparison-zed.md§3) enforceable by reference, closing the half-migration drift class (K.3.2, M.7). - §5.8.1/§5.8.2 refined (2026-06-24, CR.6) -- modes own their command declarations + the
Effectboundary made explicit. The mode-owned surface now names command declarations (action:*+:ex-commandCommandSpecs, registered viaboot.commands_mut()-- the multibuffer/diff pattern), not just keymaps + handlers. TheEffectvocabulary is documented as the host boundary by design, with the genericEffect::ApplyEdit { target, edit, cursor }primitive letting a mode drive an arbitrary edit with no feature-specificEditor::do_<x>; the acid test is sharpened to "zero per-featureEditor::do_<x>+ zeroAction::<feature>", with the lifecycleEffectappliers (&mut Editor-bound: pane / actor / spawn) called out as legitimate host residue (the cycle constraint;host-provider-boundary.md). Worked example: the diff subsystem (diff-extraction.md§4). For how anEffectactually reaches its host + renderer appliers — the "everything inout.effectswas already host-applied" invariant and the sync-direct-call vsapply_effect_hostvs async-inbound paths — see the audit../audit/effect-dispatch.md. - Options and groups are typed identities, not strings. Cross-crate uniqueness enforced by Rust's type system; display-name uniqueness by
linkmeaggregation. Strings appear only at boundaries (:set, TOML, plugin manifests). See §5.12 augmentation andmode-architecture.md§6.4 / §6.8. :customizeshipped in v1. Group-oriented (a mode's options or anOptionGrouplikeLsporPicker), buffer-backed (customize-modemajor), TUI-first. Same store as:set. TOML write-through deferred to v1.x. Seemode-architecture.md§6.7.- Hover popup is a markdown-mode buffer in floating geometry. §5.9 reframed: popups carry buffer content with a popup-shape view rather than being non-buffer primitives whose rendering bypasses the standard pipeline. Resolves the K-hover regression (no syntax highlighting, no consistent rendering with
:helppanes) by routing hover content through the samecompose_visible_linespipeline as any other buffer.
Changes from v0.3
This revision pivots editing semantics, plugin model, and UI layout to align the design with the project's four paramount goals: performance, extensibility, extensible vim modal editing, and asynchronicity.
- Strict vim grammar. Modal editing follows vim semantics exactly:
[count] ["register] (operator [count] (motion | text-object) | motion | text-object | action). Kakoune-flavored selection-first composition is dropped. Multi-cursor is no longer the default model -- the data layer accommodates it as a clean post-1.0 extension; v1 invariants assume a single selection with vim's visual extents. - The vim grammar is the public command API. Every operator, motion, text object, register, range, and count is a first-class typed command. Keymaps are configuration that bind chord sequences to command invocations -- the default vim keymap is itself a config file. Extending the grammar (new motions, text objects, operators) is first-class, not an afterthought. Tree-sitter-driven motions and text objects are a clean future plugin, requiring no host changes.
- Unified command / grammar dispatch (deviation from vim). Vim's
:ex-command world and its functional / plugin world are merged into oneCommandRegistrywith one dispatcher. Operators, motions, text objects, ex-commands, plugin contributions, and command-palette entries all share the sameCommandInvocationshape and flow through the sameexecute. The:-line is a parser front-end that maps vim ex-syntax to typed invocations. Vim user UX is preserved exactly; the simplification lives below the parser. Wins: single dispatch site, plugins-as-peers-of-builtins, lower per-call overhead, cleaner mental model.
The remaining v0.4 changes are a set of vim/emacs unifications that bridge the two editor cultures:
- Hooks and autocmds unified as typed event subscriptions (§5.10). One event registry with typed payloads; vim
:autocmdsyntax becomes sugar forsubscribe(event, filter, invocation); emacs's hook idiom maps to the same primitive. - Self-documenting help system (§5.11). Every command, option, event, mode, and keybinding carries metadata.
:describe-key,:describe-command,:describe-event,:describe-option,:describe-modeopen buffer-backed help views. Emacs-class introspection from v1. - Unified position history (§5.1). One ring per buffer + one global ring. Items are pushed by tagged sources:
AutoJump(vim's auto-push behavior),ExplicitMark(emacs's set-mark),PluginPush(LSP hops, etc.). Different keybindings walk different source-filtered views of the same data. - Visual mode IS the active region. When Visual is active, the current selection is the default
rangearg for any range-accepting command. Vim's "operate on visual selection" and emacs's "operate on region" become the same mechanism. - Macros are recorded
CommandInvocationsequences, not keystrokes. Replay survives keymap changes, plays back faster, and is editable as data (a buffer-backed view of a recorded macro is editable before save). - Typed options + customize-as-buffer-view (§5.12). Every option is a typed registered value (
name, type, default, doc, group, validator).:setis a parser front-end producingset-optioninvocations. A customize buffer view shows options grouped by topic, type-aware, writes back to user TOML. - Rich minibuffer. The
:line,/search line, and every interactive prompt are full buffers with a major mode (command-line,search-line,git-commit-line,repl-input, ...). Full vim grammar applies. Tree-sitter highlights the command syntax. Live error indicators, parameter hints, type validation, completion popups, and substitution preview render as inline decorations on the editor and minibuffer buffers. See §5.9.10.
A small set of further unifications -- interactive arg specs, :g/:v as normal commands, history pickers, *messages* / *scratch* / *compilation* as buffers, :redir as an effect-capture wrapper -- are documented in Appendix B.
- Iconography and sprites in v1 (§5.6.7). A sprite atlas (separate from the glyph atlas; same GPU pipeline) supports file-type icons in the file tree, severity icons in gutters and diagnostics lists, language logos in tab strips, status-line indicators, picker leading icons, and notification badges. Plugins register
SpriteSets; users override individual sprites in TOML. Distinct from Path 4 (full inline media blocks), which remains post-1.0. - WebAssembly Component Model plugin host from day one. WIT interfaces are the canonical plugin API. Plugins are cross-language (Rust, Zig, Go, AssemblyScript, anything that compiles to component-model WASM), capability-gated, crash-isolated, and fuel-limited. Performance is the gating constraint, not WASM-vs-native: AOT compilation, lazy instantiation, resource handles for zero-copy buffer access, native host APIs for tree-sitter / ripgrep / regex / I/O, and a strict host-call overhead budget (p99 < 500ns typed call, < 5μs round-trip for grammar-extension calls). Built-in motions, text objects, and operators stay native in
lattice-grammar-- WASM is for extensions, not for replacing the hot path. - Everything is a buffer. No fixed sidebars or bottom panels. File tree, outline, symbol list, diagnostics, search results, terminal, REPL -- all are buffers, distinguished only by mutability flags, content provider, and major mode. Users place them in panes via the same split/window operations used for code buffers. The pane tree, tabs, popups, pickers, notifications, mode line, header line, and command/echo area remain.
- Modal mode stays orthogonal to major/minor modes. Major mode = content-type identity (rust, markdown). Modal mode = a buffer-level state machine (Normal, Insert, Visual, Op-pending, Command, Search). The two axes do not collapse into each other.
Changes from v0.2
This revision adds UI specification and a performance baseline:
- New Section 5.9: UI Components -- formalizes the window/pane tree, popups, pickers, panels, sidebars, notifications, status lines, and the command/echo area. This was a meaningful gap in v0.2.
- Status segment registry specified -- defined contribution model for the mode line, header line, and gutter.
- Picker abstraction specified as a first-class primitive used by file finder, command palette, symbol search, and plugin-defined pickers.
- Section 9 (Plugin API) expanded with explicit UI contribution primitives.
- Section 6 (Core Protocol) expanded with UI commands and events.
- New Appendix A: Performance Comparison -- concrete comparison with Neovim and Emacs across all relevant dimensions, with methodology and risk assessment. Intended as a reference baseline during implementation.
Changes from v0.1
(Preserved for context.)
- Rendering subsystem restructured around a
Renderertrait abstraction with multiple specialized implementations. - Rich-buffer rendering pipeline added as a first-class capability.
- Major modes and minor modes added as the primary customization model.
- Future web-rendering capability acknowledged but deferred.
- Performance commitments restated with concrete numbers per rendering path.
1. Vision
A text editor that combines the modal editing power of Vim with the extensibility model of Emacs, built on a modern, asynchronous, multi-threaded core that never blocks the UI. The editor is GPU-accelerated, has first-class support for tree-sitter and LSP, and exposes a plugin API where extensions are sandboxed WebAssembly modules -- meaning a slow or buggy plugin cannot freeze the editor, leak memory, or crash the host.
The guiding principle: the user's keystrokes are sacred. Every architectural decision is in service of the rule that keystroke->glyph latency stays imperceptible -- indistinguishable from the terminal/compositor echoing the key, always within one display frame (<= 8.3ms at 120Hz) -- regardless of what the editor is doing in the background: indexing a million-line repo, running an LSP request, evaluating a misbehaving plugin, or re-shaping rich text in a markdown buffer. One frame is a physical ceiling -- the goal is to never miss a refresh, not to fill a budget; the bar is "as fast as the best-in-class reference (vim / the compositor), ratcheted down forever", not a fixed number.
A secondary principle, equally non-negotiable: the architecture must compose. A buffer of plain code, a buffer of richly-styled markdown, and a future buffer of rendered web content are all the same kind of thing as far as the core is concerned. Specialization happens at the rendering layer, not the data layer.
2. Goals and Non-Goals
2.1 Goals
The four paramount goals -- in priority order when they conflict -- are performance, extensibility, extensible vim modal editing, and asynchronicity. Everything else serves these.
- Strict vim modal editing. The full vim grammar -- counts, registers, operators, motions, text objects, ex-ranges, dot-repeat, marks, macros -- is preserved exactly. Specific edge cases may be deferred for v1, but the language semantics are not altered.
- The vim grammar is the public command API. Operators, motions, text objects, registers, ranges, and counts are typed commands. Keymaps are configuration that bind chord sequences to command invocations -- the default vim keymap is itself a config file, not hardcoded behavior. Any user or plugin can invoke any command, compose commands, or build entirely new editing flows.
- Unified command / grammar dispatch. Vim's split between ex-commands and functional APIs is merged into one
CommandRegistryand one dispatcher. The:line is a parser front-end producing typedCommandInvocations; everything below the parser is one normal call path. Plugins and built-ins are peers of the dispatcher, not separated worlds. - Vim/emacs unifications. Hooks and autocmds unify as typed event subscriptions (§5.10). Self-documenting help via metadata on every registered primitive (§5.11). Jump list and mark ring unify as position history with tagged sources (§5.1). Visual mode IS the active region. Macros are recorded
CommandInvocationsequences. Options are typed with a customize buffer view (§5.12). The minibuffer is a rich editing surface with full vim grammar, tree-sitter syntax highlighting, and live error / parameter-hint decorations (§5.9.10). - Extensible grammar. Registering new motions, text objects, and operators is first-class -- not bolted on later. Tree-sitter-driven motions ("next function," "inner argument," "outer class") are a clean future plugin requiring no host changes.
- Imperceptible input latency -- keystroke->glyph indistinguishable from the terminal/compositor echoing the key, always within one display frame (8.3ms at 120Hz, 16ms at 60Hz; a physical ceiling, not a target) on a mid-range laptop, for buffers up to 100MB, regardless of background load. The bar is match-or-beat Vim/Neovim, ratcheted down by CI -- not a fixed number.
- Truly non-blocking architecture. No plugin, LSP request, file I/O, syntax parse, or text-shaping operation can stall the UI thread. Ever. Multi-threaded by construction.
- First-class tree-sitter integration: incremental parsing, syntax highlighting, structural motions, structural selection, language injections.
- First-class LSP integration: diagnostics, completion, hover, go-to-definition, rename, code actions.
- GPU-accelerated rendering with sub-pixel-precise text, smooth scrolling, layered rendering paths optimized per content type.
- Rich-buffer rendering: variable fonts, sizes, and weights within a single buffer, with editing latency indistinguishable from code editing.
- Major mode / minor mode customization: composable, content-type-aware customization. (Modal state -- Normal/Insert/Visual/etc. -- is orthogonal; see §5.2.)
- Everything-is-a-buffer UI. File tree, outline, diagnostics, search results, terminal, REPL -- all are buffers placed by the user into panes. The editor enforces no fixed sidebar or bottom-panel layout.
- Modern transient UI surface: tabs, splits, popups (completion, hover, signature help), pickers (file, symbol, command palette), mode line, header line, notifications, command line / echo area.
- WebAssembly Component Model plugin host from day one. Plugins are sandboxed WASM components: cross-language (anything that compiles to component-model WASM), capability-gated, crash-isolated, fuel-limited. Performance-disciplined: AOT compilation, lazy instantiation, resource handles for zero-copy buffer access, native host APIs for hot work (tree-sitter, ripgrep, regex, I/O). Strict per-call overhead budgets enforced in CI. Built-in motions / text objects / operators stay native; WASM is for extensions. Multi-threaded by construction -- each plugin instance has its own
wasmtime::Storeand runs as a tokio task; many plugins execute in parallel across cores. - Cross-platform: macOS, Linux, Windows. Wayland and X11 both work day one on Linux.
- Headless mode: the core can run without a UI.
2.2 Non-Goals (for v1.0)
- Native web-page rendering. Architecture accommodates a future
WebRendererbut it's not in scope. - Collaborative real-time editing (CRDT/OT). Data model won't preclude it.
- Built-in terminal emulator. Plugin material (and a buffer, like everything else).
- Email, IRC, calendar, file manager. Out of scope.
- Backwards compatibility with Vim or Emacs configs/plugins.
- Web/mobile targets.
- Tree-sitter alternatives.
- Inline arbitrary HTML/CSS layout in buffers. Floating elements, multi-column flows, CSS-grid in buffer body are out.
- Kakoune-style selection-first composition. Vim-grammar semantics only.
- Multi-cursor as the primary editing model. The selection data type is a set so multi-cursor is a clean post-1.0 extension; v1 invariants assume a single selection with vim's visual extents.
- Pluggable editing paradigms in v1. No emacs/readline-style alternative to vim modal editing in v1. The command API is paradigm-agnostic so an alternative can be added post-1.0 without redesign.
- Fixed dock layout. No left/right sidebar or bottom-panel as a first-class concept. Panels-as-buffers compose via the pane tree.
- In-process scripting language / sub-keystroke REPL. No Lua, no embedded Scheme, no
M-x ielm. WASM (Rust today; any component-model language tomorrow) is the single extension substrate. Live evaluation is the*scratch:rust*plugin-authoring workflow described in §10, with 1-3 s compile latency, not a sub-keystroke evaluator. A community-shipped plugin can offer the latter as an extension; the host does not. - Backwards-compatible config syntax beyond TOML. Lua / vimscript / elisp config files are not supported; config is TOML for static data and Rust→WASM (
init.rs, §5.12) for code. Extensions are WASM. There is one extension substrate. - A function-call / palette / scripting syntax on the
:line. The:line is vim's ex-syntax DSL, full stop -- a parser front-end that produces typedCommandInvocations for the unified dispatcher (§5.2.1). Code paths (plugins,init.rs, the Rust functional API) constructCommandInvocationvalues directly via the WIT host, which is the canonical typed surface for non-typing input. Two input surfaces (vim DSL for users, typedCommandInvocationfor code), one dispatcher. We deliberately do not attempt to fold typing-into-:and code-construction into a single shape; the cost (a sugar layer that re-implements vim DSL inside a function-call parser, plus the cognitive overhead of "is this a function or a vim shorthand?") outweighs the gain (a unification we already have atCommandInvocation).
3. Architectural Overview
The editor is structured as three strictly separated layers communicating exclusively via typed message passing over async channels.
+------------------------------------------------------------------+
| UI Layer (renderer peers) |
| |
| Window/tab management | Pane tree | Input loop |
| Popup + picker (buffer-backed) | Echo area | Buffer-backed views|
| |
| +------------------------------------------------------------+ |
| | lattice_host::Renderer (marker trait: Theme, | |
| | PaneRenderRegistry) -- consumes a per-pane DisplayMatrix| |
| | +----------------+ +----------------+ +---------------+ | |
| | | TuiRenderer | | GpuiRenderer | | Future: Web | | |
| | | (lattice-ui- | | (lattice-ui- | | (placeholder| | |
| | | tui, ratatui) | | gpui, GPUI) | | ) | | |
| | +----------------+ +----------------+ +---------------+ | |
| +------------------------------------------------------------+ |
+------------------------------+-----------------------------------+
CommandInvocation | RenderState snapshots
v ^
+------------------------------------------------------------------+
| Core Layer |
| |
| Buffers (rope) | Documents | Selections | Undo stack | |
| Grammar dispatcher | Tree-sitter | LSP clients | |
| Major/Minor Mode Registry | File I/O | cell-grid worker | |
| Workspace | Search | Modeline registry |
+------------------------------+-----------------------------------+
(in-process typed mailbox + event bus)
v ^
+------------------------------------------------------------------+
| Plugin Layer (WASM sandboxes) -- 📝 host built |
| (Phase 7); editor-side loading is Phase 8 |
+------------------------------------------------------------------+
Critical architectural property: the UI is just the most privileged client of the core. Major and minor modes are a native lattice-mode trait (not WASM plugins) — the default keymap and grammar never cross the WASM boundary; WASM is the substrate for extensions (and, in a later phase, for shipping modes as components). See §5.8.
Threading model:
- UI / render thread (one per renderer peer): owns the window / terminal, input event loop, layout, rendering. Consumes published snapshots; does no blocking work, no I/O, no parsing (paramount goal #4, §5.7).
- Editor actor (a dedicated
current_threadtokio task): owns the writableDocumentand dispatches grammar;&mut Editorcannot escape it. Reads are wait-free snapshot loads (arc-swap). - Shared / LSP executors (tokio multi-thread): LSP clients, background subsystems, plugin instances (each plugin owns its own
wasmtime::Storeas a task). - Blocking pool (
spawn_blocking): file ops, large files, tree-sitter parsing, cell-grid builds, diff computation — all CPU/IO work off the actor and render threads. (An earlier draft named arayonpool for render-prep; that was never adopted —spawn_blockingcovers it.)
3.1 Core vs plugin: the fast path / orchestration split
CLAUDE.md goal #1 (performance) and goal #2 (extensibility) compete on every subsystem decision. The discipline that resolves the conflict:
Fast path stays in core. Configuration / orchestration / authoring goes to plugins. The trait surface between them is the extensibility seam.
The WASM Component Model boundary has real cost (typed-call p99 budget < 500ns; round-trip < 5μs per §5.5). Anything that fires per-keystroke or holds keystroke-hot-path state pays that cost on every input event, and even with batching it accumulates against the one-frame keystroke-to-glyph ceiling (8.3 ms at 120 Hz, 16 ms at 60 Hz). We earn extensibility around those subsystems via traits, not inside them via WASM dispatch.
Concretely:
| Subsystem | Core (Rust crate) | Plugin (WASM Component) |
|---|---|---|
| LSP | wire layer, actor, document sync, diagnostics broadcast bus, supervisor primitives, position-encoding shim. Per-keystroke hot path (didChange, hover paint). | server installer / config UI / lifecycle preferences (lighthouse, Phase 8b); per-buffer source contributions (AI completion, project linters). |
| Snippets | parser, body type, render walker, active-snippet state machine, placeholder navigation. Per-keystroke hot path during placeholder edits. | snippet pack discovery + management UI; project-snippet sync; authoring tools. |
| Picker | widget, keymap, popup geometry, matcher / ranker default impls. Used by ~10 surfaces; each fires off the keystroke path. | picker sources (fuzzy file finder, project-grep, command palette content) via the existing trait surface. |
| Insert-mode completion | state machine, aggregator, popup widget, completion-popup minor mode, matcher / ranker traits. | sources beyond the bundled set (LSP + snippets + buffer-words + path + tree-sitter): AI / Copilot, project-specific generators. |
| Modal engine | grammar, dispatcher, builtins (motions / operators / text objects), modal state machine. | new motions / text objects / operators via the grammar-extension trait surface. |
| Renderer | the cell-grid / DisplayMatrix substrate (lattice-cells) + the lattice-ui-tui (ratatui) and lattice-ui-gpui (GPUI) peers, pane tree, buffer-backed popups. | renderer overlays (e.g. inline-error decorations, custom gutter content); modeline elements. |
| Buffer / Document | rope, undo, edit dispatch, snapshot model. | content providers for non-file buffers (REPL, terminal, scratch:rust, magit-clone, diff viewer). |
What this isn't: a closed shop. The trait surface is rich (CandidateGenerator / CandidateMatcher / CandidateRanker / CandidateAnnotator / SourceGenerator / Hook / EventSubscriber / SpriteSet / StatusSegment / FoldProvider / etc.). Plugins extend lattice through these traits — that's where extensibility lives. What the trait surface does not expose is the keystroke-frequency state machines themselves: those stay in core and the trait surface taps into them.
What goes plugin: anything that's "an opinionated tool built on top of the editor" rather than "the editor's keystroke response." A magit clone, a fuzzy file finder, a git-blame inline overlay, a markdown-preview pane, a test-runner integration — all clearly plugin material. Phase 8b lists these explicitly.
Implication for build order: core grows first (Phases 4–6), establishing the trait surfaces plugins will consume; the plugin host (Phase 7) then ships against a concrete, exercised set of seams rather than speculative ones.
4. Technology Stack
| Concern | Choice | Rationale |
|---|---|---|
| Language | Rust, edition 2024, pinned 1.94 (rust-toolchain.toml) | Memory and data-race safety; mature async; strong editor ecosystem. |
| Async runtime | tokio (multi-thread shared/LSP pools + a current_thread editor actor) | Default; integrates everywhere. |
| Buffer | ropey | Battle-tested rope; O(log n) edits; cheap clones. |
| Parser | tree-sitter | Incremental, error-recovering, ubiquitous. |
| Regex / search | fancy-regex | Backreferences + look-around for / and :s. |
| Diff | imara-diff | Fast Myers/histogram diff for the diff subsystem (§5.13). |
| LSP types | lsp-types | Generated bindings; we write our own client. |
| TUI renderer | ratatui + crossterm (lattice-ui-tui) | First-class terminal peer (headless / SSH), not a fallback. |
| GPU renderer | GPUI (gpui, optional gui feature; lattice-ui-gpui) | Purpose-built GPU UI; peer of the TUI over a shared DisplayMatrix. |
| Layout / text shaping | 📝 taffy / cosmic-text / parley (planned) | Not integrated — the shipped renderer is a cell grid; GPUI shapes per-line. wgpu fallback likewise not pursued. |
| Plugin runtime | wasmtime (v46) + Component Model + WASI p2 | Sandboxing, fuel + epoch limits, async host (Phase 7). |
| Serialization | serde; WIT for plugin interfaces | In-process typed channels; Component Model for plugins. (Cross-process MessagePack framing was dropped — the editor is one process, §6.) |
| Config | TOML (lattice.toml) | Static option tier; programmable config (init.rs→WASM) is 📝 planned (§10). |
| Clipboard | arboard | System clipboard integration. |
| CLI | clap | Standard. |
| Logging | tracing + tracing-subscriber | Structured logs, span timing; MessagesLayer tees to *messages*. |
| Build | cargo workspace (32 crates) | Crate boundaries enforce architecture. |
| Testing | cargo test, criterion | ~3.5k tests + latency/throughput benches (ratcheted in CI). |
5. Component Designs
5.1 Buffer and Document Model
The atomic unit of editable text is a Buffer, internally backed by ropey::Rope. Buffers are wrapped in a Document with metadata.
The Document is deliberately lean — it owns only text, selections, and undo. Syntax, diagnostics, modes, and modal state are not fields on it; they live in sibling crates keyed by buffer id (which is what lets lattice-core stay dependency-free at the bottom of the graph). The real shape (lattice_core::Document):
struct Document {
id: DocumentId,
path: Option<PathBuf>,
buffer: Buffer, // ropey-backed
version: u64, // bumps on any state change
text_version: u64, // bumps only on text mutation (syntax-cache trigger)
selections: SelectionSet,
undo: UndoStack, // linear undo/redo (not a branching tree)
clean_position: Option<usize>,// undo depth matching on-disk state → `dirty` is derived
coalescing: bool, // fold an insert session into one undo unit
group_has_entry: bool,
}
Where the rest lives: syntax → a lattice-syntax handle; diagnostics → lattice-lsp; language / major mode / minor modes → App.active_modes (per-buffer, in the host, because mode resolution needs lattice-config); modal state (§5.2) → the host's Editor. Keeping these off Document is what breaks the lattice-core → lattice-mode dependency (see mode-architecture.md).
Key properties:
- Cheap snapshotting via O(1) rope clone.
- Versioning for stale-result rejection (
version+text_version). - Dirtiness is derived (
clean_positionvs. current undo depth), not a stored flag. - Undo is a linear
UndoStack(undo + redo vecs; a new edit clears redo). 📝 A branching undo tree + sidecar persistence are post-v1. - Selections support a primary cursor with charwise / linewise / blockwise visual extents (vim model). The type is a set so multi-cursor is a clean post-1.0 extension; v1 invariants assume one selection.
- Modal state (Normal / Insert / Visual / Op-pending / Command / Search) is its own axis, orthogonal to major mode; see §5.2.
Synthetic Documents. Help / log / scratch / messages / terminal-scrollback views are all Document instances on the same rope-backed abstraction. They differ from on-disk Documents along two orthogonal axes: BufferMutability (ReadOnly / Interactive / Editable, §5.9.8) and the absence of a file path. The grammar and rendering layers do not branch on "synthetic vs. file-backed"; mutability gates only operator-level edits via the read-only echo path. Terminal scrollback is the most recent addition — see terminal-as-document.md — but the model also covers help (§5.11), *messages*, LSP logs, and :scratch.
5.1.1 Position history (jump list + mark ring unified)
Both vim's jump list and emacs's mark ring track "where the cursor was" -- the same underlying data, with different push policies. We unify them into one position history with each entry tagged by the source that pushed it. It lives in the host (lattice-host, state.rs), fed by grammar via Effect::RecordJump; the real PositionEntry carries a buffer + registry buffer_id + terminal_scroll_offset (not a DocumentId / timestamp):
struct PositionEntry {
position: Position,
source: PositionSource,
buffer: /* name */,
buffer_id: BufferId,
terminal_scroll_offset: Option<u32>,
}
enum PositionSource {
AutoJump, // ✅ pushed by "big motions" (vim jump list semantics)
NamedMark(char), // ✅ vim's a-zA-Z marks
ExplicitMark, // 📝 reserved (emacs set-mark) — not yet wired
PluginPush, // 📝 reserved (LSP / fuzzy-finder hops) — not yet wired
}
Different keybindings walk different views. <C-o> / <C-i> (vim) iterate the jump-list (AutoJump) entries; g; / g, walk NamedMark entries. All bindings consume the same data structure through filtered iterators -- no duplicate state. The ExplicitMark / PluginPush sources are reserved slots (the enum and filtered-view machinery exist); the plugin-push typed API arrives with the plugin loader (Phase 8).
5.2 Modal Editing Engine
A buffer-level state machine in front of the buffer. Vim semantics, with one deliberate deviation: the command line and the functional API are unified. Orthogonal to major / minor modes -- modal state is its own axis.
Modal states:
- Normal -- default; keys parse as commands.
- Insert -- text input; only a small set of editing keys are special.
- Visual -- charwise (
v), linewise (V), blockwise (<C-v>) extension of selection. - Select -- vim Select mode: same charwise / linewise / blockwise extent as Visual, but a printable key replaces the whole selection and drops into Insert. Entered via
gh/gH/g<C-h>, toggled from Visual with<C-g>, or programmatically (Effect::EnterMode(Select(_))+ a selection). A reusable primitive for select-and-overtype (snippet placeholders, rename, template fields) — built decoupled from any consumer. See select-mode.md. - Operator-Pending -- entered automatically after an operator; awaits a motion or text object.
- Command --
:ex-command line. - Search --
/and?incremental search. - Replace --
Roverstrike.
Command grammar (vim):
[count] ["register] ( operator [count] (motion | text-object)
| motion
| text-object
| action )
dw, c2iw, >ap, "ay$, 3dd, gUap, :.+1,$d all parse cleanly. Operator + (motion | text-object) enters and exits Operator-Pending automatically. Counts compose (2d3w = d6w). Registers prefix any operator. Ex-command ranges (:1,5, :%, :'<,'>, :.,+10, pattern ranges) are part of the grammar.
5.2.1 Unified command / grammar dispatch (deviation from vim)
In vim, ex-commands (:write, :%s/foo/bar/g) and functional APIs (function(), call(), autoload) are two dissociated worlds bridged by :call and :execute. We unify them. Every named primitive -- built-in operators, motions, text objects, ex-commands, plugin contributions, command-palette entries -- lives in one CommandRegistry with a typed signature. There is one dispatcher.
pub struct CommandRegistry { /* ... */ }
pub struct CommandInvocation {
pub command: CommandId,
pub count: Option<Count>,
pub register: Option<Register>,
pub range: Option<Range>,
pub target: Option<Target>, // operator's motion / text-object target
pub bang: bool, // ex-command `!` (e.g. `:q!`)
pub args: Args, // typed per the command's signature
}
/// Grammar dispatch is SYNCHRONOUS: it runs on the document actor that owns the
/// buffer and returns the Effect directly. (`execute_with_env` / `execute` /
/// `execute_motion_only` in `lattice-grammar`.)
pub fn execute(reg: &CommandRegistry, doc: &mut Document, /* … */ inv: CommandInvocation)
-> GrammarResult<Effect>;
The async envelope is the runtime handle layer, not the grammar. Grammar execute is a plain synchronous call. What makes command submission non-blocking is that it happens inside the editor actor's own task: the UI sends a CommandInvocation over the actor mailbox and gets back a Pending<Effect> (a oneshot the caller may await or drop), while the actor runs execute synchronously on its thread and publishes a fresh snapshot. So "sync grammar dispatch" and "never block the UI" are both true — the boundary is the mailbox (lattice-runtime), not the grammar dispatcher.
// lattice-runtime: the actor-handle seam the UI actually calls.
pub struct Pending<T> { /* oneshot-backed; await or drop */ }
impl RopeDocumentHandle {
pub fn dispatch(&self, inv: CommandInvocation, env: DispatchEnv) -> Pending<Effect>;
}
Veto-class hooks are 📝 planned. The design intends pre-mutation hooks (BeforeApplyEdit, BeforeBufferWrite) that run inline under a latency budget and may veto/mutate. Today the event bus is observation-only (§5.10.2) — the Before* veto/mutation path is not yet built.
Atomicity scope. A single CommandInvocation is atomic within one document: edits, selection updates, and the resulting Effect commit together or not at all. Cross-document and cross-pane invocations (📝 :bufdo / :windo are not yet built) would be a sequence of per-document atoms with explicit failure modes, not a global transaction.
Backpressure. The actor mailbox is unbounded (tokio::mpsc::unbounded). An earlier design used a bounded mailbox that returned CommandError::Busy when full, but that dropped keystrokes under load (audit slice 6/H3) and was removed — Busy is gone from CommandError. The event bus still applies bounded-queue discipline for slow subscribers (drop-with-counter, no publisher backpressure).
The vim user experience is preserved exactly:
dw-- the state machine assemblesCommandInvocation { command: ops::delete, args: Target::Motion(motions::word_forward), count: 1, ... }and callsexecute.:1,5d-- the:-line parser front-end maps the ex-syntax string to aCommandInvocationwithrange = Some(Lines(1, 5))and dispatches through the sameexecute.:%s/foo/bar/g-- parser producesCommandInvocation { command: edits::substitute, range: Some(Whole), args: (Pattern("foo"), Replacement("bar"), Flags::Global), ... }.- A plugin invoking the same command from WASM passes a
CommandInvocationover the WIT boundary; same dispatcher, same signature.
What unification gives us:
- Single dispatch site. One place to instrument, fuel-meter, log, route to undo, record for dot-repeat, benchmark.
- Plugins and built-ins are peers. The only difference between a built-in operator and a plugin-registered one is whether the body is native or WIT-bound WASM. Same signature, same invocation path.
- Performance. No double-hop through ex-runtime -> wrapped function. The
:parser does syntax -> typed args; everything after is one normal call. - Discoverability. Every command is reachable from
:, from a keymap chord, from the command palette, and from scripts. One registry, one truth. - No vimscript residue.
:call,:execute,:function,:returndistinctions die. Plugins call the dispatcher directly.
What lives in the parser, not the dispatcher. Vim's syntactic oddities -- :set wrap!, :set lcs=tab:>·, :s/foo/bar/g flag parsing, range pattern syntax -- are handled by the :-line parser, which produces typed args. Each weird-looking command resolves to a normal (command_id, args) pair. The dispatcher knows nothing about the input syntax.
Two input surfaces, one substrate. The vim DSL on : is the canonical surface for user typing; constructing a CommandInvocation directly via the WIT host is the canonical surface for code (plugins, init.rs, the Rust functional API, future scripting-shaped extensions). They meet at CommandInvocation -- byte-identical from execute(...)'s perspective regardless of origin. The DSL stays vim-shaped because vim users have decades of muscle memory and many idioms (:%s/.../.../g, :1,5d, :wq!) don't map cleanly to function-call syntax without re-implementing the DSL as a sugar layer; the typed surface stays typed because plugins want signatures, not strings. We deliberately do not unify the input surface -- only the dispatch substrate. (The corresponding non-goal is in §2.2.)
📝 Every command reachable from : via a small kind-prefix form (planned). The CommandRegistry already namespaces motions / operators / text-objects (motion: / operator: / text-object:), but the :-line kind-prefix parser below is not yet built — today motions/operators/text-objects are reached via the chord grammar, and : reaches ex-commands. The design: ex-commands keep the bare alias surface (:wq, :write foo.txt, :set number), and the other kinds become reachable through a kind-prefix word + the registered name's tail:
:motion goto-first-lineruns the sameggmotion the chord grammar reaches.:operator delete word-forwardruns the operator over the named motion (or text-object) target.:text-object inner-worderrors helpfully -- text-objects are operator targets, so they need an operator.
The kind word disambiguates the namespace cleanly without forcing colons to repeat (:motion:goto-first-line reads as two :s on the cmdline -- visual noise). The three kind words (motion, operator, text-object) are reserved on the : line; no ex-command may shadow them. Targets after the operator name are themselves looked up first as motions, then as text-objects; the bare tail (without prefix) is sufficient because the dispatch context fixes the namespace.
This kind-prefix reachability is what closes the long-standing parser gap that otherwise contradicted "every command is reachable from :" while motions / operators / text-objects were silently rejected. The : form is meant for palette / discovery / scripting use; the chord grammar (5dw, daw, gg) remains the natural surface for power-typing.
Keymaps bind chord sequences to CommandInvocations. The default vim keymap is itself a config file: "dw" binds to a delete invocation with a WordForward target. Users and plugins override or compose without recompiling.
5.2.2 The grammar as typed values
Within the unified dispatcher, the grammar's primitive concepts are still typed values -- not strings or untyped enums. They are command IDs into the registry, plus typed argument values:
pub struct OperatorId(CommandId);
pub struct MotionId(CommandId);
pub struct TextObjectId(CommandId);
pub struct ExCommandId(CommandId);
pub enum Target {
Motion(MotionId, Args),
TextObject(TextObjectId, Args),
Range(Range),
}
pub enum Register {
Unnamed, Named(char), System, BlackHole,
Expression, ReadOnly(char), Numbered(u8),
}
pub enum Range {
Span { start: RangeBound, end: RangeBound }, // :1,5 / :'<,'> / :.,+10
CurrentLine, // .
Whole, // %
Selection, // current Visual / active region
Custom(RangeId), // plugin-registered range
}
// As-built: `Custom` carries no `Args` (just the RangeId), and `RangeBound::Pattern`
// holds the pattern as a `String` (compiled at use), not a pre-built `Regex`.
pub enum RangeBound {
Line(u32), Mark(char), CurrentLine, LastLine,
Pattern(String), Offset(Box<RangeBound>, i32),
}
pub struct Count(pub u32);
The built-in command catalog includes every standard vim operator, motion, and text object as registered commands:
struct Builtins {
// Operators
pub delete: OperatorId,
pub change: OperatorId,
pub yank: OperatorId,
pub indent_left: OperatorId,
pub indent_right: OperatorId,
pub upper: OperatorId,
pub lower: OperatorId,
pub toggle_case: OperatorId,
pub replace_char: OperatorId, // `r`
// (illustrative subset; `format` / `filter` operators are 📝 not yet built)
// Motions
pub char_forward: MotionId,
pub word_forward: MotionId,
pub word_backward: MotionId,
pub line_start: MotionId,
pub line_end: MotionId,
pub first_non_blank: MotionId,
pub paragraph_forward: MotionId,
pub find_char: MotionId, // takes char arg
pub mark: MotionId, // takes char arg
pub search: MotionId, // takes (pattern, direction) args
// ...
// Text objects
pub inner_word: TextObjectId,
pub around_word: TextObjectId,
pub inner_paren: TextObjectId,
// ...
// Ex commands (unified peers of operators/motions; same dispatcher)
pub write: ExCommandId,
pub quit: ExCommandId,
pub substitute: ExCommandId,
pub set_option: ExCommandId,
pub source: ExCommandId,
// ...
}
5.2.3 Keymap resolution
Walks layered keymaps in priority order:
- Built-in vim default keymap (a config file)
- Major-mode keymap
- Active minor-mode keymaps (in activation order)
- User config overrides
- Per-buffer ad-hoc bindings
Each keymap entry is (chord_sequence) -> CommandInvocation.
Resolution runs against the layered trie in the dedicated lattice-keymap crate; feature keymaps register at KeymapLayer::MinorMode(mode_id) / MajorMode(mode_id) (never Builtin). Authoritative reference: docs/keymap-architecture.md — the trie data structure, layer merging, performance commitments, and plugin / user-config registration paths.
5.2.4 Extensibility -- first-class
Plugins extend the grammar by registering new commands. The same registration API serves operators, motions, text objects, and ex-commands; all become first-class citizens of the dispatcher:
The registration API takes (name, doc, spec); the spec's closures are Arc<dyn Fn …> and the caller's source location is captured via #[track_caller] (see §5.11.1). Registration returns a typed id (MotionId / OperatorId / …):
let next_fn = registry.register_motion(
"motion:tree-sitter-next-function",
"Move to the start of the next function (tree-sitter).",
MotionSpec { apply: Arc::new(|ctx| { /* query tree-sitter → MotionResult */ }),
jump: true, exclusive: false, args_schema: vec![] },
);
registry.register_operator(
"operator:sort-lines", "Sort the lines in the range.",
OperatorSpec { apply: Arc::new(|ctx| { /* → Effect::Edits */ }),
repeatable: true, args_schema: vec![], blockwise_per_row: false },
);
Plugins use the forgery-safe register_plugin_motion(plugin_id, name, doc, spec) family (§9), which stamps SourceLayer::Plugin. Every registration returns an id usable in keymaps, in : invocations, and in plugin-to-plugin calls. Tree-sitter-driven structural motions and text objects are ✅ shipped — 16 of them (]f/[f/]c/[c/]a/[a/]l/[l + the af/ac/aa/al/aC objects) live in lattice-syntax, registered through exactly this extension point with no host changes.
The keymap-side companion of this extension point -- how plugins and user config bind chords to the ids returned here, layer priority across builtin / major-mode / minor-mode / user / per-buffer, and the trie merge cost on overlay push / pop -- is documented in docs/keymap-architecture.md §5 (extensibility) and §6 (layered registry).
Macros, marks, registers, and dot-repeat are mechanical because every change flows through execute(invocation). The last change for . is the most recent recorded CommandInvocation. Macros record resolved grammar Action sequences -- not raw keystrokes (MacroRecording { register, actions: Vec<Action> }). Replay survives keymap changes, plays back faster (no parse pass), and is editable as data. (📝 The buffer-backed *macro:q* editing view is design intent, not yet built.)
Visual mode IS the active region. When Visual is active, the current selection is automatically supplied as the range argument to any range-accepting command. Vim users see "operate on visual selection"; users coming from emacs see "operate on region." Both reduce to: the dispatcher receives range = Some(Range::Selection) when no explicit range is given and Visual is active. This is the Range::Selection variant added to the range type for exactly this purpose.
Multi-cursor (post-1.0). The selection set already permits it. Adding multi-cursor later requires per-feature semantic spec (which operators broadcast, how registers behave, how dot-repeat interacts) but no fundamental rework of the grammar, the dispatcher, or the command API.
5.2.5 Latency classes (the keystroke contract)
Every command's CommandSpec declares a latency class that pins how the runtime should schedule its work and what budget the CI harness will enforce:
pub enum LatencyClass {
Reflex, // sync Effect must commit within keystroke budget (<2ms p99)
Display, // sync Effect within "feel responsive" budget (~10ms p99)
Background, // no user-perceived sync budget; throughput-only
}
Status. The
LatencyClassenum is ✅ declared on everyCommandSpecand the cooperativeCancellationTokenis ✅ plumbed throughexecute(hot loops poll it; a flip returnsCommandError::Cancelledwith no commit). The deadline-timer enforcement + CI budget gates described below are 📝 not yet built — the class is advisory today. The event-bus completion choreography in "Events over invocation" is also 📝 aspirational: real insert-completion runs through thelattice-completionpipeline (§5.11.3) +gen:lspsources, not the 7-stepCompletionRequested → … → CompletionRankedbus flow sketched here.
Reflex. Single-stroke editing primitives: cursor motion, char insert, mode entry, dot-repeat, simple delete, scroll. Their evaluator must commit a sync Effect within the keystroke budget. The input loop awaits the Pending::effect receiver on the keystroke path; if it's not ready by the deadline, the dispatcher cancels (see below) and the keystroke completes without a commit.
Display. UI affordances that must appear immediately even when the data behind them is incomplete: open completion popup, open picker, open hover, post status segment, render last-cached search highlights. Their sync prelude commits a "shell" Effect (popup with placeholder, picker with no items yet, status segment with a spinner); the actual content arrives later via events.
Background. No user-perceived sync work. File-watcher tick, indexer pass, plugin housekeeping, LSP didChange debounce. Fire-and-forget; effects flow into snapshots whenever they're ready.
Events over invocation
When a Display or Background command needs work it cannot complete sync-fast, it MUST publish an event rather than synchronously invoke a follow-up command. Subscribers (other commands, plugins, UI surfaces, providers) consume the event and produce their own events as additional work completes. This composes:
- Multiple subscribers can react to the same event in parallel; no single command holds a chain of awaiting work.
- Adding a new participant means subscribing to an existing event, not rewriting the originating command.
- Late arrivals join the next snapshot's commit cycle without disturbing earlier ones.
- Sorting, ranking, deduplication, filtering are themselves subscribers -- they consume raw events and emit refined events. No central coordinator orders the pipeline.
Concrete shape for completion:
- User types
.. The Reflex command "insert char" commits its syncEffect-- the period is in the buffer. - A minor mode subscribed to
BufferEditrecognises a completion-trigger char and publishesEvent::CompletionRequested { document, position, version }. - The Display command "open completion popup" runs its sync prelude -- popup appears with a spinner. Kept short so the popup itself is sync-fast.
- The LSP client subscribes to
CompletionRequested; on receipt it issuestextDocument/completionasynchronously and publishesEvent::CompletionCandidatesArrived { document, version, items, source }when the response lands. - Plugin completion providers (snippet sources, dictionary sources, AI providers) subscribe to the same
CompletionRequestedevent and emit their ownCompletionCandidatesArrivedevents as they finish -- possibly out of order. - A ranking subscriber consumes raw
CompletionCandidatesArrived, applies sorting / scoring / deduplication, and emitsEvent::CompletionRanked { document, version, ordered_items }. - The completion popup view subscribes to
CompletionRanked; each arrival updates the popup content. Each update is a syncEffectcommitted to the next snapshot. The user sees the spinner replaced by candidates the moment they arrive, and the list refining as more sources finish.
The dispatcher and the event bus (§5.10) are the only coordination primitives. Plugin authors compose new behavior by subscribing -- never by direct cross-command calls.
Cancellation contract for Reflex
Every Reflex evaluator runs against a CancellationToken. The dispatcher sets a deadline timer based on the command's class budget; on expiry, the token is flipped. The evaluator must observe the flip and return promptly:
- Target: < 100us from token flip to evaluator return.
- Concrete pattern: poll
token.is_cancelled()once per loop iteration over a buffer scan, and after every host call from a WASM evaluator. The CI test harness verifies budget compliance under cancellation by injecting flips at adversarial times.
The same token is flipped by user-initiated cancellation -- pressing Esc during a long motion, or interrupting a regex motion hitting a pathological backtrack. The runtime treats both sources uniformly: a flipped token means "stop now." The user's Esc and the deadline timer are equivalent at the evaluator's level; the surface behavior (a flash "search interrupted" echo vs. silent abort) is differentiated at the input loop, not at the evaluator.
On cancellation, no Effect is committed. The document actor sees CommandError::Cancelled and skips the snapshot publish step. The atomicity property of §5.2.1 holds: edits, selection updates, and decoration changes are committed together or not at all. A cancelled Reflex leaves the document at the version the keystroke arrived at; the user perceives the keystroke as having had no effect -- the correct framing, since they cancelled it.
Display and Background commands cancel via the same token mechanism, but their deadline is class-appropriate (~10ms / no deadline), and their async tails carry independent cancellation tokens that are flipped when a newer same-event request supersedes them (a newer CompletionRequested cancels the in-flight LSP request from the prior one).
CI enforcement
Every registered command is benchmarked under criterion against a representative buffer corpus. The harness asserts:
- Reflex commands meet their < 2ms p99 budget on normal-size buffers (1k-10k lines) and degrade gracefully (cancel, not blow) on adversarial inputs (100MB log, regex with backtracking).
- Display commands' sync prelude is < 10ms p99.
- Reflex evaluators correctly observe injected token flips within 100us p99.
- Background commands have throughput targets but no latency assertions.
A command that fails its class's budget is a CI regression on the same gate that catches §8.2 commitments.
5.3 Syntax: Tree-Sitter Integration
Tree-sitter is responsible for all structural code understanding. Bundled grammars today: Rust, Python, JavaScript, Markdown (the last as a dual block+inline grammar, with code-block injections resolving fenced languages); more are added by registering into the process-wide LangRegistry.
Update flow (Option B + C-series, slices B.1–B.5 / C.1–C.5): edit → Buffer::apply_edit returns EditDelta (~2ns) → App accumulates deltas + synchronously shifts visible_highlights to track line/byte moves (slice C.3/C.4) → end of Action: input thread does Buffer::clone (O(1) Arc bump) + non-blocking mpsc::send → syntax worker applies tree.edit() per delta + publishes intermediate snapshot with shifted byte ranges (slice C.2) + Parser::parse(.., Some(&old_tree)) on spawn_blocking → atomic final snapshot swap via ArcSwap → renderer queries new tree on next frame. Renderer never sees empty/wrong spans during the parse window: held spans are line-aligned by C.3 and byte-aligned by C.4, so they paint correctly even before the worker publishes; the intermediate publish then makes the byte alignment "official" via the cache key invalidation. The single-line tree-shape staleness in the changed region settles within ~50µs (sub-frame at 60Hz). Layered guards (no cached tree, version mismatch, byte-length mismatch) fall back to full reparse without panic. Grammar-driven edits (operators) flow through the same chokepoint via slice C.5 so the byte-shift logic and didChange fire-out apply uniformly.
Highlight queries evaluated lazily on visible viewport + overscan, cached per (snapshot_ptr, text_version, viewport_range, fold_hash). Cache hit (steady-state norm: cursor blinking, no edit) is fast; cache miss runs the QueryCursor walk (parallelizing over patterns is post-1.0 work). Slice B.3 lit this up.
Structural motions: ]f/[f, ]c/[c, af/if, ae/ie. locals.scm for scope-aware rename.
Injections are first-class: markdown code blocks, JSX in JS, regexes in strings.
5.4 LSP Subsystem
We write our own client. tower-lsp is server-side; async-lsp brings tower middleware that doesn't fit our actor model. lsp-types (LSP 3.17) provides the wire types; the rest is hand-rolled. The companion docs (lsp-architecture.md for module-level commentary, ../../user/lsp.md for the user surface, ../notes/lsp-features.md for per-method tracking) elaborate beyond the design-relevant detail captured here.
5.4.1 Crate layout
lattice-lsp is a self-contained crate; lattice-host consumes its public API (attach is mode-owned, §5.4.3). The module split mirrors the data-flow stages:
framing-- LSPContent-Lengthheader parser. Pure; stream-agnostic. Default 64 MiB per-message ceiling guards against runaway servers.jsonrpc-- JSON-RPC 2.0 typedRequest/Response/NotificationwithRequestIdcorrelation. Now lives inlattice_protocol::jsonrpcand is re-exported aslattice_lsp::jsonrpc. Standard error codes (-32700..=-32600) plus LSP extensions (-32099..=-32000).codec-- tokioAsyncBufRead/AsyncWritecodec. Oneread_message/write_messageper LSP message; reuses scratch buffers so steady-state alloc count is one per round-trip (the bodyVecfrom serde_json).transport--tokio::process::Commandchild-process spawn withkill_on_drop; captures stdin / stdout / stderr;split()yields the codec halves + retainedChildso the actor's read / write loops own independent tasks.pending--Pending<T>(oneshot::Receiverwrapper parameterised overLspError). Mirrorslattice_runtime::Pendingsemantics: async-await, blocking_recv, or sync drop.error--LspErrorenum (Transport/Codec/Framing/Server/ActorGone/ResponseDropped/Cancelled/HandshakeFailed/ResponseDecode/NotInitialized) withis_fatal/is_retryableclassifiers.capabilities-- client capability advertisement (utf-8 preferred + utf-16 fallback, stale-request support, applyEdit, configuration, workspaceFolders, textDocument synchronisation + publishDiagnostics; per-feature buckets opened per phase) and negotiatedCapabilitiessnapshot.config--ServerConfig(binary, args, env, root markers, init options, file patterns, language id) + curatedbuiltin_servers()registry (rust-analyzer, pyright, gopls, typescript-language-server, clangd, lua-language-server) +resolve_workspace_rootwalking up for marker files.actor-- per-server tokio task;ServerHandleis the editor-facing analogue oflattice_runtime::DocumentHandle(clone-cheap, Arc-internal, syncrequest<P, R>/notify<P>/cancel(id)/shutdown()).position-- utf-8 ↔ utf-16 ↔ utf-32 column conversion (byte_to_lsp_characterdispatches per negotiated encoding; utf-8 short-circuits to 1ns).sync--DocSync: per-serverHashMap<Uri, DocState>.open/record_edit/flush/flush_all/close. HonoursTextDocumentSyncKind::{Incremental, Full, None}. Maintains aStringmirror per URI so utf-16 column conversion has access to BEFORE-state line text.diagnostics--DiagnosticEventtyped payload (server_id: Arc<str>,uri,version,Arc<[Diagnostic]>);DiagnosticsBusovertokio::sync::broadcast.diagnostics_layer--DiagnosticsLayerper-URI state container keyed by(uri, server_id)so multi-server scenarios don't overwrite. Apply-side version gating (event.version < prev.versiondrops); empty-list = clear semantics. Lookup APIs:diagnostics_for(uri)/diagnostics_on_line(uri, line)/line_severity(uri, line)/severity_counts()/snapshot().pump_diagnostics(layer, rx)is the supervisor's spawn-point.logging--LogLevelordered Trace < Debug < Info < Warn < Error;LogSource∈ {Client, Stderr, LspMessage, LspShowMessage, Trace};LogRingboundedVecDeque<LogRecord>;LspLoggerArc-shared facade with global ring + per-server rings + per-server min-level overrides + per-server trace toggle. Every emission also firestracing::*at the matching level soRUST_LOG=lattice_lsp=debugusers see the same stream.supervisor--LspSupervisor: registry ofServerConfigs + per-(workspace, server-id)actor map + per-actorDocSync+ per-Uriattachment list + shared logger + sharedDiagnosticsLayer.open_buffer/close_buffer/record_edit/flush/servers_for/attach_handle/shutdown(last one runs the LSP shutdown sequence per actor).
5.4.2 Three-task topology per server
+---------+ cmd via mpsc +-------+ Message via mpsc +-----------+
|App |------------------>|Actor |----------------------->|Write Loop |
+---------+ | | +-----------+
| | |
| | LspWriter
| | |
| | server stdin
| |
| | Message via mpsc +-----------+
| |<-----------------------|Read Loop |
+-------+ +-----------+
^ ^
| LspReader
Pending map: |
RequestId -> server stdout
oneshot::Sender
A separate stderr_drain task reads each line of ChildStderr and emits a Warn / Stderr record through the logger. Four tasks total per server, but the actor task itself does no I/O on its hot path — it owns coordination state and delegates reads / writes via channels. Three reasons for the split:
- Burst tolerance. A single-task design collapses when an indexer publishes hundreds of
$/progressnotifications while the editor is also sendingdidChangeper keystroke. Splitting reads / writes onto separate tasks lets the OS schedule them across cores; the actor stays cheap. - Bounded contention. The pending-requests
HashMapis touched only on the actor task — no locks. The writer isMutex-guarded only because the handshake's initialize request needs to fire before the loop owns it. - Crash containment. A panicked
read_loopdoesn't take thewrite_loopwith it; the actor observes thempsc::Receiver::recv()returningNoneand runs the cleanup path (drain pending withLspError::ActorGone, fire shutdown sequence).
5.4.3 Per-language-per-workspace lifecycle
The supervisor keys actors by (workspace_root, server_id). Implications:
- Two
.rsfiles in the same Cargo workspace share one rust-analyzer actor. Indexing happens once. Both buffers'didOpengo through the sameDocSync; the actor's pending table fans responses back to the right caller via JSON-RPC id. - Two
.rsfiles in different Cargo workspaces get two actors. Different indexed views; no cross-workspace navigation in v1. - Two distinct languages in the same workspace get two actors. Different
server_ids.
Spawn is lazy: the first didOpen for a (workspace, server_id) triggers actor::spawn. The handshake (initialize → initialized) completes before open_buffer's reply lands, so once the supervisor's mailbox dispatches the next request the actor is fully attached. Spawn failures (binary not on PATH, handshake error, server response malformed) are logged via the supervisor's logger and the relevant attachment is skipped without sinking the buffer-open.
Attach is event-driven AND mode-owned (paramount goal #4 + the mode-ownership rule). Both the initial document and every subsequent :e <path> follow one path: the publisher sets BufferId → Uri eagerly (the URI is a deterministic uri_from_path), publishes Event::DocumentOpened { id, path, version, text }, and returns immediately. The attach itself is owned by LspMode::on_activate (M-async.5): when the LSP minor mode activates on a buffer it subscribes to DocumentOpened and submits each path-bearing event to the supervisor's mailbox via LspSupervisorHandle::open_buffer, on the shared/LSP runtime. The earlier standalone lattice_lsp::attach_driver::spawn / App::build_lsp_subsystem wiring was retired in favour of this mode-owned path — a worked example of "modes own their full surface" (§5.8.2). The render thread never parks on the LSP initialize round-trip; servers attach in the background and diagnostics flow whenever a server finishes initialising.
A BufferId ↔ Uri map lives in the host; lattice-lsp is below the host in the crate graph and can't see BufferId, so the host threads URIs into the supervisor's API.
5.4.4 Document synchronisation
DocSync::record_edit(uri, edit) translates a lattice Edit { range, kind: Replace { text } } into an LSP TextDocumentContentChangeEvent:
- Look up the
(uri, server_id)'s mirror; read the lines atedit.range.start.lineandedit.range.end.line(BEFORE-state). - Convert
Position::byteto LSPcharacterviaposition::byte_to_lsp_characteragainst the negotiated encoding. - Build the change event with the converted range + the new text.
- Apply the edit to the mirror.
- Bump the per-doc version counter.
- Push to the per-doc pending queue.
flush(uri) honours the negotiated TextDocumentSyncKind:
- Incremental -- send the queued events as one
didChange. - Full -- drop queued events; send the entire mirror as one no-range change.
- None -- clear the queue; emit nothing.
The mirror is a String rather than a Rope because the LSP layer only ever splices one contiguous region per edit; per-line indexing is rare and bounded by line count. Mirror cost is one String per attached buffer per server — a few MB at most for a typical session.
flush is debounced by the App's idle timer (~50ms after the last keystroke). One didChange per debounce window coalesces typist-pace bursts into a single wire message. close flushes pending then sends didClose and drops the mirror.
5.4.5 Cancellation model
Every request returns Pending<T> (oneshot-backed; matches §5.2.1's envelope). Two cancellation scopes:
- Actor-internal.
ServerHandle::cancel(jsonrpc_id)resolves the matchingPendingwithLspError::Cancelledand fires$/cancelRequestso the server can free its scheduling slot. Advertised viageneral.staleRequestSupport.cancel. - Editor-driven supersession. When a newer same-flavour request supersedes a stale one (cursor moves before completion returns, etc.), the editor calls
cancelon the stale id. The dispatch site for each per-feature command owns the supersession bookkeeping.
Cancellation is cooperative: a misbehaving server can ignore $/cancelRequest and run to completion. The actor still resolves the local Pending immediately; the late response, if any, is logged and discarded.
5.4.6 Diagnostics pipeline
server ---publishDiagnostics---> actor
|
DiagnosticEvent ::=
(server_id, uri, version,
Arc<[Diagnostic]>)
|
DiagnosticsBus
(tokio::sync::broadcast,
cap 256)
|
+---------------------------------------+----------------------------+
| | | |
pump_diagnostics gutter glyph :diagnostics future
task per server provider (4.1.d.iii) buffer view plugins
| (4.1.d.iv)
DiagnosticsLayer
keyed by (uri, server_id)
+ version gating
+ multi-server merge
|
diagnostics_for(uri) diagnostics_on_line(uri, line)
line_severity(uri, line) snapshot() / iter_uris() / count()
The bus is the wire-side primitive; the layer is the editor-side state container. Multiple subscribers fan out on each event; a lagging consumer drops oldest first (correct: latest publish supersedes anyway).
The renderer threads through the layer per-frame:
- The gutter prepends a 1-cell severity column per line. The most-severe diagnostic on the line picks a glyph + colour (
■red Error,▲yellow Warning,●blue Information,·dim Hint). - The body has an underline overlay applied to each diagnostic range using ratatui's
Modifier::UNDERLINED+Style::underline_color. Composes with visual / hlsearch / current_match overlays without conflict.
The :diagnostics buffer is a help-style synthesised view: one row per diagnostic with a [severity] [path:line:col message](file:path:line) markdown link the existing help-link path knows how to follow.
]d / [d per-buffer navigation queries diagnostics_for(uri) sorted by (line, col) and walks; :cnext / :cprev (vim-quickfix aliases) currently scope per-buffer but are intended to walk the :diagnostics buffer's flat list (workspace-wide) once the cross-file jump lands.
5.4.7 Logging pipeline
Layered to mirror emacs's *lsp-log* / *<server> stderr* convention on lattice's everything-is-a-buffer surface:
*lsp*-- subsystem-wide events (supervisor: spawn / handshake / crash / restart) plus cross-server messages. Owned byLspLogMode.*lsp:<server-id>:<workspace-path>*-- per-instance: stderr lines (Warn/Stderr),window/logMessageandwindow/showMessagenotifications mapped by the server'stypefield,publishDiagnosticssummaries (Debug), lifecycle events, decode failures. The full workspace path is the canonical store key (multiple instances of the sameserver-idagainst different workspaces stay distinct); the display label may shorten with~/. Owned byLspServerLogMode.*lsp:<server-id>:<workspace-path>:trace*-- full JSON-RPC wire trace. Off by default; toggle per-instance.←(inbound) /→(outbound) markers + 240-char body excerpt. Owned byLspTraceLogMode.
These three modes own their buffers end-to-end per the B' contract (§5.10.5). The App holds no LSP-specific buffer-handling code; the ex-commands (:lsp-log, :lsp-server-log, :lsp-trace-log) provision the buffer through the mode-owned creation seam (§5.10.5) with the relevant mode id and activate it. On server detach the mode's Guard drops its event subscription; the buffer survives (frozen — no further appends) for post-mortem inspection. :bd removes it.
Producer: LspLogger.log(server_id, level, source, message) runs:
- Trace gate.
level == Trace+ per-server trace toggle off → return immediately. Toggle on → bypass the level filter (deliberate opt-in). - Level filter. Drop iff
level < effective_min(server_id). tracing::*fan-out. Always fires; survives without a subscriber.- Ring push. Append to global ring (server_id None) or per-server ring.
Per-record cost ≈ 91ns; trace-off short-circuit ≈ 9ns. Both Background-class.
:lsp-log [server] opens the relevant buffer; :lsp-trace <server> toggles + opens the trace buffer; :lsp-status shows running actors with capability summary; :lsp-log-level [server] <level> adjusts gating; :lsp-log-clear [server] drops the ring.
Why two pipelines (rings + tracing): rings serve buffer views and survive without an external subscriber; tracing fan-out lets power users drive RUST_LOG-style filtering, JSON log shipping, OpenTelemetry, etc.
5.4.8 Multi-buffer / multi-server topology
Two scenarios deserve explicit treatment.
Multiple buffers, separate servers per language. Each piece of LSP state is keyed by URI or (URI, server_id). Cross-buffer isolation is structural — there's no "active buffer" mutable register that features rewrite. ]d / [d are per-buffer; :diagnostics is workspace-wide as a buffer-backed list.
Multiple servers attached to one buffer. The supervisor keeps attachments: HashMap<Uri, Vec<(workspace, server_id)>>. Two servers attach when both ServerConfigs' file_patterns match. Per-feature merge (lands per phase as features arrive):
- Diagnostics: layer keyed by
(uri, server_id); readers merge across servers. Already shipped. - Hover (4.2): concat with
[server-id]labels. - Goto-def family (4.2): race to first non-empty; priority breaks ties.
- References / symbols (4.2): union with dedupe.
- Completion (4.2): each server is a
gen:lsp:<server-id>generator into the existinglattice-completionpipeline. - Code actions (4.3): union with picker entries prefixed
[server-id]. - Rename (4.3): merge
WorkspaceEdits; conflict → priority-resolved. - Formatting (4.3): single winner (priority).
- Semantic tokens (4.4): single server (multi-server merge deferred).
Server priority is a single integer per ServerConfig; default 100. Used by the few "single winner" features.
5.4.9 Crash recovery
The read_loop detects pipe close and ends. The actor task observes the inbound mpsc::Receiver::recv() returning None, drains pending requests with LspError::ActorGone, signals the supervisor. The supervisor restarts with exponential backoff (100ms → 5s; max 5 retries) and re-issues didOpen for every URI it was tracking under that (workspace, server_id). The diagnostics layer's per-server entries clear via clear_server(&id); other servers attached to the same buffer keep working.
Restart-with-backoff lives in the supervisor (host-side knowledge of which buffers were attached); the actor only signals "I'm gone." Both halves shipped: 4.1.h the actor-side detection, 4.4 the supervisor-side restart loop (restart_history + RESTART_WINDOW + max-restarts in supervisor.rs).
5.4.10 Performance characteristics
LSP requests are §5.2.5 Background-class — no sync-prelude budget. The wire layer is benched at the framing / encode / decode / position-conversion / logging level (crates/lattice-lsp/benches/lsp.rs); all sit in nanoseconds-to-microseconds. Selected numbers (Background targets):
| Operation | Time |
|---|---|
| Content-Length parse | ~77ns |
didChange encode | ~208ns |
publishDiagnostics decode | ~1.58µs |
| utf-16 column conversion (CJK line) | ~23ns |
LspLogger::log per record | ~91ns |
LspLogger::log trace-off short-circuit | ~9ns |
The §8.2 commitment is "LSP plumbing never shows up next to editor work in a flame graph." ../operations/benchmarks.md carries the full bench table.
5.4.11 Roadmap
Features land in this order: diagnostics → completion + resolve → hover → definition / declaration / typeDefinition / implementation / references → documentSymbol / workspace.symbol → codeAction → rename → formatting (full + range + on-type) → signatureHelp → semanticTokens → inlayHint → foldingRange → documentHighlight → callHierarchy + typeHierarchy → codeLens → documentLink → inlineValue → inlineCompletion. Per-method status in ../notes/lsp-features.md.
5.4.12 Non-goals (v1)
- Notebook documents -- post-1.0; needs the rich-buffer rendering work in §5.6 + a notebook-aware buffer kind.
- Multi-root workspace folders beyond the initial root -- post-1.0; v1 is single-root per actor, multiple actors handle multiple roots.
- Server-side LSP -- lattice talks to servers, doesn't host one. The grammar API (§5.2) is the canonical extensibility surface for in-process commands.
5.5 Plugin Subsystem
Two principles: plugins are actors; the API grows from real plugins.
Runtime: wasmtime + Component Model + WASI. WIT interfaces are the canonical plugin API; plugin authors target any language with component-model toolchain support (Rust, Zig, Go, AssemblyScript, etc.). Sandbox guarantees: memory isolation, fuel limits, capability-based filesystem / network / subprocess access, crash isolation. A trapping plugin does not take down the editor.
Plugin types:
| Type | Purpose | Examples |
|---|---|---|
| Major mode plugin | Defines a content type's identity | rust-mode, markdown-mode |
| Minor mode plugin | Composable feature toggle | git-blame-mode, auto-pair-mode |
| Grammar extension | New motions, text objects, operators, ex-commands | tree-sitter-motions, git-hunk-objects |
| Buffer-backed view | Content provider for non-file buffers (file tree, outline, etc.) | file-tree, diagnostics-list |
| Feature plugin | Standalone command/UI contribution | fuzzy-finder, format-on-save |
A single plugin component can register any combination of these.
5.5.1 Concurrency model
Plugins compose with the §5.7 multi-threaded async architecture:
- Each plugin instance owns its own
wasmtime::Store. Stores are independent; there is no global plugin lock. - Each Store runs as one or more tokio tasks on the multi-thread runtime. Two plugins doing CPU work execute on two cores in parallel.
- The async ABI is the canonical pattern. Host functions are async; when a plugin calls one, the WASM stack suspends and the OS thread is released to run other tokio tasks. Wakeup resumes the plugin.
- The UI thread never invokes WASM directly. Plugin work is scheduled on the core executor; results flow back as events.
- A single Store is
!Sendwhile executing, but yields at every host call. Between yields the OS thread runs other tasks. This is normal tokio multi-thread executor behavior; no thread is pinned. - Fuel exhaustion traps cleanly. A runaway plugin's task is killed; other plugins, the document actor, the UI, the LSP clients all keep running.
- Intra-plugin native threading is out of scope.
wasi-threadsis experimental; the Component Model is moving toward the async ABI for concurrency. Plugin authors compose async tasks via host primitives instead of spawning OS threads. Heavy compute belongs in host APIs (tree-sitter, ripgrep), called from plugins.
The architectural rule "no plugin can stall the UI thread, ever" holds by construction: there is no synchronous path from UI input to plugin code.
5.5.2 Performance discipline
WASM call overhead is real but bounded. Ground rules:
- AOT compilation. Wasmtime + Cranelift compiles plugin modules ahead of time at install. Per-instantiation cost is the cost of allocating linear memory and resolving imports, not codegen.
- Module cache on disk. Re-installs and editor upgrades reuse compiled artifacts.
- Lazy instantiation. A plugin that is never invoked is never instantiated. 50 installed plugins do not contribute 50 instantiation costs to startup.
- Resource handles, not copies. Buffers, documents, ranges, edits cross the WASM boundary as opaque handles. Plugin code does not receive a copy of the rope; it calls back into native APIs to read slices.
- Native host APIs for hot work. Tree-sitter parsing and queries, ripgrep, regex (
fancy-regex), file I/O, HTTP — all native, exposed to plugins as host functions. - Built-ins stay native. Default vim motions, text objects, operators, registers, ranges, and the dispatcher live in
lattice-grammar(native Rust). The default vim keymap never crosses the WASM boundary. WASM exists for user/plugin extensions. - Plugins emit data, not draw calls. Status segments return
SegmentContent; gutter providers returnGutterContent; decorations are styled ranges. There is no per-frame WASM call. - Coarse APIs. Where a plugin would otherwise want a per-byte stream, the API takes a range and returns a slice / iterator backed by host memory.
Per-call overhead budgets (CI-enforced):
| Call class | p50 | p99 |
|---|---|---|
| Typed host function call | < 100ns | < 500ns |
| Grammar-extension round-trip (motion/text-object/operator) | < 1μs | < 5μs |
| Status / gutter segment update | < 10μs | < 50μs |
| Picker filter pass per item | < 500ns | < 2μs |
| Major-mode event handler | < 50μs | < 250μs |
Cold-start budget. Editor startup contribution from 50 lazily-loaded plugins: < 30ms total (instantiation amortized as plugins are first invoked). First-paint plugin work (status segments visible at startup): < 5ms total.
What this means concretely. A motion evaluator implemented in WASM that queries a host tree-sitter API costs ~5μs round-trip; called once per relevant keypress, this is 0.25% of the < 2ms keystroke budget. The bottleneck is the parser, not the WASM boundary.
5.5.6 Bundled plugins
📝 Status (Phase 8 / 8b). The plugin host is ✅ built (Phase 7: Component Model, WIT seams, capability/fuel/epoch, WASI-fs isolation, AOT cache, per-call CI budgets). But it is not yet wired into the editor (
lattice-hostdepends only on the wasmtime-freelattice-plugin-apicatalog, guard-tested) and no bundled plugins ship —plugins/fuzzy-finderis a seam-parity test fixture, not a loaded plugin. Everything below is the forward plan. Note much of this functionality already ships natively (fuzzy-finder =lattice-picker, grep + git =lattice-diff, snippets =lattice-snippet, outline =:lsp-symbols); the not-done work is repackaging it as WASM components + the editor-side loader (§13, Phase 8).
Updated (2026-07-20): "bundled" = core plugins, and they are NOT compiled into the binary. The distribution model was settled in
plugin-manager.md: plugins ship separately (rejectinginclude_bytes!), on two roots — a runtime root of prebuilt core-plugin.wasmbeside the binary (found via a search path, discovered at boot), and the user root~/.config/lattice/plugins/fed by a use-packagerequire+ build-on-boot layer. Core plugins are enabled by a<id>.enabledconfig gate. auto-pair is the first core plugin, shipped (the whole stack proven end to end). Read the paragraphs below with that model; theinclude_bytes!/core-plugins/<name>.wasm-only phrasing is historical.
Lattice ships with a curated set of core plugins -- WASM Component Model packages that ship with the editor (prebuilt, staged into a runtime root beside the binary; NOT compiled into it) so they're available without a separate install step. They are the same shape as user-installed plugins; they just have a higher trust default, zero install friction, and are enabled by a config gate.
The strategy: features that aren't architecturally core but are essential to ship feature-complete out of the box live here. Core stays narrow (buffers, modal grammar, command registry, renderer trait, runtime, plugin host); editor-quality wins (LSP server management, project-wide search, version-control UIs, snippets, surround / comment / auto-pair editing helpers) ship as bundled plugins. This dogfoods the plugin host on real workloads, gives third-party plugin authors high-quality reference implementations to study, and keeps the plugin API surface honest.
Trust distinction. Bundled plugins inherit the editor's trust level -- their capabilities are pre-granted at build time, no per-install consent prompt. User-installed plugins (via the bundled plugin manager) go through capability prompts on first install. Plugin manifests declare requested capabilities (fs:write:install_dir, net:http, proc:spawn, ...); the runtime gates accordingly.
Bootstrap (as built — plugin-manager.md). Core plugins live as prebuilt .wasm in the runtime root (<runtime>/plugins/<name>/, resolved via $LATTICE_RUNTIME → install prefix → exe-relative → dev workspace) — not include_bytes! (rejected: plugins ship separately, versioned independently of the binary). At boot the host discovers + instantiates them with pre-granted (bundled-tier) capabilities and a <id>.enabled gate; the loader then handles user plugins from ~/.config/lattice/plugins/, declared via a use-package require (git/local source, built on first boot into that cache).
Bundled-plugin candidates (Phase 8 -- post-Phase-7 plugin host; concrete inventory in docs/../operations/implementation.md):
- LSP server manager -- install / update / uninstall LSPs into a managed
${XDG_DATA_HOME}/lattice/lsp/<name>/<version>/tree; bundled registry of common servers; SHA-pinned downloads. Lighthouse implementation -- the first non-trivial bundled plugin we build, validating that the WIT surface is sized correctly. - Plugin manager -- install / update / uninstall third-party plugins; capability-prompt UX.
- Project / workspace fuzzy-finder (Telescope / fzf-lua equivalent).
- Project-wide grep (ripgrep wrapper, results-as-buffer).
- Git client (magit-style — git ops as buffers).
- Snippet engine (LSP-spec snippets + custom).
- Editing helpers: comment toggle, surround, auto-pairs, multi-cursor.
- Diff viewer / merge tool.
- Outline / symbols sidebar (consumes LSP
documentSymbol). - Format-on-save controller.
- Test runner integration.
- Markdown preview.
WIT prerequisites that this design imposes on Phase 7's plugin host (the first three are blockers for the LSP server manager specifically):
LspSupervisormutation through WIT -- plugins registerServerConfigs pointing at paths under their managed install dir.ServerConfigbecomes a stable WIT type.- Filesystem capability scoped per-plugin --
${XDG_DATA_HOME}/lattice/plugins/<plugin-id>/data/mounted viawasi:filesystem; writes outside it require an explicit broader capability. - Network capability --
wasi:http(preview2), gated; consent prompt on first install. - Subprocess capability -- contentious, since "spawn arbitrary process" approximates "trust this plugin completely". v1: bundled plugins only get
proc:spawn; user-installed plugins ship pre-built binary recipes (no source-build paths). Sandboxed subprocess primitives are post-1.0. - Long-running task surface --
start_task → push_output → finalizeso plugin-driven installs / scans stream stdout into a buffer-backed view without blocking the renderer. - Ex-command registration through WIT (already in §5.2.1's plan; called out here as a load-bearing dependency).
- §5.12 typed-options registration through WIT -- plugins register
lsp-manager.install_root,lsp-manager.github_token, etc. into the sameConfigRegistrycore options live in.
8b design (in progress, 2026-07): the first-wave bundled plugins now have design fragments.
plugin-auto-pair.md— the trivial-first plugin (auto+manualpairing) — forces two more general host seams beyond the above (a grammar-contextdocumenthandle so a grammar action can read buffer text, and declining / fall-through bindings), plus theplugin-treesitter-seam.mdquery seam (promoted to v1 — foundational to the tree-sitter-driven grammar of paramount #3; it publishes the host's parse tree to plugins so structural motions / text objects / folds can be plugins).lighthouse.md(the LSP server manager) is where prerequisites 1–5 above land as an implemented host-services extension. Slice plans:../operations/slice-plans/(plugin-auto-pair/plugin-treesitter-seam/lighthouse).
5.5.7 Editor-side loading, the manager view, and observability
Built (Phase 8). The runtime (§5.5.1–5.5.2) is Phase 7; this subsection is the editor-side surface that makes it reachable. Detail lives in the fragments
plugin-host.md(the seam spine + capability model) andplugin-observability.md(the trace stack); slice sequencing in../operations/slice-plans/plugin-loader.md+plugin-observability.md.
Loading. The lattice-plugin-loader crate stands the host up at boot as a PluginLoaderHandle service and drives compile → instantiate → activate off the boot thread. On-disk discovery loads plugins from ${XDG_DATA_HOME}/lattice/plugins/; :plugin-load <path> / :plugin-unload <name> / :plugin-reload <name> load / reverse / re-instantiate on demand. The user's init.rs is itself a boot-capability plugin (keymaps / autocmds / custom commands as code), auto-reloaded on rebuild. Unload reverses every registry contribution by provenance; reload mints a fresh, untripped quarantine.
The :plugins manager view. A read-only buffer (everything-is-a-buffer) listing every loaded plugin's health / tier / capabilities, live-updating on PluginCrashed, with in-view chords (r reload, x unload, K describe, gr refresh, t open trace, T cycle trace verbosity). The mode owns both its chord bindings and their handler bodies (the mode-ownership rule) — zero host Editor:: methods.
Observability (the boundary is the seam). Because every plugin interaction is host-mediated, the host records every host↔guest call — name, timing, fuel, result-or-trap, capability denials — independent of the guest's source language. This boundary trace is lattice's plugin debugger. It is never on the editor hot path: records stream via the event bus (the LSP-log precedent), formatted + appended off-thread; the one synchronous seam (grammar) gates on a per-plugin relaxed-atomic HotGate that costs ≈0 when off (benched). Surfaces: the *plugin-trace* firehose + per-plugin *plugin-trace:<name>* buffers (:plugin-trace / the manager t drill-in), a live typed plugin.trace-level option (global + per-plugin), and a wasi:logging-shaped guest import so a plugin emits its own narrative into the same buffer (Layer 2).
5.6 Rendering -- The Layered Architecture
5.6.1 The Renderer trait + the cell-grid substrate
Reconciliation note (v0.6). The original design (below, "Superseded approach") specified a rich per-buffer
Renderertrait withEditorRenderer/DocumentRendererimplementers, four selectable GPU fast paths, anaccessibility_tree, a taffy-basedDocumentRenderer, and a v1 sprite atlas. That is not what shipped. What shipped is the cell-grid /DisplayMatrixarchitecture described here.
The renderer is split into a renderer-neutral cell-grid substrate (built off the render thread) and thin peer renderers that paint it. The trait a frontend implements is a marker that surfaces its renderer-specific associated types to the host's boot/dispatch machinery — it is not a per-buffer draw API:
// lattice-host
pub trait Renderer {
type Theme; // frontend-specific theme cache
type PaneRenderRegistry; // frontend-specific per-kind provider table
}
| Impl | Crate | Role |
|---|---|---|
TuiRenderer | lattice-ui-tui | ✅ terminal peer (ratatui + crossterm) — first-class, headless / SSH |
GpuiRenderer | lattice-ui-gpui | ✅ GPU peer (GPUI; gui/window cargo features) |
MinimalRenderer | lattice-host (tests) | headless test harness |
| Web renderer | — | 📝 deferred placeholder |
The shared substrate is the cell grid (lattice-cells). A per-pane DisplayMatrix (rows of styled CellMatrix cells, plus virtual rows) is built off the render thread by a cells worker (on spawn_blocking) from the document snapshot + syntax spans + decorations, and published through RenderState (§5.7.2). Both peers consume the same DisplayMatrix: the TUI maps cells to ratatui spans; GPUI shapes each line via WindowTextSystem::shape_line against a glyph atlas. There is no per-buffer Renderer v-table, no EditorRenderer / DocumentRenderer split, and no lattice-render* crate — one grid, two painters.
Both peers are held to the same latency invariant: the paint body reads a published snapshot and does zero I/O / parsing / shaping-of-unchanged-content per frame (§5.7). Element fan-out is O(viewport rows), never O(chars).
The renderer reads mode-resolved options for every per-frame decision (wrap, line numbers, gutter width, foldcolumn, decoration providers). There is no The original design specified a BufferKind branch in the render path: every buffer flows through the one cell-grid pipeline, parameterised by its ResolvedOptions snapshot. The hover popup is a floating-geometry view of a markdown-mode buffer with a hover-mode minor contributing wrap=true, line-numbers=false, anchor=cursor -- not a separate render code path. See docs/mode-architecture.md §6.1 (resolution layers), §6.3 (caching), and §9.5 (renderer integration).Superseded approach (original v0.x design — not built)
trait Renderer { type Content/Event/Config; set_content / handle_input / layout / paint / accessibility_tree / invalidate } with EditorRenderer (editable buffers) and DocumentRenderer (popups/pickers, taffy-based) implementers, plus CanvasRenderer / WebRenderer. This was retired: the peer renderers instead consume a shared DisplayMatrix, so per-buffer paint logic lives once in the cell-grid substrate rather than behind a trait each frontend re-implements. An accessibility tree was never built. The lattice-render / -render-editor / -render-document crate trisection (§11) was likewise dropped.
The renderer reads mode-resolved options for every per-frame decision (wrap, line numbers, gutter width, foldcolumn, statusline contributors, decoration providers). There is no BufferKind branch in the render path: every buffer flows through one pipeline, parameterised by its ResolvedOptions snapshot. The hover popup is a floating-geometry view of a markdown-mode buffer with a hover-mode minor contributing wrap=true, line-numbers=false, anchor=cursor -- not a separate render code path. See docs/mode-architecture.md §6.1 (resolution layers), §6.3 (caching, with O(1) hot-path reads), and §9.5 (renderer integration).
5.6.2 Building the cell grid (the content path)
The DisplayMatrix for each pane is built off-thread from the published snapshot: walk the visible line range (+ overscan), fetch the cached highlight spans and the decoration/virtual-row contributions for those lines, and emit styled cells per line (CellMatrix) plus any virtual rows. This runs on a cells worker via spawn_blocking and publishes through RenderState; the render thread only reads the finished matrix. Fan-out is O(viewport rows).
Painting the matrix differs per peer:
- TUI (
lattice-ui-tui) maps each cell to a ratatui styled span — monospace, 24-bit color where supported (8-color fallback), damage-tracked (not full-redraw-per-frame). - GPUI (
lattice-ui-gpui) shapes each visible line viaWindowTextSystem::shape_lineagainst a glyph atlas and emits GPUI elements — variable fonts + sub-pixel positioning available, but the content is the same cell grid.
A Document.rendering_profile field exists as data (a per-buffer hint), but the renderer does not select among four separate GPU pipelines — the four "selectable fast paths" of the original design were not built. Incremental highlight + incremental cell build keep the steady state flat to ~100k lines; soft-wrap is supported on both peers.
📝 Planned rich-buffer rendering. Per-line variable-height shaping with a
LineLayoutcache + a Fenwick cumulative-height index (for markdown/org with mixed font sizes), and Path-4 inline media blocks (images, LaTeX, charts), are not built. Markdown today renders as styled monospace cells with syntax highlighting; rich per-line geometry and inline blocks are post-1.0.
5.6.3 Popups, pickers, and UI furniture
There is no separate taffy-based DocumentRenderer. Popups (completion, hover, signature, diagnostics), pickers, and buffer-backed panels are all Documents (§5.9.8) rendered through the same cell-grid pipeline in a popup-shaped view — the hover popup is a markdown-mode buffer in floating geometry, not a bespoke text-layout path. This is what makes syntax highlighting, :help-consistent rendering, and vim motions work uniformly inside popups. The work is off the input path and cannot affect editor responsiveness.
5.6.4 Cursor positioning and logical/visual translation
Core, plugins, and command dispatcher deal exclusively in logical positions (line, byte). Renderer translates to visual positions when drawing. Plugins never see pixels.
5.6.5 Compositing model
Window is a single GPU surface. Renderers render into regions. All renderers share one GPU atlas, one shader pipeline, one frame submission. A frame containing an editor pane plus a popup plus a buffer-backed view in another pane costs roughly the same as a single editor pane alone.
5.6.6 Future WebRenderer placeholder
Deferred indefinitely. Architecturally the trait accommodates Servo embedding, system webview (wry/WebView2/WKWebView), or CEF. Decision punted.
5.6.7 Iconography and sprites (📝 planned; text glyphs today)
📝 Status. The
SpriteSet/SpriteSpec/SpriteSource/Decoration::InlineSpritetypes below do not exist. Icons today are nerd-font text glyphs with a BMP-block fallback (Geometric Shapes + Misc Symbols) — theui.nerd_fontsoption toggles between the two palettes, which occupy the same cell width so column geometry is stable. The rasterized sprite atlas is the forward design.
Small graphical elements -- file-type icons in the file tree, severity icons in the diagnostics list and gutter, language logos in tab strips, status-line indicators (LSP healthy / sick, git branch), picker leading icons, notification level badges. The planned design makes them atlas-backed, line-height-sized, and rendered through the same GPU pipeline as glyphs (not inline media blocks).
Sprite atlas
A separate-from-glyphs sprite atlas holds rasterized small graphics. Sources are SVG (preferred, multi-resolution) or PNG (for fixed pixel art). At load time, plugins register SpriteSets; the host rasterizes each sprite at the device's display scale (with extra resolutions for HiDPI / scale changes) and packs them into the atlas.
pub struct SpriteSet {
pub id: SpriteSetId,
pub plugin: PluginId,
pub sprites: Vec<SpriteSpec>,
}
pub struct SpriteSpec {
pub id: SpriteId, // namespaced: "git-gutter:added"
pub source: SpriteSource, // Svg(bytes) | Png(bytes) | Builtin(name)
pub default_tint: Option<Color>, // None = use source colors
pub themable: bool, // tint follows current theme on render
}
pub enum SpriteSource {
Svg(Vec<u8>),
Png(Vec<u8>),
Builtin(BuiltinIcon), // a curated set ships with the editor
}
Where sprites appear
Sprites are referenced from the same primitives that already exist; no new rendering path:
- Decorations. A new variant
Decoration::InlineSprite { sprite: SpriteId, tint: Option<Color> }places a sprite at a byte offset in any buffer. The file tree's per-line file-type icon is exactly this -- a per-line decoration on the file-tree buffer. - Gutter segments.
GutterContent::PerLineitems can hold sprites instead of (or alongside) text glyphs. Diagnostic markers, breakpoint indicators, and git diff markers become sprites with optional tints. - Status segments.
SegmentContent::Sprite { sprite, tint }and the existing composite type let mode-line / header-line indicators show icons. - Picker items.
ItemRenderercan return rich content with a leading sprite (file-type icon in the file picker, symbol-kind icon in the symbol picker). - Tab strip. Each tab's title may carry a leading sprite (the active document's language icon).
- Notification level badge. Info / Warn / Error each have a default sprite; users / themes override.
Bundled icon set + plugin sprites
A curated builtin-icons sprite set ships with v1, covering: 60+ language file-type icons, common file kinds (folder, symlink, hidden, executable), LSP symbol kinds, severity levels (error/warn/info/hint), VCS markers (added/modified/deleted/conflict), generic UI primitives (close, expand, collapse, search, settings).
Plugins ship their own sprites. The git-gutter plugin registers git-gutter:added, git-gutter:modified, git-gutter:deleted etc. Users override individual sprites in their TOML config:
[icons]
"file-types.rust" = "/path/to/custom-rust.svg"
"git-gutter.modified" = { source = "/path/to/glyph.svg", tint = "#cccccc" }
Performance
- Atlas allocation. Sprite atlas separate from glyph atlas; both share the GPU pipeline. Default cap 32MB; LRU eviction.
- Resolution variants. Each registered sprite is rasterized at 1x, 2x, and 3x device scale up front (for HiDPI). Re-rasterized on display-scale change (rare). Cost: <50us per sprite per resolution; amortized at startup or first-use.
- Decoration overhead. A sprite decoration is one extra atlas lookup + one quad emit. No measurable impact on the monospace fast path -- the file tree with 200 visible icon decorations renders in the same frame budget as a 200-line code buffer.
- Theme tinting. Tints are per-draw-call uniforms; no atlas re-rasterization needed for theme switches.
Why this is not Path 4
Sprites fit in line height. They participate in the existing decoration + gutter + status + picker pipelines. They share the GPU pipeline with glyphs. They do not change line layout. Path 4 (inline blocks) is for non-line-height media that affects the cumulative-height index and requires per-block layout work -- a strictly more complex problem deferred to post-1.0.
5.6.8 Render-snapshot coherence (the core / renderer contract)
Every frame the renderer reads a coherent view of (buffer text, syntax, decorations, selections). These pieces live on different actors -- text and selections on the document actor, syntax trees on spawn_blocking workers, decorations from any source. The renderer cannot acquire a lock the document actor holds, and it cannot afford a synchronous "give me a snapshot" round-trip into the actor: that round-trip would put the keystroke-to-glyph budget at the mercy of the actor's mailbox depth.
The contract between core and renderer is therefore publish-versioned, copy-on-write snapshots: the document actor publishes an immutable DocumentSnapshot; the renderer reads the latest one with one atomic load per visible document at frame start, and uses that snapshot for the entire frame. The real snapshot carries only text + selections + version metadata — syntax spans, decorations, and the built cell grid publish through the separate RenderState sub-states (§5.7.2), each an arc-swap cell, and the cells worker composes them into the DisplayMatrix. Coherence still holds because each publish is a single atomic store.
pub struct DocumentSnapshot {
pub id: DocumentId,
pub version: u64, // bumps on any change
pub text_version: u64, // bumps only on text mutation
pub buffer: Buffer, // O(1) rope clone
pub path: Option<PathBuf>,
pub dirty: bool,
pub selections: SelectionSet,
// NB: syntax / decorations / cell-grid are NOT fields here — they
// publish via RenderState sub-states (§5.7.2) and the cells worker.
}
/// One atomic-load-published-pointer per document. arc-swap is the
/// canonical primitive; `load()` is wait-free.
pub struct PublishedSnapshot(Arc<arc_swap::ArcSwap<DocumentSnapshot>>);
Publish discipline (actor side)
The document actor holds the writable state. On every committed Effect, it constructs a new DocumentSnapshot:
- Fields not affected by the commit are
Arc::cloned (one word each). - Fields affected by the commit are rebuilt from the new state -- but most rebuilds are cheap:
RopeSnapshotandSelectionSetareArc-cloned plus a small mutation;DecorationLayeris rebuilt with the structural-sharing trick (a persistent map /im::OrdMap) so unchanged decorations are not copied. - The actor does an atomic
store_releaseon the published pointer.
Snapshot construction p99 budget: < 10 us for buffers up to 100MB. No syscalls, no allocations beyond the changed fragments.
Late arrivals from other actors -- tree-sitter completing a parse on a spawn_blocking worker, a plugin publishing decorations, the cells worker finishing a DisplayMatrix build -- submit their result to the document actor (or RenderState) as an event. The actor folds the result into the next snapshot it publishes. Workers never publish snapshots themselves.
Renderer discipline (read side)
At frame start, each renderer instance does one arc_swap::Cache::load per visible document. The returned Arc<DocumentSnapshot> lives for the duration of the frame. All subsequent reads -- line text, span styles, decoration ranges, selection extents, shaped glyphs -- go through that snapshot. There are no additional loads, no actor round-trips, no per-line locks, no lifetime ambiguity.
At end-of-frame the Arc drops; if the actor has since published newer snapshots and no other reader holds the old one, it's reclaimed. Lock-free reclamation is arc-swap's job.
Coherence guarantees within a snapshot
- Text + selections + decorations are always coherent with each other, because they all commit through the actor's single publish step. A selection at byte 100 corresponds exactly to that byte in this snapshot's rope, even if the actor has since committed an edit that would shift it.
- Syntax may lag the text by one snapshot: if the actor publishes snapshot N before the parse for N has landed, snapshot N carries the tree from version N-1. The renderer treats this as "highlights are one snapshot stale," consistent with §5.6.2's existing claim that one-frame-stale highlights are accepted.
- Layout cache may lag similarly for shaped buffers; the renderer falls back to the prior
LayoutCacheSnapshotfor one frame when shaping isn't done.
Cross-pane: same document, different snapshots
Two panes rendering the same document at the same vsync may capture different snapshots if their frame work straddles a publish. This is intentional. Forcing same-version across panes would require a global frame fence that holds back the leading pane's render until the trailing one is ready -- the wrong tradeoff against latency. Visually, pane B may render one snapshot behind pane A; this is below human perception at >= 60Hz.
Multi-pane selection transformation under remote edits
As-built: the snapshot /
ArcSwap-published coherence core is built; the multi-pane selection-transformation described here (cross-pane divergent snapshots, transforming selections owned by other panes) is the design contract but is n/a in the single-pane-per-document v1 — it activates when the same document is edited-and-rendered concurrently across panes.
When the actor commits an edit, the resulting AppliedEdit is applied to all open selections on that document (including selections owned by panes other than the one that issued the edit) before the next snapshot is published. The transformed selections become part of the new snapshot's Arc<SelectionSet>. Panes whose next frame uses that snapshot see selections at correct positions; panes using an older snapshot continue to see selections at the older positions -- which are still internally coherent with that older snapshot's text. There is no "selection points at byte 100 but the text shifted" torn state, ever.
This is the property emacs's marker objects provide and that vim doesn't need (vim never edits-and-renders concurrently). It pins §15:12 (multi-window state synchronization).
Why arc-swap specifically
arc-swap is an RCU-flavored primitive: lock-free read, atomic publish, refcount-based reclamation on last drop. We name it explicitly so the implementation is not free to swap to a flavor with different visibility rules (full-fence atomics across an unrelated mutex, per-thread epoch tables) that would change the renderer's correctness model. The required semantics are:
- Renderer reads a published snapshot with
load-acquireordering -- the read sees all writes the publisher ordered before itsstore-release. - Actor publishes with
store-release-- prior writes to the snapshot's interior are visible to any reader observing the new pointer. - Reclamation is by refcount drop, not by epoch fence -- the renderer's
Arc<DocumentSnapshot>keeps the snapshot alive for as long as it needs it, regardless of how many newer snapshots the actor publishes in the meantime.
Memory cost
A DocumentSnapshot is approximately: six Arc words (~48 bytes on 64-bit) plus the changed-fragment costs of the underlying immutable structures. Per-document overhead with one snapshot in flight + one being constructed: ~200 bytes regardless of file size, because Arc clones do not copy underlying B-trees. Even a 100MB buffer is one shared rope tree.
Snapshot retention: the actor keeps one published snapshot live; older ones are dropped when no reader holds them. Renderers naturally release at end-of-frame. A frozen renderer (e.g. a debugger has stopped the UI thread) would pin one old snapshot, but this is bounded -- the actor keeps publishing newer ones; old snapshots are released when the renderer thaws.
Performance contract
| Operation | Target (p99) |
|---|---|
| Snapshot publish (actor side) | < 10us |
Snapshot load (renderer side, arc_swap::Cache::load) | < 5ns |
| Whole-frame: locks held by renderer | 0 |
| Whole-frame: actor round-trips by renderer | 0 |
The renderer's frame budget (§8.2) treats snapshot acquisition as a fixed cost in the single-digit-nanoseconds range, freeing the rest of the budget for actual rendering work.
This is the load-bearing async invariant of the editor. Every other piece of the architecture -- actor mailboxes, dispatcher async returns, plugin async ABI, event bus -- assumes the renderer has a frozen, coherent view to work against. That assumption is what this section pins.
5.7 Async Runtime and Threading
| Component | Runs on |
|---|---|
| UI event loop | Dedicated UI / render thread (one per peer) |
| Grammar dispatch + buffer mutations | The editor actor (a current_thread tokio task; one document actor per doc) |
| Tree-sitter parses | spawn_blocking |
Cell-grid (DisplayMatrix) build | spawn_blocking (cells worker) |
| 📝 Text shaping (rich buffers) | spawn_blocking (planned; not built) |
| LSP I/O | tokio tasks (shared / LSP runtimes) |
| LSP file-watcher | Dedicated tokio task on the LSP runtime (§5.7.4) |
| Plugin instances | tokio tasks (wasmtime async), one Store each |
| Search / diff / index | spawn_blocking |
| File I/O | spawn_blocking |
The actor pattern for documents: each open document owned by one tokio task; mutations via mpsc; reads via O(1) rope snapshot. No locks on the hot path.
5.7.1 The architectural enforcement of paramount goal #4
"Nothing blocks the UI -- enforced architecturally, not by discipline." The UI thread reads input, reads RenderState, renders. Nothing else.
Every other editor we measured against (vim, neovim, emacs, helix, zed) keeps the UI thread free by discipline: contributors are expected to know which operations might block and route them off-thread. Lattice instead chooses primitives that make UI-thread blocking physically impossible in the steady state. Three primitives carry the load:
RenderState-- the renderer's wait-free read contract with the host (§5.7.2).Arc<ArcSwapOption<T>>/PerBufferCache<T>-- the publication primitives background tasks use to write results directly into renderer-visible state without touching&mut Editor(§5.7.3).- Subsystem-task ownership of channel rxes -- LSP file events, request responses, and similar streams are consumed by dedicated tokio tasks on the LSP runtime, not by per-tick drains on the UI loop (§5.7.4).
The compounding property: every renderer-coupled subsystem migrates onto these primitives the same way, so the architecture is uniform by the time a new feature lands.
5.7.2 RenderState -- the renderer's read contract
Renderers do not read Editor fields directly during render. They consult a typed RenderState snapshot published by the dispatcher at the end of every tick. The snapshot is wait-free for readers (one ArcSwap::load) and decoupled from whoever is mutating Editor:
// lattice-host/src/render_state.rs
pub struct RenderState {
pub active_document: Arc<ActiveDocumentRenderState>,
pub buffers: Arc<BuffersRenderState>,
pub panes: Arc<PanesRenderState>,
pub lsp: Arc<LspRenderState>,
pub syntax: Arc<SyntaxRenderState>,
pub picker: Arc<PickerRenderState>,
pub completion: Arc<CompletionRenderState>,
pub popup: Arc<PopupRenderState>,
pub messages: Arc<MessagesRenderState>,
pub modeline: Arc<ModelineRenderState>,
pub diagnostics: Arc<DiagnosticsRenderState>,
}
Each sub-state is Arc-wrapped so subsystems publish independently -- a motion republishes ActiveDocumentRenderState (per-frame critical) without forcing BuffersRenderState (:b N-rate) to churn. The split tracks read frequency, not domain boundaries. (The set has since grown past the list above — e.g. active_document is itself a nested Arc<ArcSwap<…>> cell for the hottest reads, and there are additional sub-states such as modeline elements and the resolved-options registry — but the shape is unchanged: one arc-swap cell per read-frequency band.)
Editor::dispatch(...) republishes after every action. Background tasks (LSP responses, syntax updates, etc.) publish their sub-state directly without re-snapshotting the whole world.
5.7.3 Per-subsystem publication primitives
The shape that bridges "background task" and "renderer-visible state":
Arc<ArcSwapOption<T>>-- a wait-freeOption<Arc<T>>cell. Used for single-slot caches (e.g.lsp_document_highlights). The spawned LSP request task clones theArc, awaits the response, and.store(Some(Arc::new(cache))). Renderers.load()wait-free.PerBufferCache<T> = Arc<ArcSwap<HashMap<BufferId, Arc<T>>>>(lattice_host::per_buffer_cache) -- a copy-on-writeHashMapfor per-buffer caches (inlay hints, folding ranges, semantic tokens, code lenses, document links, document colors, pull diagnostics, references, symbols, …). ThePerBufferCacheExttrait carriesget_for/insert_for/remove_for; writes clone the inner map (microseconds for ~5--50 open buffers), reads return a detachedArc<T>the renderer holds across the frame.
Both primitives share the invariant RenderState's sub-state Arcs depend on: a clone observes writes made through the original. So rs.lsp.inlay_hints.get_for(id) sees data the spawned task wrote via the editor's lsp_inlay_hints_cache.insert_for(...) without any further publication of RenderState itself.
5.7.4 The LSP file-watcher as worked example
The watcher fans workspace/didChangeWatchedFiles notifications to interested LSP servers. The naive (pre-5.8.AF.5) implementation kept the notify::Watcher and the event drain on the renderer's per-tick loop -- notify::Watcher::watch(workspace_root, Recursive) walks the entire subtree synchronously on Linux, which froze the UI for minutes on Rust workspaces with populated target/ directories.
The relocated shape:
LspFileWatcherHandlelives onEditor. Holds only acmd_tx.- A dedicated tokio task on the LSP runtime owns the
RecommendedWatcher, the notify event rx, the per-server subscription map, and an ignore matcher. Editor::refresh_lsp_file_watcheris now a fingerprint-gated, non-blockingcmd_tx.send(SyncSubscriptions { … }). NonotifyAPI call, no inotify install -- those happen inside the task viatokio::task::block_in_placeso a gianttarget/walk doesn't stall sibling LSP tasks on the multi-thread runtime.- The notify callback consults a shared
Arc<ArcSwap<WatcherIgnoreMatcher>>(.gitignore+.ignore+ baked default globs coveringtarget/,.git/,node_modules/, …) and drops events whose every path is ignored before they enter the channel. Events for paths a server registered for, but no others. - Fan-out happens inside the task: the cloned
LspSupervisorHandleis wait-free; the per-serverdid_change_watched_filesnotify is a non-blocking actor channel send.
drain_lsp_fs_events deleted entirely. Zero watcher work runs on the renderer thread.
5.7.5 Migration shape for the rest of run_tick_pending
Every LSP feature drain in run_tick_pending migrates onto §5.7.2 + §5.7.3 the same way:
- The cache field on
Editorflips fromOption<T>orHashMap<BufferId, T>to the matchingArc<ArcSwapOption<T>>orPerBufferCache<T>. maybe_request_Xclones the cache slotArcinto the spawned task. The task awaits the LSP response and writes the result directly via.store(...)/.insert_for(...). No channel construction.drain_pending_Xand itspending_X_rxfield delete entirely.- The corresponding sub-state on
RenderStatecarries a clone of the cache slot; renderers read throughrs.lsp.X.load()/rs.lsp.X.get_for(id). - Renderer-coupled side-effects the old drain ran inline (e.g.
recompute_foldsafter a fold-cache write) move to the renderer-tickmaybe_request_Xvia a "last cache version reflected" tracker, fired once per (buffer, version) transition.
📝 Target end state: run_tick_pending disappears entirely — the renderer reads editor.render_state.load_full(), paints, and hands input back, with subsystems publishing on their own cadence. This is in progress: the primitives (§5.7.2/§5.7.3) are in place and the heavy drains (LSP file-watcher, most feature caches) have migrated, but run_tick_pending still exists today for the not-yet-relocated drains.
5.7.6 Paint-time savings — deferred-decision rationale
Today's build_render_state rebuilds every sub-state per dispatch tick (naive: every Arc is fresh). Slice 3a accepted this cost (well under the 100µs budget since 9 of 10 sub-states are empty placeholders); Slice 3b populates them. The question is whether the substrate should evolve toward Zed-style reactive publication (cx.notify() shape: dirty-bit per sub-state, renderer-side Arc::ptr_eq short-circuits) once the sub-states carry meaningful work.
We evaluated this against the paramount goals and the answer is defer — and treat the design as open.
Why the savings curve is shallow in Lattice. Zed's reactive paint wins decisively because Zed has substantial "cold" UI relative to the cursor: a file-tree sidebar, an AI panel, a status bar, and a project-search panel that don't change during most editing. On a cursor move, reactive paint skips all of that.
Lattice's "everything is a buffer" model means the active pane is the cursor-coupled work; there are no cold panels to skip. The dominant on-screen surfaces (document text + syntax highlights, gutter with relativenumber, cursor-coupled diagnostic glyphs / inlay-hint annotations) all derive from ActiveDocumentRenderState or LspRenderState and change every frame during interactive use. Reactive savings would apply only to idle frames (where the user doesn't perceive the latency anyway) and to background-LSP-update frames where only diagnostics changed (real but rare in absolute frame count).
The reactive semantic is already delivered by the existing primitives. Arc<ArcSwapOption<T>> (§5.7.3) and PerBufferCache<T> (§5.7.3) propagate writes across all clones atomically. The renderer's rs.lsp.foo.load() sees the spawned task's write within nanoseconds, no re-publication of RenderState needed. What cx.notify() would add on top is a paint-side short-circuit — and that only pays when the paint work it skips is substantial relative to the bookkeeping cost.
Simpler alternatives target the same idle-frame savings. Per-pane input-keyed cache ((buffer_id, cursor.line, scroll, viewport_dims, theme_epoch, syntax_version, diag_version) → rendered output) is a strictly simpler design: no dirty bits, no event-bus subscriptions, no Arc::ptr_eq logic — just a hash-of-inputs cache. Cache key invalidates correctly: cursor moved → recompute (which is exactly what we want). Glyph atlas + line-layout caching at the renderer-peer level (GPUI's text system already caches shaping; the cell-grid caches incremental builds) targets the same hot path more locally. Tree-sitter highlight span cache keyed on (buffer_id, range, syntax_version) is the syntax-side equivalent.
Decision posture. No commitment to adopt cx.notify()-shaped reactive publication. The substrate (§5.7.2 + §5.7.3) is already reactive in the load-bearing sense; further paint-side savings are an open question, gated on bench data:
- Land Slice 3c (
Editoron its own thread). Sub-states fully populated. - Bench: where is the renderer thread actually spending its one frame (8.3 ms at 120 Hz)? On what kinds of frames?
- If the data shows redundant per-frame paint work, evaluate the alternatives in order of complexity — input-keyed pane cache first (simplest), syntax/atlas caching second, sub-state dirty bits with renderer-side
Arc::ptr_eqonly if the simpler shapes don't close the gap.
This is heuristic #2 in action: evaluate against the paramount goals, not against other editors. Zed has reactive paint; that's data, not justification.
5.8 Major Modes and Minor Modes
The major / minor mode system is the primary customization mechanism. The full design -- trait surface, lifecycle events, typed option registry, OptionGroup registry, customize surface, plugin path, migration plan -- is documented in docs/mode-architecture.md. This section anchors the high-level commitments design.md depends on; consult the companion doc for the load-bearing detail.
5.8.1 Commitments
- A buffer has exactly one major mode -- content-type identity (rust, markdown, help, file-tree, lsp-log, ...). The major mode declares parser, default LSP attachment, default keymap layer, default minor modes, rendering profile (§5.6), and mode-scoped option overrides.
- A buffer can have any number of minor modes active simultaneously. Minor modes are composable, additive, and declaratively contribute keymap layers, option overrides, event subscriptions, decoration providers, statusline segments, and lifecycle hooks. Conflicts are declared, not implicit.
- Modal state (Normal / Insert / Visual / Op-pending / Command / Search) is orthogonal to major / minor modes; the two axes do not collapse.
- Modes own their full surface; the host is a thin substrate. A mode owns not just its keymap layer but the command declarations its chords/
:-line resolve (registered viaboot.commands_mut()), the action-handler bodies those chords fire, its lifecycle subscriptions, decoration providers, completion sources, and the production of its buffers. Keymaps live atKeymapLayer::MinorMode/MajorMode, neverBuiltin. The host exposes only generic primitives (buffer-store service, event bus, action-handler registry, generic chord dispatcher, the typedEffectvocabulary + its applier). Acid test: a new provider crate adds zero per-featureEditor::do_<x>methods and zero new variants to the hostActionenum -- it drives edits through the genericEffect::ApplyEditprimitive (the host keeps only genericEffectappliers, incl. the lifecycle ones that need&mut Editor). This is the load-bearing commitment that makes the mode system the extension mechanism (paramount goal #2) rather than one feature among many -- and the single deepest structural difference from Zed. See §5.8.2. - Modes are an interface, not a distribution unit. Same
Modetrait, three implementation paths: built-in (compiled Rust against the trait), bundled plugin (WASM Component Model, ships with the editor), third-party plugin (WASM Component Model, user-installed). The host treats all three identically downstream of registration; built-ins do not pay per-call WIT overhead. (Seemode-architecture.md§2 for the rationale and the v0.4-to-v0.5 correction.) - Options and
OptionGroups are typed identities, not strings. Each is a unique Rust type; cross-crate uniqueness is enforced by Rust's type system, display-name uniqueness bylinkmeaggregation, and naming-rule constraints (mode names end in-mode; group names do not) byconst fnassertions emitted at macro expansion. Strings appear only at boundaries (:set, TOML, plugin manifests). The bare namespace is reserved for built-ins by macro-API construction; plugin options are mechanically prefixed by plugin ID at registration. (mode-architecture.md§6.4 / §6.7.1.1 / §6.8.) :setand:customizeare two front-ends on the same store.:set <opt>=<val>is the cmdline parser, single-option, immediate.:customize <name>is the form-buffer front-end, group-oriented, buffer-backed (customize-modemajor), TUI-first.<name>resolves to a mode (when ending in-mode) for a focused view, or to anOptionGroup(Editor,Lsp,Picker,Filetree, ...) for a cross-mode collection. Both ship in v1; TOML write-through is deferred to v1.x.- Mode toggle and reload. Every registered mode is user-toggleable via
:enable/:disable/:toggleplus an auto-generated ex-command per mode (:rust-mode,:lsp-diagnostics-mode). Major modes additionally support reload (call the ex-command on a buffer already in that mode ⇒ deactivate then re-activate; the trait contract is idempotent setup). Minor mode deactivation removes every contributed override (options, keymap, subscriptions, decorations) by construction because the registry, not the mode, owns the layer stack.
The detailed taxonomy of which lattice features are major modes, minor modes, or neither -- plus the migration plan to land the lattice-mode foundation crate, retire BufferKind, unify the renderer, gate LSP behind lsp-mode, and ship the :customize form view -- is in mode-architecture.md §4 (inventory) and §10 (migration).
5.8.2 Mode ownership -- the host is a thin substrate
This is the load-bearing commitment that makes the mode system the extension mechanism rather than one feature among many -- and the single deepest structural difference from Zed (comparison-zed.md §3). It is stated at design level because past sessions repeatedly half-migrated features -- a mode contributed its keymap but left the handler body in the host -- and naming the rule here is what makes "don't do that" enforceable by reference rather than by memory.
A mode owns its full surface. Not just the keymap layer, but everything the keymap implies:
- Keymaps -- pushed at
KeymapLayer::MinorMode(id)/MajorMode(id), neverKeymapLayer::Builtin. Builtin is the universal vim grammar that fires in every buffer; a feature chord scoped to one mode does not belong there. A mode contributes its layer viaMode::keymap()(orregister_<mode>_keymap(...)) in the mode's own crate; the host's generic translate pass pushes it underMinorMode(<mode>::mode_id())and a per-keystroke filter scopes the chord to mode-active buffers. - Command declarations -- the
action:*and:ex-commandCommandSpecs that the chords and the:line resolve. A mode registers them in its crate'sinstall(&mut impl SubsystemBoot)viaboot.commands_mut()(the multibuffer / diff pattern), not in the host's grammarex_commands.rsoractions.rs. The registered names are what the keymap'scmdand theActionHandlerRegistryhandler key against. (User-friendly short names register under their plain canonical form -- noex:prefix or host alias shim.) - Action-handler bodies -- the closure that runs when a chord or ex-command fires. A mode contributes global, buffer-agnostic handlers declaratively via
Mode::action_handlers()(registered once at boot, keyed byCommandId), and per-buffer session-scoped handlers inon_activate(token in theGuard). There is noEditor::do_<provider>_actionin the host. (ActionHandlerRegistry:mode-architecture.md§5.3.) - Lifecycle subscriptions, decoration providers, completion sources, option overrides -- all owned by the mode, acquired in
on_activate, released byGuarddrop. - Production of the mode's buffers -- a mode that backs a synthetic buffer owns creating it, through the
&mut-backed creation seamModeActivator::ensure_named_document(imperative, from a provider trigger fn -- e.g. compilation'sstart_compilation) or the declarative peerEffect::OpenSyntheticBuffer(from a pure ex-command). Creation is NOT on the&selfBufferStore(which can't activate a mode) and NOT inon_activate(which runs on the already-created buffer). See §5.10.5 andmode-architecture.md§5.4.
What stays in the host is purely generic: the buffer-store service, the typed event bus, the ActionHandlerRegistry, the generic chord dispatcher (walks the layer stack, resolves a chord to an ActionId), and the generic Effect applier. None of these branch on mode or BufferKind.
The Effect vocabulary is a host-owned typed enum -- by design. A mode-handler body does not mutate Editor directly; it returns an Effect and the host applies it. This is the seam that keeps modes pure (no &mut Editor), replayable (macros record the invocation, the host re-derives the effect), and -- at the WASM stage -- the capability-gated plugin boundary. A mode that needs to drive an arbitrary document edit returns the generic Effect::ApplyEdit { target, edit, cursor } primitive; the host applies it to the target buffer (the active-document pipeline, or a peer buffer's handle) -- so a new edit-driving feature adds no Editor::do_<x> method. Effects whose application is expressible as ActionContext -> Effect (cursor moves via SelectionChange, edits via ApplyEdit, messages via Echo) are fully mode-owned end-to-end.
The one host residue: lifecycle Effect appliers. Some effects intrinsically mutate &mut Editor substrate a mode crate cannot reach -- pane-tree splits, document-actor session creation/teardown, task spawning. Their appliers (do_<x>) stay host because the provider crate is below lattice-host in the graph and cannot take &mut Editor without a dependency cycle. These generic-but-host-bound appliers do not violate the acid test below: the test targets per-mode chord/handler behavior and Action::<feature> variants, not the host's Effect-applier layer. (Lifting even these bodies into the provider crate needs the host-provider-inversion seam -- a richer handler context -- which is deferred; host-provider-boundary.md.)
Acid test. A new provider crate landing requires zero per-feature Editor::do_<x> method additions in lattice-host and zero new variants in the host Action enum -- it contributes its keymap, command declarations, and handler bodies through the generic seams, and drives any edits through the generic Effect vocabulary. If a migration adds a feature-specific host Action variant or a chord/handler body on Editor, it is incomplete. (The diff subsystem is the worked example: lattice-diff owns every diff chord, handler, and command declaration; only the generic lifecycle Effect appliers remain host. See diff-extraction.md §4.)
The half-migration failure mode. Moving a mode's keymap into the mode while leaving its handler in the host's dispatcher -- or publishing substrate data through a Document trait method while the host still binds the chord -- leaves half the ownership surface with the host and silently re-introduces per-mode host logic. This is the exact drift the rule prevents; it is caught by tests (crates/lattice-host/tests/multibuffer_is_a_regular_buffer.rs), not the type system, so it must be checked at review.
Substrate-vs-helper rule (which side owns mode-relevant data):
Documenttrait method -- when generic host machinery (renderer, generic dispatch loop, position-history walker) reads the data uniformly across all buffer kinds. Example:display_line_numbers(the renderer reads it for the gutter),dispatch_with_cancel(the chord-dispatch loop reads it to execute grammar).- Substrate helper function in the owning crate -- when only a specific mode's handler reads the data. Example: composed->source translation, excerpt expansion -- consumed only by the multibuffer mode's handlers, so they are free functions the mode imports, not trait methods.
Rule of thumb: trait method = uniform-host consumer; helper = mode consumer. Mode-consumed data must not extend the Document trait surface, because doing so implies host involvement the mode-ownership rule forbids.
The trait surface, the ActionHandlerRegistry, the Guard cleanup contract, and the full migration history are in mode-architecture.md §5.2 (modes own their lifecycle) and §5.3 (modes own their action handlers).
5.9 UI Components
The UI layer's structure determines how the user actually experiences the editor. The components below are all UI-layer concerns living in lattice-ui-gpui; the core knows nothing about windows, panes, popups, or pickers.
Foundational principle: everything is a buffer. File tree, outline, symbol list, diagnostics list, search results, terminal, REPL -- all are buffers, distinguished only by mutability flags, content provider, and major mode. Users place them in panes via the same split / window operations as code buffers. The editor enforces no fixed sidebar or bottom-panel layout; the user composes their workspace from panes containing buffers of their choice.
Popups carry buffer content with a popup-shape view. Hover, signature help, and :describe-* content is real buffer content (typically markdown-mode) rendered in a floating-anchored geometry rather than a pane-shaped one. The same compose_visible_lines pipeline paints both -- a hover popup gets tree-sitter syntax highlighting, link nav, position history, and every other buffer affordance "for free", because it is a buffer. The popup-shape view contributes its own minor mode (hover-mode) that overrides options like wrap, line-numbers, and anchor=cursor. See docs/mode-architecture.md §6.7 / §9.5.
What is not a buffer:
- Pickers -- modal fuzzy-search overlays (file, symbol, command palette).
- Notifications -- corner-anchored transients.
- Mode line / header line -- per-pane status surfaces, contributed via segment registry.
- Command line / echo area -- per-window single-line input/echo (though every interactive prompt within them is a buffer per §5.9.10's rich minibuffer).
- Completion candidate list -- list of typed
CompletionItems, distinct from the content of any single completion (which renders as buffer content when expanded). - Code action menu -- list of typed actions.
The lists / overlays above are not buffer content; they're typed-data UI primitives. Anything editable or prose-shaped is a buffer regardless of where on screen it appears.
5.9.1 Window structure
+----------------------------------------------------------------+
| [Title bar -- OS-native or custom] |
+----------------------------------------------------------------+
| [Tab bar -- optional] |
+----------------------------------------------------------------+
| [Header line -- per-pane, optional] |
+----------------------------------------------------------------+
| |
| |
| Pane tree (recursive splits; each leaf holds one buffer) |
| |
| File tree, outline, diagnostics, search results, terminal, |
| REPL -- ALL buffers. The user splits panes and chooses |
| which buffer occupies each pane. There is no left sidebar, |
| right sidebar, or bottom panel as a first-class concept. |
| |
+----------------------------------------------------------------+
| [Mode line -- per-pane] |
+----------------------------------------------------------------+
| [Command line / echo area -- per-window] |
+----------------------------------------------------------------+
Floating overlays (z-ordered above all):
* Completion popup
* Hover popup
* Signature help popup
* Diagnostic popup
* Code action menu
* Pickers (file, symbol, command palette)
* Notifications (corner-anchored)
5.9.2 The pane tree
A window contains a recursive tree of panes. Each leaf is an editor view; each branch is a horizontal or vertical split.
The real tree is a recursive binary split (each split is one branch with a left/right or top/bottom child + a ratio), not an n-ary Vec of children:
// lattice-core/src/ui/pane.rs
enum PaneNode {
Leaf(usize), // index into the tab's pane list
HorizontalSplit { left: Box<PaneNode>, right: Box<PaneNode>, ratio: f32 },
VerticalSplit { top: Box<PaneNode>, bottom: Box<PaneNode>, ratio: f32 },
}
struct PaneState {
buffer_id: BufferId,
cursor: Position,
scroll: usize,
viewport_height: u32,
viewport_width: u32,
// … per-pane render/preview state
}
Splits carry a ratio (<C-w>+ / <C-w>- / <C-w>> / <C-w>= adjust it). Pane ops: split (<C-w>s / <C-w>v), close (<C-w>c), focus (<C-w>hjkl / <C-w>w), resize, only (<C-w>o), and MovePaneToNewTab. (There is no generic SwapPanes / MovePane in v1; show_gutter / show_minimap are not pane fields — gutter columns are global-ish options, minimap is 📝 unbuilt.)
The same buffer can appear in multiple panes (and tabs). Each pane holds its own cursor/scroll state; the buffer is shared.
5.9.3 Tabs
Tabs are a per-window grouping of documents, distinct from panes. A tab can hold multiple panes (a saved split layout). Tabs are optional -- users who prefer pure split-only navigation can disable the tab bar.
There is no Window struct — the host Editor is the composition root and owns the tab list directly:
// lattice-host: Editor.tabs: Vec<TabSlot>
struct TabSlot {
id: TabId,
panes: PaneNode, // this tab's pane tree
label: String, // user-editable via :tabmove/:tabnew; defaults from buffer
}
Tab operations (built): :tabnew [path], :tabnext/:tabprev (gt/gT/{N}gt), :tabclose, :tabonly, :tabmove, MovePaneToNewTab, <C-PageDown>/<C-PageUp>. The tabline shows via the tabline.show option (never/auto/always). (Move-to-window and duplicate are 📝 — there is a single window.)
5.9.4 Status lines: mode line and header line
Two horizontal status surfaces per pane, both painted through the shared cell-grid pipeline.
Mode line (bottom of pane): the persistent status surface -- modal mode indicator, file info, cursor position, encoding, line ending, major mode, minor modes summary, LSP status, plugin contributions.
Header line (top of pane, optional): contextual breadcrumb or symbol context -- file path with breadcrumbs, the current symbol (LSP documentSymbol containing cursor), tab-context, plugin contributions.
The mode line is composed of modeline elements registered in a ModelineRegistry (lattice-mode, the ML.0–ML.5 redesign). Elements are event-driven, not pull-on-trigger: content updates arrive over the event bus as ModelineElementUpdate, so an element recomputes only when its data changes — never per keystroke:
// lattice-mode/src/modeline.rs
struct ModelineElement {
id: ElementId,
zone: Zone, // Left | Center | Right
priority: i32, // ordering within the zone
// content published via ModelineElementUpdate over the event bus
}
The header line is not a second segment registry — it is delivered as view-header virtual rows (lattice-cells headerline / HeaderlineProvider); async-buffer/multibuffer providers surface progress + breadcrumb content there (the standing rule that async-buffer status lives on the headerline, not the mode line or a notification).
Elements are contributed, not hardcoded: the core ships defaults (modal indicator, file path, cursor position, encoding, EOL, major mode, LSP status) and modes / plugins register their own. Conflict resolution: same-(zone, priority) elements order by registration.
Default mode line (rust-mode example):
[NORMAL] [+] src/main.rs [42:18 (78%)] rust rust-analyzer UTF-8 LF
^ ^ ^ ^ ^ ^ ^ ^
| | | | | | | |
modal dirty file path cursor major LSP status enc EOL
5.9.5 The gutter
A vertical strip at the left of each pane showing line-aligned annotations. Like the status lines, gutter content comes from a registry of contributors.
📝 Status. The gutter columns are hardcoded today (
gutter_cols()in the cells worker) — line numbers (number/relativenumber), diagnostic markers, and diff signs. The pluggableGutterSegmentSpecregistry below is the forward design; plugin-contributed columns (breakpoints, blame heatmap) are not yet a seam.
The planned registry mirrors the modeline element model:
struct GutterSegmentSpec {
id: SegmentId,
column: GutterColumn, // ordered columns left-to-right
width: GutterWidth, // Fixed(n_chars) | Auto
content_provider: ContentProviderId,
}
Gutter columns today (left to right): diagnostic markers, diff signs, line numbers, fold indicators.
5.9.6 Popup system
Popups are floating, transient UI surfaces. Crucially, a popup is a Document in a popup-shaped view (§5.9.8 / §5.6.3) — not a bespoke non-buffer primitive with typed anchor/dismissal/interaction structs. The real shape carries only a placement + a focus mode; content, dismissal, and interaction come from the underlying buffer + its mode:
// lattice-host/src/ui/popup.rs
enum PopupPlacement {
CursorAnchored, // pinned to the symbol the popup was invoked from
Centered,
}
enum PopupFocus {
Steal, // popup captures input (State-B): pickers, code-action menus
Passive, // informational; input goes to the editor: hover, signature
}
Popup types: completion, hover, signature help, diagnostic detail, code-action menu — each a buffer with a mode, placed CursorAnchored (hover/signature/ completion pin to the invoking symbol via the stored anchor) or Centered (pickers). Focus is Passive for hover/signature, Steal for pickers / code-action menus. Dismissal (on cursor move, edit, Esc) is driven by the owning mode's lifecycle hooks — e.g. the hover-popup auto-dismiss-on-cursor-move hook lives in Editor::dispatch so it fires in both peers.
📝 A corner-anchored notification popup with stacking + timeout (the old AtPaneCorner row) is not built — see §5.9.9.
Performance: because a popup is just another buffer view, it reuses the cell-grid layout + caching; showing / hiding is sub-millisecond.
5.9.7 Pickers
A picker is a fuzzy-search overlay used pervasively across the editor -- file open, symbol search, buffer switcher, command palette, plugin-defined pickers. All of these use one primitive.
struct PickerSpec {
id: PickerId,
title: String,
content_provider: ContentProviderRef, // streams matchable items
item_renderer: ItemRendererRef, // how each item appears
preview_provider: Option<PreviewProviderRef>,
selection_mode: SelectionMode, // Single | Multi
initial_query: Option<String>,
on_select: ActionRef,
on_cancel: Option<ActionRef>,
}
trait ContentProvider {
/// Returns a stream of items matching the query.
/// Implementations should be lazy -- don't enumerate everything upfront.
fn items(&self, query: &str) -> BoxStream<PickerItem>;
}
struct PickerItem {
id: ItemId,
label: String, // matched against query
secondary_label: Option<String>, // shown but not matched
metadata: serde_json::Value, // arbitrary; used by renderer and preview
}
trait ItemRenderer {
fn render(&self, item: &PickerItem, query: &str, focused: bool) -> RichContent;
}
trait PreviewProvider {
fn preview(&self, item: &PickerItem) -> BoxFuture<RichContent>;
}
Built-in pickers:
| Picker | Content provider | Preview |
|---|---|---|
| File picker | Recursive workspace file enumeration (ripgrep-style) | First N lines of file |
| Buffer switcher | Open documents | Current view of buffer |
| Symbol picker | LSP workspace/symbol | Source surrounding symbol |
| Command palette | All registered commands | Command description and keybinding |
| Recent files | History | First lines of file |
| Live grep | ripgrep results streaming | Match in context |
Streaming results: the picker stays responsive even with millions of files because ContentProvider::items() returns a stream. Items appear as they're discovered. The first matches appear in milliseconds; the picker is usable while enumeration continues.
Preview pane: optional. When present, the picker layout splits into list-on-left, preview-on-right. Previews render through the same cell-grid pipeline as any buffer (in a read-only projection, §5.9.8). Preview is async; missing previews show a placeholder.
Plugin-defined pickers: plugins create pickers by implementing ContentProvider and optionally ItemRenderer/PreviewProvider and calling ui.open-picker(spec). Examples: a git-branch picker, a docker-container picker, a snippet picker.
✅ Graduated. The picker primitive now lives in its own lattice-picker crate (the "graduates to a sibling crate when the GPUI peer comes online" plan below has happened — both peers are online): Picker + PickerRegistry + PickerSourceSpec, a source registry, live-query subsystem, MRU frecency, and async syntax-highlighted preview. It holds a query line, a filtered candidate list (fed by registered sources), a selection cursor, and a PickerAction tag the host's accept dispatcher pattern-matches on. The first instantiation is the buffer switcher -- :b with no arg walks BufferRegistry, builds one row per entry with a kind-tagged marginalia (doc / tree / help, plus (current) on the active buffer), and <CR> activates the selected BufferId via App::activate_buffer. Filtering today is case-insensitive substring; the full pipeline-driven path (lattice-completion matcher / ranker / annotators) graduates the picker once CommandLineSlot is lifted out of the slot detector.
Renderer-agnostic by construction. The lattice-picker crate has no renderer-specific or host-specific imports beyond lattice-completion's candidate shape. Host-coupled work (walking BufferRegistry, snapshotting the LSP supervisor, parsing host buffer ids) lives on the host side; each renderer peer supplies only its own render adapter.
Layout. Vertico-style: the picker's query line takes over the cmdline / echo row at the bottom, candidates render in the row band immediately below. The selected row sits at the TOP of the band (closest to the prompt below), alternatives fan upward in match-rank order. Reuses the cmdline completion popup's per-row painter (matched ranges + marginalia), so styling is consistent across surfaces.
Live preview. While a SwitchToBuffer picker is open, every selection change activates the candidate buffer in the active pane (without pushing to position history). On <CR> the preview-active buffer is the real switch; on <Esc> the pane reverts to whatever buffer was active when the picker opened (Picker::preview_origin). Activate paths gate position-history pushes on App.previewing so preview hovers don't pollute the jump list. LSP-instance pickers (OpenLspLog / OpenLspTraceLog) skip preview today -- those targets create / surface real registry-tracked log buffers on accept; preview-on-hover for them is a follow-up.
Live sources -- sources whose candidate set is driven by re-running an external engine on each query change rather than fuzzy-filtering a fixed batch. The shape ContentProvider::items(&query) -> BoxStream<...> above describes the post-v1 fully-streaming model; v1 ships a debounced variant of the same idea that reuses the existing inline-or-future plumbing:
PickerSourceSpec.live: boolis the declarative marker. Sources opting in implementPickerSourceGenerator::on_query_changed(&ctx, &query) -> Option<SourceResult<PickerInitResult>>. Default impl returnsNone-- static sources cost nothing.- The picker primitive's
Picker::refiltershort-circuits whenlive_source_mode == true-- candidates renderraw1:1 in insertion order, no fuzzy scoring, no MRU re-ranking. The source's engine (grep, future LSP workspace-symbols) is the filter; re-ranking on top would distort the ordering it produced. - The host's main-loop drain
drain_pending_live_picker_querywatches a debounce deadline. Each keystroke pushes the deadline forward by 150ms (telescope's default); when the deadline elapses, the host callson_query_changed, routes the result (Inlineseats immediately,Futurespawns + parks the rx with the launched query stashed for stale-detection at drain time), and clears the deadline. A newer keystroke arriving while a future is in flight does not wait for it -- it pushes the deadline forward and the older future's result is dropped at drain time when itslaunched_for_queryno longer matches the picker's current query. - The first positional arg to
:picker <live-source>seeds the prompt::picker grep TODOopens withquery = "TODO"and runs the first grep immediately; subsequent keystrokes extend the same query.:picker grepwith no args opens an empty picker that waits for typing. - Grep specifically spawns
run_grepon tokio's blocking pool (spawn_blocking) so the std-syncCommand::outputcall doesn't pin an async-runtime worker. The UI thread never parks on grep.
Why this shape over the streaming model: the streaming spec assumes incremental row delivery (1 → N rows over time per query). Our backends are batch -- grep returns its full hit list when it finishes, LSP workspace/symbol returns a single response. A future-per-query is a tighter fit; the Stream variant of PickerInitResult stays in the API for sources that genuinely produce incremental output (post-v1, e.g. an LSP server that pages results).
5.9.8 Buffer-backed views (panels-as-buffers)
The auxiliary surfaces other editors expose as fixed sidebars and bottom panels (file tree, outline, diagnostics list, search results, LSP / plugin logs, terminal, REPL, debugger, test runner, git branch viewer) are all just buffers here. They are placed in panes via the standard split / window operations.
Plugins that provide such views ship as major-mode plugins with a content provider:
pub trait BufferContentProvider: Send + Sync {
/// Initial content (populated when the buffer opens).
fn initial_content(&self, ctx: &BufferContext) -> RichContent;
/// Triggers that mark the buffer dirty and re-pull content.
fn update_triggers(&self) -> Vec<UpdateTrigger>;
/// Recompute content on a trigger fire (off the UI thread).
fn refresh(&self, ctx: &BufferContext) -> RichContent;
/// Optional: handle input for interactive buffers (file tree expand,
/// diagnostics-list jump-to, terminal pty input).
fn handle_input(&mut self, ctx: &mut BufferContext, event: InputEvent)
-> Vec<BufferEvent> { vec![] }
/// Mutability classification.
fn mutability(&self) -> BufferMutability;
}
pub enum BufferMutability {
ReadOnly, // diagnostics list, search results, logs
Interactive, // file tree (expand/collapse), terminal
Editable, // regular code/text buffers
}
First-party buffer-backed views shipping with v1:
| View | Provider | Mutability |
|---|---|---|
file-tree | Workspace walker | Interactive |
outline | LSP documentSymbol | Interactive (jump-to) |
diagnostics-list | LSP diagnostics aggregator | ReadOnly |
search-results | ripgrep stream | ReadOnly |
lsp-log | Per-server stderr | ReadOnly |
plugin-log | Plugin output | ReadOnly |
terminal | PTY adapter (post-1.0) | Interactive |
The user opens any of these like opening a file: a command (:open file-tree, key binding, command palette entry) creates a new buffer of that major mode and places it in the active pane (or a new split, by user choice). To get a "left sidebar with a file tree," the user creates a vertical split and opens file-tree in the left pane; the layout is theirs to compose, save, and restore via tabs (§5.9.3).
Implementation: help / log buffers in the registry (PU.1a). The unified BufferRegistry stores help content as BufferData::Help(DocumentEntry) — the same actor-backed DocumentEntry used by BufferKind::Document, Messages, Multibuffer, and Dashboard. The old BufferData::Help(HelpBuffer) variant (an owned blob) was superseded in PU.1a: help content is now an actor-backed Document with wait-free snapshot access. App::open_help_in_pane(buffer) is the in-pane entry point (durable record + active hot-path mirror); App::open_help is the popup overlay path (transient surfaces: hover, doc lookups, error toasts). De-dup by title means re-running :lsp-log rust surfaces the existing buffer rather than spawning a duplicate. The picker (§5.9.7) and the LSP command refactor (:lsp-log / :lsp-server-log / :lsp-trace-log) consume this primitive: candidate generation walks BufferRegistry::help_ids_sorted(), on-accept activates the chosen BufferId in the current pane.
Performance: content providers are lazy. They populate on first display; they refresh on declared triggers (event, periodic, manual), all off the UI thread on the blocking pool (spawn_blocking). A buffer that is open but whose pane is not visible can be configured to suspend updates entirely.
Terminal buffers — Document-on-Normal. The terminal buffer has two sub-states (TerminalInsertMode PTY-bound, TerminalNormalMode Document-bound). Insert mode encodes keystrokes to ANSI and the renderer paints the alacritty cell grid directly. Normal / Visual operates on a synthetic, read-only Document built from the scrollback at Insert→Normal transition; the central vim grammar (motions, text objects, marks, search, registers, visual modes) runs against that Document with no kind-specific branching. The renderer still paints the cell grid; a coord adapter remaps document-space selection ranges to cell coordinates at publish time. See terminal-as-document.md for the lifecycle, coord adapter, and slice plan.
5.9.9 Notifications (📝 planned)
📝 Status. A dedicated notification subsystem (corner-anchored, stacking, timeouts) is not built. Today the echo area (
set_message/EchoLevel) and the*messages*buffer cover the case, and async-buffer status routes to the headerline by standing rule (§5.9.4). The design below is the forward plan; revisit only if the echo area proves insufficient.
Transient messages anchored to a window corner, queued and animated.
struct Notification {
id: NotificationId,
level: NotificationLevel, // Info | Warning | Error
title: Option<String>,
body: String, // can include markdown
actions: Vec<NotificationAction>, // optional buttons
timeout: Option<Duration>, // None = sticky until dismissed
source: NotificationSource, // who posted (plugin, core, lsp)
}
struct NotificationAction {
label: String,
command: ActionRef,
}
Behavior:
- Stack vertically in a chosen corner (default: bottom-right).
- New notifications appear above older ones; older ones expire.
- Maximum visible count (default 3); excess queued.
- Errors and warnings have higher visual priority and longer default timeouts.
- Notifications with actions display interactive buttons.
Use cases: "File saved", "Plugin X crashed", "LSP server restarted", "1 of 5 tests failed (click to view)", "Update available".
5.9.10 Minibuffer and echo area
✅ Shipped (2026-07-24, rich-minibuffer MB.1–MB.5). The
:command line is a buffer-backed, readline-grade editing surface (*command-line*syntheticDocumentwithcommand-line-modemajor mode). The/·?search line is the same substrate (*search-line*buffer,search-line-mode). Both are insert-only (tier 1) with<C-x><C-e>expanding in place into a full-modal mini-buffer band (tier 2). Built:<C-p>/<C-n>history walk,q:/q//q?fuzzy history pickers,command-linegrammar syntax highlighting, live error indication, parameter hints,:s///substitution preview,command-line.expand-heighttyped option, and an independentsearch_historyring. The substrate is ready forgit-commit-line/repl-input/ interactiveinput()prompts — each is a new major mode on the same one-line-buffer pattern. Seerich-minibuffer.mdfor the full design,slice-plans/rich-minibuffer.mdfor sequencing.
The design intent: the "command line" surface is a rich editing space, not a single-line widget — a real buffer with a major mode, opened transiently, painted through the cell-grid pipeline. Every interactive prompt would reuse the buffer, command-dispatch, decoration, popup, and renderer machinery -- nothing minibuffer-specific outside of "this buffer is currently the input focus."
Minibuffer as a buffer
struct Minibuffer {
document: DocumentId, // a real Document
major_mode: MajorModeId, // command-line, search-line, ...
active: bool, // visible / focused
on_submit: CommandInvocation, // what to invoke on Enter
on_cancel: Option<CommandInvocation>,
}
Built-in minibuffer major modes:
| Major mode | Triggered by | Purpose | Status |
|---|---|---|---|
command-line | : | ex-command parsing and dispatch | ✅ shipped |
search-line | / / ? | incremental search (forward/backward) | ✅ shipped |
git-commit-line | git plugin | one-line commit messages | 📝 |
repl-input | REPL plugins | interactive evaluation | 📝 |
picker-query | pickers | fuzzy-match query input | 📝 |
prompt | interactive arg specs (§B.1) | typed argument prompts | 📝 |
Each major mode brings its own keymap, parser, syntax highlighting, completion source, validator, and live-preview decorator.
What being a buffer gives us
Because the minibuffer is just a buffer:
- Full vim grammar applies inside it. You can
b,dw,0,c$while editing a:command. Thecommand-linemajor mode declares Insert as the entry mode for ergonomics, but Normal mode is one<Esc>away. - Tree-sitter syntax highlighting. The
command-linemajor mode uses acommand-linetree-sitter grammar that highlights command names, ranges, regex bodies, flags, and string literals -- exactly the way code is highlighted. - Decorations. Live error indicators (red squiggle under unknown command name; under malformed regex), parameter hints (virtual text after the cursor showing the next expected arg's name and type), live type validation (squiggle under
:set tabstop=hello), are allDecorations on the minibuffer document, identical to the LSP decorations on a code buffer. - Popups. Completion popups (command name completion, file-path completion for
:write, option-name completion for:set), hover popups (showing the docstring of the command being typed), parameter-info popups (the typed signature of a multi-arg command) -- all render through the existing popup manager. - Live preview decorations on other buffers. The
:%s/foo/bar/gparse, while incomplete, can publish decorations onto the active editor buffer:foomatches highlighted,barrendered as virtual replacement text. Implemented as a buffer-event subscription on the minibuffer'sBufferEditevent that re-runs the parser and pushes decorations to the target document. Cancelled on minibuffer dismiss; committed on submit. - Full editing power for power users. Multi-line drafting of complex commands works because the minibuffer is just a (transiently shown) buffer. Crafting a regex, a script invocation, or a long substitute command uses the same editing model as any other text.
- Plugins extend prompts the same way they extend buffers. A plugin registers a new major mode and gets all of the above for free for its own prompts.
Live error indicators -- concretely
The command-line major mode subscribes to its own buffer's BufferEdit event with a debounced (~10ms) handler:
1. On edit, the handler runs the ex-syntax parser incrementally.
2. The parser produces either:
- A complete CommandInvocation, or
- A partial parse with one or more typed parse errors at byte ranges.
3. Errors are published as Decoration::Diagnostic ranges on the minibuffer
document (red squiggle + hover text). The decoration provider is the
command-line major mode itself.
4. If the parse names a command, its registered argument schema is consulted
to render parameter hints (virtual text) at the cursor position and to
validate already-supplied args.
5. If the partial parse describes a substitution, search, or other
buffer-targeting command, decorations are also published to the *target*
buffer (live preview).
Submission (<Enter> in Insert mode, or whatever the keymap binds in Normal) runs the parsed CommandInvocation through execute(...). Cancellation (<Esc>) closes the minibuffer and removes any preview decorations.
Performance
Parsing is allocation-bounded and runs on the dispatcher's task; the minibuffer never blocks the UI. Decoration updates piggy-back on the existing decoration pipeline (§5.6). Cold open of the minibuffer is sub-millisecond (instantiate or reuse a pre-sized buffer, swap focus); subsequent edits cost the same as edits in any other buffer.
Echo area
The echo area is a separate, single-line surface used for transient one-line output from the core or commands ("File saved", "No matches", "Pattern not found", "Plugin X crashed"). It shares screen real estate with the minibuffer: when the minibuffer is active, the echo area is hidden; when the minibuffer dismisses, the echo area shows the most recent message until its timeout elapses.
A rolling history of every echo-area message and every notification is kept in a *messages* buffer (read-only, auto-scrolling) that the user can open in any pane to scroll back.
5.9.11 Scrollbars and minimap (📝 planned)
📝 Status. Neither is built. Both are forward design.
Scrollbars: modern style -- appear on scroll, fade out after a moment of inactivity. Show position indicator + diagnostics summary marks (errors/warnings as colored ticks along the scrollbar). Configurable: always-visible, on-scroll, never.
Minimap: optional thumbnail of the file rendered at small scale on the right edge of the pane, re-rendered (debounced) only when the file or scroll position changes substantially. Shows diagnostics, search matches, current viewport indicator.
5.9.12 Performance characteristics of the UI layer
A frame containing:
- One editor pane (monospace path)
- An open completion popup
- A vertical split holding a
file-treebuffer - A horizontal split holding a
diagnostics-listbuffer - A status line with 8 segments
- A header line with breadcrumbs
- 2 active notifications
...costs roughly the same to render as a frame with just the editor pane. Reasoning:
- The editor pane carries the heavy work; it's on the fast path.
- Popups, buffer-backed views, status lines are all low-traffic cell-grid content -- built once off-thread, cached, re-rendered only on change.
- Modeline elements update on event, not per-frame.
- All renderers share one GPU atlas and submission.
This is the architectural win over Emacs in concrete terms. In Emacs, every UI element shares the redisplay engine with the buffer, so a busy UI taxes editor rendering. Here, the UI layer's complexity has near-zero impact on editor input latency.
5.10 Event System and Hooks
Vim's autocmd and emacs's hooks both attach behavior to editor events. The two systems differ only in surface syntax; their semantics collapse into one primitive:
The bus is a concrete struct EventBus in lattice-runtime (not a trait in lattice-mode), holding the writable Document actors' fan-in plus typed subscriptions:
// lattice-runtime/src/events.rs (shape)
impl EventBus {
fn subscribe(&self, filter: EventFilter, sink: SubscriptionTarget) -> SubscriptionId;
fn unsubscribe(&self, id: SubscriptionId);
fn publish(&self, event: Event);
}
pub struct EventFilter {
pub kinds: Option<Vec<EventKind>>, // e.g. DocumentSaved
pub path_glob: Option<String>, // path glob (field is `path_glob`)
pub major_mode: Option<MajorModeId>,
pub predicate: Option<PredicateId>, // arbitrary plugin-supplied
}
pub enum SubscriptionTarget {
Channel(mpsc::Sender<Event>),
Invocation(CommandInvocation), // run a command in response
Plugin { plugin: PluginId, handler: HandlerId },
}
5.10.1 Event catalog
The protocol-level Event enum (lattice-protocol/src/event.rs) carries editor state transitions. The real v1 variants:
- Document lifecycle:
DocumentOpened(carries{ id, path, version, text }; §5.4.3 describes the mode-owned LSP attach that subscribes to it),BeforeSave,DocumentSaved,DocumentClosed,DocumentChanged. - Modal state:
ModalModeChanged { from, to }. - Mode lifecycle:
MajorEntered { buffer, major },MajorExiting { buffer, major },MinorActivated { buffer, minor },MinorDeactivated { buffer, minor }.MajorEnteredfires afteron_activate;MajorExitingbeforeon_deactivate. Seedocs/mode-architecture.md§7. - Selection:
SelectionsChanged. - Options:
OptionChanged. - Plugin:
Plugin(payload),PluginCrashed.
Beyond this core enum, subsystems run typed sub-buses for their own payloads (not part of the protocol Event): LSP publishes LspBufferAttached / LspLogPushed / LspDocumentChanged etc. over its own typed bus; diagnostics flow over a broadcast. 📝 The richer catalog originally sketched here — CursorMoved, AfterSave, Idle { duration }, FocusGained/Lost, BufferViewOpened, UI/Pane* events — is not protocol-level today; those are either subsystem-typed or not yet added.
5.10.2 Hook handlers may modify events for "Before"-class events (📝 planned)
📝 Status. The event bus is observation-only today (
event.rsdocuments "v1: observation-only"). The mutate/veto path below is not built.
The design: a subscription registered as Invocation(...) for a Before-class event receives the typed payload, may mutate fields the event declares as mutable, and may veto by returning Err. BeforeSave handlers could rewrite content (formatters); BeforeQuit handlers could veto. For non-Before events the return value is ignored except for error-logging.
5.10.3 Vim :autocmd and emacs hooks both desugar to this (📝 planned)
📝 Status. The
:autocmd/:add-hookex-commands are not built — thesubscribe(filter, target)primitive exists and modes subscribe in Rust, but the user-facing desugaring commands below are forward design.
" vim
autocmd BufWritePre *.rs RustFmt
; emacs
(add-hook 'before-save-hook
(lambda () (when (eq major-mode 'rust-mode)
(rust-format-buffer))))
// lattice (the underlying call -- both syntaxes desugar to this)
events.subscribe(
EventFilter {
kinds: Some(vec![EventKind::BeforeSave]),
document_pattern: Some(Pattern::new("*.rs")),
..Default::default()
},
SubscriptionTarget::Invocation(invocation_for("rust-fmt-format-buffer")),
);
The :autocmd and :add-hook ex-commands are parser front-ends for this call.
5.10.4 Performance
Subscriptions are indexed by EventKind (plus a wildcard bucket); the remaining filters (path_glob, major_mode, predicate) are AND-checked per candidate at publish time. Publishing evaluates only the matching kind bucket, never the global subscription list. WASM-hosted subscription handlers run on the publisher's tokio task via the async ABI; a slow handler does not delay other subscribers because each Invocation target is dispatched as a separate task.
Subscriptions for Before-class events are bounded in count per event; if a user installs 100 BeforeSave hooks, the save runs them in registration order and each gets a fuel budget. A handler that exhausts fuel logs and is skipped; the save proceeds.
5.10.5 Mode-owned synthetic buffers (the B' contract)
*lsp*, *messages*, future *scratch*, *compilation*, REPL buffers, plugin-emitted log streams: synthetic buffers whose content originates from a subsystem rather than user keystrokes. The contract for v1:
A mode owns the synthetic buffer it produces, end-to-end. No subsystem-specific code lives on the App. The contract splits across two host traits by their mutability:
// lattice-mode
// `&self`, thread-safe — READ / FIND + write-HANDLE. NOT creation:
// creating a named buffer must activate a mode (mutating the mode
// registry / active-modes / options cache), which needs `&mut Editor`.
pub trait BufferStore: Send + Sync {
fn find_by_name(&self, name: &str) -> Option<BufferId>;
fn handle_for(&self, id: BufferId) -> Option<DocumentHandle>;
fn insert_document_buffer(&self, id: BufferId, kind: BufferKind, /* … */);
}
// `&mut Editor`-backed — the mode-owned CREATION + activation seam.
pub trait ModeActivator {
/// Find-or-create the synthetic named `Document` and activate
/// `major` on it *by id* (runs `on_activate`). Returns the id;
/// idempotent (a second call reuses the buffer, no re-activation).
fn ensure_named_document(&mut self, name: &str, major: ModeId, flags: BufferFlags) -> BufferId;
fn activate_major_for_kind(&mut self, buffer: BufferId, kind: BufferKind);
fn activate_minor_by_id(&mut self, buffer: BufferId, mode: ModeId);
fn services(&self) -> Arc<ServiceRegistry>;
}
BufferStore is Send + Sync so a mode's tokio task can pull a handle from any thread. Creation lives on ModeActivator, not BufferStore — a &self handle cannot activate a mode. (An earlier BufferStore::ensure_named_document claimed find-or-create but its &self impl could only find, and panicked on a miss — removed.) ServiceRegistry is the typed lookup the App registers at boot — modes pull Arc<BufferStoreHandle> from their ModeContext.
Creation is the mode's responsibility, through one of two front-ends to the same primitive (Editor::ensure_named_synthetic_document), chosen by call context:
- Imperative — a provider's trigger function, called with
&mut dyn ModeActivator, invokesensure_named_documentdirectly (e.g. compilation'sstart_compilation: create the*compilation*buffer and kick off the run in one&mut Editoroperation). - Declarative — a pure ex-command's
applyclosure (which has no&mut Editor/ services in scope) returnsEffect::OpenSyntheticBuffer { name, mode_id }; the host applies it (e.g. AI-log / ACP).
Lifecycle:
- A trigger (either front-end above) creates the buffer + activates
major. - Activation runs
Mode::on_activate(ctx), which pulls the buffer'sDocumentHandleviactx.service::<BufferStoreHandle>()?.handle_for(buffer_id), subscribes to its typed source event (LspLogPushed,MessageEmitted,ReplOutput,CompilationOutputPushed, …), and spawns a tokio task that drains the subscription, coalesces records into batches, and applies oneDocumentHandle::apply_edit_batchper drain cycle. (The drain establishes before the producer's first record when the trigger creates the buffer synchronously ahead of publishing.) - Dropping the
Mode::Guard(deactivation) drops the subscription; the task exits. The buffer survives in the registry (frozen — no further appends) until:bdremoves it.
Why this matters:
- Extensibility (#2). Adding a new subsystem-owned buffer (DAP debug log, terminal output, plugin telemetry) is a new mode + a new typed event — zero App churn. Same recipe for built-in subsystems and plugin-installed ones (WIT-hosted modes pull the same
BufferStoreHandlecapability). - Asynchronicity (#4). The UI thread sees one batched edit per drain cycle per buffer — actor mailbox, no contention. Bursty subsystems (LSP trace traffic, REPL output) run on their own tokio tasks and never compete with rendering.
- Vim modal (#3). Synthetic buffers live in the same registry as path-backed Documents.
:b *lsp*,:ls,:bd,:bnoperate uniformly; no kind-specific branches. The mode'sReadOnlyoption contribution gates user writes.
Sharp edges to honor:
- No lock-across-
.await. TheBufferStoreimpl usesstd::sync::Mutex; the guard must not cross.await. Pull data out, drop the guard, then.await. Conventional in the codebase; not statically enforced. - Lazy creation. Creation (
ModeActivator::ensure_named_document, or theEffect::OpenSyntheticBufferfront-end) runs on the App / actor thread (ex-command / effect dispatch). Publishers that fire records before the buffer exists route through the event bus — the records buffer in the in-memory ring; first creation seeds the buffer from the ring.
5.10.6 messages-mode v1 (tracing-bridged echo log)
*messages* is the editor's audit log: every record App::set_message produces, plus every tracing::* event from the editor + plugins, flows through a single subscriber into one buffer.
Architecture:
messages-modeowns*messages*per §5.10.5.- The mode registers a
tracing::Subscriberat activate time. The subscriber receives everytracing::info!/warn!/ etc. invocation in the codebase, formatsHH:MM:SS [LEVEL] target: message, and appends to*messages*viaDocumentHandle. EchoLevelis a thin alias fortracing::Level; legacyApp::set_message(level, text)routes through a singletracing::event!call. One code path, one subscriber.- Typed option
messages.filter: Stringaccepts standardtracing-subscriberfilter syntax (editor=info,lsp=debug,grammar=trace). Live-editable via:set messages.filter=.... Per-subsystem verbosity for free;RUST_LOG-style filtering for power users. - Syntax highlighting:
messages-modecontributes a line-local highlight provider keyed by the bracketed level.TRACEdim,DEBUGcyan,INFOdefault,WARNyellow,ERRORred. Computed at append-time per new line; no rope re-parse. - Plugins (post-1.0 WIT host functions) get
host.log(level, target, msg)— flows through the same subscriber. Plugin telemetry shows up in*messages*without any plumbing.
Why this shape:
- Performance.
tracing's filter check is a const-time atomic load; dropped records cost ~10ns. The append is a batched edit viaDocumentHandlefrom the subscriber's task — never on the UI thread. - Extensibility.
tracingis the standard Rust observability primitive; subsystems already use it. Bridging into*messages*is one subscriber registration, not per-call-site changes. - Debugging. Cranking verbosity for a single subsystem (
:set messages.filter=lsp=trace) gives the same observation surface a Rust developer expects fromRUST_LOG. No new vocabulary.
5.11 Introspection and Help
Every registered primitive in the editor -- commands, options, events, modes, keybindings -- carries metadata. The metadata is mandatory at registration time, not optional documentation, and the :describe-... family of commands renders it on demand.
pub struct CommandSpec { // lattice-grammar/src/command.rs
pub id: CommandId,
pub name: String,
pub kind: CommandKind,
pub doc: String, // markdown, multi-paragraph
pub args_schema: Vec<ArgSpec>,
pub source: SourceLocation, // path, line, plugin (§5.11.1)
pub latency_class: LatencyClass,
}
pub struct ArgSpec {
pub name: &'static str,
pub kind: ArgKind, // typed (String, Path, Chord, ...)
pub doc: &'static str,
pub prompt: &'static str,
pub default: ArgDefault,
pub completion: Option<&'static str>, // named generator, e.g. "gen:files"
}
(The original sketch carried since_version / example_invocations / a Category — not built; the real spec is the shape above.)
Built-in introspection commands:
| Command | Opens | Content |
|---|---|---|
:describe-key <chord> | *help:key:<chord>* | Resolved command, arg presets, source layer, alternative bindings |
:describe-command <name> | *help:command:<name>* | Signature, doc, default keymap entries, examples |
:describe-option <name> | *help:option:<name>* | Type, default, current value, doc, group, validator |
:describe-event <kind> | *help:event:<kind>* | Payload, mutability, current subscribers |
:describe-mode <id> | *help:mode:<id>* | Major/minor mode summary, keymap, hooks, declared style mappings |
:describe-buffer | *help:buffer* | Current buffer's mode stack, encoding, options, keymap chain |
:apropos <pattern> | *help:apropos:<pattern>* | Fuzzy search across all metadata |
Each opens a buffer-backed help view (HelpContent / HelpBuffer in lattice-help, consistent with everything-is-a-buffer), painted through the cell-grid pipeline like any buffer; cross-references inside it are clickable / followable via standard motions.
Cost model. Metadata lives next to registrations and is only materialized when an introspection command runs. The catalog is queryable in O(1) by id and O(log N) by name; :apropos is a streaming picker (§5.9.7) over all metadata.
The keymap descriptor metadata that backs :describe-key -- the typed KeymapEntry rows, their forgery-prevented SourceLocation capture, and the catalog/registry consistency invariants -- is specified in docs/keymap-architecture.md §3.5 and §6.
5.11.1 Provenance: source-of-truth for every binding
Vim's :verbose set X? shows where an option was last changed; Emacs's C-h k only shows the function bound, not where the binding came from. Lattice unifies these: every registered / bound / set thing carries a SourceLocation recording where it was created, surfaced as a [[file:...]] link in every :describe-* output. The user can follow the link to inspect or edit the source.
pub struct SourceLocation {
pub layer: SourceLayer,
pub kind: SourceKind,
}
pub enum SourceLayer {
Builtin, UserConfig, ProjectConfig, Modeline, Runtime, Plugin(PluginId),
}
pub enum SourceKind {
File { path: PathBuf, line: Option<u32> },
CommandLine { history_index: usize }, // typed at `:`
MacroReplay { register: char, step: u32 }, // replayed
DotRepeat(Box<SourceLocation>), // chains transitively
Synthetic(&'static str), // <initial-load>, <test>
}
Forgery prevention is structural. There is no public API that takes a SourceLocation parameter. The four ways a SourceLocation can come into existence are:
- Built-in registration uses
#[track_caller]onregister_motion/register_operator/register_text_object/register_ex_command. The compiler captures the caller's(file, line)automatically -- the caller cannot supply or override it. Untrusted code can only call from a different(file, line), which is just being honest about where it actually is. - Static-slice rows use a per-row declarative macro (
keymap_entry!,option_spec!, ...).file!()andline!()expand at each row's invocation site, so the captured location matches the row. Thesourcefield on the underlying struct ispub(crate); the macro is the only construction path. - Trusted subsystems (config loader, plugin host bridge, runtime dispatcher) construct
SourceLocationfrom their own ground truth -- the loader knows which TOML file and line it parsed, the host knows the plugin's identity from itsStore<PluginCtx>, the dispatcher knows it's executing a:line. They reachpub(crate) insert_*registry methods directly. Visibility ispub(crate)today (everything trusted lives in the same crate); when cross-crate trusted subsystems land, visibility is granted via sealed-trait re-exports, never by exposing a public_atform. - Tests use
SourceLocation::synthetic("<test-fixture>")behind#[cfg(test)].
Determinism guarantee. A unit test (track_caller_captures_register_motion_call_site) registers a sentinel command at a known line and asserts the captured location matches. Any future refactor that breaks call-site capture -- wrapping register_motion in a dyn Fn dispatcher, hand-rolling source values somewhere, removing #[track_caller] from a helper in the chain -- fails CI on the line-number mismatch.
5.11.2 Generic introspection (Introspectable trait)
Every :describe-* target implements one trait:
pub trait Introspectable {
fn kind_label(&self) -> &'static str;
fn identifier(&self) -> String;
fn doc(&self) -> &str;
fn sources(&self) -> Vec<SourceEntry<'_>>;
fn extra_sections(&self) -> Vec<HelpSection> { Vec::new() }
}
pub fn render_introspection(item: &dyn Introspectable) -> Vec<String>;
render_introspection produces the help body in a uniform shape: identifier icon heading (e.g. ex:write : or motion:line-down →), doc, type-specific extra sections (e.g. Arguments: for commands), then one [[file:...]] link per source labelled (Defined at:, Bound at:, Subscribed at:, Last set at:, Overridden at:, Activated at:). Each :describe-X is a thin lookup-and-call.
extra_sections() is the open hook for type-specific structure: commands render their args_schema, options render their type and current value, events render their subscribers list, modes render their keymap and hooks. Adding a new registry means adding one trait impl; the renderer doesn't change.
Multiple sources per item are first-class. An option's :describe-option shows two source links: Defined at: for the registration (default-value source) and Last set at: for the most recent setter. A user-overridden built-in command shows Defined at: (the built-in) plus Overridden at: (the user config). The trait returns Vec<SourceEntry>; the renderer emits one labelled link per entry.
5.11.3 Completion pipeline
The :-line and (eventually) every minibuffer-shaped prompt run their candidates through a four-stage pipeline modelled after emacs's vertico / orderless / marginalia ecosystem -- composability is the architectural property, not a future ambition. The crate lattice-completion is a standalone library with its own test corpus; it depends on lattice-grammar for the CommandRegistry shape but does not depend on any UI crate.
// Each stage is a trait. Plugin authors target these.
pub trait CandidateGenerator { fn generate(&self, ctx: &GenerateContext) -> Vec<RawCandidate>; ... }
pub trait CandidateMatcher { fn matches(&self, query: &str, c: &RawCandidate) -> Option<(MatchScore, Vec<Range<usize>>)>; }
pub trait CandidateRanker { fn rank(&self, scored: &mut Vec<ScoredCandidate>); }
pub trait CandidateAnnotator { fn annotate(&self, candidate: &mut RenderedCandidate); }
// One assembled pipeline runs all four in sequence.
pub struct CompletionPipeline {
pub generators: Vec<Arc<dyn CandidateGenerator>>,
pub matcher: Arc<dyn CandidateMatcher>,
pub ranker: Arc<dyn CandidateRanker>,
pub annotators: Vec<Arc<dyn CandidateAnnotator>>,
}
Stages, mapped to the emacs analogue:
| Lattice stage | Emacs analogue |
|---|---|
CandidateGenerator | consult's sources |
CandidateMatcher | orderless (or default substring matching) |
CandidateRanker | scoring step (custom in vertico-prescient, etc.) |
CandidateAnnotator | marginalia |
Renderer (in lattice-ui-tui) | vertico |
Per-slot resolution. The :-line driver computes the current slot (command name, arg N, delimiter-syntax body) via current_slot(line, cursor, registry). The slot dictates which generator the pipeline uses for this query; the matcher / ranker / annotators come from the registry's user-configured defaults (cmdline.matcher = "match:fuzzy", etc.).
Forgery resistance. Every registration is #[track_caller]; the registry has pub(crate) insert_* companions. Same invariant as commands and keymap entries: no public API takes a SourceLocation.
Caching. Opt-in per generator via CandidateGenerator::cache_key returning Option<CacheKey>. The pipeline reads from a shared GeneratorCache before invoking generate; on miss it caches the produced candidate set. Each generator declares its own TTL via cache_ttl() (default Duration::MAX). Built-in cache strategies:
| Generator | cache_key | cache_ttl |
|---|---|---|
gen:commands | "gen:commands:v1:{generation}" (the CommandRegistry::generation() counter, bumped on every register/unregister, so a plugin load/unload invalidates the cache) | MAX |
gen:files | "gen:files:{dir}" | 1 second (filesystem mutates) |
gen:options (post-§5.12) | "gen:options:v{N}" | MAX until version bumps |
The matcher / ranker / annotators always run live; only generation is cached.
Built-ins shipped with lattice-completion:
| Stage | Built-in | Purpose |
|---|---|---|
| Generator | gen:commands | every CommandSpec |
| Generator | gen:files | filesystem walk |
| Matcher | match:prefix | exact-prefix (default) |
| Matcher | match:substring | case-insensitive contains |
| Matcher | match:fuzzy | subsequence with byte-range tracking; score decays with skipped chars; prefix-bonus |
| Ranker | rank:score | descending score (default) |
| Ranker | rank:alphabetical | A-Z |
| Annotator | anno:kind-label | → (motion), : (ex-command), f (file), etc. |
| Annotator | anno:doc-snippet | first line of doc |
Host-state generators (gen:chords, gen:registers, gen:marks, gen:buffers) live in the host crate (lattice-ui-tui) because they read App-level state; they register against the same CompletionRegistry like any plugin would.
Vertico-style rendering (post-popup work): a vertical list of candidates, one per row, with the matched byte ranges from ScoredCandidate.match_ranges painted with a distinct style. Annotations rendered right-aligned. Selected row marked. Renderer is replaceable -- when the rich minibuffer (§5.9.10) lands, the popup graduates to a tree-sitter-styled buffer view; the underlying RenderedCandidate shape doesn't change.
Insert-mode completion (Phase 4.2.g) is the editor surface that turns this pipeline into a buffer-level input flow: trigger evaluation per-keystroke, async sources (LSP / snippets / buffer-words / path / tree-sitter / plugin), multi-column popup display, side documentation popup with lazy completionItem/resolve, snippet engine with placeholder navigation, frequency-aware ranking. Spec lives in insert-completion.md; behavioural choices are explained alongside surveyed precedents (VS Code / Neovim blink.cmp / Helix / JetBrains / Sublime / Emacs corfu).
5.12 Configuration System (typed options + code-as-config)
Vim's :set option=value is a string-bag with no typing or validation, and vimscript fills the gaps with a string-shaped scripting language users have to learn separately. Emacs's customize is a typed system bridged awkwardly to setq for non-curated variables, and elisp fills the gaps with a second authoring environment plugin authors must also master. We unify both halves: a typed option registry for data, and the Rust→WASM plugin substrate (§5.5) reused as the code layer. There is no third surface and no second language.
Implementation status: the typed-option registry below ships in its own renderer-agnostic crate,
lattice-config. The current implementation uses a slightly tightened shape vs. the sketch (eachOption<T>is generic and owns its ownArcSwap<T>value cell; the registry storesArc<dyn ErasedOption>; consumers hold typedOptionHandle<T>for zero-overhead reads).:setsyntax,gen:optionscompletion,:describe-option, and theregister_core_options/register_*_optionsAPI all flow through this crate. Theoptions.toml+init.rslayers below are post-Phase-7 (gated on the WASM plugin host).
5.12.1 The typed option registry
The v0.5 model shipped: the option's type is its canonical identity, not a string name. Each option is a unique Rust type carrying its own value/default/doc, and OptionType is a trait (not an enum). The registry stores type-erased handles; consumers hold typed handles for zero-overhead reads:
// lattice-config: option_decl.rs / option.rs / option_type.rs
pub trait OptionType: 'static { /* Value assoc + parse/format */ }
// A built-in option is a unit type implementing this and carrying its metadata:
pub trait OptionDecl {
type Ty: OptionType;
const NAME: &'static str; // dotted path, e.g. "editor.line-numbers"
const DEFAULT: /* Ty::Value */;
const DOC: &'static str;
const CUSTOMIZABLE: bool;
}
// Each Option<T> owns an ArcSwap<T> value cell; the registry holds
// Arc<dyn ErasedOption>; reads are type-driven: config.get::<Tabstop>().
The registry is the single source of truth. :set, :describe-option, the gen:options completion source, the customize buffer, and any plugin call all read and write the same store. Cross-crate uniqueness is enforced by the type system; display-name uniqueness by linkme aggregation; naming rules by const fn assertions in the declaration macros (lattice-config-macros). The same model is mirrored for OptionGroup. See docs/mode-architecture.md §6.4 (option identity), §6.7.1.1 (groups), §6.8 (constraints).
Layered resolution: modal-state override → buffer-local explicit
setlocal→ active minor modes (in activation order, with explicit priority for tie-breaks) → major mode → global:set→ built-in default. Cached per buffer asResolvedOptions, invalidated on mode toggle / option write. O(1) hot-path reads. (mode-architecture.md§6.1 / §6.3.)Two front-ends on this single registry:
:set(cmdline, single-option, immediate) and:customize(form-buffer, group-oriented, buffer-backedcustomize-modemajor, TUI-first). Same store, same metadata, same validation -- different rendering. v1 ships both; TOML write-through deferred to v1.x. (mode-architecture.md§6.7.)
5.12.2 Two layers, both optional
Status. The
lattice.tomlstatic layer is ✅ built (lattice-configloader over XDG paths, plus.lattice/config.tomlfor project-local overrides). Theinit.rs→WASM layer is 📝 not built — it depends on editor-side plugin loading (Phase 8). The only shipped WASM-config piece is theconfig-plugin WIT seam (config_host.rs), which lets a plugin register options from a prebuilt component.
User configuration lives in two layered files (the config file is lattice.toml, not options.toml):
~/.config/lattice/ (XDG; + project-local .lattice/config.toml)
├── lattice.toml # ✅ static option overrides; data only; no toolchain needed
└── init.rs # 📝 planned: Rust → WASM, loaded as a plugin with `boot` capability
| Layer | Format | Toolchain | Loaded | What it expresses |
|---|---|---|---|---|
options.toml | TOML | none | deserialized into the option registry | typed option overrides; static keymap entries (chord → invocation string); static autocmds (event + filter + invocation string) |
init.rs | Rust → WASM Component | rustup + cargo-component (auto-detected; banner if absent) | loaded as a plugin via the §5.5 host with the boot capability | everything options.toml can express, plus closures, conditionals, custom command/motion/operator registration, autocmd handlers with logic, and any other plugin-shaped extension |
Either, both, or neither can be present. Both fall back to defaults when absent. Both reach the same internal state through the same functional API -- TOML via deserialize-then-call-the-setter, init.rs via WASM-call-into-the-host-which-calls-the-setter. There is no functional gap between them; the cost difference is "instant data load" vs. "5-15s first-boot compile, then cached".
The intended progression: a new user copies an options.toml example. When they outgrow declaration -- a keymap that needs context, a hook that does real work -- they migrate the affected piece into init.rs. The graduation cost is "learn the same API the plugin SDK exposes", because init.rs is a plugin -- the only thing distinguishing it from a third-party plugin is the boot capability and the well-known load path.
5.12.3 The init.rs plugin (📝 planned — Phase 8)
init.rs is a single source file. The host wraps it in a small generated crate (Cargo.toml, src/lib.rs shim, [package.metadata.component] entry) under ~/.cache/lattice/init-build/ and compiles it through cargo-component build against the published lattice-config-api crate, which re-exports the §5.5 / §9 WIT bindings under an ergonomic Rust-native shape.
// ~/.config/lattice/init.rs
use lattice::config::*;
#[lattice::init]
fn init(c: &mut Config) {
// Static settings (could equally live in options.toml).
c.set("editor.tabstop", 4);
c.set("editor.relativenumber", true);
// Programmatic -- the bit TOML can't do.
c.keymap("normal", "<C-s>", "write()");
c.keymap("normal", "<leader>fe", |ctx| {
let path = ctx.active_buffer().path()?;
ctx.invoke(format!("Tree(\"{}\")", path.parent().unwrap().display()))
});
c.autocmd("BeforeSave", "*.rs", "format()");
// Custom command registration -- same API third-party plugins use.
c.register_motion("motion:my-fancy-jump", "Jump to next paragraph header", |mctx| {
/* ... */
});
}
Config is the WIT-defined facet of the host plugins use, scoped to operations sensible at boot time: option setting, keymap registration, command registration, autocmd subscription, event-bus subscription, and the invoke(CommandInvocation) host call. Capabilities beyond boot (filesystem, network) are declared in the manifest the same way they would be for a third-party plugin and require the user's explicit acknowledgement; the boot capability alone is bounded.
5.12.4 Auto-build on first boot (📝 planned — Phase 8)
The user does not run a build command manually. Boot sequence:
- Read
options.tomlif present -- pure data; deserialize and apply. - Look for
~/.config/lattice/init.rs(or escalate toinit/if the user has split into a multi-file crate). - Compute cache key:
sha256(source + lattice_version + wit_revision). - Probe
~/.cache/lattice/init-<key>.wasm.- Hit: load via the §5.5 plugin host with the
bootcapability set; runinit(...). - Miss: spawn a background tokio task that materialises the build scaffold, runs
cargo-component build, places the artifact at~/.cache/lattice/init-<key>.wasm, then loads it. The UI shows a "Compiling config..." splash if the build doesn't complete within ~200 ms; cargo's stdout streams to:messages.
- Hit: load via the §5.5 plugin host with the
- If toolchain is missing (rustup / cargo-component), boot continues with defaults and a non-fatal banner: init.rs found but no Rust toolchain detected; install rustup + cargo-component to enable, or run
lattice config build --help. - If the build fails, boot continues with defaults; the compile error is rendered in a help-style buffer (Rust syntax-highlighted) reachable via
:describe-config-errorand surfaced as a non-fatal banner.
Subsequent boots are dominated by the dlopen-equivalent of the cached WASM artifact -- in the high-tens-of-microseconds range. The compile cost is one-time per source change or editor version.
A filesystem watcher on init.rs triggers an in-background recompile when the user saves; on success the host hot-swaps the loaded module (the §5.5 host already supports plugin reload). Live config feedback without restart, with the same compile-then-load pipeline; just incremental.
The lattice config build CLI subcommand is not on the user's critical path -- it exists as a debugging / scripting tool. Use cases:
- Pre-warming a cache on a fresh machine before first launch (dotfile-bootstrap scripts).
- Diagnosing "config didn't load" with full verbose cargo output.
- CI-checking a config (validate it compiles before pushing dotfiles changes).
- Cross-compiling and shipping a pre-built
init.wasmalongside the source for sharing -- the host loads the pre-built artifact when its hash matches the source.
5.12.5 Sources and precedence
Values come from layered sources, resolved in this order (later wins):
- Built-in defaults (each option's
DEFAULT). - Bundled config (default keymap, theme).
- User options (
~/.config/lattice/lattice.toml). ✅ - 📝 User init module (
init.rs→ WASM,bootcapability) — Phase 8, not built. - Project options (
.lattice/config.tomlat workspace root). ✅ - Per-buffer overrides (
:setlocal) + the mode-resolution stack (§5.12.1). - Programmatic /
:setinvocations during a session.
init.rs runs after options.toml is applied so it can read what TOML did and override or extend. Project-level init.rs is deferred -- arbitrary code execution by virtue of cd-ing into a directory is a real attack surface; the eventual mechanism is a per-directory trust prompt with a hashed allowlist (vim's :set exrc with explicit trust). Until that lands, project-local code-config is unsupported.
5.12.6 The :set parser front-end
:set option=value, :set option? (query), :set option! (negate), and :set option& (reset) parse into a typed ParsedSet (NameOnly / Query / Assign / Negate / Reset). Scope is chosen at the command level — :set / :setlocal / :setglobal — rather than a field. (📝 The compound += / ^= / -= append/prepend/subtract ops are not yet built.)
Validation runs at the dispatcher; type errors surface as a parse error on the command line.
:set is itself a registered command that dispatches through execute(...); init.rs's c.set(name, value) call lowers to the same invocation. There is one path that mutates an option, and it publishes the option's on_change event so subscribers (autocmds, :customize redraw, dependent options) react uniformly regardless of source.
5.12.7 Customize as a buffer-backed view
A built-in command (:customize, or the customize major mode) opens a buffer that lists every registered option grouped by group. The view is rendered with type-aware widgets: bools are checkboxes, enums are dropdowns, paths are completable strings, lists are addable-removable. Edits dispatch the same :set invocations described above. Saved customizations are written back to options.toml -- which the user can then move into init.rs if they need code around them. Filtering (/) and folding (za) work because it is a buffer.
5.12.8 Options are addressable from every entry point
- The
:setline. :describe-option <name>(introspection).- The customize buffer.
options.toml(deserializer-driven).init.rs(WIT host call:config.set/config.get).- A third-party plugin (same WIT call as
init.rs).
All six entry points produce or consume the same typed Value against the same OptionSpec. init.rs and third-party plugins use literally the same call; the only thing that distinguishes them is the capability set the host loaded them with.
5.12.9 Invariants
- No second config language, ever. TOML for data, Rust-WASM for code. Lua / vimscript / elisp / Rhai / Janet / a custom config DSL are all out of scope -- the doubling of API surface, binding maintenance, ecosystem fragmentation, and learning cost is the explicit cost we refuse. Users who want a no-toolchain logic surface use TOML's static keymap / autocmd entries; users who want logic install a Rust toolchain like every other Rust author. The graduation step from "config" to "third-party plugin" is purely packaging.
init.rsis a plugin. It is loaded by the §5.5 host, declared in WIT, capability-gated, fuel-limited, crash-isolated. The only thing privileging it is thebootcapability and the well-known load path. A bug ininit.rscannot crash the editor; it surfaces as a banner and falls back to defaults.- Exactly one path per concern. One option mutation path (
:set→execute(...)), oneCommandInvocationshape, oneEffectwire format, one cancellation token, one event bus. Config is a consumer of these surfaces, not a parallel surface. - Project-local code-config is gated on a trust mechanism. Until that mechanism ships, project-level overrides are TOML-only; project-level
init.rsis unsupported. - The compile cost is paid once per source change. Auto-build is a one-time event amortised across many boots; the user never types a build command on the happy path.
5.13 Diff System
Lattice ships a diff subsystem covering the four flows users expect from a modern editor: inline overlay against a baseline (file vs. disk, file vs. git HEAD, file vs. AI-proposed content), side-by-side two-way diff, three-pane three-way merge, and hunk motions / transfer operators (]c / [c / do / dp) registered through the central CommandRegistry (§5.2.1). The deep-dive design lives in diff-system.md; the headline architectural decisions are: hunks are data on the Core thread owned by a DiffSubsystem and RCU-published via arc-swap, not a window-local UI flag (vim's mistake); two-way and three-way share one data model with 1..=3 participating documents (ediff's correct structural choice); inline overlay and side-by-side are presentation transforms over that shared data, not separate subsystems (Zed's structural insight) — realized as a DiffSignMap plus overlay / fold / filler modules rather than one DiffMap type; the engine is imara-diff (Histogram default, what gitoxide ships) running on spawn_blocking. Two foundational primitives ship with the first slice: displacing virtual rows (deletion blocks, also consumed by multibuffer §5.14 and post-v1 inlay hints) and scroll-binding pane groups (used by side-by-side diff, vim's :set scrollbind, future :windo). Slice ledger is D.0–D.7 in ../operations/implementation.md.
5.14 Multibuffer Views
A MultibufferDocument is a Document whose content is composed of N anchored excerpts spliced from other Documents, where edits at the surface propagate back through the standard edit pipeline to the source buffers. This is the primitive that lights up project-wide diff, AI multi-file openDiff, search-as-buffer (wgrep-style), LSP references-as-buffer, and diagnostics-as-buffer with one implementation rather than five. The deep-dive design lives in multibuffer-views.md; the headline architectural decisions are: Document becomes a trait with RopeDocument and MultibufferDocument implementations (the load-bearing refactor); excerpts hold the existing anchor type (§5.1.1) and track source-buffer edits automatically; edit propagation is a translation-table lookup → dispatch through the standard pipeline — undo / macros / autocmds / LSP didChange observe one consistent edit stream regardless of pane; cross-pane coherence falls out of arc-swap (§5.6.8) because there is only one underlying buffer; providers populate excerpts by emitting mutations from a single task — built today as concrete search + narrow providers (📝 references / diagnostics / diff-as-multibuffer providers, and a generalized MultibufferProvider trait, are not yet built). Multibuffer and diff are independent designs that meet at their consumers (project-wide diff, AI multi-file edits), not at their implementations. Slice ledger is M.0–M.6 in ../operations/implementation.md.
5.15 Virtual Rows
A displacing virtual-row primitive sits as a sibling lane to CellMatrix (§5.6, cell-grid-renderer.md) and is shared by diff inline-overlay deletion blocks (§5.13), multibuffer excerpt headers and separators (§5.14), and future inlay-hint / code-lens / signature-preview consumers. The deep-dive design lives in virtual-rows.md; the headline architectural decisions are: virtual rows ride in their own VirtualRowMatrix RCU-published via ArcSwap, independent of CellMatrix so that diff/multibuffer churn does not invalidate the cell-builder's multi-axis cache; the renderer reads both lanes per frame and interleaves at slice() time via the additive CellMatrix::display_slice(scroll, height, virtual_rows) -> DisplaySlice<'_> method — existing slice() callers are unchanged, virtual-row consumers opt in; motions stay source-line-based (vim's j / k step document rows only — virtual rows are visual chrome, not navigation targets); the VirtualRowProvider trait gives diff, multibuffer, and post-v1 inlay/codelens emitters one shared registration surface. D.0a (the pure data + interleaver layer) landed 2026-05-28; the production virtual-rows worker (D.0a.1) and the first renderer consumer (D.3 inline diff) follow when their respective design fragments call for them.
5.16 Agent Integration
Lattice integrates two coding agents — Claude Code and opencode. The seam is the capability surface, not the transport: one EditorAccess port in lattice-agent serves every adapter, because "give me the selection", "write this file", "ask the user to approve this diff" mean one thing regardless of who opened the socket. The deep-dive designs live in agent-integration.md, ai-agent-protocol.md, and agent-ui.md.
Primary topology — the agent's TUI in a terminal buffer. Both agents are terminal-native (their own TUIs give prompt/readline, slash commands, model switching, and edit review), so v1 runs each in a lattice terminal buffer with a thin minor mode layered on: Claude Code dials into lattice over a loopback WebSocket / MCP for openDiff edit review (claude-code-mode); opencode's :opencode spawns its native TUI (Effect::SpawnTerminal + opencode-mode). This keeps everything-is-a-buffer (a terminal is a buffer), gives each agent's full UX for free, and avoids reimplementing a REPL lattice would always lose to the agent's own.
Alternative topology — lattice owns the conversation over headless ACP. :opencode-acp drives opencode acp headlessly and renders the conversation as a Document (*ai:opencode* under an ai-conversation major mode) — Normal-mode scrollback + an Insert-mode prompt via the generic editable-tail primitive (Mode::editable_tail(), the comint pattern) enforced by the host's read-only edit gate. Its one distinguishing win is that edit approval reuses lattice's diff subsystem (§5.13): MCP's openDiff and ACP's session/request_permission are two encodings of one review_diff verdict, the agent's write blocks on the user's accept/reject, review mode fails closed (reads auto-run, file-edits open a diff, un-reviewable mutating operations are denied unless the user opts into per-session trust mode), and a conversation/trace split keeps dialogue in the buffer and protocol diagnostics in per-process :ai-log rings. This path is kept as the substrate for a future IDE-native edit-review integration of the terminal path. Slice ledger: the "Agent integration" section in ../operations/implementation.md; archived slice plans under ../operations/slice-plans/archive/.
6. The Core Protocol
As-built note. The
CommandandEventenums in §6.1/§6.2 are an illustrative sketch of the client↔core protocol, not the built types. As shipped: clients drive the core through typedEditorHandlemethods (not a singleCommandenum with reply channels); the document actor's internal mailbox carries anActorMsg; and editing commands flow asCommandInvocationthrough the unified dispatcher (§5.2.1). The outbound event type islattice_protocol::Event— richer than the §6.2 sketch (it carries UI events,BeforeSave/BeforeQuit/OptionChanged, and the LSP/plugin events, with different mode-event names). Read §6.1/§6.2 as the shape of the contract, not the enum definitions to code against.
6.1 Commands (clients to core)
Reconciliation note (v0.6). The monolithic client→core
Commandenum below was retired (its removal is documented inlattice-protocol/src/lib.rs). There is no single wire enum; the real protocol is a typed actor handle plusCommandInvocation.
Clients drive the core through the per-document actor handle (RopeDocumentHandle in lattice-runtime), whose typed methods each return a Pending<T> (an awaitable-or-droppable oneshot):
impl RopeDocumentHandle {
fn snapshot(&self) -> Arc<DocumentSnapshot>; // wait-free read
fn dispatch(&self, inv: CommandInvocation, env: DispatchEnv) -> Pending<Effect>;
fn apply_edit(&self, edit: Edit) -> Pending<AppliedEdit>;
// … open / save / undo / redo etc.
}
Everything a client wants to do — a keystroke, a : line, a plugin operator, an LSP-mediated action — is expressed as a CommandInvocation (§5.2.1) dispatched through this handle; the grammar dispatcher runs it synchronously on the actor and the result comes back as an Effect. Registration of new commands / motions / etc. is a direct CommandRegistry call at boot (§5.2.4), not a wire message. UI-side concerns (popups, pickers, notifications, status segments) are host/renderer state, not core-protocol messages.
6.2 Events (core to clients)
The protocol-level Event enum (lattice-protocol/src/event.rs) is the reconciled list from §5.10.1 — the earlier 25-variant sketch with UI / LSP / picker events was narrowed; those are subsystem-typed sub-buses now, not core protocol:
enum Event {
DocumentOpened { id, path, version, text },
BeforeSave { .. }, DocumentSaved { .. }, DocumentClosed { .. }, DocumentChanged { .. },
SelectionsChanged { .. },
ModalModeChanged { from, to },
OptionChanged { .. },
MajorEntered { buffer, major }, MajorExiting { buffer, major },
MinorActivated { buffer, minor }, MinorDeactivated { buffer, minor },
Plugin(PluginEventPayload), PluginCrashed { .. },
}
(LSP / diagnostics / UI events flow over their own typed buses — see §5.10.1.)
6.3 Wire format
In-process only. The editor runs as a single process; the core protocol is tokio mpsc channels carrying typed values, zero serialization. The earlier cross-process MessagePack-over-socket framing was abandoned (documented in lattice-protocol/src/lib.rs). A JSON-RPC codec exists but for the LSP client (and a peer-agent channel), not the core editor protocol. Cross-process / remote-headless is 📝 not pursued in v1 (the TUI peer covers headless / SSH).
7. Data Flow Examples
7.1 User types x in normal mode (delete character) -- code buffer
UI receives KeyEvent -> translates to a CommandInvocation -> handle.dispatch(inv) -> the editor actor runs execute(...) -> returns Effect::Edits -> buffer mutates -> Event::DocumentChanged -> tree-sitter reparse on a spawn_blocking worker + cell-grid rebuild, LSP didChange (debounced), plugins notified, new snapshot published -> next frame renders.
End-to-end 1-7: <2ms. Background work in parallel.
7.2 User types in a markdown heading -- rich buffer
Same as above through buffer mutation. (📝 The shaped rich-buffer path — per-line layout cache + Fenwick height index on a spawn_blocking worker — is not built; markdown renders as styled monospace cells today, §5.6.2.)
End-to-end input: <2ms regardless of shaping completion.
7.3 User triggers completion (insert mode, after .)
Completion trigger -> LSP request (via the lattice-completion pipeline + gen:lsp source) -> 50-500ms wait (user keeps typing) -> version-aware cancellation if buffer changes -> eventual matching response published to the completion sub-state -> the completion popup (a buffer view) updates -> first paint fast; cached afterward.
7.4 User opens command palette (Ctrl+Shift+P)
1. UI receives keypress, recognizes binding for "open-command-palette".
2. UI sends Command::OpenPicker with spec:
- content_provider: built-in CommandPaletteProvider (returns all registered commands)
- item_renderer: shows command name + binding + description
- preview_provider: None
3. Picker overlay appears (a centered buffer-backed view), with input field focused.
4. User types "form".
5. Picker filters items via fuzzy match on each provider yield.
6. User presses Enter on "format-buffer".
7. Picker emits Event::PickerClosed with selected item.
8. Selection callback runs the format-buffer command.
Throughout: editor pane keeps rendering, LSP keeps running, no other UI work is blocked.
7.5 A plugin crashes mid-render
1. Plugin contributes a status segment showing "git branch".
2. Plugin's content provider, when called, panics.
3. Wasmtime catches the trap.
4. Core marks the segment as failed, logs the error.
5. Status line renders without that segment (others unaffected).
6. Notification posted: "Plugin 'git-info' crashed (segment disabled)".
7. User sees the notification; the rest of the editor is unaffected.
8. Performance Strategy
8.1 Invariants
- UI thread does no I/O, no parsing, no shaping.
- No mutex on the buffer. Actor pattern.
- Every async operation has cancellation.
- Plugins cannot block the host (wasmtime async + fuel).
- Allocations on the input path are bounded.
- The cell grid is built off-thread (
spawn_blocking); the render thread only reads a publishedDisplayMatrix. - 📝 (planned) Text shaping for rich buffers happens on workers — the shaped path is unbuilt; markdown renders as monospace cells today.
- UI furniture (popups, buffer-backed views, status) is layout-once, cached, off the input path.
8.2 Performance commitments per path
The shape here is "what's the physically credible best we can hit on this path, and what do we commit to ship at v1.0?" not "how much margin do we want above neovim?" Where the architecture genuinely permits ns numbers (atomic loads, format-only segments), we don't settle for µs targets just because incumbent editors do.
Columns
- Floor. The physics-credible best given our architecture -- derived from the cost of the underlying primitives (atomic acquire-load,
parking_lotmutex acquire, tokio cross-task wakeup, ropey rope op, ratatui draw, tree-sitter parse). When a row's floor is bounded by a known-hard limit (cache bandwidth, scheduler latency, allocator), we say so in the rationale. This isn't aspirational -- it's "what microbenchmarks of the individual ops add up to." - Target (v1). What every implementation MUST hit by v1.0. Tighter than the Today column on rows where we know the engineering path; relaxed where the Today column reflects a legitimate trade we're keeping. CI enforcement (as-built): CI records the benchmark medians as baselines (
benchmarks.md) and fails only on a handful of absolute-ceiling ratchet tests on specific hot-path rows (e.g.perf_ratchet.rs: the grammar round-trip ceiling, the typed-call / grammar p99 budgets) — not a blanket ">10% vs. main on any benchmark" gate. The ratchet bar only moves down; the ceilings are generous enough to catch a gross regression without tripping on runner variance. - Today. The current
docs/../operations/benchmarks.mdmedian. "—" / "n/a" means unmeasured. These cells are a point-in-time snapshot and drift — the live../operations/benchmarks.mdis the authoritative current number; reconcile there, not against a hand-transcribed figure here (which may lag by a few µs / ns as the CI benches move). - Stretch. Credible with N-months-of-known-engineering, not novel research. Cited paths: GPU renderer, suffix-array search index, single-thread tokio runtime, sync edit fast-path, tree-sitter
Tree::editdeltas threaded through the actor.
The "vs neovim" framing the previous revision carried is deliberately gone -- the targets here are derived from primitive costs, not from being "X× faster than vim." Where we end up significantly faster than incumbents on a row, the rationale column says why our architecture permits it; where we're constrained by physics (cache bandwidth on a 200k-line full-buffer search; tokio scheduler latency on a multi-thread async actor), the rationale states the constraint.
Read path (renderer reads buffer state)
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
Snapshot load (renderer, load_full) | ~16ns | <20ns | 16ns | ⏹️ | At the floor for load_full semantics: one atomic acquire-load + one Arc bump. The renderer's hot path uses Cache::load instead. |
Snapshot load (renderer, Cache::load) | ~300ps | <500ps | 305ps | ⏹️ | Wait-free thread-local-cached steady-state load: one Relaxed atomic compare, register-cached Arc returned. ~50× faster than load_full -- below 1ns. |
| Status segment update (1 snapshot read + format) | ~50ns | <100ns | 56ns | ⏹️ | Measured ~56ns. One ArcSwap::load_full (~17ns) + Arc deref + format! of a few u64s. Already at the practical floor. |
| Frame render (code, TUI, 80×24) | ~200µs | <500µs | highlight 178µs + compose 14µs = ~192µs | <100µs | compose_visible_lines is fast (~14µs viewport-bounded); highlight dominates. Stretch is a viewport-bounded highlight cache that survives across frames. |
| Frame render (code, TUI, 200×60) | ~325µs | <800µs | highlight 289µs + compose 35µs = ~324µs | <200µs | Same shape, larger viewport. Linear in highlighted-line count; the per-frame compose cost is essentially free. Compose dropped from 40µs to 35µs after the renderer moved to Cache::load (one snapshot pinned at frame start; no internal load_full per inner helper). |
| Frame render (code, GPU, 1080p) | ~150µs | <1ms | n/a | <300µs | Variable-font shaping cached per-line; only diff repaints. GPU path post v1 design (§5.6). |
| Frame render (markdown, GPU, 1080p) | ~600µs | <3ms | n/a | <1.5ms | Per-line layout cache + Fenwick height index; floor scales with shape-changed lines. |
| Highlight span cache hit (steady-state) | ~10ns | <50ns | 20ns | ⏹️ | Cache keyed on (snapshot_ptr, text_version, scroll, viewport_height, fold_hash). Steady-state norm -- cursor blinking, no edit -- ~100% hit; miss falls through to highlight_lines (~178µs). Pre-B.3 ran the full QueryCursor walk every frame regardless; B.3 drops it to a key compare + early return -- ~8900× speedup on the steady-state frame. |
Write path (mutation → published snapshot)
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
Snapshot publish standalone (from_document + ArcSwap::store) | ~80ns | <500ns | 101ns | <50ns | Measured ~101ns constant across buffer sizes -- Buffer::clone is O(1) Arc bump; the cost is the allocator (Arc::new) + atomic release-store. Below original 2µs target. |
InputEdit construction (per Document::apply_edit) | ~2ns | <10ns | 1.87ns | ⏹️ | Six u32 writes on the stack alongside AppliedEdit. Below the regime where caching matters; floor is "no allocation, no work beyond field copies." Measured at §8.2's floor on landing (B.1). |
tree.edit() per-edit (syntax worker, sync pre-step) | O(N) | scale-by-size | 4.4µs (80 lines) / 163µs (1600) / 4.0ms (16k lines) | -- | tree-sitter walks every node updating byte/Position fields -- O(num_nodes), NOT constant. Initial 500ns floor estimate was wrong; real scaling is per-node. For typical files (sub-1000 lines) this stays sub-100µs. Pathological for huge files; the 256-edit cap on coalesced bursts limits worst-case multiplier. Worker-side, not input-thread. |
| Buffer ship to syntax worker (input thread, per keystroke) | ~10ns | <50ns | 7.7ns | ⏹️ | Slice B.5: input thread sends Buffer (O(1) Arc bump on ropey's internal sharing) instead of pre-materialized String. Worker calls buffer.as_string() on its own thread. 24,500× faster than the pre-B.5 path at 100k lines (189µs → 7.7ns); honours goal #1 strictly even on huge files. |
Apply-edit (sync fast-path, with_document_mut(closure)) (deferred to Phase 7) | ~5µs | not v1 | not built | <2µs | parking_lot mutex acquire (~30ns) + ropey op (~1-3µs) + snapshot publish (~500ns). Deferred because the plugin-holds-mutex starvation risk is unbounded under today's cooperative-only timeout enforcement. Lands once Phase 7's WASM fuel makes plugin call duration infrastructure-bounded (rather than discipline-bounded). Until then the actor envelope below is the keystroke floor; that's well under §8.2's <16ms-per-frame budget so we're not actually constrained. |
Apply-edit round-trip (async actor, block_on(handle.apply_edit(...))) | ~50µs | <100µs | 85µs | <50µs | Two cross-thread tokio wakeups (~30µs each) + actor work. Floor is scheduler-bound on a multi-thread runtime; single-thread runtime would close to ~30µs. |
| Dispatch round-trip (motion + Effect commit) | ~50µs | <100µs (small bufs) | 78µs (10 lines) / 86µs (1k) / 513µs (50k motion walk) | <50µs | Scheduler-bound on small buffers (matches apply-edit envelope). On large buffers the motion's own walk dominates -- the word_forward walk on 50kloc is the cost, not the envelope. |
| Keystroke to glyph (code, TUI) | ~250µs | <2ms | unmeasured | <800µs | Actor-envelope edit + incremental reparse (B.2, async, off keystroke) + viewport highlight (B.3 cache, 20ns hit) + frame render. Incremental reparse landed; remaining stretch is the sync edit fast-path (deferred to Phase 7 per the row above). |
| Keystroke to glyph (code, GPU) | ~200µs | <2ms | n/a | <500µs | Same minus ratatui's terminal-write overhead. |
| Keystroke to glyph (code w/ LSP) | ~300µs | <3ms | n/a | <800µs | Decoupled: LSP results land on a later frame; the keystroke itself doesn't wait. Floor unchanged from non-LSP. |
| Keystroke to glyph (markdown render, GPU) | ~700µs | <5ms | n/a | <2ms | Inline-shape cost dominates; per-line layout cache lets unchanged lines reuse glyph runs. |
Search
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
| Literal substring, first-match-near-cursor | ~50ns | <1µs | 200ns–2µs | <100ns | memmem on one rope chunk. Floor is the SIMD prefilter cost. Today's number is fancy-regex's per-call setup; trivial-pattern fast-path could land at floor. |
| Literal substring, worst-case 200k buffer | ~300µs | <2ms | 659µs | <50µs (post-1.0) | L2 bandwidth limit on a sequential scan. Stretch needs suffix-array index (~5× memory; rebuild on edit; deferred). |
| Regex typical (lazy DFA + literal prefilter) | ~20µs | <2ms | 1.1ms | <500µs | regex crate's lazy DFA. Stretch via larger scan window amortising per-call setup. |
| Regex pathological (backref) | n/a (bounded) | abort at 50ms | 169ms (50k) | abort at 50ms | fancy-regex backtracking; bounded by 1M-iteration recursion limit. Per-search timeout via cancellation token (§5.2.5) is the credible bound. |
File open + parse
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
| Open 100MB log (first paint, viewport only) | ~80ms | <100ms | 76ms (rope only, render unmeasured) | <30ms | ropey rope construction at 1.3 GiB/s -- 76ms for 100MB measured. Initial viewport render is on top; well within budget. Tree-sitter parse runs in background; first paint shows raw text immediately. |
| Open 100MB log (full ready, syntax + folds) | ~80ms | <500ms | unmeasured | <200ms | Tree-sitter full parse (50ms-class on 100MB depending on grammar) + initial fold compute. |
| Open 10K-line markdown (full ready) | ~50ms | <500ms | n/a | <200ms | Block parse + inline injection per visible paragraph (📝 the shaped layout-cache path is unbuilt; renders as monospace cells today). |
| Tree-sitter incremental reparse (1-char edit) | scale-by-size | scale-by-size | 594µs (80 lines) / 325µs (1600) / 1.77ms (16k lines) | -- | Tree::edit byte-delta + Parser::parse(.., Some(&old_tree)) reuses unchanged subtrees. Beats full reparse 8× at 1600 lines, 14× at 16k lines. Loses at <100 lines (594µs vs full 246µs) -- tree-sitter's incremental algorithm has setup overhead that doesn't pay off for tiny inputs; both paths are sub-ms so nobody notices. Owned-Tree work in Option B + B.2's InputEdit threading lit this up. |
| Tree-sitter full reparse | scale-by-size | scale-by-size | 246µs (80 lines) / 2.5ms (1600) / 25.5ms (16k lines) | -- | Falsification anchor for the incremental row above + the file-load / cold-start path. At 16k lines this exceeds the 16ms-at-60Hz frame budget, which is why incremental matters; the keystroke path uses incremental, this row's the path file-load takes once. |
Folds
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
| Fold recompute (indent, 200-fn rust) | ~30µs | <100µs | 33µs | ⏹️ | Linear single-pass scan; near floor. |
| Fold recompute (markdown, 100 sections) | ~5µs | <50µs | 6.3µs | ⏹️ | Linear ATX heading walk; near floor. |
| Fold recompute (syntax, 200-fn rust) | ~3ms | <5ms | 3.9ms | <1ms | QueryCursor::matches traversal across many pattern alternatives. Stretch via per-pattern caching + pruning never-folded captures. |
Plugin host (§5.5; ✅ Phase 7 shipped — CI-gated)
| Operation | Floor | Target (v1) | Today | Stretch | Rationale |
|---|---|---|---|---|---|
| Typed host fn call (1 scalar in, 1 out) | ~150ns | <500ns | CI-gated <500ns p99 | <100ns | wasmtime trampoline + 2 word copies. Floor is Cranelift's ABI marshalling. |
| Grammar-extension round-trip | ~2µs | <5µs | ~340ns release (CI-gated <5µs p99) | <1µs | Two trampolines + closure invocation. Wasmtime AOT closes most of this (PH7.7d). |
| Cold start, 50 lazily-loaded plugins | ~10ms | <30ms | n/a | <5ms | Module deserialise + import resolution per plugin. Disk cache amortises across runs. |
Architectural levers, by row
The Floor / Target / Stretch numbers above are not asserted in isolation -- specific architecture decisions enable each one:
ArcSwap::Cache::load(~2ns) is the renderer's read floor; the full design.md §5.6.8 split between editor thread (writer) and render thread (reader) is what permits one atomic primitive on the read path.- Sync edit fast-path (planned) drops the keystroke round-trip from the actor's 85µs envelope to ~5µs by bypassing the mailbox for the editor thread's own writes (see #191).
- Owned
tree_sitter::Parser+Tree(Option B, post-Step-4) collapses the previous dual-parse (one for highlight + one for folds) onto a single parse per keystroke; folds, highlights, and any future query consumer all walk the sameTree. - InputEdit threading + incremental
Parser::parse(text, Some(&old_tree))(Option B.1 / B.2) shrinks the keystroke→fresh-tree window from full-reparse (~5ms on 50kloc) to incremental (~50µs floor). Backs the reparse row's tightened v1 Target. Worker still runs the parse onspawn_blocking; input thread emits the InputEdit alongsideAppliedEdit(~2ns) and the worker appliestree.edit()as the sync pre-step before the async parse. - Frame-level highlight span cache (Option B.3) drops steady-state per-frame highlight cost from ~178µs to ~10ns by skipping recomputation when no input changed. Cache key is
(text_version, viewport_range, fold_hash); invalidation is one comparison. Load-bearing for paramount goal #1's strict reading on the steady-state floor -- without it the input thread spends ~178µs/frame doing recoverable work. - Buffer-not-String to the syntax worker (Option B.5) drops the per-keystroke input-thread cost of preparing the source for the worker from O(n) (
Document::text()→ freshString, 189µs at 100k lines) to O(1) (Buffer::clone()→ Arc bump, ~8ns). Worker materialises the bytes viabuffer.as_string()on itsspawn_blockingthread. Honours goal #1 strictly regardless of buffer size; without it, B.2's incremental- reparse win was capped by an O(n) input-thread alloc. #[tokio::main]+ nested-runtime-safeblock_on(Slice C.1) ensuresHandle::try_current()succeeds from program start so the syntax worker actually spawns. Pre-C.1,mainwas synchronous and the seeded handle's worker silently never started -- Option B's incremental-reparse pipeline routed every request to a dropped channel.block_in_placewrap lets existingapply_edit_blockingetc.block_oncalls work from inside the runtime without panicking.- Intermediate-snapshot publish + synchronous span shifting (Slices C.2/C.3/C.4): worker publishes a byte-aligned snapshot immediately after
tree.edit()(before parse); App also synchronously line-shifts and byte-shiftsvisible_highlightson the input thread for every edit. Combined, the renderer never sees empty / positionally-wrong spans during the parse window -- spans only ever transition from one CORRECT set to another. Eliminates the user-visible "flicker on>>/dd" that survived all the B-series work because spans had no way to track byte movements before the worker caught up. - Grammar-driven-edits go through
publish_document_changed(Slice C.5): operators (>>,dd,cc,c,y,x, every mutation that flows through the dispatcher → actor) reach the same chokepoint as direct App edits (backspace, paste, type-key). Without this, the C.x synchronous-shift logic and the B.5 EditDelta accumulation skipped the most common edit shape; the worker fell back to full reparse on every operator and LSPdidChangenever fired. Now uniform. - memmem-driven literal search on rope chunks (B-α + B-β) replaces a naive char-by-char walk with SIMD-prefiltered scanning; backs the search floors above.
- Per-call WASM overhead budgeted in CI prevents plugin-introduced regressions from creeping into any of the bolded targets.
- Latency classes (§5.2.5) make per-call budgets enforceable; any code claiming the keystroke path declares which class it belongs to so the arithmetic doesn't drift.
CI records every benchmark median as a baseline (benchmarks.md) and hard-fails on the absolute-ceiling ratchet tests for the load-bearing hot-path rows (perf_ratchet.rs); the ratchet bar only moves down. A blanket ">10% vs. main on every benchmark" gate is not in place today — the ceilings are deliberately generous (≥100× headroom on some rows) to catch a gross regression without tripping on runner variance.
8.3 Memory
- Rope memory shared via Arc; snapshots ~free.
- Tree-sitter trees compact; old trees dropped on swap.
- Per-line layout cache (shaped buffers): ~200 bytes/line. 100K-line markdown: ~20MB.
- Glyph atlas: bounded LRU; cap 64MB default.
- Plugin memory capped at instantiation (default 64MB per plugin).
- Status segments / panel content: bounded by visible UI surface.
9. Plugin API
9.1 Principles
- Capability-based. Filesystem, network, subprocess access is granted explicitly per plugin and enforced by the wasmtime runtime.
- No global mutation APIs; submit edits. All state changes flow through the actor-protected document.
- Events are facts, not requests. Plugins observe; they don't ask the editor to do things by mutating event data.
- The plugin surface is data, not code. Plugins emit content (segment text, decoration ranges, picker items), not draw calls.
- Async by default. The Component Model async ABI is the canonical pattern. Host functions yield; the plugin task suspends; the OS thread runs other work.
- The grammar is the API. Operators, motions, text objects, registers, ranges, counts -- all are first-class WIT types and first-class extension points.
- Cross-language by construction. WIT is the source of truth. Plugins ship in any language with component-model toolchain support; Rust ships with first-party convenience bindings via
wit-bindgen.
9.2 What plugins can do
Buffer operations:
- Subscribe to events
- Read buffer content via resource handles (no copies; range-based slicing)
- Submit edits
- Persist plugin-local state
Mode contributions:
- Register major modes
- Register minor modes
- Activate / deactivate minor modes per buffer
Grammar contributions (first-class):
- Register motions
- Register text objects
- Register operators
- Register ex-commands and ex-ranges
Command contributions:
- Register named commands callable from the command line or keymap
- Register keymaps (with conflict detection against the layered keymap stack)
- Invoke any built-in or registered command (composition is plugin-visible)
UI contributions:
- Register status segments (mode line, header line)
- Register gutter segments
- Register buffer-backed views (file-tree, outline, diagnostics-list, terminal, REPL, ...)
- Open pickers (with custom content / item / preview providers)
- Show popups anchored to positions
- Post notifications (with optional actions)
- Add inline decorations (squiggles, virtual text, hints)
External I/O (capability-gated):
- Spawn subprocesses (
cap-subprocess) - Read / write files (
cap-fs-read,cap-fs-write, scoped to a path prefix) - Make HTTP requests (
cap-net-http, scoped to host allowlist)
9.3 What plugins cannot do
- Render arbitrary GPU content into the editor view (no draw calls; emit data only).
- Block or stall any other plugin or the UI (enforced by the async ABI + fuel + per-task isolation).
- Read or modify another plugin's state.
- Modify built-in keymaps directly (overrides go through the layered keymap stack).
- Access the raw filesystem outside their capability grant.
- Spawn native OS threads (use async tasks via host primitives instead; see §5.5.1).
9.4 WIT interface
Reconciliation note (v0.6). The original sketch (an
import-host-functions-that-return-idsmodel with a monolithicuiinterface and a richhost-servicesoftree-sitter-query/ripgrep-search/http-request/spawn) was not what shipped. The Phase-7wit/package inverts it: guests export per-capability "source" worlds that the host drives, plus a grammar callback trampoline.ui.witandcommand.witare empty stubs;host-servicesis minimal. The real shape:
The package is lattice:plugin-host@0.1.0 (14 wit/ files: types, plugin, buffer, grammar, picker-source, completion-source, decorations, events, config, modes, host-services, plus ui / command stubs and a test fixture).
The model: guests export worlds the host calls. A picker/completion/decoration plugin exports its source world; the host instantiates it and drives it (init / accept / gutter-decorations / …). The guest doesn't call register-picker(...) and get an id back; the host owns the registry and pulls.
Illustrative sketch, not the canonical package. The built WIT is
wit/*.wit(lattice:plugin-host@0.1.0) — the exercised seams (picker-source / completion-source / grammar / events / decorations / config / modes / keymap / host-services / logging), seeplugin-host.md+plugin-observability.md. Designed-but-not-yet-built seams: the tree-sitter query seam (plugin-treesitter-seam.md, v1), and the lighthouse host-services extensions (http-fetch/spawn-process+ task surface /register-server,lighthouse.md). The block below is kept for the shape of the contract.
package lattice:plugin-host@0.1.0;
// buffer.wit — a READ-ONLY document resource (no apply-edits from a source world)
interface buffer {
resource document {
version: func() -> u64;
line-count: func() -> u64;
text-range: func(start: u32, end: u32) -> result<string, error>;
}
}
// grammar.wit — register-by-name + a CALLBACK TRAMPOLINE (host holds callback ids)
interface grammar {
register-motion: func(name: string, doc: string, spec: motion-spec, callback: u32);
register-operator: func(name: string, doc: string, spec: operator-spec, callback: u32);
register-text-object: func(name: string, doc: string, spec: text-object-spec, callback: u32);
register-ex-command: func(name: string, doc: string, spec: ex-command-spec, callback: u32);
}
interface grammar-callbacks { // EXPORTED by the guest; host invokes by id
eval-motion: func(callback: u32, ctx: motion-ctx) -> result<motion-result, command-error>;
// … eval-operator / eval-text-object / parse-ex / apply-ex
}
// picker-source.wit / completion-source.wit / decorations.wit — guest-EXPORTED worlds
interface picker-source { // host drives these
init: func(args: list<string>) -> picker-init-result;
accept: func(item: u32) -> picker-accept;
}
// host-services.wit — MINIMAL (no ts-query / ripgrep / http / spawn)
interface host-services {
walk: func(/* introspection walk */) -> list<catalog-entry>;
register-event: func(kind: string) -> sub-id;
emit-event: func(payload: event-payload);
}
// events.wit / config.wit / modes.wit — event subscribe+emit, option contribution, mode contribution
// ui.wit, command.wit — EMPTY STUBS (reserved)
interface plugin { // lifecycle, exported by every plugin
activate: func() -> result<_, error>;
on-event: func(e: event) -> result<_, error>;
deactivate: func() -> result<_, error>;
}
Rationale for the inversion: a source world the host pulls keeps the registry, lifecycle, and teardown host-owned (no plugin-held ids to leak on crash), and the callback-trampoline keeps grammar evaluators synchronous-on-keystroke without a round-trip through a guest-side registry. The rich ui surface (popups, pickers, status, notifications) and the heavy host-services (tree-sitter-query, ripgrep, http, spawn) from the sketch are 📝 not built — plugins that need those wait on later capability seams.
9.5 Concurrency for plugin authors
Plugins are written async-first. All host-services calls are async; the runtime takes the WASM stack out of execution at every host call so other tasks (other plugins, the document actor, LSP I/O) keep moving.
Background work is expressed as additional plugin-side tasks composed via the host's async primitives. There is no std::thread::spawn; there is host-services.spawn-task(future) (or the language-binding equivalent) which schedules onto the host's tokio runtime.
The §5.5.1 concurrency model gives each plugin its own wasmtime::Store, which means:
- A plugin's tasks run on whatever tokio worker thread is free; plugin work is genuinely parallel across plugins.
- A plugin invocation that triggers re-entry into another plugin (e.g., A registers a motion that calls B's command) is composed via the host -- there is no recursive re-entry into a single Store.
9.6 Performance contract
Per §5.5.2, every WIT host function has a budget enforced in CI. Plugin authors see these as guarantees:
- Host-call latency is bounded -- typed call returns in < 500ns p99 with negligible fuel cost.
- Buffer access is zero-copy at the slice level --
get-text-rangereturns a string view backed by host memory; the boundary cost is the call, not the data. - No per-frame plugin work. The renderer does not call into plugins on the UI tick. Plugins compute on triggers; the renderer reads cached results.
9.7 Reference plugins shipping with v1.0
📝 Status. Only
fuzzy-finderexists as an actual WASM component (a Phase-7 seam-parity fixture, not yet a loaded plugin). Every other entry is either built natively (not as a plugin) or unbuilt: git markers →lattice-diff(native); markdown/rust/python modes → nativelattice-syntaxlanguage modes; file-tree → nativelattice-file-tree; outline →:lsp-symbols; diagnostics-list → the native:diagnosticsbuffer; tree-sitter-motions → shipped natively inlattice-syntax(§5.2.4). Repackaging these as WASM components is Phase 8b work.
fuzzy-finder-- file / symbol / buffer pickers (validates picker primitive end-to-end). (the one real component)git-gutter-- 📝 (native today vialattice-diff).linter-bridge-- 📝 adapter for non-LSP linters.markdown-mode/rust-mode/python-mode/javascript-mode-- 📝 (native language modes today).file-tree-- 📝 (nativelattice-file-treetoday).outline-- 📝 (native:lsp-symbolstoday).diagnostics-list-- 📝 (native:diagnosticstoday).tree-sitter-motions-- ✅ shipped natively inlattice-syntax, not as a plugin (]fnext function,af/acobjects, etc.).
The reference plugins exercise every primitive: pickers, popups, buffer-backed views, status segments, gutter segments, modes, decorations, notifications, grammar extensions. If any is painful to write, the API needs to grow.
10. Configuration and Extension Tiers
Two tiers: TOML and WASM. TOML (lattice.toml) covers configuration -- options today; keymaps / layouts / theme selection are partly TOML, partly mode-owned. WASM (Component Model + WIT) is the intended single substrate for everything else: extensions, custom motions/operators/text-objects, plugin-provided modes, and live evaluation.
📝 Status. The TOML option tier is ✅ built (
lattice-config). The WASM tier's editor-side consumption (userinit.rs, live-eval, the layout / keymap TOML expansions in the example below) is Phase 8+ — the plugin host exists (Phase 7) but is not yet wired into the editor. Treat the WASM-tier and live-eval prose as forward design.
Live evaluation (📝 planned) means plugin authoring without restart, not REPL-style sub-keystroke evaluation. The design: a built-in *scratch:rust* buffer accepts Rust source; on :eval the host writes it to a temp dir, invokes rustc --target wasm32-wasip2, and instantiates the resulting component against the same plugin host shipped plugins use — new commands / motions / decorations available immediately, ~1-3 s compile. Not built yet (depends on editor-side loading). Users wanting a sub-keystroke REPL would install a plugin exposing an evaluator over the CommandRegistry; not a host concern.
Why no in-process scripting language. A second runtime (Lua via mlua, embedded Scheme, Rhai) doubles the API surface plugin authors must learn, doubles the binding maintenance, and divides the ecosystem between "plugin-shaped" and "scripting-shaped" extensions that should be the same shape. The Rust-WASM-only choice keeps every extension on one substrate with one set of tooling. The cost is the live-eval-experience tradeoff above; we accept it.
[editor]
line-numbers = "relative"
soft-wrap = true
[ui]
tab-bar = "auto" # always | auto | never
mode-line = { segments = ["modal", "file", "git", "lsp", "position", "encoding"] }
header-line = { enabled = true, segments = ["breadcrumbs", "symbol-context"] }
notifications = { corner = "bottom-right", max-visible = 3, default-timeout = "4s" }
# No fixed sidebars or bottom panels. Layouts are buffers in panes;
# the user composes them and may save reusable layouts as named tabs.
[layouts.coding]
description = "File tree on the left, code in the center, diagnostics at the bottom."
splits = [
{ orientation = "horizontal", ratio = 0.2, buffer-view = "file-tree" },
{ orientation = "horizontal", ratio = 0.8, content = "active" },
{ orientation = "vertical", ratio = 0.25, buffer-view = "diagnostics-list", placement = "bottom" },
]
[keys.normal]
"space f f" = "fuzzy-finder.files"
"space p" = "command-palette.open"
"space e" = { command = "ui.open-buffer-view", args = { view = "file-tree", target = "split-left" } }
[major-mode.rust]
language-server = "rust-analyzer"
default-minor-modes = ["git-gutter", "rainbow-delimiters", "auto-pair"]
11. Project Layout (Cargo Workspace)
The workspace is 32 crates (Cargo.toml members), organized bottom-up so every crate depends only downward. (This tree was rewritten in v0.6 to match the code — the earlier lattice-render* trisection, lattice-modes, lattice-config-api, and lattice-headless were never built; rendering consolidated into the ui-* peers + lattice-host, modes into one lattice-mode crate, and the TUI serves the headless role.)
lattice/
|-- Cargo.toml
|-- rust-toolchain.toml # pins rustc 1.94, edition 2024
|-- crates/
| # -- bottom: shared types (no editor deps) --
| |-- lattice-protocol/ # Position/Range/Edit/Selection/Event/IDs; jsonrpc; CancellationToken
| # -- core: the editor engine --
| |-- lattice-core/ # Buffer (ropey) + Document + linear UndoStack + file I/O + search
| |-- lattice-grammar/ # vim modal state machine + CommandRegistry + dispatcher + builtins
| |-- lattice-runtime/ # DocumentActor (tokio task/doc) + snapshots (arc-swap) + Pending<T> + EventBus
| |-- lattice-cells/ # CellMatrix / DisplayMatrix / VirtualRow — the renderer-neutral cell grid
| # -- subsystems --
| |-- lattice-syntax/ # tree-sitter integration (rust/python/js/markdown), incremental reparse
| |-- lattice-lsp/ # LSP client: actor pool, diagnostics, supervisor, watcher, lsp-modes
| |-- lattice-mode/ # Mode trait (major/minor), registry, lifecycle, ServiceRegistry, modeline, event bus
| |-- lattice-keymap/ # layered keymap trie + chord resolution
| |-- lattice-completion/ # insert-completion pipeline: generators / matchers / rankers / annotators
| |-- lattice-config/ # typed-option registry (OptionDecl + trait OptionType, ArcSwap cells), :set, lattice.toml loader
| |-- lattice-config-macros/ # derive macros for typed options
| |-- lattice-picker/ # picker primitive: source registry, live-query, MRU frecency, preview
| |-- lattice-help/ # HelpContent/HelpBuffer, Introspectable, :help + :describe-* backing
| |-- lattice-theme/ # theme element registry (ArcSwap), palette resolution
| |-- lattice-snippet/ # snippet parser + registry + expansion mode
| |-- lattice-terminal/ # terminal buffer kind + PTY
| |-- lattice-file-tree/ # file-tree buffer kind
| |-- lattice-oil/ # oil-style directory buffer kind
| |-- lattice-diff/ # diff subsystem (imara-diff): signs, inline/side/three-way, ]c/[c/do/dp
| |-- lattice-multibuffer/ # multibuffer Document (excerpts) + search / narrow providers
| # -- host: the editor composition root (thin substrate) --
| |-- lattice-host/ # Editor: boot, dispatch, BufferRegistry, RenderState, pane tree, tabs
| # -- renderer peers --
| |-- lattice-ui-tui/ # TUI peer (ratatui + crossterm); first-class, headless / SSH
| |-- lattice-ui-gpui/ # GPUI peer (optional `gui`/`window` features)
| # -- integrations --
| |-- lattice-ai/ # AI coding-agent integrations (Claude Code over MCP, opencode ACP)
| |-- lattice-agent/ # agent-log buffers + modes
| |-- lattice-dashboard/ # dashboard/start buffer
| # -- plugin substrate --
| |-- lattice-plugin-host/ # wasmtime (v46) + Component Model + WASI p2; capability/fuel/epoch; every seam
| |-- lattice-plugin-api/ # wasmtime-free introspection catalog (what lattice-host depends on today)
| |-- lattice-plugin-sdk/ # guest-side SDK for authoring plugins
| |-- lattice-plugin-sdk-derive/ # derive macros for the guest SDK
| # -- binary --
| `-- lattice-cli/ # `lattice` binary (TUI default; `--features gui` links the GPUI peer)
|-- wit/ # canonical WIT interface definitions (package lattice:plugin-host@0.1.0)
| |-- types.wit # shared record/variant types
| |-- plugin.wit # top-level plugin world
| |-- buffer.wit # read-only `document` resource
| |-- command.wit # (stub)
| |-- grammar.wit # grammar contributions + callback trampoline
| |-- picker-source.wit # guest-exported picker-source world
| |-- completion-source.wit # guest-exported completion-source world
| |-- decorations.wit # guest-exported decoration world
| |-- events.wit # event subscribe / emit
| |-- config.wit # plugin-contributed options
| |-- modes.wit # mode contributions
| |-- host-services.wit # walk / register-event / emit-event
| |-- ui.wit # (stub)
| `-- trampoline-fixture.wit # test fixture
|-- plugins/
| `-- fuzzy-finder/ # the PH7 validation plugin (seam parity + overhead bench)
|-- assets/ # icons, banner, desktop-entry, bundle resources
|-- docs/
`-- tests/
📝 Bundled first-party plugins (git, grep, outline, format-on-save, …) and the
init.rs-as-WASM config path are Phase 8 / 8b — the host runtime exists, but editor-side loading is not yet wired (§5.5.6, §5.12, §13). Much of that functionality already ships natively (fuzzy-finder =lattice-picker, git diff =lattice-diff, snippets =lattice-snippet, outline =:lsp-symbols); the not-yet-done work is repackaging it as WASM components.
12. Testing Strategy
| Test type | Tool | Coverage |
|---|---|---|
| Unit | cargo test | Per-module logic |
| Integration | cargo test (workspace) | End-to-end command/event flows |
| Snapshot | insta | Editor scenarios; status segment outputs |
| Property | proptest | Buffer invariants, layout cache coherence |
| Benchmark | criterion | Hot paths in CI |
| Fuzz | 📝 cargo-fuzz (planned) | Edits, command parser |
| LSP integration | 🟡 real servers | rust-analyzer / pyright / etc. (Docker harness 📝) |
| Plugin | lattice-plugin-host guest fixtures | every seam exercised end-to-end (Phase 7) |
| Visual regression | 📝 headless screenshot diff (planned) | Rendering, popups, pickers |
| UI scenario | synthetic input in host/ui-tui tests | keystroke → published state |
Status. Unit + integration + benchmark testing is heavily exercised (~3.5k tests: ~1.5k ui-tui, ~650 host, ~190 grammar, ~180 lsp, plus the rest) and the criterion latency benches are CI-ratcheted. The
cargo-fuzz, Dockerized-LSP, and screenshot-diff / visual-regression rows are 📝 aspirational.instasnapshot +proptestare used lightly, not pervasively.
13. Roadmap
Progress (see
../operations/implementation.mdfor the live ledger): Phases 0–8 shipped. Phase 7 (the plugin host runtime) is complete —lattice-plugin-host, thewit/API package, the capability/fuel/ crash-isolation model, and every extension seam, exercised end-to-end by guest fixtures, with thefuzzy-findervalidation plugin and CI overhead gates. Phase 8 (editor-side plugin loading) landed (branchphase-8-plugin-loader): thelattice-plugin-loadercrate (on-disk discovery
:plugin-load/unload/reload+init.rs-as-WASM config), the buffer-backed:pluginsmanager view, and the full plugin-observability stack (lattice-plugin-trace: the boundary tracer, the gated hot-path grammar seam, the*plugin-trace*buffer views, the liveplugin.trace-leveloption, and thewasi:loggingguest import — seeplugin-observability.md). The mode/view system was built natively ahead of the plugin host, so Phase 8's remaining work (8b) is the bundled first-party reference plugins + repackaging the built-in modes as WASM components, not building the mode system. The week numbers + crate names below are the original plan, retained as the historical spec — several crate names never existed (lattice-render*,lattice-modes; see §11 + the Phase-5/6 superseded notes below).
Phase 0: Foundation (weeks 1-2)
Workspace, lattice-core, document/buffer/undo, file I/O, protocol enums, snapshot tests. Exit: programmatic edit roundtrip.
Phase 1: Modal Editing (weeks 3-4)
lattice-grammar crate. Vim modal state machine (Normal, Insert, Visual, Op-pending, Command, Search, Replace). Strict vim grammar parser: counts, registers, operators, motions, text objects, ex-ranges. Built-in command catalog (Operator / Motion / TextObject / Register / Range / Count types as the public command API). Default vim keymap as a config file. Macros, marks, registers, dot-repeat. Grammar extension hooks (register_motion / register_text_object / register_operator) wired and exercised by an internal test plugin. Exit: complex editing through typed command invocations in tests; default keymap fully functional.
Phase 2: Terminal UI Bootstrap (weeks 5-6)
lattice-ui-tui with crossterm/ratatui. Wire input -> core -> render. Exit: modally edit text in a terminal.
Phase 3: Tree-Sitter (weeks 7-8)
lattice-syntax, highlighting (10 grammars), structural motions, reparse pipeline. Exit: highlighted code, structural motions work.
Phase 4: LSP (weeks 9-11)
Diagnostics, completion, hover, definition, references; cancellation; version tracking. Exit: rust workflow with diagnostics, completion, jump-to-def.
Phase 5: GPU Rendering Foundation (weeks 12-14)
lattice-render, lattice-render-editor monospace path, lattice-ui-gpui compositor with splits and tabs, theme system. Exit: GPU code editor with terminal feature parity, smooth scrolling.
Superseded (as-built): the
lattice-render/lattice-render-editor/lattice-render-documentcrates below were dropped. The App→Editorcomposition (Phase 5.B) put the renderer-neutral core inlattice-hostand ships oneRenderermarker per frontend (lattice-ui-tuibuilt,lattice-ui-gpuipeer); the shared paint substrate is the cell-grid (lattice-cellsCellMatrix, seecell-grid-renderer.md), not a taffy+ cosmic-text document renderer. See §5.6 (which carries the same correction).
Phase 6: Document Renderer + UI Components (weeks 15-18)
lattice-render-document(taffy + cosmic-text).- Popup system (completion, hover, signature help, diagnostic, code action).
- Picker primitive (file picker as first user).
- Status lines (mode line, header line) with segment registry.
- Buffer-backed view scaffolding (file-tree, diagnostics-list as first users).
- Notifications.
- Command line / echo area.
Exit: full UX surface in place; everything but the plugin host.
Phase 7: Plugin Host (weeks 19-22)
lattice-plugin-host with wasmtime + Component Model + WIT bindings (wit-bindgen). AOT module cache; lazy instantiation; capability manifests; fuel limits; per-call overhead benchmarks gated in CI (typed call < 500ns p99; grammar-extension round-trip < 5μs p99). Async ABI wired through tokio. Reference plugin: fuzzy-finder (validates picker primitive end-to-end). Exit: a WASM plugin replicates the file picker without host changes; CI enforces overhead budgets.
Phase 8: Major/Minor Modes + Reference Plugins (weeks 23-25)
Reframed vs. the original plan: the mode/view system is ✅ built natively (lattice-mode registry, major/minor axes, lifecycle, the LSP sub-mode cascade, help/diff/oil/file-tree/multibuffer modes) — the everything-is-a-buffer principle is already validated. What remains for Phase 8 is (a) the editor-side plugin loader (on-disk discovery, :plugin-load, init.rs-as-WASM), and (b) repackaging built-in modes + reference plugins as WASM components (§5.8.3 goal). Built-in modes stay native by design; the as-components path is the extension story.
Phase 9: Rich Buffer Rendering (weeks 26-28)
Shaped path in lattice-render-editor. Per-line layout cache + Fenwick index. markdown-mode. Style mappings system. Exit: edit markdown with variable headings; latency indistinguishable from code.
Phase 10: Polish and v1.0 (weeks 29-32)
Live-eval (*scratch:rust* -> rustc -> dynamic plugin load). Accessibility. Cross-platform packaging. Crash reporter. Documentation. Themes. Exit: 1.0 release.
Post-1.0
Path 4 (inline blocks). org-mode. WebRenderer (decision time). Remote / SSH. Collaborative editing. Multi-cursor as first-class editing model. Pluggable editing paradigms (e.g., emacs / readline-style alternative). tree-sitter-motions plugin promoted to a bundled extension. PTY-backed terminal buffer view.
Total estimate: ~8 months solo, faster with a small team. Realistic with rework: 11-15 months.
14. Risks and Mitigations
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| GPUI API churns | Medium | Medium | UI crate isolated; protocol-only dependency; wgpu + parley fallback. |
| Plugin API (WIT) design proves wrong | High | High | The WIT is validated before 1.0 by the fuzzy-finder plugin (⭐ Phase-7 exit) + a per-seam guest fixture for every interface (grammar / picker / completion / events / decorations / config / modes / keymap / logging), exercised end-to-end in CI. (tree-sitter-motions as a plugin is deferred post-1.0 — it is not the validator.) SemVer only after 1.0. WIT changes ARE breaking; design carefully upfront. |
| ✅ WASM host-call overhead exceeds budget | High | Empirically bounded + CI-gated (Phase 7): grammar round-trip ~340ns release, typed-call <500ns p99, both green in CI. AOT + module cache; built-in grammar stays native. Risk largely retired. | |
| ✅ Plugin cold-start tax at editor launch | Medium | Lazy instantiation + AOT module cache landed in the Phase-7 host; cold-start budget stands. Re-measure once the loader (Phase 8) instantiates real plugins. | |
| Per-line layout cache invalidation bugs | Medium | High | Property tests for cache coherence; content + style hashes. |
| Tree-sitter performance on huge files | Low | High | Per-file thresholds; disable structural features above N MB. |
| LSP server quirks | High | Low | Per-server compatibility shims. |
| Async cancellation correctness | Medium | High | Property tests; tracing spans on every async op. |
| Vim grammar edge cases (revised in v0.4) | High | Medium | Vim semantics are committed -- the grammar is not modified. Specific edge cases (rare register quirks, obscure block-visual behaviors) may be deferred for v1 with explicit tests documenting the gaps. The default keymap is a config file so users can patch differences locally. |
| Grammar extension API churn (new in v0.4) | High | High | The grammar-extension API is exercised end-to-end in CI by the grammar-guest fixture (motions / text-objects / ex-commands, incl. the guest-err + trap/quarantine paths) through the sync trampoline. SemVer freeze on the grammar WIT only after the extension API has supported at least three real plugins. (tree-sitter-motions as a plugin is deferred post-1.0.) |
| Major mode plugin churn breaks buffers | Medium | High | Version pinning per plugin manifest; WIT protocol version declared per binding. |
| Rich buffer scroll on huge documents | Medium | Medium | Eager layout < 50K lines; lazy with overscan above. |
| Status segment thrash | Medium | Medium | Pull-not-push update model; per-segment update triggers; fail-isolated rendering. |
| Popup z-ordering / dismissal bugs | Medium | Low | Centralized popup manager; declarative dismissal rules; visual regression tests. |
| Buffer-backed view layout UX (new in v0.4) | Medium | Low | Ship named layouts in default config (e.g., coding, writing) so the everything-is-a-buffer model has zero-cost defaults. Users keep full freedom to compose their own. |
15. Open Questions
- Persistence of undo trees -- GC story when source file is deleted / renamed.
- Plugin distribution -- own registry or piggyback on existing infrastructure.
- Crash recovery for unsaved buffers -- periodic snapshot or append-only edit log.
- Workspace concept granularity -- per-folder or per-project-root.
- Remote development story -- designed for from day one or retrofitted in v2.
- Telemetry -- opt-in, privacy story matters.
- Plugin versioning and compatibility -- Component Model versioning, our policy.
- Built-in mode boundary -- bundled vs separate downloads (binary size vs. OOB experience).
- Live-reload of plugin-defined modes -- without restart?
- Style mapping override layering -- formal precedence: theme vs. mode vs. user vs. plugin.
- Picker keymap defaults -- Tab vs. Enter vs. Ctrl-N for next item; align with which existing tool.
Multi-window state synchronization -- when the same document is open in two windows, how do scroll / selection / focus events propagate.Resolved in §5.6.8: selections are transformed againstAppliedEditand published as part of the nextDocumentSnapshot; per-pane scroll / focus is pane-local state.- Notification persistence -- should errors persist across sessions until acknowledged.
- Default layouts (new in v0.4) -- which named layouts ship in default config so the everything-is-a-buffer model has good zero-config defaults.
- Grammar extension API surface for tree-sitter motions (new in v0.4) -- exact shape of the host's tree-sitter query API exposed to plugins (one-shot vs. cursor-based query iterator; range scoping; query caching).
Plugin async-task host primitiveResolved — each plugin owns its ownwasmtime::Storerun as a tokio task; background work is additional plugin-side tasks composed via the host's async ABI (§5.5.1 / §9.5). No separatespawn-taskprimitive was needed.- WASM AOT cache invalidation (new in v0.4) -- when do we invalidate cached compiled modules (wasmtime version, target triple, plugin checksum, all three).
- Folds (deferred) -- vim has manual / indent / syntax / expr folds; tree-sitter gives us syntax folds nearly free. Open: storage (rope-side metadata vs. computed view), interaction with motions (
zj/zk/[z/]z) and operators that target folded ranges, persistence across sessions. Replace mode (Resolved / built —R) dispatchRoverstrike ships as its ownModalState::Replacewith per-charReplaceUndoLastrestore (lattice-grammar).Live evaluation / REPL parity -- emacs'sResolved per §10 / §2.2: live evaluation in lattice means plugin authoring without restart viaM-x ielm,eval-last-sexp, scratch buffer.*scratch:rust*->rustc-> dynamic plugin load, sharing the WASM plugin host substrate. In-process REPL with sub-keystroke evaluation is an explicit non-goal; users wanting it install a community-shipped plugin.- File watcher / auto-revert (deferred) -- emacs's
auto-revert-modeand external-change detection. Open: notify-based watcher per workspace, mtime poll fallback, conflict resolution UI when external + local edits diverge. - Bookmarks and cross-file marks (deferred) -- vim's
'A-'Zglobal marks and emacs's bookmark facility cover overlapping ground. Position history (§5.1.1) handles in-process navigation; bookmarks need persistence, naming, and a picker. - Function rebinding / advice (deferred) -- emacs's
defadvice/advice-add. The dispatcher already mediates every command, so wrapping is a registry-side concern. Open: advice ordering, removal semantics, interaction with WASM-defined commands, fuel accounting for advice chains. Narrow-to-regionResolved / built — narrow-mode (N.1, 2026-06-10):zn/:narrowbuild aMultibufferDocumentHandleview of the sub-range with tree-sitter text objects;:widenrestores.- Snippets and abbrev — 🟡 snippet engine built (
lattice-snippet: parser, registry, expansion mode, integrated with completion). Abbrev (auto-expand-on-type) is still open. - Frames (multi-OS-window) (deferred) -- emacs's frame concept. Decoupling is clean (each frame is a top-level window with its own pane tree); open question is workspace boundaries (one workspace per frame vs. shared) and how the position-history ring partitions across frames.
- Session save / restore (deferred) -- emacs
desktop.el. What state is captured (open buffers, panes, layouts, registers, marks, position history, command history, ex-history, search history) and what is per-workspace vs. per-user-global. - DAP support (deferred to post-1.0 plugin) -- LSP is in-host (§5.4) for latency reasons; DAP is similar shape. Open: in-host like LSP, or first reference plugin that exercises the WIT async/event surfaces under a real adversarial workload.
- AI / completion-as-you-type integration (deferred to post-1.0 plugin) -- Copilot / Codeium / Claude / local-LLM. The everything-is-a-buffer model gives chat / inline-suggestion surfaces for free; the question is whether the plugin host's WIT API has the right shape (streaming completions, ghost-text rendering hook, accept/reject arbitration with vim modal state).
- Magit-class VCS integration (deferred to post-1.0 plugin) -- a reference plugin will land in v1.0 (basic git status / diff / blame); the open question is whether the buffer protocol exposes enough for a plugin to reach Magit-equivalent fidelity (interactive rebase, hunk staging, log graph), or if specific hooks need adding.
- EditorConfig / project-build awareness (deferred) -- whether the host reads
.editorconfignatively or via a reference plugin, and how project-detected build commands (cargo, npm, etc.) integrate with:!and the compilation-buffer (§B.4).
16. Glossary
- Buffer: rope-backed in-memory text.
- Document: buffer + metadata.
- Selection / Selection set: (anchor, head) pairs. The type is a set so multi-cursor is a clean post-1.0 extension; v1 uses a single selection with vim's visual extents (§2.2).
- Edit: a primitive change at a range.
- CommandInvocation / Event: the typed dispatch value and the typed event payload (the old monolithic
Commandenum was retired, §6.1). - Modal mode: input interpretation context (Normal, Insert, Visual).
- Major mode: a buffer's primary content-type identity. Exactly one per buffer.
- Minor mode: composable feature toggle. Zero or more per buffer.
- DisplayMatrix / cell grid: the renderer-neutral per-pane styled-cell output both peers paint (§5.6).
- Style mapping: major mode's table from syntax-tree node types to text styles.
- Layout cache: per-line cache of shaped glyphs and metrics.
- Damage region: range that has changed and needs re-rendering.
- Decoration: inline annotation (squiggle, virtual text, gutter marker).
- Operator / Motion / Text object: components of the modal grammar.
- Plugin: a WASM component extending the editor.
- Capability: explicit permission grant for a plugin.
- Fuel: wasmtime's CPU budget mechanism.
- WIT: WebAssembly Interface Types.
- Renderer: a peer frontend (
TuiRenderer/GpuiRenderer) painting the sharedDisplayMatrix; theRenderertrait is a marker surfacing itsTheme/PaneRenderRegistrytypes (§5.6.1). - Pane: a region of a window holding one document's view.
- Pane tree: recursive split structure of a tab's panes.
- Tab: per-window grouping holding a pane tree.
- Modeline element: contributed, event-driven unit of mode-line content (
ModelineRegistry); the header line is delivered as virtual rows. - Gutter column: a gutter content column (hardcoded set today; registry 📝).
- Popup: a Document in a floating popup-shaped view (§5.6.3).
- Picker: fuzzy-search overlay for selecting from a list (file / symbol / command / etc.).
- Buffer-backed view: a non-file buffer (file tree, outline, diagnostics, terminal, ...) placed in a pane like any other buffer.
- Notification: transient corner-anchored message.
- Command API: typed, scriptable surface of every editor primitive -- operators, motions, text objects, registers, ranges, counts. Keymaps are bindings from chord sequences to command invocations.
- CommandInvocation: the unified call type carrying
(command, count, register, range, args)through the single dispatcher. Vim's ex-syntax and plugin function calls both produce these. - Grammar extension: a registered motion, text object, operator, or ex-command that participates in the vim grammar exactly like a built-in.
- Position history: a per-buffer-and-global ring tracking cursor positions, with each entry tagged by source (
AutoJump,ExplicitMark,PluginPush,NamedMark). Unifies vim's jump list with emacs's mark ring. - Event / Hook: a typed editor state-transition with a payload. Subscriptions are typed; vim's
:autocmdand emacs's hooks both desugar to subscriptions on the unified event bus. - Minibuffer: a transient editing buffer for
:commands,/searches, and any interactive prompt. Has a major mode (command-line,search-line, ...), supports the full vim grammar, tree-sitter highlighting, decorations, and popups. - Echo area: single-line surface for transient core/command messages; sharing screen real estate with the minibuffer; rolling history kept in
*messages*. - Option / Customize: every option is a typed registered value with metadata; the customize buffer is a type-aware editing view that writes back to user TOML.
- Component Model: WebAssembly's typed-interface, multi-language plugin model; the runtime substrate for all plugins.
- WIT (WebAssembly Interface Types): the canonical interface description language for the plugin API.
- Resource handle: an opaque WIT type that lets a plugin reference host-owned data (a buffer, a document) without copying it across the boundary.
- Sprite / Sprite atlas: small line-height-sized graphical element (SVG or PNG) referenced by id. Used for file-type icons, severity icons, status indicators, etc. Atlas-backed; shares the GPU pipeline with glyphs but lives in a separate texture. Distinct from Path 4 inline media blocks.
Appendix A: Performance Comparison with Neovim and Emacs
This appendix establishes a baseline for "what good looks like" so we can evaluate our progress during implementation. Comparisons here are honest assessments, not aspirational marketing.
A.1 Methodology
Numbers below are from a mix of published benchmarks, our own measurements where available, and reasonable estimates from architectural analysis. They assume:
- A mid-range modern laptop (Apple M-series or recent x86 mobile, integrated GPU).
- A 60-120Hz display.
- Editing typical source code files (1K-10K lines).
- LSP enabled with a representative server (rust-analyzer, pyright, etc.).
- Default-or-minimal user configuration (i.e., not heavily-customized Doom Emacs).
The metric "keystroke-to-glyph" measures the time from when the OS delivers a key event to when the corresponding glyph is on screen, including all intermediate work (buffer mutation, parse, render, present).
A.2 Headline comparison
| Dimension | Lattice (target) | Neovim | Emacs (vanilla) | Emacs (Doom/Spacemacs) |
|---|---|---|---|---|
| Cold startup | 150-400ms | 30-80ms | 200-500ms | 500-2000ms |
| Daemon/instant launch | Possible post-1.0 | N/A | emacs --daemon | emacs --daemon |
| Steady-state typing (code) | <2ms | 2-4ms (terminal RTT) | 5-10ms | 10-25ms |
| Steady-state typing (markdown rich) | <5ms | N/A (grid only) | 10-20ms (org) | 15-40ms (org) |
| Open 100MB log file | <500ms | <500ms | Slow / requires so-long-mode | Slow |
| Tree-sitter incremental reparse | <1ms | ~1ms | ~1ms (treesit native) | ~1ms |
| LSP completion latency (popup) | <10ms after server | ~10ms after server | 20-50ms after server | 30-80ms after server |
| Plugin overhead in heavy config | Architecturally bounded | Variable; can block | Cumulative; serialized | Cumulative; can be severe |
| Worst-case input lag under plugin load | UI unaffected | Plugin can block briefly | Plugin can block significantly | Plugin can block significantly |
| Memory baseline | 150-250MB | 30-60MB | 60-100MB | 200-500MB |
A.3 Dimension-by-dimension assessment
Cold startup. Neovim wins clearly -- terminal-only, minimal init. We will not match it because we initialize a GPU surface, font system, and atlas at startup. We will be in line with vanilla Emacs and significantly faster than configured Emacs distributions. Daemon mode (post-1.0) makes subsequent launches near-instant, eliminating this penalty for the actual day-to-day workflow.
Steady-state typing (code). This is the most important metric and we should be at parity with or slightly faster than Neovim. Our pipeline (buffer mutation -> tree-sitter incremental edit -> atomic tree swap -> next-vsync render) is microseconds of Rust on the input path; the display work is GPU-accelerated with monospace fast path. Neovim's terminal-based pipeline adds a small PTY round-trip; ours doesn't. Realistic targets: ours <2ms, Neovim 2-4ms, both well below human perception. Against Emacs we should be 3-5x faster because Emacs's display engine carries machinery (variable fonts, mixed content) we don't pay for in code buffers.
Steady-state typing (rich content / markdown / org). Neovim doesn't natively render variable fonts -- its rich-document plugins (render-markdown.nvim) work through grid-cell tricks rather than true variable-font rendering. So this is apples-to-oranges. The fair comparison is Emacs org-mode, where the design (📝 shaped rich-buffer path, not yet built — §5.6.2) should be 2-3x faster because shaping would run on spawn_blocking workers off the input thread. Emacs's redisplay is single-threaded; shaping cost goes directly to perceived latency. (Today lattice renders markdown as highlighted monospace cells, which is already off-thread.)
Large file handling. Neovim and Lattice both use rope-like structures and handle 100MB+ files comfortably. Emacs uses a gap buffer that performs poorly on large files; mitigated by so-long-mode but not gracefully.
Project-wide search. All three shell out to ripgrep. This is a wash -- performance is determined by ripgrep, not by the editor.
LSP responsiveness. Neovim's built-in LSP client is fully async and well-implemented; we should be at parity in terms of raw responsiveness. Architecturally we have a subtle advantage: third-party plugins in our system cannot accidentally make LSP synchronous (the WASM async boundary prevents it), while in Neovim a poorly-written Lua plugin could. Against Emacs lsp-mode / eglot, we should be substantially faster -- both run on the main elisp thread, and heavy LSP traffic causes visible stutters in Emacs.
Plugin overhead. This is where the architectural difference is most pronounced. In Neovim, Lua plugins run on the main thread; well-written plugins are async, but synchronous plugins (still common) directly add latency. In Emacs, every elisp hook on every event runs serialized on the main thread; heavy configurations accumulate measurable lag. In Lattice, plugins are WASM actors with fuel limits and async host calls -- they cannot block the UI by construction. A plugin can be slow, but its slowness is bounded to its own computation; the editor keeps rendering.
Memory. Neovim is the lightest. We're heavier because of GPU resources, atlas memory, layout caches, and the WASM runtime. Vanilla Emacs is comparable to us; configured Emacs distributions exceed us.
A.4 What we won't beat
Honesty matters here:
- Neovim's startup time. Inherent cost of being a GUI app.
- Neovim's memory footprint. GPU and runtime overhead.
- Neovim's binary size. Our binary will be 20-80MB (Rust + GPU dependencies + bundled plugins); Neovim is 5-15MB.
- Decades of edge-case shaking. Both Neovim and Emacs have had thousands of users for thousands of corner cases. Our v1.0 will have rougher edges.
These aren't dealbreakers but they're real. Users for whom startup time and memory dominate may prefer Neovim. Users for whom decades of plugin ecosystem dominate may prefer Emacs. Our pitch is for users who want modern UX, predictable latency under load, and safe extensibility -- and are willing to trade 100MB of RAM and 200ms of startup for it.
A.5 What we should beat decisively
- Worst-case latency under heavy plugin load. This is the headline. In Emacs, twenty active plugins each adding a few milliseconds to a hook pile up into perceptible lag. In Neovim, a single ill-behaved plugin can stall the whole editor. Lattice's architecture makes plugin work parallel and bounded -- we should never exhibit the kind of "Emacs is slow today" experience that comes from accumulated hook overhead.
- Rich-buffer editing latency. With shaping on workers, we deliver code-like latency for markdown/org-style content. Emacs cannot match this on its current architecture.
- Large-file responsiveness. Rope + tree-sitter + worker-based shaping handles huge files without the Emacs-style "this file is too big" failure mode.
- Predictability. This is qualitative but real. With the actor model and explicit fast paths, latency variance should be tight. Emacs and Neovim both have "good days and bad days" depending on what plugins triggered when. We should be more consistent.
A.6 Risks to performance posture
These are the things that could erode our position if we're not vigilant:
- Allocation discipline on the input path. Rust makes allocation easy. The hot path must be allocation-free -- verified by
criterionbenchmarks andtracingspans gated in CI. - GPU driver variance. Frame times that are tight on Apple Metal might be loose on Linux Mesa or Windows DirectX. Cross-platform benchmark coverage is required.
- Tree-sitter pathological grammars. TypeScript with heavy generics, Rust with macro chains, can spike parse times. Inherit lessons from Helix/Neovim; benchmark our top 20 grammars.
- WASM cold-start overhead. Wasmtime instantiation is a few milliseconds per plugin. Loading 50 plugins serially at startup adds up. Mitigations: lazy plugin loading, AOT-cache modules, parallel instantiation.
- Atlas thrashing. If a buffer mixes many fonts/sizes, atlas eviction can cause slow first-paint of unfamiliar glyphs. Mitigation: pre-warm atlas on file open with the file's glyph census.
- UI furniture creep. Each new panel, status segment, and notification adds work to every frame. Even though it's cached, "free" repaints are not actually free at scale. Discipline: budgets per frame for UI work, profiling in CI.
A.7 The headline assertion
Better than Emacs across nearly every dimension. At parity with Neovim on the things that matter for daily editing. Worse than Neovim on startup time and memory.
If we deliver against the Section 8.2 commitments and avoid the Section A.6 risks, this is achievable. If we miss, we degrade gracefully toward "comparable to Emacs," which is still acceptable. The unacceptable outcome -- slower than Emacs in any common workflow -- is precluded by architecture, not by optimization effort. That's the point.
Appendix B: Vim / Emacs Unifications (smaller wins)
This appendix collects the smaller-scale unifications that fall out naturally from the architecture. Each reuses the existing command registry, event bus, buffer model, and minibuffer.
📝 Status (most of Appendix B is forward design). Built today: B.1 interactive arg specs (incl.
Chord-capture), B.10:describe-buffer. Partial: B.2 — only:global(:g) ships;:v/:vglobaland the whole:windo/:bufdo/:tabdo/:argdofamily are 📝 unbuilt. B.4 —*messages*ships;*scratch*/*compilation*do not. Not built (📝): B.3 history pickers, B.5:redir, B.6:!/shell-execute, B.7 minor-modes status segment, B.8 idle hooks, B.9 notification-to-*messages*teeing (notifications themselves are unbuilt, §5.9.9). Read the rest as design intent.
B.1 Interactive arg specs (emacs (interactive ...) done right)
Each CommandSpec carries an args_schema: Vec<ArgMetadata> (§5.11). The schema declares per-argument:
- name, type, doc
- prompt text (used when invoked interactively without a value)
- completion source (used by the minibuffer completion popup)
- validator (used for live error indicators in the minibuffer)
- default value or "use selection" / "use cursor word" / "use last response"
Three entry paths consume the same schema:
:line. Thecommand-lineparser fills positional args from the input string; missing args trigger a follow-up prompt in apromptminibuffer with the schema-supplied prompt text and completion.- Keymap binding. A binding may pre-supply some args (
"<leader>fr"->format(scope=region)) while leaving others to prompt. The unsupplied args trigger sequential prompts. - Command palette. Selection of a command opens a guided form built from the schema -- one prompt per missing arg, in declaration order, with completion and validation throughout.
Result: one declaration covers : invocation, keymap-with-prompt, and palette-with-form. There is no second mechanism.
Per-kind input modes. ArgKind is more than a type tag -- it picks the cmdline's input mode while the cursor sits in that arg's slot. v1 implements Chord: when the active arg has kind == Chord (e.g. :describe-key's sole arg, the future :map <lhs> <rhs>), the cmdline switches into chord-capture: every key event renders to its canonical chord token (<C-c>, <Up>, gg) and gets appended; <BS> deletes one full token (not a byte); <Esc> cancels, <CR> submits. The Ctrl-c -> Quit global hatch is intentionally suppressed inside chord-capture so :describe-key <C-c> is reachable; <Esc> remains the abort.
When a Chord-required arg is submitted empty (:describe-key<CR>), instead of erroring the cmdline pre-fills with the command word + space, arms a one-shot auto-submit, and shows the schema-supplied prompt as a status hint. The very next chord captured fires the lookup -- no second <CR> needed. This is the v1 implementation of the missing-arg prompt path; richer prompt minibuffers (multi-arg, String / Pattern / completion-driven) layer onto the same arming mechanism.
B.2 :g and :v are normal commands
:g/TODO/d " delete every line matching TODO
:v/^[^#]/d " delete every line not starting with #
Both are registered ex-commands taking (pattern: Regex, body: CommandInvocation). The :g parser produces a CommandInvocation for global whose body arg is itself a parsed CommandInvocation for delete with range = Some(CurrentLine). The dispatcher iterates matching lines and invokes body for each. Same registry, no special form.
📝 The same pattern is intended to let :vglobal (:v), :windo, :bufdo, :tabdo, :argdo be normal commands taking a body invocation arg — none of those are built yet; only :global ships.
B.3 Histories as pickers over registries
Command history, search history, register history, position history (§5.1.1) all live in registries. A built-in picker over each gives:
:history-command-> picker over recent:invocations:history-search-> picker over recent search queries:history-register <reg>-> picker over content stored in a numbered register:history-position-> picker over the position history with source filtering
Plugins can register additional histories the same way.
B.4 Scratch, messages, compilation, REPL -- all buffers
*scratch* a persistent editable buffer for ad-hoc text / experiments
*messages* rolling log of echo-area messages and notifications
*compilation* output of `:compile` and external builds, errors made jumpable
*shell:zsh* a PTY-backed terminal buffer (post-1.0)
*repl:python* a comint-style REPL for the python plugin
Each is a regular buffer with a major mode tailored to its purpose; users open them in panes like any other buffer. The buffer-naming convention *name* is itself convention, not a special path -- the buffer just doesn't have a file backing.
B.5 :redir as an effect-capture wrapper
:redir > out.txt
:set
:redir END
becomes a single composed call: capture-output(target=Path("out.txt"), invocations=[set]). The capture command takes an output target (file path, register, scratch buffer) and a list of invocations to run; it tees Effect::Output from each invocation into the target. No special-form parser; composition is data.
B.6 :!cmd and :read !cmd -- subprocess effects through the dispatcher
:!ls -la becomes shell-execute(cmd="ls -la") returning an Effect::Output. :read !ls -la is shell-execute composed with insert-output-at(position=CurrentLine.below). Subprocess capability is gated; the parser front-end produces typed invocations regardless.
B.7 The status line / mode line sees minor modes the emacs way
Emacs's mode line shows the active minor modes as a flat list; vim's shows none of this by default. The default minor-modes status segment shows them as a small comma-separated list with a configurable filter (hide always-on minor modes; show only the unusual ones). Click / cursor-move on a minor-mode label opens its :describe-mode view.
B.8 Idle hooks
Many emacs workflows depend on run-with-idle-timer. We promote idle to a first-class event: Idle { duration } fires when the user has been quiescent for the declared duration. Subscriptions filter by duration threshold. Plugins use this for low-priority work (refresh git status, prefetch LSP hovers, etc.) without polling.
B.9 The *messages* buffer is also where notifications live
Notifications (§5.9.9) are transient corner-anchored UI but they are also logged to *messages* so the user can scroll back. Errors that lack a corresponding notification still surface there. Closing the user-facing notification doesn't lose the record.
B.10 Buffer-local commands
A buffer can have commands registered against it specifically (the buffer-local keymap, its major mode, an active minor mode). :describe-buffer enumerates which commands are reachable in the current context, with their resolved key bindings. Both vim's :map and emacs's C-h b collapse into this.
Appendix C: Lattice vs Zed -- architecture and performance positioning
Zed is the editor whose design Lattice converges with most closely, because the underlying technology choices (Rust, GPU, tree-sitter, LSP, WASM extensions) are the only modern stack that hits sub-frame latency without compromising extensibility. This appendix names where the convergence is deliberate, where the divergence is, and what we should and should not borrow.
C.1 Where the convergence is deliberate
| Choice | Lattice | Zed |
|---|---|---|
| Language | Rust | Rust |
| GPU rendering | GPUI preferred / wgpu fallback | GPUI (same renderer) |
| Text storage | ropey | custom sum_tree |
| Async runtime | tokio multi-thread, dedicated LSP runtime | smol/futures, single executor + worker pool |
| Tree-sitter | yes | yes |
| LSP | native client | native client |
| Plugin substrate | WASM (Component Model + WIT) | WASM (extension API) |
There is no reason to differentiate by picking a worse foundation. Where we share with Zed, we share deliberately.
C.2 Where Lattice diverges deliberately
C.2.1 Modal editing as the public API, not a key mapping
Zed exposes a non-modal Rust command surface and emulates vim on top of it via a vim_mode = true toggle. Lattice's grammar (operators × motions × text objects × registers × ranges × counts) is the public command API: there is one dispatcher (§5.2.1); ex-commands, normal-mode chords, and plugin contributions all flow through CommandInvocation → execute(...). A plugin that adds a motion is extending the grammar, not bolting on a binding. Adding tree-sitter-driven text objects lands natively because the grammar is the surface.
This is the largest architectural distinction, and it is also the most constraining one. Modal editing is paramount goal #3.
C.2.2 Everything-is-a-buffer is enforced, not implied
Zed has sidebars, bottom panels, and a buffer area as distinct UI primitives, each with its own focus and layout rules. Lattice has buffers placed in panes via splits (§5.9). File tree, diagnostics list, terminal -- all are buffers. :bn / :bp / :ls / :bd work uniformly across kinds because there is no "sidebar" concept to special-case. The UX cost is real (no draggable sidebar; you split a pane and :Tree into it). The benefit is that every text operation works on every buffer kind, with one code path.
This is the design call the running auto-memory feedback (buffers_no_special_case, synthetic_buffers, mode_owns_its_buffers) keeps enforcing.
C.2.3 WIT is the canonical plugin API (today, not aspirationally)
Zed extensions are written against a Zed-specific Rust crate that compiles to WASM. Lattice commits to WIT (§9.4) as the canonical interface: any language with a Component Model toolchain (Rust today; Zig, Go, AssemblyScript tomorrow) speaks the same plugin protocol. Per-plugin instances each own a wasmtime::Store and run as tokio tasks -- crash-isolated, fuel-limited, capability-gated. Plugin overhead is CI-enforced: typed-call < 500 ns p99, grammar-extension round-trip < 5 µs p99 (§8.2).
Zed's API is faster to ship features against; Lattice's is faster to port between language ecosystems and harder to silently regress under feature pressure.
C.2.4 Asynchrony is architectural, not disciplinary
Both editors target sub-frame latency. The mechanism is the meaningful distinction:
| Aspect | Lattice | Zed (typical) |
|---|---|---|
| Keystroke → glyph @ 120 Hz | within 1 frame; ≈0.7 ms measured (TUI, §8.2) | ~8--12 ms |
| Per-call WASM overhead | < 500 ns p99 (CI-gated) | unbudgeted |
| UI-thread blocking | architecturally impossible (§5.7.1) | discipline-enforced |
| Cold-start file open → first frame | not yet committed | ~50--80 ms typical |
Zed keeps the UI thread free by convention: contributors know which operations might block and route them off-thread. Lattice (§5.7) chooses primitives that make UI-thread blocking physically impossible in the steady state -- RenderState for reads, Arc<ArcSwapOption<T>> / PerBufferCache<T> for writes, dedicated subsystem tasks for everything else. The compounding property: every renderer-coupled subsystem migrates onto these primitives the same way, so the architecture stays uniform under feature pressure.
This is the technical moat. The rendering pipeline is literally the same as Zed's. The differentiator is that Lattice cannot regress into UI-blocking code paths in the way every editor (Zed included) eventually does.
C.3 What Lattice should borrow from Zed (and what we evaluated and didn't)
C.3.1 The cx.notify() reactive model — evaluated, not borrowed
Zed's UI redraws are driven by typed observation of state changes: a subsystem that mutates state calls cx.notify(), observers re-paint; clean entities skip paint code. We initially noted this as a candidate to import after Slice 3c. On re-evaluation against the paramount goals (heuristic #2), the savings curve doesn't transfer to Lattice -- Zed's reactive paint wins on its sidebar / panel / status-bar surfaces (cold relative to the cursor), but Lattice's "everything is a buffer" model has no comparable cold surfaces. The dominant on-screen work (document + syntax + relativenumber gutter + cursor-coupled diagnostic glyphs) changes every frame during interactive use.
The reactive semantic is already delivered by the existing primitives (Arc<ArcSwapOption<T>> + PerBufferCache<T>; §5.7.3): writes propagate to all clones atomically without re-publication. What cx.notify() would add is a paint-side short-circuit, and simpler alternatives (per-pane input-keyed cache; glyph-atlas / line-layout caching at the rendering crate; tree-sitter span cache) target the same idle-frame savings without the substrate complexity. See §5.7.6 for the full rationale and the deferred-decision posture: open question, gated on bench data from Slice 3c.
C.3.2 AI as a buffer kind (post-stability)
Zed's AI integration is a buffer kind, not a sidebar -- the chat history, prompt input, and notebook-style edit loops all live in a buffer with its own major mode. This maps cleanly onto Lattice's everything-is-a-buffer + WIT plugin surface: an AI plugin contributes a Document-buffer kind with chat semantics, plus optionally a grammar extension for prompt cmdlines. Deferred until the core stabilizes; the shape is already compatible with the design today.
C.4 What Lattice should NOT borrow from Zed
C.4.1 Non-modal editing as the default
Vim's grammar is Lattice's identity (paramount goal #3). A "config that disables modal" path is post-v1 territory; the design starts modal.
C.4.2 Imposed layout primitives
Zed's UI carries a set of pane / panel / sidebar primitives with explicit layout rules. Lattice does not impose any layout: panes + splits + buffers compose into whatever the user wants. The architect's position is explicit: enforcing layouts is anti-flexibility. The user expresses preferred layouts via their init.rs / keymap (a hypothetical "open file-tree on the left at 30% width" is a startup command, not a built-in layout primitive). Lattice's model offers more flexibility because it does not pre-decide layout.
C.4.3 Custom rope
Building a better rope than ropey is pure complexity-debt for marginal wins. Not on the v1 critical path.
C.4.4 Single-language extension API
WIT is the more portable target. Plugin authors who already write Rust are happy either way; everyone else benefits from us not having Lattice-specific bindings.
C.5 The one-line framing
Lattice = vim's grammar + emacs's extensibility + Zed's hardware bet, with paramount-goal-4 architectural asynchrony as the differentiator.
End of design document v0.4.