Skip to main content

Module wasm_grammar

Module wasm_grammar 

Source
Expand description

Loading grammars from WebAssembly, and the stores parsers need to run them.

Design: plugin-languages.md §2.5. Slice plan: LG.3b.

load turns a wasm side module into an ordinary [tree_sitter::Language], which is the fact the whole design rests on: downstream, HighlightConfiguration, folds, injections, indents and the incremental reparse path cannot tell where a grammar came from.

§The one thing that is not transparent

A wasm-backed Language can only be used by a Parser that owns a [WasmStore]. A parser without one fails set_language outright (“Failed to load the Wasm store”), so this is not something that can be left to chance. The costs, measured (see benchmarks.md):

operationcost
WasmStore::new5.1 ms — it compiles tree-sitter’s wasm libc
load_language102 ms — Cranelift compiling the grammar
binding an already-loaded Language into another store68 µs

Three properties fall out of that table, and each is load-bearing:

  1. load_language is not cached by the Engine. Loading the same bytes into a second store pays the full 102 ms again. So a grammar is loaded exactly once, at registration, and the Language is kept.
  2. A Language outlives the store it was loaded from, and is portable into any other store for 68 µs — three orders of magnitude cheaper than recompiling it. So load drops its loading store immediately; nothing has to keep it alive.
  3. A Tree survives its parser’s store being taken back, which is what makes the pooled store below safe.

And one constraint that is easy to violate and fails opaquely: every store must share the engine the grammar was compiled with. A Language from engine A cannot be instantiated in a store from engine B — WasmStore::new returns a bare Wasm error with no explanation. That is why [engine] is a process-wide OnceLock rather than something callers pass in: there is exactly one engine, so the mistake is unavailable.

§Two strategies, because two call shapes

[set_language] gives a parser its own store and leaves it there. That is right for crate::Syntax’s long-lived parser: 5 ms once per buffer whose language is wasm-backed, off the keystroke path, and the store must stay because the parser needs it for every later reparse.

[with_pooled_store] lends a thread-local store for the duration of one parse and takes it back. That is right for injection highlighting, which builds a fresh Parser per injection, per highlight call — a markdown file with twenty fenced blocks would otherwise pay 20 × 5 ms on every highlight. Property 3 above is why taking the store back after the parse is sound.

Native grammars touch none of this: both entry points check Language::is_wasm first and do nothing when it is false.

Functions§

load
Compile a grammar from a wasm side module.