Expand description
Loading grammars from WebAssembly, and the stores parsers need to run them.
Design:
plugin-languages.md §2.5.
Slice plan: LG.3b.
load turns a wasm side module into an ordinary
[tree_sitter::Language], which is the fact the whole design rests on:
downstream, HighlightConfiguration, folds, injections, indents and
the incremental reparse path cannot tell where a grammar came from.
§The one thing that is not transparent
A wasm-backed Language can only be used by a Parser that owns a
[WasmStore]. A parser without one fails set_language outright
(“Failed to load the Wasm store”), so this is not something that can be
left to chance. The costs, measured (see benchmarks.md):
| operation | cost |
|---|---|
WasmStore::new | 5.1 ms — it compiles tree-sitter’s wasm libc |
load_language | 102 ms — Cranelift compiling the grammar |
binding an already-loaded Language into another store | 68 µs |
Three properties fall out of that table, and each is load-bearing:
load_languageis not cached by theEngine. Loading the same bytes into a second store pays the full 102 ms again. So a grammar is loaded exactly once, at registration, and theLanguageis kept.- A
Languageoutlives the store it was loaded from, and is portable into any other store for 68 µs — three orders of magnitude cheaper than recompiling it. Soloaddrops its loading store immediately; nothing has to keep it alive. - A
Treesurvives its parser’s store being taken back, which is what makes the pooled store below safe.
And one constraint that is easy to violate and fails opaquely: every
store must share the engine the grammar was compiled with. A Language
from engine A cannot be instantiated in a store from engine B —
WasmStore::new returns a bare Wasm error with no explanation. That is
why [engine] is a process-wide OnceLock rather than something callers
pass in: there is exactly one engine, so the mistake is unavailable.
§Two strategies, because two call shapes
[set_language] gives a parser its own store and leaves it there.
That is right for crate::Syntax’s long-lived parser: 5 ms once per
buffer whose language is wasm-backed, off the keystroke path, and the
store must stay because the parser needs it for every later reparse.
[with_pooled_store] lends a thread-local store for the duration of
one parse and takes it back. That is right for injection highlighting,
which builds a fresh Parser per injection, per highlight call — a
markdown file with twenty fenced blocks would otherwise pay 20 × 5 ms
on every highlight. Property 3 above is why taking the store back after
the parse is sound.
Native grammars touch none of this: both entry points check
Language::is_wasm first and do nothing when it is false.
Functions§
- load
- Compile a grammar from a wasm side module.