We put a 5MB embedding model in the browser

Semantic search usually means a server: an embeddings API, a vector database, a network round trip per query. ternlight (open source, MIT) skips all three. It is a 5-7 MB WebAssembly file that turns text into a 384-dimensional vector on the user's own CPU, in a few milliseconds, with nothing sent anywhere. We took it apart, built our own implementation searching this site's blog and news, and shipped it as a demo at /search.

What ternlight actually is

It is a small transformer, distilled from all-MiniLM-L6-v2, trained from the start as a ternary model: every weight is -1, 0, or +1, so inference is just adds and subtracts, no multiplication. That is the same idea behind Microsoft Research's BitNet b1.58. Ternary weights pack to 2 bits each, so the whole model, its tokenizer, and a hand-written Rust inference engine fit inside one `.wasm` file. No runtime download, no ONNX runtime, no GPU.

Why this fits a site with no backend

`next.config.mjs` sets `output: 'export'`. There is no server here at request time, full stop, which normally rules semantic search out entirely: it is the one search technique that usually needs an API call to an embeddings model. ternlight is the exception, because the embedding model ships to the browser instead of living behind an endpoint. The search is real, the constraint stays intact.

What we built

The /search page embeds every blog and news post's title, subtitle, and description once, on page load, then embeds whatever you type and ranks the corpus by cosine similarity. Repeated searches reuse the corpus vectors, so only the query gets re-embedded per keystroke.

const index = corpus.map((d) => ({ ...d, vector: embed(d.text) }));

// on every keystroke:
const qVector = embed(query);
const ranked = index
  .map((d) => ({ ...d, sim: cosineSim(qVector, d.vector) }))
  .sort((a, b) => b.sim - a.sim);

One real gotcha: webpack needs `experiments: { asyncWebAssembly: true }` to bundle a `.wasm` file imported as an ES module, which is not on by default in Next 15. One line in `next.config.mjs` and the static export builds clean.

mini vs base: the tradeoff, live

ternlight ships two tiers behind the same API. We put a toggle on the page instead of picking one for you:

Switch tiers on the page and you reload a different multi-MB model live, then watch the same query re-rank against it. That gap, 2.5ms versus 5.1ms per embed, a few hundredths of a point of retrieval quality, is the entire size-versus-quality decision this class of model asks you to make, made visible instead of buried in a config file.

What this is not

This is retrieval, not RAG. ternlight finds the posts closest in meaning to your query; nothing summarizes or answers in prose on top of that, because there is no generation step and no model call leaving the browser to do one. Full RAG would need an LLM in the loop, which is a heavier, different thing than "free-form on-device search." It also only searches title, subtitle, and description right now, not full post bodies, and the corpus is under twenty items. Small scale is exactly where this shines: no index to build, no postbuild step, embed the whole thing in the time it takes the page to paint.

Next step: try it yourself at /search, toggle mini and base mid-search, and watch the latency number move. All credit for the model itself to ternlight and its live demo; what we built is our own implementation of it, pointed at our own content.