Zayd Mulani

A few months ago I got obsessed with a question: can you run a real LLM entirely inside a browser tab, with zero backend, zero GPU, and zero install?

The answer is yes. Here's what I built.

ghost is a single HTML file that downloads a quantized language model into your browser's cache on first visit, then runs inference locally in WebAssembly forever after. Fully offline after that first download. No API key. No npm. No build step. Open the file, pick a model, chat.

How it works

The inference engine is wllama — a WebAssembly binding for llama.cpp. It runs GGUF quantized models directly in the browser using WASM SIMD. I pin it to a specific version so the JS and WASM files always match (learned this the hard way after a fun debugging session involving mismatched memory imports).

Models are downloaded from HuggingFace on first load and cached via the browser's Cache API. On every subsequent visit they load instantly from cache, no network needed.

Features

Three models: Qwen2.5 1.5B (smart), Qwen2 0.5B (fast), TinyLlama (lightweight)
Markdown rendering from scratch — no library, just regex transforms
RAG: drag a .txt or .pdf onto the chat window. It chunks the text, embeds each chunk using wllama's embedding API, stores vectors in memory, and retrieves the top-3 relevant chunks on each message. Fully local, fully offline
Voice input via the Web Speech API — mic button auto-sends on silence
Multi-turn conversation memory capped at 10 turns
PWA installable — works on mobile home screen too

The hard parts

Getting wllama to load from a cached model was genuinely tricky. Blob URLs created in the main thread aren't accessible from wllama's internal Web Worker. IndexedDB chunk reconstruction hit a 2GB ArrayBuffer limit on Windows Chrome. The final solution was using wllama's built-in loadModelFromHF with useCache: true which handles everything internally.

The embeddings API requires toggling a flag (embeddings: true) that conflicts with normal chat completion — so I toggle it on before each embedding call and back off in a finally block.

Try it

Live: https://zaydmulani09.github.io/ghost/ghost.html
GitHub: https://github.com/zaydmulani09/ghost

Speed on CPU is ~3-8 tokens/sec depending on model. Not fast, but it works, and every token stays on your machine.