- Entry date
- 15 March 2026
- Category
- Inference
- Lead over the world
- ~1–2 years ahead
- Access
- 🔒 Protected
We built an operating system for the GPU — VRAM as a page table, models as processes — and pushed a real model to 147 tokens a second on a gaming card.
- 147.2 tok/s
- measured on one RTX 3090 (FP16)
- 2.8–4.5×
- speculative decoding speedup
- 2×
- context from INT8 KV-cache (cos 0.9999)
- ~122K
- lines of Rust in the engine + OS
Honest evaluation
The 147.2 tok/s inference-engine result is measured and real; a separate ternary-7B speed claim inside the same project was measured, found false, and retracted.
Runnable proof — see it work
What we learned — a preview
The honest edges are open even when the engine is not. The full report — what it was, what we built, and the measured internals — is protected.
- An on-disk ternary-7B speed claim was RETRACTED: the generic-kernel ternary path measured ~7× slower than Q4, and one on-disk perplexity figure did not reproduce. The footprint win is real; the speed win, so far, is not.
- GPU utilisation is structurally limited (~14% on some paths) — a physics check, not a bug to be optimised away cheaply.
- The wider platform this feeds still carries a large backlog of type errors and is not integrated end-to-end; 'compiles and runs in parts' is not 'shipped'.
- The aspirational step-time targets remain honestly flagged as targets, not results.
The deep body of “PRISM — a GPU operating system, and datacentre-class inference on a single 3090” is behind access.
We open the demos, the specs, and the method; we protect the engines, the model internals, and anything that touches commercial, safety-critical work. This report describes an engine — so its details are gated, and no client, company, or project is named. Members can read it in full; if you have a genuine reason to see it, tell us who you are and why.
Proofs & sparks
We demonstrate rather than assert. Each ✅ proof is a visible result with a hard figure.
- GPU-as-OS inference147.2 tok/s on one 3090an inference engine that treats GPU/VRAM like an OS treats CPU/process memory — measured at 147.2 tok/s on a single RTX 3090 (engine source protected).