The Lab · Quanta
A verified inference compiler for NVIDIA GPUs.
Quanta compiles a model straight to PTX and runs it through the CUDA driver API, one CUDA graph per token. Every kernel must agree bit for bit with a reference interpreter, or the build fails. It is how NovaTerra will run on a GPU its owner controls.
The measured result
1.27× llama.cpp, on one small model, at batch 1.
On Qwen2.5-0.5B Q6_K, batch-1 greedy decode on an RTX 3090, a Quanta-emitted engine reached 844 tokens a second against llama.cpp's 661, measured in the same session. Its output was bit-identical to its own CPU reference.
- tokens a second, Quanta-emitted engine
- 844
- tokens a second, llama.cpp, same session
- 661
- ratio, this one workload only
- 1.27×
- logit words differing from the CPU reference
- 0 / 24.6M
| Model | Qwen2.5-0.5B-Instruct, GGUF Q6_K |
|---|---|
| Workload | Batch 1, greedy decode, fixed 50-token continuation |
| Card | One NVIDIA GeForce RTX 3090 (24 GB, sm_86), i9-10900K host |
| Method | Quanta and llama.cpp alternated in the same session, both orders pooled |
| Clock | SM clock median 1,905 MHz, 68 to 75 °C, quiet host |
| Date | 5 August 2026 |
Where it does not hold
The limits, stated with the result.
A ratio without its conditions is not a result. These travel with it.
- This is the small, launch-bound end of the range. At 0.5B and batch 1 the card is mostly idle, so cutting launches pays off most.
- At 7B (Qwen2.5-7B Q4_K_M decode) our engine reached 0.88× llama.cpp. That engine used hand-written CUDA, not Quanta-emitted PTX.
- Prefill is not faster: llama.cpp is about 5 to 6 times quicker at reading the prompt.
- Greedy decoding only. There is no sampling, batching or tensor-core path yet.
- Individual kernels are slower than llama.cpp's. The gain is fusion: about 126 graph nodes per token against llama.cpp's 438 operations, in one CUDA graph.
- The raw llama.cpp log from that session was not kept. Quanta's side is reproducible from saved result files; the llama.cpp side is being re-measured now.
How it works
Checked, then fused.
Quanta is a language and compiler built in our lab. Its GPU backend emits PTX directly, with no cuBLAS and no hand-written kernels in the Quanta path.
1. Compile
About 70 Quanta kernel sources cover the decode path: quantised GEMV, fused RMSNorm and QKV, RoPE, grouped-query attention with a KV cache, SwiGLU and on-device argmax.
2. Check
Each build runs a reference interpreter beside the GPU code. If any output word differs, the build fails. Our evidence includes the controls that are shown to fail.
3. Launch
The whole token is captured as one CUDA graph and launched through the CUDA driver API, so the GPU spends less time waiting between small kernels.
More on the language itself: the Quanta research page and a recorded run of the compiler check.
Roadmap
What comes next.
01
Re-run and publish the evidence
The same alternating benchmark on the RTX 3090, with raw logs for both engines, clocks, temperatures and the model file's checksum. If the ratio comes in lower, we will say so here.
02
Ada, Hopper and Blackwell
The PTX path is tuned for sm_86 today. Next we retarget it to newer NVIDIA architectures and profile it with Nsight Systems and Nsight Compute.
03
A comparison against TensorRT-LLM
TensorRT-LLM becomes a published baseline alongside llama.cpp, at 0.5B and larger sizes, decode and prefill, losses included.
04
A local tier for NovaTerra
NovaTerra already routes each model tier through an OpenAI-compatible interface. The plan is a tier that runs on a GPU the member owns, so private data stays on their node.
How Quanta fits into the product: NovaTerra’s technical approach. The source is not public yet. We intend to open it once the evidence folder and reproducible scripts are ready. Questions to jon@vocabotics.com.