Skip to main content

The Lab · Quanta

A verified inference compiler for NVIDIA GPUs.

Quanta compiles a model straight to PTX and runs it through the CUDA driver API, one CUDA graph per token. Every kernel must agree bit for bit with a reference interpreter, or the build fails. It is how NovaTerra will run on a GPU its owner controls.

The measured result

1.27× llama.cpp, on one small model, at batch 1.

On Qwen2.5-0.5B Q6_K, batch-1 greedy decode on an RTX 3090, a Quanta-emitted engine reached 844 tokens a second against llama.cpp's 661, measured in the same session. Its output was bit-identical to its own CPU reference.

Re-run in progress. These figures were measured on 5 August 2026. We are repeating the benchmark with raw logs for both engines, and this page will show the new figures, whichever way they fall.
tokens a second, Quanta-emitted engine
844
tokens a second, llama.cpp, same session
661
ratio, this one workload only
1.27×
logit words differing from the CPU reference
0 / 24.6M
Conditions
ModelQwen2.5-0.5B-Instruct, GGUF Q6_K
WorkloadBatch 1, greedy decode, fixed 50-token continuation
CardOne NVIDIA GeForce RTX 3090 (24 GB, sm_86), i9-10900K host
MethodQuanta and llama.cpp alternated in the same session, both orders pooled
ClockSM clock median 1,905 MHz, 68 to 75 °C, quiet host
Date5 August 2026

Where it does not hold

The limits, stated with the result.

A ratio without its conditions is not a result. These travel with it.

  • This is the small, launch-bound end of the range. At 0.5B and batch 1 the card is mostly idle, so cutting launches pays off most.
  • At 7B (Qwen2.5-7B Q4_K_M decode) our engine reached 0.88× llama.cpp. That engine used hand-written CUDA, not Quanta-emitted PTX.
  • Prefill is not faster: llama.cpp is about 5 to 6 times quicker at reading the prompt.
  • Greedy decoding only. There is no sampling, batching or tensor-core path yet.
  • Individual kernels are slower than llama.cpp's. The gain is fusion: about 126 graph nodes per token against llama.cpp's 438 operations, in one CUDA graph.
  • The raw llama.cpp log from that session was not kept. Quanta's side is reproducible from saved result files; the llama.cpp side is being re-measured now.

How it works

Checked, then fused.

Quanta is a language and compiler built in our lab. Its GPU backend emits PTX directly, with no cuBLAS and no hand-written kernels in the Quanta path.

  1. 1. Compile

    About 70 Quanta kernel sources cover the decode path: quantised GEMV, fused RMSNorm and QKV, RoPE, grouped-query attention with a KV cache, SwiGLU and on-device argmax.

  2. 2. Check

    Each build runs a reference interpreter beside the GPU code. If any output word differs, the build fails. Our evidence includes the controls that are shown to fail.

  3. 3. Launch

    The whole token is captured as one CUDA graph and launched through the CUDA driver API, so the GPU spends less time waiting between small kernels.

More on the language itself: the Quanta research page and a recorded run of the compiler check.

Roadmap

What comes next.

  1. 01

    Re-run and publish the evidence

    The same alternating benchmark on the RTX 3090, with raw logs for both engines, clocks, temperatures and the model file's checksum. If the ratio comes in lower, we will say so here.

  2. 02

    Ada, Hopper and Blackwell

    The PTX path is tuned for sm_86 today. Next we retarget it to newer NVIDIA architectures and profile it with Nsight Systems and Nsight Compute.

  3. 03

    A comparison against TensorRT-LLM

    TensorRT-LLM becomes a published baseline alongside llama.cpp, at 0.5B and larger sizes, decode and prefill, losses included.

  4. 04

    A local tier for NovaTerra

    NovaTerra already routes each model tier through an OpenAI-compatible interface. The plan is a tier that runs on a GPU the member owns, so private data stays on their node.

How Quanta fits into the product: NovaTerra’s technical approach. The source is not public yet. We intend to open it once the evidence folder and reproducible scripts are ready. Questions to jon@vocabotics.com.

    We use cookies.

    Quanta: a verified inference compiler for NVIDIA GPUs · vocabotics