A capable model on hardware you own: inference and training, our own stack

The promise "a capable model on hardware you own" is easy to say and hard to mean. Meaning it requires a stack you actually control — no cuBLAS, no PyTorch underneath — running a real model and producing the same tokens as the reference. That is what this lane measures. And, in the spirit of the house, it measures the ceilings too.

A real model, generating real text, on our own stack

✅Shipped & measured

A real Qwen2.5-0.5B runs a full forward pass in QUANTA, from QUANTA-emitted PTX with zero cuBLAS and zero cudart, on real GGUF weights. Measured: 6.10 ms/token = 163.9 tok/s — a 5,591× speed-up over the 34-second single-thread baseline — with argmax matching llama.cpp (" The" → " problem") and GPU results equal to CPU to 3.84e-04. 🟢

Then the full loop: interactive generation in QUANTA with a KV-cache, real GQA attention, a BPE tokenizer (151,936-vocab, read from the GGUF), and sampling. "The capital of France is" → " Paris. It is the largest city in Europe…", token-for-token identical to llama.cpp across the 20 generated tokens of that prompt (it diverges after token 20), decoding at 151.7 tok/s. 🟢 A real LLM, generating correct text end-to-end, on a stack we built ourselves.

Be clear about what that is and is not: an exact agreement on twenty tokens of one prompt is a correctness result, not a speed comparison. Our head-to-head against llama.cpp is a separate, int4 short-context bench, and it reads 305 tok/s for us against llama.cpp's 313 — level, not ahead — with prefill 5–30× behind because we do not batch the prompt. Our own governing note on this says it plainly: we can claim bit-exact full decode against a CPU reference, but not an unqualified "bit-exact vs llama.cpp" — llama parity holds on 2 of 3 prompts.

Flat where a transformer climbs

✅Shipped & measured

The costly signal for the engineers: 128k long-context decode runs 2× faster than llama.cpp, and the latency line stays flat where a transformer's climbs. 🟢 Sparse attention reaches 59.7× at 65k tokens; the WMMA GEMM hits 92.8% of cuBLAS, bit-exact, with int4 fused decode at 2.9× (M=1) and an 8.7× cut in HBM traffic. The whole thing trains at 21 ms per step, 377 MiB peak, on one 3090 — a median training step over 250 steps, not a serving latency; we publish no serving latency for the mind.

Training, too — as a compiler pass

✅Shipped & measured

Inference is only half a stack. Training works in QUANTA: a real MLP trained end-to-end, loss 0.15 → 0.0015 (a 99% reduction), gradients finite-difference verified to 5.6e-7, forward native ≡ interpreter. 🟢 Autodiff is implemented as a bit-identical IR-to-IR pass — the backward graph is derived, not hand-written — with four optimizers. The compiler is transformer-ready: attention backward and Adam are authoring work, not compiler changes. QUANTA now infers and trains.

What we will not claim

🔬Research

Our GPU architect keeps us honest, so we keep the reader honest:

• We are not the fastest. 🔬 92.8% of cuBLAS is fast and bit-exact — but it is 92.8%, not 105%. We will not say "fastest".
• Prefill is behind. 🔬 Decode is our strength; prefill still trails llama.cpp, and paged-KV is not done.
• 7B is parity, not lead. 🔬 At 7B, decode is ~128 tok/s against llama.cpp's ~146 — parity, not an advantage. A fast QUANTA-authored full-model decode is in progress, not shipped.

Why it ladders back

If a capable model runs on the 3090 you already have, generating the same tokens as the reference and provable at every step, then capability stops being something you rent. That is the access half of an abundant world.