A capable model on hardware you own: inference and training, our own stack

The promise "a capable model on hardware you own" is easy to say and hard to mean. Meaning it requires a stack you actually control — no cuBLAS, no PyTorch underneath — running a real model and producing the same tokens as the reference. That is what this lane measures. And, in the spirit of the house, it measures the ceilings too.

A real model, generating real text, on our own stack

✅Shipped & measured

A real Qwen2.5-0.5B runs a full forward pass in QUANTA, from QUANTA-emitted PTX with zero cuBLAS and zero cudart, on real GGUF weights. Measured: 6.10 ms/token = 163.9 tok/s — a 5,591× speed-up over the 34-second single-thread baseline — with argmax matching llama.cpp (" The" → " problem") and GPU results equal to CPU to 3.84e-04. 🟢

Then the full loop: interactive generation in QUANTA with a KV-cache, real GQA attention, a BPE tokenizer (151,936-vocab, read from the GGUF), and sampling. "The capital of France is" → " Paris. It is the largest city in Europe…", token-for-token identical to llama.cpp, decoding at 151.7 tok/s. 🟢 A real LLM, generating correct text end-to-end, on a stack we built ourselves.

Flat where a transformer climbs

✅Shipped & measured

The costly signal for the engineers: 128k long-context decode runs 2× faster than llama.cpp, and the latency line stays flat where a transformer's climbs. 🟢 Sparse attention reaches 59.7× at 65k tokens; the WMMA GEMM hits 92.8% of cuBLAS, bit-exact, with int4 fused decode at 2.9× (M=1) and an 8.7× cut in HBM traffic. The whole thing is 21 ms per step in 377 MiB on one 3090.

Training, too — as a compiler pass

✅Shipped & measured

Inference is only half a stack. Training works in QUANTA: a real MLP trained end-to-end, loss 0.15 → 0.0015 (a 99% reduction), gradients finite-difference verified to 5.6e-7, forward native ≡ interpreter. 🟢 Autodiff is implemented as a bit-identical IR-to-IR pass — the backward graph is derived, not hand-written — with four optimizers. The compiler is transformer-ready: attention backward and Adam are authoring work, not compiler changes. QUANTA now infers and trains.

What we will not claim

🔬Research

Our GPU architect keeps us honest, so we keep the reader honest:

• We are not the fastest. 🔬 92.8% of cuBLAS is fast and bit-exact — but it is 92.8%, not 105%. We will not say "fastest".
• Prefill is behind. 🔬 Decode is our strength; prefill still trails llama.cpp, and paged-KV is not done.
• 7B is parity, not lead. 🔬 At 7B, decode is ~128 tok/s against llama.cpp's ~146 — parity, not an advantage. A fast QUANTA-authored full-model decode is in progress, not shipped.

Why it ladders back

If a capable model runs on the 3090 you already have, generating the same tokens as the reference and provable at every step, then capability stops being something you rent. That is the access half of an abundant world.