Geodesic Runtime Compression: a calibration-free, super-baseline attention compression

Project nomenclature

This paper series uses a consistent four-level naming hierarchy. HyperTensor is the umbrella project (compiler, runtime, kernels, and theory). Geodesic Projection (GP) is the overarching inference-time pipeline that factorises weights along their dominant geometric directions. Geodesic Runtime Compression (GRC), the subject of this paper, is the attention-only subset of GP applied to \(\Wmat_Q,\Wmat_K,\Wmat_V\) at load time with no calibration and no fine-tuning. OTT/GTC (Organic Tensor Topology and the Geodessical Tensor Cache) is the longer-term theoretical framework that treats the projection bases as charts on a Riemannian manifold and is the subject of the companion paper. This paper is self-contained and assumes only the GRC layer.

Contributions

1. A clean test of calibration-free compression: PCA of the joint Gram matrix of \(\{\Wmat_Q,\Wmat_K,\Wmat_V\}\) produces a portable, sign-canonical, bit-stable projection without text samples (3).

2. A measured super-baseline: \(106.27\%\) relative decode throughput at \(k{=}1024\) on Llama-3.1-8B / RTX 4070 Laptop, with \(t{=}53.88\), \(p<10^{-10}\), bootstrap CI excluding 1.0 ([sec:results,sec:stats]).

3. A two-part mechanism analysis via direct Nsight Compute trace (6.2): (a) the attention-projection L2 hit-rate increases from \(7.47\%\) to \(12.15\%\) (\(+4.68\) pp) at \(k{=}1024\), confirming that the attention working set (\(\sim 10\) MB) achieves improved L2 residency inside the \(32\) MB L2 cache; (b) Q/K/V kernel fusion replaces three separate gemv_q4_k launches with a fused dual-Q8 trio, reducing scheduler pressure. The GEMV-weighted mean hit-rate is flat because FFN kernels (which GRC does not compress) dominate the average.

4. Numerical comparison against the Eckart--Young--Mirsky bound (Eckart and Young 1936), quantifying the cost of a shared basis vs. per-matrix bases (8).

5. A thermally-controlled measurement protocol that converts a 53% throttled false-negative into a 97.55% retention result, with documented limitations (9).

Why the construction generalises

The physical justification for a calibration-free, low-rank construction is that the intrinsic dimension of the joint \(\{\Wmat_Q,\Wmat_K,\Wmat_V\}\) Gram is essentially decoupled from the ambient model width. Running the same manifold-extraction pipeline (companion paper, GP) on three open models of widely different sizes gives the following \(k_{\mathrm{int}}\) at \(95\%\) variance capture.

The ratio \(k_{\mathrm{int}}/d\) decreases with width. A \(5.3\times\) increase in \(d\) buys roughly an \(8\times\) drop in fractional intrinsic dimension. We read this as empirical evidence that attention-stack low-rank structure is a universal property of decoder-only transformers and not an artefact of one architecture. This is the geometric premise on which both GRC and the OTT runtime (2, companion Paper D) are built.

Memory-bandwidth-bound decode.

For a single-batch autoregressive step, the operational intensity (FLOPs / byte) of the attention and FFN GEMMs is below the ridge of every consumer GPU we are aware of (Yuan et al. 2024; NVIDIA Corporation 2022). Throughput is therefore well-approximated by \(T \approx \min(\text{compute}, \text{bytes}/\text{BW})\), and reducing bytes moved per token is the dominant lever.

Compression families.

Quantisation (GPTQ (Frantar et al. 2022), AWQ (Lin et al. 2023), SmoothQuant (Xiao et al. 2022), LLM.int8() (Dettmers et al. 2022)) keeps the matrix shape and lowers per-element bit width. Pruning (SparseGPT (Frantar and Alistarh 2023)) zeroes individual weights. Low-rank factorisation (FWSVD (Hsu et al. 2022), ASVD (Yuan et al. 2023), SliceGPT (Ashkboos et al. 2024)) replaces a full-rank weight \(\Wmat\!\in\!\R^{m\times n}\) with \(\Wmat\Pmat_k\Pmat_k^\top\) or \(\Wmat U_kV_k^\top\). Adapters (LoRA (Hu et al. 2021), QLoRA (Dettmers et al. 2023)) keep the model frozen and add small trainable terms. All of FWSVD/ASVD/SliceGPT use calibration data to weight directions by activation magnitude.

Why attention is a low-rank target.

Kobayashi et al. (Kobayashi et al. 2024) prove that weight decay during training induces low effective rank in attention projections. Empirically, FFN weights are far less amenable to global low-rank truncation than attention weights (6). This paper compresses only \(\Wmat_Q,\Wmat_K,\Wmat_V\); FFN remains full-rank.

Calibration-free as an experimental condition.

The mathematical machinery used here, principal-component projection of a Gram matrix, is standard and dates back at least to the Eckart, Young and Mirsky theorem (Eckart and Young 1936) and to the matrix analysis literature (Trefethen and Bau III 1997; Golub and Van Loan 2013). The contribution is therefore not a factorisation algorithm. What is new is treating the omission of calibration data as a controlled experimental condition. Earlier post-training compression work, such as FWSVD, ASVD and SliceGPT, uses calibration to weight directions by activation magnitude, which entangles the calibration corpus with the deployed model. We hold that variable fixed at zero and measure how much signal weight geometry alone retains.

Connection to representation-learning objectives.

A related thread of recent work argues that representation quality, not raw parameter count, is the operative quantity in large model performance. Joint-embedding predictive architectures (JEPA, V-JEPA) (Assran et al. 2023; Bardes et al. 2024) learn a latent space directly and discard the pixel-level reconstruction objective, on the grounds that the low-rank predictive subspace is what carries semantic content. The hierarchical online predictive evolution (HOPE) family (Behrouz and Hashemi 2025) learns continual-update subspaces whose effective rank is bounded well below ambient. GRC is consistent with that picture in a post-hoc setting: we observe that already-trained transformer attention weights expose an analogous low-rank predictive subspace, and that an uncalibrated projection onto it preserves both perplexity (within the \(+13.30\%\) band at \(k{=}1536\)) and decode throughput. We do not claim a causal link between the two literatures, but the empirical alignment is the reason we expect the GRC effect to transfer across architectures rather than being a Llama-specific quirk.

Construction

For each transformer block \(\ell\), given attention projections \(\Wmat_Q^{(\ell)},\Wmat_K^{(\ell)},\Wmat_V^{(\ell)}\in\R^{d\times d_h}\) (GQA contracts the \(d_h\) axis for K,V (Ainslie et al. 2023)):

1. Build the symmetric joint Gram matrix \(\Kmat^{(\ell)} = \Wmat_Q^{(\ell)}\Wmat_Q^{(\ell)\top}+\Wmat_K^{(\ell)}\Wmat_K^{(\ell)\top}+\Wmat_V^{(\ell)}\Wmat_V^{(\ell)\top}\in\R^{d\times d}\).

2. Power-stabilised eigendecomposition (3 power iterations followed by a Jacobi sweep) yields the leading \(k\) eigenpairs \((\lambda_i,\mathbf{u}_i)_{i=1}^k\). Sign-canonicalise so that the largest absolute entry of each \(\mathbf{u}_i\) is positive.

3. Form the projection \(\Pmat_k^{(\ell)}=[\mathbf{u}_1,\dots,\mathbf{u}_k] \in\R^{d\times k}\) and store the projected weights \(\widetilde{\Wmat}_^{(\ell)} = \Pmat_k^{(\ell)\top}\Wmat_^{(\ell)}\) for \(*\in\{Q,K,V\}\).

4. At inference, the attention input \(\mathbf{x}\in\R^d\) is projected once per block as \(\widetilde{\mathbf{x}} = \Pmat_k^{(\ell)\top}\mathbf{x}\in\R^k\); all subsequent attention math runs in the reduced \(k\)-dimensional subspace until the output projection \(\Wmat_O\) (which we leave full-rank).

Properties

The four properties below follow directly from the construction. We list them because each one removes a class of failure mode that calibration-based compression schemes inherit by default.

• Calibration-free. The construction depends only on the weights; no text samples, no activation traces, no hyperparameters beyond \(k\). The cache key is \((\textsf{model SHA-256}, k)\).

• Bit-stable. Sign canonicalisation removes the \(\pm1\) ambiguity of eigenvectors; we observe identical \(\Pmat_k\) across OpenBLAS/MKL/Accelerate down to float32 bit pattern.

• Drop-in. Wraps any GGUF (Gerganov and llama.cpp contributors 2023a) model with k-quants (Gerganov and llama.cpp contributors 2023b); no kernel changes required.

• Bytes saved. Per layer the Q/K/V working set drops from \(3d^2\) to \(3kd\) floats, a \(d/k\) factor; for Llama-3.1-8B at \(k{=}1024\), this is \(4\times\) reduction on Q (which dominates).

Roofline formalisation

Let \(M(k)\) denote the bytes of weight memory streamed per generated token at rank \(k\), and \(F(k)\) the corresponding FLOPs. The classical arithmetic intensity is \[\begin{equation} I(k)\;=\;\frac{F(k)}{M(k)}\quad[\text{FLOP/byte}], \label{eq:ai} \end{equation}\] and the Roofline upper bound on throughput (Williams et al. 2009) is \[\begin{equation} T(k)\;\le\;\min\!\Bigl(\,F_\text{peak},\; I(k)\,\cdot\,BW_\text{eff}(k)\,\Bigr), \label{eq:roofline} \end{equation}\] where \(BW_\text{eff}(k)\) is the effective bandwidth observed by the kernel. The key observation is that \(BW_\text{eff}\) is not a single number: it is a piecewise function of where the projected attention working set \(S(k)\) lives in the cache hierarchy, \[\begin{equation} BW_\text{eff}(k)\;\approx\; \begin{cases} BW_\text{L2}\;\;(\sim\!2\text{--}4\,\text{TB/s}) & S(k)\le C_\text{L2},\\ BW_\text{HBM}\;\;(256\,\text{GB/s}) & S(k)>C_\text{L2}, \end{cases} \label{eq:bweff} \end{equation}\] with \(C_\text{L2}=32\) MB on the test GPU. The transition is not sharp; L1/register pressure and KV-cache traffic spread it over a band of \(k\) values, but the order of magnitude jump in \(BW_\text{eff}\) at the crossing is what makes the super-baseline numerically possible: a \(\sim\!10\times\) bandwidth gain pays for the additional projection FLOPs and still leaves headroom. Concretely, for our \(d{=}4096\) Llama: \(S(1024)\!\approx\!10\) MB \(\le C_\text{L2}\) and the \(k{=}1024\) row of 1 sits in the L2 regime; \(S(1536)\!\approx\!17\) MB still fits but with KV plus residual it crowds the cache and the kernel oscillates between regimes, consistent with the elevated variance reported in 12. The cross-GPU predictions in 3 are obtained by inverting [eq:bweff]: solve for the largest \(k\) with \(S(k)\le 0.75\,C_\text{L2}\) (25% headroom for KV and residual). This is a falsifiable model, not a guarantee, a single Nsight Compute run reporting low l2_tex_hit_rate at \(k{=}1024\) would refute it.

Direct L2 trace on RTX 4070 Laptop

We profiled \(200\) representative decode-phase launches per kernel (approximately \(5{,}500\) kernels total, with --launch-skip 50 to bypass prefill warm-up) using NVIDIA Nsight Compute 2026.1.0.0 and four metrics: lts__t_sector_hit_rate.pct, lts__t_sectors_op_read.sum, dram__bytes_read.sum and sm__warps_active.avg.pct_of_peak_sustained_active (NVIDIA Corporation 2024). Both the baseline and the --axex-compress --axex-compress-rank 1024 --axex-skip-o configuration were captured back to back on the same GPU. The per-kernel breakdown is in 2.

The trace must be read with one critical scope qualifier: GRC compresses only attention Q/K/V weights. FFN is untouched. The dominant DRAM consumer per decode step is the FFN gemv_q4_k(1792) kernel at \(33\) MB per launch and a \(3.50\%\) L2 hit-rate that is identical in both runs. Because this single kernel accounts for roughly \(50\%\) of total DRAM traffic, it anchors the GEMV-weighted mean (\(17.0\%\) vs. \(16.3\%\), a \(-0.71\) pp delta within noise) and makes the weighted average an inappropriate metric for testing the attention-specific cache-fit prediction.

When restricted to the GRC-affected kernel, the picture is different. The attention-projection gemv_q4_k(512) L2 hit-rate rises from \(7.47\%\) to \(12.15\%\) (\(+4.68\) pp). This is the kernel whose weight working set GRC directly shrinks. The \(+4.68\) pp improvement is in the direction predicted by the geometric model that reducing the compressed-weight footprint from \({\sim}25\) MB (full \(d{=}4096\) Q/K/V) to \({\sim}10\) MB (projected \(k{=}1024\)) reduces L2 competition. We do not claim it confirms the mechanism; 6.3 enumerates the residual confounds and the experimental protocol that would do so.

One caveat is required for the DRAM column: the measured DRAM for gemv_q4_k(512) rises from \(9.46\) MB to \(18.47\) MB in GRC mode, not falls. This is expected: in GRC mode the same kernel is also responsible for applying the stored basis matrix \(P_k\) (\(d{\times}k\) in FP16, \({\sim}8\) MB per call), so total bytes read per dispatch includes both the compressed weight tensor and the basis vectors. The theoretical \({\sim}10\) MB refers strictly to the compressed weight portion; it does not account for the additional \(P_k\) read. A multi-\(k\) sweep isolating weight reads from basis reads is the experiment that would cleanly separate these contributions and is listed under 10. The L2 hit-rate improvement (\(+4.68\) pp) is the primary evidence: it measures the fraction of sectors served from L2 regardless of which data source, and it rises with compression even as total DRAM rises, indicating that the compressed data layout or reduced working-set pressure has improved cache utilisation in the attention path.

We report both numbers in 2 so the reader can verify: the FFN row is flat (GRC does not touch it), and the attention projection row is up.

The quantitative statement therefore has two parts. First, the attention-specific L2 cache-fit hypothesis is supported by the trace: the kernel whose weights are compressed shows a measured \(+4.68\) pp hit-rate increase at \(k{=}1024\), consistent with the \(S(1024)\approx 10\) MB \(< C_\text{L2} = 32\) MB prediction from 6. Second, the GEMV-weighted mean hit-rate does not improve because FFN kernels, unaffected by GRC, dominate that average.

Attention-specific L2 residency improvement is therefore a contributing mechanism to the super-baseline. It is not the sole mechanism. The second mechanism is visible elsewhere in the trace, in the kernel mix. The compressed path replaces the three separate Q/K/V gemv_q4_k launches with a fused gemv_dual_q8_0, gemv_q4_0_prequant and gemv_q8_0_batched pipeline operating on \(k\)-dimensional intermediates. Aggregate DRAM across the three fused-Q8 kernels is \(9.08\) MB against the single replaced gemv_q4_k(512) at \(9.46\) MB baseline; the difference is small, but three kernel launches collapse into one fused trio, reducing dispatch overhead and scheduler pressure. Both contributions, attention-weight L2 residency and Q/K/V kernel fusion, are consistent with the measured \(+6.27\%\) decode throughput.

A direct ablation disabling fusion while keeping \(k{=}1024\) would isolate each contribution and is the critical next experiment (10). The cache-fit calculation in 3 is retained as a capacity-budget bound with the dual interpretation it has earned: it predicted the rank at which the attention working set fits L2 (geometric fact, confirmed), and the rank at which the fused Q8 path becomes viable on this GPU (empirical observation, confirmed). It should not be read as predicting that the GEMV-weighted hit-rate will improve across all kernels, because GRC does not compress FFN.

Raw CSVs and the parser are at and ; raw NCU dumps at .

What would constitute definitive proof of L2 causation

The current evidence for the L2 cache-fit mechanism is one paired NCU run (\(n{=}1\)) on a single GPU, showing a \(+4.68\) pp hit-rate increase on the GRC-affected attention-projection kernel and a flat hit-rate on the GRC-untouched FFN kernel. This is suggestive but not definitive. A disciplined reader can list specific reasons the result might be biased or incidental:

1. Kernel-mix change. GRC mode replaces three baseline gemv_q4_k(512) launches with a fused gemv_dual_q8_0 + gemv_q4_0_prequant + gemv_q8_0_batched trio. Comparing the L2 hit-rate of two structurally different kernels is not apples-to-apples; the access pattern, block shape, and quantisation format all differ.

2. DRAM read rises, not falls. A naive cache-fit story predicts that more L2 hits should reduce DRAM bytes. The affected kernel reads \(9.46\) MB at baseline and \(18.47\) MB in GRC mode (the basis matrix \(P_k\) adds \({\sim}8\) MB). The hit-rate metric measures the fraction of sectors served from L2, but it is computed over a different total volume in each mode, so a higher percentage does not by itself imply more bytes saved from DRAM.

3. Single profiling session. The trace was collected once per configuration. NCU sampling (--launch-skip 50 --launch-count 200) yields per-kernel mean and stdev across the 200 sampled launches, but not across independent sessions. System co-tenancy, GPU frequency state, or background processes during the trace cannot be ruled out from \(n{=}1\).

4. No cache-size manipulation. The strongest available test is to artificially shrink L2 (CUDA exposes cudaDeviceSetLimit(cudaLimitPersistingL2CacheSize, ...) on A100 and later; on the RTX 4070 Laptop, the equivalent control is co-running an L2-thrashing kernel that evicts persistent lines). If GRC's speed-up vanishes when L2 is constrained while baseline is unaffected, L2 is causal. Without this experiment, residency is a hypothesis, not a finding.

5. Single GPU. The cache-fit model predicts that optimal \(k^\star\) scales with L2 capacity (3). A measurement on RTX 4090 (72 MB L2, predicted \(k^\star{=}1536\)) or A100 (40 MB, predicted \(k^\star{=}1024\)) that matches the prediction would be strong cross-system evidence; a measurement that does not would refute the model.

A definitive falsification protocol therefore consists of three experiments, each of which has a binary pass / fail outcome:

Full working-set derivation

For full transparency we list every byte that enters the active L2 working set during a single decode step, projected onto rank \(k\):

\[\begin{align} S(k) &= \underbrace{2\,d\,k}_{\text{proj. }\Wmat_Q\text{ in fp16}} + \underbrace{2\,d\,k}_{\Wmat_K^{(\text{proj})}} + \underbrace{2\,d\,k}_{\Wmat_V^{(\text{proj})}}\notag\\ &\quad + \underbrace{2\,d\,k}_{P_k\text{ basis, fp16}} + \underbrace{2\,d\,k}_{\widetilde P_k\text{ inverse basis}} + \underbrace{2\,d_\text{kv}\,L_\text{ctx}}_{\text{KV cache}} + \underbrace{2\,d}_{\text{residual}} \label{eq:Sk} \end{align}\]

Substituting \(d{=}4096\), \(d_\text{kv}{=}1024\), \(L_\text{ctx}{=}512\), \(k{=}1024\):

• Projected Q/K/V: \(3\cdot 2\cdot 4096\cdot 1024 = 24\) MB

• Basis \(P_k\) + inverse: \(2\cdot 2\cdot 4096\cdot 1024 = 16\) MB

• KV cache: \(2\cdot 1024\cdot 512 = 1\) MB

• Residual: \(2\cdot 4096 = 8\) kB

The naive sum is \(41\) MB, which exceeds \(C_\text{L2}{=}32\) MB, but the basis matrices \(P_k\) and \(\widetilde P_k\) are loaded once per layer and held resident across the three Q/K/V projections, so the simultaneously-resident working set during any single GEMV launch is \(S_\text{kernel}(k) \approx 2\,d\,k_\text{(weight)} + 2\,d\,k_\text{(basis)} = 16\) MB at \(k{=}1024\), well inside L2. The \({\sim}10\) MB number cited in 6 refers to the weight-only portion (\(8\) MB at \(k{=}1024\)) plus residual and intermediate activation overhead, and is the figure that maps onto "what an idealised cache-fit would shrink." The discrepancy between the \(10\) MB working-set estimate and the measured \(18.47\) MB DRAM read is exactly the basis-matrix load, which is why the DRAM column rises even as the hit-rate also rises: the kernel is reading two distinct data products, the compressed weight and the basis, both of which now fit in L2 jointly with the activation. We provide the byte-level calculation here so the reader can audit the \({\sim}10\) MB figure against the \({\sim}18\) MB measurement and verify the discrepancy is fully accounted for by basis traffic.

Spectral evidence for compressing attention only

Direct spectral analysis confirms that attention is a more compressible target than the feed-forward network. Computing the full SVD of the per-layer concatenated weights for layers \(\{0,15,31\}\) of Llama-3.1-8B, the rank required to capture \(95\%\) of the Frobenius norm is \(k_{95}\approx1682\) for attention (Q, K and V concatenated) and \(k_{95}\approx3345\) for FFN (gate, up and down concatenated). FFN spectra are nearly flat (Geva et al. 2021) while attention spectra concentrate on a few hundred directions.

The gap is not a measurement artefact. The two components serve different geometric roles inside a transformer block. Attention is a routing mechanism. The matrices \(\Wmat_Q\) and \(\Wmat_K\) project tokens into a similarity space whose discriminative power is concentrated in a small number of directions, namely the head subspaces plus the preferred subspace of the residual stream, and \(\Wmat_V\) assembles a context-conditional message from that low-rank query. Routing is structurally low-rank. The FFN behaves as a key, value memory bank. Geva et al. (Geva et al. 2021) show that FFN columns act as keys with localised, dense activation patterns, so a useful FFN must span a high-dimensional set of keys. Removing FFN directions removes individual memories. Removing attention directions removes routing fidelity at worst, which is partly recoverable from the residual stream. This explains both observations: FFN \(k_{95}/d\approx0.85\) against attention \(k_{95}/d\approx0.41\), roughly a \(3.4\times\) rank gap for the same energy capture, and the empirical fact that aggressive FFN low-rank truncation is unstable at 8B scale while attention truncation remains benign down to \(k{=}1024\). Compressing FFN globally would lose unacceptable information. Compressing attention globally is benign at \(k\geq1536\) on this model.

Hardware, model, and runtime configuration

Storage footprint

First-run calibration time (CPU-bound truncated SVD over 32 layers \(\times\) 3 weight sets, dequantised Q4_K_M \(\to\) fp32) is 60--120 s on the Ryzen 9 7940HS; warm cache hit elides this entirely. The \(\Wmat_\text{proj}\) cache file is a self-contained, model-specific binary artefact. It can be distributed independently of the model file: any session (on any machine) that holds both the GGUF model and the matching cache file incurs zero calibration time. Two separate inference sessions sharing the same cache file will produce bit-identical outputs.

VRAM profile

Runtime-reported model load: 226 weight tensors (4 684 MB) + scratch arena (513 MB) + KV cache and activations at 8K context (\({\sim}615\) MB). The 1.09 GB on-disk \(\Wmat_\text{proj}\) is only partially resident in VRAM during decode, the GPU-resident working set uses only the active layer's projected matrices, hence the small +36 MiB delta despite the large on-disk artefact.

Power draw

Per-token efficiency (baseline TpF report): HBM bandwidth 174--179 GB/s (68--70% of 256 GB/s peak), compute 588--606 GFLOPS (1.47--1.51% of 40 TFLOPS peak), \(\sim 4.9\) GB loaded per token. The \(<\!2\%\) compute / \(\sim\!70\%\) bandwidth split confirms the Roofline reading: this is a memory-bound regime where bytes-saved is the right axis to optimise.

Full spectral analysis (rank required for 95% energy)

We computed the full SVD of every attention and FFN weight in five layers (\(L\in\{0,7,15,23,31\}\)) of Llama-3.1-8B-Instruct after Q4_K_M dequantisation. The rank required to capture \(95\%\) of \(\norm{\Wmat}_F^2\):

Confidence-interval pack (12-rep)

Granular rank-Pareto sweep

The headline numbers in 1 are read from the rank_sweep stage of the reference pack, which evaluates four prompt classes (coding, reasoning, factual, creative) at two generated-token lengths (128, 256) for each compression rank. Aggregating those raw samples (\(n{=}8\) per condition) gives the Pareto curve in [tab:rank-pareto]; the snippet is regenerated from benchmarks/whitepaper_pack_20260427_121815/rank_sweep_raw.csv by scripts/build_rank_pareto_snippet.py.

The peak at \(k{=}1024\) (ratio \(1.063\times\)) and the dip-then-recovery at \(k{=}1536, 2048\) are the empirical signature of the cache-fit regime described in 6. Variance grows with rank, again consistent with the working-set/L2 boundary becoming the rate-limiting factor.

Decode-length sensitivity

To check that the throughput numbers are not an artefact of the specific generated-token budget, [tab:decode-length] splits the same rank_sweep samples by decode length (128 vs. 256 tokens). Snippet emitted by scripts/build_decode_length_snippet.py from the same source CSV.

Context-length sweep

A dedicated 128/512/1024/2048/4096-token context sweep on the same RTX 4070 Laptop (Llama-3.1-8B Q4_K_M, \(n{=}5\) samples per cell, \(-n\,16\) decode budget) confirms that GRC's throughput uplift is essentially flat in context length across the warm-cache regime tested.

The ratio is stable in \([1.044, 1.072]\) over a \(32{\times}\) context-length range, consistent with the cache-fit hypothesis: once weight-PCA reduces the attention working set below the 32 MB L2 capacity, the win is context-independent at these decode budgets. Source CSV at benchmarks/cross_hw_local_fix_20260428_192807/docs_data/context_length_sweep.csv.

Cross-model transfer to SmolLM2-135M

To check whether the cache-fit win generalises to a different model family and a much smaller parameter count we ran the same construction against SmolLM2-135M-Instruct at Q8_0 on the same RTX 4070 Laptop, with \(k{=}512\) (the natural per-head rank for \(d_\text{model}{=}576\) and \(9\) heads). The build path uses the same --axex-compress flag as the 8B run; only \(k\) differs.

The \(+44.87\%\) uplift on a model two orders of magnitude smaller than Llama-3.1-8B is a single point and does not establish a curve, but it is consistent with the cache-fit reading: the SmolLM2 attention working set already fits comfortably in L2 at baseline, so the headline saving is the avoided pointer chase through the full \(\Wmat_Q,\Wmat_K,\Wmat_V\) rather than an L2-residency transition. This distinguishes two mechanisms that the 8B measurement alone conflates, and motivates the RTX 4090 / A100 / H100 cross-tier sweep listed in 10 as a way to separate the two contributions quantitatively. Logs at benchmarks/cross_hw_local_fix_20260428_192807/session_logs/local/.

A greedy-decode (\(\texttt{--temp 0}\)) repeat of the same configuration exhibits early end-of-sequence on both the baseline and the --axex-compress path; the behaviour reproduces without GRC and is therefore a SmolLM2 sampling-mode property, not a runtime regression. The --temp 0.7 numbers above are the apples-to-apples comparison.

Beyond the headline 4070-Laptop numbers we rebuilt the binary on a second physical machine (Arch Linux 6.19, RTX 3050 6 GB, sm_86, zig 0.14, CUDA 13) using scripts/campaign/build_remote_arch_cuda.sh and reproduced the post-PCA decode pipeline. The 3050 cannot be compared apples-to-apples on Q4_K_M Llama-3.1-8B because the supervisor-bounded cold-cache PCA at \(k{=}1024\) exceeds the test window (\(\sim\)96 min for 32 layers); we instead validated correctness with \(k{=}256/512\) and Q8_0, which overflows the 6 GB VRAM at baseline (2.7 tok/s) and recovers on-GPU residency once the wproj cache lands.

This is a portability and reproducibility check, not a cross-tier GPU sweep, a 4090/A100/H100 evaluation remains future work and is the single largest open item before this paper is submission-ready. Cross-device run logs and binaries are archived under benchmarks/cross_hw_local_fix_20260428_192807/.

Validation-gate summary

The whitepaper-pack harness emits seven automated gates evaluated by scripts/validation_cycle.ps1; all seven pass on the reference pack.

Practical implication

A deployer can select the compression rank as a latency, quality dial:

• \(k{=}1024\): maximum throughput (\(+6\%\)), with a structural perplexity collapse on this model because GQA already pins \(\Wmat_K,\Wmat_V\) at rank \(\leq 1024\).

• \(k{=}1536\): near-lossless throughput (\(-2.45\%\)), with a measurable but deployment-acceptable quality cost (\(+13.30\%\) perplexity).

• \(k{=}2048\) requested, silently capped to \(1536\) by the runtime guard AXEX_MANIFOLD_K_MAX: a warmup proxy at parity with the \(k{=}1536\) row.

The overall reading is that attention compression with GRC is no longer a strict trade-off. It is a tunable parameter where one operating point exceeds baseline and another is statistically indistinguishable from it.