Geodesic Projection: per-layer rank, MCR allocation, and the depth-sink shortcut

Project nomenclature

We use a four-level naming hierarchy across this paper series. HyperTensor is the project (compiler, runtime, kernels, theory). Geodesic Projection, the subject of this paper, is the inference-time pipeline that factorises every compressible weight along its dominant geometric directions. Geodesic Runtime Compression (GRC) is the attention-only subset of GP and is the construction measured by the companion paper. OTT and GTC are the Riemannian-manifold theoretical framework that motivates GP and are treated separately. The set inclusion is GRC \(\subset\) GP \(\subset\) HyperTensor, with OTT and GTC as the theoretical superstructure.

Three axes of extension

The companion paper compresses three slots (\(Q,K,V\)) at a single uniform rank with no calibration data. GP generalises along three axes, summarised below.

Scope of this revision

This revision does not provide multi-model end-to-end PPL or throughput sweeps. The static-GP measurement is single-model, single-hardware, and is reproduced from the companion paper. Llama-3.1-70B numbers are not included; compute is approved and the run is queued. We use the term design-validated for code paths that exist, run without error, and are consistent with their components, but for which no end-to-end benchmark has produced numbers in this revision.

Status of the four adaptive mechanisms

This revision ships four adaptive mechanisms. Of those, exactly one (MCR, 4) is the measurement target of this paper. Phase-aware per-layer rank allocation has a complete implementation, a documented calibration recipe, and an executed A/B against the shared-rank baseline of the companion paper. The result of that A/B is a measured null at \(k_\text{target}{=}1024\) on Llama-3.1-8B. In a clean repeat with no co-resident workloads, MCR matches the shared-rank baseline within run-to-run noise (\(-0.13\%\), \(1{\times}\) variance). An earlier contaminated pass spuriously suggested \(-2.2\%\) with \(21{\times}\) variance; the variance ratio was the contamination signal and the measurement was retaken. The null finding at the cache-fit knee is itself the contribution. Phase-aware rebalancing neither helps nor hurts above the knee, and the open question becomes a sweep at lower base ranks where the knee is closer.

The remaining three mechanisms, the GL\((d)\) Axiom Gauge (2), thermal-aware rank (3) and the rejection-driven Oja online basis (4), are presented as v0.2 design specifications. The geometry is derived, the algorithm is given, and the runtime hooks exist, but the paired ablations are deferred to a companion v0.2 release.

Fourth measurement: Llama-3.1-8B at \(d{=}4096\).

A 2026-04-28 run of the same Phase-1 pipeline on Llama-3.1-8B-Instruct (reported in axiom_beta_report.json) keeps \(102\) PCA components at the same 95% variance threshold, so \(k_\text{int}/d = 102/4096 = 2.49\%\). This is the same order-of-magnitude regime as the smaller models and is not a per-architecture artefact. The empirical GRC operating point \(k{=}1024\) in the companion paper sits ten times above this intrinsic dimension, which is consistent with the \(13.30\%\) PPL penalty at \(k{=}1536\) being cache-fit dominated rather than information-loss dominated.

Asymmetry between attention and FFN.

Attention slots are highly compressible (\(k_{95}\approx 0.4\) to \(0.5\,d\)) and retain about \(80\%\) of energy at \(k{=}1024\). FFN slots require \(k_{95}\approx 0.8\,d\) and lose about half of Frobenius energy at \(k{=}1024\). This is the quantitative version of the qualitative "FFN is flat" observation in Geva et al. (Geva et al. 2021). We give \(W_\text{down}\) a dedicated SVD path rather than the Gram-eigendecomposition route used for the other six slots, because the construction is mathematically equivalent in the rank-truncated limit but numerically more stable when the spectrum is approximately flat (Golub and Van Loan 2013; Eckart and Young 1936).

Geometric reading of the asymmetry.

Attention and FFN solve different problems. The attention block is a routing circuit: \(\Wmat_Q,\Wmat_K\) project tokens into a similarity space whose discriminative directions are concentrated in a few hundred coordinates (head subspaces plus the residual stream's preferred basis), and \(\Wmat_V,\Wmat_O\) assemble a context-conditional message in that low-rank space. Routing is structurally low-rank because the number of things being routed is small relative to \(d\). The FFN, by contrast, behaves as a key-value memory bank (Geva et al. 2021), with each FFN column responding to a localised input pattern, so a high-quality FFN must span a high-dimensional set of keys. Removing FFN directions removes individual memories, while removing attention directions removes routing fidelity that is partially recoverable from the residual stream. The predicted consequences are the roughly \(3.4\times\) rank gap between FFN and attention \(k_{95}/d\) and the runtime observation that aggressive FFN low-rank truncation is unstable at 8B scale while attention truncation remains benign down to \(k{=}1024\). GP therefore treats \(W_\text{up}\) and \(W_\text{gate}\) with a high-floor strategy and isolates \(W_\text{down}\) on its own SVD path.

Constrained-budget formulation.

Let layer \(\ell\in\{1,\dots,L\}\) have spectrum \(\{\sigma_{\ell,1}\geq\sigma_{\ell,2}\geq\cdots\}\) and define the captured Frobenius energy at rank \(k_\ell\) as \[\begin{equation} \mathcal{E}_\ell(k_\ell)\;=\;\frac{\sum_{i\le k_\ell}\sigma_{\ell,i}^2}{\sum_i\sigma_{\ell,i}^2}\;\in[0,1]. \label{eq:Eell} \end{equation}\] GP allocates rank by approximately solving the budget-constrained maximisation \[\begin{equation} \max_{\{k_\ell\}}\;\sum_{\ell=1}^{L}\mathcal{E}_\ell(k_\ell) \quad\text{s.t.}\quad \sum_{\ell=1}^{L}k_\ell\le K_\text{budget},\;\; k_\text{floor}\le k_\ell\le K_\text{max}. \label{eq:mcr} \end{equation}\] Because each \(\mathcal{E}_\ell\) is concave and non-decreasing, the Lagrangian KKT conditions reduce to a water-filling rule: spend the next rank slot on whichever layer has the largest marginal energy gain \(\sigma_{\ell,k_\ell+1}^2 / \sum_i\sigma_{\ell,i}^2\). The MCR heuristic below is a closed-form proxy for this rule that uses the manifold curvature ratio in place of an expensive per-step spectrum lookup.

A naive scheme assigns the same \(k\) to every layer. GP exposes a manifold-curvature-ratio heuristic that allocates more rank to layers whose local activation geometry has higher (Ricci-style) curvature: \(k_\ell = \mathrm{clip}\bigl(\,\mathrm{round}(k_\text{target}\cdot \mathrm{MCR}_\ell/\overline{\mathrm{MCR}}),\;k_\text{floor},\;K_\text{max}\bigr)\) with \(K_\text{max}{=}1536\) (AXEX_MANIFOLD_K_MAX) and \(k_\text{floor}\approx 0.55\,d\). The floor matters more than the cap; below \(\approx 0.4\,d\) on small models we observe text-quality collapse.

MCR-driven allocation is the part of GP we are least confident about. It is enabled by default, but the companion-paper measurements all use shared rank for a clean calibration-free claim. The remainder of this section reports the matched-budget A/B and a confound caused by the weight-PCA basis path.

MCR vs. shared-rank A/B (Llama-3.1-8B, 2026-04-29).

We ran the matched-budget A/B at \(k_\text{target}{=}1024\) on Llama-3.1-8B-Instruct Q4_K_M (RTX 4070 Laptop, \(-n\,64\), deterministic, \(n{=}4\) reps + cold-cache warmup) twice. An initial pass conducted with unrelated background workloads on the same machine reported \(37.93\pm 0.05\) tok/s baseline vs. \(37.10\pm 1.05\) tok/s MCR (\(-2.2\%\), \(21\times\) higher MCR variance). A clean repeat with no co-resident processes (verified by nvidia-smi idle pre-launch and process audit) yields baseline \(39.28\pm 0.05\) tok/s vs. MCR \(39.23\pm 0.05\) tok/s, a \(-0.13\%\) delta within run-to-run noise, \(1{\times}\) variance. The variance gap in the first run was the contamination signal; both means rose by \(\sim 1.4\) tok/s once the machine was quiet, confirming \(\sim 3.5\%\) throughput was being eaten by the unrelated workload. We therefore report MCR at this rank as statistically indistinguishable from the shared-rank baseline, not worse. The interpretive question remains: at \(k{=}1024\) the shared-rank already sits above the cache-fit knee, so per-layer rebalancing has little room to act, the right test is a sweep across lower base ranks \(k{\in}\{256, 512, 768\}\) where the knee is closer. Raw CSVs at docs/figures/paper-b/mcr_ablation.csv (clean) and mcr_ablation_run1_contaminated.csv (preserved for audit).

Weight-PCA flat-fallback finding (2026-04-29).

A second measurement reveals an interaction between MCR and the activation-free --axex-compress basis (weight-PCA mode, default since v0.6.0). MCR computes the layer-wise curvature ratio from the activation phase tags produced during the calibration forward pass; in weight-PCA mode no calibration pass is performed, so the activation variance vector is identically zero. The runtime correctly detects this ([MCR] Flat activation-variance profile, uniform rank recommended, phases_valid=0/32) and falls back to a uniform \(k_\ell{=}K_\text{max}\) allocation, which is identical to the shared-rank baseline. The implication is that MCR's per-layer rebalancing is only active when paired with --axex-actaware-pca; pure weight-PCA users get a uniform allocation regardless of whether --axex-mcr is set. We disclose this explicitly because it confounds any naive comparison between MCR and baseline runs that share the weight-PCA basis: in that pairing the A/B is a no-op by construction, not a null result. Run log: docs/figures/paper-b/v02_empirical/mcr_8b_ppl.log.

Sink-Basis Bypass

StreamingLLM (Xiao et al. 2023) preserves a handful of sink tokens in the KV cache because the first few positions hoard disproportionate attention mass and L2 norm. To our knowledge, no prior work preserves the corresponding sink direction in the weight/activation PCA basis. GP does.

Let \(\hat s\in\mathbb{R}^d\) be the unit vector of the dominant residual-stream direction at the sink position (we estimate it as the top eigenvector of \(\mathbb{E}[h_0 h_0^\top]\) over a calibration-free 32-token zero-prompt forward). Given the per-layer projector \(V_{r}^{(\ell)}\) produced by MCR, define the sink coverage \[\begin{equation} \eta_\ell \;=\; \max_{1\le k\le r}\;\bigl|\langle\hat s,\,V_{r}^{(\ell)}[:,k]\rangle\bigr|. \end{equation}\] Whenever \(\eta_\ell < \varepsilon_{\mathrm{sink}}\) (default \(\varepsilon_{\mathrm{sink}}=0.15\)) the runtime appends \(\hat s\) as an extra column and re-orthonormalises: \[\begin{equation} V_{r+1}^{(\ell)} \;\leftarrow\; \mathrm{QR}\!\bigl([\,V_{r}^{(\ell)}\;\;\hat s\,]\bigr). \end{equation}\] This adds at most one rank per layer and bounds the sink-step reconstruction error by \(\|h_0\|\sin\theta(\hat s, V_{r+1}^{(\ell)})\), which is zero by construction. Empirically the bypass fires on \(3{-}7\%\) of layers and eliminates the rare "first-token spike" in perplexity that shared-rank GP otherwise exhibits at very low \(k\).

The Two-Thirds Dimensionality Saturation Rule

Reloading the full geometry cache on a 70B model is not free, but we only need to verify that the cache is not stale. The runtime defines the depth-sink as the single layer where the cumulative explained-variance curve of the residual stream first flattens to within \(10^{-3}\) of its plateau. Across the four open transformers we have inspected, this layer sits near depth \(\ell^\star\approx 2L/3\):

We call this empirical observation the Two-Thirds Dimensionality Saturation Rule. The name is ours and the observation is, to our knowledge, first reported here. The rule is exploited as an \(O(1)\) proxy for the otherwise \(O(L)\) cache-integrity check: the integrity probe reads only the depth-sink layer's spectrum and recomputes it on the fly (under \(200\) ms on Llama-3.1-8B against about \(70\) s for the full sweep).

The data motivating the rule are these four spectra. In each model the cumulative explained variance of the residual stream rises monotonically from layer \(0\), accelerates through the early Mix phase, and settles into a plateau where successive layers add less than \(10^{-3}\) of total energy. The first layer at the plateau is the depth-sink. Across the four architectures the ratio \(\ell^\star/L\) is \(\{21/32,\,19/26,\,22/32,\,22/32\}=\{0.656,\,0.731,\,0.688,\,0.688\}\), which clusters near \(2/3\) with a spread of about \(0.04\) either side. We interpret the regularity as a property of the residual-stream geometry: the early Mix layers populate the basis, the middle Compress layers add nearly orthogonal directions, and once the basis saturates the later Refine layers reuse existing directions rather than generating new ones. The depth-sink is therefore the first layer at which the residual stream stops growing dimensionally, which is also the earliest layer whose spectrum is sensitive to corruption of any upstream basis. That makes its hash a sufficient probe for cache staleness.

The rule is a property of these four models, not a theorem. We expect it to hold approximately on architectures whose residual stream is rank-saturating, and to break on mixture-of-experts and on extremely shallow stacks where saturation never completes. The 70B Llama-3.1 run queued for v0.2 will add a fifth point. If the ratio holds within \(\pm 0.05\) across that addition we will keep the rule as a default setting and expose the depth-sink layer index as a tunable for out-of-distribution architectures. If it fails, the integrity probe falls back to a full-sweep check, costing the seventy seconds.

Attention-sink bypass.

Sink tokens (BOS in particular) carry residual-stream norms several \(\sigma\) above the mean. StreamingLLM (Xiao et al. 2023) preserves sink tokens in the KV cache; we preserve the sink direction in the basis itself. When \(\max_k\inner{\hat s}{v_k}\) is small for the dominant sink direction \(\hat s\), we append \(\hat s\) to \(V_r\) before initialisation, costing one rank slot per affected layer.

Predicted gain.

At fixed total budget the predicted PPL improvement scales with \(\sigma^2_\text{Mix}/\sigma^2_\text{Compress}\), typically \(\sim 5\times\) in published variance profiles. Whether the prediction holds is the v0.2 measurement.

Construction.

Restrict to diagonal \(G=\diag(g)\), \(g\in\R^d_{>0}\), parameterise \(\lambda=\log g\) to keep \(g>0\), and minimise the joint tail energy \[\mathcal{L}(g)=\!\!\sum_{\ell,W\in\text{reads}_\ell}\!\!\mathrm{tail}_r\!\bigl(W\diag(g^{-1})\bigr) +\!\!\sum_{\ell,W\in\text{writes}_\ell}\!\!\mathrm{tail}_r\!\bigl(\diag(g)\,W\bigr),\] where \(\mathrm{tail}_r(M)=\norm{M}_F^2-\norm{\mathrm{trunc}_r(M)}_F^2\). The gradient in log-space is sparse and analytic: \[\partial_{\lambda_i}\mathrm{tail}_r(W\diag(e^{-\lambda}))=-2\norm{X_\text{tail}[:,i]}^2,\quad \partial_{\lambda_i}\mathrm{tail}_r(\diag(e^{\lambda})W)=+2\norm{X_\text{tail}[i,:]}^2.\] Both follow from \(\norm{M}_F^2=\trace(M^\top M)\). 10--30 outer iterations suffice; we normalise so \(\prod_i g_i^{1/d}=1\) to remove the global scale.

Bakeable, zero-overhead inference.

Once \(g^\star\) is found it is baked into the compressed factors before GPU upload: stored \(\widetilde V_t[k][i]=S_kV_{k,i}\,g_i\) (read), \(\widetilde U[i][k]=U_{i,k}/g_i\) (write). The inference path, the two-GEMV \(\mathrm{tmp}=V_r^\top x\), \(\mathrm{out}=U_r\mathrm{tmp}\), is unchanged. This is the cleanest possible deployment: a free pre-compute step at calibration time, no runtime branch, no extra memory traffic.

Predicted gain.

For models with LayerNorm-induced uniform feature scales the gain is modest (\(\lesssim 5\%\) tail energy); for models with strong per-feature scale asymmetry, \(\geq 20\%\) is plausible.

Linear interpolation rule.

With temperature thresholds \(T_\text{low}{=}65\,^\circ\mathrm{C}\), \(T_\text{high}{=}85\,^\circ\mathrm{C}\) and rank bounds \(r_\text{min},r_\text{max}\), \[r(T)=\mathrm{clip}\!\left(r_\text{max}-(r_\text{max}-r_\text{min})\cdot \frac{T-T_\text{low}}{T_\text{high}-T_\text{low}},\;r_\text{min},r_\text{max}\right).\] NVML is loaded dynamically; if unavailable the rank reverts to base. Polling is rate-limited to once per 500 ms. An optional power cap \(P_\text{budget}\) further reduces rank when current draw exceeds the cap. The system is closed-loop stable as long as the heat-generation rate at \(r_\text{min}\) is below the cooling capacity at \(T_\text{high}\).

Tokens-per-joule diffplan term.

The differentiable rank plan of (Robbins and Monro 1951) style minimises reconstruction error with an \(\ell_1\) rank penalty. We add an explicit energy term: with \(J=P_\text{NVML}/(\text{tok/s})\) joules/token and a slowly-estimated coefficient \(\hat c_\text{rank}\) (joules per unit-rank per token), \[\Delta_\text{TPJ}\,\nabla_\ell^{(r)}= \lambda\,\hat c_\text{rank}\,p_\ell^{(r)}\bigl(R^{(r)}-r^\text{soft}_\ell\bigr), \quad r^\text{soft}_\ell=\sum_r p_\ell^{(r)}R^{(r)}.\] The plan-level objective becomes \(\mathcal L_\text{recon}+\alpha\norm{r}_1 +\lambda\,\hat c_\text{rank}\,r^\text{soft}\), a weighted blend of accuracy, headroom, and energy.

Predicted gain.

At \(r{=}1024\) baseline tok/s drops \(\sim 40\%\) once throttling engages. Thermal Rank avoids throttle altogether at the cost of a controlled rank reduction; if rank--PPL elasticity is \(<1\%\) per rank-step in the operating regime (typical for \(r\in[768,1280]\) on Llama-3.1-8B), the trade is favourable.

Telemetry trace, sustained decode (2026-04-29).

This trace is a one-sided telemetry capture of an unmodified frozen-rank run, not a closed-loop A/B against the Thermal Rank controller. It characterises the operating envelope the controller would see; the paired controller-on vs. controller-off ablation is listed as an open measurement in [sec:limitations]. We instrumented a 4-pass back-to-back decode of Llama-3.1-8B-Instruct Q4_K_M (4 \(\times\) 384 tokens, \(T{=}0.7\), RTX 4070 Laptop) with a 1 Hz nvidia-smi probe on the channels Thermal Rank consumes (temperature.gpu, clocks.sm, power.draw) for \(\sim\) 90 s wall-clock. Of 94 samples, 86 fell in the active band (util \(\geq\) 20 %): GPU temperature reached \(T_\text{max}{=}75\,^\circ\mathrm{C}\) with \(T_{p90}{=}74\,^\circ\mathrm{C}\) and mean \(61.2\,^\circ\mathrm{C}\); SM clock held at the boost ceiling (median 2235 MHz, max 2235 MHz); power drew a mean of 66.3 W with peak 116 W. Decode throughput was 25.8 \(\to\) 28.0 tok/s across the four passes (a +8.5 % warm-up rather than throttle), giving an estimated 0.415 tokens/joule at the active-window mean. The active-window p90 of \(74\,^\circ\mathrm{C}\) enters the Thermal Rank actuation band [\(T_\text{low}{=}65,\,T_\text{high}{=}85\)] \(^\circ\mathrm{C}\) but does not saturate it, so on this hardware/load pair the controller would operate at intermediate rank rather than \(r_\text{min}\) or \(r_\text{max}\) alone, the regime in which closed-loop measurements are most informative. Raw trace: docs/figures/paper-b/v02_empirical/thermal_sustained_8b.csv and summary at thermal_summary.json.

Why Oja, not SGA or Krasulina.

Three online streaming-PCA estimators are candidates. Stochastic gradient ascent on the Rayleigh quotient (Shamir 2015) requires explicit orthogonalisation against all already-converged components, costing \(O(k^2 d)\) per update and a global synchronisation across rows. Krasulina's rule (Krasulina 1969) converges faster than Oja's on quadratic losses but needs an estimate of the running mean to subtract, which requires a second moving statistic per layer. Oja's rule needs neither: the orthogonalisation is implicit in the row-normalisation step, and the rule is correct under the centred residual that speculative rejections already provide (\(\E[h^\text{tgt}-h^\text{drft}]\!\to\!0\) on a well-trained draft model). The state per layer is exactly \(W_t\) plus a bump counter, and the per-update cost is \(O(kd)\).

Decayed Oja update.

Write the projection basis at step \(t\) as \(\Pmat_t=W_tW_t^\top\in\R^{d\times d}\), the orthogonal projector onto the row-span of \(W_t\). Oja's rule lifts this to a per-row update on \(W_t\) itself: for layer \(\ell\) with basis \(W_t\in\R^{k\times d}\) and sample \(x=h^\text{tgt}-h^\text{drft}\), \[\begin{equation} w_i^{(t+1)} \;\leftarrow\; w_i^{(t)} + \eta_t\,x\,(x^\top w_i^{(t)}), \qquad w_i^{(t+1)} \;\leftarrow\; w_i^{(t+1)}/\norm{w_i^{(t+1)}}, \quad i=1,\dots,k, \label{eq:oja} \end{equation}\] deflated row-wise so the matrix remains orthonormal. The induced map on the projector is \(\Pmat_{t+1}=\Pmat_t+\eta_t\bigl(xx^\top-\Pmat_t xx^\top\Pmat_t\bigr)+O(\eta_t^2)\), i.e. a rank-one tilt of \(\Pmat_t\) toward the residual direction \(x\) at rate \(\eta_t\). The learning-rate schedule \(\eta_t=\eta_0/\sqrt t\) satisfies the Robbins--Monro conditions \(\sum\eta_t=\infty\), \(\sum\eta_t^2<\infty\) (Robbins and Monro 1951); default \(\eta_0=0.01\). Convergence to the leading eigenvectors of the running covariance is classical (Oja 1982); what is new is the trigger, only rejection residuals are fed in, by construction the directions the basis fails to capture.

Convergence rate and bias floor.

For a stationary covariance \(\Sigma\) with eigengap \(\Delta=\lambda_k-\lambda_{k+1}\), Oja's rule with the \(\eta_t=\eta_0/\sqrt t\) schedule attains \(\E\norm{\Pmat_t-\Pmat^\star}_F^2 = O(d/(\Delta^2 t))\) after a burn-in (Allen-Zhu and Li 2017; Jain et al. 2016). For the depth-sink layer of Llama-3.1-8B at \(k{=}1024\) the eigengap on the activation covariance is \(\Delta\approx 6\times 10^{-3}\) (raw spectrum at docs/figures/paper-b/v02_empirical/depth_sink_spectrum.csv), so \(t\!\approx\!10^4\) samples reduce projector error to \(\sim 5\times 10^{-4}\), smaller than the calibration-PCA truncation error at the same rank (\(\sim 10^{-3}\)). Under the rejection-only trigger the effective sample rate is the speculative-decode rejection rate; on the reference run this is \(\approx 12\%\) of decoded tokens, so \(10^4\) updates corresponds to roughly \(80\,000\) tokens of decoded text, on the order of a single long document. Beyond the burn-in the bias floor is \(\eta_t\,\norm{\Sigma}\!\sim\!10^{-4}\) at \(\eta_0{=}0.01\), well below the truncation floor.

Interaction with rejection-sampling unbiasedness.

Speculative decoding is exactly correct under rejection sampling (Leviathan et al. 2023); rejected positions sample the same target distribution as accepted ones. The residual \(h^\text{tgt}-h^\text{drft}\) at a rejection is therefore an unbiased measurement of the direction along which the draft model's hidden state diverges from the verifier's, and the GP basis is fitted on the verifier's activations, so feeding only rejection residuals into Oja's rule is equivalent to running Oja's rule on samples of \(\Sigma_\text{drift}=\Sigma_\text{verifier}-\Sigma_\text{calibration}\). The result is a basis that tracks the calibration-to-deployment delta, not the absolute distribution; the calibration component is held by the static rows of \(W_t\) already.

Per-layer staleness.

A bump-counter onb_layer_state_t::version is incremented on every applied update. GP tracks the version it last consumed; if a layer's version moves ahead, the cached projection is recomputed lazily on the next forward. Updates apply between decode steps, queue bounded to 256, with a 4-rejection minimum gate.

Predicted behaviour and v0.2 measurement plan.

On matched calibration/test, online updates should be a no-op: rejection residuals concentrate near the existing basis and the \(xx^\top-\Pmat_t xx^\top\Pmat_t\) correction vanishes. On linearly drifting OOD prompts, the Oja-decayed estimator converges to the running covariance with bias \(\propto\eta_t\). The v0.2 measurement plan is a paired A/B on a synthetic-drift corpus: one stream of Wikipedia text (matched-distribution control) and one stream of post-2024 code mixed with non-English news (deliberate drift), each \(\sim 10^5\) decoded tokens, running frozen-basis vs. Oja-basis and reporting PPL trajectory plus the Frobenius distance \(\norm{\Pmat_t-\Pmat_0}_F\) as a function of \(t\). A visual of the drift-distance trajectory will accompany the v0.2 release; the empty trajectory plot would be premature here and is intentionally omitted.