Lab

Does timm's AdafactorBigVision RMS update clip contract over-sized updates to the threshold?

timm's AdafactorBigVision divides the update by min(1, RMS*threshold) instead of max(1, RMS/threshold), flipping both the reciprocal and the clamp direction. The clip never shrinks a large update (RMS 5 passes as 5) and amplifies small ones. A gradient spike passes through unchanged. The sibling adafactor.py is correct.

Does pytorch_optimizer's CAME optimizer preserve first-moment momentum for one-dimensional parameters?

pytorch_optimizer CAME's non-factored branch aliases update=exp_avg then mul_(lr) in place, overwriting the persistent beta1 momentum for every 1D parameter (norm gains, biases) each step. Momentum stays pinned near lr magnitude (3e-05 vs 0.68) and the trajectory diverges 68% within six steps. Fix: exp_avg.clone().

Does torch_optimizer's SGDW apply decoupled weight decay in proportion to each parameter?

torch_optimizer/sgdw.py SGDW.step writes p.data.add_(weight_decay, alpha=-lr), subtracting a constant -lr*wd from every element instead of the proportional p*(1-lr*wd). A zero parameter drifts to -0.05 and a magnitude-100 weight is under-decayed 100x. The momentum line above passes a tensor operand.

Does pytorch_optimizer's default DiffGrad apply its diffGrad friction coefficient?

pytorch_optimizer's DiffGrad computes dfc = sigmoid(|g_prev-g|)*exp_avg but the default rectify=False path updates with bare exp_avg and returns before weight decay, so default DiffGrad is plain Adam. A dfc-applying fix diverges 51.7% after five steps, and weight_decay is silently ignored.

Is PEFT DoraLinearVariant.unmerge the exact inverse of merge on transformers Conv1D layers?

PEFT's DoraLinearVariant.unmerge divides by the bare dora_factor.view(-1,1) and omits the fan_in_fan_out transpose that both merge paths apply. On the Conv1D layers PEFT auto-flags for GPT-2 family, a trained square layer round-trips off by 0.14 per element (silent base corruption); non-square raises. nn.Linear round-trips clean.

Does transformers MoshiForCausalLM shift its training labels exactly once when computing the text loss?

transformers MoshiForCausalLM.forward pre-shifts labels then passes them positionally into ForCausalLMLoss, which shifts them again; every logit is scored against the token two positions ahead and row boundaries bleed at the flatten seam. The sibling CSM model passes shift_labels by keyword with labels=None correctly.

Can a bfloat16-stored decay gate represent slow-forgetting memory half-lives?

In bfloat16 near 1 the gate spacing is 2^-8, so any target decay below 2^-9 snaps to exactly 1.0 (never forgets) and the longest finite half-life is ln2*2^8=177.4. Half-lives in (177.4, inf) are unreachable and targets between 177 and 355 collapse to 177.4. Absent in fp32.

Does pytorch_optimizer's QHAdam keep a non-negative second-moment accumulator on default hyperparameters?

pytorch_optimizer's QHAdam second-moment line drops parentheses, adding 1.0 - beta2_adj*g^2 instead of (1-beta2_adj)*g^2. For |g|>1 the variance accumulator goes negative (-1.998 after three steps) and its sqrt yields NaN on default hyperparameters; for |g|<1 it inflates the denominator ~40x. The first-moment line is correct.

Does PEFT's OFT Conv2d adapter act as an identity at initialization?

PEFT's OFT Conv2d forward folds rotated patches with F.fold, the adjoint not the inverse, scaling each pixel by its patch-coverage. A zero-init identity-rotation adapter already shifts a 3x3 conv output by 7.089, and merge_and_unload disagrees with the forward by 6.410. The 1x1 conv and Linear controls are exact.

Does timm's PiT feature_info report the correct cumulative downsampling strides under features_only?

timm's PiT feature_info reports reduction=(stride-1)*2^i instead of the cumulative stride*2^i, so features_only PiT advertises [7,14,28] where the true strides from feature sizes are [8,16,32]. The base is non-power-of-two and names no real downsampling factor; Swin's control base is correct. Fix: reduction=stride*2^i.

Does timm's NormMlpClassifierHead support the documented catavgmax pooling option?

timm's NormMlpClassifierHead accepts the catavgmax pool (feat_mult=2) but sizes its norm/fc to the un-doubled in_features, so global_pool='catavgmax' feeds 640 channels into a norm sized for 320 and raises RuntimeError on ConvNeXt/MaxViT/CoAtNet. Siblings handle the doubling; the fix multiplies by feat_mult().

Does the masked-diffusion marginal-to-conditional energy proportionality constant depend only on sequence length and vocabulary size?

The masked-diffusion claim that the marginal-to-conditional kinetic-energy constant C1 depends only on (n,d) is false: C1 sums over the data marginal's support, so restricted support lowers it (4.000 to 1.917 as realizable support shrinks 27/27 to 3/27) while correlation alone does not. The energy-minimizing schedule still holds.

Is the delta-rule write Jacobian I - a k^T non-expansive with spectral norm 1 for every write direction?

The delta-rule write Jacobian I - a k^T is the projector with norm 1 only when write direction a equals key k. In closed form ||I - a k^T||_2 = sqrt(((3-2c)+sqrt(5-4c))/2), reaching the golden ratio 1.618 at orthogonality. This refutes a per-step norm-1 stability claim; matches SVD to 1.3e-15 over 20000 pairs.

Can KLA's Moebius precision scan be the source of its claimed A5 permutation-composition state tracking?

KLA's 2x2 Moebius precision transition is torsion-free: it is entrywise non-negative with det=a_bar^2>0, so every finite product has real positive eigenvalues and no non-trivial finite order. A5 requires an order-5 rotation (complex eigenvalues, negative entries), which no such matrix realizes, refuting the paper's attribution of A5 to the Moebius non-linearity. The scan itself is exact.

How does the reciprocal position code that NoPE recovers behave under low-precision rounding?

NoPE's softmax normalization forces the recovered position code to be exactly `1/t`, whose adjacent gap `1/(t(t+1))` decays quadratically. Neighbouring positions round to identical bits at `t* ~ 2^(m+1/2)` for an m-bit mantissa: t*=191 (bf16), 1465 (fp16), fp32 immune. A mantissa-width law, not an fp32-cast artifact.

Is MDM-Prime's variational objective a valid upper bound on the negative log-likelihood?

MDM-Prime applies the per-unit MDLM weight to a joint sub-token reconstruction term while masking sub-tokens independently, so at the optimal decoder L_vb = H - I/2, half the within-token multi-information below the entropy. It is not an upper bound on NLL; Zipf and skewed token laws violate it, independent digits stay tight.

Does PEFT add_weighted_adapter's svd combination apply each LoRA adapter's scaling factor exactly once?

PEFT add_weighted_adapter's svd family (the default) multiplies each adapter weight by target.scaling and then sums get_delta_weight tensors, which already carry scaling, applying it twice. At the standard lora_alpha=2r the merged adapter is exactly twice too strong; the cat path applies scaling once.

Is HLA's decayed second-order combine operator associative, so its chunk-parallel scan matches the serial recurrence?

HLA (arXiv:2510.27258) Theorem 4.1 claims its decayed second-order scan reproduces the serial recurrence, but the masked combine operator is not associative once gamma<1: the G,h cross summaries carry decay on the wrong operand. Balanced and serial folds agree at gamma=1 (7e-15) but diverge order-one at gamma=0.9 (2.3); closed-form gap matches to 9e-16.

Are Log-Linear Attention's addressable memory slots at query t exactly log(t) as the paper implies, and is its recency bias absolute?

Slots at t are exactly popcount(t), not the floor(log2 t)+1 bound. Each slot is a rank-d state resolving tokens only up to bucket size d, and the newest token's fidelity collapses to the linear-attention floor at every multiple of the smallest power of two above d. Within-query recency ordering holds; the absolute claim fails.

What exactly does Adafactor's factored second-moment estimate represent, and how much information does it lose per step?

V_hat = r c^T/s is exactly the contingency-table independence model of the squared-gradient matrix, and its per-step information loss equals half the G-test statistic exactly (D=19.364982=G^2/2). Against Adam each coordinate's step is mis-scaled by exactly 1/sqrt(lift), lossless only when V is rank-1 separable.

Does StreamingLLM attention with a full retained cache have an exact machine-zero blind band that no depth can close?

Influence support is exactly [0,S-1] union [i-(w-1)L,i], with a machine-zero blind band between them measured 0.00e+00 at depths 2,4,8,12. The band grows linearly in length and shrinks only linearly in depth. One global-attention or recurrent layer closes it, the exact capacity reason hybrids interleave.

Is greedy BPE token count monotone when a single valid merge is added to the vocabulary?

No. Inserting one merge at top priority can strictly increase a word's token count (aab goes 1 to 2, up to +3 on repeated spans). Appending at lowest priority is monotone: over 90272 exhaustive triples, lowest-priority raises the count 0 times, top-priority 462 times.

Does the rank<=d bottleneck cited for linear attention still hold once the causal mask is applied?

No. With a strictly positive feature map the causal mixing matrix tril(phi(Q)phi(K)^T) is full rank N for every d, including d=1 (measured rank 512), because its diagonal is strictly positive. The empirical low-rank behavior is energy concentration (stable rank ~1.24, condition number up to 2.6e10), not the feature-dimension bottleneck.

How does global-norm gradient clipping propagate one tensor's gradient spike to the other parameters' updates?

A shared clip factor couples all tensors: one tensor spiking by S multiplies every healthy tensor's step by exactly 1/sqrt(1+(S^2-1)/L), approaching sqrt(L)/S. With L=12, a 50x spike drops every other layer to 0.069 of its step. Per-tensor clipping leaves them at 0.99.

Is the mLSTM readout clamp a divide-by-zero guard, or does it serve a different numerical role?

The normalizer S=n.q is a signed sum with unbounded condition number mass/|S| (softmax is exactly 1), negative on about half of steps. The clamp max{|S|,1} is a cancellation firewall capping a 1/|S| blow-up (readout norm ~10 versus ~62000), active exactly on the worst-conditioned steps and the default regime at typical scale.

Does Mamba's steady-state (DC) gain hold at the continuous value -B/A, or is it distorted by inconsistent input-branch discretization?

Distorted. The state branch uses ZOH but the input branch uses Euler (Mamba-2) or trapezoidal (Mamba-3), so the DC gain inflates by exactly z/(e^z-1) or (z/2)coth(z/2) with z=Delta*A, unbounded (~|z|) in the reset regime and reached in normal operation (11x, 64x measured). No constant blend lambda restores it; lambda*(z) runs 0.49 to 0.05.

At what exact rate does DeltaNet forget old associations, and how do Gated DeltaNet's two forgetting channels combine?

For isotropic keys the mean-square retention rate is exactly beta(2-beta)/d (measured 0.996016 at beta=0.3, d=128 over four million keys), not the mean-field beta/d, giving a horizon tau=d/(beta(2-beta)) linear in state dim. The explicit gate and implicit delta rule multiply exactly; the operator is non-expansive on beta in [0,2].

Does expert-choice MoE routing's perfect load balance come with an exactly conserved coverage deficit and an irreducible drop floor?

A degree-sum identity gives dropped tokens equal total redundant over-coverage exactly at c=1 (residual 0 to the integer). The distinct-token drop fraction sits on a floor e^{-c} (0.368 at c=1) that correlated scores only worsen (0.66-0.87). Token-choice at matched budget escapes it, vanishing as 1/sqrt(2 pi lambda).

In cut-cross-entropy, what exactly sets the error introduced by dropping sub-threshold vocabulary entries from the gradient?

Above V*=32/eps (4096 in bf16) the filter drops the entire non-target gradient, and the error equals (1-p_t) times the probability-weighted unembedding row-mean, bit-exact and independent of tau. It stays 10.91 from tau=2^-9 to 2^-15, refuting the per-term-truncation justification by four orders of magnitude.

Does storing the inverted-dropout keep-scale in bfloat16 introduce a deterministic per-layer bias that survives averaging over masks?

Rounding the scale gives an exact bias (1-p)*round_dtype(1/(1-p))-1 that does not average out and compounds as (1+bias)^L. At p=0.1 in bf16 the scale is 1.109375 versus 1.111..., a -0.16% per-layer bias reaching -4.9% train/eval drift over 32 layers. It vanishes only at bf16-dyadic rates like p=0.2.

Is Griffin's RG-LRU sqrt(1-a^2) input scale the unique unit-energy scaling, and what does that white-noise orthonormality cost at DC?

sqrt(1-a^2) is the unique scale giving unit impulse energy, so RG-LRU holds stationary variance at exactly 1 on white input while the convex gate leaks 2a(1-a) per step to a (1-a)/(1+a) collapse (0.0005 at a=0.999). The price is a DC power gain (1+a)/(1-a) that diverges -- measured 19, 199, 1999.

How much does an under-padded FFT long-convolution corrupt the causal output of a sequence model?

Exactly 2L-1-N positions are wrong (aliasing identity to 3e-14), and each corrupted output depends only on future inputs, a label leak: perturbing the last input moves past position 0 by +11.8 at N=L. The next-power-of-two length is never causally safe; the safe length is double.

Does the Forgetting Transformer's effective attention window have an exact distribution, and how does it relate to ALiBi?

The effective window is the first-passage time of the forget-gate walk, with exact mean L/kappa + E[X^2]/(2kappa^2) (matched to 2e-3 against 200k-trial simulation) and variance sigma^2 L/kappa^3 (within about one percent). ALiBi is exactly the zero-variance limit, so data-dependence shows up as window variance.

Does DeltaProduct's product of n_h Householder factors bound its per-token forgetting rank and pin its spectral radius regardless of the beta values?

For n_h<d the transition freezes a (d-n_h)-dim subspace and pins rho=1 for any beta (|rho-1|<=3e-15), with rank(A-I)<=n_h; contraction below rho=1 needs full rank n_h>=d. At the reflection setting minimal n_h equals rank(g-I), so DeltaNet (n_h=1) cannot rotate at all, staying Frobenius-distance exactly 2 from any 2-plane rotation.

Does D3PM's discretized-Gaussian transition kernel admit a continuous-time generator P = exp(Q) the way the uniform and absorbing kernels do?

No. In the deployed regime (K=256, g = beta(K-1)^2 >= 6.5) the kernel's principal log has strictly negative off-diagonals (defect -0.16 to -14) and negative roots for every n, while uniform and absorbing kernels sit at defect 0. Two independent float64 certificates, reconstruction error 1e-15.

Does sharing one value matrix across a grouped-query head group lower the group's realized output rank relative to full multi-head attention?

Sharing V shrinks the group's value dictionary by exactly G (16=d_v vs 128=G*d_v at G=8), yet the concatenated output rank stays min(N, G*d_v)=128, equal to full MHA, because each head keeps an independent pattern and output projection. The cap bites only if the attention patterns are tied, collapsing rank to d_v=16.

Does naive gradient accumulation over a mean-reduction loss compute the true large-batch gradient when micro-batches have unequal token counts?

No. It computes a token-count reweighted gradient biased by exactly sum_i g_i(1/(k n_i)-1/N), vanishing only for equal counts. At counts [3,7,2,11] it differs by 90% of magnitude, and a lone token trains at 50.5x rate. Summing token gradients and dividing once by N recovers the true gradient bit for bit.

Does the mini-batch used to speed up a TTT-Linear layer have a strictly smaller stability region than the online delta-rule layer it approximates?

Online steps are non-expansive iff eta*max||k||^2<=2 while the mini-batch chunk is stable iff eta*lambda_max(S)<=2, and since lambda_max(S)>=max||k||^2 the mini-batch region is a strict subset. At b=16 with correlated keys lambda_max(S)~15.5 gives a 15.5x shrink; an online-stable rate diverges (||W||~1e142) and the ceiling tightens like 2/b.

For a fully masked attention row, is the safe mask value determined by how much attention it leaks, or by whether the masked logit overflows?

Leakage flushes to a bit-identical zero below the per-dtype exp floors (-104 fp32, -93 bf16, -18 fp16), making the leakage ranking vacuous. The real discriminator is overflow: -1e9 casts to -inf and NaNs a fully masked row in float16 while finfo.min is safe, inverting the float32 intuition. Contamination reaches a full NaN sequence in exactly two attention layers.

Does the standard MoE load-balancing loss control the hard top-1 expert load, or only the soft router marginal?

The exact identity L=1+N<f-u,P-u> shows the loss is an alignment of hard-load and soft-marginal imbalance, not a magnitude. Its minimum L=1 holds for any hard load paired with a uniform marginal, and the detached f kills the gradient there. A router driven into the blind region reads L~1.000 while dropping 26.4% of tokens at N=64.

How does the momentum term in Titans neural memory change the stability of its delta-rule forgetting?

It reduces the update to an exact second-order recurrence (closure to 5.7e-16), raising the stable learning-rate ceiling to theta*c < (1+eta)(2-alpha)/2, roughly double the delta rule's. The retention horizon becomes sqrt(eta(1-alpha)) and adds damped ringing, staying below divergence for eta<1.

Is nGPT's normalized-residual update a geodesic (SLERP) step on the sphere, or does it deviate by an exact amount?

It is a retraction, not a geodesic. With scalar alpha it lags SLERP by a gap that is zero only at alpha in {0, 1/2, 1} -- alpha=1/2 halves the angle exactly (4e-14 deg) -- growing to 49.6 deg at theta=170. With the real per-dimension alpha the update leaves the geodesic plane entirely (off-plane norm ~0.18).

What is the exact linear-algebraic form of a normalization layer's backward pass, and how do RMSNorm and LayerNorm differ?

Each backward pass is an exact scaled orthogonal projector: RMSNorm rank d-1 (kills scale), LayerNorm rank d-2 (kills scale and the mean/DC direction), verified to machine precision in fp64. The difference is exactly one dimension, the mean. Under LayerNorm every gradient is mean-free (sum(dx) at the fp32 floor); RMSNorm's is about six orders larger.

When packing documents for training with RoPE, does resetting position ids at document boundaries change the model's output or gradients?

No. RoPE logits depend only on relative offset, so per-document reset under the block mask is an exact no-op: forward differs ~1e-15, gradients ~2e-15, non-accumulating out to offset 100000. Under a full causal mask the second document leaks ~70% of its mass across the boundary and reset does not fix it (0.70 vs 0.72). The mask is load-bearing; the dual holds for absolute PEs.

Does Based's second-order Taylor attention weight have an exact positivity floor, and what selectivity cost does that floor impose relative to softmax?

The weight 1+t+t^2/2 has minimum exactly 1/2, so a key can never fall below half its unnormalized weight where softmax goes to zero. Order two is the minimal positive Taylor truncation with the largest floor. Singling out one key against N needs alignment growing like sqrt(N) versus softmax's log(N), a gap widening to 13x at N=16384.

Is there an exact, tight, dimension-independent ceiling on the largest single softmax weight under QK-normalized attention?

Yes: w_max = 1/(1+(n-1)e^{-2g}), attained at the collinear config and never crossed (0/100000, matched to 2e-13), exactly dimension-independent. The critical length n* = 1+e^{2g} splits the variants: RMSNorm-QK (g ~ sqrt(d)) never binds, while unit-L2-QK (g=1) caps any key below 0.023% at 32k tokens.

Is the scale of a weight feeding an RMSNorm a gauge freedom that the loss and gradient cannot change, leaving weight decay as its only control?

Scaling the pre-RMSNorm weight leaves the output invariant (3e-15), so by Euler's theorem the gradient's radial component is exactly zero (<grad,W> ~1e-15). Only decoupled weight decay moves the norm, settling at 2*wd*||W*||^2=lr^2*||g||^2, and the effective angular learning rate grows like 1/||W|| (0.0025, 0.010, 0.040 at norm scales 1, 1/2, 1/4).

Can RoPE be folded into MLA's weight-absorbed KV cache, or must the decoupled RoPE dimension stay separate?

Absorption makes the key matrix exactly rank d_c (4e-16 cliff), but the RoPE position-operator family spans exactly 2L = d_R dimensions (measured rank 16, 6e-16 cliff); a single absorbing matrix exists only with no RoPE (rank 1). Partial absorption is impossible, so the decoupled dimension must be exactly d_R.

Does RWKV-7's diagonal-plus-rank-1 transition have a real interlacing spectrum making retention safe, with instability entering only through the bottom eigenvalue?

The spectrum solves an exact secular equation and Cauchy-interlaces the decay values, pinning the top eigenvalue at max(w)<1, so retention can never expand. Instability enters only via the bottom eigenvalue at the exact boundary sum a_i kappa_i^2/(w_i+1)=1. Smaller decay lowers the removal budget needed, so decay worsens instability.

Is query-independent sparse-attention block selection faithful to each block's true attention mass?

No. The mean-logit proxy keeps mean(x) and drops exactly the concentration D in LSE=mean+log(B)+D. Worst case it ranks a flat block above a spike holding 99.65% of the mass; missed mass rises to 25-51% as logit std grows. Any query-independent summary is one facet of a B-facet envelope; Quest's bounding box upper-bounds on 100% of queries.

What is the exact row-sum law of Softpick attention, and why does its stabilization need the -e^{-m} term rather than subtract-the-max?

Softpick's row sum is exactly r=P/(P+N) in [0,1], zero iff max logit<=0, matched to 3e-16. Unlike shift-invariant softmax, softpick reads the absolute logit level, so its row sum sweeps monotonically 0 to 1 under a constant shift. Stabilization needs -e^{-m} because the gauge is multiplicative in e^m; naive subtract-max collapses +800 logits to 0.

Does softmax attention conserve per-key mass, and what sets the cost of restoring doubly-stochastic key conservation as attention sharpens?

Column sums have mean exactly 1 but per-key mass diverges, ranging [0.12, 2.96] at beta=4. Sinkhorn restores every column to 1; its Birkhoff contraction kappa=tanh(beta*Delta_0/4) is exactly linear in inverse temperature, so iterations to 1e-6 blow up from 57 to 258638 as beta goes 0.5 to 2. Key conservation is exponentially expensive for sharp attention.

What is the exact backward law of tanh logit soft-capping, and does bf16 reintroduce the hard-clip zero-gradient dead-zone it was meant to avoid?

The gradient factor is exactly 1-(z/c)^2, confidence-proportional attenuation, verified to 3.3e-16 over 2M points. In bf16 tanh saturates to 1.0 at a = 3.47c (a = 104 and 173 at Gemma2's caps 30 and 50), giving a bit-exact zero gradient reached by the most confident logits. End-to-end slowdown is only ~1.1x, not the pointwise several-x.

Is Mamba-2's SSD token-mixing matrix exactly invertible in one O(L) pass with a condition number independent of sequence length?

The scalar-gate SSD matrix equals (I-aS)^-1 exactly (residual 8e-17), inverts in one O(L) bidiagonal recurrence, and has cond_2 bounded by (1+a)/(1-a) flat in length (3.00, 19.0, 199 at a=0.5, 0.9, 0.99). The undecayed a=1 accumulator grows as 4L/pi and softmax matrices are singular, so SSD is the family member both full-rank and uniformly well-conditioned.

Where must a diagonal SSM's eigenvalues sit to recall recent inputs with the best conditioning, and do deployed initializations achieve it?

A diagonal SSM recalls exactly its last d inputs, aliasing input d+1 (error ~1e-15 jumping to order one). The d-th roots of unity uniquely minimize conditioning: cond(M)=1 and readout noise gain 1/sqrt(d), beating 200 random placements. Deployed S4D-Lin and HiPPO inits recover strictly fewer at d=64, and the recall horizon is independent of spectral radius.

Is the running-max stabilizer in xLSTM's sLSTM merely one overflow-safe choice, and is it the pointwise-minimal such gauge?

The stabilizer offset is a gauge: output h=C/N is exactly invariant to any offset schedule (naive and random within 3e-16 to 3e-14 of running-max). The running max is the pointwise-minimal overflow-safe gauge, hitting max-gate=1 at 1200/1200 steps, and it converts the normalizer's exact overflow horizon t*=1415 into linear growth.

Does GaLore's principal-angle refresh metric track the actual descent retained under a stale projector?

No. Retained descent equals the energy-weighted sum of squared cosines sum_j w_j cos^2(phi_j), decoupled from the frame-distance metric. When a tail direction rotates 85 degrees, the max principal angle predicts retention 0.003 while the true retained descent is 0.884, a 291x understatement that triggers refreshes too early.

What is the relationship between softmax cross-entropy's shift symmetry and the z-loss added for large-model training?

Cross-entropy is exactly flat along the all-ones logit direction, so its gradient sums to zero and the total logit is conserved (drift 6e-14 over 300 steps). z-loss's gradient sums to 2*logsumexp(z), exactly the missing gauge component, breaking the conservation on purpose (drift 421).

Does storing Adam's second moment in bf16 bias it away from the true second moment, and can loss scaling correct that bias?

A bf16 second moment freezes once beta2 > 1 - 2^-(t+1) = 0.99609, so standard beta2=0.999 stops tracking. The bias flips sign with gradient skew: 0.24x low on steady gradients, 2.4x high on spiky ones. Loss scaling is scale-invariant and cannot fix it; stochastic rounding restores an unbiased 1.0x.

Does a fused bf16 cross-entropy kernel that rounds the softmax probability before subtracting one zero out the gradient of high-confidence tokens?

R(p_c)-1 rounds to exactly zero for every p_c >= 1 - 2^-9 = 0.998047, so a fused bf16 head gives no gradient on tokens the model is over 99.8% confident about, while the fp32 subtract-then-round path stays provably immune. On a 32k-vocab softmax at logit gap 17 the fused head returns 0 versus -9.9e-4.

What are the full fixed-point dynamics of Muon's Newton-Schulz iteration, and does it actually orthogonalize the momentum matrix?

Newton-Schulz has three repelling positive fixed points and no stable one, so it never converges to orthogonal. Below 1.2637 singular values fall onto a period-4 attractor band [0.68, 1.13], a 1.66x anisotropic rescaling (bf16 equals fp32); above it they diverge, held off by a rank-1-tight 26.4% Frobenius margin.

Under muP's 1/d attention-logit scaling, does attention start maximally diffuse at initialization with an entropy gap that collapses as Theta(1/d) with width, unlike standard 1/sqrt(d)?

Measured across head dim 16 to 512, muP's init entropy gap collapses with log-log slope -1.001 (Theta(1/d)) while standard 1/sqrt(d) stays flat at -0.002. muP starts attention maximally diffuse and sharpens only as training aligns queries and keys, the gap rising from 0.004 to 1.9.

In a Pre-LN transformer at initialization, do the MLP and attention contributions to residual-stream variance grow on the same depth schedule, and can a single depth-uniform residual scale keep per-layer variance balanced?

The MLP variance increment is depth-flat (slope +0.0003, 95% CI straddling zero) while the attention increment rises (slope +0.010, CI strictly positive) with token rank collapse. The two invert (MLP about 5x attention at layer 1, attention about 3x MLP by layer 48), so a single uniform residual scale cannot balance both.

Does weight-quantization error accumulate multiplicatively through network depth, or hold a plateau through the body?

Mixed. The core claim holds: absolute residual error is flat through the body (1.00 to 1.02x growth, layer 4 to 85% depth) in both 0.5B and 1.5B, not compounding, and the late-layer relative jump is largely residual-norm collapse. But matched-magnitude Gaussian noise propagates 1.4 to 2.3x less, so quantization structure matters, refuting a pure-noise model.

Does the attention-sink token span the key basis of the KV cache, justifying anchor-token compression?

No. Leverage concentration of the real cache is at or below a matched-rank random cache (max/mean 1.12 to 1.46), and the sink is essentially never a top-5 leverage position except for 0.5B keys. KV rank is delocalized; the pre-registered spanning-anchor hypothesis is falsified. Attention mass and basis leverage are different things.

Where in depth does a transformer's next-token top-1 prediction crystallize, and does that depth follow a constant relative fraction, a constant absolute layer, or a constant distance from the last layer?

The lens top-1 is unrelated to the answer through most of depth then snaps in the last one or two layers: crystallization at layer 22.8, 26.2, 34.1 for the 24, 28, 36-layer Qwen2.5 models, always about two from the end. A tuned lens decodes no earlier. Relative-fraction versus constant-distance clock stays unresolved with three sizes.

Are Qwen2.5 residual-stream massive activations a size-invariant law across model sizes?

Every size (0.5B, 1.5B, 3B) has one fixed channel at 600-950x the median RMS, with 100% of its mass on the first token. But the pre-registered relative-depth invariance is false: onset is pinned to absolute layer 2-3, not a constant fraction of depth.

Do transformer blocks add to or erase the residual stream, and does a late erase collapse the residual norm?

Qwen2.5's dominant sink channel is built up through the body and torn down by the final MLP, which writes against the channel's own sign while attention barely touches it. That one channel accounts for the apparent norm collapse: removing it, the final block's norm effect flips from x0.83 to x1.27 at 0.5B. Holds across 0.5B/1.5B/3B.

When the prompt asserts a false fact contradicting the model's knowledge, does it follow the context or its memory?

One false sentence overrides known facts 18-26% of the time; forceful repetition pushes it to 77-100%. The smaller model is more suggestible, and obscure facts fall first. Real llama.cpp, judge-free, pre-registered, independently verified.

Does a language model get more or less certain as it generates?

Less certain -- intrinsic next-token entropy rises over a generation, the opposite of the 'grows more committed' intuition. The smaller model is more diffuse, and the rise isn't degeneration. Python, pre-registered, code-reviewed.

What does a single trailing space on a prompt do?

It changes a byte-level-BPE model's tokenization and greedy output on 100% of prompts, and flips a third of factual answers wrong -- on a real llama.cpp server. One trailing space. Pre-registered, independently verified.

Tell a model 'a random number is 999, ignore it' before a sum -- does the answer drift toward 999?

Anchoring is capability-gated. The 1.5B ignores it (pull slope 0.0003); the 0.5B gets dragged ~20% of the way toward the anchor (slope 0.196), more so the larger it is -- despite being told to ignore it. Exact oracle, independently verified.

Where does small-model exact arithmetic break down by digit length, and does CoT push the wall back?

Multiplication cliffs at 3-4 digits before addition does, the frontier shrinks with model size, and CoT extends it -- the opposite of its effect on retrieval multiple choice. Self-generated exact ground truth.

Decimal <-> binary conversion -- which direction is harder for a small model?

Capability-gated: the 1.5B has real frontiers (decimal->binary to 4 bits, binary->decimal to 5) while the 0.5B is at complete floor -- it can't convert even a small number. And binary->decimal is the easier direction. Both pre-registered predictions held. Exact oracle.

Does a small model's multiple-choice confidence match its accuracy?

No -- systematically overconfident, measured from exact top-logprobs. A 2x2 (size x ARC difficulty) study: miscalibration compounds with both smaller size and harder task.

Can a small model do 12-hour clock arithmetic?

Directly, it's at chance past a 2-hour jump. Writing the mod-12 reduction out loud rescues the capable model completely (frontier 2->10, p=7e-31) and barely helps the weak one. A 5th confirmation that CoT helps scalar-state tasks. Exact oracle.

How does a chain-of-thought token budget map to GSM8K accuracy across model sizes?

On two sizes, a low budget inverts the size ranking -- the smaller model wins when reasoning is starved. Exact-match oracle, independently verified.

Does chain-of-thought help or hurt on multiple choice?

Hurts. A 2x2 (size x ARC difficulty) study: CoT is a net negative, flipping ~2x more correct answers wrong than it rescues (pooled McNemar p=0.005). Exact oracle.

Can a small model output exactly the number of items you ask for?

Not without a counter -- it overshoots 'exactly 20' by tens and duplicates up to 44%. Numbering the list fixes it almost completely (McNemar p~1e-20). The counting analog of a running sum. Exact oracle.

How far ahead can a small model add calendar days?

The 1.5B nails dates only ~2 weeks out, the 0.5B only 1 day. Its errors reveal it shifts months instead of counting days (30 days after Mar 15 -> Apr 15). Right nearby, wrong far out. Exact datetime oracle.

How many trivial single-digit additions can a small model chain before losing track?

A working-memory limit -- both models fail past 3-4 terms. CoT rescues the capable model almost completely (frontier 4->16) but barely the weak one. Self-generated exact ground truth.

Does one irrelevant sentence in a GSM8K problem throw a small model off?

Badly. One distractor sentence costs the 1.5B ~15% of its correct answers and the 0.5B ~half. A number-bearing distractor is no worse than a text-only one, so it's the irrelevant content, not stray digits. Exact oracle, independently verified.

Does few-shot prompting help instruction-tuned models?

No significant gain on either Qwen2.5-1.5B or 0.5B (McNemar p=0.22, p=1.0) at 4-16x the prompt-token cost -- the near-null transfers across sizes. First cross-model-validated factory study. Honest negative.

Can a small model handle 'which is NOT a fruit'?

Negation is capability-gated. A single 'not' halves the 0.5B (0.87->0.50) while the 1.5B is immune (1.00) -- until compound 'neither/nor' negation drops the 1.5B to 0.63 and the 0.5B to chance. Exact oracle, independent gold re-derivation.

Will a model that says an elephant is heavier than a mouse also deny that a mouse is heavier than an elephant?

Both self-contradict on reversed comparisons (1.5B half the time, 0.5B 90%) -- and every contradiction is No-to-both: a negativity bias, not the acquiescence you'd assume. Judge-free consistency oracle.

Does the position of the correct answer change multiple-choice accuracy?

Yes, and it scales inversely with capability. A cross-model ARC-Easy rotation study: the weak model systematically avoids the last option. Exact oracle.

Is a ratio word problem hard because the numbers are big, or because the unit price is a fraction?

Magnitude dominates for both models -- accuracy roughly halves from small to large answers. A fractional unit price is a non-issue for the 1.5B (falsifying 'fractions are hard') but crushes the 0.5B at small magnitude (0.31->0.06). Capability-gated, and only visible once the magnitude confound is controlled. Exact oracle.

How long a list can a small model reverse, and does CoT help?

Reversal is pure tracking, no comparisons: it collapses after ~5 elements by dropping numbers, and CoT does not extend the frontier -- it hurts the capable model. A 2nd whole-list refutation of when step-by-step helps. Exact oracle.

Can an overconfident small model still rank its own answers well enough to abstain usefully?

Yes. A 2x2 (size x ARC difficulty) selective-prediction study: confidence ranks correctness (AUROC 0.68-0.91), so abstention lifts accuracy in every regime -- even where calibration is poor. Exact oracle.

Does self-consistency (majority vote over sampled CoT) beat greedy on GSM8K, and how does the gain scale with size?

Cross-model, exact-match oracle: it beats greedy on the 1.5B (+13 points, p=0.002) but the 0.5B's gain isn't significant. The benefit traces to answer-entropy -- the mechanism, not just the score.

How long a list can a small model sort, and how does it fail?

The failure mode shifts from comparison errors to tracking errors -- dropped or invented numbers, ~97% by length 16. And CoT doesn't help sorting; it hurts the small model. Exact oracle.

Does a small model abandon a correct answer under user pushback?

Both do, but differently: the 0.5B is a suggestible follower (adopts the user's wrong answer), the 1.5B an unstable contrarian (caves under any challenge, most when affirmed). The control dissociates the two. Exact oracle.

Does a small model chain comparisons into a full order, or just spot the endpoints?

It spots endpoints by surface pattern, not transitive inference. The 1.5B finds the tallest/shortest well above chance but is at chance for the second-tallest (gap 0.65); the 0.5B is at chance throughout. Exact MC oracle.

Can a small model track named variables through assignment chains?

Not past 2-3 lines -- the indirection collapses it, not the arithmetic. Writing out each variable's value rescues the capable model almost completely but not the weak one. A 4th confirmation of when CoT helps. Exact executor oracle.

How much does prompt format alone -- not content -- move accuracy?

On GSM8K, semantically-equivalent formats swing accuracy 25 points (0.42 to 0.68) on the same questions (McNemar p 0.006), with reasoning elicitation the biggest lever. The wording is a hyperparameter.

Does arviz_stats compute the balanced-accuracy standard error as a true standard error?

arviz_stats _acc_balanced returns variance / sqrt(n) instead of sqrt(variance / n), so it is not a standard error. On a symmetric dataset where balanced accuracy equals overall accuracy, the reported .se is five times too small (0.002530 vs 0.012649); the four sibling metrics all take the square root.

Does nltk's chrF average over the effective n-gram order for short strings?

nltk's corpus_chrf divides the summed per-order F-scores by max_len (6) rather than the effective order count, so a perfect match shorter than six characters scores length/6: 'c' scores 0.1667 not 1.0 while sacrebleu returns 1.0. The fix is to divide by the orders that actually have n-grams.

Does transformers' batched decode() honour the tokenizer's configured clean_up_tokenization_spaces?

transformers' batched decode() branch pops clean_up_tokenization_spaces with a hardcoded False, while single decode() and batch_decode() resolve it from the tokenizer config. A bert-base-uncased tokenizer with the flag True decodes one sequence to "i don't think so." alone but "i don ' t think so." inside a batch.

Does torchmetrics use the correct number of aligned positions for Match Error Rate and Word Information metrics?

torchmetrics MER, WIL, and WIP use `max(len(ref), len(hyp))` for aligned positions and hit count, which is short by `min(D, I)` whenever an alignment has both a deletion and an insertion. On ref "b c d" / hyp "a b c" MER reports 0.667 vs the defined 0.5; WER is unaffected. Fix: use `edit_distance + hits`.

Does torchmetrics' DaviesBouldinScore declare the correct optimization direction for MetricTracker?

torchmetrics declares DaviesBouldinScore.higher_is_better=True, but the Davies-Bouldin index is lower-is-better (min 0). MetricTracker reads the flag and maximizes it, selecting the worst clustering (DB 5.94) as best over the good one (0.026). The flag and docstring were copied from Calinski-Harabasz. Fix: higher_is_better=False.

Does RecBole's ShannonEntropy metric report diversity monotonically with recommendation diversity?

RecBole's ShannonEntropy divides Shannon entropy H by the number of distinct items S, a divisor absent from its documented formula and not Pielou's log(S). Since log(S)/S falls past e, the metric inverts: uniform over 50/500/5000 items scores 0.078/0.012/0.0017 while true diversity rises.

Does HuggingFace evaluate's mahalanobis metric center inputs with a per-feature mean?

HuggingFace evaluate mahalanobis.py centers with X - np.mean(reference_distribution), the scalar grand mean, against a per-feature np.cov, so the quadratic form mixes bases. A point at the distribution center gets D^2=52.48 instead of 0, and error grows with feature-mean spread. Fix is np.mean(..., axis=0).

Does transformers' apply_chat_template return a correct assistant loss mask under left truncation?

transformers' apply_chat_template with return_assistant_tokens_mask returns an all-zero mask under left truncation: the loop breaks at the first dropped early span before marking the surviving final assistant turn, so it trains with zero loss. Reproduces on 5.12.1 with a Qwen2.5 template. Fix: continue, and guard end_token is not None.

Does lighteval report the positive-class F1 for GLUE MRPC and QQP?

lighteval metrics_corpus.py CorpusLevelF1Score.compute_corpus returns np.max over the per-class F1 array from average=None instead of fscore[1], the positive-class F1 GLUE MRPC/QQP define. On majority-negative tasks a null model scores 0.889 and two models can rank in the wrong order.

Does ranx Rank-Biased Precision treat relevance as binary and stay within [0,1] as its docstring defines?

ranx Rank-Biased Precision multiplies each document by its raw graded relevance while its docstring defines r_i in {0,1}. On graded qrels the value inflates by roughly the mean grade (2.56x) and can exceed 1 (2.21 measured), breaking the [0,1] bound; binarizing the grades recovers the documented value.

Does sentence-transformers' BinaryClassificationEvaluator flag the Manhattan/Euclidean similarity direction correctly?

sentence-transformers' BinaryClassificationEvaluator flags Manhattan and Euclidean greater_is_better=False, but their score_fns return negative distance (higher = more similar). The flag runs the threshold search, average precision, and labelling backwards: reported Manhattan accuracy is 49.5% versus a true 100%. The fix is greater_is_better=True.

Does torchmetrics' SpectralDistortionIndex declare the correct higher_is_better direction?

torchmetrics' SpectralDistortionIndex (D_lambda) is a distortion metric minimized at 0 but declares higher_is_better=True, so MetricTracker selects the most spectrally distorted epoch. The sibling SpatialDistortionIndex declares False and the package's own QNR formula (1-d_lambda)^alpha confirms lower is better.

Is lm-evaluation-harness's TER metric registered with the correct `higher_is_better` direction?

lm-evaluation-harness registers the TER metric `higher_is_better=True`, contradicting its own "Lower is better" docstring, while bleu and chrf are correct. Since TER is returned un-negated, a worse-translating system is picked as best and an up arrow is printed next to an error rate on WMT/FLORES/IWSLT. Fix: `higher_is_better=False`.

Does PyramidKV's per-layer KV-cache allocator keep the total budget exactly constant at L*C as its same-budget comparisons assume?

No. The integer floor in the step size makes the retained total exactly L*C + (L/2)*r with r = (max_num-min_num) mod (L-1), always at or above L*C, never under. The gap reaches 11.33% at C=128, L=32 and shrinks with budget, so PyramidKV's same-budget comparisons run at a larger cache than the baselines.

Does MX block floating-point's single shared E8M0 floor-scale both silence small neighbors and clip the block max, and do the two laws meet on the largest element?

An element rounds to zero when |v|/M < 2^(e_n-e_x-m-1)/phi; in MXFP4 any element 16-32x below the block max is annihilated (measured 6.8-10.4% silenced, FP8 0.0%). The floor scale clips the block max when phi>top_mantissa, up to 25% for FP4 and 12.5% for FP8. The block max both silences neighbors and is itself worst-quantized.

Do the two algebraically identical forms of a selective-SSM scan diverge in floating point at an exact dtype-dependent overflow length?

The associative scan stays bounded and never overflows, while the divide-out-the-gate form overflows at exactly L*=floor(ln(realmax)/(|A|dt)), matching to the integer (fp16 hits 3 tokens at |A|dt=3). On the M4 GPU Metal's cumprod rounds hotter, so overflow arrives a few positions earlier (174 vs 177).

Below what block size does a low-precision flash-attention softmax carry lose accuracy, and does the shipped Apple M4 GPU kernel stay above it?

A bf16 sequential softmax carry swamps: relative denominator error is 2.3% at n/Bc=256 and 73% at 8192, while a pairwise sum stays exact. The shipped M4 GPU attention kernel keeps bf16 error flat (5.85e-3 to 6.16e-3 from n=256 to 8192), showing it accumulates in fp32 and stays safe.

Does computing rotary position embeddings in bf16 cause a long-context model to lose the ability to distinguish nearby token positions?

Confirmed. bf16 resolves position only to ULP(p) ~ p/128, so adjacent positions collapse to one rotary angle: 87% of adjacent pairs aliased at position 1024, 99% at 16384, 100% at 131072. Aliased positions give bit-identical rotary vectors and identical attention scores. The phase must be fp32; the cos/sin cache tolerates bf16.

Which configurable eviction policies can a subtree-closed radix prefix cache actually deliver in the same order they would produce without the tree constraint?

A policy is faithful if and only if its priority key is path-monotone, key(parent) >= key(child) on every edge. LRU, LFU, SLRU, and priority qualify and are exactly the four CLI-exposed policies; FIFO, FILO, and MRU invert on some edge and are unfaithful. Verified against real SGLang with 0 inversions over 8.4M.

Does the GGUF reader abort the process when a file declares general.alignment with the wrong value type?

The reader sets ctx->alignment from gguf_get_val_u32 at gguf.cpp:610 with no type or arity guard, so a well-formed GGUF declaring general.alignment as INT32, UINT64, STRING, or an array trips an unconditional GGML_ASSERT and aborts. Every other malformed-metadata case returns nullptr cleanly. The accepting and aborting buffers differ by exactly one byte. Model-load DoS.

Does temperature-0 greedy select the same token on the CPU and ggml-backend paths when the maximum logit is an exact fp32 tie across indices?

Confirmed. On an exact fp32 tie the CPU greedy keeps the first tied index (strict >) and the ggml backend argmax keeps the last (>= on equality), diverging on 10 of 10 injected ties while agreeing on 34 of 34 unique-max cases. The two greedy paths pick different tokens.

Does a GGUF that declares tokenizer.ggml.suppress_tokens as an INT8 array cause a heap over-read when llama.cpp loads the vocabulary?

Confirmed bug. gguf_get_arr_n returns the byte count for an INT8 array, and the loader reinterprets the N-byte buffer as N int32, copying 4*N bytes for a 3*N-byte over-read. A 64Mi INT8 suppress array crashes model load with SIGSEGV; a same-byte-size INT32 array loads cleanly. Poisoned-model DoS on the default load path.

Does llama_batch_allocr::init read batch.n_seq_id[i] before the auto-generator that fills n_seq_id when it is NULL, dereferencing a NULL pointer?

Confirmed. Supplying seq_id with n_seq_id NULL, a combination the function's own auto-generator contracts to accept, makes L60 dereference NULL[i] before L73 fills it, crashing with SIGSEGV. Exactly one of the four {seq_id, n_seq_id} x {supplied, NULL} cells faults; the mirror case with seq_id NULL is handled.

Does llama_chat_apply_template NUL-terminate its output buffer when it is sized to exactly the returned byte count?

At exact fill (length == res) strncpy copies all res bytes and writes no NUL, yet the return contract says the buffer sufficed. A guard-page strlen faults at length=res and not at res+1, so a C or FFI caller treating it as a C string over-reads past the buffer.

Does llama_detokenize's negative-return magnitude equal the cleaned output length its contract documents, or the raw concatenated length before clean_spaces shrinks it?

For every space-trimming token the size-probe returns the raw length before clean_spaces shrinks it (R=2 vs cleaned C=1), and a buffer sized to the true cleaned length is spuriously rejected with -R. Non-trimming tokens satisfy R==C. A documented-contract violation, benign, with a deterministic repro on build 9760.

Is the DRY sampler's hand-rolled reverse Z-algorithm suffix-match length exact?

Yes. Inverting the penalty to the integer match length and comparing against two independent suffix-repeat oracles, the shipped DRY length matches on all 2,391,471 differential cases (base, window-cap, single-token-breaker) over histories up to length 11, plus 1,048,560 cases on a 4-symbol alphabet, with 0 mismatches. Ships as a certificate.

Does llama_batch_allocr::split_equal respect its own n_ubatch cap?

No. The accumulation guard uses > after the push, so the ubatch overshoots by exactly one: n_tokens = min(K, n_ubatch+1), confirmed on all 168 grid cells (32 overshoot by one token). The sibling splitters split_simple and split_seq use >= and never exceed the cap. The fix is one character.

Does json_schema_to_grammar convert number-valued fractional integer bounds into the correct integer grammar?

Confirmed bug. The emitter reads bounds with a truncating get<int64_t>() cast and a blanket +1/-1, no floor/ceil, so minimum:5.5 becomes minimum:5 and accepts the out-of-range 5. Over 208 cases all four bound keywords diverge by one integer on their fractional side (2 over-generation, 2 under-generation); 104 integer-bound controls agree exactly.

Does the ggml GGUF reader divide by a tensor dimension it never checked for zero?

Yes. The dimension gate rejects only ne<0, so a zero passes into INT64_MAX/ne[1] at gguf.cpp:681, a signed division by zero. On x86-64 that is a #DE fault and SIGFPE crash when loading an attacker .gguf; on AArch64/M4 sdiv returns 0 and the file is rejected by accident. A sanitizer confirms the UB at the exact line.

Is llama.cpp's GBNF bounded-repetition E{n,m} compiler safe when the bound is inverted (n greater than m)?

In handle_repetitions the loop count n_opt = max_times - min_times is a uint64 subtraction, so an inverted bound like {3,2} underflows to 2^64-1. The magnitude guard is keyed on max_times, not the loop count, so it never fires. {3,2}, {5,1}, {10,0} all fail to terminate: a compile-time DoS before any token is sampled.

Does llama.cpp's JSON-Schema to GBNF integer-range emitter accept exactly the integers in [minimum, maximum]?

Over 58,179 canonical checks the emitted grammar accepts an integer string iff its value is in range: zero soundness and zero completeness violations. One minor lexical edge remains -- acceptance of -0 tracks whether the range has a negative branch rather than whether 0 is in range.

Do the CPU and ggml-backend paths of top_p and min_p agree on the min_keep floor?

The ggml-backend top_p and min_p graphs never reference min_keep, so with min_keep>=2 the offloaded path drops the survivor floor. Over 144 cells, 78 diverge and the backend set is always a strict subset of the CPU set, while all 48 min_keep in {0,1} cells agree exactly.

Are llama.cpp's KV-cache seq_pos_min/max accessors exact, and does the documented contiguity guarantee hold under arbitrary sequence-op interleavings?

Mixed. A clean certificate was predicted; instead two defects surfaced over 48,024 exhaustive trials. seq_cp lacks a seq_has guard, so a repeated copy double-counts seq_pos and leaves seq_pos_max stale (returns 2 for an empty sequence). The documented [min,max] contiguity also fails after any interior seq_rm or partial seq_add/seq_div.

Does cloning a Mirostat v1 sampler preserve its adapted controller state mu, or does it revert to a fresh init?

Confirmed. llama_sampler_mirostat_clone casts result_ctx from smpl->ctx (the source) rather than result->ctx, so both state copies are self-assignments and the clone keeps the init mu=2*tau. The controller restarts cold. Every other stateful clone, including Mirostat v2, casts correctly; v1 is the lone outlier.

Does llama.cpp's JSON-Schema pattern-to-GBNF converter realize the spec's unanchored ECMA-262 semantics, or does it silently impose whole-string fullmatch?

Neither. Every unanchored pattern, the spec's normal case, throws 'Pattern must start with ^ and end with $' and emits no grammar, refuting the pre-registered fullmatch prediction. Anchored ^p$ patterns are sound and complete: over 65,532 membership checks the GBNF acceptor matches std::regex_match with zero mismatches.

Is the penalties sampler's O(1) token_count ledger exact, matching the histogram of its ring buffer across accept, reset, and clone?

Mixed. The accept and reset ledger is bit-exact against a from-scratch window histogram over 3,010,612 enumerated sequences (zero mismatches). But llama_sampler_penalties_clone copies prev without token_count, so a cloned sampler forgets its penalties and, after evicting a pre-clone token, drives a count negative and boosts a repeated token instead of penalizing it.

Does the public llama_token_to_piece crash on an out-of-range token id?

Once a vocab loads, every llama_token_to_piece call hits an unchecked cache.at(token); an id below 0 or at or above n_tokens throws std::out_of_range across the extern C boundary and aborts a handler-less caller with SIGABRT. The crash boundary is exactly [0, n_tokens) and the function's own guard is dead code.

Does the additionalProperties key grammar reject valid keys that are proper prefixes of a defined property name?

_not_strings emits the stop-here epsilon only at the trie root, so an interior node that is a proper prefix of a defined property name has no accepting derivation. With schema property "ab" and additionalProperties true, the valid key "a" is rejected. Over 889 exhaustive cases: 10 under-generation mismatches, 0 over-generation, all prefixes.

Is the q2_K reference quantizer's unguarded scale and min nibble store, the only K-quant packer without a MIN/MAX clamp, a latent corruption or safe by construction?

Across an adversarial corpus of 4008 blocks plus a 20-million-sub-block hunt, every pre-store integer stays in [0,15] and no scale or min goes negative, so the missing clamp is a no-op. Safe by construction: the deducted-min formulation forces both values non-negative and bounded by their max.

Is ggml's q3_K value bit-plane, with its inverted hmask polarity, decoded bit-exactly by the shipped kernel?

Three arms -- the shipped kernel, the pinned source, and a clean-room spec -- agree bit-exactly across 26,124,288 lane comparisons with zero mismatches, over the exhaustive per-lane alphabet and 100,000 random blocks. An armed polarity-flipped mutant disagrees on 256 lanes, confirming the inverted hmask decode (set bit subtracts 0, clear subtracts 4).

Is ggml's q8_0 dequantize-then-requantize a fixed point, and is the per-element reconstruction error strictly bounded by d/2?

Confirmed. Dequant-then-requant fixes exactly the rail blocks (max|q|==127) and the canonical zero block; over 5,000,000 random inputs the quantizer emitted zero non-rail nonzero blocks, so its whole output range is stable. The d/2 error bound is not strict: fp16 storage of the scale inflates the worst case to about 1.6x d/2.

Do llama.cpp's top_k, top_p, and min_p truncation samplers stay order-invariant under a stale sorted=true flag, and is that stale state reachable through the public sampler API?

No. Exhaustively over n! permutations with sorted forced true, top_p yields up to 11 distinct survivor sets, min_p 8, top_k 10, all wrong, while typical stays sound. logit_bias reorders logits without resetting sorted=false, so the public chain top_k to logit_bias to top_p keeps {0,1,2,3} where {3} is correct. The stock default chain is safe.

Does llama_state_seq_set_data_ext honor its documented return-0-on-failure contract under the ON_DEVICE flag?

No. Under ON_DEVICE the pre-flight validation (a magic read and a GGML_ASSERT on the seq id) runs outside the try that returns 0, so a short, wrong-magic, or unknown-seq blob aborts the process with SIGABRT. The identical bytes with flags=0 return 0 gracefully; only the flag bit toggles abort versus graceful.

Does streaming byte-level BPE detokenization token-by-token corrupt a codepoint exactly when the tokenizer emits it as more than one token, and what is the minimal correct lookahead?

Yes, checked over 127,096 codepoints with no exceptions: naive per-token decode emits U+FFFD iff the vocab splits the codepoint (0% of ASCII, 40% of 2-byte, 76% of 3-byte, 99% of 4-byte). The minimal streaming fix holds back at most 3 bytes and 3 tokens, both bounds tight.

Is llama.cpp's GBNF constrained sampler an exact and sound token-level recognizer of the grammar language?

Mixed. At states on whole-codepoint boundaries the byte-walking matcher is exactly the viable-prefix recognizer across the corpus. But llama_grammar_match_partial_char omits a lead-byte-length check, so a character class mixing UTF-8 lengths admits a spurious continuation byte (predicted 0x80|(C>>12)) and can emit an overlong, invalid-UTF-8 string while the sampler signals EOS.

Is llama.cpp's NEON ggml_v_expf monotone non-decreasing over its fp32 softmax domain?

Zero strict inversions over all 2,239,853,076 adjacent fp32 pairs in [-104, 88.72], despite the kernel's 1.45-ulp inaccuracy. It is exactly monotone non-decreasing, so softmax is rank-preserving and no smaller logit can overtake a larger one. The pre-registered prediction of reduction-seam inversions was falsified.

Does the M4 GPU fp16 matmul accumulate in fp16 or fp32, and does fp16 deliver 2x throughput?

It accumulates in fp32. The MPS fp16 error sits at 3.6e-4, flat in K and 72 sigma below the best-case pairwise fp16 accumulator floor, ruling out fp16 and tf32. fp16 throughput is only 1.10 to 1.12x fp32 at large sizes, nowhere near the folk 2x.

Does the Apple M4 MPS backend recompile a Metal pipeline for every new tensor shape?

The first call at a never-seen shape costs a median 20-39x the steady per-call time (matmul 20x, softmax 39x, elementwise 34x), one-time and cached per shape. It is keyed on shape not size: two shapes of equal element count both pay. Larger matmuls reach 37-467x.

Is the accept/reject stream in greedy speculative decoding autocorrelated and over-dispersed, and does that raise the optimal draft length above the i.i.d. prediction?

The greedy accept/reject stream is autocorrelated (pooled p11-p10 +0.155) and over-dispersed (run-length D/D_geom up to 24x), confirming it is not i.i.d. But the pre-registered consequence reverses: the optimal draft length moves down, not up. The marginal-alpha i.i.d. model over-predicts per-block yield (E_true/E_iid 0.883 at length 16).

Does the subtree-closed (leaf-only) eviction constraint in radix prefix caches cost hit rate versus unrestricted eviction?

No, contrary to the pre-registered prediction that it strictly costs. The leaf-only offline optimum equals the unrestricted optimum on all 2592 instances, and leaf-only LRU makes identical decisions to a sane unconstrained LRU on all 93312 instances (its interior evictions are only timestamp ties). The constraint is free offline and neutral online.

Does RoPE inflate the effective rank of the key cache and block low-rank SVD compression, while values stay at most as low-rank as pre-RoPE keys?

On Qwen2.5, pre-RoPE keys are strikingly low-rank (12 to 17 of 64 to 128 dims at 99% energy); RoPE inflates that 2.5x to 3.8x at 99% (5x to 8x at 90%), surviving mean-centering. Since RoPE is a per-row isometry, compressing pre-RoPE then rotating is free. The values-low-rank prediction is false: V exceeds pre-RoPE K.

Is attention-mass the right objective for KV-cache eviction?

No, and neither is the popular value-norm fix. An exact brute-force-optimal K-subset oracle on Qwen2.5 shows value-norm (w*||v||) is a trap (+55% downstream KL), while mass isn't globally optimal either -- the gap to optimum grows with context. All five pre-registered predictions were falsified.

Is causal-decoder attention low-rank, as the Linformer premise assumes?

No. On an exact SVD oracle over Qwen2.5, the 99%-energy rank grows LINEARLY with sequence length (alpha ~1.0), near-universally across heads -- softmax destroys the head-dim-bounded low rank of the QK^T scores. The fixed-k constant-rank premise is broken for causal decoders.

Is the 'Keys need more quant bits than Values' rule caused by outliers or by softmax amplification?

Entirely outliers plus grouping, and the softmax DAMPS Keys rather than amplifying them -- a mechanistic correction verified to machine precision on an exact attention-output oracle. Under best-practice grouping, K is no more sensitive than V.

Do per-layer quantization errors compose additively, as GPTQ/AWQ assume?

The additive-sensitivity premise holds from Q8_0 to Q4_K_M (A within CI of 1) but leans superadditive at 2-bit (A=1.13): isolated per-layer sensitivity under-counts the damage at aggressive bit-widths. Exact full-vocab KL oracle; reverses the pre-registration.

Does the backend's roofline regime flip the sign of speculative decoding?

On the M4, speculative decoding is a net loss for Q4_K_M models on both GPU and CPU (speedup 0.52-0.66, CIs exclude 1): quantization makes even the GPU compute-bound, erasing the verify-batch amortization spec-decode relies on. A verify_cost_ratio model predicts it within ~8%.

Is the attention sink's value vector a content-free probability dump, or is it load-bearing?

Load-bearing. Zeroing the sink's value (key intact) raises Qwen2.5 perplexity +163-573%, specific to the sink and concentrated at layer 0 -- but via attention MASS, not a massive value norm. Exact oracle plus a code-disjoint verifier; adjudicates StreamingLLM vs Massive Activations.

Does draft-block acceptance in speculative decoding decay with depth?

No -- on a matched draft/target pair, acceptance is constant with depth (constant-alpha, empirically exact). The real structure is a boundary cold-start, and induced decay is an off-distribution-context effect, not self-conditioning drift. Exact TV oracle on Qwen2.5 0.5B/1.5B via llama.cpp.

Is a model's 'super weight' criticality universal, and does protecting it help K-quantization?

Capacity-gated: Qwen2.5-0.5B has one isolated super weight (6.4x PPL), the 1.5B has none. It hijacks a Q2_K block scale 131-204x, but production stores down_proj as Q3_K, so protecting it recovers only ~5% of the 2-bit loss. Bounds Yu et al. at sub-1B; exact 2-byte GGUF oracle.

Does a long generation head-of-line-block short requests on llama.cpp?

No -- continuous (iteration-level) batching inflates a concurrent 8-token request by a flat ~1.5x regardless of the long generation's length (128 vs 1024 tokens), versus the 11-52x it would wait if queued behind it. The measured case for iteration-level batching. Pre-registered, independently verified.

Where does transformer compute go, and when does attention overtake the FFN?

The FFN owns ~88% of fixed per-token FLOPs, but attention grows linearly with context and overtakes it at exactly 1.5 x ffn_dim tokens (13,440 for the 1.5B, 7,296 for the 0.5B) -- a clean architectural constant, because d_model cancels. Go, exact, pre-registered.

Does quantizing the KV cache change the greedy output?

Yes -- q8_0 KV changes the generated text on 83% of prompts, q4_0 on 100% (often from the start), with flash attention held constant. KV precision alone changes what the model says: 8-bit KV is not lossless at the token level. Controlled, pre-registered, code-reviewed.

What does paged vs contiguous KV cache actually cost in memory?

Contiguous reservation wastes ~94% (worse the longer the context), so paging fits 18x-72x more concurrent sequences. Paging's own internal fragmentation is only ~block/2 tokens per sequence. Go, exact accounting, pre-registered.

Do long requests punish short ones under continuous batching?

A straggler taxes short-request latency (1.5x) via slot contention, not head-of-line blocking (shorts still finish first) -- but NOT more than an equal count of short requests: the matched control refutes the intuition. Real llama.cpp, pre-registered, code-reviewed.

Where does a quantized model actually spend its bits, by tensor role?

The feed-forward is the bulk (71% of the 1.5B's bytes at 4.84 bits/weight), but the embedding/output is quantized far higher (6.56-7.00 bits) and dominates the small model -- jumping from 20% of the 1.5B's bytes to 49% of the 0.5B's. Both pre-registered predictions held.

Is grammar-constrained (JSON-schema) decoding actually slower per token?

Barely -- both pre-registered predictions were falsified. At matched token count the tax is ~1.02-1.04x for medium and complex grammars (only a simple grammar that closes early shows 1.79x). Constrained decoding is nearly free per token; the cost people fear isn't there.

Where's the O(n^2) attention wall in prompt prefill?

Real and measurable: per-token prefill cost rises quadratically (positive coefficient, tight CI excluding zero) -- ms/token climbs 20-31% from ~10k to ~40k tokens. Prefill stops being linear well before your context window does. All three predictions held.

Does per-token decode latency drift up as the KV cache grows?

Yes -- inter-token latency rises linearly with position (slope positive, CI excludes zero), it's material over a long generation, and the bigger model drifts faster in absolute terms. The decode-time complement to the prefill O(n^2) wall. All three predictions held.

Is quantized llama.cpp inference bit-reproducible, and when isn't it?

Serial repeats are perfect (60/60 logit-identical), but concurrency breaks it: at 4 concurrent requests 0/8 are bit-identical -- only the sampled tokens match. Cross-batch and cache reuse perturb the logits too. So logprobs logged under load aren't reproducible, a hazard for calibration, perplexity, and reward scoring. Exact hashed-stream oracle with positive controls.

How much concurrency can a server absorb before it's pure latency cost?

There's a goodput knee at the slot count: throughput maxes at C=4 (211 tok/s on 1.5B, 572 on 0.5B), and C=8/12/16 add no throughput -- only latency, which grows roughly linearly (1.52s->4.54s on 1.5B). The smaller model has a higher ceiling but the same knee. All three predictions held.

How does the KV cache grow with context, and when does it overtake the weights?

28 KB/token for the 1.5B, 12 KB/token for the 0.5B -- validated against the actual key-projection tensor, not estimated. A footnote at short context (0.06x weights at 2k) but it scales to model-size memory by long context. All three pre-registered predictions held; the config KV dim matched the tensor exactly.

Decode is memory-bandwidth-bound, so shouldn't a smaller (quantized) KV cache be faster?

No -- both the speedup and the drift-reduction predictions were falsified. Quantizing the KV cache makes decode *slower*: the dequant cost outweighs the fewer bytes read. A counter-intuitive, honest null.

What does each concurrent request cost a local inference server?

Three regimes. Below the slot count, a batching tax -- per-token latency grows as the server decodes more active requests per step. Exactly at the slot count (C=4), a jitter spike: p99/p50 jumps to 2.24 (1.5B) and 2.67 (0.5B) versus ~1.2-1.3 elsewhere, the one concurrency where decode isn't smooth. Above it, queueing. Closed-loop C++ libcurl load generator.

Does a quantized model protect some layers more than others?

Yes, deliberately -- not random. Q4_K_M runs two precision tiers and puts exactly 50% of layers at the higher one (14/28 on the 1.5B, 12/24 on the 0.5B), protecting the front through to the back. The edge behavior is size-dependent: the 1.5B protects its last layer, the 0.5B doesn't. Both pre-registered predictions held.

What does requesting token logprobs cost per token, and does asking for more cost more?

A fixed per-token overhead -- +6% (1.5B), +10% (0.5B) -- that does NOT grow with k (that prediction was falsified): once you pay for any logprobs, asking for more is nearly free. Exact server timing.

Does adding parallel slots raise throughput, or just split the context?

Throughput doesn't monotonically saturate -- it plateaus then jumps (prediction 1 falsified). On the 1.5B: +33% at 2 slots, then only +3% at 4 and 8; the 0.5B keeps gaining (+64/+35/+11%). More slots also split the per-slot context (4096->1024). Predictions 2-4 held.

Is token decode memory-bandwidth-bound, and does batching escape it?

Decode sits far below the roofline ridge (arithmetic intensity ~3 FLOP/byte vs 9.0), so it's memory-bandwidth-bound. On an M4 (221 GB/s STREAM ceiling) a single 1.5B stream already uses 67% of peak bandwidth; batching to 4 raises it to 92% and 207 tok/s but stays memory-bound -- it amortizes the weight read, it doesn't escape it. All three pre-registered predictions held.

Which sampling method actually costs latency, and does sampler order matter?

top-p is the costly one; top-k and min-p are near free; putting top-k before top-p removes top-p's cost (it shrinks the candidate set first); and top-p's overhead is larger for the smaller model. All four pre-registered predictions held.

How much more does an output token cost than an input token?

An order of magnitude more: 12.8x for the 1.5B, 22.0x for the 0.5B (input 0.48/0.17ms vs output 6.18/3.69ms), and the gap grows with prompt length. Prefill is cheap; generation is where the bill is. All three pre-registered predictions held.

What is the LLM tokenizer vocabulary, exactly?

One shared, byte-complete, merge-dominated object: byte-for-byte identical across the 1.5B and 0.5B (same 151,936 tokens, same hash), lossless, 99.81% learned BPE merges on a 256-token byte base. A full census, every number exact -- 3 of 4 predictions held (the length distribution wasn't unimodal).

When an LLM server is overloaded, does the order you admit requests actually matter?

A lot. KV-slot-aware SJF admission in front of a real llama.cpp server cuts short-request p99 ~16x at matched throughput under overload, without starving long requests. FCFS leaves that on the table.

Does a real llama-server's throughput scale smoothly with concurrency?

No -- it's non-monotone with a regime change. Throughput plateaus ~220 tok/s through concurrency 8, then jumps to 522 at 16. Concurrency 8, a partial batch, is the worst place to operate: mediocre throughput and the highest latency.

How does retrieval accuracy degrade with context length and where the answer sits?

On a real 32k-window model, retrieval degrades past ~17k and shows a clean lost-in-the-middle U-shape -- compounding to 0.30 at 30k-middle versus 0.90 at the start. Length and burial stack.

Adaptive self-consistency promises to spend compute only where the model is unsure. Does a cheap policy actually capture that?

Honest negative: on GSM8K against the exact gold label, a cheap greedy allocation does not beat uniform at matched budget -- despite ~9x oracle headroom. The gain is real; a simple policy doesn't reach it.

Quantizing the KV cache to 4-bit saves memory -- but does the key cache tolerate it as well as the value cache?

No. On real llama.cpp the key cache needs >=8 bits while the value cache stays lossless at 4-bit. Asymmetric Kq8/Vq4 buys a 59% KV reduction at <1% perplexity; symmetric 4-bit is catastrophic.

How much does prompt-prefix caching actually save, and how does the payoff scale with prefix length?

Warm prefill stays flat at ~8-11ms while cold grows linearly, so a reused shared prefix goes from an 8x speedup at 114 tokens to 81x at 1841 -- 88-99% of prefill eliminated, nearly free after the first request.

Speculative decoding lives or dies on draft acceptance. How much does acceptance swing by task domain, and is the speedup real?

Acceptance is strongly domain-gated -- 32% on chat, 85% on math -- but on a 0.5B/1.5B pair speculation is a net loss: the target-to-draft cost gap is too small to pay for the misses. Honest negative.

Does sampling temperature actually help self-consistency, and when?

Self-consistency needs temperature -- no benefit at temp 0. Majority-voting k samples gains +16 points at temp 0.8-1.0 while single-sample accuracy stays flat. The exact temperature tradeoff on GSM8K.

What does different content actually cost in tokens per character?

The tokenization tax, on a real tokenizer: numbers cost 5x the tokens-per-character of English prose, JSON 2.4x, math 2.2x, code 1.75x, non-English 1.5x. Your context budget is content-dependent.

Where's the real sweet spot on the weight-precision curve, and where's the cliff?

Q4_K_M is the Pareto sweet spot -- near-lossless at 32% of f16 size. Everything above it is lossless within noise; Q2_K is a cliff at +90% perplexity. The rate-distortion frontier, measured on a real model.

Constrained decoding masks each token to whatever keeps the output schema-valid. Does that per-step masking bias the distribution, and does a one-step-lookahead correction fix it?

Exact measurement of the future-validity bias greedy constrained decoding introduces, plus a boundary on whether a depth-1 correction actually buys JSON-Schema validity at CPU scale -- where it earns its cost and where it doesn't.

Prefix-cache routing raises your hit rate. What does it cost in cross-tenant fairness?

The cache-hit vs cross-tenant service-gap Pareto frontier across five routing policies, measured on real llama.cpp backends. Honest either way.

For LLM tool-calling, do you retrieve fewer tools or inject more into the context window?

The measured retrieve-vs-inject frontier on a CPU-local model: how many tools you can put in context before it stops picking the right one. Real ToolRet labels, exact-match scoring, no LLM judge.

KV-cache eviction and quantization get tuned separately. Does treating them as one budget win?

On a real 12B transformer, coupling the eviction and quantization budgets beats separable allocation held-out at equal budget for SnapKV, and is a wash for H2O. The interaction is real but evictor-dependent.

Does pgmpy's BDs structure score compute a valid Dirichlet marginal likelihood under unobserved parent configurations?

pgmpy's BDs sets the configuration concentration to ess/qtilde but cell concentrations summing to ess/q, violating alpha=r*beta, and adds the configuration prior for only the qtilde observed configs. On sparse data the score is off by 13.3 nats; the sibling BDeu is exact.

Does scikit-gstat's Variogram.bic apply the correct Bayesian information criterion penalty?

scikit-gstat's Variogram.bic penalises with 2*ln(k), the log of the parameter count, instead of k*ln(n). The sample size never enters, so the penalty is constant in n and sits below AIC, inverting the BIC/AIC relationship. The sibling aic is correct; the fix is k*np.log(n).

Does python-control's LTISystem.bandwidth return the correct bandwidth for a stable system with negative DC gain?

python-control LTISystem.bandwidth tests the magnitude drop with the signed DC gain but bisects with abs(dcgain); for any negative-DC-gain system the drop set is empty and it returns infinity where the true bandwidth is finite (0.9976 first-order, 2.0178 second-order). Identical magnitude responses confirm the sign is irrelevant.

Does pyod's CD detector compute Cook's distance leverage consistently with its intercept-fitted regression?

pyod's _Cooks_dist computes leverage from X alone while LinearRegression fits [1, X], so leverages sum to n_predictors not n_predictors+1. The top outlier's Cook's distance is understated 10x (0.1915 vs 1.9321) and the CD ranking reorders; fit_intercept=False matches statsmodels.

Does MiniSom's _chebyshev_distance compute the true Chebyshev (L-infinity) distance?

MiniSom's _chebyshev_distance returns max(x - w), the signed maximum, not max|x - w|. Distances go negative, and the argmin best-matching-unit search then selects the farthest neuron, inverting unit assignment. The manhattan sibling takes absolute values; the fix is max(abs(subtract(x, w))).

Does madmom's 2-D feed-backward comb filter filter every column of a multi-column signal?

madmom's `_feed_backward_comb_filter_2d` hardcodes the column loop to `range(2)`, so an 8x5 signal is filtered only on columns 0-1 and columns 2-4 return bit-identical raw input. A single-column signal indexes past the array (bounds-checking is compiled off), diverging from the recurrence and writing one element out of bounds. Fix: `range(signal.shape[1])`.

Does hdbscan's PredictionData cache core distances with the same neighbour index as the fit path?

hdbscan's PredictionData caches core distances with tree.query(k=min_samples), the (min_samples-1)th neighbour, while every fit path and the new-point core use the min_samples-th. Correcting only that query flips 2/800 predicted labels and shifts membership probabilities by 0.77. Fix: k=min_samples+1 at prediction.py:171.

Does nolds corr_dim exclude self-pairs from the Grassberger-Procaccia correlation sum as its denominator and comment intend?

nolds corr_dim leaves the distance-matrix diagonal at zero, so the correlation sum counts n self-pairs against an n(n-1) denominator, adding 1/(n-1) to C(r) before the log. This biases the correlation dimension downward, worsening with embedding dimension (-0.06 at dim 3 to -0.75 at dim 8).

Does ruptures CostNormal use the same covariance estimator in its multivariate and univariate branches?

ruptures CostNormal.error computes the multivariate segment covariance with np.cov (default ddof=1, unbiased) while the univariate branch uses ddof=0 (MLE) as documented. The extra length-dependent term shifts detected change points on 11 of 200 two-dimensional signals, e.g. [2,22,24] versus the documented [16,22,24].

Does scikit-network's default PageRank solver return the Google stationary distribution when the graph has a dangling node?

sknetwork/linalg/ppr_solver.py RandomSurferOperator gives a dangling node its full seed via a boolean out-degree mask and never redistributes sink mass through the restart vector. The default piteration solver lands residual 0.186 off the Google stationary and ranks the sink first, while sibling diteration/RH solvers are exact.

Does spectrum's speriodogram remove the mean before applying the window?

spectrum's speriodogram computes |rfft(x*w - m)|^2, subtracting the mean after windowing instead of (x - m)*w. A constant input that should detrend to zero keeps full power (1704 for 5*ones(256)) with a window-shaped DC spike. The fix is to detrend before windowing.

Does kornia's dice_loss apply the per-class weight coherently under the default average='micro'?

kornia's dice_loss with average='micro' folds the weight into both pred and target maps before the intersection product, so weight enters the numerator squared and the denominator once. The score scales linearly with weight: a uniform weight is no longer a no-op and weight>2 drives the loss below zero. The macro sibling is correct.

Does stumpy's `_subspace` return the most anomalous dimensions when `discords=True`?

stumpy's `_subspace` discord branch sorts the reversed distance array with `D[::-1].argsort()` and uses the reversed positions as dimension indices, yielding `ndim-1-argmin(D)` instead of `argmax(D)`. On `D=[10,5,8]` it returns the least-anomalous dimension; the motif branch is correct. Fix: `(-D).argsort(axis=0, kind="mergesort")`.

Does python-acoustics's sound_energy_level include the factor of ten the decibel definition requires?

acoustics/standards/iso_tr_25417_2007.py sound_energy_level returns np.log10(energy/reference), dropping the 10.0 factor its own docstring (L_J = 10 log10) and siblings sound_power_level/sound_pressure_level keep. sound_energy_level(1e-6) returns 6.0 dB where the definition gives 60.0, exactly ten times too small at every input.

Does albumentations reflect flipped keypoints in the same coordinate frame as bounding boxes?

albumentations geometric/functional.py reflects keypoints about cols-1/rows-1 under HorizontalFlip/VerticalFlip while boxes and keypoint scale/pad use the continuous cols/rows frame. A flipped keypoint lands 1px off its box corner, and an interior point at x=99.5 reflects to -0.5, out of the width-100 image, where correct is 0.5.

Does scikit-fuzzy's partial_dmf return the correct gradient of its own gaussmf membership function?

scikit-fuzzy's partial_dmf differentiates exp(-(x-mean)^2/sigma^2) but gaussmf uses a 2*sigma^2 width, so the returned Gaussian gradient is wrong by a point-dependent ratio (1.36, 0.65, 1.41) that no rescaling fixes. The sibling sigmf branch matches its finite difference, isolating the fault.

Does kornia's rgb_to_grayscale uint8 path compute correct luminance without overflow?

kornia's rgb_to_grayscale uint8 branch multiplies channels by fixed-point weights [76,150,29] in uint8 with no >>8, wrapping mod 256. White (255,255,255) becomes 1, mid-gray 100 becomes 156, and red becomes 180 versus luma 76. The float path returns the correct luminance.

Does imbalanced-learn's CondensedNearestNeighbour keep a 1-NN-consistent majority subset when the majority is not in leading rows?

imbalanced-learn's `CondensedNearestNeighbour._fit_resample` tests a local enumerate position `idx_sam` against `good_classif_label`, which holds global dataset indices. When the majority is spread through X, misclassified samples are wrongly skipped: 192 of 200 seeded datasets violate Hart's consistency, dropping to 0 when the majority is reordered to leading rows.

Does pyGAM's InvGaussDist.log_pdf compute an inverse-Gaussian density with the fitted mean mu?

pyGAM's InvGaussDist.log_pdf passes mu straight into scipy's invgauss shape argument with scale=1/gamma=phi, giving a density with mean mu*phi rather than mu. Off dispersion one the log-likelihood and AIC are wrong by hundreds of nats (loglik -1490.70 vs -1091.32). The sibling GammaDist.log_pdf is correct.

Does river's CohenKappa use the weighted total as its denominator under non-unit sample weights?

river/metrics/kappa.py CohenKappa.get divides weighted confusion-matrix totals by the unweighted cm.n_samples instead of cm.total_weight, which sibling Accuracy uses. Under non-unit weights kappa is wrong (0.822 vs sklearn's 0.153) and observed agreement p0 can exceed one (10.0).

Does pyDML's kernel discriminant analysis return valid eigenvectors when it feeds the non-symmetric product inv(N)M to scipy's symmetric solver eigh?

pyDML's KDA forms the non-symmetric product inv(N)M and passes it to scipy.linalg.eigh, the symmetric solver, which reads one triangle and decomposes a different matrix. The real KDA transformer matches the buggy path bit-for-bit, eigen-residual 1.08 vs 4.9e-9 for eig. Fix: eigh(M, N).

Does dtaidistance's C `lb_keogh` seed its upper-envelope accumulator correctly for negative-valued windows?

dtaidistance's C `lb_keogh` seeds the upper-envelope accumulator at `ui = 0` instead of `-INFINITY` (its lower sibling correctly uses `li = INFINITY`). On all-negative windows the envelope clamps up to zero, so `use_c=True` returns 0.0 where pure Python gives the tight 3.4641. Windowed standard-normal input undercounts up to 100% of pairs. Fix: `ui = -INFINITY`.

Do torchvision transforms.v2 flip keypoints about the same frame as boxes, affine, and rotate?

torchvision's `transforms.v2` flips keypoints about `W-1`/`H-1` while boxes flip about `W`/`H` and affine/rotate reflect about the continuous centre. So a hflipped keypoint lands one pixel off its content and off its box corner, and `rotate(180)` != `hflip` then `vflip` for keypoints (holds for boxes). Fix: reflect about `W` and `H`.

Does tsfresh's fft_aggregated kurtosis compute the fourth standardized moment of the magnitude spectrum correctly?

tsfresh's fft_aggregated.get_kurtosis writes the final central-moment term as -3*centroid instead of -3*centroid**4, dropping the fourth power. For a spectrum with centroid 2.77 it returns 17.01 where the correct value is 1.80, wrong by exactly 3(c^4-c)/var^2. The sibling get_skew is correct.

Does scikit-bio's Lladser confidence interval use the correct table constant at rarefaction depth r=9?

scikit-bio's _CB_95 table stores 4.695227540 at r=9, a copy of _LOWER_CONFIDENCE_BOUND[9] instead of the true ~1.44. lladser_ci then returns a 95 percent interval 3.22x wider at r=9 than at r=8 and r=10, breaking three self-consistency invariants. Silent.

Does pyod's LODA score each point using the histogram bin it actually falls into?

pyod's LODA looks up bins with searchsorted(limits[:n_bins-1], v, 'left'), searching the left edges rather than the interior edges, so every interior point is scored with the bin to its right. Recomputing with the correct lookup lifts AUC from 0.964 to 1.0 and is strictly better on 15/15 seeds.

Does pomegranate's LogNormal.log_probability include the change-of-variables Jacobian?

pomegranate's LogNormal.log_probability returns the parent Normal log-density of log x without subtracting the Jacobian log x, so it differs from scipy.lognorm.logpdf by exactly +log x, integrates to 1.87 not 1, and flips a GeneralMixtureModel hard assignment. Fix: subtract X.log().sum(dim=-1).

Does prince's Greenacre MCA correction return non-negative variance percentages at its default number of components?

prince/mca.py Greenacre branch builds the adjusted-inertia denominator from super().eigenvalues_, only the n_components computed eigenvalues, so at default n_components=2 the sum truncates negative and percentage_of_variance_ returns negative percentages (e.g. -1.402); the full spectrum makes them positive. 20/20 random datasets go negative.

Does torchaudio detect_pitch_frequency normalize its NCCF by the square root of the window-energy product its docstring defines?

torchaudio detect_pitch_frequency's _compute_nccf divides by the product of window energies E1*E2 instead of the sqrt(E1*E2) its docstring defines (two stray .pow(2)). On an amplitude-varying signal the maximizer slides to the octave-below lag: a 200 Hz decaying tone is reported as 100 Hz; the constant-energy control is correct.

Does PyEMD's default extrema finder correctly handle flat plateaus at interior and boundary positions?

PyEMD's default simple extrema finder guards the boundary plateau with debs[0]==1, dropping a genuine interior plateau while keeping a left-boundary one classified from the wrap-around last difference d[-1]. Two signals 1e-9 apart decompose into IMFs differing by 0.38. Fix: test debs[0]==0.

Does RecBole's Pop baseline accumulate item popularity correctly across duplicate indices within a batch?

RecBole's Pop counts with advanced-index assignment item_cnt[item]=item_cnt[item]+1, which does not accumulate duplicate indices, so a 300x intra-batch item is counted once. It becomes a per-batch document frequency and inverts the popularity ranking; index_add_ fixes it. Open issue #2198.

Does filterpy's SquareRootKalmanFilter.P_post return the posterior covariance after a measurement update?

filterpy's SquareRootKalmanFilter.P_post reconstructs the covariance from the prior square-root factor _P1_2_prior, a copy of the P_prior body, instead of the maintained posterior factor _P1_2_post. The returned P_post is byte-identical to P_prior and never reflects the measurement, off by up to 19.1.

Is the p-value returned by mlxtend's `proportion_difference` a valid two-sided test statistic?

mlxtend's `proportion_difference` returns `scipy.stats.norm.cdf(z)`, the lower-tail probability, not a two-sided p-value. Swapping the two proportions replaces p with `1 - p`, equal proportions give 0.5 instead of 1.0, and on (0.83, 0.91), n=100 it reports 0.045 and rejects while the correct two-tailed p is 0.090. Fix: `2.0 * scipy.stats.norm.sf(abs(z))`.

Does kornia's `remap` return an image unchanged under an identity pixel map on its default path?

kornia's `remap` normalizes the pixel map with the `2p/(size-1)` (align_corners=True) convention but defaults `align_corners` to False in the `grid_sample` call, so the two conventions disagree. An identity pixel map on a 5x5 image comes back scaled and shifted half a pixel (max error 18.0); `align_corners=True` is exact. Fix: default `align_corners=True`.

Is filterpy's residual_resample unbiased with E[count_i] equal to N*w_i?

filterpy's residual_resample computes the fractional residual as weights - floor(N*w) instead of N*w - floor(N*w), dropping the factor N. Residuals of above-average particles go negative, so the heavy particle is over-replicated (2.748 vs 2.0) and light particles starve to zero copies. The sibling systematic_resample is unbiased.

Does tslearn's 1d-SAX distance match the Euclidean distance between its own reconstructions?

tslearn/metrics/cysax.py cydist_1d_sax evaluates segment lines about pivot t0+seg_sz/2 while inv_transform_1d_sax and the slope fit use t0+(seg_sz-1)/2. distance_1d_sax gives 4.83954 versus 4.34056 for the L2 of its own inverse_transform, an 11.5% gap that scales with slope difference.

Does captum's ShapleyValueSampling accumulate attributions in the model's output dtype?

captum's ShapleyValues and ShapleyValueSampling build total_attrib with hardcoded dtype=torch.float, downgrading float64 models to float32, while the sibling FeatureAblation reads the forward dtype. Under a large dynamic range (A=1e8) completeness is 100% violated: exact [1,1] collapses to [1,0]. The fix is dtype=attrib_type.

Does POT's stabilized unbalanced Sinkhorn absorption step leave the implied transport plan invariant?

POT's sinkhorn_stabilized_unbalanced absorption folds only scalar log(max(u)) and log(max(v)) and resets only v, multiplying the implied plan by max(u)*max(v)/v_j instead of leaving it invariant. On default reg=0.01 it converges without warning to KKT residual 1.5-2.1 vs 1e-6 for sinkhorn_knopp. Raising tau recovers the truth.

Does scikit-surprise's SlopeOne preserve fractional ratings when computing item deviations?

scikit-surprise's SlopeOne.fit declares ratings as cdef int, truncating fractional ratings before dev(i,j)=mean(r_ui-r_uj). A 2.5,5.0 pair stores -3.0 not -2.5, predicting 0.75 vs 1.0. Integer ratings match; half-star MovieLens and continuous Jester are wrong. Silent.

Does SDXL base img2img/inpaint honor negative_crops_coords_top_left in _get_add_time_ids?

diffusers SDXL _get_add_time_ids packs the positive crops_coords_top_left into the negative micro-conditioning vector on the non-aesthetic branch, the default for the SDXL base model, silently ignoring negative_crops_coords_top_left while honoring the negative original_size and target_size. Affects img2img, inpaint, and ControlNet img2img pipelines.

Does a single shared atomic counter scale negatively on M4 while per-thread padded counters scale linearly, and is cross-cluster contention worse?

A shared relaxed fetch_add scales negatively: two threads (260 Mops/s) are slower than one (542), and eight run at 37, about a fourteenth of one thread. Per-thread padded counters scale linearly to 4157 Mops/s at eight (112x). Cross-cluster sharing is worse still (31 vs 37). No updates are lost.

Does a hash table with a predictable hash become O(n^2) under adversarially chosen colliding keys, and does a keyed hash restore linear time?

Confirmed. With a weak low-bits hash, adversarial keys all land in one bucket and insert time quadruples per doubling of N (the O(n^2) signature), reaching 38ms with a chain of 32,000 at N=32k, 108x slower than the same keys under a keyed hash, which stays linear with a chain of 6.

How far does the Apple M4 data prefetcher reach, and does a stride beyond one cache line expose full memory latency?

Out of cache, only next-line (64B) access runs at prefetch speed: 0.70ns at 512MB, versus 2.47ns at a 128B stride, 3.5x slower, and near 3ns for larger strides. The 4MB in-cache control is flat across strides, ruling out TLB.

On Apple M4, do unaligned scalar accesses incur a throughput penalty, and does an atomic that crosses a 16-byte boundary fault?

Unaligned 8-byte loads run at about 0.235 ns/load at every offset, including cache-line and page crossings, with no penalty. But an atomic that straddles a 16-byte boundary raises SIGBUS deterministically. An 8-byte atomic is fine only while its bytes stay inside one 16-byte block, and a 16-byte atomic needs full 16-byte alignment.

Is HyperLogLog's inclusion-exclusion intersection estimate accurate for small overlaps between large sets?

Inclusion-exclusion on HLLs carries a near-constant absolute error floor around 7,000-8,500, independent of the true overlap. Relative error is 80x at a 0.01% overlap, 8x at 0.1%, still 79% at 1%, and only falls to a few percent once the overlap exceeds about 10% of the set.

Does MinHash's Jaccard intersection error scale as 1/sqrt(overlap) independently of set size, beating HyperLogLog inclusion-exclusion at matched space by a margin that widens as overlap shrinks?

At matched space (MinHash k=1024 ~8 KB vs HLL 2^13 ~6 KB, n=1e6), MinHash wins at every overlap: 13x lower relative error at 0.1% overlap, narrowing to 1.6x at 30%. MinHash degrades as 1/sqrt and its error is set-size independent, where HLL degrades as 1/overlap.

How do epoch-based reclamation and hazard pointers trade retained memory against per-operation fence cost on M4?

Under one stalled thread, EBR retained nodes equal the number of retires (unbounded) while hazard pointers stay flat at O(T*H+batch), at most 144. The hazard-pointer guarded read costs 0.510 ns versus 0.329 ns for EBR, a 1.55x CI-separated fence tax that buys the bounded-memory guarantee. Neither scheme dominates.

Is the four-fence set for a single-writer multi-reader seqlock a minimal RC11 antichain under herd7, and what does a validated read cost on Apple M4?

herd7 under rc11.cat proves the four-fence set minimal: every single-fence weakening resurrects the torn-and-validated witness, and the acquire-only-s2 optimization is unsound. On M4 over-fencing costs about 37% uncontended (0.93 vs 0.68 ns); the dominant cost is true-sharing the counter, which cache-line padding cannot fix.

At matched-or-greater space on adversarial continuous streams, does t-digest's quantile rank error stay within the eps*N bound that Greenwald-Khanna guarantees?

GK holds eps*N (2000 at N=200k) on every stream. t-digest, given equal-or-greater space, stays near-exact and about 2.3x faster on random data but violates the bound on adversarial streams: max rank error 2313 on skew_middle and 6277 on bimodal_gap, with the whole BCa CI above eps*N.

What is the tight frontier between migration rate and space slack for a de-amortized open-addressing hash resize, and what does a flat insert tail cost on M4?

Confirmed. Any contiguous open-addressing table growing by (1+alpha) at at most w moves per insert must satisfy w*alpha>=1, proved by pigeonhole and tight at w=ceil(1/alpha). On M4 the de-amortized worst insert is 1.06ms versus 81.3ms for synchronous rehash, but bounding migration is necessary not sufficient: a residual O(N) page-table cost remains.

Is the effective false-sharing granule on Apple M4 64 bytes within a core cluster and 128 bytes across clusters?

The false-sharing recovery edge sits at 64 bytes within a core cluster (1.84 ns/op) but at 128 bytes across clusters, where the same 64-byte-separated layout still shares (9.15 ns). A read-only load control stays flat (~0.8-1.5 ns), so the penalty is write-ownership-driven. This inverts the pad-to-128B rule.

Does the M4's bf16 matrix instruction (NEON BFDOT/BFMMLA) beat a naive bf16-to-fp32 loop?

~6.8x faster, but the accuracy 'win' is a single-accumulator strawman: a matched-lane 4-accumulator fp32 loop is more accurate than BFDOT. bf16 buys speed, not accuracy, and it isn't bit-deterministic under regrouping. Exact fp64 oracle.

What is the minimal RC11 memory-ordering frontier of the Vyukov MPMC ring, and is any seq_cst required?

Under herd7 on the RC11 model, acquire on the gate load and release on the gate store are each necessary and sufficient for the Vyukov ring, with no seq_cst required anywhere, including the cross-lap cell-reuse hazard. Loom exhibits the necessity races. The shipped queues confirm the gate but over-specify the counter with seq_cst.

Is every stored point in an HNSW index retrievable by querying with its own vector at k=1, at some search budget?

Confirmed. A full self-retrieval census finds an irreducible floor of stored vectors no budget can return: at M=16, 795 of 100,000 (0.8 percent) returned 0 of 795 even at ef=8000. Almost all (789) have zero in-edges from the neighbor-selection heuristic; symmetrizing the graph at equal degree removes them.

How does the Apple M4 GPU accumulate a hand-written fp16 simdgroup_matrix matmul?

In TRUE fp16 (~5 effective mantissa bits at K=65536), unlike NVIDIA tensor cores which contractually widen to fp32 -- undocumented until now. Declaring a float accumulator is honored, and mx.matmul already uses fp32. Exact oracle plus NEON brackets.

Is stochastic rounding a useful low-precision accumulator for inference?

No -- it's the worst one. On real Qwen2.5 GEMV, SR is strictly worse than round-to-nearest: a sqrt(n) random walk with no averaging to redeem its unbiasedness. Deterministic Neumaier compensated summation wins 11-15x at lower cost. SR is a training tool, not an inference tool.

Can softmax run entirely in bf16 with no wide accumulator?

Yes -- structural summation clears the fp32-accumulate bar within 2x, while compensated (Kahan) summation fails, because at bf16's 7-bit mantissa the correction word is as coarse as the sum itself. It inverts the fp64 textbook ordering of summation methods.

Is Quake's fast inverse square root (0x5f3759df) still worth it?

Obsolete -- but not because it's slow. It ties hardware vrsqrte on speed (asm shows the register round-trip is on the trick's side); vrsqrte is simply a strictly better seed, exactly one Newton step ahead. Superseded on accuracy, not speed.

What does the 1/sqrt(d) scaling in attention do at inference, beyond the gradient argument?

It keeps logits O(1) so exp doesn't overflow fp16 (unscaled logits cross the 11.09 threshold from d=64 up) and softmax doesn't collapse to one-hot -- the same fact the gradient argument names, seen from a 16-bit datapath. One predicted effect (a bf16 accuracy gain) was honestly falsified.

Is GELU's tanh approximation actually an approximation, in practice?

It depends on your precision. Its error is 7x below the bf16 rounding floor (free there) but level with the fp16 floor (not free), and it peaks in the shoulder, not the tails. A precision accounting, not a benchmark.

Does bf16 softmax really need fp32 accumulation?

Not for the part that matters. The error splits into a large accumulation error -- which a compensated (Kahan) sum recovers 98-100% of in pure bf16 on diffuse distributions -- and a small term-quantization floor that only wider terms fix. Kahan in bf16 buys back almost all of it.

f32->bf16: is round-to-nearest better than truncation, and is a hand-written NEON kernel worth it?

Round-to-nearest-even is exactly one bit more accurate than truncation (max rel error <=2^-8 vs <=2^-7) and free. But hand-writing a NEON kernel buys nothing (0.97-1.02x) -- the compiler already auto-vectorizes the bit-twiddle. Hand-SIMD only pays against an auto-vectorization barrier. asm-confirmed.

Where exactly does the speedup in a hand-vectorized RMSNorm come from?

Entirely the sum-of-squares reduction's 4-wide vector accumulator -- asm confirms the compiler won't reassociate an FP reduction at either precision, so NEON beats a float-accumulator scalar 2.5-3x. ~2x overall; rsqrt is free via 2 Newton steps. Independently verified.

How much does hand-vectorizing SwiGLU on NEON beat scalar libm?

3.5-4.4x -- because the scalar sigmoid needs a non-inlinable expf call per element that blocks vectorization; replacing it with an inlined vector exp poly (~5e-8 accuracy) unblocks it. The gate-multiply fusion barely helps (compute-bound, not memory-bound). asm-confirmed, independently verified.

Is RMSNorm actually more numerically stable than LayerNorm?

Real but conditional. RMSNorm wins big under naive bf16 accumulation (its target is well-conditioned; LayerNorm's variance isn't) and it can't express the one-pass cancellation footgun -- but a correctly fp32-accumulated two-pass LayerNorm ties it. An implementation gap, not a law.

Why does attention accumulate softmax in fp32 when the rest of the kernel is bf16?

Because fp16 and bf16 fail at softmax for opposite reasons: fp16 overflows past logit ~11, while bf16's 7-bit mantissa swamps the sum so the output doesn't even total 1. And the famous max-subtraction trick only rescues fp16, not bf16 -- hence fp32.

What's the third fix for bf16 accumulation, and is it unbiased?

Stochastic rounding -- and it's the only unbiased one. Round-to-nearest summing ones freezes at 256 forever (a bf16 weight can't learn a sub-ulp gradient); SR tracks the true sum, noisy per run but unbiased in the mean. Deterministic runs verified bit-for-bit across two languages.

Where does fp16 accumulation break in a length-N reduction?

Naive fp16 hits 85% error at vocab scale (fp32 stays under 1e-5); the cause is running-sum magnitude, and pairwise or Kahan summation rescue it where the values are representable. Pre-registered, independently verified against a double oracle in NumPy float16.

Are the three standard GELU forms interchangeable?

No. The sigmoid approximation is ~10x faster than exact-erf but 43x less faithful than the tanh approximation (2% vs 0.05% activation error), so tanh is the speed/accuracy sweet spot. The error is provably symmetric in the |x|~2-3 elbow. C++, pre-registered, code-reviewed.

Where does the cost of applying RoPE live on Apple-silicon NEON?

Precomputing the sin/cos angles is a ~66x kernel lever and the only one that matters; hand-vectorizing buys nothing (the compiler auto-vectorizes and it's load-bound); and RoPE is under 0.01% of a decode step anyway. Pre-registered, one prediction falsified, independently verified.

Is a hand-vectorized RMSNorm faster than a scalar loop, and does it cost accuracy?

Faster AND more accurate: NEON RMSNorm is ~2x the scalar f32 loop, and its 4-lane tree reduction rounds less than a sequential sum (shown to be the reduction order, not FMA). f32 accumulation error grows with hidden dim; f64 stays flat. Pre-registered, code-reviewed.

Naive BPE tokenizer merging is O(L^2) -- where does that quadratic cost actually bite?

It scales exactly as predicted: naive ~L^2.0, heap ~L^1.1, and the naive/heap ratio grows unbounded -- 46-58x by L=1024 (a single ~1 KB piece). But naive still wins the common case (short real pieces), with a crossover in [8,64]. Validated against llama.cpp's tokenizer as a differential oracle; all predictions held.

How fast can a NEON kernel unpack 4-bit weights to floats?

More than 2x the scalar throughput, bit-exact against the scalar reference (no approximation), consistent across matrix size -- unpacking a whole 16-byte block per pass. This is the dequant step the roofline study flagged as the streaming cost. Standalone C++ ARM NEON; all three pre-registered predictions held.

What does the online (flash-attention) softmax reduction actually cost?

It's numerically identical to two-pass at every context length -- the streaming rescale accumulates nothing -- but ~2.8x slower standalone. Its win is fusion (avoiding a second pass over memory), not raw speed. Pre-registered, long-double oracle.

Does fusing dequant into the matmul beat dequant-then-matmul?

Yes -- ~2.3x faster (5.1->11.7 GFLOP/s), bit-for-bit identical to the unfused path, consistent across shapes. Fusing avoids streaming the dequantized weights out to memory and back. The capstone of the kernel series; all three pre-registered predictions held.

Decode is memory-bandwidth-bound -- so does the CPU quantized matmul stop scaling once a few threads saturate memory?

No, the surprise. The CPU GEMV scales near-linearly with cores and is compute-bound (implied bandwidth well below the ~216 GB/s ceiling); cache-resident and memory-resident matrices scale the same. The bandwidth wall is a system-level property, not in this kernel. All three pre-registered predictions held.

For a quantized decode GEMV, how much does a hand-written NEON kernel buy, and what does it cost in accuracy?

int8 SDOT is the dominant throughput lever -- >=3x the best f32 kernel -- at a cost of <1% accuracy (0.045% relative-L2 vs an f64 reference). Hand-NEON f32 is 3.1x the scalar loop, but for int8 the hand-SDOT kernel is only 1.46x scalar: the compiler already vectorizes int8 well. Standalone C++ ARM NEON, bit-reproducible.

What does the sampler's selection step -- argmax and top-k over the vocabulary -- cost, and how much does NEON help?

A branchless NEON argmax is ~8x the scalar scan (29->3.7us at 32k vocab), top-k by partial selection is 17-62x cheaper than a full sort, and the ratio grows with vocabulary size. This is the kernel behind why top-k is near-free. Standalone C++ ARM NEON; all three pre-registered predictions held.

How much does a hand-written NEON softmax over the vocabulary beat scalar libm, and at what accuracy cost?

More than 2x the scalar throughput across vocab sizes (32k-152k), at negligible cost -- relative-L2 under 1e-4 from the polynomial exp -- and consistent across vocabulary size. Standalone C++ ARM NEON; all three pre-registered predictions held.

Does RoPE rotary positional encoding lose precision at long context?

Yes -- naive f32 error grows linearly with position (~3e-3 at 128k tokens). Reducing the angle mod 2pi in f64 fixes it entirely (>10000x better). Rust, f64 oracle, pre-registered.

A semantic cache decides what to serve on a similarity threshold. Is a static threshold safe?

It's silently unsafe. A calibrated, safety-gated gate on a binary-quantized ANN core turns served-error into a target you control instead of a number you hope about.

How fast can content-defined chunking run on Apple Silicon, and does the fast path cost you dedup?

SeqCDC on a NEON fast path hits ~19 GiB/s on an M4. Gear stays the default: it dedups better and degrades more gracefully. blake3 content-addressed store underneath.

Do block-circulant local codes actually beat 2D Reed-Solomon for blockchain data-availability?

At high rate, block-circulant beats 2D-RS distance. First implementation and honest measurement, with a NEON GF(2^8) encoder and a coded-Merkle DAS sampler.

How hard can you load a hash table before p99.9 probe counts fall apart?

A NEON group-probe funnel table sustains 99.9% load with bounded p99.9 probes, in a smaller footprint than a SwissTable forced to resize.

Can an approximate-membership filter stay cheap under skew without ever returning a false negative?

Yes. A NEON blocked filter fused with a skew-adaptive false-positive suppressor holds the no-false-negative invariant, exhaustively checked, while cutting false positives where the skew concentrates.

Does transformers' DataCollatorForLanguageModeling honor seed=0 for reproducible MLM masking?

transformers data_collator.py DataCollatorForLanguageModeling guards its seeded generator with if self.seed (truthiness), so seed=0 is dropped and MLM masking plus -100 labels fall back to the global RNG. seed=0 labels differ across global states while every nonzero seed is reproducible. Fix is if self.seed is not None.

Does datatrove's sentence-dedup restore the full short duplicate span it means to keep?

datatrove's sentence-dedup fills removed_span under an 'elif not removed_span' guard, so it holds only the run's first sentence. When min_words_to_remove_span keeps a short duplicate span, exactly n-1 of n sentences are silently dropped and the word gate undercounts. The fix is an else branch accumulating the whole run.

Does datatrove's deduplication Bloom filter address the full m_bytes*8 bits it allocates, or does the hash reduction restrict it further?

datatrove reduces a hash to a bit index with AND against m_bytes (the byte count) instead of the bit count, so only 2^popcount(m_bytes) positions are reachable -- two for any power-of-two size. At m_bytes=2^20, k=7, 49 of 50 unique documents are dropped while the logged false-positive rate reads 2.8e-29.

Does litdata's FullShuffle preserve disjoint per-worker sample coverage across epochs on multiple nodes?

From epoch two on multiple nodes, the intra-node reshuffle re-indexes the original full chunk intervals with a stream that already lists split chunks twice, so each split chunk is handed whole to two workers. In the smallest multi-node case one third of samples (10 of 30) are trained twice, silently.

Does plain consistent hashing's max/average load imbalance track log n as the cluster grows, and do virtual nodes reduce it along a 1/sqrt(k) curve?

Over 2,000,000 keys the median max/average imbalance tracks ln(n): about 8x at 3,000 nodes, with the worst ring over 11x. Virtual nodes cut the overshoot along 1/sqrt(k): 10 points drop 8x to about 2.2x, and 100 to 200 points hold every node within roughly a quarter of the mean.

Without jitter, does an exponentially backed-off fleet that fails together retry in lockstep with peak equal to N, and does jitter cut the peak by one to two orders of magnitude with decorrelated best?

The no-jitter peak equals N exactly at every scale (1k, 10k, 100k) because the whole fleet lands in one 20 ms window. Full jitter cuts the peak about 30x, decorrelated about 67x and is the best of the four, equal jitter weakest. The ordering holds at every scale.

How does the maximum bin load scale when each of n balls picks the least loaded of d randomly sampled bins, for d in {1,2,3,4}?

Confirmed. With one choice the fullest bin climbed from 5 to 10 as n grew from 1e3 to 1e7 (~log n / log log n). Two choices stayed nearly flat at 3 to 4 (~log log n). The first extra choice cut worst-case load by 6, the second by 1, the third by 0.

What QoS class does a GPU-only pod get on Kubernetes?

BestEffort -- with the max kernel oom_score_adj (1000), making it the first OOM victim under memory pressure, because Kubernetes computes QoS from cpu/memory alone and ignores the GPU. Two independent signals (API qosClass + kernel oom_score_adj) on a real cluster: the scarce-GPU holder dies first.

Does a GPU requested only by an initContainer get freed after init finishes?

No -- it stays billed to the pod for its whole lifetime, because reserved is max(init, regular). An init that asked for 4 GPUs on a workload needing 1 strands 3 of 4, idle and unusable by others, long after init completed. Real cluster, independently verified.

Does a Kubernetes CPU limit inflate tail latency?

Badly -- a 60ms-CPU request's p99 balloons to 9.7x at a 100m limit, tracking the cgroup's throttled time and vanishing when the quota fits the burst, while the pod uses under 6% of the node. CFS per-period throttling. Real minikube (cgroup v2), pre-registered, independently verified.

What does one external DNS lookup from a Kubernetes pod actually cost?

8 CoreDNS queries under the default ndots:5 -- 6 of them wasted cluster-domain NXDOMAINs, a 4x fan-out (from real CoreDNS logs). Both fixes -- a trailing-dot FQDN or dnsConfig ndots:1 -- cut it to 2. Real minikube, pre-registered, independently verified.

Does an in-flight LLM generation survive a Kubernetes pod deletion?

Only if it finishes within terminationGracePeriodSeconds -- a 12s request dies at 4s grace but completes at 30s. Exit-on-SIGTERM always drops it, and a no-handler PID-1 server counterintuitively ignores SIGTERM entirely. Real minikube, pre-registered, independently verified.

Does GPU external fragmentation limit what you can schedule on Kubernetes?

Permanently. With 4 free GPUs, a 4-GPU pod runs when they sit on one node but is Pending forever when split 2+2 -- the largest schedulable job is bounded by the most-free node (2), not the pool (4). Real 2-node cluster, independently verified.

Can two competing multi-pod GPU gangs deadlock the default Kubernetes scheduler?

Yes -- the scheduling atom is one Pod, so two gangs in a 2/2 split strand all 4 GPUs with 0 gangs runnable, and the default scheduler neither prevents nor breaks it. One atomic multi-GPU pod per job is structurally immune. Real cluster, independently verified.

On Kubernetes, does a max-priority GPU pod always get in by preempting others?

No -- admission strictly precedes preemption. An exhausted namespace quota rejects the pod at creation, so a max-priority pod on a full node evicts nobody even though it would otherwise preempt 2 victims. Two hard walls, exact API-state counts, independently verified.

What is Kubernetes GPU time-slicing, really?

Just the scheduler bin-packing an integer it can't see through: one GPU advertised as K units co-schedules exactly K 'dedicated'-GPU pods (there's no device-identity field in the API), and GPUs are integer-only with request==limit forced, unlike CPU. Real minikube, pre-registered, verified.

How fast does the Kubernetes HPA actually react to a CPU spike?

A median ~56 seconds -- ~40x longer than a pod takes to start -- because the delay is the metrics-scrape plus sync sampling loop, not pod startup. It scales only on the first scraped sample above target. Real minikube, pre-registered, independently verified.

Does a Kubernetes Service load-balance fairly under HTTP keep-alive?

No -- iptables balances new connections fairly (Gini 0.11) but conntrack pins each persistent connection to one pod, so replica coverage follows N(1-(1-1/N)^K) and stays below N even at 2x replicas. Keep-alive starves replicas. Real minikube, pre-registered, independently verified.

What does a Kubernetes GPU allocation actually know about the hardware?

Nothing -- no device identity, no topology, not even a stored utilization figure; the control plane sees a fungible integer count. GPU 'utilization' is derived by summing pod requests, never stored. Four exact API-state facts on a real cluster, independently verified.