Search publications
“Lemma A.2 (Separation of unequal-label segments). For sufficiently large integers M and integers 1 <= K <= M, there are K binary words of length P = M^2 such that any two length-a segments of distinct codewords satisfy LCS <= 0.99a whenever a >= a0 := ceil(40 log2(MP)) = ceil(120 log2 M).” · Show in paper
“Divide the p levels into consecutive blocks of B levels each. At the first level of every block after the first, we reset the selection cell to the whole space. Older retained references are kept unless they meet the conflict rule specified below. Independently of the disorder, banks and path, draw independent thresholds r_i, uniform on I_i = [t_0 + 4(i-1)\rho, t_0 + (4(i-1)+1)\rho].” · Show in paper
“The batched implementation adds 1.6–6.7% to the prefill time of the prefill anchor across the four pairs, whereas the per-layer implementation adds 5–51% (Figure 4b). ... The request takes 2.96 s, close to the 2.80 s of Llama-3.2-Instruct-1B alone and 3.0× faster than the 8.83 s of Llama-3.1-Instruct-8B alone (Figure 1).” · Show in paper
Measured median latencies on one A40 GPU in Figure 1, Figure 4, and Section 4.3 directly support the reported percentages and speedups.
“Following GPTQ, we compute an upper-triangular factor U of the damped inverse Hessian, U⊤U = (H + λI)−1, (9) ... The block error is then propagated to the future blocks through the back-substitution: Wf:,F ← Wf:,F − EQU−1Q,QUQ,F. (13)” · Show in paper
“Proposition 7.1 (Embedded comparator cost). There are finite alphabets A_1, ..., A_J, depending only on Y, k and the fixed constants, and online random maps ξ_t : Y → A_1 × · · · × A_J with the following properties. Each coordinate cell at level j has diameter at most 40r_j and an anchor within 20r_j of all its points. Thus ξ_t is injective. Give the full alphabet tree the leaf distance when the first differing coordinate is j. For the selected hidden members X_t^1, ..., X_t^k, P := E ∑_t ∑_{a=1}^k d_T((^a), ξ_t(X_t^a)) ≤ CℓQ + . (7.2) Here C is absolute and < ∞ is independent of the input law and its horizon.” · Show in paper
The proof in Sections 6 and 7 carefully separates conditioning on the filtration for pilot potentials from averaging over the partition tape for separation and retirement probabilities.
“As shown in Table 5, using the token contextual embedding xt as input performs better, particularly under tight budgets, suggesting that xt provides a stronger long-horizon utility signal than the projected KV representations alone.” · Show in paper
Table 5 directly supports the claim.
“Let Cb,r be the following set of shared-endpoint actions on the scale-b grid. The outside endpoints are exactly r, consecutive child endpoints agree, all endpoint shifts are within 6b of p0 = r− − l, and the shift p at every internal cut satisfies |p − p0 − vbΦj(p)| ≤ τθb/M.” · Show in paper
“(1) 4-bit NormalFloat, an information theoretically optimal quantization data type for normally distributed data that yields better empirical results than 4-bit Integers and 4-bit Floats.” · Show in paper
Empirical results across perplexity (Table 2), zero-shot accuracy (Figure 3), and MMLU (Table 4) consistently support that NF4 outperforms FP4 and Int4.
“we compute during prefill and mark positions with as boundaries, where and are the sequence mean and standard deviation, and boundaries closer than tokens are merged, keeping the highest-surprisal one.” · Show in paper
“This yields the lower bound \hat{Z}_{\mathcal{E}} = n_e \cdot \exp(q^\top \bar{k}/\sqrt{d}) \le Z_{\mathcal{E}}, with relative error O(\sigma^2) controlled by the logit variance (see Appendix B). With both components estimated, the corrected output takes the form \hat{f}(q) = \hat{w}_{\mathcal{R}} f_{\mathcal{R}}(q) + (1 - \hat{w}_{\mathcal{R}}) \hat{f}_{\mathcal{E}}(q), \hat{w}_{\mathcal{R}} = Z_{\mathcal{R}} / (Z_{\mathcal{R}} + \hat{Z}_{\mathcal{E}}).” · Show in paper
“For a prescribed center path c(σ) and a spatial scale δ > 0, the curl field V(σ, x) = ∇x × [ζ((x − c(σ))/δ) (c˙(σ) × (x − c(σ)))/2] is divergence-free and equals c˙(σ) on a neighborhood of the moving center.” · Show in paper
“Theorem 5.2 (One-step coverage transfer).” · Show in paper
“Let Ds > 0, and let Q satisfy (7.1), with ∥v∥2 ≤ Ds. Put n = r/(2Ds), P = (Im − vv^T/(4D_s^2))^{1/2}. For a deterministic x ∈ R^d, let Z, G1, . . . , Gm be independent standard Gaussian vectors in R^d. Then the output of Q(x + nZ; (P ⊗ Id)G) has exactly the law of Q(x; G).” · Show in paper
“Online Activation Quantization. We process X via: (1) Reordering and Primary Quantization. X is reordered, followed by block-wise quantization QX = round(X/sX). (2) Residual Compensation. We isolate outliers Xo, compute residuals Ro = Xo − sXo · QXo , and quantize them into QRo . (3) Augmentation. We concatenate along dimension K as QXaug = [QX | QRo ] with combined scales sXaug = [sX | sRo ].” · Show in paper
“After the first 768 generated tokens, we encode the oldest 512 generated tokens into latent memory tokens at a 16x compression ratio, resulting in 32 latent tokens, and keep the most recent 256 tokens uncompressed. Generation then continues from the compacted context and latest raw tokens, and the procedure repeats after every further 512 generated tokens.” · Show in paper
“FlashJoLT replaces this SVD with a randomized approximation [11] that computes only the leading qcap token-mode singular vectors. The feature-mode SVD is kept exact because dh is small (128), and the rest of the pipeline, including the Lagrangian allocator, is unchanged.” · Show in paper
“We use a local-lemma argument [3, Section 2, pp. 616–617], recalling the required finite probability estimate with its proof.” · Show in paper
“If it overflows, discard the task's workspace and replace the task by two right tasks that fix the next bit of its M_0 mask to zero and one, respectively. Process these tasks depth first, in a fixed order of the top-mixer bits.” · Show in paper
“Table 2 assigns NVFP4 to the QK⊤ and dS paths and retains the softmax-sensitive P V , P⊤dO, and dOV ⊤ products in BF16. Quantized views are reused between forward and backward to avoid precision mismatch and repeated memory traffic.” · Show in paper
“Theorem 4.1 (Two binary substitutions). Let A be a finite alphabet with A >= 2 letters. Each of the two constructions below gives an integer width w >= 1 with w = O(log(2A)), an absolute constant a > 0, and a map c : A -> {0, 1}^w such that, simultaneously for every integer L >= 0 and all x, y in A^L, a w r_L(x, y) <= (c(x), c(y)) <= w r_L(x, y). (23) Here c extends to words by concatenation. For arbitrary finite words, ED(c(x), c(y)) <= w ED(x, y). (24)” · Show in paper
Both probabilistic existence bounds and deterministic matching recovery arguments are proved without gaps.