Search publications
No empirical token-footprint measurements are provided, and a fixed 4-tuple schema does not strictly guarantee tokens unless the length of text within each tuple field is also bounded.
“The routing summary projects token keys into a -dimensional space with a learned per-head matrix and averages them with surprisal weights : .” · Show in paper
“Append m − 1 copies of η, and average the m cyclic permutations of the m pair registers.” · Show in paper
“Then EJ,T1,...,TJ−1,T |E(p,q)∼ω⟨fT (cT (p)) − mT,H(p) , gT (q)⟩| ≤ 4BB′ (D/K) 1/4 . Every realized history has at most MK−1 values. In particular, a history can be fixed before drawing the fresh test so that its average absolute discrepancy in (3.7) is at most the right side.” · Show in paper
The information-theoretic chain rule, Pinsker's inequality, and conditional independence steps are all sound.
“For any randomized quantization algorithm Q : → {0, 1}^{b·d} with bit-width b and any reconstruction map : {0, 1}^{b·d} → R^d, there exist a hard input instance x ∈ such that: Dmse(Q) := E[∥x − (Q(x))∥^2_2] ≥ 1/4^b.” · Show in paper
The lower bound proof in Theorem 3 (pages 14–15) is rigorous and follows directly from Yao's minimax principle, Lemma 3, and the pigeonhole principle.
“By contrast, we will aim to quantize the weights of all rows in the same order, and will show that this typically yields results with a final squared error that is similar to the original solutions. ... This reduces the overall runtime from O(drow · d^3_col) to O(max {drow · d^2_col, d^3_col}), i.e., by a factor of min {drow, dcol}.” · Show in paper
The complexity reduction follows from updating once per column, and empirical comparisons in Tables 1 and 8 support comparable accuracy to OBQ.
“Each activation block is quantized as xbt,b = QactF (xt,b; cactt,b), cactt,b = DF(xt,b), (14) where DF is the format-specific dynamic scale rule for activations. Because activation scales are computed online for every token, DF is kept lightweight and selects cactt,b for each token block using an element-wise MSE criterion.” · Show in paper
“Proposition 6.8 (Localization into current groups). For an active pattern in (11), there are poly(k) fixed search leaves such that, up to total absolute length-weighted error at most Ce^{-k}|R|, each leaf contributes \sum_{G \text{ current}} H_I(, , , ), = \overline{L}_{l,G,s}\mathcal{B}_{l,s}z_l. (14)” · Show in paper
The binary localization argument in Proposition 6.8 is clearly explained and all error terms are controlled by (13) and Lemma 6.1.
“LoRA-SVD protects a compact projection subspace in BF16 while retaining full-shape NVFP4 computation, reducing the linear-only loss gap from 1.40% to 0.61%.” · Show in paper
Table 3 and Section 4.2 directly support the reduction of the linear-only loss gap from 1.40% to 0.61% on 1B/20B-token pretraining.
“We first bound the normalization of the successful event, then expose a common conditional density. We control entropy loss in two clipping steps, then deduce concentration near the endpoints of a fixed interval. Finally, one prefix and one threshold will satisfy every calibration and order requirement simultaneously.” · Show in paper
“Use the axis-neighbor Metropolis chain on \mathcal{V}. Each of the 2e signed coordinate directions is proposed with probability q_B = 2^{-q_{\mathrm{bin}}}, q_{\mathrm{bin}} = \lceil \log_2(8d) \rceil; the unused probability, or a proposal leaving the cube, holds.” · Show in paper
“From the figure we conclude that, under the prefix-aware batching policy, more than 90% of iterations are able to complete the forward computing within 30ms, whereas the proportion of iterations that complete the forward computing within 30ms is less than 10% for the FCFS policy.” · Show in paper
AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving SystemThe controlled ablation in Figure 13 and the preliminary experiment in Figure 3 directly support the claimed latency reduction.
“In vLLM, we adopt the first-come-first-serve (FCFS) scheduling policy for all requests, ensuring fairness and preventing starvation. ... we implement an all-or-nothing eviction policy, i.e., either evict all or none of the blocks of a sequence. Furthermore, multiple sequences within one request (e.g., beam candidates in one beam search request) are gang-scheduled as a sequence group.” · Show in paper
“Use the torus coordinates (x, z) = (u, v/u), (u, v) = (x, xz).” · Show in paper
“There is an extraction of a consistent tuple F′ of deterministic length ℓ′ = Θ(k), still in one of the two stream orientations, and a new discrete context C′ for which H(C′) ≤ Λ′ ≤ CqP log(w + 2)(σ + wP) + Cwσ, |domain of each slot of F′| ≤ Cq^d , Δ′ ≤ CK∗, 1 ≤ w ≤ Cσ^{1+η/2}/D.” · Show in paper
The standalone nature of the new context C' and the uniformity of all constants across admissible histories are carefully verified in Section 6.5 and the proof of Lemma 6.1 on page 47.
“On GSM8K [41] with 8-shot chain-of-thought prompting [42] and exact-match scoring (n=1319 per cell), JoLT is near-lossless at 2× and 3× on both Mistral and LLaMA-2 (Table 7 in Appendix C). All free-zone deltas fall within their 95% confidence intervals of zero, with the largest shift at −1.21 pp (Mistral 3×).” · Show in paper
Tables 7 and 10 support the claims with the stated caveats (single model and single seed for LongBench).
“The results demonstrate that FP4Quant consistently outperforms Tahquant in preserving model quality under FP4 communication compression. Under TP, Tahquant suffers from severe accuracy degradation (loss 3.4848, PPL = 32.6163), whereas FP4Quant with G = 64 achieves substantially lower loss (2.2306, PPL = 9.3050) and maintains a stable convergence trajectory close to the full-precision baseline, as illustrated in Fig. 5. Under PP, FP4Quant further demonstrates near-lossless performance (loss 2.1525, PPL = 8.6105), matching BF16 accuracy and outperforming Tahquant.” · Show in paper
The numerical metrics in Table 2 and convergence curves in Figure 5 support the claim for G = 64 (and G <= 256, as qualified on page 9).
“Compressing the unknown block between these kernels gives a 2-by-2 matrix. The Hadamard equations bound its distance from sqrt(6) times its unitary polar factor. A closed cover of U(2), the group of 2-by-2 unitary matrices, therefore accounts for every feasible completion.” · Show in paper
“Specifically, let Qmse be the quantization map corresponding to TurboQuantmse with a bit-width of b − 1. For any x ∈ the residual vector, defined as r := x − _mse (Qmse(x)), has a small L2 norm... We can then apply the QJL quantization map Qqjl on this residual vector, resulting in an overall bit-width of b” · Show in paper
“Figure 6 confirms substantial gains over FP16: Qwen2.5-7B achieves 2.0×–2.5× speedup (PRO 6000) and Llama 3.1-8B reaches 3.5× (RTX 5090), while memory usage drops by 1.5×–2.8×. Crucially, residual overhead is marginal: compared to uncompensated NVFP4, latency increases by only 3%–9%.” · Show in paper
The latency and memory numbers in Figure 6, Figure 8, and Table 10 support the reported speedup, memory reduction, and 3%–9% overhead over uncompensated NVFP4.