Search publications
“Across both Qwen3 models and all three FP4 formats, a SoScale variant obtains the best average score in every model-format group, preserving 96.44%–97.17% of the BF16 average accuracy under MXFP4, 98.99%–99.20% under NVFP4, and 98.38%–99.40% under HiF4. Compared with the strongest non-SoScale baseline in each group, the best SoScale variant improves the average score by 0.21 to 0.77 percentage points; relative to local ScaleSearch, the gains are 0.47 to 1.10 points.” · Show in paper
The reported relative BF16 percentages and improvements over the strongest non-SoScale baselines and ScaleSearch match the exact values in Table 2.
“We evaluate eight open-sourced models at various context sizes across 18 task configurations. Each task can then be represented with a vector of model performance at various context sizes. The 18 task vectors are then clustered via agglomorative clustering algorithm, using correlation coefficient as the distance metric.” · Show in paper
“Given a protection budget ρ ∈ [0, 1] (sensitive-frac), the K=⌈ρL⌉ layers with largest εℓ are loaded in fp16; all others in NF4 with double quantization; lm_head stays fp16.” · Show in paper
“FlashJoLT replaces this SVD with a randomized approximation [11] that computes only the leading qcap token-mode singular vectors. The feature-mode SVD is kept exact because dh is small (128), and the rest of the pipeline, including the Lagrangian allocator, is unchanged.” · Show in paper
“Our additional ablations demonstrate that training on longer sequences does not always lead to better performance on RULER” · Show in paper
Both the LWM ablation in Figure 4 and the main results in Table 3 support that longer training context does not guarantee superior RULER performance.
“Figure 4 (middle-right) shows that the 34B model is significantly better than the 6B model on RULER for both performance at length of 4K and the relative degradation, suggesting the benefit of scaling model sizes for better long-context modeling.” · Show in paper
Figure 4 (middle-right) clearly shows Yi-34B-200K outperforming Yi-9B-200K and Yi-6B-200K across all tested lengths.
“Evaluation results demonstrate that our method substantially narrows the accuracy gap, achieving on average a 70.63% reduction in accuracy loss compared to MXFP4 and a 37.30% reduction relative to the latest NVFP4 on LLM benchmarks. Furthermore, our design delivers up to 1.91× speedup and 1.75× energy savings over state-of-the-art accelerators.” · Show in paper
The reported 70.63% and 37.30% accuracy loss reductions match the Table 2 averages, and the 1.91x speedup and 1.75x energy savings are supported by the normalized latency and energy comparisons in Figure 13.
“The results demonstrate that FP4Quant consistently outperforms Tahquant in preserving model quality under FP4 communication compression. Under TP, Tahquant suffers from severe accuracy degradation (loss 3.4848, PPL = 32.6163), whereas FP4Quant with G = 64 achieves substantially lower loss (2.2306, PPL = 9.3050) and maintains a stable convergence trajectory close to the full-precision baseline, as illustrated in Fig. 5. Under PP, FP4Quant further demonstrates near-lossless performance (loss 2.1525, PPL = 8.6105), matching BF16 accuracy and outperforming Tahquant.” · Show in paper
The numerical metrics in Table 2 and convergence curves in Figure 5 support the claim for G = 64 (and G <= 256, as qualified on page 9).
“GPT-2 has a 50% time horizon of only 2 seconds, while o3 has a 110-minute time horizon and succeeds at several tasks over 4 hours. Also, o3 lies above the long-run trend (p = 0.006); this is robust to methodological ablations like using continuous scoring (Appendix H), which may imply the trend in 2024 and early 2025 is faster.” · Show in paper
The logistic regression estimates, p-value, and 2024-2025 comparison support the claim as qualified with 'may imply'.
“Relative to the unedited model, the residual-stream changes of adapter and hook have cosine 0.972–0.992 and 0.984–0.996 across eight layers from 18 to 41. Their norm ratios are within 0.4% of 1. Over completion tokens, KL(hook k adapter) is 1.1 × 10−4 and 1.2 × 10−4. This is 99 and 110 times smaller than the edit’s own effect, KL(unedited k adapter) = 0.011 and 0.013.” · Show in paper
The teacher-forced forward-pass measurements in Table 12 directly support the claimed cosine similarities, norm ratios, and KL ratios.
“The partial Tucker approximation X˜(rT , rd) computes the mode-k singular vectors of the token and feature unfoldings via the sequentially truncated higher-order SVD (ST-HOSVD) [22] and truncates them to ranks rT and rd, while keeping identity factors on the layer and head modes.” · Show in paper
“The Target KV Cache is partitioned into the following three regions accordingly, Sealed History: [0,H) (CPU) Active Tail: [H,Cr) (GPU) Speculative Frontier:[Cr ,Lr)(GPU) 0 ≤ H ≤ Cr ≤ Lr.” · Show in paper
“The base LLM stays frozen. We train only the projections , the thresholds , and the rank predictor , under 0.1% of the backbone’s parameters, once per backbone on generic text (Section 4.1). The objective combines four terms: .” · Show in paper
“In our MTP procedure, the model's last layer activations from the previous step and token embeddings are fed into the MTP head. The MTP head generates future tokens sequentially using a separate embedder and a 4-layer Transformer block that cross-attends to the KVs of the main model (Figure 1), thus eliminating the need for MTP prefill and supporting any draft length.” · Show in paper
“Figure 2 shows that configurations with stronger OOD degradation also exhibit larger drift from the pretrained model. Higher-rank QLoRA checkpoints have larger symmetric KL divergence and larger effective dense update magnitudes. The strongest forgetting regime, QLoRA r=64, also has the largest SVD intruder excess, indicating a larger change in the leading spectral structure of adapted weight matrices.” · Show in paper
Figure 2 clearly shows monotonic increases in symmetric KL, dense RMS drift, and SVD intruder excess with QLoRA rank, exceeding the FFT baseline.
“M2XFP achieves the lowest perplexity on all models except OPT-6.7B, where BlockDialect is better by only 0.03.” · Show in paper
Tables 3 and 4 directly substantiate the perplexity rankings and the reasoning accuracy recovery.
“Comparing full fine-tuning (FFT) with quantized low-rank adaptation (QLoRA) at ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition–retention frontier. Low-rank QLoRA preserves out-of-domain (OOD) performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at an increasing cost in performance on unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime.” · Show in paper
The five-seed results in Figure 1 and Table 5 directly support the full claim, including the relative positions of FFT and QLoRA ranks 8, 16, 32, and 64.
“We evaluate serving with a closed-loop concurrency sweep. At concurrency C, the benchmark harness admits at most C requests to the endpoint at a time. When one request completes, the next request or session turn is dispatched. For multi-turn profiles, each slot advances a session in turn order, preserving the session-context snowballing that makes agentic prompts grow across turns, increasing prefix-cache pressure.” · Show in paper
“In summary, our results show that while LLMs are effective at retrieving isolated facts, their capabilities degrade substantially when required to reason over long contexts.” · Show in paper
Can LLMs reason over extended multilingual contexts? Towards long-context evaluation beyond retrieval and haystacksFigures 1, 5, 6, and 8 clearly demonstrate the performance gap between retrieval tasks and reasoning tasks across all tested languages.
“We get a suitable α by running a quick grid search on a subset of the Pile (Gao et al., 2020) validation set. To get the statistics of activations, we calibrate the smoothing factors and the static quantization step sizes once with 512 random sentences from the pre-training dataset Pile, and apply the same smoothed and quantized model for all downstream tasks.” · Show in paper