Search publications
“Under that control the ranking has no measurable advantage (Appendix N lists every setting). Across five models and three budgets, top-K beats the median matched draw in 11 of 15 settings, and 50 of the 150 draws beat top-K where chance predicts 75. A sign test does not separate that from chance (p=.12), and the median gap is 0.01 perplexity.” · Show in paper
The parameter-matched control in Table 19 and downstream comparisons in Section 7.2 directly support this negative result.
“As shown in Table 8, compared to an average gold patch, model generated patch files that apply correctly are less than half the total length (74.5 versus 30.1 lines) of gold edit patch files, and rarely edit more than a single file.” · Show in paper
Table 8 confirms that all evaluated models' applied patches average at most 30.1 total lines (less than half of All Gold's 74.5 lines) and 1.0-1.1 files edited.
“For every plotted setting, BridgeKV improves over the fine-tuned decode anchor while keeping the same small model for generation. However, the magnitude of the gain depends on the task and model pair. As shown, BridgeKV improves the accuracy of the directly fine-tuned decode anchor by up to 8.4% relative (NarrativeQA, for Qwen2.5-Instruct-7B→Qwen2.5-Instruct-0.5B). In the worst case, the relative improvement is 0.2% (SAMSum, for Llama-3.1-Instruct-8B→Llama-3.2-Instruct-3B)” · Show in paper
Figure 3 confirms that BridgeKV outperforms the parameter-matched decode anchor in all 30 settings, with relative gains ranging from 0.2% (SAMSum, Llama 8B->3B: 45.3 to 45.4) to 8.4% (NarrativeQA, Qwen 7B->0.5B: 56.8 to 61.5).
“As shown in Table 3, we observe that for the Llama and Mistral model, KIVI only has up to 2% accuracy drop despite the KV cache being stored in 2bit. ... Since Falcon-7B adopts multi-query attention and only has one head for KV cache, it is already highly compressed compared to Llama-based models. Thus, in Table 3, 4bit KIVI is needed to maintain the accuracy, while 2bit KIVI may have a large accuracy drop in this case.” · Show in paper
The reported tables and figures accurately reflect these accuracy bounds and the distinction between Llama/Mistral and Falcon-7B.
“Across the three datasets, SpecStream achieves average throughput speedups of 1.41× and 1.32× over TriForce on Qwen3 and InternLM2.5, respectively. The corresponding speedups over SD+Offload are 1.44× and 1.62×.” · Show in paper
The throughput values plotted in Figure 5 (e.g., Qwen3: 263.4 vs 209.5/229.8 on GSM8K, 17.7 vs 12.2/10.5 on LongBench v2, 16.5 vs 10.8/11.0 on MRCR) support the reported average speedups.
“As shown in Figure 5, we see that as total context length increases, Claude 2’s performance drops considerably; behavior that is also observed in other models. In our evaluation settings, models see a lot of code that may not be directly related to solving the issue at hand, and they seem to frequently struggle with localizing problematic code needing to be updated.” · Show in paper
Figure 5 and Tables 2-3 directly demonstrate that longer input contexts degrade model resolve rates even when recall of relevant files increases.
“At inference time, we use the learned retention coefficients to rank all cached KV entries globally. For each cached token (ℓ, h, i) present at decoding step t, we assign the retention score G̃ℓ,h,i(t) = ∑T s=t+1 βs−i ℓ,h,i = βt+1−i ℓ,h,i (1 − βT−t ℓ,h,i)/(1 − βℓ,h,i), for βℓ,h,i < 1.” · Show in paper
“It transfers. On held-out cities and clinics, the share of tool turns handled with the right tool and the right inputs rises from 25% to 67%, and on the unseen business from 25% to 79%. Bookings without consent fall from 71 to 1 on the held-out test, and all 10 emergency calls are transferred at once (§5).” · Show in paper
Tables 1 and 2 directly report the exact percentages and counts cited in the claim, with bootstrap confidence intervals supporting the gains.
“For decode kernels, the performance gap between sparse and dense KV-Cache is negligible (within 1%). For prefill kernels, there is approximately a 10% performance gap.” · Show in paper
Figure 12 shows decode bandwidth utilization within 1% (83–85%) and prefill TFLOPs/s within ~5–15% (around 10% on average).
“We show that fixed wall-clock and output-token budgets can materially change apparent agent performance: For example, extending the time limit closes a 20 pp harness gap on GPT 5.4 and shrinks a 30 pp gap on GPT 5.4 mini to 5 pp.” · Show in paper
Figure 6 directly shows the 20.3 pp gap on GPT 5.4 closing completely (74.9% vs 75.3%) and the 29.9 pp gap on GPT 5.4 mini shrinking to 5.3 pp (68.2% vs 62.9%) at 32x timeout.
“O(1) Constant Input Context: Because messages is freshly constructed on every iteration of the while loop with only system_prompt and the tiny state_cursor JSON, input token cost and attention complexity never grow—whether you run 6 stages or 600 stages.” · Show in paper
The code in fetch_batch_openai and fetch_batch_anthropic directly confirms that only system_prompt and user_payload (containing state_cursor) are sent on each API call.
“Choosing a separate harness per model on training tasks improves held-out full-credit attainment by 2.53–2.73 percentage points over a single shared harness. Mean-score gains remain within ±0.14 points, and R0.8 gains within 0.31 points. Matching harnesses to models thus matters more for complete delivery” · Show in paper
Table 10 and the accompanying text in Appendix C.1 confirm that full-credit gains are positive in all 20 repetitions and substantially larger than mean-score or R0.8 gains.
“Expanding from {q,v} at r=8 (53.4%, 344K params) to the full attention set {q,k,v,o} (59.2%, 590K params) recovered 5.8 points at the cost of 71% more trainable parameters and a 6.3% memory increase (1.59 to 1.69 GB). Adding feed-forward layers {q,v,FFN} (58.6%, 786K params) matched {q,k,v,o} in accuracy but required 33% more parameters, an unfavourable trade-off. Notably, LoRA r=16 on {q,v} (59.6%, 590K params) matched {q,k,v,o} at the same parameter budget, suggesting that, in this setting, rank is a more efficient lever than module expansion.” · Show in paper
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter ModelTable 2 directly supports the exact accuracy, parameter count, and memory comparisons between module expansion and rank increase.
“Consistent with prior task-adaptation results (Biderman et al., 2024), the strong rank-dependent frontier observed on OSM is not evident in this math setting. In our experiments, it is therefore most pronounced when adaptation installs new factual associations while preserving OOD behavior.” · Show in paper
Table 3 supports the near-identical math performance and small OOD drops across FFT and QLoRA r=16, 32, and the authors appropriately bound the conclusion to their experimental settings.
“Agentic LLM workloads occupy a hardware regime that no prior inference benchmark captures. Across 22 workload profiles, 9 models, and 14 GPU configurations (3,197 sweep rows, 148,077 NCU records), AGENTPERFBENCH identifies a dual pressure that separates agentic serving from chat.” · Show in paper
The benchmark comparison, saturation sweeps, and roofline/capacity analyses directly support the claim that agentic workloads exhibit this dual-pressure regime uncaptured by prior benchmarks.
“From Table 5, we observe that group sizes 32 and 64 yield similar results, whereas the performance significantly decreases when the group size reaches 128.” · Show in paper
Tables 5, 6, and 7 directly support the stated sensitivity to group size 128 and the robustness across residual lengths 32 and 128.
“We fine-tune T5-small (60M parameters) for natural-language-to-SQL generation on WikiSQL. Each example is serialized as: translate English to SQL: <Q> table: <H1 | H2 | ···> and the target is the gold human_readable SQL string. We train on 5,000 examples and evaluate on 500 validation examples to maintain a tractable single-GPU sweep.” · Show in paper
“AQLoRA-q averages 4.8 ± 2.0% and is also faster in every session, by 2.1 to 8.1%. Neither result depends on the estimator or the exclusions (Appendix O).” · Show in paper
Both the 4.8 ± 2.0% speedup and the accuracy parity across all tested models and tasks are supported by the tables.
“Overall, the combination of adaptive smoothing with GPTQ proves to be the most effective method, yielding the lowest perplexity scores. For instance, this approach reduces perplexity by 0.47 in per-group quantization compared with the baseline.” · Show in paper
Table 3 confirms that B+AS+GPTQ consistently achieves the lowest perplexity across all tested models and granularities, including the 0.47 reduction on per-group LLaMA-3-8B.
“Gemma 4 31B is the top open model in the dense category, and both Gemma 4 31B and 26B-A4B show performance equal to much larger open models.” · Show in paper
Table 4 directly shows Gemma 4 31B as the highest-ranked dense open model and both 31B and 26B-A4B achieving Elo scores comparable to or exceeding much larger open MoE models.