Search publications
“Our models demonstrate smooth test-time scaling. In Figure 3, we sweep over the different reasoning modes of the model (low, medium, high) and plot accuracy versus average CoT+Answer length. We generally see log-linear returns on most tasks, where longer CoTs provide higher accuracy at a relatively large increase in final response latency and cost.” · Show in paper
Figure 3 clearly demonstrates smooth, monotonic accuracy improvements from low to medium to high reasoning effort on AIME 2025 and GPQA Diamond.
“This paper delivers cheap, precise, verifiable control of a coding agent’s decisions by LLM-written weight edits, and a measured account of when such control does and does not raise solve rate.” · Show in paper
The manuscript thoroughly substantiates both the precise, dose-dependent decision control and the nuanced, negative/unconfirmed solve-rate findings with appropriate controls and pre-registered tests.
“The rank ordering of configurations is consistent across all three seeds: full fine-tuning > LoRA r=32 ≈ LoRA r=16 > LoRA r=8 > LoRA r=4 > LoRA r=2. Lower ranks show higher variance (std up to 0.063 for r=2), suggesting sensitivity to initialization at minimal parameter budgets. QLoRA configurations are among the most stable, with std of 0.006 (INT8) and 0.015 (NF4), likely because quantization regularizes the optimization landscape.” · Show in paper
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter ModelTable 3 and Figures 1–3 directly substantiate the consistency of rank ordering and the reported standard deviations across seeds.
“Unless otherwise stated, the main system results use a 200-request trace with 1024 input tokens and 1024 output tokens per request. Request arrivals follow an exponential distribution with an arrival rate of 2.0 requests/s.” · Show in paper
“For GLM-5.3 and Kimi K3, accuracy barely moves: only one of their six scaffold pairs differs significantly, Kimi K3’s Pi against OpenCode by 1.7 points. Inkling is the exception, with Pi trailing the other two scaffolds by 5.8–7.5 points.” · Show in paper
Table 3, Table 4, and Figure 4a directly substantiate the accuracy spreads, p-values, and token-cost differences.
“In Figure 13, LCLMs achieve the highest accuracy across all compression ratios on GSM8K, with particularly strong gains over baselines at higher compression ratios.” · Show in paper
Table 9 and Figure 13 directly support the claim across all three compression ratios.
The claim relies on analytical derivation and external skill specifications (skills_compilation_v2.txt) that are not included in the manuscript, and no empirical benchmark or token-counter evaluation was conducted.
“At deployment time, a one-time profiling script collects data for the analytical models by running production traces at various request rates against the target model hardware combination. The step latency and batch contents are recorded at each step and used to train the analytical models. Total profiling time is ~30 minutes per model/hardware combination.” · Show in paper
“Our evaluations show that both state-of-the-art proprietary models and our fine-tuned model SWE-Llama can resolve only the simplest issues. The best-performing model, Claude 2, is able to solve a mere 1.96% of the issues.” · Show in paper
The experimental results in Tables 2 and 5 support the overarching finding that all evaluated models resolve only a tiny fraction of SWE-bench issues (under 2% for Claude 2, and 3.79% for Claude 3 Opus in the updated Table 5).
“Per-group rank allocation beats uniform rank at all 15 grid cells (ratios 2, 3, 4, 6, 8 at three context lengths), with a peak advantage of 0.97 PPL at n=512, 4×.” · Show in paper
Tables 12 and 13 confirm all reported perplexity differences.
“Experimental results validate our theoretical findings, showing that for KV cache quantization, we achieve absolute quality neutrality with 3.5 bits per channel and marginal quality degradation with 2.5 bits per channel.” · Show in paper
The experimental results in Figure 4 and Table 1 directly support the claimed quality neutrality at 3.5 bits and marginal degradation at 2.5 bits.
“Harness choice substantially affects performance within individual models. General-task score gaps across harnesses range from 5.0 to 9.5 points within nine models, and reach 39.2 for Qwen (Table 2). ... Multimodal gaps are larger for several models: Kimi ranges from 28.1 with Codex to 72.7 with OpenClaw, with preferred harnesses differing across backends (Table 3).” · Show in paper
The reported ranges match the values in Tables 2 and 3.
“We evaluate models using the Harbor framework with the Terminus-2 agent harness [10], which interacts with each task environment through a single long-horizon terminal session. Additionally, for GPT-5.3, we instead use Codex [28] as the agent harness.” · Show in paper
“We evaluate representative MLLMs, including closed-source systems (Gemini-3 [5], ChatGPT-5[15]) and open-weight models (Qwen3 [1], Gemma-3 [4]), under a unified evaluation pipeline. All models are tested with standardized input formatting and identical decoding settings (temperature = 0.2, max tokens = 2048) to ensure fair comparison. For retrieval-augmented experiments, we employ ColPali as a multimodal retriever at the page level, retrieving top-k = 3 relevant pages per query. Retrieved content is concatenated with the prompt before generation. Then for result evaluation, we adopt Qwen-Plus (temperature = 0.2) as an LLM-as-a-judge to score final answer correctness and evidence alignment on a 0–5 scale. We further analyze reasoning behaviors using a structured error taxonomy. In addition, we report ROUGE [11] and BLEU [17] as complementary automatic metrics.” · Show in paper
The counts (1/10, 2/10, 9/10) and position audit match §5.4, and the authors appropriately note that the other three base models hold cold as well.
“For total budget B, we estimate pass@k by splitting B across k independent trajectories of B/k tokens each and counting a task as solved if any trajectory’s first correct submission falls within budget. pass@k is computed task-wise using the unbiased estimator of Chen et al. (2021).” · Show in paper
“In contrast, we propose hybrid sparse attention (HSA), which takes advantage of two-dimensional reduction, in both sequence and head dimensions together to achieve better estimation accuracy on KV token indices with top-k attention scores.” · Show in paper
“In Fig. 7, streaming access for the full query group yields the largest throughput gain. With 16K inputs, K2 improves throughput over K1 by an average of 83.2% on Qwen3 and 117.1% on InternLM2.5.” · Show in paper
The ablation results in Figure 7 and Section 5.3.1 directly substantiate the individual and cumulative gains of streaming verification.
“We further analyze Yi-34B, which claims context length of 200K and achieves reasonably good performance on RULER among open-source models. Our results demonstrate large degradation in Yi's performance as we increase input length and task complexity.” · Show in paper
Figures 2 and 3 and the quantitative error analysis in Section 5 substantiate each of the reported failure modes.
“Gemma 4 31B is closest in size and significantly better across the board, while E2B roughly matches Gemma 3 27B performance with 10x less parameters.” · Show in paper
Table 5 supports both parts of the claim, noting that Gemma 4 models were evaluated in thinking mode whereas Gemma 3 27B was evaluated in non-thinking mode.