1. /76
    2026-02-02

    R-Stitch Dynamic Trajectory Stitching for Efficient Reasoning

    R-Stitch hands decoding from a small model to a large one when token entropy rises and back once it falls; R-Stitch+ replaces the threshold with an RL-trained, latency-aware router.

    arXiv-2507
  2. /75
    2026-02-02

    Estimating LLM Uncertainty with Evidence

    LogTokU treats raw logits as evidence to split token uncertainty into aleatoric and epistemic parts, separating four states that softmax probability conflates, to detect hallucinations without sampling.

    arxiv-25
  3. /74
    2026-01-29

    XAttention Block Sparse Attention with Antidiagonal Scoring

    XAttention scores attention blocks by the sum of their antidiagonals and selects blocks by a cumulative threshold, with a worked example of the strided sampling that cuts estimation cost to 1/S.

    ICML-25sparse-attention
  4. /73
    2026-01-29

    FlexPrefill A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

    FlexPrefill chooses a query-aware or vertical-slash sparse pattern per head and per input during prefill, and uses cumulative attention to decide which blocks to compute.

    ICML-25sparse-attention
  5. /72
    2026-01-07

    Think Big, Generate Quick LLM-to-SLM for Fast Autoregressive Decoding

    A frozen LLM encoder encodes the prompt, a projector adds its representation to the SLM's embeddings, and the SLM decodes autoregressively, plus why the approach depends on encoder-decoder LLMs.

    ICML-24-Workshop
  6. /71
    2026-01-07

    Entropy Adaptive Decoding Dynamic Model Switching for Efficient Inference

    Entropy Adaptive Decoding switches between a small and a large model by a rolling-window entropy threshold, accepting output divergence for speed, with notes on where its evaluation falls short.

    arXiv-2502
  7. /70
    2026-01-07

    Beyond the 80 20 Rule High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning

    Only about 20% of CoT tokens are high-entropy forking tokens that steer the reasoning path; restricting RLVR gradients to them beats full-token training on Qwen3-32B.

    NeurIPS-25
  8. /69
    2025-12-07

    SLED A Speculative LLM Decoding Framework for Efficient Edge Serving

    SEC-25
  9. /68
    2025-12-06

    KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider

    ATC-25KVCache
  10. /67
    2025-12-01

    Token Level Routing Inference System for Edge Devices

    arXiv-2504