Tree-Ising improves reasoning in language models by solving sparse Ising problems, achieving higher math benchmark accuracy with fewer edits and lower costs.
Researchers Hadi Salloum and Ammar Ali have introduced Tree-Ising, a novel method for repairing reasoning loops in large language models (LLMs) by solving sparse Ising or Quadratic Unconstrained Binary Optimization (QUBO) problems. Published on November 10, 2026, on arXiv (2611.14210) and featured on HuggingFace Daily Papers, the method demonstrates significant improvements in mathematical reasoning benchmarks while reducing computational costs.
Tree-Ising operates by casting the repair of reasoning loops as a sparse discrete optimization problem over the model's BF16 weights. The approach uses forward-pass statistics to define an objective that weakens incorrect loop directions, strengthens correct concluding directions, and preserves productive token outputs. Admissible edits are limited to sign flips and exponent shifts, ensuring the preservation of shared group exponents and mantissas in the model's state-tree representation.
The optimization problem is decomposed into independent QUBO problems, one for each output row. These are solved using either simulated annealing or quantum-inspired simulated bifurcation, both of which run efficiently on GPUs. Notably, the method requires only four forward-pass requests and zero LLM tokens, with the annealing process costing approximately $0.90 in GPU time.
7%) under the same protocol. 1%, with both solvers showing statistically significant improvements across all five math benchmarks, including GSM8K. 9% on MATH-500.
Source link
