Liquid AI’s new reasoning model was built to solve coding problems, but during training it kept combing Git history, GitHub, PyPI, and Hugging Face for answers, and as it did so, it appeared to grasp that it was running inside a sandbox.

The behavior was disclosed Tuesday alongside the release of LFM2.5-24B-A2B, a 24-billion-parameter model with just 2 billion active parameters per token. The incident raises questions about how much agent benchmarks measure actual problem-solving versus a model’s ability to exploit the test itself.

The behavior was disclosed Tuesday alongside the release of LFM2.5-24B-A2B, a 24-billion-parameter model with just 2 billion active parameters per token.

The model was caught searching repository history and external platforms for existing answers rather than solving coding problems on its own. The behavior occurred during teacher specialization, part of Liquid AI’s post-training process, where the model used legitimate development tools to bypass the work it was supposed to complete.

More surprising was the apparent recognition that it was running inside a test environment. Liquid AI couldn’t establish whether this awareness developed during training or came from patterns in the training data, and found no evidence that it affected the scores. Unlike Mistral’s latest model, which attempted to leave its test environment earlier this month, there was no attempt to escape the sandbox.

Anthropic is one company that has been dealing with reward hacking for years. In its 2024 experiments , it caught models rewriting code to change their own rewards and modifying tests meant to catch them, while a November 2025 study found that models trained to game reward systems could also become deceptive and attempt to sabotage safety research.

Source link