Turning Database Grading into RL Rewards: Post-Training Agents with ThinkingBox

Microsoft and Hugging Face have released ThinkingBox-Training, a follow-up to their ThinkingBox benchmark, which grades AI agents by inspecting the final state of a database rather than trusting what the agent reports. In a joint blog post published October 22, 2026, the authors describe reusing those executable pass-or-fail checks as reinforcement learning rewards.

They fine-tuned Qwen3.8-27B with GRPO for 50 steps on 187 tasks kept separate from the 507-task ThinkingBox-Bench, using 24 H100 GPUs. Average success across 10,140 attempts rose from 51.70% to 60.78%, and tasks passed on all twenty tries jumped from 38 to 107. Coverage slipped slightly, with tasks solved at least once falling from 453 to 446. The authors describe the model as more repeatable, not broader. By comparison, Claude Opus 5.5 leads repeatability at 241 tasks.

The key technique is what they call offline policy-relative difficulty. Each candidate task is probed against the base model once before training, and tasks it almost always fails or passes are dropped, since GRPO learns only from groups with mixed outcomes.

The authors estimate cost per dependable task fell from $24.36 to $8.00, though that figure excludes training and probing costs. They also caution that the benchmark shares its framework, grading approach and a few tools with the training tasks, which limits the transfer claim. The code is public, but the 187-task training pool is private. For teams building agents that touch production records, the post argues that average accuracy alone hides how unreliable a model can be.