RLVR and Reasoning Models
How reasoning models are trained with verifiable rewards and GRPO, how they game imperfect checkers, why you never reward the chain of thought, and why most teams should distill instead.
Reward the answer with a checker
RLVR (reinforcement learning with verifiable rewards) trains reasoning models without a learned reward model. The policy writes a long chain of thought and an answer; a deterministic verifier checks it: did the answer match, did the tests pass, does the proof check.
The reward is often just 1 for correct, 0 for wrong, sometimes plus a small format reward. Long chains of thought, self-verification and backtracking emerge from this loop rather than being supervised. DeepSeek-R1 showed reasoning can be incentivised through pure RL with no human-labelled reasoning traces.
- +Checkable prompts
- +Policy
- +Verifier
- +Gradient update
Lesson 8 of 8 in Training & Adaptation