There’s an intuitive but simplified picture of reasoning: let a model think for longer, and it’ll be more likely to be correct. This aligns well with our human intuition: spend more time and effort on thinking and it’s more likely that we’ll arrive at a correct answer.
But this picture doesn’t seem to explain everything. During RL training, models learn to write longer chains of thought, and they get more accurate as they do. Yet if you take a trained model and force any one chain to run longer, it helps for a while, then stops helping.1 Sometimes the answer even gets worse, as if it’s overthinking.2 If a chain of thought were truly converging toward some optimum, more steps shouldn’t make it worse. So is it really the length of each chain that helps at inference time? Or is it something more subtle?
