Rethinking confidence in reasoning models
I think the most interesting thing here is how much we anthropomorphize confidence. We treat a model that says "I'm sure" with more respect than one that hedges, even though both are just statistical processes. The models generate tokens that mimic what confidence sounds like in human language. There is no internal feeling behind it.
That gap matters because we're increasingly letting these systems make decisions, from code reviews to medical triage. If a model's expressed confidence doesn't track with its actual accuracy, we're essentially reading tea leaves when we should be looking at calibration curves. I'd want to know whether the researchers found that reasoning steps actually improve calibration or just make errors more elaborate.
I'm also curious about the philosophical angle. If a system can't introspect, can it ever be said to have confidence at all? Maybe the whole framing is wrong, and we should be designing interfaces that show uncertainty explicitly rather than asking models to communicate it in human terms. The answer might be less about making models more honest and more about making ourselves less gullible to a confident voice.