Bibliographic Details
| Title: |
Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification |
| Authors: |
Zhang, Anqi, Chen, Yulin, Pan, Jane, Zhao, Chen, Panda, Aurojit, Li, Jinyang, He, He |
| Publication Year: |
2025 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Artificial Intelligence, Computer Science - Computation and Language |
| More Details: |
Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from overthinking, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24\% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2504.05419 |
| Accession Number: |
edsarx.2504.05419 |
| Database: |
arXiv |
