This paper provides the first comprehensive overview of the current state of knowledge on metacognition for large language models (LLMs). It bridges the gap in understanding when and how LLMs can exhibit effective metacognitive abilities to advance AI reliability and intelligence.
- Analyzes and taxonomizes the landscape of metacognition in LLMs.
- Summarizes recent technical advancements, including methods and benchmarks for measuring and evaluating these abilities.
- Details techniques to elicit, improve, and apply metacognition in LLMs.
- Discusses applications, open questions, challenges, and promising directions for future work.
The authors aim to provide a detailed review to stimulate meaningful research and discussion in this emerging field.