Chain-of-Thought Prompting: definition + examples

Chain-of-Thought (CoT) prompting is a technique that improves the reasoning capabilities of large language models (LLMs) by including intermediate reasoning steps in the prompt. Instead of asking for a direct answer, the prompt demonstrates a sequence of logical steps that lead to the final answer. This method was introduced by Wei et al. (2022) in the paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." The core idea is that autoregressive LLMs, which generate text token by token, can be guided to produce a coherent chain of reasoning when shown examples of such chains. Technically, CoT prompting works by conditioning the model on a few examples (few-shot CoT) or by simply appending the phrase "Let's think step by step" (zero-shot CoT, Kojima et al., 2022). The model then generates a sequence of intermediate thoughts before outputting the final answer. This improves performance on tasks that require multi-step reasoning, such as math word problems (GSM8K accuracy rose from ~18% to ~58% with CoT in GPT-3), symbolic reasoning, and common-sense reasoning. CoT is particularly effective for models with at least 100 billion parameters; smaller models may not benefit or may generate incoherent chains. A key variant is "Self-Consistency" (Wang et al., 2022), which samples multiple CoT paths and selects the most consistent answer, further boosting accuracy. Another variant, "Tree-of-Thoughts" (Yao et al., 2023), generalizes CoT by exploring multiple reasoning branches. CoT prompting is distinct from fine-tuning because it requires no parameter updates; it is a pure inference-time technique. However, it can be combined with supervised fine-tuning (e.g., training on CoT-augmented datasets like MathQA) to create models that inherently reason step-by-step. Common pitfalls include: (1) CoT can be brittle to prompt phrasing; (2) it increases token usage and latency; (3) models may generate plausible but incorrect reasoning (hallucination in the chain); (4) it does not help with tasks that don't benefit from intermediate steps (e.g., simple fact retrieval). As of 2026, CoT is standard practice in production LLM systems (e.g., GPT-4, Claude 3, Gemini 1.5) and is often combined with retrieval-augmented generation (RAG) and tool use. Research focuses on automating CoT prompt generation (Auto-CoT, Zhang et al., 2023), improving faithfulness of reasoning, and extending to multimodal CoT (e.g., Zhang et al., 2023 for visual reasoning). CoT is also a foundational component of agentic systems where LLMs decompose tasks into sub-steps.