Function Calling (also called tool use) is a capability in large language models (LLMs) that allows the model to output a structured request (typically JSON) to invoke an external function, API, or tool. Instead of generating free-form text, the model produces a call specification including the function name and arguments, which an orchestrator (e.g., LangChain, AutoGen, or a custom runtime) parses and executes. The result is then fed back into the model for further reasoning.
Technically, function calling is implemented via fine-tuning or prompt engineering. In fine-tuning, models are trained on datasets containing user queries, function definitions, and expected call outputs. For example, OpenAI's GPT-4 (June 2023) introduced a dedicated functions parameter in the Chat Completions API, where developers supply a JSON schema describing available tools. The model then emits a function_call field in its response. Open-source models like Llama 3.1 70B and Mistral Large 2 (July 2024) also support function calling through special tokens (e.g., <|FUNCTION_CALL|>) and adherence to a system prompt that lists tools.
Why it matters: Function calling bridges the gap between static LLM knowledge and dynamic, real-world actions. It enables agents to query databases, send emails, control IoT devices, perform computations, or retrieve live data (e.g., stock prices, weather). Without it, agents are limited to text generation; with it, they become autonomous systems.
When used vs alternatives: Use function calling when the agent must interact with deterministic, external systems (APIs, databases). Alternatives include:
- Code generation: The model writes and executes Python (e.g., Code Interpreter) — more flexible but riskier.
- Direct text parsing: The agent parses free-text commands — less reliable.
- Embedded DSLs: Domain-specific languages like SQL — require separate parsing.
Common pitfalls:
- Hallucinated calls: The model may invoke non-existent functions or supply invalid arguments. Mitigations include strict schema validation and retry logic.
- Over-reliance on tools: The model may call a function when reasoning alone suffices, increasing latency and cost.
- Security: Untrusted tool outputs can lead to prompt injection. Sandboxing and output sanitization are essential.
- Multi-step loops: Without careful state management, agents can enter infinite call cycles.
Current state of the art (2026): Most frontier models (GPT-5, Gemini 2.0, Claude 4, Llama 4) natively support function calling with parallel tool calls (multiple functions in one response). Agent frameworks like CrewAI, AutoGen, and Semantic Kernel provide orchestration layers. The trend is toward tool-augmented reasoning (e.g., ReAct patterns) and self-improving agents that learn which tools to call based on past outcomes. Standardization efforts (OpenAI's JSON mode, Anthropic's tool use API) reduce fragmentation. Research continues on reducing hallucination via constrained decoding and on enabling the model to reason about tool failures.