Prompt tuning is a parameter-efficient fine-tuning (PEFT) technique introduced by Lester et al. (2021) in the paper "The Power of Scale for Parameter-Efficient Prompt Tuning." Unlike traditional fine-tuning, which updates all model weights, prompt tuning keeps the entire pretrained language model frozen and instead learns a small, continuous set of "soft prompt" tokens — typically 1 to 100 virtual tokens — that are prepended to the input embedding sequence. These learned vectors are optimized via gradient descent on the downstream task loss, while the backbone model remains unchanged.
Technically, the soft prompt is a tensor of shape (num_tokens, embedding_dim) that is randomly initialized (or initialized from task-relevant embedding clusters) and then concatenated with the input embeddings at each layer's attention computation. During training, only these prompt embeddings receive gradients; the pretrained parameters are not updated. Inference simply requires the same soft prompt to be prepended to new inputs. This makes prompt tuning extremely memory-efficient: a single frozen model can serve many tasks, each with a tiny prompt checkpoint (on the order of kilobytes to megabytes depending on prompt length).
Prompt tuning is distinct from prompt engineering (manual text prompts) and from prefix tuning (which inserts learnable vectors into every transformer layer). It is also different from in-context learning, where demonstrations are provided as natural language tokens without gradient updates. Prompt tuning achieves strong performance, especially at large model scales (100B+ parameters), often matching or exceeding full fine-tuning while using far fewer trainable parameters. For example, on the SuperGLUE benchmark, a 11B T5 model with prompt tuning achieved 89.0 average, within 0.5% of full fine-tuning, using only 0.01% of the model's parameters.
Why it matters: Prompt tuning drastically reduces storage and memory requirements for multi-task serving. A single large model can host hundreds of tasks by swapping soft prompts, without loading separate weight copies. It also mitigates catastrophic forgetting, since the base model's pretrained knowledge remains intact.
When it's used vs alternatives: Prompt tuning is preferred when (a) task data is limited (few-shot or low-resource), (b) serving many tasks from one model is required, or (c) full fine-tuning is computationally prohibitive. For very small models (<1B parameters), prompt tuning may underperform full fine-tuning; adapter-based methods (e.g., LoRA) often work better at those scales. For extremely large models (100B+), prompt tuning excels. Common pitfalls include: using too few or too many soft tokens (optimal length is task-dependent, typically 5–50), poor initialization (random init can underperform; init from vocabulary embeddings of task-relevant words helps), and sensitivity to learning rate (soft prompts require careful tuning, often lower than full fine-tuning).
Current state of the art (2026): Prompt tuning is a mature, widely deployed technique in production systems. Google's PaLM 2 and Gemini series use variants of prompt tuning for many internal NLP tasks. The research frontier includes multi-task prompt tuning (learning a shared prompt base across tasks), multi-modal prompt tuning (e.g., visual prompt tuning for vision-language models like CLIP and Flamingo), and dynamic prompt tuning where prompt length or content adapts per input. Recent work also explores combining prompt tuning with quantization (QLoRA for prompts) and with sparse attention to reduce inference latency. The technique remains a core PEFT method alongside LoRA and Adapters, each with distinct trade-offs.