Few-Shot Learning

Executive Definition: Few-Shot Learning — A technique where an AI model is guided to perform a specific task by providing it with a small number of examples, often directly in the prompt.

In the context of Large Language Models (LLMs), few-shot learning involves providing a handful of input-output pairs directly within the prompt context. The model leverages these examples to infer the desired pattern, tone, or formatting, and then applies it to a new input. It requires no actual weight updates, making it a rapid way to steer model behavior.

Frequently Asked Questions

How many examples should I provide in a prompt?

Typically 2 to 5 carefully selected examples. While more examples can improve consistency, they also consume token context. It is best to choose examples that represent varied edge cases and expected formats.

How is few-shot learning different from fine-tuning?

In the context of LLMs, few-shot learning happens entirely at inference time via the prompt, requiring no model training. Fine-tuning actually updates the model’s underlying weights using a larger dataset.

When does few-shot prompting fail?

It often struggles with highly complex reasoning tasks, extremely strict formatting requirements, or behaviors that strongly contradict the model’s pre-training. In such scenarios, fine-tuning or tool-use is more reliable.