Instruction Tuning

Executive Definition: Instruction Tuning — A fine-tuning process that trains a base language model to follow natural language commands and instructions.

Base language models are trained merely to predict the next word in a sequence. Instruction tuning uses supervised learning with pairs of instructions and desired responses to teach the model how to act as a helpful assistant, directly answering questions or completing specified tasks.

Frequently Asked Questions

What is the difference between instruction tuning and RLHF?

Instruction tuning teaches a model to follow commands using supervised learning on predefined datasets. Reinforcement Learning from Human Feedback (RLHF) is a subsequent step that aligns the model’s responses with human preferences.

Can I instruction-tune an open source model?

Yes. Developers can use techniques like Low-Rank Adaptation (LoRA) and public instruction datasets to instruction-tune open-source base models on relatively modest hardware.

Why do base models need instruction tuning?

Without instruction tuning, a base model might respond to a question by generating more questions instead of an answer. Instruction tuning aligns the model’s behavior with the user’s intent to receive a helpful response.