Epoch

Executive Definition: Epoch — One complete pass of the entire training dataset through a machine learning algorithm during the training process.

An epoch indicates that the model has processed every example in the training dataset exactly once. The training process generally spans multiple epochs—a few for fine-tuning, or potentially hundreds when training from scratch. Selecting the right number of epochs is crucial; too many can cause overfitting, while too few can result in underfitting.

Frequently Asked Questions

How many epochs should I train for?

There is no universal number. The best practice is to monitor validation loss and implement early stopping when it stops improving. For LLM fine-tuning, 1 to 5 epochs is typical, whereas smaller models might require hundreds.

What happens if I train for too many epochs?

Training for too long leads to overfitting. The model begins to memorize the training data rather than learning generalizable patterns, causing its performance on unseen data to degrade even as training metrics improve.

Is one epoch ever enough?

Yes, especially for exceptionally large datasets. Many modern foundation models and large language models are trained for less than one full epoch over their massive training corpora.