Foundation Model

Executive Definition: Foundation Model — A large-scale artificial intelligence model trained on a vast quantity of unlabelled data that serves as a versatile base for many downstream tasks.

Foundation models, such as GPT-4, Claude, and Llama, act as core infrastructure platforms that can be rapidly adapted for specialized use cases. Because training them from scratch demands exorbitant computational resources and specialized engineering, the standard industry practice is to adopt an existing foundation model and adapt it via prompt engineering, RAG, or fine-tuning.

Frequently Asked Questions

Should my company build its own foundation model?

Generally, no. Training a foundation model from scratch requires millions of dollars in compute infrastructure. Instead, organizations should leverage existing models and customize them via fine-tuning or RAG to meet specific business needs.

What is the difference between a foundation model and a fine-tuned model?

A foundation model is the broad, general-purpose base trained on massive datasets. A fine-tuned model starts as a foundation model but undergoes additional, targeted training on specialized data to excel at specific tasks.

Are open-weights foundation models as capable as proprietary ones?

The performance gap is continually shrinking. Open-weights models like Llama 3 and Mistral frequently achieve parity with proprietary models on many benchmarks, particularly when fine-tuned for a specific domain.