Weights

Executive Definition: Weights — The internal numeric parameters in a neural network that are optimized during training to map inputs to accurate predictions.

Weights are the mathematical values that store a neural network’s learned knowledge. During training, optimization algorithms iteratively adjust these millions or billions of parameters to minimize prediction errors. When a model is ‘released’ or downloaded, the weights are the core files being shared.

Frequently Asked Questions

What does it mean when a model ‘releases its weights’?

It means the trained model files are publicly available for download. Anyone can run the model locally without relying on an API. Llama and Mistral are examples of models with released weights.

How large are model weight files?

It depends on parameter count and precision. A 7B parameter model at 16-bit precision is about 14GB. A 70B model can be 140GB+ at full precision, though quantization reduces this significantly.

Can weights be modified after training?

Yes, through fine-tuning. You continue training the model on new data which adjusts the weights. PEFT methods like LoRA add small adapter weights without changing the original ones.