Also called: base training
The first, giant training run where a model learns general patterns from a broad dataset.
Why it mattersMonths of compute and a nine-figure bill. Everything after this is comparatively cheap.
See also Fine-Tuning, Foundation Model