← Back to the AI Glossary 📖 Training & Fine-Tuning

Pre-training

The initial, most expensive phase of training an AI model, where it learns general language patterns from a massive, broad dataset before any specialisation happens.

This is the phase that costs the huge sums of money and compute you hear about in AI news — often hundreds of millions of dollars for a frontier model.
One of 60 free AI glossary terms
Plain-language definitions for the AI jargon you'll actually run into — no email needed, ever, for this section.
Browse the Full Glossary →
More in Training & Fine-Tuning
Synthetic Data
Training data generated by an AI model itself, rather than collected from real-world sourc...
Fine-Tuning
Taking an already-trained model and training it further on a smaller, more specific datase...
Knowledge Cutoff
The date after which an AI model has no training data — it genuinely doesn't know about an...
Reinforcement Learning from Human Feedback (RLHF)
A training technique where human reviewers rate a model's responses, and the model is adju...