← Back to the AI Glossary 📖 Infrastructure & Ops

Inference

The process of actually running a trained AI model to generate a response — as opposed to training, which is building the model in the first place.

Every time you send a message to an AI chatbot and get a reply, that's one inference request — training already happened long before, once, ahead of time.
One of 60 free AI glossary terms
Plain-language definitions for the AI jargon you'll actually run into — no email needed, ever, for this section.
Browse the Full Glossary →
More in Infrastructure & Ops
Latency
The delay between sending a request to an AI model and receiving a response — lower latenc...
Prompt Caching
A technique where an AI provider temporarily stores parts of a prompt it has seen before,...
API (Application Programming Interface)
A defined way for one piece of software to talk to another — in AI, it's how developers se...
Cold Start
The extra delay before an AI system responds to its very first request in a while, as oppo...