Inference

Inference is running a trained model to get an answer, as opposed to training, which is producing the model in the first place. Every question a user asks is an inference; training happens once, or occasionally.

The distinction matters to a budget because the costs behave differently. Training is a large one-off expense. Inference is a per-use expense that grows with adoption — an AI feature that succeeds costs more every month, which is a pleasant problem only if it was modelled in advance.

Inference is also where latency lives. A model that takes eight seconds to respond is a different product from one that takes eight hundred milliseconds, regardless of how good the answer is, and the design has to account for the wait.