Inference
Inference is a model answering — using its already-trained weights to produce output. Training is how those weights were learned; inference is every use after that.
A model has two lives. Training happens once, at the publisher, on enormous clusters — that is where the weights are learned. Inference is everything afterwards: every chat, every completion, every API call replays those frozen weights against new input. When this site says a model "runs" somewhere, it means inference.
What inference demands is exactly what the fit pages measure: enough memory to hold the weights and the growing context cache, and enough memory bandwidth to stream a large share of those weights for every single token generated. That is why generation speed tracks memory speed, and why the same model feels different on different machines.
The whole local-versus-cloud question is a question about inference: pay a provider per token to run it on their hardware, or hold the weights yourself and pay in electricity and patience instead.
Where you see it on the radar
The cost calculator prices the same model's inference three ways — cloud API, your own hardware, and a rented GPU — from measured prices and stated assumptions.
Price your inference three ways →Related terms