Offloading
Offloading splits a model between graphics memory and ordinary RAM when it does not fully fit — the model runs, just noticeably slower.
Local runtimes place a model layer by layer. When the graphics card fills up, the remaining layers go to system RAM and run on the CPU. Nothing crashes; every token simply has to wait for the slow part of the machine.
How much slower depends on how much spilled. A few offloaded layers can be tolerable; running half a model from system RAM turns an interactive chat into a coffee break, because generation speed follows the slowest memory the model touches every token.
Offloading is a fine tool for occasionally using a model one size above your hardware, and the wrong default for daily work. The honest framing: it is a trade you should choose knowingly, not a fine print behind an optimistic compatibility claim.
Where you see it on the radar
Our fit labels say "Offload required" instead of counting a spilled model as a fit — and rental recommendations never assume offloading at all.
See where your hardware's real limit is →Related terms