Guide · compute
Buy,or rent?
Renting a GPU by the hour is the part of local AI most people have never tried, and the part most often priced wrong in both directions. This guide explains how it works, puts a measured hourly rate next to a sourced purchase price, and says plainly when each path wins.
The short answer
- Rent the spike, own the routine. A job you run a few times a month is cheaper rented for years; a model you talk to every day is cheaper owned within months.
- Memory decides what you can rent, exactly as it decides what you can buy: a model that needs 80 GB needs an 80 GB card in both worlds. The finder tells you the number before you spend anything.
- The consumer cards that exist on both sides of the ledger — RTX 4090 and RTX 5090 — break even at 1,995–2,139 rented hours at today's medians. The table below shows the arithmetic.
How GPU rental works
A GPU marketplace lists machines that other people own — a data-center rack or a well-cooled basement — with a price per hour set by the host. The big clouds sell the same thing at list prices; the marketplace we measure, Vast.ai, is where the hourly rate is set by supply and demand, which is why it moves and why we record it every hour.
- Pick a class by memory. The model file plus its context cache must fit the card. The finder computes that for every tracked model, and the fit ladder on each GPU page shows the verdicts.
- Pick an offer. On-demand means the machine is yours until you stop it. Interruptible offers are cheaper but can be reclaimed when someone outbids you — fine for a batch job you can restart, wrong for an afternoon of interactive work.
- Start it with a container image — a runtime such as Ollama or vLLM, or a notebook — and pull the model onto the machine. A 20–40 GB download takes minutes on a data-center line, and those minutes are billed.
- Run, then stop. Billing runs while the instance exists, idle or not. Disk space you keep between sessions and data you move out are usually billed separately; the offer card says how much.
- Your data was on someone else's machine. For most experiments that is fine. For anything you would not send by e-mail, it is the reason to own.
The break-even, in measured numbers
Two consumer cards can be bought at a published launch price and rented at a measured rate, so for them the break-even is plain division: how many rented hours cost as much as the card.
| Card | Launch price | Rented, median $/h | Rented hours to match | In human time | Owning still costs |
|---|---|---|---|---|---|
| RTX 409024 GB | $1,599NVIDIA launch MSRP, Oct 2022 source ↗ | $0.80rising59 verified offers | 1,995 h | ≈ 5 years at 8 h/week≈ 16 months at 4 h/day | $0.138/helectricity at 450 W + 100 W |
| RTX 509032 GB | $1,999NVIDIA launch MSRP, Jan 2025 source ↗ | $0.93rising64 verified offers | 2,139 h | ≈ 5 years at 8 h/week≈ 18 months at 4 h/day | $0.169/helectricity at 575 W + 100 W |
The purchase side is the launch price from the vendor's own announcement — street prices move weekly and we refuse to quote one. The rental side is today's median across verified on-demand offers; the range on the rental page shows how far below the median a patient renter gets. Electricity uses the vendor board power plus 100 W for the rest of the machine at $0.25/kWh, the calculator's default — change it there. Not in the table: resale value (favours owning), setup time and storage (favour owning at high hours), and speed — a data-center card and a consumer card are not like for like, and we publish no tokens-per-second figure.
The rented classes, a working month each
Every class we measure, priced for a working month of 160 hours — eight hours on twenty working days — next to how many of the tracked models run comfortably on its memory at 8K context. The counts are the fit engine's estimate from measured file sizes, not a live reading.
| Class | Memory | Median $/h | Working month | Runs comfortably (estimated) |
|---|---|---|---|---|
| RTX 4090 | 24 GB | $0.80rising59 offers | $128 | 22 of 45 tracked models |
| A100 | 40 GB | $0.91steady14 offers | $145 | 29 of 45 tracked models |
| RTX 5090 | 32 GB | $0.93rising64 offers | $150 | 29 of 45 tracked models |
| H100 | 80 GB | $2.67falling18 offers | $427 | 32 of 45 tracked models |
| H200 | 140 GB | $3.98steady13 offers | $636 | 35 of 45 tracked models |
| B200 | 179 GB | $6.88steady8 offers | $1,100 | 35 of 45 tracked models |
Buying an H100 is not a consumer decision. The cards are sold through system vendors and there is no public list price we can verify, so there is no purchase column for the data-center classes. What we can measure is the rental market: at today's median an H100 hour costs $2.67, a working month $427 — for memory that runs 32 of 45 tracked models comfortably.
Rent when
- + The job is occasional — a fine-tune, a batch evaluation, one big experiment.
- + The model needs more memory than any consumer card has. An 80 GB class is a rental in practice, not a purchase.
- + You want to try a size class before spending on hardware. One afternoon answers the question.
- + You are early. Nothing teaches what you actually need like a month of hourly receipts.
Buy when
- + You run models daily. Owned hardware has no marginal cost — past the break-even, every hour is electricity only.
- + Your data must not leave your machine.
- + You want zero setup: a local runtime is always on and already holds your models.
- + You already own a capable card. Run the finder before you spend a cent.
Or both
Most people who do this seriously end up hybrid: a 16–24 GB card for the daily models — the budget guide says which — plus a rented 80 GB class for the rare big job. The calculator prices exactly that split for any tracked model.
What to watch out for
- Idle billing. An instance you forgot on a Friday costs the whole weekend. Stop it; keep a disk snapshot if you need it back.
- Storage and transfer. Many hosts price disk per GB-month and outbound traffic separately. A large model pulled fresh every session is time and money; a persistent volume can be cheaper.
- Interruptible means interruptible. The cheaper tier can vanish mid-job. Use it for work you can restart.
- Prices move. The medians here shift day to day; the rental page keeps the daily history so you can see the trend before you commit to a pattern.
- Speed is not in this guide. We publish no tokens-per-second figure for any hardware; the methodology says why.
- Privacy. Whatever you upload sits on hardware someone else controls. If it must stay private, it stays home.