Self-hosting an LLM is cheaper than an API only above a break-even volume, and it is higher than most people guess. You divide the fixed cost of the GPU (rented or owned) by the tokens you actually push through it, and compare that to the API’s per-token price. At low or bursty volume the API wins easily, because it charges only for tokens you use while your own GPU bills every hour it is powered on. The self-hosting vs API calculator finds the crossover for your own numbers.
The core trade-off
An API is pure variable cost: send a million tokens, pay for a million tokens, send nothing and pay nothing. Self-hosting flips that. You pay a fixed hourly rate for a GPU whether it is flat out or idle, and the price per token depends entirely on how many tokens you manage to run through it.
That single fact drives everything. A rented H100-class card might cost roughly $2 to $3 per hour (rates move constantly and vary by provider and commitment). Owning one is a capital purchase plus power, cooling, and rack space amortized over its life. Either way the meter runs 24/7. The only way to make each token cheap is to keep the card busy.
Finding the crossover
Break-even is where fixed cost per token equals the API price per token. Work it in three steps:
- Fixed monthly cost. Take your GPU hourly rate times 730 hours a month. A single card at $2.50/hour is about $1,825/month.
- Realistic throughput. Estimate tokens per second under your model and quantization, times utilization. A server that could do 2,000 tokens/second but runs at 30% utilization effectively delivers 600.
- Effective cost per token. Divide monthly cost by monthly tokens. Compare to the API’s blended input+output price per token.
The trap is step two. A raw GPU-hour rate assumes 100% utilization. Real traffic is spiky — busy during work hours, near-dead overnight — so the honest number is far lower.
Worked example: where the lines cross
Say a self-hosted setup costs $1,825/month for the GPU and, at 30% real utilization, processes 600 tokens/second. That is roughly 1.56 billion tokens a month. Effective cost:
| Line item | Value |
|---|---|
| GPU cost / month | $1,825 |
| Effective throughput | 600 tokens/sec |
| Tokens / month | ~1.56 billion |
| Self-host cost / 1M tokens | ~$1.17 |
Now compare to APIs. A cheap small model might run $0.20 per million tokens blended; a mid-tier model $2 to $5; a flagship $10 or more (all approximate and always changing).
- Against a $0.20/1M cheap model, self-hosting at $1.17/1M is roughly 6× more expensive. You never break even — the API is just cheaper.
- Against a $5/1M mid-tier model, self-hosting is about 4× cheaper at this volume, and the crossover sits far below 1.56 billion tokens.
So the answer is not “self-hosting is cheaper” or “the API is cheaper.” It is: cheaper relative to which model, at what utilization. Push utilization to 70% and your effective self-host cost drops to about $0.50/1M, moving the crossover way down. Let the card idle at 15% and it doubles to $2.34/1M, pushing break-even out of reach.
The costs a GPU-hour rate hides
The rented rate is the floor, not the total. Add these before you trust a break-even number:
- Idle time. The biggest one. Bursty traffic means you pay for hours that move zero tokens. This is baked into the utilization figure above — get it wrong and every other number is fiction.
- Engineering hours. Someone has to run inference servers, batching, model updates, and on-call. That salary cost often dwarfs the GPU bill for a small team.
- Redundancy and headroom. One card is a single point of failure. Real uptime means a second card and spare capacity for spikes, both mostly idle by design.
- Cold starts and model loading. Large weights take time to load into VRAM. Scale-to-zero to save money and you pay in latency; keep warm and you pay in idle hours.
- Throughput reality. Memory-bound or heavily quantized models run slower than the headline. Fitting a model on a smaller card via quantization can cut throughput — see GGUF quantization sizes explained for how that trade works.
So when does it actually make sense?
Self-hosting wins when you have high, steady volume against a model whose API equivalent is expensive, plus a reason beyond price — data residency, privacy, a fine-tuned model, or predictable latency. If your traffic is bursty, low-volume, or served fine by a cheap hosted model, the API almost always costs less and saves you the ops burden.
Before committing hardware, run both scenarios. Plug your token volume, GPU rate, and honest utilization into the self-hosting vs API calculator to see the crossover, then check whether simply moving to a cheaper hosted model with the model switch savings calculator closes the gap without any infrastructure at all. Often it does.
