Put in your traffic, token counts, and uptime. The calculator shows the API bill, the GPU bill, and exactly where the estimate can go wrong.
Peak capacity for synchronous traffic
Elastic runtime for offline batch jobs
KV cache and quantized model fit
Editable prices and measured throughput
Nothing is sent anywhere. Every price and speed assumption stays visible, so you can swap in numbers from your own bill or benchmark.
On-demand US pricing shown below. Region, availability, networking, storage, commitments, and support can materially change the bill.
You pay only when tokens flow, absorb bursts without idle replicas, and get mature reliability. Self-host when traffic is steady, model control or data locality matters, and your team can own the serving stack.
Queue work, use continuous batching, rent GPUs only for the processing window, and consider interruptible capacity. First compare against the API provider’s own 50%-off batch tier.
Route predictable, privacy-sensitive volume to your deployment and overflow to a token API. This avoids paying for rarely used peak capacity while retaining model control.