Key facts
The formula is intentionally transparent and editable; it is not a vendor benchmark.
Batch size improves throughput but can add latency, so the result should be compared with p95 targets.
Long context reduces effective throughput because prefill and KV cache pressure matter.
Cost/request
$0.00
Requests/hour
$0.00
Seconds/request
$0.00
$/1M output tokens
$0.00
Formula notes
How to interpret the result
Treat this page as a planning worksheet. Static examples and transparent formulas are useful for narrowing options, but real procurement still needs source quotes, workload benchmarks, storage and egress estimates, and failure-mode checks.
For buyer workflow context, read Buyer Guide, How Compute Is Priced, and GPU Cloud Price Comparison.
FAQ
Is this an exact LLM inference benchmark?
No. It is a planning calculator. Real throughput depends on model architecture, quantization, kernels, KV cache, speculative decoding, batching policy, hardware, and server implementation.
Why does context length change the result?
Longer prompts increase prefill work and KV cache pressure. That can lower useful tokens per second even if the GPU-hour price is unchanged.
Can I compare managed API token prices with this?
Yes, but compare at the task level. This calculator estimates self-hosted infrastructure cost, while managed APIs include operations, reliability, model quality, safety systems, and product overhead.