How Compute Is Priced

The unit tells you what market you are in

The GPU market uses different pricing units for different jobs. Training buyers usually start with $/GPU-hour, because the work keeps one or many GPUs busy for long stretches. Inference buyers usually care about $/1M input tokens, $/1M output tokens, $/request, latency, and cache hit rates. Rendering markets can use benchmark units such as Render Network's OctaneBench-hour model rather than raw GPU-hour pricing.

A provider price is therefore only comparable after you normalize it. A one-GPU H100 price is not the same as an eight-GPU node price if the workload needs NVLink, local NVMe, high-speed networking, or a particular region. Likewise, a low token price is not cheap if queueing, context length, tool calls, search grounding, or output-heavy prompts dominate the bill.

• $/GPU-hour: best for dedicated training, fine-tuning, batch rendering, simulations, and long-running model servers.
• $/token: best for managed LLM APIs where the provider abstracts the hardware and charges separately for input, cached input, output, priority, or batch lanes.
• $/request or $/tool call: best for agent-facing APIs, small inference calls, embeddings, and x402-style machine payments.
• Benchmark-hours: useful when the market cares about work completed, such as frames rendered, rather than raw accelerator identity.

Spot, interruptible, reserved, and committed prices answer different risk questions

Spot and interruptible capacity is a risk discount. You accept reclaim, interruption, or changing availability in exchange for a lower price. That can be rational for checkpointed training, batch data processing, offline embeddings, or render queues. It is usually the wrong default for interactive inference unless your system can route around interruptions without visible user impact.

Reserved and committed capacity is the opposite trade: you pay for predictability. Lambda publishes 1-Click Cluster pricing from 16 GPUs upward, CoreWeave and hyperscalers sell enterprise capacity, and marketplaces such as Vast.ai advertise reserved tiers. The real comparison is not just hourly price. It is whether the commitment maps to your utilization curve.

If a cluster is 90 percent utilized for months, reserved capacity can be sane even at a higher headline rate than transient spot deals. If utilization is uncertain, reserved capacity can silently become the most expensive option because idle GPUs still meter against the commitment.

The hidden meters are often not hidden, just ignored

Most surprise bills come from ignored meters: persistent disks, network volumes, inter-region transfer, data transfer out, idle serving workers, model cache storage, load balancers, NAT, public IPv4, snapshot storage, and support commitments. GPU marketplaces tend to market the accelerator line item because it is legible. Buyers need to price the rest of the workload path.

For training, storage and data locality can matter as much as the GPU quote. A cheap H100 instance that spends time pulling terabytes across regions may lose the price advantage. For inference, the biggest cost lever may be reducing output tokens, caching repeated context, batching requests, or separating prefill and decode work, not negotiating one more percentage point off the node rate.

How to compare honestly

Start with the workload, not the provider list. A useful comparison says: this model, this precision, this context length, this throughput, this latency target, this region, this fault tolerance, this storage footprint, this egress pattern, and this expected utilization. Then normalize every quote to the cost of completed work.

For training, compute cost per successful checkpoint is more useful than sticker price per GPU-hour. For inference, cost per accepted user request is more useful than raw model token price because retries, moderation, tool calls, rejected outputs, and long responses all change the denominator.

• Model the steady state and the burst state separately.
• Record every source price with the access date, because GPU prices move.
• Separate secure cloud, community cloud, verified host, and unverified host offers.
• Treat egress, storage, support, and observability as first-class meters.
• Benchmark your own model on each stack before moving production traffic.