Compute Market Glossary

Core terms

GPU marketplace: A venue where buyers rent accelerator capacity from one provider, many hosts, or a decentralized supply network. The term can describe a peer exchange such as Vast.ai, a decentralized marketplace such as Akash, or a GPU-first cloud catalog.

GPU-hour: One GPU allocated for one hour. It is a useful starting unit but incomplete without utilization, memory, CPU, storage, network, and failure behavior.

Accelerator: Hardware specialized for parallel compute. In this market it usually means NVIDIA GPUs, but it can include AMD GPUs, TPUs, NPUs, or custom inference chips.

H100, H200, B200, A100, L40S: GPU model names that imply different memory capacity, bandwidth, interconnect, precision support, availability, and price. A model name alone is not a workload guarantee.

VRAM: GPU memory. LLM inference is often constrained by model weights and KV cache memory, so VRAM can matter more than raw FLOPS.

NVLink / high-speed interconnect: GPU-to-GPU communication fabric. Critical for large multi-GPU training and some large-model inference deployments.

Spot / interruptible / preemptible: Discounted capacity that may be reclaimed. Good for checkpointed and fault-tolerant work; risky for user-facing paths.

Reserved capacity: Capacity committed for a term. Useful for steady utilization; wasteful if the workload is uncertain.

MIG: NVIDIA Multi-Instance GPU, a way to partition supported GPUs into isolated instances for smaller workloads.

Dynamic batching: Combining incoming inference requests into batches so the accelerator is used more efficiently.

Continuous batching: LLM-serving scheduling that keeps adding and removing requests during generation instead of waiting for a fixed batch to finish.

KV cache: Attention key/value memory used during transformer generation. It can dominate memory use in long-context inference.

Per-token pricing: A managed model API price based on input tokens, cached input tokens, output tokens, and sometimes priority or batch lanes.

x402: An HTTP-native payment protocol that lets servers request payment and clients, including agents, pay programmatically.

DePIN: Decentralized physical infrastructure network. In compute, it refers to networks that aggregate physical GPU or CPU supply from many operators.

OctaneBench-hour: A Render Network pricing unit based on rendering benchmark performance rather than raw GPU identity.

Metrics worth tracking

Utilization tells you whether rented hardware is doing useful work. Low utilization can make cheap GPUs expensive.

Throughput tells you completed work per unit time: tokens/sec, images/minute, frames/hour, samples/sec, or jobs/day.

p95 latency tells you how slow the slower user-facing requests are. Average latency can hide bad inference economics.

Cost per useful unit is the best buying metric: cost per accepted answer, trained checkpoint, processed image, rendered frame, or completed agent task.