What Is Tokens per Watt?
Tokens per watt measures how many tokens an inference system produces for each unit of energy it consumes. It ties the output of an AI system (tokens generated) directly to its largest operating input (power), which makes it a clean way to compare the efficiency of GPUs, serving stacks, and whole data centers.
The metric rose to prominence with the token factory framing, where NVIDIA’s Jensen Huang presented the relationship Revenue = Tokens per Watt times Available Gigawatts. The point for operators is blunt: once you are serving inference at scale, the number of tokens you can produce per watt sets a ceiling on both your capacity and your margins.
Why It Matters Now
Training is episodic. A run finishes and the cluster frees up. Inference is continuous: every request produces tokens around the clock, and Deloitte projects inference will account for roughly two-thirds of all AI compute in 2026. When a workload runs continuously, energy stops being a background cost and becomes the thing that determines whether the workload is profitable.
That shifts what infrastructure optimizes for. A GPU tuned for raw training throughput can have very different tokens-per-watt economics than one tuned for inference, and the serving software matters just as much as the silicon. Techniques like continuous batching, paged attention, and quantization raise tokens per watt without changing the model itself.
Improving Tokens per Watt
The largest levers are matching the model to the right GPU, keeping accelerators busy through batching, and reducing wasted memory movement with efficient KV cache handling. Autoscaling matters too: idle GPUs still draw power, so scaling capacity to actual request volume protects the metric.
Saturn Cloud is the platform layer where these levers are settings rather than a build project. It matches workloads to the right GPU across providers, serves with high-throughput inference engines, and autoscales to real demand, so operators can drive tokens per watt up across the GPU capacity they already run.
