THE COMPUTE DESK

A practical field guide · By Alex Yen

AI compute, from
hardware to economics.

A GPU’s hourly price is only the beginning. What fits in memory, how machines connect, and how much capacity you use all shape the cost of getting work done.

Start with one number / A worked example

The same GPU.
A different effective cost.

Suppose you rent eight GPUs at $3 per GPU-hour for 30 days. The bill is $17,280. At half utilization, you spread that bill over half as many hours of useful work.

Change utilization to see the cost per productive GPU-hour.

10%100%
Billed capacity-time8 GPUs × 720 hours
ProductiveIdleOne square = 1% of billed time
Price per rented GPU-hour$3.00
Cost per productive GPU-hour$6.00
$17,280 ÷ 2,880 productive GPU-hours = $6.00

Illustrative model. Utilization here means the share of rented time doing useful work, at unchanged throughput. Fixed rental cost only; storage, networking, and other charges are excluded. This is not a live quote. Model a capacity commitment →

The guide

Understand the constraint. Then test the decision.

01 / Hardware

First, know what
you are renting.

A GPU does the model’s calculations. Its memory holds the model and the data it is working on. Servers supply power, storage, and connections; racks combine them into larger systems.

A faster chip helps only if the job fits and the rest of the system can keep it supplied.

Inside a 72-GPU rack →Compare accelerator specifications →From one rack to a cluster →
Conceptual accelerator
Drag to rotate
Compute

The central dies perform calculations.

Memory

Nearby packages keep data close.

Schematic geometry, not a production layout. Explore the rack model for a specific GB200 NVL72 configuration.

02 / Networks

Chips work together.
Communication takes time.

In distributed training, GPUs calculate part of a result and exchange information before the next step. Some communication can happen alongside computation. The rest adds time to the step.

That is why a network constraint can leave expensive processors waiting.

Model time spent communicating →Compare two network layouts →
An illustrative training stepSeconds
01013
10 + 3 = 13 seconds23% of the step is waiting

If unhidden communication doubles to 6 seconds, the same calculation takes 16 seconds per step.

Illustrative timings, not a hardware benchmark. The network model lets you change bandwidth, message size, and overlap.

03 / Economics

Commit to the work
you expect to use.

Reserved capacity costs money even when it sits idle. On-demand capacity can cost more per hour but lets you pay only when you need it.

The useful question is how often each additional GPU will be needed. Stable demand and occasional peaks call for different commitments.

Find a capacity commitment →Explore the historical price atlas →

One additional GPU / Illustrative rates

Reserved$2/hour

Paid every hour

On demand$6/hour

Paid when used

$2 ÷ $633⅓%break-even usage
If this extra GPU is needed more than a third of the time, reserving it costs less under these assumptions. Availability, minimum terms, and extra charges are outside this example.

04 / Contracts

A service promise needs
a measurement rule.

“99.9% uptime” leaves questions open. Is availability measured across the whole fleet or for each GPU? Which failures count? Is the remedy a refund, a credit, or the right to leave?

The same outage can produce different remedies under different definitions.

Compare contract exposure →

Make the promise testable

  1. 01
    Define the unit

    A GPU, a job, or the entire service?

  2. 02
    Count the interruption

    Which failures and exclusions apply?

  3. 03
    Calculate the remedy

    What fee does the credit percentage apply to?

Use the contract model to compare illustrative clauses. Actual remedies depend on the signed agreement.

The working models

Choose a question.

Each model includes its assumptions, sources, and limits.

01 / HardwareRack anatomy

Inspect trays, memory, and interconnects.

 02 / HardwareCluster layout

Follow connections between racks.

 03 / HardwareAccelerator comparison

Compare capacity, bandwidth, and power.

 04 / NetworksCommunication time

Change the network; inspect the wait.

 05 / NetworksNetwork topology

Compare paths, switches, and link counts.

 06 / EconomicsCapacity planning

Balance a commitment against demand.

 07 / EconomicsPrice atlas

Compare the archived July 2026 observations.

 08 / ContractsContract exposure

Apply different terms to the same outage.