How a cluster is connected

Decision supported

Which fabric assumptions belong in the quote and design review?

Primary output

Planning BOM, traffic tradeoff, and failure domain

Model boundary

Illustrative two-tier designs; vendor port allocation may differ

Counts are planning estimates under the displayed endpoint, radix, rail, oversubscription, and fabric-plane assumptions—not a vendor design or cabling schedule.

Topology diligence

Training traffic has a shape, and rails can be wired to it. In the illustrated mapping, collective channels keep corresponding GPU ranks on dedicated rails and use the high-bandwidth NVLink domain for intra-node movement. Actual NCCL algorithms and channel mappings are topology-, message-, and workload-dependent; NCCL can select rings, trees, NVLS, and other algorithms.

The cost gap is countable only after the design assumptions are explicit. The calculator uses a simple two-tier Clos planning rule for fat-tree and groups servers under each rail switch. When a rail needs more than one switch group, connections between those groups are not counted. It is intended to expose quote assumptions, not replace the supplier's port map.

Rails trade generality for an expected traffic pattern. Cross-rank traffic such as mixture-of-experts all-to-all routing, embedding lookups, parameter-server flows, storage, and data loading may need an intra-node hop before crossing the appropriate rail. That path can consume NVLink capacity shared with collective traffic. Measure all-to-all as well as all-reduce when the cluster will serve mixture-of-experts or other communication-heavy inference workloads.

Failure modes are shaped by the wiring and its redundancy. In the single-plane toy case, a fat-tree leaf failure removes its attached endpoints while a rail-switch failure removes one rail across many nodes. Whether jobs stall or reroute depends on redundant planes, routing, scheduler placement, and collective configuration. Both examples are mapped onto one physical row on the pod page — see the failure stripe on the pod page →

The toy cluster shows one channel and one cross-rank pair so the wiring stays readable. Use the controls for a planning estimate, then ask the supplier for the actual logical topology, physical port map, redundancy behavior, and measured collective results.