Inside an AI Factory – Power, Cooling, and GPUs Explained

Summary

Kian presents a 16-chapter course explaining how an AI data center turns grid power into model output. The course follows three paths: electricity from the grid to the GPU, data between chips and systems, and heat from the chips back outside.

Its central point is that more GPUs or more power capacity do not automatically produce more usable compute. Power delivery, cooling, networking, workload scheduling, construction, and reliability all have to work together.

Key Technological Concepts

• Electricity and power delivery: The primer introduces voltage, current, resistance, Ohm’s law, and power. Because resistive heat loss rises with the square of current, transmitting power at higher voltage can reduce losses. The course also explains AC versus DC, three-phase AC, power factor, transformers, rectifiers, inverters, and electrical protection.

• Rack and chip power: Power is stepped down through campus and building equipment, then distributed to rack power shelves and power-supply units (PSUs). PSUs convert AC to DC; voltage-regulator modules (VRMs) further regulate power near the GPU. Local capacitors and battery backup units help buffer rapid changes in demand. The presenter emphasizes that short, low-resistance paths matter because chips operate at low voltage and require high current.

• Backup power and reliability: Online UPS systems use rectifiers, DC links, inverters, batteries, and capacitors to keep loads powered during interruptions while generators start. The course compares redundancy approaches such as N, N+1, N+2, 2N, and 2N+1, and discusses the risks of shared buses and bypass paths. It distinguishes infrastructure redundancy tiers from availability targets (“nines”).

• Rack-scale computing: A server tray contains CPUs, memory, storage, networking, and GPUs. Models are loaded from storage to system memory and then to GPU memory. The course uses Nvidia’s GB300 NVL72 rack as a recurring example, describing its compute trays, power shelves, bus bars, and NVLink switches. It also notes that rack power demand is rising and that future systems may require denser or wider rack designs.

• Training and inference workloads: Training involves forward and backward passes, gradients, and optimizer state, so it typically requires more memory than inference. Inference is divided into prefill, which processes the prompt and builds the KV cache, and decode, which generates output tokens. The presenter explains throughput, time to first token, inter-token latency, and per-user interactivity, as well as how context length increases KV-cache requirements.

• Scheduling and power variation: GPU loads can fluctuate during training, particularly when systems exchange or synchronize data. The course shows why a workload’s time-varying power profile matters, not just its average consumption. Batching, staggering jobs, and shifting flexible work can improve utilization or reduce peak demand. Continuous batching is contrasted with fixed batching.

• Networking and interconnects: The course separates scale-up communication within a rack from scale-out communication between racks and sites. It discusses NVLink, InfiniBand, Ethernet/RoCE, switch topologies such as leaf-spine, copper versus optical links, co-packaged optics, and optical circuit switches. Network bandwidth and latency affect both model loading and distributed training.

• Cooling and heat rejection: Modern high-density racks require liquid cooling because liquids can carry more heat than air. Cold plates collect heat from chips, and a coolant distribution unit (CDU) transfers it from the equipment loop to a separate facility loop. Rear-door heat exchangers can capture heat that escapes into rack air, while immersion cooling is presented as another approach. The course compares dry coolers, air-cooled chillers, evaporative cooling towers, and water-cooled systems, including trade-offs in electricity use, water consumption, climate suitability, and cooling capacity.

• Site selection and power supply: Data-center projects must account for grid connections and interconnection queues, substations, fuel or local generation, transmission equipment, fiber access, land conditions, water availability, and construction timelines. The course compares grid supply, grid-parallel generation, export-only arrangements, and behind-the-meter or islanded power. It also contrasts simple-cycle gas turbines, which can respond faster, with more efficient combined-cycle turbines, which are better suited to sustained loads.

• Construction and operations: The course covers greenfield and brownfield development, prefabricated equipment skids, factory testing, commissioning, rack handling, and phased deployment. Operational examples illustrate automated cooling controls, workload shifting during grid constraints, recovery from power or cooling failures, and the importance of removing single points of failure.

• GPU-cloud economics: The final business discussion compares on-demand, spot, and committed-capacity contracts; managed inference versus bare-metal GPU rentals; and how utilization, electricity costs, contract length, and financing affect profitability. The presenter describes contracts and capacity backstops as tools that can help finance GPU infrastructure, while noting that unsold capacity remains a risk.

Examples and Analysis

The course applies its concepts to real or illustrative cases, including the Abilene/Stargate campus, SpaceX/xAI’s Colossus sites, Microsoft Fairwater, Google’s cooling controls and grid-demand response, Meta’s training and rack designs, Cloudflare’s multi-site outage, and a Toronto cooling disruption.

These examples highlight recurring trade-offs: bringing compute online quickly versus maximizing efficiency, adding redundancy versus managing cost, and matching facilities to the workloads they will actually run.

Guide and Tutorial Content

This is an educational course rather than a product-review video. It includes:

• A fundamentals primer for viewers unfamiliar with electricity and computing concepts.
• Sixteen chapters covering topics from power fundamentals through networking, cooling, construction, operations, and economics.
• Worked calculations, diagrams, case studies, and knowledge checks.
• Slides, written explanations, and a glossary linked from the course homepage.
• Suggested further reading, including The Electricity Wars, the presenter’s chips course, and cited SemiAnalysis materials.

Main Speaker and Sources

• Main speaker: Kian, the course presenter.
• Publisher: freeCodeCamp.org.
• Sources and references cited: SemiAnalysis articles and podcasts; company materials and examples from Nvidia, Google, Microsoft, Meta, Crusoe, Oracle, and other data-center and infrastructure vendors; the Open Compute Project; and technical materials from equipment manufacturers.