Video summary

Doing The Optimization Research Billion Dollar Companies Won't Do.

Main summary

Key takeaways

Technology

Purpose & Audience

  • The channel has spent ~2 years analyzing how different game engines render GPU graphics pipelines (optimized vs. non-optimized) to identify what “good” and “bad” pipelines have in common.
  • This video shifts from analysis into a “how modern games should be rendered” guide for:
    • Gamers: to recognize/criticize poor optimization choices (and influence market demand).
    • Developers / graphics engineers: to know what to request from engine distributors/licensors.
  • The speaker emphasizes that micro-optimizations can add up; saving even ~0.1 ms can matter when chained across lighting/AA.

Key “Product Feature” / Optimization Framework

  • A pipeline diagram/chart is recommended to be followed while watching (available as an exclusive gift via Patreon).
  • The diagram is structured in three layers:
    1. GPU pass steps per frame (middle layer): which GPU passes to run.
    2. Workflow nodes (top layer): engine/art workflow integration matters as much as raw GPU efficiency.
    3. Issues to avoid / visual goals (bottom layer): what the pipeline achieves and which passes demonstrate poor vs. optimal approaches.
  • Marketing guidance: studios should advertise concrete metrics like hardware clear costs, full-screen overwrites, and “hijack clears,” ideally with resolution/hardware context.

Rendering Pipeline Concepts (Major Sections)

1) GPU Clears & Resource Management (Micro-Optimization)

  • Avoid clearing with full-screen primitives: “full screen triangle/quad for clearing” is claimed to be slower than hardware clears, hurting performance.
  • Prefer overwriting pixels instead of clearing + rewriting, except where overwriting can’t cover the full surface (e.g., emissive-only buffers, dual-normal buffers used only by certain materials).

Introduced / Labeled Techniques

  • “Hijack clear”: piggyback a clear-like overwrite while writing main outputs to additional render targets “for almost free.”

Buffer Behavior Guidance

  • Deferred G-buffers (e.g., base color and normals) generally don’t require clearing if depth is cleared and later passes overwrite precisely.
  • Buffers likely needing hardware clears include:
    • MSAA coverage
    • emissive
    • dual normal
    • any buffers only partially written

2) Frame Compute Shaders (Careful GPU-Driven CPU Replacement)

  • Starting-frame compute shaders are discussed as useful only when they remain very small.
  • Core critique: modern engines try to move CPU work to GPU, but that can bloat GPU pipelines unnecessarily.
  • Motivating example: Unreal Blueprints vs C++ (speaker claims Blueprints can be up to 10× slower).
  • The speaker advocates GPU-driven approaches, but argues against overusing them.

3) Depth Prepass Strategy (Especially for Alpha-Tested Content)

This is one of the biggest technical arguments.

  • Overdraw is a major cost driver in base passes.
  • A depth prepass can reduce overdraw, but the video argues partial alpha-tested prepasses can be counterproductive if configured wrong.

Main Thesis

  • In certain scenarios, disabling the alpha-tested prepass and relying on proper depth-test/write behavior can improve performance substantially.

Quantified Claims from Tested Assets

  • A case where alpha-tested prepass + base pass cost ~13% more than a configuration without the prepass, depending on whether depth writing is enabled.
  • Larger examples show significant differences too, with the claim that prepasses become less beneficial when they don’t write useful depth for main shading.

Quad Overdraw Complication

  • GPU pixel shaders execute in 2×2 “quads.”
  • A more honest metric is “quad invocations,” not just pixel shader invocations.
  • Dense geometry prepasses can effectively multiply quad overdraw, harming performance.

Marketing / Debug Guidance

  • Show customers “depth surface area” from prepasses using debug views.
  • Ensure the prepass doesn’t do expensive per-pixel work:
    • no alpha testing
    • no displacement sampling

4) API Discussion (DX11 vs DX12 vs Vulkan)

  • Vulkan is described as “pro-consumer” and “developmently ideal,” but the speaker argues optimization debugging tools are less mature than DX11 tools.
  • Limited testing concerns:
    • Results suggest DX11 shading can be faster than DX12 for pixel shading (speaker claims 2× faster in Unreal tests).
  • Emphasis: testing was within Unreal Engine implementations, so Vulkan/DX12 behavior could be engine-specific.

Base Pass / G-Buffer / Workflow Design Decisions

G-Buffer Precision & Packing

  • 32-bit depth is chosen (reverse-Z implied), linked to hardware quirks and quality.
  • Mentions:
    • Reverse-Z rendering (referenced via NVIDIA material).
    • Normal encoding quality (e.g., 10-bit normals vs. higher precision).
    • G-buffer packing for multiple uses (storing flags/lighting channel info in limited bits, e.g., RGBA/alpha bits).
  • The G-buffer layout is described as undecided, but with intended buffer roles outlined.

New Material/BRDF Workflow

  • Proposes reworking classic metallic/specular PBR channels:
    • Metallic blending should use a dual material profile ID so paint thinning/blending doesn’t break physically-correct metal behavior.
    • Specular channel packing: encode index of refraction in a higher-precision material profile and scale appropriately into the specular channel.
  • The pipeline is positioned as more artist-workflow-friendly while constrained by GPU throughput realities.

Artist / LOD / Transition Guidance

  • Foliage/asset guidance reframed:
    • not “polycount matters most,” but overlap and quad overdraw matter most.
  • LOD transitions:
    • uses ALU-based jittered Bayer matrix (speaker critiques some transitions for popping/shadow issues).
  • Mentions using SV coverage (sample coverage tracking) and how multisampling affects cost multipliers.

Lighting Passes (Deferred, Shadows, GI, Reflections)

Shadows: Non-Cascaded / Perspective Shadow Maps

  • Introduces perspective shadow maps to reduce shadow aliasing by rendering shadow maps in light space closer to what the player actually sees.
  • Claims:
    • perspective shadow maps reduce wasted depth shading by scaling resolution within a view-dependent bound.
  • Advocates:
    • scaling the shadow field of view with player perspective
    • keeping total shadow map resolution under an upper threshold (speaker mentions “48 squared” as an expensive waste point)
  • Critiques:
    • heavy cascade / alpha-tested shadow costs
    • suggests approximated techniques instead of expensive exact ones

Deferred Lighting Optimization: Stencil Volume Lighting

  • Argues per-light GPU compute “single draw” approaches are slower than stencil volume lighting.
  • Describes the classic stencil trick:
    • render light volume into stencil, then run lighting shader only where stencil indicates it affects pixels.
  • Notes MSAA interaction:
    • stencil volumes can get slower under MSAA, so early-stencil testing is needed.

GI / Indirect Lighting: Avoid RSM

  • Suggests indirect lighting built from:
    • tracing shadow map information first, then falling back to world-space tracing.
  • Claims reflective shadow maps (RSMs) are too expensive (about 0.3–0.6 ms each frame) and unnecessary if you already have other shadow/baked data.

Reflections

  • Similar philosophy to GI:
    • use non-ray methods first to reduce world-space tracing needs.
  • Considers simplified representations and dynamic cube maps as options.

MSAA + Anti-Aliasing Strategy (SMAA + “SMAA4X” Style Temporal AA Replacement)

Anti-Aliasing Goal

  • Emphasizes MSAA is essential, but:
    • MSAA doesn’t resolve jagged edges via temporal accumulation like TAA.
    • instead, MSAA targets temporally unstable edges; morphology handles remaining aliasing.

SMAA4X / SMAA2TX Temporal Logic

  • Advocates a modern temporal direction:
    • Two-frame blending: blend current frame with previous frame’s SMAA output, not infinite-history accumulation.
  • Claimed benefits:
    • less ghosting/blur
    • fewer instability artifacts
  • Critiques many TAA implementations:
    • accumulation history buffers can cause smearing, ghost trails, ringing/halos, and expensive history buffers.

Temporal Issues Addressed

  • Particle / visual effect ghosting: solved via responsive masks authored by artists.
  • Foliage ghosting: attributed to missing motion vectors for alpha-tested/deformed foliage; suggests motion vectors are feasible and needed.
  • Texture blurring:
    • proposes edge-only temporal blending
    • techniques to prevent full-frame texture jitter contamination
    • mentions CMAA/TSCMA ideas, SV coverage/masks, and sample location control

Texture Quality Mitigation

  • Isolate where temporal blending is allowed (edge mask based blending).
  • Mentions potential programmable sample locations to better match MSAA patterns without jittering textures/lighting unnecessarily.

Motion Blur (Velocity Packing + Perceptual Correctness)

  • Motion blur is treated as requiring careful velocity handling, separate from simplistic camera-only blur stereotypes.
  • Pipeline described:
    • build velocity (packed into RG11B10) and use it to blur
    • use tiled/max-tile resources with caution: tile dilation can cause blocky artifacts/outlined ghosts
  • Proposed direction:
    • single-direction motion blur behavior (more like natural peripheral perception) rather than sampling opposite directions
    • artist-controlled camera motion blur cutoff based on distance with a smooth U-shaped mask
  • Also argues for plausible blur direction to avoid uncanny “object centers blur both ways” results.

Post-Processing & Composite Effects

  • Composite effects should use half-resolution buffers to control cost.
  • Critiques native-resolution composite pipelines as choking performance.
  • Uses half-res depth and RGBA buffers for composites (e.g., lens rain and related effects).
  • Mentions:
    • responsive masks in alpha to improve temporal anti-aliasing friendliness
  • Notes tone mapping and DOF/bloom ordering needs refinement:
    • “DOF after AA, AA after tone mapping” is cited as the proposed order, but currently “needs refinement.”

Tone Mapping (Contrast Toe + Hue Preservation; GT7-Like Tone Mapper)

  • Heavy focus on tone mapping curves and perceptual correctness.
  • Central praise for GT7-style tone mapping:
    • mostly-linear region + contrast toe (darker-to-black transition) + highlight shoulder compression
    • creates deeper blacks, richer contrast, and vibrant colors without oversaturation
  • Critiques “display mapping” approaches:
    • tends to wash out colors; hue shifts differ
  • Also discusses:
    • autoexposure importance and calibration based on environmental values affecting “naked eye perception”
    • white balance automation and how hue-preserving tone mapping can look warmer without it
    • tone-mapped output and color management choices (potential exploration of BT2020)

UI Rendering

  • Notes UI needs gamma correctness and can benefit from separate render targets.
  • Mentions examples of engines that implement multiple gamma settings for different displays.

Main Speakers / Sources (Implied by Subtitles)

  • Primary speaker: the video narrator/host (“me and my team” throughout)
  • Referenced companies / games:
    • Unreal Engine / Epic Games
    • Crytek (Crysis 3)
    • Guerrilla Games
    • Fox Engine / Kojima (Day(s) Gone)
    • Digital Foundry (mentioned critically as an influencer/source of claims)
    • NVIDIA (reverse-Z, tone mapping references, DL/AA/upscaler references)
    • Striking Distance Studios (Kalisto Protocol topics like character/eye/SSS)
  • Referenced individuals:
    • Brian Carris (called out regarding Unreal’s TAA failures / implementation context)
    • John Linnaman (mentioned for promoting/“gaslighting” claims about lighting/gib reference)
    • Alan Wake 2 (referenced as a title/context for pipeline discussion)

Original video