Why Is AI Rewriting Code in Rust? (We Converted a Neural Network) : Part 1

Summary

The video demonstrates converting micrograd, Andrej Karpathy’s small Python neural-network engine, into Rust with help from an AI coding agent. The Rust version reproduces the Python program’s results exactly while running much faster.

Why Code Is Moving to Rust

• Performance: The video points to Rust-based Python ecosystem tools such as UV and Ruff, which claim substantial speed improvements over the tools they replace.
• Memory safety: It cites Microsoft and Google figures indicating that memory-safety issues account for a large share of serious vulnerabilities. Rust’s ownership and borrowing rules prevent many such errors before a program runs.
• Adoption: The video mentions Google’s reported decline in Android vulnerabilities classified as memory-safety bugs—from 76% in 2019 to 24% in 2024—and credits the use of memory-safe languages for new code.
• AI-assisted migration: Rust’s strict compiler can be challenging for programmers, but the video argues that an AI can use compiler errors as feedback and revise its code.

How Micrograd Works

Micrograd represents calculations as a graph of values. During backpropagation, it traverses the graph in reverse to calculate how each value affects the result, then adjusts the network’s weights to learn. The Python implementation is slow because training creates many small objects and repeatedly walks the graph.

Rust Concepts and Conversion Walkthrough

The video explains three Rust concepts:

1. Ownership: Each piece of memory has one owner and is freed when that owner is done.
2. Borrowing: Values can be temporarily shared under rules that prevent simultaneous conflicting access.
3. Shared graph data: Micrograd’s nodes can share children and update the same gradients, which conflicts with Rust’s ordinary borrowing rules. Two possible designs are shared, checked handles or a flat list of nodes referenced by indices.

For the conversion, the presenter instructed the AI not to change the Python files, to avoid unsafe code, and to run tests and a linter after each change. Before translating, the AI noticed that Python’s set-based child ordering caused tiny run-to-run numerical differences. It fixed the order and saved the original results for comparison.

The AI chose shared handles to keep the Rust implementation close to the Python design, prioritizing an exact translation over maximum speed.

Results

• Exactness: All 637 compared values—including losses, accuracies, learning rates, and final weights—matched, with zero differences.
• Speed: For 100 training steps on 100 points and a 337-parameter network, the Python version took about 69 seconds; Rust took about 0.5 seconds, roughly 137× faster.
• Larger workload: Python hit a recursion error on the larger network unless its recursion limit was raised. With the limit raised, the run was estimated to take about an hour and a half; Rust completed in 22 seconds, using an iterative backward pass.
• Alternative design: A separate flat-list Rust implementation, built off-camera, completed the smaller run in about 0.04 seconds—around 12× faster than the shared-handle version. The video notes that performance gains come from both the language and the design.
• Safety caveats: The final library contained no unsafe code, and a deliberate unsafe-code test was rejected by the compiler. However, Rust cannot prevent incorrect logic, and programs can still fail at runtime. The video also cautions that translating a large C++ codebase is much harder than converting this small Python project.

Main Speakers and Sources

• Main speaker: The video’s narrator and presenter, who conducts the conversion walkthrough.
• Demonstrated source: The AI coding agent used for the translation.
• Technical project: Andrej Karpathy’s micrograd.
• Organizations and research cited: Microsoft’s security and research teams; Google’s Chrome and Android teams; DARPA’s Tractor program; and MIT Lincoln Laboratory.