Why We Haven’t Reached Superintelligence, with David Krakauer

Summary

• What counts as intelligence: David Krakauer frames intelligence as finding efficient ways to solve difficult problems. On this view, intelligence comes in degrees, while life is usually treated as a binary condition. Krakauer argues that even simple organisms display adaptive problem-solving, though its degree and form vary.
• Evolution, information, and brains: Natural selection incorporates information into genomes through differences in survival and reproduction. Krakauer describes this process as slow, especially as organisms grow and generation times lengthen. Brains provide a faster way to gather and use information about changing environments.
• Collective intelligence: Groups can produce abilities that individual members do not have, as with networks of neurons or coordinated animal groups. But a capable group does not automatically make a capable whole: the discussion contrasts emergent coordination with the disorder of crowds and human societies.
• Signals, communication, and language: Animal calls and other signals can convey useful information, such as warnings about predators. The speakers discuss how these differ from human language, which can combine a finite set of elements and rules to express an open-ended range of ideas. Where to draw the line between complex signaling and language remains unsettled.
• Intelligence and consciousness: The conversation distinguishes problem-solving from conscious experience, emotion, and self-awareness, drawing on a definition by consciousness researcher Anil Seth. It also explores how much thinking occurs unconsciously—an ability that predates language—and how practice can turn skills into largely automatic expertise.
• Dreams, insight, and multimodal thought: The episode discusses the story that August Kekulé arrived at the structure of benzene after dreaming of a snake biting its tail. The example is used to consider whether insight can arise through imagery and unconscious processing, with language later helping people explain and share it. Gestures, vision, and other senses are also described as supporting thought.
• Language models and problem-solving: Krakauer argues that large language models begin from language rather than from an organism’s embodied, pre-linguistic experience. They can solve difficult problems, but may do so inefficiently, and success on a benchmark does not by itself establish human-like intelligence. The speakers contrast a model’s statistical heuristics with methods that guarantee correct answers when properly applied.
• AI, tools, and dependence: The discussion distinguishes tools that support and train a person’s abilities from tools that replace them. An abacus and slide rule are presented as tools that can build mathematical intuition; maps can teach spatial orientation, whereas turn-by-turn GPS may allow users to navigate without learning those skills. The concern is that outsourcing abilities may weaken them and make people dependent on technology providers.
• Energy, information, and entropy: The speakers compare AI’s use of large data centers to the Industrial Revolution’s use of fossil fuels: both unlock a stored resource while generating costs or “externalities.” The analogy extends to information: language models can rapidly draw on a vast cultural archive, but may also amplify misinformation and disorder. The discussion presents this as a conceptual comparison, not as a claim that information disorder and physical entropy are identical.
• AI and scientific problems: Deep-learning systems are discussed as useful for high-dimensional problems such as protein folding, where many variables interact. In the example of AlphaFold, neural-network methods can scale more efficiently than exhaustive physical simulation and can draw on evolutionary as well as physical and chemical information.
• Scale, emergence, and understanding: Some phenomena can be described at a higher level without tracking every component—for example, fluids through macroscopic properties such as pressure and temperature. Other problems may lack a useful higher-level description and require attention to many interactions. Krakauer presents AI as a tool for exploring such difficult problems, rather than evidence that AI will necessarily become a master or “superintelligence.”
• What science should aim for: The episode distinguishes solving problems from making the world comprehensible. It closes by emphasizing the value of asking good questions and remaining open to unresolved ones.

Researchers and Sources Mentioned

• Featured participants: David Krakauer; Neil deGrasse Tyson; Chuck Nice; Gary O’Reilly.
• Researchers, scientists, and intellectual sources: Carl Sagan; Alexander von Humboldt; Charles Sanders Peirce; Ivan Pavlov; Anil Seth; August Kekulé; Srinivasa Ramanujan; Henri Bergson; Albert Einstein; Carl Linnaeus; André Leroi-Gourhan; Dorothy Cheney and Robert Seyfarth (apparently the researchers referred to in the transcript); Geoffrey Hinton.
• Authors and literary sources: Cormac McCarthy, “The Kekulé Problem”; Andrea Wulf, The Invention of Nature (a biography of Humboldt); Philip K. Dick, Do Androids Dream of Electric Sheep?; Douglas Adams, The Hitchhiker’s Guide to the Galaxy; Rainer Maria Rilke.
• Scientific project and institutions: AlphaFold and DeepMind; the Santa Fe Institute.
• Uncertain attribution: A remark about approximation is tentatively associated in the conversation with Bertrand Russell.