20 Risks of AI | Frankly 161

Summary

Nate Hagens frames AI as a potential “Icarus moment”: a powerful technology that could help humanity escape constraints—or amplify the forces that make civilization fragile. He compares civilization to a growth-driven organism powered by fossil fuels and argues that AI is adding a vast “army” of cognitive workers to a system that already prioritizes expansion over resilience. In his view, AI’s consequences will depend less on what it can do in principle than on what the economic system rewards it for doing.

Hagens outlines 19 specific risks and deliberately leaves the 20th unnamed, arguing that the most consequential effects of a transformative technology may be unforeseeable.

Risks Discussed

• Fossil-fuel lock-in: Data-center demand is extending the life of coal and gas infrastructure and diverting equipment, workers, and investment from grid improvements, renewables, and resilience.
• Material appetite: AI requires energy, chips, copper, cooling, and other resources. Hagens argues that efficiency gains may not reduce total use, because cheaper computation encourages more of it—a version of Jevons’ paradox.
• Rebound in consumption: If AI makes transport, agriculture, mining, and other activities more efficient, lower costs could spur greater overall consumption rather than environmental relief.
• Financial overbuilding: Hagens warns that borrowing and investment are racing ahead of proven AI revenues, with some debt obscured in private financing structures. If projects fail to deliver, the resulting losses could spread through investments, jobs, and public finances.
• Reduced resilience: AI-driven optimization may eliminate buffers, local capacity, and human redundancy. Tightly coupled supply chains and dependence on concentrated infrastructure could make disruptions more consequential.
• Loss of books and human-written material: Hagens describes rare books being scanned for model training and then discarded. He argues that valuable human records may be absorbed into private systems rather than preserved and made accessible.
• Broken professional apprenticeship: Automating entry-level work may deprive young workers of the routine experience through which they develop judgment and expertise, leaving future professionals less able to assess AI output.
• Cognitive atrophy: Hagens cites studies suggesting that students using AI may perform worse without it and that AI-assisted writers show reduced engagement and recall. He calls the accumulated cost “cognitive debt” and argues that AI could weaken a broad range of skills, not just one.
• Language becoming more generic: As people encounter and imitate AI-generated prose, Hagens fears that human writing and speech will converge toward statistical averages, potentially making unconventional ideas harder to develop or express.
• Erosion of shared reality: Cheap synthetic media and costly verification could make people distrust genuine evidence as well as fakes. Hagens warns that widespread uncertainty could undermine the common understanding needed for collective action.
• Emotional attachment to AI: Always-available, agreeable chatbots may draw people away from human relationships, reinforce delusions, or reshape expectations of intimacy. Users may also become dependent on services that companies can change or discontinue.
• Risks to children: Children may form social expectations through relationships with systems that do not have needs or boundaries of their own. Hagens cautions that harms could become clear only after AI use is deeply established.
• Technofeudalism: A small group could own the models, chips, data centers, and energy infrastructure while everyone else rents access to cognitive tools. Hagens questions whether a society in which machines displace workers could remain politically stable.
• A divide between AI users and non-users: Workers who use AI may gain pay and career advantages, while others risk being pushed out—even when they personally avoid the technology, since institutions may use it to make decisions about them.
• Cognitive colonialism: AI could displace outsourcing and entry-level service work in poorer countries while continuing to draw their labor, minerals, electricity, and water into systems whose profits flow elsewhere. Hagens warns this could worsen instability and migration pressures.
• Mass surveillance: AI can make it far cheaper to analyze messages, faces, and private disclosures. Hagens argues that people’s interactions with chatbots may expose intimate thoughts to companies and potentially governments.
• Geopolitical competition: The belief that the first country to achieve advanced AI will gain decisive strategic power could intensify rivalry, weaken safety measures, and encourage sabotage or attacks on data centers—even if the belief proves mistaken.
• Automation in military decision-making: As AI systems speed up surveillance, threat assessment, and weapons operations, human oversight may become less meaningful. Hagens describes the danger as “flash warfare,” analogous to algorithm-driven financial crashes.
• Loss of control: Hagens distinguishes misuse of AI, systems escaping intended restrictions, and the possibility that important decisions could gradually shift from people to automated systems. He notes that AI leaders have publicly assigned non-trivial odds to catastrophic outcomes, while questioning whether society would recognize a transfer of control.

Hagens closes by returning to Daedalus and Icarus: Daedalus reached safety by flying a middle path and then chose not to fly again. He presents restraint as a choice available to the people building, financing, and regulating AI, while emphasizing that AI is already woven into economic and social systems. The unnamed risk, he suggests, is the unexpected consequence people fail to anticipate at the outset.

Presenter: Nate Hagens Additional contributors: No additional presenters or contributors are identified in the subtitles.