Why Building Superintelligence Means Human Extinction (with Nate Soares)
Future of Life Institute
@futureoflifeinstituteAbout
The Future of Life Institute (FLI) is a nonprofit working to reduce global catastrophic and existential risk from powerful technologies. In particular, FLI focuses on risks from artificial intelligence (AI), biotechnology, and nuclear weapons. The Institute's work is made up of three main strands: grantmaking for risk reduction, educational outreach, and advocacy within the United Nations, US government and European Union institutions. FLI has become one of the world's leading voices on the governance of AI having created one of the earliest and most influential sets of governance principles: the Asilomar AI Principles.
Latest Posts
Video Description
Nate Soares is president of the Machine Intelligence Research Institute. He joins the podcast to discuss his new book "If Anyone Builds It, Everyone Dies," co-authored with Eliezer Yudkowsky. We explore why current AI systems are "grown not crafted," making them unpredictable and difficult to control. The conversation covers threshold effects in intelligence, why computer security analogies suggest AI alignment is currently nearly impossible, and why we don't get retries with superintelligence. Soares argues for an international ban on AI research toward superintelligence. LINKS: If Anyone Builds It, Everyone Dies - https://ifanyonebuildsit.com Machine Intelligence Research Institute - https://intelligence.org Nate Soares - https://intelligence.org/team/nate-soares/ PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) Episode Preview (01:05) Introduction and Book Discussion (03:34) Psychology of AI Alarmism (07:52) Intelligence Threshold Effects (11:38) Growing vs Crafting AI (18:23) Illusion of AI Control (26:45) Why Iteration Won't Work (34:35) The No Retries Problem (38:22) Computer Security Lessons (49:13) The Cursed Problem (59:32) Multiple Curses and Complications (01:09:44) AI's Infrastructure Advantage (01:16:26) Grading Humanity's Response (01:22:55) Time Needed for Solutions (01:32:07) International Ban Necessity SOCIAL LINKS: Website: https://podcast.futureoflife.org Twitter (FLI): https://x.com/FLI_org Twitter (Gus): https://x.com/gusdocker LinkedIn: https://www.linkedin.com/company/future-of-life-institute/ YouTube: https://www.youtube.com/channel/UC-rCCy3FQ-GItDimSR9lhzw/ Apple: https://geo.itunes.apple.com/us/podcast/id1170991978 Spotify: https://open.spotify.com/show/2Op1WO3gwVwCrYHg4eoGyP
Essential AI & Tech Books
AI-recommended products based on this video
Apple AirPods 4 Wireless Earbuds, Bluetooth Headphones, Personalized Spatial Audio, Sweat- and Water-Resistant, USB-C Charging Case, H2 Chip, Up to 30 Hours of Battery Life
Media Management and Artificial Intelligence: Understanding Media Business Models in the Digital Age
12 Set Graduation Gifts for Her Him Graduation Drinking Glass Cups The Tassel was Worth The Hassle 16 oz Glass Cup with Lid Straw for Women Men College High School Master Degree
When the Moon Hits Your Eye
When the Body Says No: The Cost of Hidden Stress
The Housemaid Is Watching
The River Is Waiting (Oprah's Book Club): A Novel
Whole: Rethinking the Science of Nutrition
The Signs: The New Science of How to Trust Your Instincts
The Ultimate Book of Random Fun Facts: Over 1000 Interesting Facts And Trivia Quizzes About History, Science, Sports, Animals, Space and Anything In Between!
The Book. The Ultimate Guide to Rebuilding a Civilization - Inspirational Science Books for Adults - Unique Artifact - Knowledge Encyclopedia with Over 400 Pages of Detailed & Catchy Illustrations
The Pandy - Emotional Healing on Demand, Realistic Panda Plush for Emotional Support, Mimics Natural Movements for Realistic Comfort, Pandy Ai Plush with Ultra-Soft Fur, The Comfort Hug You Have
Surrounded by Idiots: The Four Types of Human Behavior and How to Effectively Communicate with Each in Business (and in Life)
Strong Ground: The Lessons of Daring Leadership, the Tenacity of Paradox, and the Wisdom of the Human Spirit
eufy Security Solo Indoor Cam C120, 2K Security Indoor Camera, Plug-in Camera with Wi-Fi, IP Camera, Human & Pet AI, Voice Assistant Compatibility, Night Vision, Two-Way Audio, HomeBase not Compatible
eufy Security Indoor Cam E220, 2K, Pan & Tilt, Indoor Security Camera, Wi-Fi Plug-in Camera, Human & Pet AI, Voice Assistant Compatibility, Night Vision, Motion Tracking, HomeBase not Compatible
Harry Potter and the Goblet of Fire (Interactive Illustrated Edition)
Strong Ground: The Lessons of Daring Leadership, the Tenacity of Paradox, and the Wisdom of the Human Spirit
The Second Brain: A Groundbreaking New Understanding of Nervous Disorders of the Stomach and Intestine
Dying for an iPhone: Apple, Foxconn, and The Lives of China's Workers