Nathaniel Soares is an American artificial intelligence author and researcher known for his work on existential risk from AI. In 2014, Soares co-authored a paper that introduced the term AI alignment, the challenge of making increasingly capable AIs behave as intended. Soares is the president of the Machine Intelligence Research Institute (MIRI), a research nonprofit based in Berkeley, California. In 2025, Soares co-authored If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky. In the book, he argues that creating vastly smarter-than-human AI (or superintelligence) “using anything remotely like current techniques” would very likely result in human extinction; he further contends that AI alignment remains nascent and that international regulatory intervention will likely be required to prevent developers from racing to build catastrophically dangerous systems. The book received mainstream attention and mixed reviews: The Guardian selected it as "Book of the day" and described it as "clear" though its conclusions are "hard to swallow"; The Washington Post characterized it as a polemic that offers few concrete instructions; and The New Yorker featured it in "Briefly Noted." Coverage also included reporting in Wired, Semafor, and SFGATE.
Life Soares received his Bachelor of Science degree (in computer science and economics) from George Washington University in 2011. Soares worked as a research associate at the National Institute of Standards and Technology and as a contractor for the United States Department of Defense, creating software tools for the National Defense University, before spending time at Microsoft and Google. Soares left Google in 2014 to become a research fellow at the Machine Intelligence Research Institute. He served as the lead author on MIRI's research agenda, which in January 2015 was cited heavily in Research Priorities for Robust and Beneficial Artificial Intelligence, an open letter calling for AI scientists to prioritize technical research “not only on making AI more capable, but also on maximizing the societal benefit of AI.” This included “research on the possibility of superintelligent machines or rapid, sustained self-improvement (intelligence explosion).” Shortly after joining MIRI, Soares became the institute's executive director. In 2017, he gave a talk at Google outlining open research problems in AI alignment, and arguing that the alignment problem looks especially difficult. In 2023, MIRI shifted from a focus on alignment research to a focus on warning policymakers and the public about the risks posed by potential future developments in AI. Coinciding with this change, Soares transitioned from the role of executive director to president, with Malo Bourgon serving as MIRI's new CEO.
Publications Garrabrant, Scott; Benson-Tilsen, Tsvi; Critch, Andrew; Soares, Nate; Taylor, Jessica (2017). "A Formal Approach to the Problem of Logical Non-Omniscience". Proceedings Sixteenth Conference on Theoretical Aspects of Rationality and Knowledge. pp. 221–235. Soares, Nate (2018). "The Value Learning Problem" (PDF). In Yampolskiy, Roman (ed.). Artificial Intelligence Safety and Security. Chapman & Hall. ISBN 9780815369820. Soares, Nate; Fallenstein, Benya (2017). "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda" (PDF). In Callaghan, Victor; Miller, James; Yampolskiy, Roman; Armstrong, Stuart (eds.). The Technological Singularity. Springer. ISBN 9783662540312. Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer; Armstrong, Stuart (2015). "Corrigibility" (PDF). AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25–26, 2015. AAAI Publications. pp. 74–82. Soares, Nate; Levinstein, Benjamin (2020). "Cheating Death in Damascus" (PDF). The Journal of Philosophy. 117 (5): 237–266. doi:10.5840/jphil2020117516. Yudkowsky, Eliezer; Soares, Nate (2025). If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All. Little, Brown and Company. ISBN 9780316595643.
References
