Back to home page
Essay
Interactive Summary

Appendices

Essay navigation

Why This Scenario?

How We Got Here: The Path to Tool AI 2035

How Tool AI Transformed Key Domains Across Society

Key Uncertainties and Tensions

The Human Experience in Tool AI 2035

Appendices

How This Scenario Was Created

This scenario is part of the AI Pathways project, an initiative of the Foresight Institute’s Existential Hope program. Rather than focusing on risks or speculative timelines, AI Pathways presents two vividly realized and contrasting visions of what a desirable AI-driven future might look like. 

The project brings together leading thinkers, including Vitalik Buterin, Glen Weyl, Anton Korinek, and Allison Duettmann, who contributed to crafting these narratives.

Videography by Petr Salaba (AI-generated).

Explore the AI Pathways Project

  • Tool AI 2035 – A future shaped by advanced, but purposefully narrow, AI systems that enhance human decision-making without striving for full autonomy or generality.
  • d/acc 2035 – Imagines a decentralized, democratic, defensive model of technological progress. Emphasizes plural acceleration, privacy-first infrastructure, and community-governed resilience.

Both scenarios are part of AI Pathways and invite reflection on the values and paths we choose in shaping AI’s role in our world.

Contributors to this scenario

The development of the Tool AI report was led by Linda Petrini and Beatrice Erkers, grounded in expert interviews and iterative feedback from across domains. The scenario should not be seen as the official views of any individual contributor.

  1. Adam Marblestone (Convergent Research), 
  2. Anton Korinek (University of Virginia), 
  3. Anthony Aguirre (Metaculus, Future of Life Institute), 
  4. Saffron Huang (Anthropic), 
  5. Joel Leibo (DeepMind), 
  6. Rif A. Saurous (Google), 
  7. Cecilia Tilli (Cooperative AI Foundation), 
  8. Ben Reinhardt (Speculative Technologies), 
  9. Bradley Love (Los Alamos National Laboratory), 
  10. Konrad Kording (University of Pennsylvania), 
  11. Jeremy Barton (Nano Dynamics Institute), 
  12. Owen Cotton-Barratt (Researcher), 
  13. Kristian Rönn (Lucid Computing).

We’re deeply grateful to anyone who contributed their time and insights to this experiment.

Metaculus Forecasting integration

To engage a broader audience, we’ve launched a set of forecasting questions on Metaculus tied to key scenario milestones, along with a  $5,000 Commenting Prize for the top eight contributors.

Participants are encouraged to share thoughtful insights, and shape this future through collective dialogue.

Context: Existential Hope Program

The term “Existential Hope” refers to the capacity to envision futures where humanity not only survives, but flourishes in ways we can currently only imagine, and to be able to better work towards those futures. It complements the better-known concept of existential risk.This project sits within the Foresight Institute‘s broader mission to drive long-term, future-positive technology, where imagination is both a tool and a catalyst for change.

Master Reference List

Core Scenario References

  1. Aguirre, A. (2025). Keep the Future Human. [Essay]. https://keepthefuturehuman.ai/
  2. Carlsmith, J. (2025). AI for AI Safety. [Essay Series]. https://joecarlsmith.com/2025/03/14/ai-for-ai-safety
  3. Drexler, K.E. (2019). Reframing Superintelligence: Comprehensive AI Services as General Intelligence. Future of Humanity Institute Technical Report #2019-1. https://www.fhi.ox.ac.uk/reframing-superintelligence.pdf
  4. Bengio, Y. et al. (2025). Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path? arXiv:2502.15657. https://arxiv.org/abs/2502.15657
  5. Cooperative AI Foundation (2025). Cooperative AI Grantmaking and Research Areas. https://www.cooperativeai.com/grants/2025
  6. Vaintrob, L., & Cotton-Barratt, O. (2025). AI Tools for Existential Security.
  7. Finnveden, L. (2024). What’s Important in “AI for Epistemics”?. LessWrong. https://www.lesswrong.com/posts/D2n5uduYGXuexkv7v
  8. RAND Corporation (2024). How AI Can Automate AI Research and Development. https://www.rand.org/pubs/commentary/2024/10/how-ai-can-automate-ai-research-and-development.html
  9. Tool AI 2035 Interview Transcripts (2025). Internal document summarizing expert interviews.

Domain-Specific References

Science
Healthcare
Education
Economy
Climate & Energy
Governance
Law & Justice
Additional PDF Sources
  • [ARXIV] Characterizing AI Agents for Alignment and Governance. arXiv preprint arXiv:2504.21848, 2025. https://arxiv.org/abs/2504.21848
  • Leibo, J.Z. et al. (2024). A Theory of Appropriateness with Applications to Generative Artificial Intelligence. Google DeepMind, Mila, University of Toronto, Max Planck Institute.
  • [Author unspecified]. (2025). AI for Epistemics — Concrete Projects.
  • Dafoe, A. et al. (2020). Cooperative AI. Nature Machine Intelligence, 2(6), 366–368. https://doi.org/10.1038/s42256-020-0186-2

Explainers

Natural Science LLMs

Large Language Models (LLMs) fine-tuned on natural science literature and datasets to assist with scientific tasks such as literature review, hypothesis generation, or experimental design. Examples include models trained on papers from arXiv, PubMed, or domain-specific corpora in chemistry, biology, and physics.

Graph-Based Model Architectures

AI models that represent and process data as graphs—networks of nodes (entities) and edges (relationships). These architectures are especially suited for structured data like molecules, knowledge graphs, and citation networks. Common examples include Graph Neural Networks (GNNs).

Knowledge Work

Cognitive labor that involves handling information, problem-solving, and generating new ideas or insights. In the context of AI, it refers to domains like research, writing, design, legal analysis, or education, which are increasingly being augmented by AI tools.

Autonomous Systems

Systems capable of performing tasks without real-time human input, typically using AI and sensor integration. In the Tool AI scenario, the focus is on narrow or task-specific autonomy—e.g., lab robots or logistics systems—not generalized agents.

Alignment

The field of AI alignment studies how to ensure that AI systems pursue goals that are beneficial to humans. In the context of Tool AI, this often involves building systems that are controllable, corrigible, and value-aware, especially when used in high-stakes domains like science or governance.

Interpretability

Techniques and frameworks used to make AI model outputs and internal reasoning understandable to humans. Interpretability is essential for debugging models, ensuring safety, and maintaining trust—particularly when AI systems are used in scientific or policy settings.

AI-Assisted Literature Synthesis Tools

Software tools that use natural language processing to extract, summarize, and analyze findings across large volumes of academic literature. They are designed to help researchers stay current, identify trends, and integrate fragmented knowledge across disciplines.

In Silico

Latin for “in silicon,” this term refers to computational simulations or experiments performed using digital models rather than in vitro (in the lab) or in vivo (in living organisms). Common in drug development, molecular biology, and materials science.

Epistemic Stack

Scientists interact with a dynamic “epistemic stack” rather than static papers. This infrastructure creates auditable provenance trees from high-level claims down to raw data or foundational papers, with AI-generated outputs explicitly tagged with their source materials and training data, allowing scientists to trace where ideas originated and verify the underlying sources. A researcher can query the global knowledge base to ask, “What is the most significant contradiction in the literature regarding protein X?” and receive a visualized map of the conflicting evidence.

Consilience-as-a-Service

A proposed service model where AI tools help integrate evidence and insights across scientific disciplines to form coherent explanations or predictions. It aims to overcome fragmentation in the scientific landscape by enabling synthesis across fields.

Theory Glut

A situation where AI systems can generate a large number of plausible scientific theories or hypotheses faster than they can be experimentally tested. This creates challenges for prioritization and validation in research workflows.

Validation Bottlenecks

The constraints in scientific progress caused by the limited capacity to test, reproduce, or verify new findings—especially when AI accelerates hypothesis generation faster than experimental pipelines can keep up.

Digital Twins

Digital replicas of physical systems—ranging from organs to entire ecosystems—that are continuously updated with real-world data. In science and engineering, digital twins allow for simulation, prediction, and intervention planning without physical testing.

Human-in-the-Loop (HITL)

A design approach where human judgment is integrated into AI workflows to provide oversight, feedback, or final decision-making. HITL systems are especially important in domains where accountability and context are critical.

Natural Language Processing (NLP)

A field of artificial intelligence focused on enabling machines to understand, generate, and interact with human language. NLP powers applications such as translation, summarization, question answering, and conversational agents. In the Tool AI context, NLP is used to interface with scientific knowledge, automate communication, and support knowledge work.

Reinforcement Learning from Human Feedback (RLHF)

A technique for fine-tuning AI models by optimizing for human preferences instead of static datasets. It involves showing models multiple outputs, asking humans to rank them, and training the model to generate preferred responses. RLHF is widely used to make large language models more aligned with human intent, tone, and ethical considerations.

Fusion

The process of combining atomic nuclei to release energy, often considered a long-term solution for sustainable power. AI tools are increasingly used to model plasma behavior, optimize reactor design, and interpret fusion experiment data.

Pareto-topia

A term coined by Eric Drexler and Mark S. Miller to describe futures where progress benefits some without harming others—i.e., Pareto improvements. It reflects a vision of societal evolution toward broadly beneficial outcomes, avoiding zero-sum tradeoffs.

Habermas Machines

An AI system developed by DeepMind, Stanford, and MIT to support group deliberation. It uses large language models to generate and refine group statements based on participant input, aiming to help diverse groups reach consensus. Inspired by philosopher Jürgen Habermas, though he has since distanced himself from the project.

Preference-Mapping Systems

Digital tools or AI models that help elicit, represent, and aggregate individual or collective preferences. These systems are useful for aligning decisions—e.g., policy or research funding—with stakeholder values.

Molecular and Materials Simulation Platforms

Software tools used to simulate the behavior of molecules and materials at the atomic level. These platforms often use physics-based models, AI, or hybrid approaches to accelerate materials discovery and design.

CAD Environments

Computer-Aided Design platforms adapted for scientific and technological work. In the Tool AI context, they may be used for designing lab experiments, biological systems, molecular structures, or hardware prototypes.

Individual Choice-Based Contribution Markets

Funding or coordination mechanisms where individuals allocate resources (e.g., tokens, votes, dollars) based on personal choice. Examples include quadratic funding or retroactive public goods markets, enabling decentralized prioritization of innovation.

Compute

Short for computing power, usually measured in floating-point operations per second (FLOPs) or GPU hours. Compute is a key resource for training, fine-tuning, and deploying AI models, and its availability often shapes who can participate in frontier AI development.

Sorcerer’s Apprentice Problem

A metaphor from Goethe’s poem and Disney’s Fantasia, used in AI ethics to describe scenarios where automated systems continue running or cause unintended consequences after losing human control. It highlights the need for robust fail-safes and scope limits.

Universal Basic Income (UBI)

A policy proposal in which all individuals receive a regular, unconditional cash payment from the government or another institution, regardless of employment status. In the context of the Tool AI scenario, UBI is discussed as a possible response to AI-accelerated shifts in the labor market, particularly in knowledge work and service sectors. It is seen as one potential mechanism to ensure economic stability and individual agency amid widespread automation and productivity gains.

Immune Monitoring Systems

Technologies designed to continuously or periodically track immune system markers—such as T-cell populations, antibody levels, and cytokine activity—to assess health status, disease progression, or treatment response. In AI-enabled biomedical research, these systems generate high-dimensional datasets that can be used for personalized medicine, early diagnostics, or in silico modeling. They are increasingly integrated with digital twins and AI tools to simulate and optimize immune interventions.

Safe Harbor

A legal provision that offers protection from liability under specific conditions. In the Tool AI context, safe harbor frameworks exempt AI systems from strict liability if they lack certain high-risk characteristics, such as combining high autonomy, generality, and intelligence, thereby incentivizing constrained designs.

Tool AI

Artificial intelligence systems designed for high competence in specific tasks while deliberately limiting autonomy and generality. Tool AI is built to operate under human control, avoiding independent goal pursuit across domains and maintaining transparency and accountability in high-stakes applications.

d/acc

Short for decentralized, democratic, defensive, and differential acceleration. A strategic approach to technological development that emphasizes plural, bottom-up innovation while integrating defensive measures and spreading benefits across diverse actors.

Constitutional AI

An alignment approach that trains AI models to follow a set of explicit rules or principles (“a constitution”) to guide outputs, reducing or replacing the need for human feedback during fine-tuning. This method aims to ensure consistent, value-aligned behavior without relying solely on post-hoc oversight.

Income Floor

A guaranteed minimum income from any combination of sources such as universal basic income, dividends, or welfare transfers, ensuring that all individuals can maintain basic living standards regardless of employment status.

Lifelong Learning

The practice of continuing education and skill development throughout a person’s life rather than concentrating all formal learning in early adulthood. This can include modular courses, vocational training, and informal learning integrated into daily life.

Federated Learning

A collaborative machine learning approach where models are trained locally on devices or institutional servers, and only model updates are shared with a central server. This allows the model to benefit from decentralized data without transferring sensitive raw data, preserving privacy and data sovereignty.