Solipsistic Superintelligence is Unlikely to be Cooperative

2026-06-02

AI Summary

This work addresses a critical limitation in current AI design paradigms, which center on unilateral optimization and neglect the interdependence among agents, their environment, and other entities—thereby hindering the emergence of effective cooperation even in superintelligent systems. The paper proposes a novel non-solipsistic paradigm that treats multi-agent interdependence as a foundational design principle. By dynamically evaluating environmental contexts, institutionalizing mechanisms, and structurally integrating human agency, the approach embeds cooperative capabilities directly into AI system architectures. The study systematically demonstrates the self-undermining nature of unilateral optimization, introduces a dynamic evaluation platform featuring adaptive adversaries, and reconceptualizes institutions as design primitives. This framework establishes both theoretical foundations and practical pathways toward coexistent, sustainable human-AI collaborative intelligence.

Abstract

AI's central challenge is shifting from capability to coexistence. The dominant paradigm in AI research focuses on developing powerful agents that treat the world as an exogenous and stationary source of feedback. We contend that superintelligence, an extremely capable task solver, born out of such a solipsistic approach to AI design, is unlikely to be cooperative. Deploying AI systems induces endogenous non-stationarity, resulting in a train-test-deploy gap where historical distributions diverge from the deployment context. We refer to this as the self-undermining property of unilateral optimization. Closing this gap requires AI that participates in cooperation: the equilibrium-selection process through which multiple actors navigate their interdependence. We call for a non-solipsistic research paradigm that treats this interdependence as a core design principle rather than approaching cooperation as a task to solve. This entails building dynamic evaluation testbeds involving adaptive counterparties, treating institutions as design primitives, and preserving human agency as a structural feature of the systems we build.

Problem

• Solipsistic Superintelligence
• Cooperation
• Endogenous Non-stationarity
• AI Coexistence
• Interdependence

Innovation

• non-solipsistic AI
• endogenous non-stationarity
• cooperative superintelligence
• dynamic evaluation testbeds
• human agency