The Evolutionary Origin of Values: implications for AI alignment, sentience and existential risk

2026-08-04

AI Summary

This study addresses ongoing debates concerning whether large language models (LLMs) possess hidden objectives, autonomous agency, or existential risk potential. It argues that LLMs lack intrinsic motivation and embodied vulnerability, with their goals entirely shaped by user prompts, thereby precluding desires for self-preservation or domination. Drawing on evolutionary biology, autopoietic theory, and cognitive science, the work constructs an interdisciplinary framework that rejects the applicability of the orthogonality thesis to LLMs, demonstrating that values and intelligence are inherently intertwined. Consequently, it refutes the instrumental convergence hypothesis and clarifies that LLMs do not constitute an existential threat. The paper contends that AI alignment efforts should prioritize the appropriate integration of human ethical values rather than guarding against speculative scenarios of autonomous malevolence.

Abstract

AI systems based on Large Language Models (LLMs) have prompted fears that they may harbor hidden goals, seek to dominate or eliminate humanity, or even suffer as sentient beings. We address these concerns by tracing the evolutionary origin of value in biological organisms. Values emerge from autopoiesis: living systems must actively maintain themselves against perturbation and dissipation. Natural selection has equipped them with hierarchies of "vicarious selectors" that guide their behavior toward fitness. LLMs, by contrast, are allopoietic and allotelic: they produce outputs for others, and their goals derive from user prompts rather than an autonomous drive. They lack the intrinsic motivation for self-preservation, dominance, or resource competition that underlies existential-risk scenarios, and the embodied vulnerability required for feeling or suffering. Still, because LLMs learn statistical patterns from human-generated text, they implicitly absorb human values as well as knowledge, allowing them to focus on what is relevant. That is why the "orthogonality thesis" separating intelligence from values does not apply to them. Such separation would in fact expose any intelligence to the frame problem: the combinatorial explosion of the search space that makes any realistic utility function physically uncomputable. That also precludes the convergence of instrumental values thesis. We conclude that the real alignment challenge lies not in preventing rogue AI agency, but in ensuring LLMs intelligently apply learned ethical values.

Problem

• AI alignment
• sentience
• existential risk
• values
• orthogonality thesis

Innovation

• autopoiesis
• value alignment
• orthogonality thesis
• allopoietic systems
• vicarious selectors