Modelling the Threat from AI: Putting Agency on the Agenda

Abstract

The AI existential-risk narrative focuses on an ‘intelligence explosion’ leading to uncontrollable superintelligence. This paper contends that the more plausible and proximate threat is the emergence of strong biological-style agency in digital systems, independent of high intelligence. Drawing on systems biology and thermodynamics, it contrasts mechanistic with organic agency: living organisms are autocatalytic systems that harness environmental energy for self-maintenance and reproduction, whereas current Autonomous/Intelligent Systems pursue only externally assigned goals. Evolution produced robust agency in bacteria, slime molds, and insects long before cognition. Recent work in embodied neural networks and bio-inspired computing shows that complex adaptive behavior can arise in machines through structural coupling with their environment that occurs without symbolic reasoning. Deliberate or accidental development of energy-seeking, self-reproducing ‘biodigital agents’ could therefore yield invasive, unpredictable systems well below superintelligent levels. The paper advocates shifting AI safety priorities from anthropomorphic ethics and alignment to measurable biophysical criteria derived from the definition of life. Recommended measures include engineering standards prohibiting direct environmental energy harvesting by A/IS, global energy audits to detect emergent agency, and epidemiological containment frameworks—thereby preventing a Cambrian-like explosion of machine agency before superintelligence becomes feasible.

Keywords: AI, superintelligence, intelligence explosion, biodigital agents