The State of AI Governance Research: AI Safety and Reliability in Real World Commercial Deployment

2025-04-28

AI Summary

This study identifies a structural imbalance in AI governance research between deployment-phase concerns—such as bias, hallucination, misleading applications, and addictive design—and pre-deployment activities—including alignment and evaluation. Through bibliometric analysis of 1,178 AI safety and reliability papers published globally from 2020 to 2025, augmented by cross-institutional topic modeling and mapping to the AI deployment lifecycle, the authors quantitatively demonstrate that enterprise research on deployment-phase issues has declined by 37%, with pronounced gaps in high-risk domains such as healthcare, finance, and misinformation mitigation. The study introduces a multidimensional risk classification framework and attributes this imbalance to commercial incentives that systematically neglect observable real-world deployment behaviors. Based on these findings, the paper proposes governance interventions including enhanced external data access and the development of market-behavior observability infrastructure to strengthen accountability and evidence-based regulation.

Technology Category

Philosophy and Ethics of AI: Safety, Robustness & TrustworthinessMachine Learning: Ethics, Bias, and FairnessHumans and AI: Other Foundations of Human Computation & AI

Application Category

Responsible Web: Human-perceived consequences of algorithmic deployment on the webSecurity and Privacy: Data transparency and provenanceEconomics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Drawing on 1,178 safety and reliability papers from 9,439 generative AI papers (Jan- uary 2020 - March 2025), we compare research outputs of leading AI companies (An- thropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of Washington). We find that cor- porate AI research increasingly concentrates on pre-deployment areas — model align- ment and testing & evaluation — while attention to deployment-stage issues, such as model bias, has waned, as commercial imperatives and existential risks have come into focus. We find significant research omissions in high-risk deployment areas, including healthcare applications, commercial and financial contexts, misinformation, persuasive and addictive features, hallucinations, and copyright usage in training and inference. AI research’s corporate concentration risks exacerbating these oversights. We recommend measures that expand external researcher access to deployment data and improve sys- tematic observability of AI systems’ in-market behaviors.

Problem

• Corporate AI research focuses on pre-deployment, neglecting deployment-stage issues
• Significant research gaps exist in high-risk domains like healthcare and finance
• Lack of observability in deployed AI risks deepening knowledge deficits

Innovation

• Focus on pre-deployment model alignment and testing
• Identify gaps in high-risk deployment domains
• Recommend external access to deployment data