Open Problems in Technical AI Governance

Open Problems in Technical AI Governance

September 20, 2024

Anka Reuel*, Ben Bucknall*, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Jeffrey Ladish, Sanmi Koyejo, Mykel J. Kochenderfer and Robert Trager. *Equal contribution. Please cite as: 'Reuel, A., Bucknall, B., et al. (2024)'

View Journal Article / Working Paper >

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivisation, or compliance.

In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalogue of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.‍

Image for Examining AI Safety as a Global Public Good: Implications, Challenges, and Research Priorities

Examining AI Safety as a Global Public Good: Implications, Challenges, and Research Priorities

March 11, 2025
Image for Looking ahead: Synergies between the EU AI Office and UK AISI

Looking ahead: Synergies between the EU AI Office and UK AISI

March 11, 2025