One key challenge with AI governance is demonstrating that agreed-upon rules are being applied consistently. Corporations bound by rules want their rivals to be similarly bound. Countries might refuse to place rules on the behavior of their companies unless they can see that their rivals are behaving similarly.
At the heart of these problems is verification, the problem of demonstrating to one actor that another actor is complying with a mutually agreed set of rules. An ideal verification scheme would demonstrate to actor A that actor B is indeed following a set of rules—all while ensuring that both actors’ most important secrets remain secure. This latter point is important, since secrets of immense importance do exist. A verification scheme might be reasonable if applied to low-sensitivity data like aggregate road traffic patterns, but impractical to apply to state secrets or the Coca-Cola formula.
Our team at the Oxford Martin AI Governance Initiative has recently released a report (with the help of many other researchers) exploring this space: Verification for International AI Governance. We explore five families of potential international AI agreements and how verification might work for each family. The report has three overall takeaways.
First, a number of potential AI agreements look a lot like historical agreements relating to sensitive information and dual-use resources. Perhaps unsurprisingly, those kinds of AI agreements are roughly as verifiable as their historical analogues. The key limiter on this kind of international cooperation is that the state sending sensitive information or resources has to believe that the risk of misuse of these assets is tolerable in comparison to the benefits of the agreement.
Second, some potential AI agreements present fundamentally new challenges. For example, verifying that AI regulations are being faithfully followed may require access to the detailed content of computations (including data and code). This is difficult to achieve, since even a human inspector in the same room as a computer can learn essentially nothing about the computations being done on it unless they are empowered with an array of tools. Luckily, decades of work have gone into hardware tools and cryptographic techniques that allow for credible claims to be made about data without revealing that data. For low-sensitivity computations, this is already possible today at a limited scale via technologies such as confidential computing—which is available on some recent hardware like the NVIDIA Hopper (H100 and others) and Blackwell chips. For extremely high-sensitivity computations, achieving sufficient security for verification will require serious investments in datacenter security and associated verification infrastructure. See below for a schematic of one approach along these lines. Building a scalable verification system along these lines may take a few years of intense effort.
Third, a variety of actors (individuals, nonprofits, corporations, and states) can take actions today that will expand our ability to do AI verification tomorrow. This includes research and development as well as policy. Some of these actions fit easily with corporate incentives to provide innovative and trustworthy AI services. However, others are also needed, such as serious investigations around what it would take for states to build international verification schemes even if they don’t otherwise trust each other. While industry has laid an important foundation for AI verification, it would be unexpected if they achieved the level of security preservation that states demand in their arms control agreements. Overall, the actions of many different actors can foster a dynamic AI verification ecosystem that has the potential to significantly reduce both security concerns (for people, corporations, and states) and the costs of compliance.
In sum, we’re cautious optimists about the future of AI verification. Some things can be verified today, and with smart investments and policies, it seems possible to drastically expand the possibilities for AI verification in the next few years. Such progress would in turn open up economic and political opportunities that may end up shaping not only the AI industry, but the structure of the global economy and the character of relations among states. Our report maps out some of this landscape, but much more remains to be done. This is the first blog post in a series looking at the AI verification puzzle and why it matters: for AI governance, for the AI industry, and for our societies in general. Stay tuned (and subscribe) if you want to stay abreast of this fast-moving conversation.