AI Governance Has a Theory Problem

The constraint on enterprise AI adoption has moved from model capability to governance. However, most of the frameworks meant to address these governance problems are being written far from where deployments actually happen. This is causing a deep structural disconnect between high-level ethical theories and operational reality.

We see this every day at Credal. The highest ROI agents increasingly need to read from and write to production data, and they cannot go live without the right guardrails in place. The blocker is almost always something specific and esoteric: a permissions model that can't express what the security team needs, an audit requirement no one anticipated, an OAuth flow that won't pass review. Solving each governance blocker requires getting extremely specific and following through until you have the key to deployment.

This is why we believe the deepest understanding of the right governance philosophy comes from practitioners. Governance challenges are uncovered in practice; they cannot be solved through theory alone. It is also why we believe startups like ours, building enterprise AI in production, have an important role in shaping the broader governance space.

So when the United Nations convened the first-ever Global Dialogue on Artificial Intelligence Governance in Geneva on July 6th and 7th - bringing together more than 4,200 participants, 1,800 organizations, and 170 UN Member States - we went to make that argument.

The Scientific Panel's Finding: Governance Is the Bottleneck

The event opened with the Preliminary Report of the Independent International Scientific Panel on AI, the first global scientific body established by the UN to assess AI's impact on society. Its central finding was that the primary constraint on AI adoption is no longer model capability, it's governance. As agentic AI advances faster than governance frameworks can keep pace, organizations need standardized evaluations, continuous monitoring, and practical governance mechanisms to deploy safely at scale.

For those of us building governed AI systems, this is a familiar conclusion. But hearing it as the headline finding of the UN's flagship scientific report marks a real shift in the global conversation: from how to constrain AI to how to make AI deployable. This can only be answered in practice.

What We Argued on Stage

The highlight of Credal's participation was CEO Ravin Thambapillai's panel seat for a discussion on "Safe, Secure and Trustworthy AI: Interoperability and Compatibility of Approaches," where he joined United Kingdom Parliamentary Under-Secretary of State Seema Malhotra, World Meteorological Organization Secretary-General Celeste Saulo, and other international leaders.

Drawing on Credal's experience building governed AI systems for government agencies, financial institutions, and healthcare organizations, Ravin made two arguments that cut against the prevailing instincts of the governance world:

First, standards are won through implementation, not consensus. The Model Context Protocol won because it published practical reference implementations that solved real developer problems, and adoption followed. Governance frameworks will work the same way: a framework grounded in operational experience will outperform an elegant set of abstract principles, because only one of them holds up in a real security review.

Second, governance and adoption are a flywheel, not a tradeoff. The default framing treats governance as a brake on adoption: more of one means less of the other. We think that framing is backwards. Strong governance is what unblocks adoption; it is the thing that lets a bank or a hospital say yes. And broader adoption is what improves governance, because it generates the operational experience that frameworks need to be any good. Governance accelerates adoption, and adoption improves governance.

Two Priorities for the Ecosystem

If those arguments are right, two priorities follow, and Ravin proposed both:

Develop open reference implementations for well-governed agentic AI systems. The ecosystem cannot wait for principles documents over the next few years. It needs working examples that organizations around the world can implement consistently, the governance equivalent of what MCP's reference servers did for tooling.

Use AI within governments and regulatory bodies. It is difficult to write practical rules for a technology you have never operated. Firsthand experience with AI systems provides a deeper understanding of their opportunities, limitations, and risks. The governments that adopt AI internally will write the most practical, interoperable frameworks. The ones that don't will regulate something that doesn't exist.

From Principles to Implementation

The shift from principles to practical implementation is already underway, and Geneva made that visible at a global scale. At Credal, we live that shift every day, and we'll bring the perspective it gives us to the next Global AI Dialogue in New York in 2027.