What Responsible AI Governance Requires in Practice

23 Oct, 2026

AI was used in researching and editing of the article

There is a useful difference between asking whether artificial intelligence is capable and asking whether an institution is ready to rely on it. The first question concerns performance: can a system produce a useful answer or complete a task? The second concerns responsibility: what evidence justifies its use, and who remains answerable when it fails? The distinction reflects the approach taken by frameworks like the US National Institute of Standards and Technology’s AI Risk Management Framework, which treats AI as a sociotechnical system.¹

This post does not argue against AI adoption. It attempts to map the structural tensions that emerge when abstract principles must become operational practice, to examine what current research suggests about institutional reliance, and to suggest that the most productive framing may not be "pro-adoption" versus "anti-adoption," but rather the conditions under which institutional trust and technical capability can safely coexist.

Legitimate Concerns Worth Taking Seriously

Any serious engagement with enterprise AI must begin by distinguishing between the different forms of assurance institutions seek. Conflating these forms weakens the case for thoughtful integration.

First, the definitional question. Ethics asks what ought to be done; governance concerns how those judgments become institutional practice; compliance concerns applicable legal obligations.² A voluntary framework can guide an organisation without creating a statutory duty. Treating a management standard as proof of ethical alignment, or a legal review as proof of operational safety, risks producing a false sense of security.

Second, the unit of analysis. It is tempting to describe a model as trustworthy in the abstract. However, reliability, safety, and fairness must be assessed in relation to a system’s intended use and operating conditions.³ A language model used to draft an internal meeting agenda and the same model used to communicate an eligibility decision possess similar technical capabilities, but vastly different consequences. Evaluating the model alone, rather than the combination of model, data, interface, and institutional process, is a reasonable basis for institutional caution.

What the Current Evidence Base Suggests

A balanced discussion must take demonstrated benefits seriously without turning local findings into universal predictions. In Generative AI at Work, published in The Quarterly Journal of Economics, researchers studied the introduction of an AI assistant among thousands of customer-support agents.⁴ Access increased productivity by approximately 15% on average, with less experienced agents seeing the most significant improvements in speed and quality.

This is a valuable finding. It provides evidence of benefits in a real workplace rather than merely demonstrating benchmark performance. However, intellectual honesty requires acknowledging the boundaries of this evidence. The study concerned a particular form of assistance in a specific operational setting. It does not establish that similar improvements should be expected in complex institutional decision-making or healthcare.

Similarly, a recent synthesis of educational research examined hundreds of papers and identified a smaller number of high-quality causal studies.⁵ The conclusion was nuanced: performance often improved while students had access to AI, but outcomes on assessments without AI were mixed, and tools with pedagogical guardrails outperformed general-purpose systems. The governance implication is straightforward: institutions must measure the outcome they actually value. If the objective is durable learning or better decision-making, assisted task completion is insufficient evidence.

The Structural Tension: Transparency, Oversight, and Agency

The deeper question AI deployment surfaces is not purely evidential; it is architectural. Institutions are unlikely to trust systems that are opaque, difficult to supervise, or prone to automation bias. This is not irrational resistance; it is a rational response to a genuine design mismatch. Several structural realities suggest possible directions for alignment, without presuming to offer a complete solution:

• The limits of generated explanations. Research on chain-of-thought reasoning has demonstrated that the step-by-step text produced by language models does not always faithfully disclose the actual influences on a model’s answer.⁶ This does not mean explanations have no value, but it establishes that they should not automatically be treated as a complete audit trail. Institutions require inspectable evidence around the system—source records, tool actions, and human approvals—rather than relying solely on plausible narratives generated by the model.

• Meaningful human oversight. "Human in the loop" is a useful starting point, but it does not describe the quality of oversight. Effective oversight requires that the reviewer understands the system’s limitations, has access to the necessary information, possesses the capacity to exercise judgment, and holds the authority to reject or escalate the proposed action.⁷ Adding a reviewer to a process does not automatically improve it if the arrangement does not support responsible judgment.

• Permissions in agentic systems. The distinction between an assistant that drafts a message and an agent that sends it is a matter of authority to change something outside the conversation.⁸ Capability should not automatically confer permission. Consequential actions require explicit limits on accessible systems, transaction authority, and layered mitigations against vulnerabilities like prompt injection, where untrusted content alters a system's behaviour.⁹

The Question of Fairness and Equity

One dimension that deserves sustained attention is the distributional impact of AI systems, both in terms of algorithmic fairness and institutional access. Fairness is often discussed as a technical defect that can be removed through adjustment. However, foundational research has demonstrated that multiple mathematical fairness conditions for risk scores cannot generally be satisfied simultaneously.¹⁰ A system can perform well under one fairness measure and less well under another. The necessary response is not a single metric, but a governance framework that explains which errors are most consequential, whether performance differs across groups, and how individuals can seek review.

Beyond algorithmic metrics, responsible governance must consider the distribution of benefits. The customer-support study noted earlier found that benefits differed significantly across workers.¹¹ If AI assistance can deliver substantial benefits, the question of which workers or students have access to well-governed systems carries significant equity implications. Access should be understood broadly: an application may be technically available while remaining difficult to use in a relevant language, inaccessible to a person with particular needs, or unsupported when an error occurs. Evaluating equity requires looking at access, capability, and recourse, ensuring that institutional caution does not inadvertently widen existing gaps.

A Framing for Continued Inquiry

The most productive framing, in my view, is neither "AI must be adopted unconditionally" nor "AI must be treated as inherently untrustworthy." It is, rather: Under what conditions is reliance on this system justified—and how will we know when those conditions no longer hold?

This is an empirical and design question, not a binary one. It requires treating governance as a process that can learn, rather than a one-time declaration. A rigorous assessment defines the purpose, maps the system, establishes evidence requirements, pilots within boundaries, and preserves the ability to stop.¹²

I do not think responsible AI governance can be reduced to a choice between enthusiasm and caution. The evidence supports a conditional position: outcomes depend on the task, the user, the system design, and the institutional arrangement. I do not know what the right pace of adoption is for every institution and every use case. I suspect no one does, yet. What I think is important is that the conversation remains grounded in evidence, attentive to the sociotechnical reality of deployment, and oriented toward the people affected—ensuring that responsible judgment is possible before deployment, and meaningful correction is possible afterwards.

Footnotes

1. National Institute of Standards and Technology (NIST), AI Risk Management Framework (AI RMF 1.0), 2023.
2. Illustrating the distinct roles of voluntary frameworks (NIST), management standards (ISO/IEC 42001), and statutory obligations (EU AI Act).
3. NIST AI RMF emphasis on contextual risk management, third-party components, and the deployment environment.
4. Brynjolfsson, E., Li, D., & Raymond, L. R. (2025). "Generative AI at Work." The Quarterly Journal of Economics.
5. Stanford Accelerator for Learning, The Evidence Base on AI and K–12: A 2026 Review, March 2026.
6. Turpin, M., et al. (2023). "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting." NeurIPS.
7. Synthesizing NIST's generative-AI profile on automation bias and Singapore's guidance on meaningful human control.
8. Infocomm Media Development Authority (IMDA), Singapore, Model AI Governance Framework for Agentic AI, January 2026.
9. OWASP Top 10 for Large Language Model Applications, focusing on prompt injection and least privilege.
10. Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). "Inherent Trade-Offs in the Fair Determination of Risk Scores." arXiv.
11. Brynjolfsson et al., regarding the distributional effects of AI assistance on varying skill levels.
12. A synthesis of lifecycle principles from NIST, the OECD AI Principles, and Singapore's AI Verify framework.