We recently asked whether not using AI could become a professional liability risk. But there’s another side to that question: If organizations are going to use AI, how do they govern it responsibly?
That question is getting harder as AI models become more sophisticated. Organizations may be able to see what goes into a system and what comes out without having a complete, reliable view of how the model reached its conclusion. That changes the governance question. Instead of relying on “Can we see how the AI thinks?” organizations need to ask: “Can we control what it does and reconstruct what happened?”
Vendor certifications and safety evaluations matter, but they don’t replace controls within your own organization. The NIST AI Risk Management Framework emphasizes ongoing governance and management of AI risk. For organizations putting that into practice, five controls are especially important.
Capture what you can: inputs, outputs, data access, tool calls and actions. If something goes wrong, you should be able to reconstruct what the system was asked to do and what it actually did.
Give AI systems only the permissions they need. Limit access to data, tools, systems and the actions they can take. The goal is simple: if the system behaves unexpectedly, limit the potential consequences.
Human approval for everything defeats much of AI’s value. Instead, establish clear thresholds. The higher the stakes — and the harder an action is to reverse — the stronger the case for human review.
If you can’t reliably review how the model reached an answer, independently check important results. That could mean predefined tests, another source or system, or periodic human review. The level of validation should reflect the consequences of getting the answer wrong.
AI governance also means preparing for vendor changes. Prices, products, terms and capabilities can change. Know how you would switch providers, maintain critical functions manually and retain the data and documentation you need.
Being able to shut down an AI system can be useful, but it isn’t a substitute for governance. By the time you need the kill switch, the system may already have taken an inappropriate action. Logging, access controls, approval thresholds and monitoring help reduce the likelihood and impact of that action in the first place.
As we discussed in our earlier article on AI and professional liability risk, organizations increasingly need a thoughtful approach to how AI fits into professional decision-making. That means recognizing AI’s value while putting practical controls around its risks.
At RiskVersity, we help organizations identify emerging risks like AI and translate them into practical steps that protect their people, operations and organizations. AI may be changing quickly, but the fundamentals of good risk management still apply: understand the exposure, establish appropriate controls and be prepared when something doesn’t go as planned.
You may not always be able to see exactly how an AI system reached its conclusion. But you should know what it is allowed to do — and be able to determine what it did.
Corporate Office
1435 Vine Street, Suite 326
Cincinnati, Ohio 45202
513-644-1085
Charlotte Office
525 N Tryon Street, Suite 1600
Charlotte, NC 28202