AI governance in organisations: roles, policies and a 90-day plan to get started

What is AI governance?

AI governance is the full set of rules, processes and structures that steer the development, use and monitoring of AI within an organisation. It determines who reviews, approves, operates and controls AI applications.

Above all, AI governance governs the collaboration between business units, IT, data protection, legal and senior management when AI is used. A single compliance project or an IT policy on its own does not cover this. Many of these disciplines are not new: at its core, it is about sound corporate organisation that keeps pace with AI. Existing standards such as ISO 27001 or the Standard Data Protection Model (SDM) can serve as guidance.

The legal framework for AI governance is set mainly by four areas. The AI Act covers roles, risk classes, transparency and AI literacy. The GDPR covers personal data, legal bases, purpose limitation and the data protection impact assessment. Information security covers access, robustness, testing and logging. Employment law comes into play through the link to employees and the involvement of the works council.

Five reasons speak in favour of embedding AI governance in your organisation:

• Data quality: AI needs correct, complete and consistent data as its foundation.
• Efficiency and synergies: Shared rules foster innovation and operational efficiency.
• Risk management: Without governance, companies face legal, financial and reputational damage.
• Trust and acceptance: Governance builds trust among customers, employees, partners and other stakeholders.
• Regulatory obligation: Article 10 of the AI Act explicitly requires Data Governance for high-risk AI systems.

How fast a company can deploy AI, and how well it stays in control, therefore also depends on how its AI governance is set up.

How is AI governance structured?

AI governance can be built like a house with three levels. The foundation creates transparency across all AI applications, the pillars govern day-to-day work through policies and processes, and the roof defines responsibility, roles and oversight. Each level builds on the one below.

The three levels of an AI governance framework at a glance:

• Foundation, inventory and classification: Which AI systems, models and use cases exist, how high is their risk, and which data and vendors are affected?
• Pillars, policies and processes: Submission, assessment, approval, controlled operation, and monitoring and review form the rules for day-to-day work.
• Roof, structure and oversight: Roles and responsibilities, committees and overarching monitoring ensure steering at organisational level.

The order is no coincidence. In practice, companies like to start with the roof: they set up an AI Board which, at its first meeting, asks which AI systems the company actually has. Without a reliable answer to that question, you end up with a roof but no house.

Anyone building AI governance should therefore start with the foundation and can put the basic framework in place in around 90 days.

How do you build AI governance in 90 days?

A robust basic framework for AI governance can be created in 90 days and then matures step by step. A transformation programme lasting several years is not required.

The 90-day roadmap for AI governance is divided into four phases:

1. Days 1–30: Start the AI inventory, take immediate measures and define roles.
2. Days 31–60: Establish risk classification, the AI policy and the process for approving use cases.
3. Days 61–90: Set up the risk log, monitoring, training and a governance board (AI Board).
4. Thereafter: Embed reviews, audits and continuous improvement as an ongoing task.

The 90-day plan begins with the AI inventory because without an overview of the AI applications in use, risk management is not possible.

What belongs in an AI inventory and how do you assess the risk?

The foundation of AI governance consists of three building blocks: an AI inventory of all systems, models and use cases, a risk-appropriate classification and an early review of data and vendors. Only on this basis can you decide how strict the review and controls for an AI use case need to be.

What belongs in an AI inventory?

An AI inventory documents each AI use case using six elements, capturing the key components of an AI tool based on the available facts:

• Purpose: What problem does the application solve?
• Data and input: What data does it need?
• Performance and output: What counts as a successful use of AI?
• Human oversight: How do you prevent harm to the company?
• Technology: Which model and which application are used?
• Risk assessment: What is likely to go wrong? Here you assess potential harm, its severity and its likelihood.

How do you classify the risk of an AI use case?

The risk of an AI use case results from four perspectives. Together, they determine how strict the review and controls need to be:

• Purpose: Does the AI merely assist, does it make a recommendation, or does it make a decision?
• Data and data subjects: Are personal, confidential or special categories of personal data involved, such as health data under Article 9 GDPR? Such data increases the risk.
• Autonomy and impact: What are the consequences of incorrect output, and which control mechanisms are in place?
• Technology and vendor: How are interfaces, model training and hosting governed, and what dependencies exist?

One example is applicant scoring. For risk classification, it makes a difference whether the AI merely recommends a rejection or an offer, or whether the rejection is sent to applicants directly via the AI. From the technology and vendor perspective, you check, for example, whether the service provider trains the tool with synthetic or anonymised data or with real data from the engagement.

Which questions about data and vendors do you need to clarify early on?

Data and vendors belong in the assessment of an AI use case from the outset. You should answer four questions before introducing it:

• Data origin: Are you allowed to use the data for this purpose? This applies, for example, to customer data collected ten years ago as part of a customer relationship.
• Data flow: May the data be fed into this tool?
• Contractual and security position: How are training, hosting, deletion and subcontractors governed?
• Changes: How do you identify and assess changes to the model or contract?

With the AI inventory, risk classification and a clarified data basis, the foundation is in place on which the rules for day-to-day work are built.

What does an AI policy in a company govern?

An AI policy gives employees clear rules for working with AI day to day: which tools are approved, which content is off-limits, how they check results and whom to contact if problems arise. Together with AI literacy, it is one of the main pillars of AI governance.

These pillars also include the data protection impact assessment, the fundamental rights impact assessment from December 2027 for certain deployers of high-risk AI such as public bodies, risk management for AI vendors including procurement processes, the selection of training data and a training concept.

Which rules belong in an AI policy?

An AI policy covers four areas of rules:

• Approved tools: Which applications may employees use, for example via whitelisting of AI tools?
• Content requirements: Which content is prohibited or requires approval?
• Output rules: When and how must results be checked, documented or labelled, especially when used externally?
• Reporting channels: How and to whom do employees turn in case of uncertainty or incidents, for example an AI owner or AI officer?

How do you build AI literacy in your organisation?

AI literacy is part of AI governance. Under Article 4 of the AI Act, companies that provide or deploy AI systems must take measures to promote the AI literacy of their staff. The law does not set out an explicit obligation to provide evidence, but the Federal Network Agency (Bundesnetzagentur) recommends documenting these measures. Effective AI training meets four requirements:

• Role-based: Senior management, developers, business users and control functions use AI differently and need their own learning objectives.
• Use-case-specific: Training addresses the purpose, limits, affected groups and specific risks of the applications actually in use.
• Practical: Employees practise prompting, handling confidential data, checking output, labelling and escalation routes.
• Verifiable: Target groups, content, attendance, effectiveness checks and regular updates are documented.

The AI policy and AI literacy govern the use of approved tools, while use case approval governs how a new use case makes its way into operation.

How does the approval of an AI use case work?

The approval of an AI use case follows four steps: submit, assess, decide and follow up. The principle “No AI use without approval” sounds strict, but it does not mean that every use of an AI tool will have to go before a seven-member committee in future. What is meant is a reliable process in which relevant AI use cases are made visible, reviewed in a risk-appropriate way and decided transparently.

Every relevant AI use case goes through these four steps:

1. Submit: A use case profile describes the purpose, data and owner of the application.
2. Assess: A risk, data protection and security check and, for external solutions, a vendor check reveal the risk profile. The depth of the review depends on the actual risk.
3. Decide: The options are approval, approval with conditions or rejection. Conditions might include a limited user group or not using personal data.
4. Follow up: The owner, measures, evidence and a fixed review date are documented.

The question of the owner is deliberately placed at the start. As long as an AI project runs under the banner of innovation, there are many people involved and many opinions. As soon as it comes to who bears responsibility for productive use, things can go very quiet.

In most cases, the most sensible decision is likely to be approval with conditions. An approval is not an open-ended operating licence for all later developments of the system.

A use case approval is therefore only complete once a fixed date has been set for the next review of the AI use case.

How do you turn AI risks into concrete measures?

A risk log translates AI risks into management decisions. For each risk, it defines when it becomes relevant, which measure applies and who is responsible for it.

Simply naming a risk helps no one. It is technically correct that generative AI can hallucinate. For business units, however, this warning is about as helpful as being told that the road can be wet when it rains.

These five typical AI risks can be operationalised in the risk log as follows:

The model change deserves particular attention. Providers can update a model, replace it or change its behaviour while the user interface looks exactly the same. Technically and in terms of risk, the system can change considerably, which is why a model change may trigger a new change or approval process.

A risk log turns a vague “could be problematic” into a clear rule: if the problem occurs, a defined measure applies with a named person responsible.

What does the ongoing operation of AI systems require?

Once an AI system is in operation, the question of the conditions under which it may continue to run is at least as important as the initial approval. This includes controlled operation with human oversight and monitoring, as well as incident and change management for incidents and changes.

What does controlled operation of an AI system involve?

The controlled operation of an AI system rests on four elements:

• Human oversight: People review critical inputs and results. The reviewer needs the time, expertise and authority to stop a use.
• Monitoring: Quality, errors, complaints and anomalies are observed continuously.
• Documentation: Approvals, conditions, tests and decisions remain traceable.
• Review cycle: The frequency of reviews depends on the risk and the pace of change.

If human oversight exists only on paper, the “human in the loop” is nothing more than a nice decoration.

In monitoring, complaints are a particularly valuable source of information: if reports of unusable output pile up, it is worth taking a closer look. Documentation does mean extra work, but governance cannot be built on people's memories.

When it comes to the review cycle, a stable internal low-risk use case needs a different review cycle from an AI agent that constantly receives new data sources, serves external users and carries out actions independently.

How do you deal with AI incidents and changes?

Even with the best AI governance, something will go wrong at some point. Incident management ensures that an incident is detected, reported, contained, assessed and then followed up with corrective action.

Equally important is change management, because changes can significantly shift the risk profile of an AI system. These four changes trigger a re-assessment if they are significant:

• a new purpose
• new data
• a new model
• a new interface

An internal HR chatbot, for example, initially only answers general questions about travel expenses. Later, someone has the idea of connecting the personnel file in order to provide personalised answers. Technically, this may take just a few clicks, but from a governance perspective it is a completely different matter.

Lessons learned from incidents and changes feed back into controls, the AI policy and training.

Oversight during ongoing operation must match the risk and the pace of change, and it requires clearly assigned responsibilities.

Who is responsible for AI governance in a company?

AI governance distributes responsibility across several roles. Senior management decides on priorities and risk acceptance, the business unit is responsible for purpose and operation as risk owner or system owner, and IT, Legal, data protection, HR and the works council take on their own tasks, from technical controls and legal review through to co-determination rights.

These five roles share the tasks in AI governance:

• Senior management: prioritisation, resources, risk acceptance and decision-making
• Risk owner, system owner or business unit: defining the purpose, subject-matter quality, risk minimisation and operation
• IT, data and security: architecture, access, technical controls and monitoring
• Legal, compliance and data protection: legal requirements, legal documentation and review
• HR and works council: guardrails, employee data and co-determination rights

The parties involved pursue legitimate but differing interests. Management wants controllability and the ability to make decisions, while business units want benefits, speed and clear approvals. Users need practical rules for data, output and escalation.

Good AI governance enables speed as long as the risk remains manageable. To achieve this, control functions need to be involved early on rather than performing a final check. An email such as “Our project goes live on Monday, can you confirm by Friday that everything is fine?” is, strictly speaking, also a form of involvement, but if in doubt it leads to long delays.

Contributing is not the same as being responsible, which is why AI governance defines who makes the final decision at each step.

What governance does an AI agent built in Copilot Studio need?

An AI agent built in Copilot Studio needs AI governance that keeps pace with it: it is quick to build but can have far-reaching effects. Microsoft Copilot Studio is a platform that business units use to create and manage AI agents without traditional software development. An agent there can be a chatbot, a phone bot or an automated process running in the background.

In the fictitious case study “HR Compass”, an agent answers employees' frequently asked HR questions around the clock. It launches in Teams, with a website or phone bot planned later. It draws its knowledge via RAG, i.e. connected knowledge sources, from the HR handbook, works agreements and SharePoint. In future, it is intended to create tickets and initiate simple HR processes.

Risks can quickly arise from this sensible use case. Overly broad SharePoint permissions can surface confidential HR content in answers. A public website test without login fundamentally expands the user base. If the agent can initiate HR processes, faulty output becomes a real action. And HR, IT, data protection and the business user who built the agent each expect the others to give the approval.

Four key questions determine what AI governance HR Compass needs:

1. Classify: Which functions only provide information, and which intervene in employee processes? HR information, external channels and autonomous actions are classified separately, rather than classifying the entire agent across the board.
2. Limit: Which data sources, interfaces, user groups and channels are necessary for the purpose? Personnel files and external web or API sources are excluded for the time being.
3. Approve: Who is responsible for subject matter, technology, data protection, information security and go-live? Stage gates, i.e. fixed checkpoints before each expansion stage, ensure a documented decision by the business, IT and control functions.
4. Monitor: How are quality, misbehaviour, changes, incidents and shutdown controlled? In operation, this includes separate development, test and production environments, rules to prevent data leakage, groups in Microsoft Entra (Microsoft's identity and access management) for user management, and logging. In reviews, quality, incorrect answers, access, connectors and incidents are monitored and regularly reassessed.

The principle for limiting is: connect only what the purpose requires.

For emergencies, the agent also needs a kill switch. That sounds like James Bond but means something simple: in the event of an incident, someone must be able to shut down the agent and know who holds the admin rights.

HR Compass shows how AI governance safeguards a good use case: it identifies the points at which the use case could turn into a risk before it goes live.

Conclusion: good AI governance starts with the inventory

AI governance is a shared set of rules whose basic framework can be built in around 90 days. If you first know which AI applications are running in your company, you can assess risks realistically and align the depth of review accordingly. This creates speed where risk is low and control where employees, sensitive data or autonomous actions are involved.

Whatever you call it, AI governance becomes sustainable through interdisciplinary teams that are involved early and have clear responsibilities. Your own rules can be based on proven standards. Because models, data and usage are constantly changing, continuous monitoring takes the place of a one-off check.

Frequently asked questions

#1 Does every company need an AI Board?

An AI Board makes sense for high-risk AI use cases, exceptions and significant changes, but it is not a good starting point for AI governance. Without an AI inventory, the committee does not know which AI systems are running in the company. In a 90-day roadmap, the governance board (AI Board) is therefore only set up in the third phase.

#2 Does every use of AI in a company need to be approved?

Relevant AI use cases should go through an approval process whose depth of review depends on the risk. An internal low-risk use case needs less review than an AI agent involving personal data or autonomous actions. Approval with conditions, for example for a limited user group, is often likely to be the most sensible decision.

Does a model change by the AI provider trigger a new review?

A model change by the AI provider may require a new risk assessment, even if the user interface looks unchanged. Providers can update or replace models and thereby change the behaviour of an AI system. Change management identifies such changes and triggers a re-assessment where they are significant.

#3 What is a kill switch for AI agents?

A kill switch is the defined means of shutting down an AI agent in the event of an incident. This includes having decided in advance who holds the admin rights. Together with a dashboard, KPIs and escalation routes, the kill switch ensures ongoing oversight of the agent.

#4 Which standards can AI governance be based on?

AI governance can be based on existing standards such as ISO 27001 or the Standard Data Protection Model. The company builds its own AI governance that fits its processes and uses the standards as a framework. Many disciplines of AI governance are not new. What is new, above all, is the speed.