Hey đ
Iâm Oliver Patel, author and creator of Enterprise AI Governanceâwhich today celebrates its 30th edition! Thanks for all your support and engagement on this journey!
This two-part series provides my personal view, from the enterprise AI governance coalface, on the top 10 challenges facing AI governance leaders today.
This is not a theoretical, policy, or legal analysis. Rather, it is a contemporary examination of the messy and relentless reality of being tasked with leading AI governance in an enterprise contextâwhere AI is being adopted at scale and promoted as integral to the future competitiveness and relevance of the organisation.
The purpose of this two-part series is to guide AI governance professionals with practical insights on how to tackle some of the most pressing challenges we face. Below are the 10 challenges this series covers in detail. Part 1 covers challenges 1-3 (as they are inextricably linked) and part 2 (coming next week) covers challenges 4-10.
The lack of an established blueprint for enterprise AI governance, and the immense, interdisciplinary challenges we face each day, is exactly why I am writing my upcoming book, Fundamentals of AI Governance. It is packed full of practical tips and strategies for enterprise AI governance leaders, including a visual deep dive on each of these 10 challenges. To secure a 25% discount during the pre-launch period, sign up at the link below.
Top 10 Challenges for AI Governance Leaders in 2025
The âdemocratisation dilemmaâ. How to maintain robust oversight and promote compliance when the ability to develop, deploy, and use AI is democratised and widely accessible?
Volume and velocity. How to keep up with the sheer volume and rapid pace of enterprise AI initiatives, whilst cutting through the noise and deploying finite resources and expertise on the highest value work?
Refining the risk-based approach. How to respond to the AI risk âvibe shiftâ and effectively target governance on the relatively small proportion of AI systems and use cases that could pose significant risks?
Protecting confidential business data. How to protect confidential business data when there is immense hunger to experiment with and use the latest AI applications that are released on the market?
Ongoing vendor due diligence and oversight. How to move beyond âpaper-basedâ vendor due diligence and apply continuous oversight on the performance, trustworthiness, and safety of externally provided AI applications?
AI engineering: building with foundation models. How to determine your rights, responsibilities, and liabilitiesâas well as the novel risks and tangible mitigationsâwhen building AI systems with foundation models provided by external organisations?
Open-source AI model oversight. How to effectively govern the widespread access and use of open-source AI models, to safeguard your organisation from legal, compliance, and cyber security risks, whilst promoting innovation?
Embedding compliance by design. How to build AI systems that promote compliance by design and default, to make it seamless for your workforce to do the right thing?
Agentic AI governance: taking the human out of the loop. How to promote responsible, meaningful, and empowered human oversight of AI, when the fundamental goal of agentic AI is to take the human out of the loop?
Digital governance silos and inefficiencies. How to effectively streamline and integrate your disparate digital governance and risk management processes and capabilities, to improve the user experience and accelerate AI innovation, whilst also strengthening your compliance posture?
1. The âDemocratisation Dilemmaâ
How to maintain robust oversight and promote compliance when the ability to develop, deploy, and use AI is democratised and widely accessible?
The launch of ChatGPT in November 2022, and the subsequent generative AI boom, was a watershed moment. Not only because of fundamental advances in model architecture or training and inference techniques, but because powerful generative AI applications were now, for the first time, at the fingertips of all employees. By some user adoption measures, ChatGPT was the fastest-growing consumer application of all time. Fast forward nearly three years, and it is evident that the democratisation of AIâand its widespread accessibility and availabilityâis perhaps the most important development impacting the enterprise AI world.
Things have moved on from everyone merely having access to powerful chatbots for simple personal assistance and productivity tasks. The ability to customise, use, and share generative AI applications, for scalable, business critical use casesâand even to build AI powered-tools for production deployment in business workflows without any hard technical skillsâis significantly increasing the volume of AI activities and use cases for which oversight is often lacking.
For example, non-technical employees can use no-/low-code agent builder platforms for âdrag-and-dropâ creation of domain specific chatbots and assistants that can retrieve and surface information, call APIs, execute tasks, and augment and automate important business workflows. Similarly, non-technical employees can also build and scale custom GPTs and generative AI âprojectsâ that are augmented with vast amounts of data and documentation, and optimised to perform in a particular way, via detailed prompts and instructions
All major AI product releases highlight that the trend towards democratisation is accelerating. And it is safe to assume that what the average non-technical employee can do with AI will only become more sophisticated over time.
This compels AI governance leaders to reevaluate how to govern the development and use of AI. Up until now, most organisations distinguished between the AI models and AI systems that their data science and engineering teams build, for production deployment, and the everyday use of generative AI applications by the wider workforce. With respect to the latter, AI governance frameworks do not typically require AI governance and risk assessments, or the implementation of lifecycle controls and risk mitigations. Aside from following the AI usage policies and guidelines, employees are generally free to use approved generative AI applications for their personal work without oversight.
However, this position wonât hold for much longer. For example, if anyone can use a no-code AI platform to build a high-risk CV screening tool that dozens of other employees can use, then, given the compliance responsibilities and ethical implications, this cannot be considered an everyday use of AI warranting no formal governance.
Remember, under the EU AI Act, if your organisation uses a general-purpose AI system for recruitment and candidate selection, and it was not intended to be used for this purpose, you could inadvertently become the provider and the deployer of a high-risk AI systemâwithout even realising.
Practical solutions: the three Gs for dealing with AI democratisation
Guidance: Educate all employees on their responsibilities and the way in which their seemingly everyday use of AI can give rise to compliance obligations. Furthermore, ensure that any AI use case or AI system that has the potential to be classified as a high-risk AI system under the EU AI Act, or even a potentially prohibited AI practice, undergoes rigorous AI governance assessment and review, irrespective of the AI platform, technology, or approach used to develop and deploy it. Remember, the law is technology neutral.
Guardrails: Implement guardrails and technical safeguards into the AI platforms that are widely available and âdemocratisedâ, so that certain types of AI use cases cannot be pursued or progressed to production (e.g., those that could be prohibited), and to enable potential compliance risks and incidents to be monitored, detected, and flagged on a continuous basis. This could involve monitoring specific content in prompts, documentation, and model outputs to flag potential violations or data-related risks.
Greenlight: Make it clear precisely which AI platforms and tools are available and approved for use, as well as which data sources, AI-powered workflows, and patterns have been approved for production use. This enhances organisation-wide understanding of what is permissible, effective, and safe, thereby enhancing your AI governance posture without slowing the business down.
2. The Volume and Velocity Challenge
How to keep up with the sheer volume and rapid pace of enterprise AI initiatives, whilst cutting through the noise and deploying finite resources and expertise on the highest value work?
The volume and velocity challenge is directly linked to, and amplified by, the âdemocratisation dilemmaâ. The easier it becomes to develop, use, and scale AI applications for business-critical use cases, the more AI use cases there are to govern.
However, surging AI volume and velocity has other causes, such as increased investment in AI capabilities across the board, strong incentives for corporate leaders to demonstrate successful AI adoption, reduced barriers to entry due to the widespread availability of foundation models, rapid advances in AI engineering techniques to capitalise on these models, and technology vendors introducing myriad AI capabilities into every product, platform, and offering.
In this context, AI governance functions are struggling to keep up with the sheer volume of AI initiatives and use cases which they are tasked with assessing, reviewing, monitoring, and approving. Furthermore, the rapid pace at which this volume is increasing, combined with the ever-changing nature of the AI use cases and their underpinning technologyâ makes it even harder to know how to tackle this problem and structure your AI governance functionâs workload.
There are three common symptoms of this challenge, all of which are detrimental to long-term AI governance success.
First, the volume challenge leads to practitioners with expertise on AI ethics, regulatory compliance, and broader legal or technical issues spending excessive time on lower value work, such as processing and reviewing inherently low-risk AI use cases, or providing similar advice repeatedly.
Second, the velocity challenge means that the nature of AI use cases and AI technology is changing fast. Risk assessment questions, lifecycle control frameworks, and training and guidance can quickly become outdated. For example, risk assessment and documentation artefacts designed for generative AI will likely be missing the mark for the new agentic AI systems that are being developed and prepared for production.
Finally, increased âdemandâ on the AI governance function means more noise and more potential distractions. This can lead to mission drift and not prioritising the most important work that is essential to protect the organisation.
Practical solutions: the three As for handling AI overload
Automate: Leverage automation and AI to speed up the process of triaging, classifying, and assessing AI use cases, as well as assigning governance and oversight pathways and reviewers (based on the inputs provided in AI governance assessment submissions). Given the volume, it may not be feasible to stick with manual human review of every single submission. Therefore, leveraging automation enables smarter allocation of work to AI governance experts. However, assurance and quality control processes should be implemented to ensure the automation is effectively serving its purpose. It should also be implemented in parallel to the human-led process, at least initially.
Artefacts: Develop and distribute reusable governance artefacts for common AI patterns and applicationsâsuch as chatbots, research agents, and document processing. When the twentieth team wants to build a document assistant bot, they need not start from scratch. Pre-approved patterns, risk mitigation control sets, and documentation templates are win-win, as they support the business and free up AI governance SME time for higher-value work.
Adapt: Dealing with velocity requires constant and targeted adaptation. Continuously and periodically review and assess whether your policies, risk assessment processes, guidelines, and artefacts are fit for purpose, making focused and impactful changes where necessary.
3. Refining the risk-based approach
How to respond to the AI risk âvibe shiftâ and effectively target governance on the relatively small proportion of AI systems and use cases that could pose significant risks?
Democratisation is making it easier for anyone to develop and deploy AI, which is challenging the notion of what type of AI activities do and do not require oversight and governance. It is also a chief contributor to the volume and velocity challenge, which risks overwhelming enterprise AI governance teams, unless they effectively adapt and restructure their work.
These three trendsâdemocratisation, volume, and velocityâare also contributing to an AI risk âvibe shiftâ, which many AI governance leaders are noticing and impacted by.
The use of AI is now ubiquitous and embedded in the daily lives of millions of people and workers worldwide. A Harvard Business Review study from April 2025 found that âtherapy and companionshipâ was the top generative AI use case. A more recent study from economists at OpenAI and the National Bureau of Economic Research found that âpractical guidanceâ is the most common conversation theme for ChatGPT, with the top three topic areas being 1) tutoring or teaching, 2) âhow toâ advice, and 3) health, fitness, beauty, and self-care.
As AI adoption increases, the perception of AI risk is changing. The common everyday use of AI is undoubtedly impacting how its risks and potential dangers are perceived. For many in the corporate world, it may be hard to connect more abstract risk themes, such as bias, explainability, and safety, with their everyday use of AI, which could feel merely assistive and somewhat harmless. Furthermore, if the organisation you work for is yet to suffer or be impacted by a major AI incident, and it is not an issue that is much discussed by the company leadership, this can also make the importance of AI governance feel harder to grasp.
But this is not just about perception. As AI adoption increases, the actual proportion of fundamentally low risk and non-material use cases is also likely increasing. Although this will vary for each organisation, the more people that use AI, and the more it is deployed to augment, optimise, and automate the vast range of back-office processes that, on balance, probably do not have the potential to materially impact peopleâs lives, the more important it is for AI governance leaders to focus their attention and resources on what matters most.
AI governance practitioners need to be alert to the resulting âvibe shiftââas well as the actual changes in the nature and distribution of AI use cases and their inherent risk levels, that are being driven by these trends.
As I have always said, AI governance does not mean governing all AI. You need to be proportionate, pragmatic, and risk-based in your approach. Trying to wrap your arms around anything and everything just because it has âAIâ in it is futile.
A better path forward is to refine and reevaluate your risk-based approach to AI governance, including how you classify the risk level of AI systems and AI use cases, and deploy resources and mandate governance and oversight accordingly.
This refinement needs to account for the fact that ostensibly everyday uses of AI can give rise to high-risk use cases, but also that most AI use cases and initiatives should probably not be classified as high-risk, irrespective of their development techniques and underpinning technology.
This is because as AI adoption surges, the volume of use cases and activities is becoming almost impossible to manage. Therefore, itâs never been more important to filter out the noise with razor sharp triaging, risk classification, and prioritisation.
In sum, to govern AI effectively in 2025, it is crucial for AI governance leaders to adapt to the societal and corporate âvibe shiftâ that is underway. We must be targeted, selective, and proportionate in our approach, to stay credible and focused on what really matters. Structure your approach so that a large majority of your teamâs time and invaluable expertise is spent on the small proportion of AI activities that pose significant risks.
Practical solutions: the three Ps for refining your risk-based approach
Prioritise: Lead the conversation to redefine exactly what constitutes high-risk AI in your organisation in 2025. Instead of navigating opaque vibe shifts, formally agree and document your leadershipâs risk appetite and risk-based approach to AI, in light of all the themes covered above. Most AI use cases donât warrant intensive governanceâpretending otherwise will undermine your credibility. Once you have done this, ensure your teams are spending most of their time on what is truly high-risk.
Proportionate: Apply governance intensity and scrutiny that matches the actual risk level. The rigour applied to and experts involved in the AI governance review, the amount of technical documentation required, the lifecycle controls that must be implemented, and the level of continuous monitoring and governance oversight must all flex, depending on the nature of the AI system or AI use case. This ensures your governance effort scales with impact and risk, not just with AI adoption.
Prove: Combat the vibe shift head-on by making your catches and wins visible. If you prevent an AI system from being deployed that poses potential ethical, legal, compliance, or reputational risksâor if a similar incident materialised in another organisationâdocument and communicate this to the right people in a subtle yet informative manner. This could be packaged up as periodic AI governance impact reports. If senior stakeholders canât see the fires youâre preventing, they may question why you need the fire extinguisher.