AI Center of Excellence (AI CoE) Build Playbook
An AI CoE sits outside individual business units and owns the four things that make AI-at-scale actually work: governance, the shared platform, the initiative portfolio, and enablement. This is the EPC Group playbook refined across 14 AI CoEs built since 2023.
The four things an AI CoE owns
- AI governance framework and standards — taxonomy, policies, RACI, incident response.
- The shared AI platform — Azure OpenAI, Copilot Studio, custom LLM infrastructure.
- AI initiative portfolio + prioritization — every use case tracked, budgeted, measured.
- Enablement and adoption — the training, playbooks, and tooling that get AI used at scale.
First 90 days
- Days 1-30 — charter definition, sponsor alignment, current-state AI inventory (shadow AI + sanctioned AI), governance framework baseline.
- Days 31-60 — platform architecture decision (Copilot + Azure OpenAI + Copilot Studio + custom LLM as needed), first 3 use case wins scoped, adoption plan draft.
- Days 61-90 — platform deployment, first 3 use cases in pilot, governance framework v1 published, quarterly cadence with executive sponsor established.
Frequently Asked Questions
What is an AI Center of Excellence?
An AI Center of Excellence (AI CoE) is a cross-functional team responsible for defining, deploying, and governing AI initiatives across the enterprise. It sits outside individual business units — reporting typically to the CEO, COO, or the vCAIO / CAIO — and owns four things: (1) the AI governance framework and standards, (2) the shared AI platform (Azure OpenAI, Copilot Studio, custom LLM infrastructure), (3) the AI initiative portfolio and prioritization, (4) enablement and adoption across business units.
What does an AI CoE team look like?
The right team size depends on scale. Minimum viable CoE (small enterprise, under 5,000 employees): 4-6 people — one AI governance lead, one platform architect, one MLOps engineer, one adoption/enablement lead, plus a fractional vCAIO. Mid-enterprise (5,000-25,000 employees): 8-12 people adding a data scientist, a Copilot Studio developer, and a legal/compliance specialist. Large enterprise (25,000+ employees): 15-25 people with sub-teams for governance, architecture, engineering, and adoption. EPC Group builds and hands off the CoE — the target state is the customer team operating independently within 12-18 months.
What is the first 90 days?
Days 1-30: charter definition, sponsor alignment, current-state AI inventory (shadow AI + sanctioned AI), governance framework baseline. Days 31-60: platform architecture decision (Copilot + Azure OpenAI + Copilot Studio + custom LLM as needed), first 3 use case wins scoped, adoption plan draft. Days 61-90: platform deployment, first 3 use cases in pilot, governance framework v1 published, quarterly cadence with executive sponsor established. Every EPC Group CoE build follows this cadence.
How is the AI CoE funded?
Three funding models we see in practice. (1) Centralized cost center — CoE budget comes from IT or the CEO office; business units get shared services at no charge. Simple but creates prioritization tension. (2) Chargeback — business units pay for CoE services (platform seats, use case dev, advisory hours) via internal chargeback. Aligns incentives but requires internal FinOps. (3) Hybrid — CoE governance + platform is a cost center; use case dev and advisory hours are chargeback. This is what EPC Group most commonly recommends because it separates the non-negotiable (governance) from the optional (use case dev).
What metrics should an AI CoE track?
Four categories. (1) Governance metrics — number of AI initiatives with approved risk assessment, number under active monitoring, audit findings open vs closed. (2) Platform metrics — active Copilot licenses vs deployed, Copilot Studio agents in production, Azure OpenAI PTU utilization, cost per model call. (3) Use case metrics — initiatives in ideation / build / production, TCO per initiative, ROI per initiative measured as time-savings or revenue impact. (4) Adoption metrics — percent of workforce actively using AI tools, satisfaction scores, training completion rates. EPC Group builds Power BI dashboards for all four categories at CoE launch.
What track record does EPC Group have building AI CoEs?
14 AI Centers of Excellence built since the practice launched in 2023. Delivered across healthcare, financial services, government, energy, and manufacturing. Every engagement is senior-architect-led — the same architect who scopes the CoE build stays through hand-off. Not a "here's a CoE-in-a-box, good luck" delivery model.
Talk to a senior architect
Email contact@epcgroup.net or call 888-381-9725.
North America's oldest continuous Microsoft Gold Partner (2000 until Microsoft retired the program in 2022) — today holding all six Microsoft Solutions Partner Designations.