The direct answer
A private AI solution keeps prompts, business data and, where required, the model itself inside boundaries the organization controls. With Microsoft technology there are three ways to do that, and most enterprises need more than one: regulated records may have to stay on servers in their own datacenter, while everyday drafting and search can run on models hosted in their own Azure tenant.
Three deployment patterns
| Pattern | Where the model runs | Fits | Status (per Microsoft Learn) |
|---|---|---|---|
| Tenant-hosted | Models sold by Azure, including Azure OpenAI and open-weight families, deployed in your Foundry resource in a geography you choose | Most enterprise workloads that need scale without running GPUs | Available now; data at rest stays in the chosen Azure geography, but Global deployments may process prompts in any Azure region and Data Zone deployments only within the US, EU or APAC data zone; Standard and Regional Provisioned keep processing inside the geography (Microsoft Learn, Deployment types in Microsoft Foundry Models, accessed 2026-10-28) |
| On-premises | Foundry Local on Azure Local: an Arc-enabled Kubernetes cluster in your datacenter, CPU or GPU, models from the Foundry catalog or your own registry | Sovereignty or regulatory rules, low latency next to the data, disconnected sites | Preview |
| On-device | Foundry Local inside the application on the user’s Windows, macOS or Linux device | Offline work and sensitive data that should never leave the laptop | Single-user inference; not designed as a server stack |
Requirements you can verify
“Private” is not one setting. Write each requirement down, then check it against the deployment instead of trusting a label. With models sold by Azure, data at rest stays in the Azure geography for every deployment type, but the deployment type decides where prompts are processed: a Global deployment may process them in any Azure region.
| Requirement | How to meet it | How to check it |
|---|---|---|
| Prompts are processed only inside the EU (or the US) | A Data Zone Standard or Data Zone Provisioned deployment in a region of that zone, or a Standard deployment in your geography | The deployment’s SKU reads DataZoneStandard, DataZoneProvisionedManaged or Standard; an Azure Policy on the subscription denies GlobalStandard |
| Stored data stays in our geography | Any deployment type: files, Responses API history and other stored data stay in the resource’s Azure geography | The resource’s region; check that the feature you rely on is generally available, because preview features may not support every data commitment |
| Microsoft staff do not review our prompts | Approved modified abuse monitoring | ContentLogging = false in the resource’s JSON view or the Azure CLI output |
| No public network path to the models | A private endpoint, with public network access set to Disabled | The private endpoint connection shows Approved, and nslookup inside the network returns its private IP |
| Inference keeps working with no internet connection | Foundry Local on Azure Local (preview) on your own cluster | Run the workload with the site disconnected and record the result |
| Data never leaves the user’s device | Foundry Local on the device | Run inference with networking off and record the result |
Record the result per workload. The private AI deployment record (CSV) has a row per workload for the data class, the pattern, the deployment type, the processing and storage boundaries, abuse monitoring, network access, key management and how each was verified, with sign-off columns. Its example rows are labeled SYNTHETIC-EXAMPLE, not client data.
Where multi-model AI comes in
A private deployment is rarely one model. EPC Group’s multi-model AI architecture puts every model behind one governed gateway, so each request goes to the model and location its data allows: open-weight models such as Llama or Mistral on infrastructure the client controls for regulated data, tenant-hosted models for general work, and Microsoft 365 Copilot where it already fits. Access control, logging and evaluation are shared across all of them, so a model can be swapped without rebuilding the application. The multi-model AI launch for Power BI shows the same pattern applied to analytics, and the engineering playbook covers the gateway layer.
How EPC Group delivers it
- Classify the data each AI workload touches and decide which of the three patterns each class allows.
- Pick the models per workload, including open-weight options for on-premises use.
- Stand up the gateway: identity with Microsoft Entra ID, logging, content safety and cost tracking in one place.
- Deploy the on-premises or on-device pieces and the tenant-hosted models, then evaluate quality on the client’s own test data.
- Hand over runbooks and the governance controls auditors will ask about.
Primary sources
- Data, privacy, and security for Foundry Models sold by Azure — Microsoft Learn, accessed 2026-10-27
- What is Foundry Local on Azure Local? — Microsoft Learn, accessed 2026-10-27
- What is Foundry Local? — Microsoft Learn, accessed 2026-10-27
- Foundry Models sold by Azure — Microsoft Learn, accessed 2026-10-27
- Understanding deployment types in Microsoft Foundry Models — Microsoft Learn, accessed 2026-11-02
- How to configure network isolation for Microsoft Foundry — Microsoft Learn, accessed 2026-11-02
Frequently Asked Questions
Can a private AI model run without sending data to the cloud?
Yes. EPC Group uses two Microsoft options for this. Foundry Local runs models on the user’s device, where data never leaves the device and no Azure subscription is required; Foundry Local on Azure Local runs models on an Arc-enabled Kubernetes cluster in your own datacenter and can operate in disconnected environments (Microsoft Learn, accessed 2026-10-27).
Are prompts sent to Azure-hosted models used to train them?
No, according to Microsoft. For models sold by Azure, including Azure OpenAI models, prompts, completions, embeddings and training data are not available to other customers or to the model providers, and are not used to train generative AI foundation models without your permission or instruction (Microsoft Learn, accessed 2026-10-27).
Which open-weight models can run on-premises?
Microsoft’s Foundry Local catalog covers chat models such as GPT OSS, Qwen, DeepSeek, Mistral and Phi, and Foundry Local on Azure Local can deploy models from the Foundry catalog or from your own registry (Microsoft Learn, accessed 2026-10-27). EPC Group chooses the model per workload rather than standardizing on one.
Is Foundry Local on Azure Local generally available?
Not yet. Microsoft lists Foundry Local on Azure Local as a preview, so features can change before general availability (Microsoft Learn, accessed 2026-10-27). EPC Group plans production designs around that status and keeps a tenant-hosted fallback for workloads that cannot wait.
How do we prove where prompts are processed?
Read the deployment type. GlobalStandard may process prompts in any Azure region; DataZoneStandard only inside the US, EU or Asia Pacific data zone; Standard inside the resource’s geography. Assign an Azure Policy that denies the deployment types your rules do not allow, so a new deployment cannot drift. Stored data, such as files and Responses API history, stays in the resource’s geography either way (Microsoft Learn, accessed 2026-11-02).
Why use more than one model for private AI?
Because no single model or location fits every workload. In EPC Group’s multi-model AI architecture, regulated data stays on models running on infrastructure the client controls, while general productivity work uses Microsoft 365 Copilot or tenant-hosted models, all behind one governed gateway with shared logging and access control.
Talk to a senior architect
A one-hour scoping call with a senior EPC Group architect covers which data must stay on your infrastructure, which models fit each workload and what the gateway needs. Email contact@epcgroup.net or call 888-381-9725.
Documented and anonymized engagements
Named engagements come first; anonymized summaries follow under the anonymized-engagement rule — the client is not named and every statement is EPC-reported. Documents behind named engagements are in the Evidence Center.
- Azure Data Warehouse and Power BI: Medical Devices (2025)Anonymized engagement summary · Medical device manufacturer · 2025
- Copilot and SharePoint AI Governance: Regional Bank (2024)Anonymized engagement summary · Regional bank · 2024Engagement phase: governance and readiness planning before a Copilot rollout; no rollout is reported
- Generative AI for Contract Audits: Transit District (2023)Anonymized engagement summary · Public transit district · 2023Engagement phase: discovery, assessment and design of an Azure generative AI capability, not a Microsoft 365 Copilot rollout; go-live is not reported
- Backup Servers to Azure: Title and Settlement ServicesAnonymized engagement summary · National title and settlement services company · 2021
- AI Governance Framework: Hospital NetworkAnonymized engagement summary · Healthcare
- Azure Cloud Migration: Global ManufacturerAnonymized engagement summary · Manufacturing
