CI/CD for Microsoft Fabric: Version Control and Deployment Pipelines

Why CI/CD Matters for Microsoft Fabric

Too many enterprises treat their Fabric workspaces like shared spreadsheets — everyone edits in production, there is no change history, and a single mistake breaks reports for thousands of users. This is not acceptable for enterprise data platforms.

CI/CD for Fabric solves three critical problems: (1) accountability — every change is tracked, attributed, and reversible; (2) quality — automated testing catches breaking changes before they reach production; (3) compliance — regulated industries require audit trails for data pipeline changes. If your organization handles data subject to governance and compliance requirements, CI/CD is not optional — it is a regulatory expectation.

The good news: Fabric now provides native Git integration and deployment pipelines. Combined with Azure DevOps or GitHub Actions, you can build a fully automated CI/CD pipeline that rivals what software engineering teams have had for decades.

Fabric Git Integration: Setup and Configuration

Fabric's Git integration connects a workspace to an Azure DevOps Repos or GitHub repository. Here is the step-by-step setup EPC Group uses for enterprise clients:

Prerequisites

• Fabric workspace on a Fabric capacity (F64 or higher recommended for enterprise)
• Azure DevOps project with Repos enabled, or a GitHub repository
• Workspace admin permissions for the user configuring Git integration
• Service principal with Fabric API permissions (for automated deployments)

Configuration Steps

1. Create the repository structure. EPC Group recommends one repository per Fabric domain (Sales Analytics, Finance Reporting, etc.), with branches for each environment: dev, test, main (production).
2. Connect the development workspace. In Fabric, navigate to Workspace Settings > Git Integration. Select your provider (Azure DevOps or GitHub), authenticate, choose the repository, and link to the dev branch.
3. Initial commit. On first connection, Fabric serializes all workspace items to the repository. Review the initial commit — verify that all items exported correctly and no sensitive data (credentials, connection strings) is exposed in the serialized JSON.
4. Configure branch policies. In Azure DevOps or GitHub, set branch policies on test and main: require pull requests, minimum 1 reviewer, build validation (CI pipeline must pass), and no direct commits.
5. Connect test and production workspaces. Create separate Fabric workspaces for test and production. Link each to the corresponding branch (test and main).

Fabric Deployment Pipelines: Environment Promotion

Fabric's native deployment pipelines provide a governed promotion path from development to test to production. Unlike Git branch merging (which deploys serialized item definitions), deployment pipelines handle environment-specific configuration: different data source connections, capacity assignments, and parameter values per stage.

Setting Up a Three-Stage Pipeline

The deployment pipeline compares items between stages, showing what has changed (new, modified, deleted). Promotion is selective — you can deploy specific items or entire workspaces. For Power BI semantic models and reports, the pipeline also handles dataset refresh after deployment to ensure test and production reflect the latest schema.

Automating with Azure DevOps and GitHub Actions

Native deployment pipelines work well for manual promotion with a UI. But enterprise CI/CD requires automation — triggered by code commits, gated by tests, and logged for audit. Here is how EPC Group implements end-to-end automation:

Azure DevOps Pipeline Example

# azure-pipelines.yml
trigger:
  branches:
    include:
      - main

pool:
  vmImage: 'ubuntu-latest'

stages:
  - stage: ValidateAndTest
    jobs:
      - job: LintNotebooks
        steps:
          - script: |
              pip install ruff
              ruff check ./notebooks/ --select E,W,F
            displayName: 'Lint Python notebooks'

      - job: ValidateSemanticModels
        steps:
          - script: |
              # Validate TMDL schema
              dotnet tool install -g Microsoft.PowerBI.Tabular.Editor
              # Run Best Practice Analyzer rules
            displayName: 'Validate semantic models'

  - stage: DeployToTest
    dependsOn: ValidateAndTest
    jobs:
      - job: PromoteToTest
        steps:
          - task: PowerShell@2
            inputs:
              targetType: 'inline'
              script: |
                # Authenticate with service principal
                $token = (Get-AzAccessToken -ResourceUrl "https://api.fabric.microsoft.com").Token
                # Trigger deployment pipeline promotion
                $body = @{ sourceStageOrder = 0; isBackwardDeployment = $false } | ConvertTo-Json
                Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/deploymentPipelines/{pipelineId}/deploy" \
                  -Method POST -Body $body -Headers @{ Authorization = "Bearer $token" }
            displayName: 'Promote Dev to Test'

  - stage: DeployToProduction
    dependsOn: DeployToTest
    condition: succeeded()
    jobs:
      - deployment: PromoteToProd
        environment: 'fabric-production'  # Requires approval
        strategy:
          runOnce:
            deploy:
              steps:
                - task: PowerShell@2
                  inputs:
                    targetType: 'inline'
                    script: |
                      # Same pattern, sourceStageOrder = 1
                  displayName: 'Promote Test to Production'

GitHub Actions Alternative

For teams using GitHub, the workflow is similar. You will trigger on a push to main, run validation, and call Fabric REST APIs using a service principal.

To enhance security, follow these steps:

• Use GitHub Environments with required reviewers for the production deployment stage.
• Store the service principal credentials in GitHub Secrets, never in the repository.

Notebook Versioning Best Practices

Notebooks are the most challenging Fabric item to version control effectively. They contain code, markdown, configuration, and potentially output cells with data. EPC Group's best practices:

• Clear outputs before committing. Output cells can contain sensitive data (query results, sample rows, PII). Configure notebooks to auto-clear outputs on save, or add a pre-commit hook that strips outputs from .ipynb files.
• Extract shared logic into libraries. Move reusable functions (data quality checks, logging, configuration loading) into Fabric Environment libraries or custom wheel packages. This reduces notebook size, enables unit testing, and avoids code duplication across notebooks.
• Use parameterized notebooks. Never hardcode file paths, table names, or dates. Use Fabric pipeline parameters passed to notebooks at runtime. This makes notebooks environment-agnostic and testable.
• Enforce code review for all notebook changes. Require pull requests with at least one reviewer for any notebook change. Data engineering notebooks are production code — treat them with the same rigor as application code.
• Add automated testing. Write pytest tests for notebook helper functions. Run these in the CI pipeline using a Python step. For full notebook testing, use Fabric REST APIs to trigger notebook execution in the test workspace and validate output table row counts and data quality metrics.

Lakehouse and Warehouse Deployment Considerations

Lakehouses and Warehouses present unique CI/CD challenges because they contain both metadata (schema definitions, shortcuts) and data. Git integration versions the metadata, not the data. Here is how to handle this:

Enterprise CI/CD Maturity Model for Fabric

Not every organization needs full automation on day one. EPC Group uses a four-level maturity model to help enterprises adopt CI/CD for Fabric incrementally:

Frequently Asked Questions