Engineering at Scale: Microsoft Fabric Data Engineering
Enterprise Guide 2026: Lakehouse architecture, Spark notebooks, data pipelines, Delta Lake, and governance best practices for production-grade data engineering.
Microsoft Fabric Data Engineering is the Spark-based workload inside Microsoft Fabric for building enterprise data pipelines, lakehouses, and Delta Lake tables. This guide covers Lakehouse architecture, Notebooks, medallion architecture, Delta Lake, data governance best practices, and cost optimization strategies for enterprise teams in 2026.
Key Facts
- Fabric Data Engineering uses Apache Spark for data transformation, with Notebooks and Spark Job Definitions as the primary authoring surfaces.
- Data written by Spark is instantly available to Fabric Warehouse (T-SQL), Real-Time Intelligence (KQL), and Power BI (Direct Lake) — without copying.
- Delta Lake is the default table format in Fabric Lakehouses. It supports ACID transactions, time travel, and schema evolution.
- Reserved F64+ capacity saves approximately 40% vs. pay-as-you-go pricing for predictable Spark workloads.
- EPC Group: 500+ Microsoft Fabric projects, 29 years of Microsoft data platform experience.
What Is Data Engineering in Microsoft Fabric?
Featured Snippet: Data engineering in Microsoft Fabric is a unified field. It combines lakehouse architecture, Apache Spark notebooks, Data Factory pipelines, and Delta Lake storage into one SaaS platform.
Fabric removes the need to connect different Azure services. Data engineers benefit from:
- OneLake storage
- Spark compute
- Orchestration
- Governance
All these features are available in a single experience with zero infrastructure management.
Microsoft Fabric has changed how enterprise data engineering teams build, manage, and oversee data platforms. In the past, creating an enterprise data pipeline on Azure required setting up and integrating at least four different services:
- Data Factory
- Azure Synapse Analytics
- Azure Data Lake Storage
- Azure Databricks
- Azure Data Lake Storage Gen2 for storage
- Azure Synapse or HDInsight for Spark processing
- Azure Data Factory for orchestration
- Power BI for downstream analytics
Each service had its own security model, billing structure, and operational overhead.
Fabric simplifies the entire data stack into a single, capacity-based SaaS experience. It features a Lakehouse that merges the flexibility of a data lake with the organization of a data warehouse.
- They write transformations in Spark notebooks.
- They orchestrate workflows with Data Factory pipelines.
- The results flow directly into Power BI, all on the same OneLake storage layer.
This setup eliminates data movement between services, separate access control configurations, and cluster management.
This guide addresses key areas of Fabric data engineering that enterprise teams must understand by 2026. These areas include:
- Lakehouse architecture
- Spark notebooks
- Data pipelines
- Shortcuts
- Delta Lake optimization
- The medallion pattern
- Performance tuning
- Governance
- Cost management
Whether you are migrating from Azure Synapse, Databricks, or starting from scratch, this is your complete reference.
Fabric Lakehouse Architecture
The Fabric Lakehouse is essential for data engineering. It combines the benefits of both a traditional data warehouse and a data lake.
Key features include:
- Schema-on-write: This allows for structured tables.
- Schema-on-read: This enables the ingestion of raw files.
- Unified storage layer: All data is managed on a single platform.
Every Lakehouse in Fabric automatically creates two endpoints. One is a Spark endpoint for notebook-based processing. The other is a SQL analytics endpoint for T-SQL queries. Both endpoints access the same Delta tables in OneLake. This design reduces data duplication and removes ETL requirements.
This dual-endpoint architecture allows:
- Data engineers to write Spark transformations.
- Analysts to query the same tables using SQL.
- Each user to utilize their preferred tool.
OneLake Storage
Single data lake for the entire organization. All Fabric workspaces share OneLake, eliminating data silos and redundant copies across teams.
Delta Lake Format
All Lakehouse tables use Delta format — ACID transactions, time travel, schema evolution, and Z-order optimization built into every table.
Dual Endpoints
Every Lakehouse exposes Spark and SQL analytics endpoints simultaneously. No data movement — one table, two access patterns.
Files + Tables
Store unstructured files (CSV, JSON, images) in the Files section and structured Delta tables in the Tables section — same Lakehouse.
EPC Group suggests using a multi-Lakehouse pattern for enterprise deployments. This approach includes creating one Lakehouse for each domain. Key domains include:
- Sales
- Finance
- Operations
All of these will be organized within a shared workspace.
Key benefits of this approach include:
- Domain isolation for enhanced security
- Improved governance
- A unified data fabric
Cross-Lakehouse references can be created using shortcuts.
Each domain team is responsible for managing their own Lakehouse lifecycle. This includes:
- Schema changes
- Access policies
- Data quality
These responsibilities do not affect other teams.
Notebooks and Apache Spark in Fabric
Fabric Notebooks are crucial for data transformation. They run Apache Spark on Microsoft-managed compute pools. This eliminates the need for:
- Cluster provisioning
- Spark configuration
- Node management
When you open a notebook and run a cell, Fabric automatically allocates Spark resources from your capacity. These resources are released when the session ends. This serverless model helps reduce the operational overhead that often makes Spark difficult to manage in traditional environments.
Notebooks support PySpark, Spark SQL, Scala, and R. For most enterprise data engineering tasks, PySpark and Spark SQL handle 95% of use cases.
Fabric also allows for notebook parameterization. This feature lets you call notebooks from pipelines with runtime parameters. As a result, you can use the same transformation logic to process:
- Different datasets
- Various date ranges
- Multiple environments
This approach prevents code duplication.
Notebook Best Practices for Enterprise
- Use notebook resources to store shared utility functions and import them across notebooks — eliminates code duplication.
- Set session timeouts (default 20 minutes idle) to prevent capacity waste from forgotten sessions.
- Pin Spark libraries at the workspace level, not per-notebook, for consistent dependency management across teams.
- Use mssparkutils for credential management — never hardcode storage keys or connection strings in notebook cells.
- Structure notebooks with clear markdown sections: parameters, imports, transformations, validation, output — for readability and debugging.
- Enable V-Order on write operations (spark.sql.parquet.vorder.enabled = true) to optimize downstream DirectLake performance.
- Use the high-concurrency mode for development environments — shares a Spark session across multiple notebooks, reducing startup time.
Data Pipelines and Orchestration
Fabric Data Factory pipelines are essential for enterprise data engineering. They offer a visual, drag-and-drop interface for creating multi-step data workflows. These workflows can:
- Ingest data from source systems
- Execute Spark notebook transformations
- Run stored procedures
- Trigger downstream refreshes
For teams already using Azure Data Factory, Fabric pipelines are architecturally the same. They share the same activity types, expression language, and linked service model.
The main difference between standalone ADF and Fabric is the level of integration. Fabric pipelines can directly reference:
- Lakehouse tables
- Notebooks
- Semantic models
All of these are within the same workspace. This means there are no connection strings or linked services needed.
A pipeline can:
- Copy data from an external SQL Server into a Lakehouse table
- Trigger a Spark notebook to transform the data
- Refresh a Power BI semantic model
All of this happens in a single orchestrated workflow with complete lineage tracking.
90+ Connectors
Ingest from SQL Server, Oracle, Salesforce, SAP, REST APIs, file systems, and cloud storage — same connector library as Azure Data Factory.
Dependency Management
Chain activities with success/failure/completion dependencies. Build complex DAGs with parallel branches and conditional logic.
Scheduling & Triggers
Schedule pipelines on cron expressions, tumbling windows, or event triggers. Support for parameterized schedules across environments.
For enterprise-scale orchestration, EPC Group recommends using a hub-and-spoke pipeline pattern. This method includes a master pipeline that oversees child pipelines for each data domain. The data domains are:
- Sales
- Marketing
- Finance
- Operations
- Customer Support
- Sales
- Finance
- HR
Each child pipeline manages the full bronze-silver-gold transformation for its domain. The master pipeline oversees cross-domain dependencies and gives combined alerts on success or failure.
This setup scales easily as new data domains are added. It also simplifies debugging by isolating failures to specific domains.
OneLake Shortcuts: Federated Data Access
Shortcuts are among Fabric's most powerful and often underused features for data engineering. A shortcut acts as a reference pointer to data stored outside your Lakehouse. This data can be found in:
- Databases
- Data warehouses
- External files
- External databases
- Cloud storage
- Other data sources
- Another Lakehouse
- ADLS Gen2
- Amazon S3
- Google Cloud Storage
The data shows up as a native table or folder in your Lakehouse. You can query it using Spark and SQL.
Importantly, the data is never physically copied to OneLake. This means you incur no OneLake storage fees for shortcut data.
Shortcuts address three key challenges in enterprise data engineering:
- Data residency: Ensures data remains in its regulated location.
- Cost optimization: Eliminates the need for storage duplication.
- Incremental migration: Allows referencing legacy storage while building new pipelines.
Shortcuts make it easier to share data across workspaces and tenants. For instance, a finance team can:
- Create a shortcut to the sales team's gold-layer tables.
- Access the data without requesting a copy.
Shortcut Use Cases in Enterprise
- Multi-cloud federation: Reference S3 or GCS data from Fabric without migrating storage to Azure.
- Compliance boundaries: Keep HIPAA-regulated data in its approved ADLS Gen2 account while analyzing it in Fabric.
- Cross-domain data mesh: Each domain publishes gold-layer tables, other domains consume via shortcuts.
- Migration bridge: Shortcut legacy ADLS data into Fabric Lakehouse while incrementally rebuilding pipelines.
- Cost avoidance: Reference a 10 TB dataset via shortcut instead of duplicating it — saves ~$230/month in storage alone.
Delta Lake and Medallion Architecture
Every table in a Fabric Lakehouse is a Delta Lake table. Delta Lake provides important features such as:
- ACID transactions
- Schema enforcement
- Time travel (versioned history)
- Merge operations (upserts)
For data engineers, this means you will achieve warehouse-grade reliability on lake-scale storage. You will avoid issues such as:
- Corrupted partial writes
- Schema drift that disrupts downstream reports
- The inability to roll back a bad transformation
The medallion architecture is the best way to organize Delta tables in a Fabric Lakehouse. It offers a clear and traceable data lineage.
This lineage tracks data from raw ingestion to business-ready analytics.
- Each layer serves a specific purpose.
- Each layer has its own quality standard.
- Each layer follows a defined access pattern.
Bronze (Raw)
Ingest raw data exactly as received from source systems. Append-only, no transformations. Add ingestion metadata columns (source_system, ingestion_timestamp, batch_id). This layer serves as the immutable audit trail — you can always reprocess from bronze if silver/gold logic changes.
Silver (Conformed)
Apply cleansing, deduplication, type casting, null handling, and business key resolution. Enforce schema with Delta schema enforcement. Merge (upsert) patterns for slowly changing dimensions. Silver tables are the single source of truth for conformed enterprise data.
Gold (Business)
Produce aggregated, denormalized, business-ready tables optimized for Power BI DirectLake mode. Star schema design with fact and dimension tables. V-Order optimized for maximum query performance. Gold tables serve analysts, reports, and AI/ML feature engineering.
Performance Optimization
Fabric data engineering performance relies on three key factors:
- Table optimization: How you write data.
- Spark tuning: How you process data.
- DirectLake compatibility: How downstream consumers read data.
Getting all three right can mean the difference between a platform that provides sub-second dashboards and one that frustrates analysts with delays.
Performance Optimization Checklist
- Enable V-Order on all gold-layer tables — this Fabric-specific optimization sorts data for maximum DirectLake query performance.
- Run OPTIMIZE regularly on Delta tables to compact small files — target 128 MB per file for optimal Spark read performance.
- Apply Z-ORDER on columns frequently used in WHERE clauses — enables data skipping and reduces scan volume by 80%+.
- Use table partitioning by date for large fact tables (>100 GB) — but avoid over-partitioning on high-cardinality columns.
- Set spark.sql.shuffle.partitions based on data volume — default 200 is too high for small tables, too low for billion-row jobs.
- Use Delta table VACUUM to remove old file versions — reduces storage cost and improves file listing performance.
- Broadcast small dimension tables in joins (< 100 MB) to avoid expensive shuffle operations.
- Cache intermediate DataFrames that are used multiple times in a notebook — avoids recomputation.
Data Engineering Governance
Governance in Fabric data engineering is crucial and integrated at all levels of the platform. OneLake provides automatic lineage tracking, which includes:
- Source ingestion
- Bronze transformations
- Silver transformations
- Gold transformations
- Power BI reports
Additionally, Microsoft Purview works seamlessly with Fabric to:
- Classify sensitive data
- Apply sensitivity labels
- Enforce access policies across the entire data estate
For enterprise data engineering teams, governance is important in four main areas:
- Access Control: Who can read or write to which Lakehouse tables.
- Data Classification: What sensitivity level does each column have.
- Lineage: Where did this data come from and what transformations were applied.
- Quality: Are the data values accurate, complete, and timely.
Fabric addresses the first three areas natively. However, data quality requires engineering discipline through validation notebooks and monitoring.
Access Control
Workspace roles (Admin, Member, Contributor, Viewer) control Lakehouse access. Row-level security and object-level security for fine-grained table protection.
Data Classification
Purview automatically scans Lakehouse tables for PII, financial data, and health information. Sensitivity labels propagate from source to downstream artifacts.
Lineage Tracking
Automatic lineage from pipeline ingestion through notebook transformations to Power BI reports. No manual documentation required — Fabric tracks every dependency.
Quality Monitoring
Build validation notebooks that check row counts, null rates, schema conformance, and business rule compliance after each pipeline run.
Cost Management for Fabric Data Engineering
Fabric has a pricing model based on capacity. You buy Capacity Units (CUs) that are shared across all Fabric workloads within a capacity. Data engineering uses CUs in the following ways:
- For data processing tasks
- To support analytics workloads
- For data integration activities
- Data processing tasks
- Data storage needs
- Data transformation activities
- When Spark notebooks execute
- When pipelines run copy activities
- When Dataflow Gen2 transformations process data
OneLake storage is billed separately at about $0.023 per GB per month, which is the same as the ADLS Gen2 hot tier.
One common cost mistake in Fabric data engineering is over-provisioning capacity for development environments. A single F64 capacity costs $4,096 per month when reserved.
This capacity is enough for a 10-person data engineering team. It can effectively handle development and testing workloads.
For production workloads that require heavy Spark processing, consider using F128 or F256. However, EPC Group recommends starting with F64.
Scale your capacity based on actual usage metrics rather than estimated workload projections.
Cost Optimization Quick Wins
- Reserved capacity saves 40% vs PAYG — commit to F64+ for 1 year on any production workload.
- Spark session timeouts: Set to 5-10 minutes for dev, 2 minutes for production to reclaim idle capacity.
- Schedule heavy batch jobs during off-peak hours (nights/weekends) when interactive usage is low.
- Use shortcuts for reference data — a 5 TB shortcut costs $0 in OneLake storage vs $115/month for a copy.
- VACUUM Delta tables weekly — removes old versions, reducing storage by 20-50% on active tables.
- Monitor capacity utilization in the Fabric admin portal — sustained >80% means you need to scale up or optimize.
Frequently Asked Questions: Fabric Data Engineering
What is data engineering in Microsoft Fabric?
Data engineering in Microsoft Fabric is a unified discipline that combines lakehouse architecture, Apache Spark notebooks, data pipelines, and Delta Lake storage into a single SaaS platform. Fabric data engineers use OneLake as the centralized storage layer, write transformations in PySpark or Spark SQL notebooks, orchestrate workflows with Data Factory pipelines, and leverage shortcuts to connect external data sources — all without managing infrastructure. The result is a modern data engineering experience that eliminates the complexity of stitching together separate Azure services like Synapse, Data Factory, and ADLS Gen2.
What is the Fabric Lakehouse and how does it work?
The Fabric Lakehouse is a combined data lake and data warehouse that stores data in open Delta Lake (Parquet) format on OneLake. It supports both SQL analytics and Spark-based data engineering on the same data without duplication. You create tables that are automatically registered in the SQL analytics endpoint for T-SQL queries and simultaneously accessible via Spark notebooks for transformations. The Lakehouse eliminates the traditional choice between a data lake (flexible but ungoverned) and a data warehouse (structured but rigid) — delivering both capabilities on a single copy of data.
How do Fabric Notebooks compare to Databricks Notebooks?
Fabric Notebooks run Apache Spark on Microsoft-managed compute — no cluster provisioning or configuration required. They support PySpark, Spark SQL, Scala, and R with built-in visualization and collaboration features. Databricks Notebooks offer more advanced features like MLflow integration, Databricks Connect for local IDE development, and more granular cluster control. For standard data engineering workloads (ETL, data cleansing, aggregation), Fabric Notebooks are equally capable with significantly lower operational overhead. For advanced ML engineering and custom Spark tuning, Databricks retains an edge.
What are Fabric Shortcuts and when should I use them?
Fabric Shortcuts are pointers to external data sources that make data appear as if it lives in your Lakehouse without physically copying it. Shortcuts support ADLS Gen2, Amazon S3, Google Cloud Storage, and Dataverse. Use shortcuts when: (1) data must remain in its source system for compliance, (2) you want to avoid storage duplication costs, (3) you need to federate data across organizational boundaries, or (4) you are migrating incrementally and want to reference legacy storage during transition. Shortcuts are read-only by default and respect the security policies of the source system.
What is the medallion architecture in Microsoft Fabric?
The medallion architecture (bronze-silver-gold) is the recommended data organization pattern in Fabric Lakehouse. Bronze layer ingests raw data from source systems with minimal transformation — preserving the original format for auditability. Silver layer applies cleansing, deduplication, schema enforcement, and business logic to create conformed datasets. Gold layer produces aggregated, business-ready tables optimized for reporting and analytics. In Fabric, each layer is a set of Delta tables in the Lakehouse, with Spark notebooks or Data Factory pipelines orchestrating the transformations between layers.
How does Fabric handle data pipeline orchestration?
Fabric uses Data Factory pipelines for orchestration — a visual, code-free interface for scheduling and sequencing data movement and transformation activities. Pipelines support 90+ connectors for ingesting data from cloud and on-premises sources. You can chain Spark notebook executions, stored procedures, Dataflow Gen2 transformations, and copy activities into multi-step workflows with dependency management, retry logic, and alerting. Pipelines also support parameterization, allowing the same pipeline to process different datasets or environments dynamically.
What are the cost management strategies for Fabric data engineering?
Key cost strategies include: (1) Use reserved capacity (F64+) for predictable workloads — saves 40% vs pay-as-you-go. (2) Implement workspace-level capacity assignment to isolate cost by team or project. (3) Use Spark session timeouts to prevent idle compute consumption. (4) Leverage V-Order optimization on Delta tables to reduce query compute. (5) Schedule heavy pipelines during off-peak hours to smooth capacity utilization. (6) Use shortcuts instead of data copies to eliminate redundant storage. (7) Monitor capacity metrics in the Fabric admin portal and set alerts for sustained high utilization.
How does governance work for Fabric data engineering?
Fabric governance is built on Microsoft Purview integration and OneLake security. Data engineers benefit from: automatic data lineage tracking across notebooks and pipelines, sensitivity labels that propagate from source to downstream tables, workspace-level access control with Entra ID, row-level and object-level security on Lakehouse tables, and endorsement workflows (certified/promoted) for dataset quality signaling. All data in OneLake is encrypted at rest and in transit. Purview Data Catalog automatically discovers and classifies Lakehouse tables, enabling data stewards to manage the entire engineering lifecycle from a single governance plane.
Can I migrate my existing Azure Synapse or ADF workloads to Fabric?
Yes. Microsoft provides migration paths from Azure Synapse Analytics and Azure Data Factory to Fabric. Synapse Spark pools map directly to Fabric Spark notebooks with minimal code changes. ADF pipelines can be migrated to Fabric Data Factory with the pipeline migration wizard — most activities transfer directly. Synapse SQL dedicated pools require more effort, as Fabric uses a different SQL engine. EPC Group recommends a phased migration: start with new workloads on Fabric, migrate existing ADF pipelines next, then gradually transition Synapse workloads as Fabric capabilities mature. We have completed 50+ Fabric migrations for enterprise clients.
Related Resources
Microsoft Fabric Consulting Services
Enterprise Fabric implementation, migration, and optimization services from EPC Group.
Read moreMicrosoft Fabric Enterprise Guide
Comprehensive overview of Microsoft Fabric capabilities, licensing, and adoption strategy.
Read moreFabric vs Databricks Comparison
Head-to-head comparison of Microsoft Fabric and Databricks for enterprise data platforms.
Read moreNeed Help Building Your Fabric Data Engineering Platform?
EPC Group has successfully completed over 50 Microsoft Fabric implementations for enterprise clients. Our certified Fabric engineers specialize in:
- Lakehouse architecture design
- Production pipeline deployment
- Delivering scalable data platforms
Schedule a free data engineering assessment today.
Microsoft Fabric Data Engineering: Enterprise Guide 2026
Microsoft Fabric Data Engineering is the Spark-based workload within Microsoft Fabric. It helps build enterprise data pipelines, lakehouses, and Delta Lake tables.
This guide includes:
- Lakehouse architecture
- Notebooks
- Medallion architecture
- Delta Lake
- Data governance best practices
- Cost optimization strategies for enterprise teams in 2026
Key facts
- Fabric Data Engineering uses Apache Spark for data transformation, with Notebooks and Spark Job Definitions as the primary authoring surfaces.
- Data written by Spark is instantly available to Fabric Warehouse (T-SQL), Real-Time Intelligence (KQL), and Power BI (Direct Lake) — without copying.
- Delta Lake is the default table format in Fabric Lakehouses. It supports ACID transactions, time travel, and schema evolution.
- Reserved F64+ capacity saves approximately 40% vs. pay-as-you-go pricing for predictable Spark workloads.
- EPC Group: 500+ Microsoft Fabric projects, 29 years of Microsoft data platform experience.
Fabric Lakehouse architecture
The Fabric Lakehouse is the primary data storage and processing workspace for enterprise data engineering. It stores data in Delta Lake format on OneLake.
- All data lands in OneLake — the unified data lake accessible to every Fabric workload.
- Delta Lake tables provide ACID transactions, time travel (up to 30 days default), and schema evolution.
- Tables are accessible from Spark (Notebooks), SQL (Lakehouse SQL Endpoint), and Power BI (Direct Lake).
- Managed tables store data inside the Lakehouse. External tables reference data in ADLS Gen2, S3, or Google Cloud Storage via shortcuts.
Medallion architecture in Fabric
Medallion architecture organizes data into three layers. Each layer adds quality and semantic structure to the raw source data.
- Bronze (raw) — ingest source data as-is. No transformations. Preserve the original for reprocessing if needed. Store in Delta Lake for audit capability.
- Silver (cleaned) — deduplicate, standardize data types, enforce schema, apply business rules. This is the governed data layer that analysts can trust.
- Gold (business-ready) — aggregate, join, and model for specific business domains. Power BI semantic models typically read from Gold tables via Direct Lake.
When to use Fabric shortcuts
Shortcuts reference external data from inside a Lakehouse without physically moving it into OneLake. Use shortcuts when:
- Data must remain in its source system for compliance or contractual reasons.
- You want to avoid storage duplication costs for large datasets already in ADLS Gen2.
- You need to federate data across organizational boundaries (partner data sharing).
- You are migrating incrementally and want to reference legacy storage during transition before full cutover.
Data engineering governance in Fabric
Governance for data engineering teams covers four areas. Build all four into the architecture — not as afterthoughts.
- Access control — who can read and write which Lakehouse tables. Use Fabric workspace roles (Admin, Member, Contributor, Viewer) plus OneLake ACLs for table-level permissions.
- Data classification — what sensitivity level does each column contain? Apply Purview sensitivity labels to Lakehouse tables to classify PHI, PII, and financial data.
- Lineage — where did this data come from and what transformations were applied? Fabric's built-in lineage view in the admin portal tracks this automatically for Notebooks and pipelines.
- Quality — are data values accurate, complete, and timely? Implement Delta Lake expectations, data quality tests in Notebooks, and Data Activator alerts for freshness and accuracy thresholds.
Fabric cost optimization strategies
Seven strategies to reduce Fabric compute costs for enterprise data engineering workloads.
- Use reserved capacity (F64+) for predictable workloads — saves approximately 40% vs. pay-as-you-go.
- Implement workspace-level capacity assignment to isolate and track cost by team or project.
- Use Spark session timeouts to prevent idle compute consumption between Notebook runs.
- Apply V-Order optimization on Delta tables to reduce query compute for frequently read tables.
- Schedule heavy pipeline runs during off-peak hours to smooth capacity utilization.
- Use shortcuts instead of data copies to eliminate redundant storage costs.
- Monitor capacity metrics in the Fabric admin portal and set alerts for sustained high utilization before you hit throttling.
Frequently asked questions
What is Microsoft Fabric Data Engineering?
Fabric Data Engineering is the Spark-based workload in Microsoft Fabric. It helps users build data pipelines, lakehouses, and Delta Lake tables. Users can create and manage projects using Notebooks and Spark Job Definitions.
Data written to the Lakehouse is immediately available to all other Fabric workloads — Warehouse (T-SQL), Real-Time Intelligence (KQL), and Power BI (Direct Lake) — without copying.
What is the medallion architecture in Microsoft Fabric?
Medallion architecture structures a Fabric Lakehouse into three Delta Lake table layers:
- Bronze: raw source data with no transformations.
- Silver: cleaned, deduplicated, and schema-enforced data.
- Gold: aggregated, business-ready data for analytics.
Power BI semantic models usually access Gold tables through Direct Lake mode for optimal performance.
What is a Fabric Lakehouse vs. a Fabric Warehouse?
Both options store data in OneLake and are accessible to Power BI. However, there is a key difference:
- Lakehouse: It is Spark-native and uses Delta Lake format.
- This format is optimized for data engineering, machine learning, and mixed workloads.
Warehouse is built on T-SQL. It is designed for SQL-first analytics teams and uses familiar database query patterns.
Data in Warehouse and Lakehouse can be accessed in two ways:
- Through the Lakehouse SQL endpoint
- By using Warehouse cross-database queries
How do I control costs in Microsoft Fabric?
Here are seven strategies for optimizing costs:
- Use reserved F64+ capacity for a 40% discount.
- Isolate costs by workspace.
- Set Spark session timeouts.
- Apply V-Order on Delta tables.
- Schedule heavy pipelines during off-peak hours.
- Use shortcuts instead of making data copies.
- Monitor capacity in the Fabric admin portal with utilization alerts.
EPC Group conducts Fabric cost optimization reviews as part of its managed services engagements.
Does Fabric Data Engineering support real-time data?
Fabric Real-Time Intelligence is a unique yet integrated workload. It uses KQL databases and Fabric event streams. This combination enables fast ingestion and querying of streaming data.
Data in KQL databases is accessible from Fabric Notebooks via a connector, and can be materialized into Delta Lake tables for batch analytics use cases.
Architect your Fabric data engineering platform
EPC Group designs and implements enterprise Fabric data engineering platforms — from OneLake architecture through Medallion build and governance. Call (888) 381-9725 or schedule a discovery call.