Microsoft Fabric Data Engineering in 2026: Architecture Decisions That Actually Matter

OneLake, Spark vs T-SQL, Direct Lake mode, Eventstream, pipeline orchestration. The architecture decisions data engineering teams actually have to make once their organization picks Fabric, from inside Thinklytics engagements.

What Fabric data engineering actually involves

Once a company picks Microsoft Fabric, the procurement decision is done. The work that follows is architectural: which storage pattern, which compute engine per workload, which Power BI mode, which orchestration shape. Most teams arrive without enough opinion-formed answers and ship a tenant that runs but cannot scale past the first three or four workloads without painful rework.

This is what we have learned from running Fabric data engineering inside 24+ Power BI Premium and Fabric engagements through 2025 and 2026. Concrete decisions, the trade-offs, and what we actually recommend.

Fabric workload patterns across 24+ engagements

Production tenants rarely commit to a single engine. Most run multiple workload types in parallel, sized to their existing skill mix and data shape.

• Tenants running at least one Lakehouse (Spark + Delta)
• Tenants running at least one Warehouse (T-SQL native)
• Tenants running both engines in production
• Tenants running Eventstream + KQL for real-time use cases

Source: Thinklytics Microsoft practice synthesis, 2024-2026

OneLake: the storage decision underneath everything

OneLake is Fabric's tenant-wide storage layer. One namespace, one set of access controls, one shortcut feature that lets a single OneLake path mirror data from S3, ADLS, Dataverse, or another OneLake region without copying. For new builds, OneLake replaces the per-workspace ADLS Gen2 containers, the dedicated SQL pool storage that came with Synapse, and the cache layer Power BI Premium used to manage separately.

The decisions teams have to make on day one:

• Workspace partitioning strategy. The default of "one workspace per team" creates an access-control mess at 8+ workspaces. We recommend one workspace per logical domain (Sales, Finance, Operations, Engineering, etc.), with workspace-level permissions held by the domain owner.
• Shortcut vs copy for inbound data. Shortcuts work well when the source is read-mostly and your egress costs are low. Copies (via Fabric Mirroring or Data Factory) win when the source is high-write or when you need a stable point-in-time view independent of source mutations.
• Medallion architecture (bronze, silver, gold) vs flat layout. Medallion is the default for any tenant with more than 2 workloads. Flat layouts work for prototypes and pay technical debt at the first production incident.

Lakehouse vs Warehouse: pick one per workload, not per company

Fabric ships two analytical engines that look similar from the outside: Lakehouse (Spark + Delta files, T-SQL read-only endpoint) and Warehouse (T-SQL native, ACID transactions, traditional indexes). The most common architectural mistake is choosing one for the whole tenant. The right pattern is per-workload.

Lakehouse wins for:

• Workloads with significant unstructured or semi-structured data
• Workloads where the team already runs PySpark code from Synapse or Databricks
• ML feature engineering where Spark's library surface is the value
• Bronze and silver layers in a medallion architecture

Warehouse wins for:

• Workloads where T-SQL is the team's strongest skill
• Workloads with concurrent ad-hoc SQL users (Warehouse handles concurrency more cleanly)
• Gold layers that get consumed by Power BI semantic models and tabular cubes
• Workloads with strict ACID transactional requirements

Lakehouse vs Warehouse: per-workload decision

• Lakehouse. Spark + Delta. Unstructured prep, ML feature engineering, bronze and silver layers. Wins when PySpark code already exists or scale-out parsing is required.
• Warehouse. T-SQL native. ACID transactional reads, concurrent ad-hoc SQL, gold layers consumed by Power BI semantic models. Wins on SQL ergonomics and concurrency.

Most production tenants run 3 to 5 Lakehouses and 1 to 2 Warehouses. The Warehouse usually serves the certified gold layer that Power BI Direct Lake reads from.

Source: Thinklytics Microsoft practice, 2026

We see most production Fabric tenants running 3 to 5 Lakehouses and 1 to 2 Warehouses, with the Warehouse usually serving the certified gold layer that Power BI Direct Lake reads from.

Spark vs T-SQL: when each wins

Inside the Lakehouse, the compute choice is per notebook, not per workspace. Two practical rules:

• If the transformation is row-by-row, joins-heavy, and would be expressible as a SQL query in under 100 lines, write it as a SQL endpoint query against the Lakehouse. T-SQL on the Lakehouse SQL endpoint is reasonably fast for analytical reads and avoids spinning up a Spark session.
• If the transformation involves unstructured input parsing, ML feature derivation, or scale-out across multi-billion-row inputs, write it as a PySpark notebook. The startup cost of a Spark session amortizes well when the workload is large.

Most teams arrive convinced that one engine should win everything. After 6 to 9 months they settle into the per-workload pattern naturally.

Direct Lake mode: the Power BI shift most teams miss

Direct Lake is the 2026 default Power BI mode for Fabric-native deployments. It reads Delta files in OneLake directly without an import step, gets within 10 to 15 percent of Import performance on most analytical queries, and side-steps the dataset refresh complexity that broke Power BI Premium deployments at scale.

The shift teams miss: Direct Lake changes the semantic model design. With Import, the model designer optimizes for compression and refresh-friendliness. With Direct Lake, the model designer optimizes for Delta file partitioning and Lakehouse query pruning. The DAX and the relationships are the same, but the optimization headers point at different problems.

Power BI mode selection in Fabric, 2026

Direct Lake is the new default above 10 GB. Import survives at the small end. DirectQuery only when a true live source exists and the source can sustain query rate.

Source: Thinklytics Power BI Premium and Fabric practice, 2024-2026

We recommend Direct Lake for any new semantic model above 10 GB. Below that, Import is still simpler operationally. DirectQuery survives only for live-source requirements your source system can actually sustain.

Pipeline orchestration patterns that hold up

Fabric Data Factory replaced Azure Data Factory inside the tenant. The pipeline shape is similar but the orchestration patterns we have seen survive production:

• Use Fabric pipelines for the orchestration layer (triggers, dependencies, retries). Use Spark notebooks for the actual transformation work. Pipelines that try to do transformations inline using Copy activities scale poorly past 50 datasets.
• Use Dataflows Gen2 sparingly. They are easy to author and harder to maintain at scale. We see them work well for departmental data prep where the author owns the artifact end-to-end. They break down when the artifact graduates to a shared production workload.
• Implement a "platform repo" for shared utilities (logging, observability, retry helpers) that Lakehouse notebooks import. Without this, every notebook reinvents error handling and the platform has no consistent debugging story.

Real-time intelligence with Eventstream and KQL

Fabric's real-time intelligence layer (Eventstream for ingestion, KQL Database for query, Activator for action) is the part of Fabric most teams underestimate. KQL is excellent at time-series queries and pattern detection across high-volume events. It is bad at the kinds of relational analytical workloads Warehouse handles.

When Fabric real-time intelligence (Eventstream + KQL) pays back

Sub-minute latency is the bar. Above that, Lakehouse with frequent micro-batches is simpler operationally.

• Operational metrics streamed from production systems. Latency, error rates, throughput surfaced sub-minute
• Telemetry pipelines for customer-facing apps. AI models reacting to user behavior in near-real-time
• Anomaly detection on infrastructure or fraud. 24-hour batch latency is too slow for the use case
• Hourly or daily analytical reporting. Lakehouse + micro-batches is cheaper and simpler
• Monthly close-cycle reconciliation. Warehouse with scheduled refreshes is the right engine

Source: Thinklytics Microsoft practice, 2026

Common use cases we ship on the real-time stack:

• Operational metrics streamed from production systems (latency, error rates, throughput) into dashboards that update sub-minute
• Telemetry pipelines for customer-facing apps where AI models react to user behavior in near-real-time
• Anomaly detection on infrastructure or fraud signals where the 24-hour batch latency of a Warehouse is too slow

When the use case is "I need real-time data" but the latency requirement is actually 1 to 4 hours, we steer teams toward Lakehouse with frequent micro-batches. The operational complexity of KQL only pays back when latency requirements are sub-minute.

The Fabric data engineering team shape

A Fabric tenant running 4 to 8 production workloads needs roughly:

• 1 platform owner: capacity, governance, security, F-sku decisions, OneLake namespace strategy
• 2 data engineers: pipelines, Lakehouse ingestion, T-SQL or Spark transformation work
• 1 analytics engineer: semantic models, Direct Lake mode optimization, DAX patterns
• 1 part-time architect across all four roles: 0.25 FTE worth of design time

Fabric data engineering team shape by workload count

Headcount scales sub-linearly with workload count once the platform owner role is in place. The skill mix matters more than the headcount.

• 1-3 workloads: 2-3 people total (platform + 1-2 engineers)
• 4-8 workloads: 4-5 people (platform + 2 DEs + 1 AE)
• 9-12 workloads: 5-7 people (add SRE-style on-call)
• 12+ workloads: split platform from workload ownership

Source: Thinklytics Microsoft practice synthesis, 2026

Below 4 workloads, collapse to 2 to 3 people. Above 12 workloads, split platform ownership from workload ownership and add SRE-style on-call for capacity-throttling events.

The skill mix matters more than the headcount. Fabric demands engineers who are comfortable in both SQL and Python, who understand columnar storage trade-offs, and who can read a Power BI semantic model without panic. Teams that come from a pure SQL background or a pure Spark background usually need 4 to 8 weeks of cross-training before they ship at speed.

How Thinklytics works on Fabric data engineering

Senior Microsoft-practice partners who have shipped Fabric and Power BI Premium at Fortune 500 telecoms, regional health systems, and B2B SaaS companies. We start with a 4-week current-state assessment that covers capacity sizing, workload inventory, OneLake vs ADLS posture, and Power BI mode audit, then a phased build sized to the workload count.

Most net-new Fabric tenants land at $260,000 to $540,000 for the foundation plus the first two production workloads. Synapse-to-Fabric migrations scale higher because the existing inventory drives the work. Read the Microsoft Fabric consulting article for the procurement-side companion to this engineering-side piece.

• $260K-$540K Typical Fabric data engineering engagement. Foundation plus first two workloads in production for a net-new Fabric tenant. Synapse-to-Fabric migrations scale higher because existing inventory drives the work. 4-week current-state assessment ships first.

Frequently asked questions

What does OneLake actually replace?

OneLake is Fabric's unified storage layer, built on ADLS Gen2 under the hood and addressable via a single tenant-wide namespace. For new builds it replaces the per-workspace ADLS containers, the Synapse dedicated SQL pool storage, and the Power BI Premium dataset cache that teams used to manage separately. The shortcut feature lets a single OneLake path mirror data from S3, ADLS, or Dataverse without copying, which is the main 2026 reason teams pick it over warehouse-native storage.

When should we use Spark vs T-SQL inside Fabric?

Spark wins for unstructured data prep, ML feature engineering, and any code your team already wrote in PySpark for Synapse or Databricks. T-SQL wins for analytical workloads where SQL ergonomics, query optimizer maturity, and dashboard latency dominate. The decision is per workload, not per company. Most production Fabric tenants run both in parallel: Spark notebooks for the messy ingestion and feature layer, T-SQL warehouses for the cleaned analytical layer.

Direct Lake mode vs Import vs DirectQuery, which Power BI mode is right?

Direct Lake is the 2026 default for Fabric-native deployments above 10 GB. It reads Delta files in OneLake without an import step, gets near-Import performance, and side-steps cache refresh complexity. Import still wins for small datasets under 1 GB and for workloads that hit data sources outside OneLake. DirectQuery survives only for actual live-source requirements that your source system can sustain at query rate.

How do we migrate Synapse Spark notebooks to Fabric?

Three steps. Move the notebook code as-is, repoint storage paths from ADLS to OneLake, and rebuild the pipeline orchestration in Fabric Data Factory. Plan 4 to 8 weeks per Synapse workspace including testing. The trap is environment parity: secrets, linked services, and managed identities all have different shapes in Fabric, and notebooks that worked under Synapse-managed identity often fail under Fabric Trusted Workspace Access until that's reconfigured.

What is the right team structure for Fabric data engineering?

For a tenant running 4 to 8 workloads in production: 1 platform owner (capacity, governance, sec), 2 data engineers (pipelines + Lakehouse), 1 analytics engineer (semantic models + Direct Lake), and 1 part-time architect across all four. Below 4 workloads, collapse to 2 to 3 people. Above 12 workloads, split platform ownership from workload ownership and add a SRE-style on-call for capacity events.

Can we use dbt with Fabric?

Yes, via dbt-fabric (the Microsoft-maintained adapter) targeting Fabric Warehouse or Lakehouse SQL endpoints. The pattern works well for cleaned analytical layers where T-SQL is the right engine. dbt does not target Spark Notebooks in Fabric: that layer stays in native PySpark or Fabric notebook code. Most teams that ship dbt with Fabric run dbt for the gold layer and Spark notebooks for bronze and silver.

How does Apache Iceberg fit into Fabric's storage model?

OneLake is Delta-Parquet native, not Iceberg-native. Microsoft has shipped read support for external Iceberg tables via the Fabric mirroring feature in 2026, but write workloads still target Delta. Teams that need true Iceberg portability across Snowflake, Databricks, and Fabric typically standardize on external Iceberg tables and use OneLake as a query surface, accepting the small write overhead.

How does Thinklytics scope a Fabric data engineering engagement?

We start with a 4-week current-state assessment (capacity, workload inventory, OneLake vs ADLS posture, Power BI mode audit), then a phased build sized to the workload count. Most net-new Fabric tenants land at $260,000 to $540,000 for the foundation plus the first two workloads in production. Synapse-to-Fabric migrations scale higher because the existing inventory drives the work. Read the Microsoft Fabric consulting article for the procurement-side companion to this.