Practical articles for engineers working with Microsoft Fabric, Azure, Spark, and modern data and DevOps platforms.
A Data Pipeline references a connection, not a string — and that connection has a different ID in every workspace. Here's how the Connection Reference variable type swaps it per environment.
Deterministic hash surrogate keys or IDENTITY columns for your dimensions? A practitioner's pros and cons across load order, idempotency, join cost, and reprocessing.
Variable Libraries plus fabric-cicd's environment parameter give you a versioned, typed way to swap Key Vault names, storage accounts, and other per-environment constants automatically.
Copy pipeline via Virtual Network Data Gateway failing with LakehouseOperationFailed / InternalServerError at the _delta_log commit step? The fix is a single Azure subnet configuration change.
Learn how to enable and leverage Change Data Feed (CDF) with Fabric Mirroring to build true incremental data pipelines and unlock near real-time data processing capabilities.
VACUUM keeps Delta tables healthy, but it can also break Change Data Feed replay windows. This walkthrough shows exactly when incremental pipelines fail and how to design a safe Fabric pattern with guardrails and fallback recovery.
Most lakehouse pipelines are still doing one expensive thing repeatedly: reprocessing entire tables, even when only a tiny fraction of rows has changed. This post explains the CDF-driven load pattern, why it works, where it can hurt, and how to implement it safely in Microsoft Fabric.
Introduction When building a modern lakehouse on Microsoft Fabric, one of the most powerful — and often underexplored — features is Mirrored Databases. Instead of...