All posts are sorted by newest first. The structure is ready for future pagination and filtering by topic.
Most lakehouse pipelines are still doing one expensive thing repeatedly: reprocessing entire tables, even when only a tiny fraction of rows has changed. This post explains the CDF-driven load pattern, why it works, where it can hurt, and how to implement it safely in Microsoft Fabric.
/implementing-change-data-feed-for-incremental-processing-in-microsoft-fabric-lak
Microsoft FabricSpark
Read articleIntroduction When building a modern lakehouse on Microsoft Fabric, one of the most powerful — and often underexplored — features is Mirrored Databases. Instead of...
/how-microsoft-fabric-mirroring-works-for-azure-sql-database-a-technical-deep-div
Microsoft FabricAzure
Read articleA practical checklist for designing APIs that are easy to use and evolve safely.
/build-better-apis
Data Engineering
Read articleHow small pull requests improve review quality and reduce release risk.
/ship-features-faster
Data Engineering
Read articleData quality isn't just a checkbox — it's the backbone of reliable analytics. Learn how to set up automated quality checks and profiling with Microsoft Purview and Fabric Lakehouse.
/data-quality-made-simple-a-quick-guide-to-microsoft-fabric-purview-setup
Microsoft Fabric
Read articleHow to keep your Spark jobs portable, safe for concurrency, and easier to orchestrate by using explicit ABFSS paths instead of attaching a default Lakehouse in Microsoft Fabric.
/run-spark-sql-in-fabric-notebooks-without-attaching-a-default-lakehouse
Microsoft FabricSpark
Read articleSpeed up your Fabric pipelines by running notebooks in parallel — without breaking your Spark cluster. Learn how High Concurrency Mode works and how to configure session tags.
/how-to-use-concurrent-sessions-in-fabric-data-pipelines
SparkData Engineering
Read articleWhen defining identity columns in Databricks tables, two common options for automatic identity value generation are GENERATED BY DEFAULT and GENERATED ALWAYS. This article explains the differences and when to use each.
/generated-by-default-vs-generated-always-in-databricks
Data Engineering
Read articleMicrosoft Dynamics suite is a widely used product for implementing ERP and CRM solutions. This article shows how to upgrade Synapse Spark pool version to 3.4 in Synapse Link for Dataverse.
/synapse-link-for-dataverse-support-for-synapse-spark-version-3-4
Microsoft FabricAzureSpark
Read articleMicrosoft Dynamics is a widely used product for implementing ERP and CRM solutions. In this article, we will compare export to datalake with synapse link for data...
/export-to-datalake-vs-synapse-link-enhancing-data-access-in-d365-finance-and-ope
Microsoft FabricAzure
Read articleAzure Data Factory can securely access credentials stored in Azure Key Vault. This article describes how to set up key vault secrets for Azure Data Factory linked services.
/use-key-vault-secrets-in-azure-data-factory
AzureData Engineering
Read articleDatabricks can connect to various sources for data ingestion. This article describes how to manage Azure Key Vault-backed secret scopes in Databricks using the GUI.
/manage-secret-scopes-in-databricks
AzureSpark
Read article