6 articles tagged Spark.
VACUUM keeps Delta tables healthy, but it can also break Change Data Feed replay windows. This walkthrough shows exactly when incremental pipelines fail and how to design a safe Fabric pattern with guardrails and fallback recovery.
Most lakehouse pipelines are still doing one expensive thing repeatedly: reprocessing entire tables, even when only a tiny fraction of rows has changed. This post explains the CDF-driven load pattern, why it works, where it can hurt, and how to implement it safely in Microsoft Fabric.
How to keep your Spark jobs portable, safe for concurrency, and easier to orchestrate by using explicit ABFSS paths instead of attaching a default Lakehouse in Microsoft Fabric.
Speed up your Fabric pipelines by running notebooks in parallel — without breaking your Spark cluster. Learn how High Concurrency Mode works and how to configure session tags.
Microsoft Dynamics suite is a widely used product for implementing ERP and CRM solutions. This article shows how to upgrade Synapse Spark pool version to 3.4 in Synapse Link for Dataverse.
Databricks can connect to various sources for data ingestion. This article describes how to manage Azure Key Vault-backed secret scopes in Databricks using the GUI.