Synapse Link for Dataverse: Support for Synapse Spark Version 3.4
Microsoft Dynamics suite is a widely used product for implementing ERP and CRM solutions. This article shows how to upgrade Synapse Spark pool version to 3.4 in Synapse Link for Dataverse.
Microsoft Dynamics suite is a widely used product for implementing ERP and CRM solutions. Synapse Link for Dataverse is a functionality to extract data from Microsoft dynamics systems for analytics and integration purposes.
In this article, We will look into how to upgrade Synapse spark pool version to 3.4 in Synapse link for dataverse.
#What is Synapse Link for Dataverse?
Synapse Link is a feature within Dataverse designed to efficiently manage and export data from microsoft Dynamics systems to external systems such as Azure Data Lake (CSV or delta) or Microsoft Fabric.
#Export data using synapse link in Delta format
When setting up a connection to extract data from Dynamics 365 to Azure Data Lake in Delta format, it's essential to configure a Synapse Spark pool during the Synapse Link setup. This step ensures that the data is processed efficiently and stored in the optimal Delta format within your Azure Data Lake.
Synapse Spark pool is used to convert data from CSV to Delta format in Data lake.
With support for Synapse Spark Pool version 3.3 coming to an end on 31st March 2025, it's crucial to upgrade your existing Synapse Link to Spark Pool version 3.4. This upgrade will ensure continued support, enhanced performance, and access to the latest features, keeping your data integration processes running smoothly. By making the switch ahead of time, you'll avoid potential disruptions and ensure your system remains optimized and future-ready
#How to upgrade synapse spark version to 3.4?
- First step is to create a new synapse spark pool with version 3.4 in synapse workspace
- Second step is to upgrade the existing synapse link in dataverse environment.
Pro tips:
- Here is a link to microsoft documentation.
- The Spark pool upgrade occurs only when a new Delta Lake conversion Spark job is triggered.
- You must create a new Synapse Spark pool with Spark version 3.4, using the same or higher nodes hardware configuration within the same Synapse workspace