When you’re working with Dynamics 365 Sales (CRM), one of the biggest challenges you’ll face as a database administrator is keeping your integrations efficient. The good news is that Dynamics 365 provides Change Tracking, and our SSIS Integration Toolkit for Microsoft Dynamics 365 exposes this capability through a purpose-built EntityChanges Source Type that makes incremental sync straightforward in SSIS.
Let’s see how Change Tracking works, how our EntityChanges Source Type leverages it, and how you can put everything together into a resilient, token-based incremental strategy. We’ll show you the steps in the process and share practical tips and patterns used in the field.
Why Incremental Sync Matters
Every project that moves data between Dynamics 365 and other systems benefits from only transferring what changed since the last run. If you reload all records every time, even when only a handful have changed, you waste resources:
- Performance drops as queries and data flows become unnecessarily large.
- ETL windows stretch, delaying downstream reports and processes.
- API call limits are consumed faster, increasing the risk of throttling.
For example, imagine syncing an entity with 2 million records where only 200 have changed in the last hour. Pulling all 2 million each time is a massive drain. Incremental sync fetches just the 200, saving bandwidth, time, and processing power.
What is Change Tracking in Dynamics 365?
Change Tracking is a server-side feature in Dynamics 365 that lets you detect inserts, updates, and deletes on entities without scanning entire tables. The platform maintains internal versioning and returns only the deltas since a given point in time.
Change Tracking offers several important advantages, helping you work with smaller result sets and maintain system consistency without overwhelming your environment. Yet, managing tokens, queries, and pushing changes requires a lot of code. We have a better way.
Dynamics 365 EntityChanges Source Type
In our SSIS Integration Toolkit for Microsoft Dynamics 365, EntityChanges is a Source Type of the Dynamics 365/CRM Source component. It leverages Dynamics 365 Change Tracking under the hood to retrieve only incremental changes for a specified entity—no custom API code required.
Prerequisites
- Dynamics 365 (CRM) 2015 Update 1 or later.
- Change Tracking enabled on each entity you plan to synchronize.
Initial Setup and Configuration
Configure the Dynamics 365 Source: Begin by setting Source Type = EntityChanges within the SSIS component. Select the entity you want to track and map the Input Token Variable and Output Token Variable. This ensures the system knows where to read from and where to write the token.
Run Initial Load: On your first run, provide an empty Input Token. This instructs Dynamics 365 to perform a one‑time full load, giving you a complete dataset. The Output Token captured at this point will serve as your reference for incremental runs.
Persist Tokens: Always store the Output Token, often in a SQL table keyed by entity. By persisting this token, you can safely reuse it as the Input Token in subsequent runs, allowing you to maintain continuity in your incremental sync.
Schedule Incremental Loads: Use SQL Server Agent to schedule regular runs at a cadence appropriate for your business needs. This prevents tokens from expiring and keeps your integration jobs consistent and reliable.
Pull Changes and Handle Deletes: Get those changed records and push them to your downstream system. Also, make use of the Secondary Output provided by the EntityChanges Source Type. Route these IDs to a delete process in your downstream system, whether you require a hard‑delete or a soft‑delete strategy.
Resilience: Incorporate retry logic and design an expired‑token branch that can gracefully reset the token. If a token becomes invalid, the system can perform a full sync and then return to incremental loads, minimizing downtime and maintaining data accuracy.
This setup minimizes API calls and keeps your warehouse or downstream apps aligned with Dynamics 365 using only delta changes.
Common Pitfalls to Avoid
Here are some of the most common pitfalls we see when setting up incremental sync with Change Tracking:
- Forgetting to enable Change Tracking: This is the most basic but also the most common mistake. If Change Tracking is not enabled on the entity, no incremental records will ever be returned, and your process will always perform full loads.
- Not storing tokens securely: Tokens are critical to maintaining the chain of incremental syncs. If they are lost or corrupted, you will be forced to run a full reload, which defeats the purpose of incremental design.
- Ignoring deletes: Change Tracking includes IDs of records that have been removed. If you do not process these, your downstream systems will accumulate stale data, leading to mismatches and inaccurate reports.
- Over‑scheduling jobs: Running jobs too frequently increases API calls and risks throttling. On the other hand, under‑scheduling jobs can lead to token expiration, forcing you into a full reload. Striking the right balance in scheduling is key.
By being aware of these pitfalls and proactively planning for them, you can create a more reliable process that is easier to maintain, setting up your integration for long‑term success.
Change Tracking and the SSIS Integration Toolkit Enable Efficient Dynamics 365 Integrations
Efficient Dynamics 365 integrations come down to smart data movement. By using Change Tracking with the EntityChanges Source Type, you avoid the heavy cost of reloading entire datasets and instead focus only on what matters—the incremental changes. Whether you’re syncing to a warehouse or feeding operational systems, our SSIS Integration Toolkit for Dynamics 365 provides a robust approach that keeps systems in sync without wasting resources.
If you’d like help designing or optimizing your incremental strategy, contact KingswaySoft—we’re happy to collaborate on an approach that fits your environment and goals.
By KingswaySoft | www.kingswaysoft.com
The post Using Change Tracking and Entity Changes for Efficient Dynamics 365 Integrations appeared first on CRM Software Blog | Dynamics 365.