Change Data Capture

Change data capture allows for constant data updating, listening to changes in the source database, and streaming the data to the target database as it is modified - and fully respects transactional integrity.

Omni Loader is a very performant migration solution, and its parallel workers significantly speed up the migration process. While it migrates terabytes of data extremely quickly, the process is not transaction aware. This is why we have incorporated CDC (change data capture) into Omni Loader’s migration engine.

As Omni Loader is a fully self-tuning solution, we have made CDC an automatized feature that requires no additional setup and can be activated with a single click.

Deploying CDC in Omni Loader

After selecting the source and target database, you can choose to start listening, by clicking on the button of the same name. This continuously monitors the source database for changes and accordingly updates the target database, displaying the number of transactions, updates, writes, and deletes that occurred. While listening, you are offered options to pause or stop listening. Pausing allows Omni Loader to continue listening but does not update the target database or the number of transactions. It is aware of the changes and is ready to replace the data when resumed. By choosing to stop listening, Omni Loader fully stops CDC and does not listen or stream any further data. Resuming is still a single click away and will reinitialize CDC from the start, triggering full (massively parallel) data load to populate the target tables.

What sets Omni Loader’s CDC apart

One of the main advantages of using CDC within Omni Loader is the ease of use. While CDC solutions typically require a comprehensive setup, Omni Loader requires none and fully sets up everything on its own. We have fully eliminated the multi-day setup times typically required by CDC solutions. Omni Loader uses parallel workers during the initial full load which substantially cuts load time down. After the initial load, its engine streams the data using one worker in real-time. This allows for the target database to always be up to date regardless of the rate of changes in the source database.

Why CDC is important and what companies look for

Change data capture allows for continuous data transfer as changes occur. This enables companies to keep their systems updated at all times. For these reasons, the demand for CDC had grown rapidly in the past few years.

Omni Loader was designed to abstract all the complexity away. This brings the following benefits without overwhelming the user:

Interested? Request a demo