Data Processing Using Change Data Capture For Real-Time Data Synchronization

Abstract

In an era increasingly reliant on data, seamless integration and synchronization between various systems are crucial for maintaining consistency, accuracy, and sustainability of information. Change Data Capture (CDC) is a highly effective method for detecting data changes in real-time, allowing systems to record each modification without reprocessing the entire dataset. By integrating CDC with an Event-Driven Architecture, systems can process only data changes, reducing server load and ensuring data consistency across systems. This study examines a data synchronization system built using CDC and Event-Driven Architecture on a dataset of 1,000,000 records. The test results indicate strong performance, with the fastest average processing time recorded at 2.6622 milliseconds per data entry and a data loss rate of 0. These findings demonstrate that the combination of CDC and Event-Driven Architecture offers an efficient, fast, and scalable solution for real-time data synchronization on a large scale.