Speed Performance Analysis of Big Data Processing with Micro Batching at the ETL Process for Event-Data

Abstract

Speed performance in Big Data Processing is one of the key aspect of presenting a real-time data for a real-time analysis in various use cases. One of such data is an Event-Data or an Activity Data that is often used for making analysis and business decisions in real-time. This research focuses on analyzing the speed performance of Big Data Processing in the Load stage of Extract, Transform, Load using Micro-Batching. The speed performance is evaluated using Load-Testing with K6 and the configuration of 10 VUs and 10000 data. The Load-Testing results of the speed performance shows that implementing Micro-Batching results in 63.63% faster in performing all the request, 59.03% faster in HTTP Request Duration, and 62% faster in HTTP Request Waiting duration.