Enhanced Late-Straggler Algorithm With On-Demand Etl For Big Data Retrieval

• Main Author: Zakaria Katrawi, Anwar Hussein
• Format: Thesis
• Language: English
• Published: 2022
• Subjects: T1-995 Technology(General)
• Online Access: http://eprints.usm.my/60026/

The growth of digital information is phenomenal. Digital documents dominate nearly every aspect of doing business to the point that it is hard to imagine doing without them. With an unprecedented potential lurking in its depths, the ongoing digital information revolution also presents risks and challenges, mainly when dealing with the extraction and analysis of digital data. The conventional method ETL of Big Data processing consists of Extraction, Transformation, and Loading integrated into a warehouse. Using this method without any optimization often leads to a delay in data retrieval, known as the straggler problem, which is a situation that arises when tasks are delayed due to low processing on some nodes. The straggler problem is considered by many as a major problem, especially when the data resources are important and if these resources are inefficiently used. Hence, detecting and, therefore, eliminating the straggler problem early is crucial to enhancing the ETL performance.