{'fraction_non_alphanumeric': 0.017653167185877467, 'fraction_numerical': 0.004153686396677051, 'mean_word_length': 5.469798657718121, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Industrial Application of Automated Regression Testing in Test-Driven ETL Development

While there are many commercial Extraction, Transformation and Loading (ETL) platforms that facilitate fast and easy development of ETL applications, ETL testing is largely a manual process. In order to allow frequent releases of reliable ETL applications, an automated regression testing approach is needed. This paper presents 1TH, one such test harness tool and a methodology for agile ETL development. 1TH builds on the existing techniques in testing and database research areas. It addresses practical challenges with nondeterministic values and, to a limited extent, test dependence. 1TH is applied in an industrial ETL development, and evaluated in a case study involving production releases once every couple of weeks for more than three years. 1TH introduced minimal delays in the development process and caught numerous regressions before they escaped to production.