{'fraction_non_alphanumeric': 0.023875114784205693, 'fraction_numerical': 0.0, 'mean_word_length': 5.300578034682081, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
On-the-fly data pipeline for image processing enabling real-time persistence correction

In this paper, we present our concept of an on-the-fly image processing pipeline for ground-based telescopes. Our focus lies on the correction of undesired but unavoidable detector effects like persistence. State of the art systems like ESO’s NGC perform operation of detectors and acquisition of science data. Instrument operators monitor the science data stream during measurement. Detector issues like persistence worsen the image quality and impair the inspection tasks. To overcome these difficulties, we develop the real-time data processing system EIDAS (Enhanced Infrared Detector Acquisition System) as an add-on to the NGC. Therewith, detector effects in the raw image stream are corrected on-the-fly and astronomical objects are visualized more clearly for direct inspections. EIDAS accompanies CPU, FPGA and GPU as well as algorithms optimized for this hardware architecture. The system is designed generically, matching various types of detectors and their respective science data output