{'fraction_non_alphanumeric': 0.01683937823834197, 'fraction_numerical': 0.0038860103626943004, 'mean_word_length': 5.130952380952381, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
EVENT INDEX BASED CORRELATION ANALYSIS FOR THE JUNO EXPERIMENT

The Jiangmen Underground Neutrino Observatory (JUNO) experiment is mainly designed todetermine the neutrino mass hierarchy and precisely measure oscillation parameters by detectingreactor anti-neutrinos. The total event rate from DAQ is about 1 kHz and the estimated volume of rawdata is about 2 PB/year. But the event rate of reactor anti-neutrino is only about 60/day. So one of thechallenges for data analysis is to select sparse physics signal events in a very large amount of data,whose volume can not be reduced by using the traditional data streaming method. In order to improvethe speed of data analysis, a new correlated data analysis method has been implemented based onevent’s index data. The index data contain the address of events in the original data files as well as allthe information needed by event selection, which are produced in event pre-processing using theJUNO’s Sniper-based offline software. The index data are subsequently selected by using refinedselection criteria with Spark so that the volume of index data is further reduced. At the final stage ofdata analysis, only the events within the time window are loaded according to the event address in theindex data. A performance study shows that this method achieves a 14-fold speedup compared tocorrelation analysis by reading all the events. This contribution will introduce detailed software designfor event index-based correlation analysis and present performance measured with a prototype system.