{'fraction_non_alphanumeric': 0.01847826086956522, 'fraction_numerical': 0.0, 'mean_word_length': 5.222972972972973, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A big data cleansing approach for n-dimensional RFID-Cuboids

Radio Frequency Identification (RFID) technology has been widely used in manufacturing sites for supporting the shopfloor management. Huge amount of RFID-enabled production data has been generated. In order to discover invaluable information and knowledge from the RFID big data, it is necessary to cleanse such dataset since there is large number of noises. This paper uses n-dimensional RFID-Cuboids to establish the data warehouse. A big data cleansing approach is proposed to detect, remove and tidy the RFID-Cuboids so that the reliability and quality of dataset could be ensured before knowledge discovery. Experiments and discussions are carried out for validating the proposed approach. It is observed that the proposed big data cleansing approach outperforms other methods like statistics analysis in terms of finding incomplete and missing cuboids.