{'fraction_non_alphanumeric': 0.015414258188824663, 'fraction_numerical': 0.0, 'mean_word_length': 4.708791208791209, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An Empirical Study of Open Data JSON Files

JSON is challenging XML as lingua franca of data exchange formats. Likewise, the JSON schema initiative has been gaining momentum to the point where it is considered by many as a defacto standard. Both the JSON data format and the JSON schema formalism allow for great degrees of freedom in modelling nested and semistructured data. In this article, we introduce a scalable tool, jHound for profiling JSON document collections. We use this tool to pursue the question how real-life datasets are structured, and whether developers actually make use of the features offered by these languages when modelling their data. For this analysis, we focus on JSON documents available as open data. While this sample can only deliver a biased snapshot of what real JSON is like, we gain first insights into the common practices in modelling data with JSON. The jHound tool is provided and available as open source, so that scientist and practicioners can apply it for profiling of other JSON data sources.