{'fraction_non_alphanumeric': 0.026901669758812616, 'fraction_numerical': 0.0, 'mean_word_length': 5.273255813953488, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Storage and index support for data intensive web applications

In this paper, a system named as DisGR, for Distributed Graph Repository, that is designed and developed for supporting Chinese Web related research, is introduced. The system is designed based on a graph data model, TGM (for Tagged Graph Model), that is designed for representing Web data, especially forum and BBS data. DisGR supports the query language TGM-L that aims at analytical tasks for TGM data. For high-scalability and availability purpose, DisGR is designed for clusters with shared-nothing architecture. DisGR has several characteristics such as column-based storage, descriptive language support, and flexible user-defined function support. DisGR is different to other database systems with similar purpose in three perspectives. First, catalog is maintained by a set of servers connected via a DHT overlay. Second, signatures with different granularities are used for data distribution and query optimization. Last but not the least, update is supported via timestamps and regularily reorganization.