{'fraction_non_alphanumeric': 0.02276176024279211, 'fraction_numerical': 0.0015174506828528073, 'mean_word_length': 5.0228310502283104, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Implementing a Searchable Database, Reporting Broken Links, and Refining Search Results

MBARI uses EndNote, a software specifically designed for bibliographies, to keep track of publications. Static HTML pages produced from the EndNote database are published on the web. It is possible to put the EndNote database online, which would allow anyone with an EndNote client to search the references. However, it is necessary to make the publications database searchable for any visitor with a web browser, regardless of whether they have EndNote or not. To that end, we decided to customize Reference Manager Web Publisher, a companion product to EndNote. Section 1 discusses issues with the current static HTML pages, why we chose Reference Manager, and the steps taken to customize the software. For indexing/searching of the internal and external web sites, MBARI uses HtDig, a web indexing/searching freeware. As part of the database building process, HtDig generates a report of broken links. However, the report is quite unreliable—it reports links that are not broken, fails to report actually broken links, and lists the broken URLs but not the referring URLs. Section 2 explains issues with the current broken link reporting feature of HtDig and why we selected Web Link Validator software to report broken URLs.