{'fraction_non_alphanumeric': 0.02029136316337149, 'fraction_numerical': 0.0, 'mean_word_length': 5.263843648208469, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Everything, for ever? The preservation of South African websites for future research and scholarship

The amount of material published on the worldwide web continues to grow. This is also true of South Africa. Much of this material is banal and ephemeral, but it nevertheless forms part of South Africa's documentary heritage. It will have value for future historians and social scientists just as previous centuries’ printed broadsheets, newspapers and ‘penny horribles’ are used today as raw material for research. The proliferation of websites worldwide poses enormous challenges to heritage institutions such as national, research and repository libraries. In the developed countries the capturing, organization and preservation of websites have become an important theme in the professional literature of information science. Many difficulties have to be overcome before we can be sure that an adequate proportion of this material is preserved for future use. The difficulties are not only of a technical nature, but also organizational, economic, political, legal and ethical. This paper draws on experience gained in two current projects funded by the Andrew W. Mellon Foundation. The first is the Political Communications Web Archiving Project, undertaken in the United States under the aegis of the Center for Research Libraries, Chicago, which has attempted to develop a model for the preservation of websites for use by area studies researchers at US universities. The second is a South African project on the legal deposit of electronic publications, managed by the Foundation for Library and Information Service Development (FLISD) on behalf of the National Library of South Africa. The paper outlines the major issues that have to be addressed when a national system for the preservation of websites is set up, with special emphasis on ‘soft’ issues (political, legal and moral) rather than technical issues.