{'fraction_non_alphanumeric': 0.019079685746352413, 'fraction_numerical': 0.0016835016835016834, 'mean_word_length': 5.703007518796992, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
DATA ANALYSIS OF THE FIRST TWO PBR REPLICATIONS IN READERS PROJECT

This Technical Report describes two empirical studies carried out in the context of the Project “Readers: A Collaborative Research to Develop, Validate and Package Reading Techniques for Software Defect Detection” — where Brazilian and American researchers investigate the effectiveness of software requirements documents inspection techniques under diverse technical and cultural settings. The studies conducted are replications of a previous experiment on a family of reading techniques named PBR — Perspective Based Reading, and had as subjects undergraduate students enrolled in computing courses at ICMC/USP and DC/UFSCar. Identified as replications R1 and R2, respectively, they compared the PBR and Checklist techniques for Requirements Documents analysis. Four metrics were used to evaluate the data collected: Defects Found, Occurrences of Defects, Effectiveness and Efficiency. Although both replications produced similar results for one of the two requirements documents used, some conflicting results were produced for the other document. In R1, PBR did better than the Checklist technique on one of the documents, in agreement with results from previous studies. However, for the other document, Checklist did better in terms of the subject’s effectiveness, one of the metrics applied for analyzing the results. These conflicting results are discussed, possible sources of variation amongst the experiments are identified and actions to mitigate such problems in future replications are proposed. Neither Checklist nor PBR led to complete uniformity of defect reporting, but with PBR a higher percentage of subjects achieved the same higher performances (within each perspective) in both replications.