{'fraction_non_alphanumeric': 0.023642943305186973, 'fraction_numerical': 0.004583835946924005, 'mean_word_length': 5.169642857142857, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Autism Developmental Disabilities Monitoring Network surveillance: A reply to Drs Durkin, Bilder, Pettygrove, and Zahorodny

We appreciate Drs Durkin, Bilder, Pettygrove, and Zahorodny’s thoughtful response to our editorial regarding the most recent autism prevalence estimates from the Center for Disease Control and Prevention Autism Developmental Disabilities Monitoring (ADDM) Network. It appears that we agree on several points, disagree on others, and have been misinterpreted on at least one point. We agree with the authors that record review is much more cost-effective than direct assessment of an entire population. When condition-specific laboratory tests are used, this type of surveillance can be quite effective. Where this type of surveillance is particularly effective is when a biomarker and specific associated laboratory test exist, such as in infectious disease and many types of cancers. This is not the case for autism spectrum disorder (ASD). A fundamental assumption underlying these methods is that the properties of the laboratory test—its ability to discriminate between those with and without the health condition—do not change over time. Because there is no biomarker for ASD, ADDM Network surveillance efforts are dependent, not on a specific laboratory test, but on a disparate body of information found in a child’s record. As research using ADDM data has found, gold standard diagnostic assessments rarely are used in community practice. Therefore, ADDM clinicians must search for specific words and phrases found in children’s records. There are a number of potential pitfalls with this method. First, it is completely dependent on the clarity of the information found in the records. In addition, if these words and phrases are used with different frequency over time and across site, it could dramatically affect the observed prevalence. While ADDM network criteria may be consistent over time, community clinician behavior may not be, with more clinicians including the language of ASD diagnosis, even when children would not meet the criteria if they were directly assessed by an experienced clinician. We disagree with the authors’ implication that the only alternative to this strategy is to conduct direct assessment of hundreds of thousands of people. Many successful prevalence studies of psychiatric and developmental disorders have been conducted using two-stage, populationbased sampling. Typically, this is done by first screening a large portion of the population using an inexpensive screening tool, then more carefully assessing a large fraction of those with very high screening scores and a smaller fraction of those with low screening scores. In fact, this is exactly the method currently in use by one of the ADDM sites to validate the current surveillance strategy. We eagerly await the results of this validation study and would be delighted if our concerns about the ADDM surveillance efforts prove to be unfounded. We agree with the authors that some studies have been conducted to validate the current surveillance strategy. We disagree, however, on the interpretation of those studies’ results. In the most recent validation study, Bakian et al. (2014) tested the extent to which external clinical reviewers agreed with the study reviewers when they reviewed children’s records. The authors found good agreement (intra-class coefficient = 0.84). Inter-rater reliability is important, but does not speak to the validity of the method. That is, experts can agree on what they see in the record, but may decide something else entirely if they observe the child directly. The only validation study to directly assess children in the ADDM Network study was conducted by the same group of researchers (Avchen et al., 2011). To our point, they reported that ADDM Network strategy resulted in a sensitivity of 0.60 and positive predictive value of 0.79. This means that 21% of identified “cases” did not in fact have ASD. They also reported a negative predictive value Autism Developmental Disabilities Monitoring Network surveillance: A reply to Drs Durkin, Bilder, Pettygrove, and Zahorodny