{'fraction_non_alphanumeric': 0.01818181818181818, 'fraction_numerical': 0.0021390374331550803, 'mean_word_length': 4.66969696969697, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Estimating population size and duplication rates when records cannot be linked

The capture‐recapture approach to estimating the size of a population is a well‐studied area of statistics. The number of distinct individuals, NA and NB, on each of two lists, A and B, and the number common to both lists, NAB, are used to form an estimate of the binomial probability of being on one of the lists, which then allows an estimate to be made of the size of the population. Critical to the method is an accurate count of NAB. We consider situations in which this count is not available. Such problems arise in a variety of behavioural health contexts in which the need for protection of privacy may prevent sharing identifying information, so it is not possible to specifically match an individual who appears on one list with an individual on the other. Suppose that the birth dates and/or other demographics of individuals on each list are known. We introduce two methods for estimating the duplication rates and the size of the population. Conditioning on the set β of birth dates of those on list B, NA and NB, the maximum likelihood estimators (MLEs) and their variance are derived. The MLEs are based on the proportion of individuals on list A whose birth dates fall in β. This approach is particularly useful if list B itself contains duplicates. The second model utilizes the full sample distribution of the birth dates. We generalize this approach to accommodate multiple demographic characteristics. The approaches are applied to the problem of estimating duplication rates and the population size of veterans who have mental illness in Kings County, NY. The data are lists of those receiving service from the Veterans Administration system and from providers funded or certified by the New York State Office of Mental Health. Copyright © 2003 John Wiley & Sons, Ltd.