{'fraction_non_alphanumeric': 0.024705221785513758, 'fraction_numerical': 0.008983717012914094, 'mean_word_length': 5.261862917398946, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Error, Reliability and Health-Related Digital Autonomy in AI Diagnoses of Social Media Analysis

The rapid expansion of computational tools and of data science methods in healthcare has, undoubtedly, raised a whole new set of bioethical challenges. As Laacke and colleagues (2021) rightly note, the use of AI technologies in mental health, especially the use of AIDDs (AI depression detectors) in social media, has the potential both to underor to over-detect signs of mental disorders and to drastically limit a user’s or a patient’s autonomy. The novelty of Laacke and colleagues’ analysis consists in extending our traditional ethical concepts such as autonomy to measure their usefulness in crossing areas at the boundary of the digital and of the health system. Here we point to two significant challenges that their argument for a new conception of health-related digital autonomy continues to face as a result of the technological opacity inherent to healthcare machine-learning algorithms (HC-MLAs). We contend that Laacke and colleagues can reach a more optimistic conclusion about a responsible use of AI in mental health practices only by addressing these challenges. This is particularly important given the recent prominent call for closer inspection and regulation of emotional recognition software systems in general (Crawford 2021). The first challenge highlights three interrelated epistemic concerns: the problem of proxies, the problem of measurement bias, and the problem of explainability vs. efficacy in healthcare. It is important to realize that in order for AI technologies to reach an allegedly ‘depression diagnostic’, they often use non-medicallyrelevant proxies derived from statistical correlations in elements of data sets that are not recognizable to medical professionals as medical phenomena. Because of this, these technologies are prone to generate measurement bias, in which what is thought to be tracked, in this case depression, does not correspond to the statistically salient elements used by machine learning methods such as deep neural networks (Sun et al. 2018; Suresh and Guttag 2019) used for language or image analysis in social media. Hence, the AI that Laackey et al. refer to may not be capturing “depression” per se but rather other medically irrelevant phenomenon such as syntactic distance between words, times of posting or search terms related to famous sad songs. Furthermore, multiple protected features—such as socio-cultural traits of language use, socio-economical backgrounds, race, gender, etc.— could be used by ML to label someone as “depressed.” This could lead to the normalization of emotive expression, considering that data is more immediately available and more often analyzed for majority groups. Importantly, the opacity regarding salience in machine learning methods (Alvarado 2020), and particularly in deep neural networks (Sun et al. 2018) make it so that researchers and medical professionals could not know exactly what it is that AI is taking into consideration or how to correct it if it fails. One could respond to this initial challenge that the question in mental healthcare is not one of explainability—whether AI are truly tracking depression for the right reasons—but rather one of efficiency (London 2019). While efficiency could possibly trump explainability in medicine, this tradeoff should not be made without taking ‘catastrophic’ failures in some machine learning systems seriously (Lynch et al. 2017). This is the third aspect of the epistemic challenge for the use of