{'fraction_non_alphanumeric': 0.02891573211866117, 'fraction_numerical': 0.018364228673566466, 'mean_word_length': 5.506846745451135, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Characteristics of publicly available skin cancer image datasets: a systematic review

Publicly available skin image datasets are increasingly used to develop machine learning algorithms for skin cancer diagnosis. However, the total number of datasets and their respective content is currently unclear. This systematic review aimed to identify and evaluate all publicly available skin image datasets used for skin cancer diagnosis by exploring their characteristics, data access requirements, and associated image metadata. A combined MEDLINE, Google, and Google Dataset search identified 21 open access datasets containing 106 950 skin lesion images, 17 open access atlases, eight regulated access datasets, and three regulated access atlases. Images and accompanying data from open access datasets were evaluated by two independent reviewers. Among the 14 datasets that reported country of origin, most (11 [79%]) originated from Europe, North America, and Oceania exclusively. Most datasets (19 [91%]) contained dermoscopic images or macroscopic photographs only. Clinical information was available regarding age for 81 662 images (76·4%), sex for 82 848 (77·5%), and body site for 79 561 (74·4%). Subject ethnicity data were available for 1415 images (1·3%), and Fitzpatrick skin type data for 2236 (2·1%). There was limited and variable reporting of characteristics and metadata among datasets, with substantial under-representation of darker skin types. This is the first systematic review to characterise publicly available skin image datasets, highlighting limited applicability to real-life clinical settings and restricted population representation, precluding generalisability. Quality standards for characteristics and metadata reporting for skin image datasets are needed.


Introduction
Digital health innovation has the potential to improve health care by increasing access to specialist expertise. 1,2 Among the myriad of machine learning applications in health care, medical image classification, particularly for dermatology, has advanced substantially in recent years, 3 and includes diagnosis of skin cancers from dermoscopic or macroscopic photographs. [4][5][6][7][8][9][10] Advances in machine learning algorithm diagnostic accuracy have largely been driven by utilisation of deep learning architectures made possible through greater availability of computing power and large repositories of digital images for algorithm training. 11,12 For this purpose, large numbers of digital images easily accessible through publicly available datasets have been used in dermatology. 13 Publicly available datasets used for developing machine learning algorithms circumvent barriers to dataset procurement, such as having appropriate technological infrastructure, regulatory approvals, time, and financial investment for large-scale digital image acquisition from participants. 14 Furthermore, publicly available datasets can be used as a benchmark for direct comparison of algorithm performance. 15,16 Skin cancer incidence continues to rise globally, placing increasing demands on health-care services. [17][18][19][20][21] Digital solutions to address this demand have been reflected by accelerated teledermatology adoption during the COVID-19 pandemic. 22,23 Machine learning algorithms have potential for automated diagnosis of skin mali gnancies through digital image analysis, and diagnostic accuracy of machine learning algorithms has been shown in the past 5 years to be comparable to, or even surpass, dermatologists in controlled experimental settings. [4][5][6][7][8][9] Publicly available skin image datasets, such as those hosted through the International Skin Imaging Collaboration (ISIC) archive, 13 are increasingly used to develop machine learning algorithms for skin cancer diagnosis. [24][25][26] Additionally, although primarily aimed for use as educational resources, dermatology atlases (compilations of photographs of skin diseases) containing digital images are frequently used as a source of skin lesion images for algorithm development. 27 However, with training data from circumscribed populations, often curated retrospectively, machine learning algorithms are susceptible to overfitting, and their generalisability is heavily influenced by the participants and images used for training, which are prone to selection bias. 28 Algorithms used for skin lesion classification frequently underperform when tested on independent datasets. 26,29,30 Further examples of the susceptibility of machine learning algorithms to biases for clinical factors such as age, sex, ethnicity, and socioeconomic status have also been reported in diverse areas of health care and artificial intelligence. [31][32][33][34][35] This underlines the importance of detailing the exact composition of datasets through metadata reporting, to ensure the generalisability of algorithms to real-world populations. Furthermore, the recently developed concept of health data povertysystematic data disparities leading to inequalities in health care-emphasises the need to ensure diversity, transparency, and usability of datasets. 14,36,37 Demographic and clinical metadata can also greatly influence machine learning algorithm development and validation, with classification accuracy increasing for skin lesions when subject and lesion metadata are integrated into models. 38,39 This more closely reflects clinical practice, as dermatologists conduct in-person history taking and evaluation of skin lesions via naked eye, dermoscopic, and physical examination (eg, lesion e65 www.thelancet.com/digital-health Vol 4 January 2022 Review palpation), and not through image analysis alone. Completeness of metadata provides information about the population, disease, and data types on which the algorithm was trained or validated, which is essential for extrapolating assumptions of generalisability of algorithm performance to other populations.
To the best of our knowledge, there are no guidelines or quality standards that characterise optimal skin image datasets that could be used to train machine learning algorithms, and these datasets have not previously been systematically characterised. This review aimed to identify publicly available skin image datasets used to develop machine learning algorithms for skin cancer diagnosis, categorise their data access requirements, and systematically evaluate their characteristics including associated metadata.

Search strategy and selection criteria
This systematic review adheres to PRISMA guidelines; 40 for the PRISMA checklist see the appendix (p 12). As the review did not evaluate a direct health-related outcome, it did not meet criteria for registration of the protocol with PROSPERO. 41 Searches were conducted on MEDLINE, Google, and Google Dataset Search, on Sept 4, 2020. The MEDLINE search was updated on Sept 1, 2021. The database was searched from inception with MeSH terms and keywords including "dataset" OR "database", "artificial intelligence" OR "machine learning", "skin cancer", and "imaging" OR "dermoscopy". The full search strategy is given in the appendix (p 2). Two independent reviewers (DW and AJX) screened titles and abstracts for articles describing skin cancer image datasets, studies using datasets to train or test machine learning algorithms for skin cancer diagnosis, or review articles detailing any skin cancer image datasets. Two independent reviewers (AJX, DW, HI, or SMK) subsequently reviewed full-text articles for publicly available datasets and attempted to access them at source. Full texts were reviewed for publications where abstracts were unavailable. Google Translate was used for seven non-English articles (four in German, one in French, one in Spanish, and one in Chinese).
Google and Google Dataset searches were completed by two independent reviewers (DW and AJX) to identify publicly available skin cancer image datasets. These searches were completed using the search term "skin cancer image dataset", and repeated with "melanoma image dataset" (appendix p 2). The number of new skin cancer image datasets found on each search result page were recorded. Reference lists of any online articles were also reviewed for named publicly available datasets. Once 20 consecutive search results no longer mentioned any new datasets, the number of search results for review was rounded up to the nearest multiple of 50, and no further webpages were reviewed beyond this point.
To be included for data extraction, datasets had to contain images of either cutaneous melanoma, basal cell carcinoma (BCC), or cutaneous squamous cell carcinoma (cSCC). Datasets could contain any form of nonradiological skin lesion images, such as macroscopic clinical photographs or dermoscopy. There was no restriction on geographical origin, patient population, or language.
Datasets were excluded if they did not contain images of skin cancers (eg, skin rashes or benign lesions only). Histopathological image and radiological image datasets were also excluded. Datasets containing text or numericonly data and images of non-human subjects were excluded, as were inaccessible datasets that were described as open access but were either inactivated or unable to be found by two reviewers (DW and SMK). All datasets included in the review were agreed by consensus by three authors (DW, AJX, and SMK).

Data analysis
Identified datasets meeting inclusion criteria were grouped into categories based on a system used by Khan and colleagues. 14 These included open access datasets (freely accessible or easily accessible via registration or email) and regulated access datasets (requiring payment, formal institutional agreements, or ethical approval).
A data collection template was designed to evaluate dataset characteristics, including number of images, number of participants (ie, individuals from whom images were taken, regardless of whether their active consent was reported), country of origin, publication date, imaging modality, image capture device, image format, and number of skin lesion categories. Further items reviewed for reporting are listed in the appendix (p 3). Metadata labels associated with individual images were also reviewed for included datasets. Images and metadata of included datasets were manually reviewed by two independent reviewers (DW and SMK), together with corresponding articles or supplementary infor mation describing the dataset. Discrepancies were resolved by discussion between reviewers, and a third party with methodological expertise (RNM) if required.
All images and accompanying data from open access datasets were imported into Databiology Lab (LS, JC, LZ, and CdBP), a data management and analysis platform. Wherever possible, image files were registered without downloading to reduce data duplication. Metadata was recorded in entities with a number of attributes and values. Search queries were created to evaluate metadata associated with images (collectively or by individual dataset).
We found that articles describing machine learning algorithms also used images from online dermatology atlases to train algorithms. Therefore, atlases were also included if they met the aforementioned inclusion criteria. Atlases were defined as mainly educational electronic resources that either included "atlas" in the title or displayed images on multiple webpages. This was in contrast to datasets, which contained multiple images See Online for appendix www.thelancet.com/digital-health Vol 4 January 2022 e66 Review downloadable in one or a few compressed files (.zip or .rar).
As characteristics and metadata varied greatly within atlases and between atlases, a simplified data extraction template was used (appendix p 4). Images from atlases amenable to download or linkage were imported to Databiology Lab for search and examination of available metadata (LS, JC, LZ and CdBP). For atlases for which this was not possible, a sample of images from each atlas was reviewed by two reviewers (DW and SMK). If enough images were available, ten consecutive images each of BCC, cSCC, and melanoma were reviewed by each reviewer, either from the top (DW) or bottom (SMK) of webpages or atlas indexes. Atlases were deemed to have metadata reporting for an item (eg, lesion site) if more than 50% of reviewed images contained the metadata label.

Results
The MEDLINE search returned 1897 articles, which were screened by title and abstract (figure 1). Review of 584 full-text articles identified 26 datasets and 20 atlases meeting inclusion criteria. Review of 200 webpages from each of the Google and Google Dataset searches, along with supplementary screening, identified 25 datasets and 12 atlases. Combining these results, 22 duplicate datasets and 12 atlases were excluded, leaving 29 individual datasets and 20 individual atlases included for review.
Attempts to access data at source revealed eight regulated access datasets (table) and three regulated access atlases. [56][57][58] Of the eight datasets, six required ethical committee or institutional approval, 10,50,53-55 one required a £75 fee and a licencing agreement, 51 and one required a competition invitation. 52 Thumbnails of images from Asan, Hallym, SNU, and Severance datasets were available for download, but access to full-size images required formal approval from local data access or ethics committees, 50 or the originating hospitals. 10 All three regulated access atlases required payment for access to digital images (US$200; £145). [56][57][58] Char acteristics of these regulated access datasets and atlases were gathered from information at source or accompanying publications; the images themselves or associated metadata files were not downloaded and reviewed.
Inaccessible datasets, which could not be analysed further, included one dataset from China (XiangyaDerm), which was described to be publicly released 59 but could not be found via the stated URL, and one atlas that had been inactivated (DermQuest). One further dataset might have been available upon reasonable request to the authors, 60 although we did not receive a response to our request for access. Datasets that contained images that were also included in another dataset, and therefore did not contain any unique images, were not classed as an additional independent dataset. All 1512 images in the ISIC 2018 task 3 challenge were included in the ISIC 2019 challenge, so the former was not included as an individual dataset. This also applied to SD-198 and SD-128 datasets, 61 images of which were included in SD-260. 49 As images from preceding ISIC challenges have been re-used in later challenges (appendix p 5), ISIC images were grouped by their original datasets, as presented in the ISIC archive image gallery (accessed March, 2021), to avoid duplicate images being counted more than once in analyses. Images not included in the ISIC archive image gallery and only available via ISIC challenges were listed separately (ISIC 2019 and 2020 challenge test sets, and ISIC 2018 challenge task 1-2 test set).
Overall, 21 open access datasets and 17 open access atlases included images that could be downloaded without barriers, and were included for data extraction. The access links to individual datasets are detailed in the appendix (p 6).
14 (67%) of 21 datasets detailed country of origin (table). Of these, 11 originated from one country only: six from Europe, two from Oceania, one from North America, one from South America, and one from Asia (appendix p 7). Three datasets contained images from multiple  Review medical records, clinical consensus between reviewers, diagnosis as described originally in an atlas, serial imaging, follow-up of patients over a specified time period, confocal microscopy, and histopathology. Completeness of reporting for dataset characteristics is summarised in figure 3A. Reporting for individual datasets is displayed in the appendix (p 8). The number of individuals from whom images were taken was reported in eight datasets (38%), data collection period in eight (38%), inclusion or exclusion criteria in 12 (57%), device used for image acquisition in 12 (57%), and whether there was image processing or adjustment by a reviewer in four (19%). Ethical approval was reported in ten datasets (48%), and participant consent in seven (33%).
Overall, the open access datasets contained 106 950 skin lesion images (median 2298, IQR 429-9232). 60 189 (56·3%) were included in datasets hosted within the ISIC archive, 20 220 (18·9%) are currently being used in ongoing ISIC challenge test sets, and 26 541 (24·8%) were from non-ISIC datasets (figure 2C). No ground truth labels were available for the 20 220 images currently being used in ongoing ISIC challenge test sets (table), although these might be released in future. Considering the eight datasets that reported number of participants, there were a total of 48 419 images from 5130 participants.
In the SKINL2 dataset, 48 56 These images were made publicly available in 2018. §The number of participants was not reported for the 240 full-size photos available for download. However, for the larger dataset of 2201 images, the number of participants was reported as 1608. ¶Images from this dataset include images from DermIS and DermQuest (deactivated) atlases. ||Images from this dataset include images from the deactivated DermQuest atlas. **A further iteration of this dataset containing additional images is described by Han and colleagues; 10 however, the images cannot be made available in totality due to privacy regulations. Patient ethnicity data were available for 1415 images (1·3% of all images), and Fitzpatrick skin type data for 2236 (2·1%). Skin type and ethnicity data associated with images is shown in the appendix (p 10) for datasets for which metadata labels were available. Aggregate data for ethnicity were available in one dataset containing 170 images (White participants only), 47 and skin type in another dataset that contained 200 images (all images were Fitzpatrick skin type II or III). 15 Aggregate metadata for the larger SNU dataset were given for age, sex, and ethnicity, 10 but further detail was not provided for the subset of downloadable 240 full-size images.
17 open access atlases were identified from the search ( figure 1 and appendix p 11). These were mostly hosted by countries in Europe (seven), North America (six), and Oceania (two). One atlas was hosted from South America and one from Japan. 12 atlases contained macroscopic photographs only, one contained dermoscopic images only, and the remaining four contained both dermoscopic and macroscopic photographs, with three containing paired dermoscopic and macroscopic images. Regarding metadata, images were either presented alone (six of 17), with a caption containing variable clinical or educational material (three), with site metadata in the webpage name (three), with structured metadata labels (three), or both structured metadata and captions (two).
Four of 17 atlases were imported into the Databiology platform. Barriers to importing images and metadata from the remaining 13 atlases included images being spread across multiple webpages, aged websites, and presentation in non-English languages.
There were 1407 images from four atlases imported into the Databiology platform, comprising 771 BCC, 206 cSCC, and 430 melanoma images. Of these, 0% had associated metadata for age, 0% for sex, 54·4% (765 images) for site, 0% for ethnicity, and 0% for Fitzpatrick skin type. A few atlases included watermarks, some of which were in close proximity to the lesion; 62 this can affect algorithm performance by making the lesion more difficult to assess and by causing the algorithm to associate the watermark with the lesion.

Discussion
In this first review of publicly available dermatology datasets for skin cancer, 106 950 images from 21 open access datasets were identified, along with 17 open access atlases that are freely accessible to researchers and the public. Although this represents a rich data resource for innovation, lack of transparency in metadata reporting for clinically essential characteristics (such as ethnicity and Fitzpatrick skin type) limits the clinical utility of these images alone. These issues are not limited to dermatology datasets, but have also been reported in ophthalmology 14 and radiology. 63 Additionally, we identified eight regulated access datasets containing at least 63 687 images, and three regulated access atlases. We also found that images are, in some cases, reused between datasets. Difficulties were encountered when importing images for the majority of atlases, with implications for usability.
Reporting of metadata was limited in many datasets and atlases, which, if used for training or validating machine learning algorithms, would have implications Dermoscopic only (13) Macroscopic and dermoscopic (1) Light field and dermoscopic (1) ISIC archive (56·3%) ISIC challenge (18·9%) Non-ISIC (24·8%) e70 Review for generalisability. Machine learning algorithms used for medical image classification are known to underperform on images collected from populations independent to those on which the algorithms were trained. An image classifier algorithm trained and validated predominantly on images of east Asian skin underperformed on skin lesion images of White patients from the USA. 29 Likewise, the majority of algorithms submitted to the ISIC 2018 task 3 challenge performed worse on test images from an external institution independent from the training dataset. 26 A limited number of studies of machine learning algorithms for medical image classification include validation with external datasets, highlighted by two recent systematic reviews which found that only 18-36% of studies validated their algorithms using externally sourced images, meaning the reported accuracy of algorithms might be generally overestimated. 3,64 These findings highlight the dangers of implementing algorithms for widespread use on broad populations without dataset transparency, especially if algorithm training was undertaken using a restricted demographic cohort. Algorithm under performance and misdiagnosis have serious implications for patients with skin cancer; they not only risk missing treatable malignancies, but can also result in avoidable surgical procedures and cause unnecessary anxiety. 65 Our review identified limited metadata reporting for datasets and atlases, therefore raising concerns about which populations are represented and to what extent any artificial intelligence algorithms developed using these are generalisable.
Limited metadata reporting could be due to the substantial time, effort, and approvals required to collect this information, but might also be because of the current lack of consensus and guidance regarding which metadata items are most important to report for skin image datasets. This is reflected in the large variability of metadata reporting between datasets, highlighting the need for consensus standards.
Although consensus recommendations for the development of imaging biomarkers in cancer and checklists aiming to improve algorithm development transparency have been proposed that highlight the importance of cohort characteristics and reproducibility to target populations, 66,67 to the best of our knowledge, no consensus standards or guidelines specifically exist for the publication of skin cancer image datasets that have the potential to be used in machine learning. Nevertheless, it is encouraging to note that organisations, such as ISIC, are exploring this area through formation of a metadata and DICOM working group, and have recently published a DICOM supplement for dermoscopy images. 68 In our opinion, the utility of atlases for clinical machine learning algorithm development might be limited, as we identified only five atlases that included systematic metadata (excluding the three atlases that contained site metadata only in the webpage name).
Relevant details from captions are also difficult to extract in an automated fashion.
To improve algorithm applicability, algorithms should be developed using metadata-rich datasets that accurately describe the skin image populations, with clearly stated inclusion and exclusion criteria. Ideally, datasets should be representative of and reflect the intended population where the algorithm will be deployed, to maximise generalisable performance. 69 Retrospectively collected images are frequently highly selected and might not represent the patient population to which they would be applied. In our review, inclusion and exclusion criteria were reported in only 57% of datasets, and the majority of datasets did not explicitly state obtaining ethical approval (52%) or individual patient consent (67%) for dataset publication. For these datasets, it was not known whether consent was waived, not sought, or not reported. Publication of images without consent can have implications for preservation of privacy. Nevertheless, for retrospectively collected images, consent might not Review specifically be required if images were anonymised, for example through exclusion of identifying features. 42,44,47 However, selection bias can occur where specific groups are excluded (eg, facial images or identifiable tattoos). Systematic prospective data collection with associated metadata might be required to reduce selection bias and best represent the cohorts from which the data are acquired. 70 Encouragingly, more recently developed datasets are heterogenous, containing images from various sources, including those not used in the initial training set. 26,42 Nevertheless, merging of individual datasets into larger, composite datasets requires accurate reporting of characteristics and metadata; without this, there is a risk of bias being magnified. Datasets containing paired images from different modalities better resemble clinical practice and could lead to more complex machine learning algorithms that can assess images of the same lesion from multiple modalities, in addition to demographic and clinical metadata. 38 In our review, datasets containing dermoscopic images only or macroscopic images only were the commonest, accounting for 19 (91%) of 21 datasets and 13 (81%) of 17 atlases. Only one dataset and three atlases contained paired macroscopic and dermoscopic images, and one dataset contained paired light field and dermoscopic images. We did not identify any datasets for other non-invasive imaging techniques to diagnose skin cancer, such as reflectance confocal microscopy, or optical coherence tomography. Light field image sets present interesting opportunities to characterise lesions in three dimensions (and at different depths of field), 48 although specialist equipment such as plenoptic cameras and training might be required for departments acquiring these images.
Health data poverty, defined as "the inability for individuals, groups, or populations to benefit from a discovery or innovation due to a scarcity of data that are adequately representative" is an emerging problem in digital health. 14, 36 We found unequal geographical distribution of datasets and atlas origin, with 11 (79%) of 14 datasets and 15 (88%) of 17 atlases originating from Europe, Oceania, and North America exclusively. Only one dataset originated from Asia, two from South America, and none from Africa. In contrast to a recent review of publicly available ophthalmological image datasets, where a large proportion of datasets originated from China, 14 we identified none in this review, highlighting that different specialties might have disease-specific biases.
Fitzpatrick skin type was not only poorly reported but also very poorly representative. Of the 2436 images from three datasets where skin type information was available (either as an aggregate or for individual images), 15,45,48 only ten images were from individuals with Fitzpatrick skin type V, and only a single image was from an individual with Fitzpatrick skin type VI. The ethnicity of these individuals was either Brazilian or unknown. Of the two datasets containing data on ethnicity (1585 images in total), 45,47 no images were from individuals with an African, Afro-Caribbean, or South Asian background. Coupled with the geographical origins of datasets, there was massive under-representation of skin lesion images from darker skinned populations. This is also reflected in dermatology educational resources 71 and images of COVID-19-related rashes published in the literature. 72 This bias is especially concerning as prevalence, presentation, and types of skin cancer vary in skin of colour populations, with associated poorer outcomes, 73,74 and would be even more alarming when detecting skin cancers that preferentially affect patients of skin of colour (eg, Kaposi sarcoma). 75 Geographical and ethnic disparities in digital health data have similarly been identified in other areas of digital health care where certain populations are underrepresented by the amount of digital health data available. 14,36,63 Biased datasets can lead to digital solutions benefitting populations that have advanced data infrastructure that facilitates data collection, but can result in the exclusion or even harm of data-poor subpopulations. Skin cancer machine learning algorithms should be developed using inclusive and representative populations to reduce algorithmic biases. Strategies proposed to achieve this include increasing awareness of health data poverty within the machine learning and digital health com munities, improving dataset transparency, investment into routine prospective collection of digital data from health-care systems, and transparent, effective communication to increase inclusion of all population groups. 36 Our systematic review had several limitations. Although our search strategy incorporated searches from multiple sources, our search was limited to MEDLINE, Google, and Google Dataset Search, and our search terms were in English only. Additionally, seven non-English full-text articles were reviewed using Google Translate. Recent publications that were published but not yet PubMed indexed might have been missed. Similarly, our search was not specifically designed to find atlases, and therefore some atlases might not have been identified. Furthermore, due to variability within and between atlases that were included in this review, it was difficult to quantify their characteristics to the same degree of detail as the datasets. We undertook reasonable efforts to download and import atlas images into our platform, but for some atlases this was not feasible, and we did not pursue this further as we would argue that these atlases might have limited usability given the challenges in downloading and importing images. We attempted to reduce the number of duplicate images in our analyses by excluding datasets for which all images were used in other datasets, and by removing images with the same identifier within the platform. However, it is possible there were some images that overlapped between datasets. Furthermore, we could not assess if there were multiple images taken of the same lesion (eg, from different views). The number of unique lesions, together with the number of subjects, could be helpful items to include in future open access datasets. Finally, reviews of this nature give a snapshot of the present situation, which can rapidly change without docu mentation of previous iterations. Datasets and atlases can be created, modified, or deactivated overnight, and in the future we expect more to emerge rapidly. Therefore, regular updates of systematic reviews and similar resources are needed to maintain usefulness to researchers and clinicians.
Future work could evaluate datasets containing images of all dermatoses, including those of rashes. 10 Factors such as positioning, rotation, colour balance, and even presence of surgical skin markings have all been shown to influence machine learning algorithm performance and any variability of these factors in datasets warrants further evaluation. [76][77][78] This has implications for the use of watermarked atlas images, which might require processing such as cropping prior to use. Furthermore, only 19% of open access datasets in our review stated whether image processing or adjustment by a reviewer occurred. Therefore, in addition to metadata standards, imaging standards are needed to ensure reproducibility of datasets and improve generalisability of the algorithms that they train. This is another area that is currently being explored by an ISIC working group, 11 along with use of DICOM. 79 To the best of our knowledge, this is the first systematic review of pubicly available skin lesion images comprising predominantly dermoscopic and macroscopic images available through open access datasets and atlases. Key characteristics and metadata were variably reported, with inadequate applicability of datasets to real-life clinical settings, and restricted geographical distribution of datasets globally, limiting generalisability. This review highlights the need for quality standards for minimum characteristics and metadata reporting for skin image datasets. Ensuring inclusion of representative population cohorts in skin image datasets might require prospective image collection using defined criteria. Health data poverty is an under-recognised but fundamental cause of the growing digital health divide and has been demonstrated to be a problem for skin image databases. Ensuring equitable digital health includes building unbiased, representative datasets to ensure that the algorithms that are created benefit people of all backgrounds and skin types.

Contributors
DW, RNM, XL, and SMK were responsible for the conceptualisation of this systematic review. RM and XL devised the methods. DW, AJX, HI, and SMK did the dataset searches. DW, SMK, LS, JC, LZ, and CdBP performed the data extraction. LS, JC, LZ, and CdBP imported and reviewed data using the software. AKD, XL, and RM supervised the project. DW and LS wrote the original draft. SMK, AJX, HI, AKD, XL, and RM reviewed and edited the manuscript. DW, SMK, and LS accessed and verified the underlying data. All authors had access to the data presented in this paper and the appendix prior to submission.

Declaration of interests
LS, JC, LZ, and CdBP were employed by Databiology (Oxford, UK) when this work was completed. All other authors declare no competing interests.

Data sharing
The review protocol is available upon reasonable request to the corresponding author.