{'fraction_non_alphanumeric': 0.030927463009743774, 'fraction_numerical': 0.010068567304222302, 'mean_word_length': 5.540240736370073, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Artificial Intelligence for Pediatric Ophthalmology

PURPOSE OF REVIEW: Despite the impressive results of recent artificial intelligence (AI) applications to general ophthalmology, comparatively less progress has been made toward solving problems in pediatric ophthalmology using similar techniques. This article discusses the unique needs of pediatric ophthalmology patients and how AI techniques can address these challenges, surveys recent applications of AI to pediatric ophthalmology, and discusses future directions in the field. RECENT FINDINGS: The most significant advances involve the automated detection of retinopathy of prematurity (ROP), yielding results that rival experts. Machine learning (ML) has also been successfully applied to the classification of pediatric cataracts, prediction of post-operative complications following cataract surgery, detection of strabismus and refractive error, prediction of future high myopia, and diagnosis of reading disability via eye tracking. In addition, ML techniques have been used for the study of visual development, vessel segmentation in pediatric fundus images, and ophthalmic image synthesis. SUMMARY: AI applications could significantly benefit clinical care for pediatric ophthalmology patients by optimizing disease detection and grading, broadening access to care, furthering scientific discovery, and improving clinical efficiency. These methods need to match or surpass physician performance in clinical trials before deployment with patients. Due to widespread use of closed-access data sets and software implementations, it is difficult to directly compare the performance of these approaches, and reproducibility is poor. Open-access data sets and software implementations could alleviate these issues, and encourage further AI applications to pediatric ophthalmology. KEYWORDS: pediatric ophthalmology, machine learning, artificial intelligence, deep learning


INTRODUCTION
The increased availability of ophthalmic data, coupled with advances in artificial intelligence (AI) and machine learning (ML), offer the potential to positively transform clinical practice. Recent applications of ML techniques to general ophthalmology have demonstrated the potential for automated disease diagnosis [1], automated prescreening of primary care patients for specialist referral [2], and scientific discovery [3], among others. Acting as a complement to ophthalmologists, these and future applications have the potential to optimize patient care, reduce costs and barriers to access, limit unnecessary referrals, permit objective monitoring, and enable early disease detection.
To date, most AI applications have focused on adult ophthalmic diseases, as discussed by several reviews [4][5][6][7][8][9][10][11]. Comparatively little progress has been made in applying AI and ML techniques to pedi-atric ophthalmology, despite the pressing need. In the United States, there is a shortage of pediatric ophthalmologists [12] and fellowship positions continue to go unfilled [13]. Globally, this shortage is even more pronounced and devastating-for example, retinopathy of prematurity (ROP), now in its third epidemic, has resulted in irreversible blindness in over 50,000 premature infants due to worldwide shortages of trained specialists and other barriers to adequate care [14,15].

KEY POINTS
• Pediatric ophthalmology has unique aspects that must be considered when designing AI applications, including disease prevalence, cause, presentation, diagnosis, and treatment, which differ from adults.
• Most recent AI applications focus on ROP or congenital cataracts, although many other areas of pediatric ophthalmology could benefit from AI.
• Reproducibility and comparability between current AI approaches is poor, and would be improved with open-access data sets and software implementations.
• Evaluation on experimental data sets should be augmented with clinical validation prior to deployment with patients.

UNIQUE CONSIDERATIONS FOR PEDIATRIC OPHTHALMOLOGY
Ophthalmic disease prevalence, cause, presentation, diagnosis, and treatment all differ between adult and pediatric patients-dissimilarities that are important to consider when developing AI applications. Common diseases in children include amblyopia, strabismus, nasolacrimal duct obstruction (NLDO), retinopathy of prematurity (ROP), and congenital eye diseases. The adult population, by contrast, is affected by cataracts, dry eye, macular degeneration, diabetic retinopathy, and glaucoma. For diseases that occur in both children and adults, the presentation, cause, and treatment often differ. Glaucoma is a good example, as the cause and presentation in congenital glaucoma patients are both unlike those in adultonset glaucoma patients. Optimal management of glaucoma, including surgery, also differs for these two populations.
Infants and children have distinct characteristics from adults that affect their ophthalmology visits. Given their developmental capabilities, there is generally less information gleaned from a single eye exam of a child, so several visits may be required to accurately diagnose or characterize that child's disease. There is also a stronger reliance on the objective exam because of the infant's or child's inability to effectively communicate. Children's short attention spans and unpredictable behavior often necessitate a quick exam that allows the physician to gain the child's trust while keeping him or her at ease. Despite this, there are portions of the clinic visit that take longer, such as restraining a child to administer dilating drops and then waiting for that child to be fully cyclopleged. Ancillary testing that requires patient cooperation may not be possible in an awake child, and eye exams under anesthesia are not uncommon. Similarly, children are typically placed under general anesthesia for eye procedures, whereas adults may require only topical or local anesthesia. Techniques for more accurate diagnosis and disease prediction could help reduce the high cost and risk of repeated exams and surgeries under anesthesia.
Other distinguishing factors pertain to the pediatric patient's growth and development. In most children, visual development occurs from birth until age 7 or 8; eye diseases affecting children during this period can cause permanent vision loss due to amblyopia or reduced visual abilities. Additionally, during development, significant ocular growth occurs, causing changes in refractive error that complicate surgical planning for congenital cataract patients.
Retinal imaging, too, differs for pediatric and adult patients. Factors such as children's lack of fixation and small pupils can create blur, partial occlusion, and illumination defects, all of which degrade image quality. For infants being screened for ROP, their fundus images are more variable and have more visible choroidal vessels, making classification comparatively difficult [16].

CLINICAL APPLICATIONS OF AI
This section surveys recent AI applications to pediatric ophthalmology, organized by disease (see Table 1). The approaches discussed in this survey would more precisely be called applications of MLthe largest subfield of AI concerned with learning models from data. We have provided a brief overview of AI and ML and their relationship in supplemental material, but the interested reader is encouraged to consult a more extensive tutorial on these topics [e.g. 5]. To limit its scope, this review focuses on applications with a goal of having the AI aspects directly impact clinical practice; we omit studies where ML was used primarily for statistical analysis.

Retinopathy of Prematurity (ROP)
The most significant AI advances in pediatric ophthalmology apply to ROP, a leading cause of childhood blindness worldwide [14,15,40] [44] and enable neonatology-led screening programs [45].
Early computational approaches to detecting plus disease from fundus images focused on vessel tortuosity. One early attempt to objectively quantify tortuosity used the spatial frequency of manual vessel tracings [46]. Since then, there have been several tools developed to determine vessel tortuosity and width via classic image analysis, including Vessel Finder [47] Like many ML methods, these systems can provide a confidence score in their predictions. i-ROP-DL exploits this notion directly by combining the prediction probabilities via a linear formula to compute approach, while the other two deep learning methods classify single images. The other key difference is that these systems are trained on different non-public ROP data sets of varying sizes and labelings (Table 2). The use of non-public data sets and closed implementations (only DeepROP is open source) complicates comparison and reproducibility [59].
Current methods for ROP detection are capable of coarse-grained classification, such as discriminating severe from mild ROP; they do not specifically assess disease stage or zone (e.g., [17 ]). In fact, all systems except DeepROP [17 ] and MiGraph [19] examine only the posterior pole view, either ignoring other views or explicitly cropping them out. While the literature suggests that severe disease rarely develops without changes in posterior pole vasculature [60], providing additional outputs of the zone and stage could improve the interpretability of the system's assessment and improve performance.

Pediatric Cataracts
Pediatric cataracts are more variable than adult cataracts, and surgical removal depends upon cataract severity and deprivational amblyopia risk. Slit lamp exams enable cataract visualization but can be challenging and subjective, and slit lamp image quality can vary (e.g., based on the child's cooperativeness, image amplification, and interference from eyelashes and other eye disease or structures) [32 ].
CC-Cruiser [30-32 ] is a cloud-based platform that can automatically detect cataracts from slitlamp images, grade them, and recommend treatment. After automatically cropping the slit-lamp image to the lens region, it uses three separate CNNs (modified AlexNets [61]) to predict three aspects: cataract presence, grading (opacity area, density, location), and treatment recommendation (surgery or non-surgical follow-up). CC-Cruiser was evaluated in a multicenter randomized controlled trial within five ophthalmology clinics, demonstrating significantly lower performance in diagnosing cataracts (87.4%) and recommending treatment (70.8%) than experts (99.1% and 96.7%, respectively), but achieving high patient satisfaction for its rapid evaluation [31 ].
Children who require surgery face potential complications that differ from those that adults face [62]. Zhang et al. applied random forests and naïve Bayes classifiers to predict two common post-operative complications, central lens regrowth and high intraocular pressure (IOP), from a patient's demographic information and cataract severity evaluation [28]. Another approach [29] uses a CNN to detect severe posterior capsular opacification warranting surgery, employing a ResNet [63] pretrained on ImageNet with a costsensitive loss to handle data set imbalance.

Strabismus
Strabismus affects 1 in 50 children and can cause amblyopia, interfere with binocularity, and have lasting psychosocial effects [64][65][66][67][68]. A CNN was used to detect strabismus based on visual manifestation in the eye regions of facial photos [33 ], which would be especially useful for telemedical evaluation. For inoffice evaluation, which in contrast permits the use of specialized screening instruments, strabismus can be detected using a CNN based on fixation deviations from eye-tracking data [34], or with very high sensitivity and specificity from retinal birefringence scanning [35].

Vision Screening
Like strabismus, refractive error can cause amblyopia, but is difficult for pediatricians to detect. Instrument-based vision screening is recommended [69] and most devices have adjustable thresholds for signaling a screening failure. Using video frames from one such instrument that combines Brückner pupil red reflex imaging and eccentric photorefraction, Van Eenwyk et al. trained a variety of ML classifiers to detect amblyogenic risk factors in young children, with the most successful being a C4.5 decision tree [70].

Reading Disability
Reading disability affects approximately 10% of children [38], but objective and efficient testing for it is lacking [37]. Abnormal eye tracking is non-causally associated with reading disability [37,38]. Two studies used SVMs to identify reading disability from eye movements during reading, either predicting reading disability risk in children ages 8-9 [37], or detecting reading disability in adults and children ages 11+ [38]. The children in both of these studies are older than the optimal age for diagnosis, so validation in a younger cohort could be useful.

Refractive Error
High myopia is associated with numerous visionthreatening complications [71]. Children at risk for high myopia can take low-dose atropine to halt or slow myopic progression [72,73]

OTHER OPHTHALMIC APPLICATIONS
This section reviews applications of ML to pediatric ophthalmology that are not tied to specific diagnoses.

Visual Development
ML has the potential to provide scientific insight into visual development. For example, adults who had cataract surgery and aphakic correction in infancy have exhibited diminished facial processing capabilities [101,102]. This impairment was originally blamed on early visual deprivation [101,102], but more recently, it was conjectured to be caused by the aphakic correction and high initial acuity experienced by these infants [103 ]. The hypothesis is that many visual proficiencies, such as facial recognition, are facilitated by the gradual increase in visual acuity during normal visual development. When tested in CNNs via initial training with blurred images, gradual acuity development increased generalization performance and encouraged the development of receptive fields with a broader spatial extent [103 ]. These results provide a possible explanation for the decreased visual proficiencies of congenital cataract patients, and suggest the potential for temporary refractive undercorrection to help restore visual development [103 ].

Pediatric Retinal Vessel Segmentation
Although many programs have been developed for vessel segmentation in adults or premature infants, fundus images in older children have unique traits, including light artifacts, that complicate segmentation [104]. Fraz et al. [104] developed an ensemble of bagged decision trees that use multi-scale analysis with multiple filter types to do vessel segmentation in pediatric fundus images. Another tool, CAIAR [25], has been validated in school-aged children [105]. CA-IAR was first applied to infants with ROP and uses a generative model of the vessels fit via maximum likelihood to a multi-scale representation of the retinal image [25].

Ophthalmic Image Synthesis
Through their multi-layered representation, deep learning methods such as generative adversarial networks [106] are able to synthesize novel realistic images, including retinal fundus images [107,108]. Such synthesized images can compensate for data scarcity, preserve patient privacy, and depict variations on or combinations of diseases for resident education [109,110].
One recent technique to synthesize highresolution images, progressive growing of GANs (PGGANs), was used to synthesize realistic fundus images of ROP (see examples in Figure 1) [111 ]. The PGGAN was trained on ROP fundus images in combination with vessel segmentation maps obtained from a pre-trained U-net CNN [58]. GANs have also been used to synthesize retinal images of diabetic retinopathy, including the ability to control high-level aspects of the presentation [77,112]. While many of the GAN-synthesized images display believable pathologic features, some do contain "checkerboard" and other generation artifacts.  [111 ]. The top row shows real images that were not included in the training set, and the bottom row shows the most similar synthesized images. (Image from [111 ], reused with permission.)

CURRENT LIMITATIONS AND FUTURE DIRECTIONS
Current applications to pediatric ophthalmology have several limitations that offer avenues for future work.
Disagreement on reference standards An ML classifier's performance is fundamentally limited by the quality of the training data, which are manually labeled by clinicians. However, there is often significant variation of the diagnosis and treatment among physicians, given the same case information [23, 42, 43,113], which complicates determination of the correct labels. When ML was used to identify factors influencing ROP experts' decisions for plus disease diagnosis, the most important features were venous tortuosity and vascular branching [23,43], neither of which are part of the standard "plus disease" definition of arteriolar tortuosity and venular dilatation [114,115]. Most approaches use the majority label from multiple experts as the label for each training instance, or combine the majority label given to imagery with the clinical diagnosis [116]. An alternative approach puts cases with any amount of disagreement up for adjudication among the experts, resulting in a consensus label and reducing errors, as demonstrated for diabetic retinopathy [76].
Need for pediatric-specific models It would be advantageous for pediatric ophthalmology to benefit from the large amount of work in AI for adult ophthalmology. However, due to the unique aspects of pediatric disease manifestation, ML models trained on adult patients may make errors when directly applied to pediatric patients. Transfer learning [56,57] and multi-task learning [117,118] techniques may offer a solution to this problem, providing mechanisms to adapt adult models to pediatric patients given a small amount of pediatric ophthalmic data. These methods could also reuse knowledge across models of different diseases or populations-for example, integrating knowledge across multiple smaller pediatric data sets of different ophthalmic diseases to help compensate for the lack of data on any one disease. Notice that, by pretraining on ImageNet, many of the CNN-based methods surveyed here already employ transfer learning of basic image features to compensate for using small data sets; transferring from adult ophthalmic data sets may provide further advantages.
Poor reproducibility and comparability Almost all the ML studies discussed here, even those that focus on the same disease, are trained and evaluated on different data sets. In many cases, the data sets and software source code are not available publicly, complicating reproducibility and scientific comparison across algorithms [59].
Most ML research relies on publicly accessible data sets and software implementations for evaluation and comparison. One simple way to encourage further applications of AI to pediatric ophthalmology is through the public release of data sets in strict compliance with HIPAA regulations, and with special regard to the additional HIPAA restrictions for minors. Even small pediatric ophthalmic data sets could be of use when used in combination with adult data through transfer learning techniques, as mentioned above. For the largest impact, these open data sets should be hosted in a widely used ML repository.
Lack of temporal information Most of these systems detect disease based upon one snapshot in time, without consideration of longitudinal imaging of the case [16]. In some diseases, such as ROP, rapid change is associated with poorer outcomes [47,119], suggesting that temporal information may have a role in predicting severe disease.
Uninterpretable "black-box" models Despite their predictive power, the "black-box" nature of most state-of-the-art ML methods, such as deep neural networks, complicates their application in medicine. It is often challenging to quantitatively interpret the inference process of such models, understanding how they arrived at their predictions [120,121]. Since they focus on correlations between the input and desired output, in some cases ML models may fixate on confounding factors instead of pathological information [122]. Interpretable ML methods provide a potential solution to benefit clinicians, allowing, for example, examination of intermediate decision steps within a deep network, natural language justifications for a decision, or visualization of image features that contribute to a decision [121]. While these methods seek to improve the interpretability of black-box models, other approaches seek to improve the predictive power of models that are already interpretable, such as the MediBoost algorithm for growing decision trees via gradient boosting [123].

CONCLUSION
There is a large potential for current and future AI applications to pediatric ophthalmology, and there are some diseases, such as NLDO, congenital glaucoma, and congenital ptosis, without any published applications of AI to our knowledge. Automated disease detection, the most common use case, could augment telemedical efforts to broaden access to care, improve efficiency, and result in earlier diagnoses. However, other less-utilized capabilities of this technology, including disease grading and outcome prediction, have the potential to enhance clinical care. All AI methods deployed in clinical care must ultimately match or surpass physician performance while meeting the unique requirements of both clinicians and pediatric patients, suggesting the need to augment evaluations on experimental data sets with clinical trials.

Financial support and sponsorship
E.E.'s work was partially supported by the Lifelong Learning Machines program from DARPA/MTO under grant #FA8750-18-2-0117. The funders had no role in the research presented in this article, nor in its preparation, review, or approval. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA or the U.S. Government.

Conflicts of interest
There are no conflicts of interest.

ONLINE SUPPLEMENT: A BRIEF OVERVIEW OF AI AND ML
Artificial intelligence (AI) is the broad field concerned with the study of intelligence and its computational manifestation within machines. It spans a broad set of problems that are all interrelated, from basic search (e.g., route finding on a map, or sequences of moves in a chess game) to logical reasoning (e.g., theorem proving, logistics planning) to reasoning under uncertainty (e.g., Bayesian abductive reasoning) to multi-agent systems (e.g., markets of trading agents) to robotics (e.g., computer vision and perception, control of dynamical systems) to learning. Machine learning (ML) is perhaps the largest subfield of AI and is concerned with this latter problem of learning from experience (i.e., data). Most recent news headlines and research concerning the application of AI techniques to problems in other disciplines (including the title of this article) would more precisely be termed applications of ML. Modern statistical ML is primarily concerned with the optimization of a model (e.g., a classification or regression model) to fit a given set of training data in such a manner that the model will be able to generalize to new data.
As a simple example, the training data might consist of demographical, biometric, and imaging data of a chosen cohort of 10,000 patients gathered from hospital records. Each record (called a data instance) could be characterized as a set of categorical, ordinal, and numeric features that are derived from the patient's record, and would be labeled according to the patient's diagnosis. ML algorithms could then train a classifier model (e.g., a decision tree, logistic regression) to predict the labeled diagnosis of a patient given the set of features derived from their record. Critically, the performance of the classifier should be assessed on new patients from the same population (i.e., patients with similar demographics that were not present in the training data), using application-dependent metrics (such as accuracy, sensitivity/specificity, receiver operating characteristic (ROC) curves).
This example focused on a supervised learning setting, in which each patient's data instance had a corresponding categorical label and we trained a classification model. If the labels had instead been numeric values, we could have trained a regression model using other supervised learning algorithms. Other settings include semi-supervised learning, in which only some data instances are labeled; unsupervised learning, which focuses on discovering patterns in unlabeled data (e.g, clusters of patients with similar biomarkers), and reinforcement learning, which seeks to learn a policy that can determine sequences of actions to execute to achieve a goal (e.g., the sequence of treatments to administer to an ICU patient, or the movements a robot should perform to tie a ligature). There are numerous different ML techniques, which vary according to the model representation (e.g., decision trees, linear classifiers, neural networks, logical rules), the mathematical technique used to optimize the model (e.g., greedy heuristics, gradient descent, evolutionary computation), and the evaluation metric used to assess the quality of model fit to the data (e.g., accuracy, precision and recall, posterior probability). Note that these metrics focus on performance on data and do not necessarily relate to the model acquiring generalizable knowledge. Consequently, ML models are learning patterns of correlations between the inputs and the desired outputs, not causal knowledge. This may cause them to exploit confounding details instead of physiological aspects. For example, an image classifier tasked with predicting disease severity might erroneously focus on identifying the type of camera (portable vs. fixed) or the presence of chest tubes or other medical devices, rather than pathological information, simply because these other confounding details are highly correlated with the desired output [122].
Deep learning (DL) methods are one subgroup of ML techniques that have shown exceptional impact to a wide variety of applications. Although DL techniques have been studied for decades, recent advances in computational algorithms and hardware have enabled these models to be trained at scale on large data sets, leading to their impact. DL is concerned with training models with numerous layers of processing, such as deep neural networks. Convolutional neural networks (CNNs) are one popular type of deep network that are often used for image classification. These models take raw input, such as a fundus photograph, and extract layered features from the input image, where higher levels of the deep neural network typically focus on increasingly abstract features that are built upon lower-level features. This automatic discovery of features is called representation learning, since the model identifies commonalities within the given input data as a way to re-represent it at different levels of abstraction. Although fundamentally an unsupervised learning technique, deep learning models can easily be adapted for classification, regression, and reinforcement learning. Despite its success and popularity, DL typically requires large data sets for training (e.g., thousands or hundreds of thousands or millions of examples, depending on the complexity of the decision), which may be problematic in certain medical applications.