{'fraction_non_alphanumeric': 0.024362078599366736, 'fraction_numerical': 0.01426708884336003, 'mean_word_length': 4.959156492785794, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Expanding the role of medical retina virtual clinics using multimodal ultra-widefield and optical coherence tomography imaging

Background An increasingly elderly population with a corresponding increase in ophthalmic conditions has led to increased pressure on hospital eye services (HES). In this study, we evaluated the use of a medical retina virtual clinic (MRVC), which has expanded into assessing all new medical retina referrals, where the need for urgent treatment was not clear. Methods Retrospective analysis of all new patients who were seen in the MRVC between April 2016 and May 2018. Pro forma sheets were used in the MRVC to record the patient history, visual acuity, and type of imaging required. Two consultants reviewed the completed pro formas and images and provided a final diagnosis and management plan. These results and reasons for face-to-face (F2F) clinic appointment requests were analyzed. Results Six hundred ten new referrals were enrolled in the virtual clinic. The most common diagnosis was diabetic eye disease (59.9%). In the virtual clinic 44.1% were followed up, 28.1% were discharged, and 27.8% were booked an F2F clinic appointment (urgent/routine). The main reason for F2F clinic was to offer treatment. Urgent F2F appointments took place on average 11.9 days after virtual clinic attendance. In only two cases was the image quality felt to be inadequate to assess the retina. Conclusions MRVC is an effective way of triaging medical retina referrals to allow those patients needing treatment to be seen promptly in the medical retinal service. The use of multimodal ultra-widefield and optical coherence tomography imaging allows assessment of a wide range of retinal pathologies and is a promising solution to alleviate the burden on HES.


Introduction
Many ophthalmic conditions occur more often with increasing age. According to figures produced by the UK Office for National Statistics, the population over 85 years of age will be 2.5 times larger by 2035. 1 Age-related macular degeneration (AMD) case numbers are predicted to rise by nearly 60% from 2015 to 2035 and likewise diabetic retinopathy. 2 This is leading to increased pressure on ophthalmic health care provision. In 2016, the Royal College of Ophthalmologists (RCOphth) estimated that at least 20 patients per month suffer severe visual loss due to delays in hospital assessment. 3 Since the introduction of the Diabetic Retinopathy Screening Service (DRSS), the incidence of blindness in the UK has fallen and diabetic retinopathy is no longer the number one cause of blindness in the working-age population in England. 4,5 Screen-positive patients, who have suspected maculopathy, proliferative diabetic retinopathy, have a high risk of disease progression, or where there is concern that there may be pathology outside the two standard 45 degree images taken, are referred to the hospital eye services (HES). However, studies have shown that only about 10% of patients referred for diabetic retinopathy actually require treatment. 6 A national study had shown that about 75% of new referrals to the HES from the DRSS were for maculopathy. 7 Pathologic markers on color images such as exudate are used as surrogates for underlying edema. These patients will then be referred into the HES for a specialist clinic appointment to assess for the presence of edema, specifically if there is clinically significant macular edema (CSME) for which treatment will be recommended. 8 For this purpose, optical coherence tomography (OCT) has been shown to be more accurate than clinical examination in identifying and quantifying any underlying edema. 9 In one study, it was found that three quarters of such referrals did not require hospital treatment and were discharged back to the screening programme. 2 Since about half of all new DRSS referrals to the HES could be considered as false positives, one could argue that demand reduction measures should be undertaken to target this situation. In addition to DRSS referrals, in the UK, most patients are referred to the HES after a visit to their optometrist, who are increasingly acquiring new technology such as OCT and detecting more retinal changes that may or may not actually need treatment.
The Way Forward Project was a national project supported by the RCOphth to identify solutions to cope with the increasing demands that the HES are facing. One of the suggestions it proposed was the use of virtual clinics. 2 There are two types of virtual clinics: a) synchronous, where it involves real-time web-based consultation and b) asynchronous, where attendance and consultations occur at different times. 10 The latter is virtual in the sense that a clinician reviews a patient's data (visual acuity, OCT, and color images) instead of the conventional clinic appointment where a history is taken and ocular examination performed.
Throughout the country, asynchronous virtual ophthalmology clinics have been running, especially for glaucoma disease monitoring but also more recently for medical retina diseases, for example AMD. In a survey of 27 eye departments in the UK, 17 were found to be running virtual AMD clinics of various formats. Studies have shown that virtual clinics in the HES setting have increased clinic capacity, streamlined referral rates, and also improved the patients' health care experience. [10][11][12][13][14][15][16][17][18] Imaging availability and choice have an integral role to play in determining the success of virtual medical retinal clinics. Not having access to good imaging services or not selecting the appropriate imaging studies may lead to inability to visualize the pathology accurately and subsequently the need to request face-to-face (F2F) appointments to assess the patient, as discussed in a recent study by Kortuem et al. 19 Several studies have shown that ultra-widefield (UWF) imaging has a better sensitivity and specificity in identifying and accurately staging retinal pathologies, such as diabetic retinopathy, compared to standard two-field imaging. [20][21][22] It also has the advantages of being less time consuming and produces less ungradable images. 21,23 The latest version, the California (Optos PLC, Dunfermline, United Kingdom), has a better range of focus and a higher resolution image, so usually one image per eye is sufficient compared to three images with the older P200 system. Additional imaging modalities are also available such as multicolor, autofluorescence, and OCT angiography (OCTA) with OCT and UWF autofluorescence with UWF color. To maximize the scope of patients that can be seen in a retinal imaging virtual clinic safely and efficiently, there is a need to adopt an individualized imaging approach when assessing each referral.
In the Eye Department of the Newcastle Hospitals NHS Foundation Trust, we have expanded the use of virtual

2339
Expanding the role of retinal virtual clinics with widefield and OCT imaging imaging clinics for retinal pathology, as the quality and speed of retinal imaging have developed, initially using such clinics for diabetic retinopathy and subsequently expanding into choroidal nevus, sickle cell retinopathy, and now for all medical retinal pathologies.
In this study we evaluated the use of our virtual retinal clinics over the last 2 years. We assessed the number of patients where we could not make a diagnosis; the number of patients whose imaging was inadequate to make a diagnosis; the number of patients needed to be brought back for treatment or were brought back for follow-up with an F2F appointment; the number that could be followed up in the virtual clinic; and the number of patients who were discharged from the HES. The impact of the virtual clinic on waiting times for clinic appointments was also reviewed as was the time taken to review the images.

Materials and methods
Pro forma sheets ( Figure 1) that had been designed to collect information at the time of imaging from new patients seen analyzed. Data were collected per patient and where two eyes could be involved, such as in diabetic retinopathy, only the more severe diagnosis or diabetic grade was recorded as the diagnostic outcome. All data were reviewed manually and anonymized data recorded into an Excel database from completed pro forma records and also outbound patient letters using the Hospital e-Records system. The project was registered as a service review on the Newcastle Upon Tyne Hospitals Foundation Trust clinical effectiveness register. This report is an audit of the outcomes of this service. Ethics approval and patient consent were not required for such an audit. It was carried out in line with the requirements of the Declaration of Helsinki.
Established in April 2016, the initial virtual clinic was just for patients with diabetic eye disease and it developed slowly. We then incorporated choroidal nevus and a sickle cell retinopathy assessment into this virtual clinic. Subsequently, the virtual clinic has been expanded to triage all new referrals of macular pathology where there was uncertainty of the need for treatment, based on reviewing the referral letter. A total of 610 patients were assessed in this two-year period.
Initially three UWF color images (Optomap P200; Optos PLC, Dunfermline, UK), using eye steering, and an OCT image were obtained per eye. Choroidal nevus assessment involved acquiring UWF color images and OCT, if the nevus was assessable by OCT, whereas sickle cell retinopathy assessment involved taking UWF color images. With the introduction of the Optos California, only one image was acquired per eye. In addition, the Spectralis OCT2 was used (Heidelberg Engineering, Heidelberg, Germany), which has a quicker acquisition time than the standard Spectralis OCT, and is capable of obtaining an OCTA image. The combination of Optos California and OCT2 increased the speed and accuracy of imaging, which led to rapid expansion of the virtual clinic service.
Patients were referred by the DRSS, optometrists, and general practitioners. The referral letters were assessed by a consultant ophthalmologist as suitable for the virtual clinic if they appeared to have a retinal problem but it was uncertain if treatment would be required. If a patient was referred with documented retinal hemorrhage, due to macular degeneration, or diabetic new vessels for example, then a direct clinic appointment where treatment could be provided was given.
If a patient was deemed suitable for these imaging clinics, the suspected diagnosis would be entered on the virtual clinic pro forma and the tests to be performed by the clinic staff on attending the clinic specified. This included visual acuity measurement, Tonopen measured intraocular pressure, the need to dilate, and the specific type of imaging requested by the consultant ophthalmologist. This could include UWF color (Optos), autofluorescence, OCT, OCTA, and multicolor. For the Optos and OCT, dilation may not be needed. For multicolor and autofluorescence, the results are better if the pupils are dilated. The tests were performed by nurses and ophthalmic photographers. In addition, a few specified questions were asked, in particular to clarify if patient was symptomatic. An explanatory letter was sent to patients prior to their attendance of the nature of the clinic, in that they would get imaging but not be seen by a doctor, and be informed by letter shortly after of the outcome in terms of diagnosis and further management.
Virtual clinic appointments have now been set up 5 days a week, morning and afternoon, as no specific clinic space or specialist needs to be available. All the nurses or optometrists who measure the vision have been trained to see such patients and photography is available every day. This gives greater flexibility for patients attending.
The referral letter, completed pro forma, and images were reviewed by two consultant ophthalmologists and the results entered on an electronic medical record system (Medisoft Ophthalmology, Medisoft Limited, Leeds, UK). All reviews were planned to be completed within 5 days of the virtual clinic and a diagnosis and a management plan made. For diabetic eye disease the retinopathy and maculopathy were graded according to the national guidelines. 24 Retinopathy was graded as none (R0), mild nonproliferative (R1), moderate or severe nonproliferative (R2), and proliferative (R3), which is further divided into stable (R3S) or active (R3A) and maculopathy graded as absent (M0), present (M1), and also specifically clinically significant maculopathy, if present. A letter containing the diagnosis and management plan was sent to the patient and their original referrer. Management plans included the need for: • Urgent F2F appointment for probable treatment.
• Follow-up in the virtual clinic.
• Discharge/discharge to diabetic screening.
The reasons for urgent and routine F2F appointments were further explored.

Results
During the two-year study period, there were 610 new referrals that were enrolled in the medical retina virtual clinic, specifically 340 for diabetic eye disease, 80 for choroidal

2341
Expanding the role of retinal virtual clinics with widefield and OCT imaging nevus, 25 for sickle cell retinopathy, and 165 for other nondiabetic macular pathologies. Table 1 illustrates the patient demographic details and the breakdown of number of clinic appointments into four major pathology categories. The patient did not attend (DNA) appointment rates was overall 13.8% (n=84) across the whole medical retina virtual clinic services. Hence, the total number of virtual clinic appointments that took place and therefore reviewed was 526. The mean processing time of clinical pro forma from patient attendance at the virtual clinic to review outcome was 3.6 days (range 0-5 days). On average, the records and images of ten virtual clinic patients could be processed in an hour by the consultant ophthalmologists reviewing the images. In a normal F2F clinic, five patients would be seen in an hour.

Diagnoses of patients assessed in the virtual clinics
Diabetic eye disease accounted for 59.9% (n=315) of all patients, of which 7.0% (n=22) had proliferative diabetic retinopathy (R3 equivalent), and 50.5% (n=159) had maculopathy (M1 equivalent). Of the 158 patients who were diagnosed with M1 grade, 34.0% (n=54) had CSME, which might benefit from treatment. Two patients, having been referred as R1M1, had new vessels (R3) detected on the UWF image. The other common diagnoses included choroidal nevus, AMD, central serous retinopathy, and retinal vein occlusion in decreasing frequency behind diabetic eye disease. Choroidal nevus was diagnosed in 88.8% (n=71) of the 80 patients assessed for this pathology and sickle cell retinopathy was diagnosed in 47.1% (n=8) of the 17 patients who attended the virtual clinic for assessment of this pathology (4 DNA appointments). Table 2 illustrates the detailed breakdown of diagnoses of all the patients reviewed in the virtual clinics.

Type of imaging performed
All patients who were reviewed in the virtual clinic for sickle cell retinopathy and choroidal nevus had Optos imaging, 12.5% (n=10) of patients in the latter group also had OCT. All the patients (n=315) who were reviewed for diabetic eye disease had OCT and 87.0% (n=274) of them also had Optos imaging. Finally, in the group of patients who were reviewed for other nondiabetic macular pathology (n=114), all of them had OCT and 36.8% (n=42) of them also had Optos imaging. Overall, 20 patients (3.8%) and 29 patients (5.5%) also had autofluorescence imaging and multicolor imaging, respectively.

Outcomes of all medical retina virtual clinic appointments
Of the total 526 patients who attended the virtual clinic, 44.1% (n=232) could be followed up in the virtual clinic, 27.8% (n=146) were offered an F2F clinic appointment, and 28.1% (n=148) resulted in patient discharged from the HES. The outcomes were further categorized according to the pathology, as shown in Table 3.
Of the patients who had diabetic eye disease (n=315), 28.0% (n=88) were invited for an F2F clinic appointment, 50.8% (n=160) were offered a follow-up in the virtual clinic, and 21.2% (n=67) were discharged back to the DRSS.
In the group of patients who were seen for macular pathologies other than diabetic retinopathy in the virtual clinic (n=114), 41.2% (n=47) were scheduled an F2F clinic appointment, 43.9% (n=50) discharged, and 14.9% (n=17) could be followed up in the same virtual clinic.
Of the 80 patients who were assessed for choroidal nevus, 47.5% (n=38) were to be followed up in the virtual clinic, 13.7% (n=11) were scheduled an F2F clinic appointment, and 38.8% (n=31) were discharged. None of the patients seen for sickle cell retinopathy (n=17) required an F2F clinic appointment, instead all of them will continue to be reviewed in the virtual clinic annually. Table 4 serves to illustrate the reasons why the patients were offered an F2F appointment both urgently or routinely. By far, the most common reason was that the clinician deemed that treatment (urgent or nonurgent) might be beneficial to the patient and so an F2F discussion was recommended. Other less common reasons included patients needing further investigation or they were referred to another hospital specialist for further evaluation or treatment, such as for cataract assessment or management of an epiretinal membrane. department by screening and further stratifying a large cohort of patients with low-risk medical retina referrals, so patients who actually need treatment are seen without delay, either as they get booked directly with a short wait, or are seen in the virtual clinic first and then can get treatment soon after. We believe that we have been able to demonstrate this result in our study where we were able to see most of the patients who were found to need treatment in less than 2 weeks after their virtual clinic appointment, which occurred within a month of referral. Before starting the virtual clinic such patients were waiting 4 months and in some it was found that their vision had reduced significantly as a result of not starting treatment promptly. Many referrals to the HES were from the DRSS with either R1M1 or R2M1, who would not need treatment unless in fact there was vascular proliferation (R3) or CSME. Another large group was patients who had had a routine optician check and some distortion was found on testing with an Amsler grid or pathology was noted on a color photograph or OCT but the patient was not symptomatic. Therefore, when an urgent macular assessment was requested by the optometrist, it not only caused patient anxiety, but also capacity problems and delays in patients who could have otherwise benefited from treatment. On top of that, we discovered that two-thirds of the patients who were referred to the HES for a medical retina consultation neither required an F2F clinic appointment nor treatment and almost half of them could just be followed up in the virtual clinic routinely either annually or biannually.
Being able to streamline referrals and generating greater clinic capacity do come with some tradeoffs. One of the known limitations of a virtual clinic is that occasionally, a clinical diagnosis may not be reached and the patient has to be recalled for an F2F clinic appointment, resulting in duplication of work and consuming more resources. This could be due to a few factors, namely not performing a full history and examination, inadequate image quality, and also instances where clinical symptoms or the recorded level of vision were unexplained by seemingly normal imaging On further analysis of the patients who were offered an urgent F2F appointment, all of them were seen within a month with the mean time being 11.9 days after the virtual clinic review. Sixteen patients received intravitreal anti-vascular endothelial growth factor injections and three received retinal laser treatment. Prior to the formation of the virtual clinic, the waiting time to be seen for an apparently nonurgent referral had reached 4 months. The waiting time for a virtual clinic appointment is currently 1 month.

Discussion
The ultimate aim of this new virtual clinic service is to free up capacity in the medical retinal services in the eye

2343
Expanding the role of retinal virtual clinics with widefield and OCT imaging findings. In our study, we did not encounter any significant issue with this, perhaps because of the type of imaging we used, which was with UWF color imaging (Optos California) and OCT, with the addition of autofluorescence and multicolor in some cases. 20 Studies directly comparing diagnosis from Optomap UWF imaging and OCT with slit-lamp biomicroscopy have demonstrated a good agreement. 9,25,26 Indeed it has been shown that more pathology may be found with this imaging approach than on clinical examination. 26 In addition, we did record if the patient was symptomatic at the time of image acquisition. We also had the original referral letter that included some information on the patient. Some virtual clinics for diabetic maculopathy are just using OCT, which is the best way to assess for CSME, but this does not allow a full diabetic retinal assessment and in a few cases will lead to new vessels being missed. 21 In this study, two patients had new vessels detected having been referred as R1M1.
The ungradable rate was very low compared to many imaging studies. We think this was for two reasons. First, the patients had already been assessed either by photographic diabetic screening or by an optometrist, so it was clear from the referral information that imaging should be possible. Second, we found that the combination of OCT and Optos color imaging was very effective at providing enough information on a retina to make a diagnosis even if there was some opacity in the visual axis.
A limitation of a virtual clinic is that an opportunity for patient education is lost, for example for those with early AMD or some diabetic changes. We propose to address this by designing information leaflets that could be added to the letters for patients.
Continued audit of the clinic is required and monitoring of capacity as a proportion of patients are now coming back for follow-up in the virtual clinic. Quality control systems need to be established such as double reporting of some of the cases to look for agreement.
There is an argument about the cost effectiveness of this virtual clinic model, which was not fully explored as part of the scope of our study. For example, it may put more pressure on the photography department and nursing staff; however, we could see twice as many patients in the virtual clinic compared to a F2F clinic in the same time. A recent systematic review provides supportive evidence in favor of this type of virtual clinic. 27 It could be argued that rather than investing in virtual clinics it would be better to employ more doctors and expand the clinical service. Unfortunately, at this time in the UK, we have found that is not possible due to work force planning issues meaning there are not enough qualified doctors who could do this work as well as there being budgetary constraints. Lack of physical space is also an issue in our department, as it is in many, meaning that extra clinics are not easy to schedule. In our unit we already had the imaging equipment described but if this was bought especially for such clinics then that would add a significant start-up cost to the service. In a previous paper we have shown that in fact reviewing wide field and OCT imaging can be more accurate than clinical assessment for detecting diabetic retinopathy so such equipment can improve both a F2F clinical service as well as allow quality virtual clinics. 26 Developments in imaging such as the combined UWF color/ OCT camera, the Monaco, from Optos, may help to streamline the imaging process further.
Virtual clinics for some aspects of ophthalmology have been used for some time such as for glaucoma and diabetic retinopathy, but few have taken the next step of using imaging to assess all cases, especially cases where the referral is unclear if treatment is required. One report has used a similar approach for medical retina referrals using a combination of OCT, Optos, and standard color images and found a similar beneficial effect for finding patients who actually needed treatment but had more ungradable cases and requirements for F2F consultations due to a less targeted imaging regime. 19 An important consideration for any health care service is the patients' experience, satisfaction, and specifically how we address their expectation of this innovative service. Although patient perspectives were not covered as part of the scope of this initial study, a survey study is currently being devised to prospectively assess this important factor and also to identify solutions to improve the patient DNA appointment rates. Reassuringly, a few studies have shown that the virtual clinic concept was generally well received by patients and has received good feedback nationally. 17,28 Currently, two experienced consultant ophthalmologists are assessing the virtual clinic appointment results of 20-30 patients every week and are using several different individual software systems that run simultaneously to view different imaging results (OCT, Optos) and input results into the patient's hospital electronic medical record. Going forward, a purpose-built unifying software could be designed to increase the efficiency of data review. The development of a network of imaging centers to enable centralized review would be able to increase patient access and our area of coverage. Additional staff could be trained to support image review and automated image reading software may become available.
For overall patient safety it should be remembered that in the UK diabetics are asked to attend annual photographic screening and most patients attend an optometrist for annual eye checks so discharging a patient from a virtual clinic or indeed any HES clinic does not usually mean that a patient will not have further checks of their eyes and so can be referred back if necessary.
In this service evaluation study, we have demonstrated the expansion of the roles of a medical retina virtual clinic to cover more retinal pathologies with the help of the latest imaging facilities. It also allowed us to generate more F2F clinic capacity to ensure that patients who need urgent treatment or evaluation get seen quickly. Further modification and optimization of this new service will enable us to determine the overall best model of care to allow us to develop the highest quality of care that we can offer to our patients within our increasingly limited resources.