{'fraction_non_alphanumeric': 0.025437964962802977, 'fraction_numerical': 0.02501799856011519, 'mean_word_length': 5.01986276634164, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Clinical Telemedicine Utilization in Ontario over the Ontario Telemedicine Network

Abstract Introduction: Northern Ontario is a region in Canada with approximately 775,000 people in communities scattered across 803,000 km2. The Ontario Telemedicine Network (OTN) facilitates access to medical care in areas that are often underserved. We assessed how OTN utilization differed throughout the province. Materials and Methods: We used OTN medical service utilization data collected through the Ontario Health Insurance Plan and provided by the Ministry of Health and Long Term Care. Using census subdivisions grouped by Northern and Southern Ontario as well as urban and rural areas, we calculated utilization rates per fiscal year and total from 2008/2009 to 2013/2014. We also used billing codes to calculate utilization by therapeutic area of care. Results: There were 652,337 OTN patient visits in Ontario from 2008/2009 to 2013/2014. Median annual utilization rates per 1,000 people were higher in northern areas (rural, 52.0; urban, 32.1) than in southern areas (rural, 6.1; urban, 3.1). The majority of usage in Ontario was in mental health and addictions (61.8%). Utilization in other areas of care such as surgery, oncology, and internal medicine was highest in the rural north, whereas primary care use was highest in the urban south. Conclusions: Utilization was higher and therapeutic areas of care were more diverse in rural Northern Ontario than in other parts of the province. Utilization was also higher in urban Northern Ontario than in Southern Ontario. This suggests that telemedicine is being used to improve access to medical care services, especially in sparsely populated regions of the province.


Introduction
T elemedicine can improve access to health and medical care services, especially in sparsely populated and less developed regions, by bridging the geographic distance between patients and service providers. However, providing access does not mean that the services will be used, or used to capacity. 1 Our objective was to examine telemedicine use in the Province of Ontario, Canada, with a focus on traditionally underserved areas such as rural and Northern Ontario.
Northern Ontario is sparsely populated, with 88% of the province's area but only 6% of the population. 2 Approximately 34% of Northern Ontario's population resides in rural areas, compared with 11% of Ontario as a whole. 3 Although Southern Ontario's rural areas are connected to an extensive road network, many of the small communities in the north are accessible only by train, airplane, or winter (ice) road. This creates different contexts for healthcare service delivery in rural versus urban areas in different regions of the province.
Northern and rural regions of the province are often medically underserved. Although most people in Canada are within 5 km of a physician, some rural residents are more than 100 km from the closest physician. 4 Only 16% of Canadian family physicians and 2.4% of specialists were located in rural and small-town Canada, compared with 21.1% of the Canadian population. 4 In Ontario, only 6% of physicians practice in Northern Ontario, and 71% of these Northern Ontario physicians practice in urban areas. 5 The Ontario Telemedicine Network (OTN) is a not-for-profit organization funded by the Ontario Ministry of Health and Long Term Care (MOHLTC). OTN uses technology to connect patients and providers throughout Ontario. 6 OTN, the largest telemedicine service provider in Canada 7 and one of the largest in the world, 8 has 2,026 OTN sites across Ontario. Onequarter of OTN sites are in Northern Ontario (n = 552), and half of those (n = 286) are located in the rural parts of the north. 9 There is limited research on telemedicine use in Ontario. OTN usage rates in Ontario ranged from 9 to 22 sessions per 1,000 from 2010 to 2014. 7 Per capita rates for Ontario exceed all provinces except Newfoundland and Labrador and the Territories, areas that are sparsely populated. However, these studies do not differentiate between utilization in urban and rural parts of the province. In a study on the uptake of telemental health services in the province of Alberta, utilization rates were lowest in urban centers where residents had access to more healthcare services that were provided in-person. 11 Australia researchers report that 75% of telemental health patients reside in rural or remote locations. 12 In the United States, patients living further from the clinic were more likely to seek electronic visits for selected primary care services. 13 These studies demonstrate the uptake of telemedicine in rural areas.
Health service organizations in rural areas were more likely to use telemedicine than those in urban areas. A survey on telemedicine in Western Australia found a significant positive correlation between distance to Perth, the major urban center in the area, and the number of telehealth services received. 14 Telemedicine in Newfoundland and Labrador was developed exclusively for people in remote communities; in 2008, there were 3 specialist sites in St. John's (the capital of Newfoundland and Labrador) and 28 remote sites where patients could connect to specialists in St. John's. 15 OTN is an exemplar of how telemedicine can provide an extensive range of medical services over large geographic areas with services that are integrated and incentivized within existing physician payment models. In this article we use medical service billing data to compare the number and type of clinical telemedicine utilization in four regions of Ontario, categorized as urban Southern Ontario, rural Southern Ontario, urban Northern Ontario, and rural Northern Ontario.
This study provides insights into the uptake of virtual medical care in medically underserved communities and regions.

Materials and Methods
This research is part of a multiphase research program funded by the Ontario MOHLTC designed to better understand how OTN is being used in Ontario in order to improve access to healthcare services in underserved communities. Data from the MOHLTC were obtained under a data sharing agreement. Although the research described in this article was exempt, ethics approval was obtained from Laurentian University's Research Ethics Board for the broader research program.
We used OTN-flagged medical service utilization data collected through the Ontario Health Insurance Plan (OHIP) and shared by the MOHLTC. These data may slightly underestimate the total number of events, as billing records may not capture visits with physicians who use alternative payment programs or who are salaried. However, billing records do capture the vast majority of virtual care services as the majority of physicians are paid fee-for-service for telemedicine consultations. Records included patient and physician geographic location (residence code), month and year of visit, and type of medical visit (billing code) for the 2008/2009 to 2013/ 2014 fiscal years. We matched patients' residence codes to Statistics Canada's Census Subdivisions (CSDs) and calculated utilization per 1,000 people for each CSD by fiscal year using 2011 census population data. We grouped CSDs as Northern or Southern Ontario, using the boundaries of the Northeast and Northwest Local Health Integration Networks 16 and as urban or rural using Statistics Canada's Statistical Area Classification system. 17  In 2011, there were 574 CSDs in Ontario with 279 in Northern Ontario. The OHIP-derived utilization rates for CSDs classified as First Nations were very low, even though we knew that these sites were used frequently. 18 Aboriginal people who are registered under the Indian Act and receive care on Reserves have most of their medical service use billed to Health Canada (a federal agency) and are not included in provincial OHIP data. 19 CSDs classified as First Nations and those with a population of under 50 were removed (n = 148), leaving 426 CSDs for analysis.
Each OTN visit generated two or more distinct OHIP codes: at least one health service code along with one telemedicine premium code. We used telemedicine premium codes 20 to identify individual patient visits. Any OTN utilization that was not covered by OHIP is not included in this analysis.
Using OHIP's Schedule of Benefits for physicians, 21 we were able to organize the billing codes by therapeutic area of care. We used the most common areas according to OTN 22 reporting and grouped everything else as ''other.'' Addiction codes were categorized with Mental Health as per OTN reporting.

Results
Of scheduled OTN visits billed through OHIP, 89.1% (n = 652,337) were completed patient visits, 10.7% (n = 78,708) were appointments that were missed or canceled, and 0.2% (n = 1,343) were dropped due to technical difficulties. Patient location was missing in 32,514 visits (5.0% of total completed visits).
Just    We also calculated utilization rates for each CSD to provide a per capita measure of telemedicine use across regions with different population sizes (Fig. 2). Utilization rate for completed OHIP billable patient visits for CSDs in Ontario ranged from no visits to 484.5 visits per 1,000 people per year during 2008/2009 to 2013/2014. The median utilization rate for Ontario CSDs was 8.4, and the mean was 30.2 for the study period. The median rate for rural Northern Ontario was higher than for urban Northern Ontario (52.0 versus 32.1), and both areas were much higher than rates in Southern Ontario (3.1 for urban Southern Ontario and 6.1 for rural Southern Ontario) (Fig. 3). Figure 4 shows distinct differences among the four geographical regions by population size and utilization rate. Northern rural areas had higher utilization rates and smaller populations than Southern rural areas. Southern urban areas had higher populations and lower utilization rates.
Northern urban areas were spread among population sizes and utilization rates. The median utilization rate for Ontario CSDs increased from 1 visit per 1,000 people per year in 2008/2009 to 19.5 in 2013/2014; in Northern Ontario, median utilization rates increased from 39.5 to 96.9 over the same period.
In Ontario, 62% of telemedicine use is for mental health and addictions, and this varies from 42% in rural Northern Ontario to 70% in urban Southern Ontario (Fig. 5). Family and general medicine use is highest in urban areas, especially urban Southern Ontario. Proportionally, rural Northern Ontario has more surgical, oncology, and internal medicine use than the other geographical areas. Usage for the catchall category of ''other clinical services'' is higher in rural than urban CSDs (15% versus 7%), regardless of region.

Discussion
Our study found increasing OTN utilization in all regions of Ontario from 2008/2009 to 2013/2014 with the greatest absolute increase for patients living in  urban southern Ontario. Per capita rates also increased over time, with higher rates in Northern Ontario, particularly in rural northern CSDs. We recognize there is a lack of standard measure to calculate telemedicine activity, which makes comparison problematic. The mean utilization rate of 30 visits per 1,000 people is higher than previously reported rates for Ontario of 9 to 22 visits per 1,000 people from 2010 to 2014. 10 The main reason for the different rates is that the other study used data collected for scheduling purposes, whereas we used data collected for physician payment, and so there may be differences in how sessions are defined and which sessions are counted.
Concerns regarding the reliability of the technology have been cited as reasons why patients and physicians hesitate to use telemedicine. [23][24][25][26][27][28] OHIP data suggest that only 0.2% of visits are dropped due to technical difficulties. The rates of technical difficulties vary for different telemedicine providers and may improve over time. For instance, telemedicine providers have reported technical difficulties in as many as one in five visits in a review of 1,000 patient visits during the start-up of a new telemedicine program. 29 Another study in a pediatric cardiac unit reported no technical difficulties in a case series of 53 patients over 9 months. 30 Technical difficulties can be mediated by investing in appropriate technology and infrastructure, including training for the people who will be using the technology. [31][32][33] The rates of missed and canceled visits are often a problem in medical settings. For OTN-flagged OHIP billings, 10.7% of all scheduled visits were missed or canceled. This is comparable to other healthcare programs, in which missed visits range from 10% to 25%. [34][35][36] Telemedicine is often used to provide mental health services to patients, especially those residing in underserved rural and remote communities with limited access to inperson services. 11,12,37,38 Praxia 10 reported that 54% of utilization in Canada in 2010/2011 was for addictions and mental health. In our study, 62% of utilization was for mental health and addictions services. The proportion of telemedicine utilization in mental health and addictions is lower in rural Northern Ontario than in other parts of the province. This is not due to lower per capita utilization for mental health and addictions in the north, but due to a higher rate of OTN utilization for other clinical areas of care, such as oncology, surgery (consultation and follow-up), and internal medicine. We suggest that telemedicine helps compensate for the lack of medical specialists with practices in northern and rural areas, thus realizing monetary and environmental benefits associated with reduced patient or provider travel as well as the potential health benefits of increased access to medical care.
The primary limitation to this research is that data on OTN are collected for billing purposes and are not designed for research. A pertinent limitation is that OHIP data underestimate patient visits in First Nation communities. In addition, OHIP data count only patient-physician interactions and exclude physician-physician consultations or care management sessions. Nonmedical clinical use is also excluded. For example, OHIP data exclude visits with other healthcare professionals such as nurses, occupational therapists, or physical therapists and exclude family visits that connect patients with loved ones in distant parts of the province. However, with the exception of telemedical services provided to patients in First Nations communities that are billed to the federal government, physicians must use OTN for clinical services to be eligible for remuneration by the province, and most physicians bill OTN clinical sessions as fee-for-service, with the possible exception of some of those physicians who are salaried or participate in alternative physician compensation models. Therefore, OHIP billing is considered to accurately capture telemedical services in Ontario, with the aforementioned exception of First Nations communities. A geographic limitation is that several of the rural Northern Ontario CSDs span several hundred square kilometers. As such, aggregate measures may conceal important differences in access to care and utilization of OTN within these CSDs.

NEXT STEPS
In the previous stage of this research, we conducted a road network analysis to find the maximum potential access to OTN for residents in Northern Ontario. In this article, we calculated utilization rates in Ontario for rural and urban areas in Northern and Southern Ontario and identified use by broad type of therapeutic care. In the next stage of this research, we will assess factors that affect OTN uptake through further analysis of administrative data, augmented by interviews with key stakeholders in order to provide recommendations regarding the future of telemedicine in Ontario.

Conclusions
Utilization rates were higher in rural CSDs than in urban CSDs and higher in Northern Ontario than in Southern Ontario. Type of use was more evenly distributed in rural Northern Ontario CSDs than in other regions, although the category of mental health and addictions was still the most frequent type of use. While we do not know if overall access to care is increasing in Northern Ontario, access to care via OTN is certainly on the rise, and utilization rates are highest in traditionally underserved rural areas. Although this article focuses on utilization in Ontario, the success of OTN's model can be used elsewhere in the world to advocate for the development of virtual care for a wide variety of therapeutic areas of care, especially in rural, remote, and underserved areas.