{'fraction_non_alphanumeric': 0.026474983986904847, 'fraction_numerical': 0.006191730125969682, 'mean_word_length': 5.324032403240324, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploring Real-World mHealth Use for Diabetes Consultations: Pros and Pitfalls of a Pragmatic Mixed-Methods Approach

. Intervention research is often highly controlled and does not reflect real- world situations. More pragmatic approaches, albeit less controllable and more challenging, offer the opportunity of identifying unexpected factors and connections. As the introduction of mHealth into formal diabetes care settings is relatively new and less often explored from the perspectives of patients and providers together, such an opportunity for exploration should be embraced. In this paper we demonstrate our experiences and results in designing and administering a pragmatic mixed-methods feasibility study to understand the impacts of a diabetes data-sharing system on patients and providers. In doing so, we aim to provide a realistic account of the pros and pitfalls of this approach to diabetes mHealth intervention research.


Introduction
Historically, research has been divided between positivist (quantitative) and constructivist (qualitative) approaches to inquiry. As a result, quantitative results are often left without much context and qualitative results present limited generalizability and replicability. Further, most scientific inquiries rely on controllable and measurable environments, which often do not resemble real-world situations; this limits our ability to provide realistic expectations of how interventions would function. Given the largely unknown nature of the rapid developing mobile health (mHealth) tools for diabetes and increased sharing of patient-gathered data, a marriage between these two perspectives should be used to explore both known and emergent factors.
Pragmatism aims to comprehensively understand a real-world situation [1]. The introduction of pragmatism turns the quantitative-qualitative dichotomy into a continuum. Because it often employs mixed methods approaches, pragmatic studies rely on the strengths of both methodologies, i.e. a reliable understanding of what has changed during the study, and an understanding of how and why these changes happened, respectively.
As a relatively new concept in the history of health research, pragmatism and mixedmethods are used less often to understand the impact of diabetes interventions, let alone those that employ mHealth technologies. Patients and providers who actively engage in diabetes self-management and care can be seen as pragmatists themselves; they are aware of and often use the most appropriate and beneficial tools, approaches, supporting factors to improve and address the ever-changing aspects of a patient's diabetes. Therefore, characteristics of pragmatism parallel the purpose, values and needs of mHealth mixed methods research and diabetes care for patients and healthcare providers (HCPs).
In this paper, we present our administrative decisions and outcomes of completing a 6-month pragmatic mixed-methods study of a diabetes data-sharing intervention for patients and providers as part of the "Full Flow of Health Data Between Patients and Health Care Systems" project [2].

Methods
HCPs involved in diabetes care were identified within our research network and contacted via phone and email. Interested offices were given a 30 minute introduction to the study over lunch, followed by a 1-hour training session about the developed datasharing system. Those who agreed to participate were also asked to identify eligible patients based on HCP's knowledge of who uses, and perception of who may be interested in using, the technology and provide recruitment material to eligible patients. Participation required patients to be 18+years, with T1D or T2D, living within the Troms/Finnmark areas of Norway, and willing to download the intervention app.
The diabetes app allowed patients to gather diabetes self-management data via their mobile phone. They were given a user guide and link to a corresponding website about the app's available functionalities. Patients could then choose which data to share, via the system, and discuss with their HCPs during diabetes consultation(s).
Methods and measures chosen were based on input from patient and HCP participants in previous co-design and development studies within this project, to ensure relevance of findings to end-users [3,4]. Data collected from patients included: automatically gathered usage-logs and patient-gathered health data on the app, 1 st questionnaire set (demographics, standardized questionnaires: WHO-5 Well-being Index, Diabetes Empowerment Scale (DES), and Healthcare climate questionnaire (HCCQ)), 2 nd study-end questionnaire set (aforementioned standardized questionnaires, the System Usability Scale (SUS) and questions about their experience and opinions of the system). To ease reporting, outcomes were gathered using a link delivered via a message in patients' app and email, and usage-log data was remotely and automatically captured. Study-end focus group meetings allowed patient and HCP participants to share their experiences and perceptions of the intervention. HCPs were asked to complete a prestudy survey about their first impressions of the system, post consultation surveys about their use of the system, as well as provide HbA1c and blood pressure for each patient participant. Triangulation and complementarity analysis will be performed on all data gathered from participants. These will be published separately.
The use of an in-house developed study-administration platform allowed the research team to remotely collect data and manage patient participation [5]. Patients were encouraged, but not required, to schedule a consultation and answer questionnaires. Reminders, support and encouragement were offered via follow-up messages each month to each participant (max. 3). Participants were not excluded based upon study engagement or "adherence". No incentives were offered besides compensating study-end focus group meeting participants for their time taken off of work and travel costs. More details are provided in the published protocol article [6]. To assess the quality of our study design's and administration's adherence to the definition of pragmatic studies-"to evaluate the effectiveness of interventions in real-life routine practice conditions" [7]-the research team used a tool developed by Loudon et al., PRECIS-2 [8]. The study was submitted to the Regional ethical committee and the response from the chairperson May Britt Rossvoll was found to be exempt (2018/719/REK nord).

Results
This study provided two levels of results: outcomes of the pragmatic study design and results of data gathered from intervention participants. Here, we will focus on the former. Level of pragmatism achieved in study design Using Loudon et al.'s PRECIS-2 Score [9], based on 9 domains, from 0 (explanatory, positivist and evidence driven) to 5 (pragmatic and reflective of real-world situations), we classified the domains as presented in Table 1. Scoring was performed and discussed between two of the authors, and while explanations are not required by this assessment, they have been added for clarity and replicability.

Outcomes of pragmatic study administration efforts
In Table 2 we describe the outcomes of our administrative activities and decisions. These activities, including design and administration, were completed by two researchers, with

Domain
Score Explanation Eligibility 4 HCPs have to be selective in who they recommend to use self-management aids, i.e. who they believe would be willing, with benefits, and who would use it correctly), which was reflected in our eligibility requirements. However, HCPs usually only suggest tested and validated technology.

Recruitme nt 3
Recruitment used existing workflows, i.e. did not require much more from an HCP than normal. However, patients were still required to finalize enrollment via our online study-admin system (used internally only) and download the app (if not previously downloaded).

Setting 4
Intervention technologies were meant to be used by participants in typical selfmanagement and consultation settings. However, the data sharing system was an added technology.

Organizati on 3
While use of the technologies was left to the participants' discretion, supported by user guides or other resources, patients did not have previous exposure to the data sharing system and HCP's are given longer presentations of only validated technologies by vendor companies.

Flexibility: delivery 3
In typical practice, HCP's would be able to access and input their notes on patients at any time, and not limited to a system that deleted the data once it was closed.

Flexibility: adherence 3
The intensity of follow-up message schedules is beyond the typical support by medical staff for self-management or health support. However no additional incentives were offered compared to normal care.

Follow-up 3
The intensity of patient follow-up, is beyond current expectations of most HCP's and staff workload, without which, use of the system would require additional training of patients, and would require hiring additional and technologically trained personnel.

Primary outcome 4
Methods and measures used to collect study results were based upon similar participants stated priorities and needs from previous studies.

Primary analysis 5
Triangulation will allow us to include all data; we consider present and missing data as equally valid indications of the participant's experience/ engagement in the trial. the exception of data extraction performed by one programmer. These results indicate the success and challenges of our pragmatic decisions.

Discussion
Traditionally, health intervention research studies are more structured and controlled, using qualitative and/or quantitative methods. However, these often do not reflect realworld situations and therefore have limited applications. A pragmatic mixed-method approach is beneficial to mHealth and informatics because it allows researchers to gather more comprehensive and diverse data from different perspectives that our relevant to the end-users that would not be possible with only one methodology or focus. Therefore, the outcomes of a pragmatic study reveal end-users' needs in real-world situations, which Domain Results of pragmatic administrative activities Eligibility x n=26 HCP offices identified and contacted (using approx. 10mins to prepare and place the call, on average once per month (for an average of 2.6 months (min=1, Max=10). x n=15 offices did not respond to any contact attempts)

Recruitment
x n=11 offices attended the research teams project presentation x n=6 offices agreed to participate and recruit patients x Unknown number of patients identified by HCPs (few offices reported provided recruitment information to 10-50 patients, but the exact number is unknown) x n=8 patients chose to enrol in the study (n=4 T1D, n=4 T2D)

Setting
x n=8 patients used the app in their real world self-management activities x n=3 HCPs were able to use the intervention in consultations with n=5 patients

Organization
x n=1 researcher sent out follow-up messages x n=1 researcher responded to patient questions and support needs x n=1 programmer resolved technical errors (one technical error for data gathering took 1 month to correct, and n=2 patient's data had to be downloaded after the intervention due to inability to automatically download their data remotely) x No questions, technical errors or needed support were reported about the system

Flexibility: delivery
x Participants used the technologies uniquely for each case

Flexibility: adherence
x n=8 patients answered the first questionnaire set, and provided usage logs x n=6 participants additionally completed the second questionnaire set and n=3 HCPs answers post consultation questionnaires after intervention testing x n=4 patients (n=1 T1D, n=3 T2D) additionally participated in a focus group meeting x n=1 nurse participated in a focus group meeting (i.e. providing her experience with a similar commercial data sharing system) but was unable to recruit patients for intervention testing

Follow-up
x n=2 patients were sent two additional (repeated) follow-up messages x The researcher took approximately 4mins to access the online study administration platform, compare the timing to each participants schedule and send each message

Primary outcome
x *See "Flexibility: adherence" Primary analysis x Triangulation will use all data in the analysis and interpretation of the results, i.e. quantitative and qualitative data will be analysed separately and then compared to one another based on common themes can direct the future development of health technologies. And while pragmatic study administration is less controllable and does take longer, analysis efforts in particular allow for a more realistic and in-depth understanding of not only what but also how and why these technologies are used -despite relatively small results.
In research, we focus on the outcomes and do not often get the opportunity to report our efforts and decisions in running a study. We believe the success of even the simplest, smallest decisions, should be mentioned so that others can benefit from our experiences.

Conclusion
Every method, question, paradigm or research intention has its place in health research. Tools and services including patient efforts, becomes increasingly more complex as the technology enables more data to be gathered and more devices to manage these data with, e.g. smartphones and smartwatches. This calls for new ways of understanding what happens both within and outside of healthcare practice -since actions leading to health and wellness happen in several phases -before, during and after consultations. Separate qualitative and quantitative studies have their place in validating findings. However, we think feasibility studies can benefit from the pragmatism approach, which helps identify real-world factors, needs and situations that should be addressed in validation studies.