{'fraction_non_alphanumeric': 0.028997305187514037, 'fraction_numerical': 0.01644958454974175, 'mean_word_length': 4.834425155584671, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 21, 'profanity_word_list.json': 0, 'sexual_word_list.json': 4, 'zh_pornsignals.json': 0}}
Usability and effectiveness of adherence monitoring of a mobile app designed to monitor and improve adherence to event-driven and daily HIV pre-exposure prophylaxis among men who have sex with men in Taiwan

Objective The UPrEPU mobile app is a self-monitoring system to enable men who have sex with men to optimize their pre-exposure prophylaxis adherence for HIV prevention. The app was designed to accommodate a rather complicated event-driven dosing schedule. We aim to evaluate the usability of the UPrEPU app and its effectiveness in improving adherence monitoring. Methods From May to October 2020, 35 participants were enrolled for the usability study and followed up for 4 months. Blood samples for the drug concentration in the dried blood spots were obtained once during the second to fourth follow-up visits. The effectiveness of adherence monitoring was analyzed using Cohen's kappa statistic to calculate the concordance between the average number of pills taken and drug concentration in the dried blood spots. Results Overall retention was 91.4% (32 participants) at the end of the study. Participants used the app for a mean of 29 days and made 2565 data entries in total, with an average of 76 data entries. The average systematic usability scale score for the app was 71.5, indicating acceptable usability. Slight agreement was reached between the dried blood spots measurement and the number of pills taken and recorded in the app (weighted kappa: 0.21). Conclusions Our user-centered UPrEPU app demonstrated that it could accommodate both daily and event-driven dosing schedules for men who have sex with men clients with acceptable usability scores. We confirmed that complex behaviors such as different drug-dosing regimens that are contingent on sexual behaviors could be incorporated into the design of a mobile app.


Introduction
Oral pre-exposure prophylaxis (PrEP) has been regarded as a highly effective and comprehensive HIV prevention strategy. The efficacy of PrEP could reach over 90% when high adherence was kept among men who have sex with men (MSM). [1][2][3] The event-driven (ED) PrEP dosing regimen -which involves taking two pills of tenofovir disoproxil fumarate/emtricitabine (TDF/FTC) between 2 and 24 h before sex, followed by two pills of TDF/FTC taken 24 and 48 h after the first pill intake-was an alternative to the daily dosing regimen recommended to MSM. 4 The ED PrEP dosing regimen among MSM has been gradually adopted in several countries, such as France, 1 the Netherlands, 5 Belgium, 6 and Taiwan. 7 A pre-coital dose of PrEP should be taken at least two hours before each sexual intercourse for an ED regimen, but users may not be able to correctly estimate the exact timing of a sexual event and thus miss the pills. 8 The post-coital dose can be easily forgotten as well. Studies have shown that 81.9% of ED PrEP users miss their pills before sex, and 4.3% miss pills after sex. 8 The effectiveness of PrEP correlates with the level of adherence. 2 Current studies use two major types to estimate PrEP adherence: self-reports and objective measurement of drug concentration. Self-reporting measures vary and often include indirect measurements such as pill counts in different time frames using different methods. A short message service is an approach that facilitates self-reporting because the participants receive messages to prompt them to report frequently, thus reducing recall bias. 9,10 Drug concentrations are often measured as a more objective way to estimate adherence and for comparison. 11 One study that used a sample of adolescent and young MSM from the United States compared four types of adherence measurement: self-reported adherence, smart pill cap openings, TFV concentrations in hair, and TFV-DP concentrations in dried blood spots (DBS). 12 Compared to the drug concentration methods, self-reported measures overestimated the adherence; smart bottle openings underestimated adherence because the bottles were not being used consistently. 12 For evaluating adherence, however, none of these measurements is perfect for ED regimens in which the timing of the sexual event should be considered. MSM PrEP users might adopt PrEP dosing regimens based on their preferences and lifestyles. [5][6][7]13 Switching between two dosing regimens is not uncommon in the real world. [5][6][7] While ED PrEP already has a complex dosing schedule, it would be more difficult to measure the real-world ED PrEP adherence with dosing regimen switching involved. Approaches to ensure and improve PrEP adherence are crucial to maximize the benefits of PrEP.
Mobile health interventions have shown promising feasibility and acceptability in improving medication adherence and preventive behavior in various medical conditions. 14,15 Moreover, mobile phone applications have been adopted as one of the mobile health technologies to improve access to HIV prevention services and adherence for antiretroviral therapy among people who live with HIV. Given the ubiquity of smartphones, mobile apps for MSM offer distinctive opportunities for public health interventions targeting this population and empowering them to self-care and to handle their healthcare. 16 Several studies have been evaluating smartphone and Internet-based interventions for HIV prevention and PrEP adherence. Most of the studies focus on MSM and aim to facilitate HIV testing or PrEP uptake by providing home delivery of prescriptions. 17 Only a few other studies have been focusing on self-monitoring of PrEP adherence among MSM; however, these studies have had limitations of daily PrEP use only and have lacked a user-centered practices design. [18][19][20] UPrEPU is a novel mobile application designed with a user-centered approach and was developed following the multiphase optimization strategy framework (comprising a preparation phase, an optimization phase, and an evaluation phase). [21][22][23] The UPrEPU app incorporates concepts of a participatory design approach, recognizes the needs and the direct input of the users, and took their needs and expectations into consideration in the design process. 24 This app was designed to assist PrEP clients, including those who used the ED dosing regimen, to better manage PrEP use by themselves. Our app allows flexibility for MSM's dynamic dosing choices and aims to improve PrEP adherence by sending reminders based on the current dosing regimen and users' sexual practices.
In this paper, we aim to describe the development of UPrEPU and evaluate the usability of this app. We further evaluated the effectiveness of adherence monitoring of the UPrEPU by comparing the pills taken and sexual intercourse recorded in the app with the drug concentration in DBS. We hypothesized that the UPrEPU app developed through a user-centered approach would be acceptable and easy to use. We expected moderate to high agreement between DBS measurement and the number of pills taken in the app among MSM PrEP users in Taiwan.

Development of the UPrEPU app
The design and development process of the UPrEPU app used a multidisciplinary and user-centered design approach. [25][26][27] Formative conceptualization qualitative interviews were conducted to understand the MSM PrEP clients' needs. PrEP clients' personas and journey maps were developed from the data collected in the interviews. One session of each participatory workshop was conducted with the personas and journey maps used to generate concept ideas and a low-fidelity prototype of the UPrEP app.
The participants for the participatory workshop included PrEP users, PrEP managers, researchers with expertise in human-computer interaction design or public health, and app developers. They were randomly assigned to two developing groups. A detailed description of the development process of the UPrEPU app was described in a previous report. 28 A low-fidelity paper prototype of UPrEPU was generated under an agreement of all participants in the workshop ( Table 1).
The research team refined a high-fidelity prototype afterward. The internal testers (research team members) accessed all features of the UPrEPU app to identify any remaining glitches, bugs, or usability concerns with the heuristic evaluation method. 29 Then, the official UPrEPU app was developed regarding the feedback from internal testers. The UPrEPU app was developed for IOS and Android operation systems. A list of final app features and descriptions is shown in Table 2.
Screenshots of the app are available in Multimedia Appendix 1. The highlighted features include visualized drug concentrations, a sex diary that allows users to create and record the details of their sexual events (including time for sex, prevention tools used in the sex, sex roles, and mood tracker), and a calendar. Participants also log their PrEP intake and choice of the dosing regimen. Based on the dosing regimen and the participants' scheduled dates with their sexual partners-a feature that is particularly important for an ED dosing regimen-the app shows reminders for pills needed to be taken in the following days on the calendar. The resources provided on the app include HIV and PrEP-related educational multimedia materials and a global positioning system (GPS)-enabled map of HIV testing locations. Data visualization in the app shows a graph integrating the mood for each sexual event and PrEP use. Users can see the pattern of their mood with different sex partners with or without PrEP protection, which is also one of the highlighted features.
Usability, feasibility, and effectiveness of adherence monitoring of the UPrEPU app Recruitment and enrollment. Participants were referred by physicians; PrEP navigators were recruited in the sexual health clinic from May to June 2020 and assessed for eligibility by an online screening tool. Eligibility requirements included the following: (1) Participants were men who were 20 years of age or older; (2) resided in Taiwan and able to understand, read, and speak Mandarin Chinese; (3) tested negative for HIV 3 months prior to enrollment for current PrEP users; (4) had laboratory results that indicated eligibility to initiate PrEP based on Taiwan's PrEP guidelines; 30 (5) currently took PrEP or were willing to initiate PrEP after enrollment; (6) reported having at least four episodes of anal intercourse with men in the previous 1 month; (7) owned an Android or Apple operating system (iOS) smartphone and were willing to download the study app; and (8) were willing to wear the Garmin wearable fitness tracker that the research team provided during the study period.
Eligible participants were asked to complete informed consent forms upon entering the study. Participants were then asked to download the UPrEPU app, and watch a short clip embedded within the app for feature demonstration. The participants were also required to answer the baseline questionnaire to finish their enrollment in the study. 31 Participants were followed up monthly for 4 months. Each monthly visit included rapid testing for HIV antigen and antibodies and participants completed a follow-up questionnaire on their mental health scales, sexual behaviors, and PrEP use. Depression and anxiety were assessed using the Table 1. Features generated from the workshops (low-fidelity paper prototype).

Planned features
Chatbot a to answer PrEP-related questions. Tailored reminder for PrEP uptake for daily/ED PrEP users Information for accessing HIV testing and PrEP.
Reward system to encourage adherence to PrEP.

Reminders Reminders and tailored feedback messages
Visualization of drug concentration. PrEP diary to record their daily PrEP use Social support through online platforms. Communication and social support through the private message function.
a Chatbot: A chatbot or chatterbot is a software app that simulates human-like conversations with users via text messages on chat. PrEP: pre-exposure prophylaxis.
Patient Health Questionnaire-9 (PHQ-9) 32 and General Anxiety Disorder-7 scale (GAD-7), 33 respectively. The System Usability Scale (SUS) for the UPrEPU app was assessed during the first follow-up visit. Blood samples for the TFV-DP and FTC-TP concentration in the DBS were obtained once during the second to fourth follow-up visits.
Outcome measurement: Usability, feasibility, and effectiveness of adherence monitoring usability We used the SUS, a reliable and low-cost usability scale, 34 as the primary outcome for usability. It was a 10-item, 5-point Likert scale yielding a global view of subjective assessment of usability. 34 In the SUS scale, even and odd questions were scored differently. Odd questions were scored 0-4 based on the 1-5 selection. Even questions were scored 4-0 on the 1-5 selection. Scores for all 10 questions were added up for a total score between 0 and 40 points. This total was multiplied by 2.5 to generate a SUS score between 0 and 100 points. 35 A product with SUS scores below 50 indicated certainty that it had usability difficulties in the field; scores in the 70 s and 80 s indicated that a product was acceptable. 36 We evaluated SUS of UPrEPU app in the month 1 visit. Participants were further asked to clarify their reasons for providing a neutral or negative response to the following question in the SUS scale: "I would not like to use this app frequently" and "app features integration had room for improvement." Table 2. Features of UPrEPU app.

Features Function
Customizable avatars • Visual representations of users and their sex partners within the app to facilitate the development of an online identity.

Dashboard
• The home screen where users can choose and switch their dosing regimens.
• Provides a visualization of the current adherence pattern.
• Shows the time and date of the last dose.
• Reminders on how many pills and when to take the next dose.

Calendar
• A calendar allows users to create, edit, and delete the PrEP and sexual event diary and HIV testing records. • Detailed information recorded for each sexual event, including the avatars, alias, sex position (bottom, top, or both), prevention strategies, and the mood at the moment of the sexual event. • Signal on the calendar to show whether users have taken PrEP correctly regarding the users' dosing regimens.

Data visualization of PrEP adherence and mood for sexual events
• Provides a 14-day summary of PrEP adherence and mood for each sexual event with sex partners.

Information modules
• Educational materials on HIV, PrEP, and other sexual health issues.
• Provides the nearest geographic location of HIV testing in Taiwan based on the users' location.

Notification
• Push notification text to remind app users to take PrEP, backfill their sex and PrEP record.

Feasibility
The primary feasibility outcomes were app engagement including the frequency of app logins, the use of app components such as PrEP-taking, and sexual behavior reports. The frequency of app logins and the use of app components such as anal intercourse and pill-taking records were logged and extracted from the app. 37 For each sexual event, participants also recorded the mood at the moment of sex on the Smiley Face Likert scale (awful, not very good, okay, really good, and fantastic). 38 Secondary feasibility outcomes of this app included a focus on and potential improvement of the app in the future. All participants were asked an open-ended question: "What features would you like to add to the app?"

Effectiveness of adherence
Adherence was estimated by the proportion of anal sex acts that used PrEP correctly. It was calculated as the proportion of "sex-days" (i.e. the number of days that anal intercourses occurred as the denominator) for which PrEP was taken correctly (numerator). Whether PrEP was taken correctly in each sexual event was determined by the algorithm in the app. An algorithm was developed regarding the ED and daily PrEP instruments as follows: Correct taking of ED PrEP was calculated as taking 2 pills 2-24 h prior to each sexual event, followed by 1 pill 24 h and 1 pill 48 h after the latest pill taken before sexual event. However, if at least one pill was taken 25-168 h (1-7 days) before each sexual event, taking only one pill 2-24 h before each sexual event was allowed. 30 Correct use of a daily dosing regimen was defined as at least 4 pills taken 168 h (7 days) before each sexual event.
The effectiveness of adherence monitoring of this app was estimated as the consistency of PrEP use between drug concentrations in the DBS sample and the pill-taking reports in the calendar component in the UPrEPU app. The DBS sample was obtained once after the first month follow-up. The procedure to measure tenofovir-diphosphate (TFV-DP) and emtricitabine-triphosphate (FTC-TP) in DBS has been described previously. 39 The lower limit of quantification was 25 fmol/sample for TFV-DP and 100 fmol/sample for FTC-TP. We used validated dosing cut-offs for TFV-DP in DBS to categorize participants' weekly pilltaking on average: 7 doses/week (>1450 TFV-DP fmol/ punch), 4-6 doses/week (800-1449 fmol/punch), 2-3 doses/ week (400-799 fmol/punch), and < 2 doses/week (below the limit quantification 350 fmol/punch). 40,41 The pill-taking record of app users in the 6 weeks before DBS collection was estimated on a weekly average and classified in line with the categories of TFV-DP. The DBS samples were stored at − 20°C within 24 h of collection and shipped on dry ice to the Skaggs School of Pharmacy and Pharmaceutical Sciences (University of Colorado Anschutz Medical Campus, Aurora, CO, USA) for analysis.

Analysis
The demographics of participants were described in numbers with percentages. The number of pills taken, the record of sexual events, mood, and PrEP protection in the app were calculated by each individual participant. App usage was described as the total of the average of data entry and days. Descriptive statistics were used to examine the SUS results with an aggregate score. Weighted Cohen's kappa coefficient analysis was used to examine the consistency between the drug concentration in the DBS samples and the self-reported PrEP diary in the app. The difference in drug concentration in

Study populations demographics
Thirty-five participants enrolled in this study from May to June 2020. The baseline characteristics of the participants are shown in Table 3. Most participants were 26-30 (16,48.6%) years of age; 14 (39%) earned less than USD $1000 per month; most participants used iPhones (24, 70.6%); and a substantial number of participants (9, 25.7%) reported they had been switching between daily and ED PrEP prior to study participation. No participants were diagnosed with HIV or syphilis during the follow-ups.
Thirty-two participants completed the 4-month followups (91.4%). Among them, 3 (8.6%) discontinued the study because of losing a device and refusing to wear the devices. Although not completing our 4-month follow-up, one person had DBS performed and examined, resulting in a sample size of 33 for DBS analysis (Figure 1).

Usability of the UPrEPU app
The results of the UPrEPU app's usability are shown in Table 4. Overall, participants found the app to be convenient to use and easy to use on their own. The average SUS score for the app was 71.5, which indicated that the UPrEPU app is acceptable and above average, with 18 participants (51.4%) who had scores of more than 70 in the SUS.
The main reasons for participants not using the app frequently included the following: infrequent sexual activities and ED PrEP-taking (n = 5, 35.7%), lack of motivations (n = 4, 28.6%), and few notifications from the app to remind users of opening the app (n = 4, 28.6%). Our participants also mentioned how the app could be improved: the data visualization for the relation between PrEP use, sex, and mood were difficult to understand (n = 6, 37.5%), and function in the calendar feature can be better integrated with other apps (n = 3, 18.8%), such as the Google calendar.

Feasibility
More than 95% (32) of the participants kept using the UPrEPU app during the study period and contributed an average of 641 data entries to the app each month (Figure 2). Participants used the app for a mean of 29 days (median: 24 days, interquartile range (IQR): 11-37 days) and made 2565 data entries in total, with an average of 76 data entries per day (median: 70, IQR: 47-106 data entries). Among the 32 participating users, data entry remained high for each follow-up (96.9%-100%).
The most used function was the PrEP intake record, with 1806 data of PrEP intake Participants recorded their mood as "really good" (264, 47.9%) and "fantastic" (181, 32.8%) in most of the sexual events (awful: 3, 0.5%; not very good: 8, 1.5%; okay: 95, 17.2%). There were 81.6% of sexual events with correct PrEP use rated at "really good" or above, and 75.3% of sexual events without correct PrEP use rated at "really good" or above.
The most wanted features for the UPrEPU app in the future was a calendar feature in the app to be integrated with their other calendar apps and keep the schedule consistent without repetitively typing content (22.9%). Some participants (8.6%) would like to have an additional note section to record their sexual events comprehensively. Given that the UPrEPU app has a lot of contexts related to sex and PrEP use, 11.4% of participants suggested that the UPrEPU app develop a social networking function to enable users to interact with each other or as an add-on function to integrate with common gay social apps, with a geosocial networking app for sexual encounters.

Effectiveness of adherence monitoring
The 33 DBS measurements from 33 participants were collected at the follow-up visits.
Overall, the median TFV-DP concentration was 821.8 (IQR: 411.9-984.8) fmol/punch, equivalent to an average of 4 doses taken per week. Twenty-three (66.7%) participants reported ED PrEP use and 10 (27.3%) reported daily PrEP use prior to the visit of DBS collection in the app. There was no significant difference in TFV-DP concentration between the ED and daily PrEP use; the medians of TFV-DP concentration were 821.8 (IQR: 439.1-961.2) and 984.6 (IQR: 763.9-1578.8), respectively (p-value: 0.145). One (3.0%) TFV-DP measurement was below the lower limit quantification (BLQ). FTC-TP concentrations were BLQ in 10 (30.3%) DBS measurements, meaning no dose intake within 48 h. Among these 10 DBS samples, 8 were from participants who reported ED PrEP use and 2 were from participants who reported daily PrEP use.
The median PrEP adherence in the app was 90%, with an IQR from 68.9% to 98.1%. Overall, there was a slight  Table 5. Comparing the self-report pills taken in the app and the estimated weekly average pills taken from dried blood spots (DBS).

Principal findings
We presented a novel technology to assist MSM PrEP users to self-monitor their PrEP adherence and allow flexibility for the user's choice of an ED dosing regimen in real-world settings, with a high proportion of ED PrEP use and regimen switching. We assessed the usability and effectiveness of this user-centered innovative app with features of sexual event and pill diary and adherence visualization to assist MSM PrEP users. First, users reported that the UPrEPU app had an acceptable and above average usability level and was easy to use on their own. Second, regarding feasibility, participants showed high engagement and provided high numbers of sex, PrEP, and mood records in the app. Third, when compared to objective measurement of drug concentration, only slight agreement was observed between the DBS measurements and the number of pills taken and recorded in the app. Our study suggested that it is feasible for MSM to use a mobile phone app to manage their PrEP use and that the UPrEPU app is an acceptable tool in terms of technology usability; however, the potential for reporting bias using the app still exists. The app, like other technology developed to improve healthcare delivery, might be most helpful when accompanying other support, such as communicating with healthcare providers and users' social network. SUS being one of the most popular indexes for usability provides us with some information regarding how our app was perceived by the users. Other studies that applied an automated directly observed therapy approach in their app to improve PrEP adherence for daily MSM users reported having a median SUS score of 85 at week 4 and 80 at week 8, 18 versus ours of 71.5 at month 1, a lower score but still in the acceptable and above average range. Other apps focusing on HIV prevention targeting different age ranges of MSM in the United States also have an above average SUS score, 42,43 showing an overall high acceptability across geographical regions and languages in applying technology in HIV prevention interventions, and have the potential to serve as a platform to apply artificial intelligence in the next step. 44 The app engagement remained high during the 4-month study period of our study. It is a key challenge for the majority of health-related apps since the level of engagement remains low or declines over time. [45][46][47] Several characteristics have been shown to enhance user engagement, including provision of personally tailored content, interactive components, social networking, and reminders. [48][49][50][51] Several common features were designed to improve adherence to PrEP, such as diary features with both a sexual activity log, medication log, and notifications. 17 The application designed for PrEP adherence improved from other studies with those common features experienced reducing user retention over time. 52 Apart from the shared common features, our app was designed to engage the users to selfmonitor PrEP use, which was one characteristic to enhance retention. 53 We added a mood log to show the correlation and trend in mental health of the user, with data visualization of the patterns of sexual activities, PrEP use, and mood. In the UPrPEU app, the avatar features provided the users with a virtual identity that enabled users to record their sexual events in a relatively safe environment and served as a personally tailored feature to enhance engagement. A data visualization of the PrEP protection level in the app also assisted users to understand their current adherence patterns and reduce their anxiety toward sexual activities. 54 Our study found a low agreement between the selfreported pill number collected by the app and the weekly pill number estimated by the DBS measurement. There was no significant difference between the proportions of overreported and underreported groups. Both overreporting and underreporting pill numbers compared to DBS measurement happened in our sample, showing that a bias in using selfreport data in an app to measure adherence can go either way. The low consistency between DBS and self-report in the app may be owing to both the incorrect self-report use in the app and noise in the DBS measurement. The underreported group may be individuals with low motivation for using the app to self-monitor their PrEP use. The main reasons for participants not using the app frequently included infrequent sexual activities and ED PrEP taking, low motivation, and not enough notifications from the app to remind the users to open the app. These reasons all led to not using our app and resulted in an underreported PrEP use compared to DBS measurement. On the other hand, overreporting individuals might be those influenced by social-desirability bias, which gives socially desirable responses instead of choosing responses that are reflective of their true state. 55 Additionally, our study sample was recruited in Taiwan and is different from the majority of study samples in studies that have assessed TFV-DP in DBS in relation to adherence to tenofovir alafenamide; such studies usually have limited race or ethnic representativeness. 40,41 Cautions are needed in terms of race/ethnicity and weight, which not only affect TFV-DP variation, 40,41,56 but also interact with the association between age and TFV-DP. 57 For example, African Americans had lower TFV-DP concentrations in DBS than White Americans which may be due to different genetic, diet, or environmental factors across races and ethnicities. 40 There were limited Asian MSM data for the estimated average of pills used by TFV-DP concentration obtained from DBS. 58 Future studies will be needed to establish the relationship between TFV-DP concentrations in DBS and adherence for Asian MSM besides samples from Thailand. 57,58 Objective measures, such as TFV-DP concentration in DBS, are generally assumed to be a more reliable measure of adherence. As demonstrated by the AMPrEP study, 52 the concept of adherence measurement for ED PrEP use is much more complex than daily PrEP use since that study involved the timing of sexual events and the number of pills taken. Moreover, objective measurement usually requires specialized equipment and trained technicians, which would be an obstacle for monitoring PrEP in the long term. Accurate self-report or technology-assist selfmonitoring of PrEP adherence should be developed and used parallel with other objective measurements.
A high percentage of sexual events were protected by PrEP in our study; however, a wide IQR suggested that there was high diversity between the participants. The protection during sexual events from ED PrEP varied in previous studies. A high proportion of correct PrEP use for high-risk sexual activities has been observed in the studies using an online diary as a data collection tool in the Netherlands and Belgium, and in the IPERGAY study in France. 59 Conversely, low adherence to ED PrEP was found among MSM and transgender women in Thailand and the United States in the HPTN 067/ADAPT study, and among MSM who sought sex partners via geosocial networking apps in China. [60][61][62] A high level of PrEP adherence in our study may be attributable to the flexibility of choosing PrEP dosing regimens and using an app. The possible explanations for the low adherence from the other two studies were that participants were assigned to one PrEP dosing regimen group without a cross-over in the HPTN 067/ADAPT study, and the participants had more frequent substance use in the Chinese study.
Conceptualization of PrEP adherence is shifting toward a prevention-effectiveness adherence (PEA) paradigm, 63 which emphasizes the consideration of other preventive measures such as condom use along with PrEP use when determining the level of protection from HIV infection. To assess PEA, comprehensive information on the timing of sexual activities, PrEP use, and the adoption of other HIV prevention strategies (e.g. condoms, serosorting, and undetectable viral load) is required. Studies have been unable to measure PEA thoroughly since such information has been hard to access without innovative electronic tools. 6,64 The UPrEPU app allows users to record their PrEP adherence patterns in the long-term and provides realtime PrEP protection levels with indications for use of PrEP for each sexual event, which enables users to better manage their risk of HIV.

Limitations
The present study included several limitations. First, we may have limited generalizability due to our strict inclusion and exclusion criteria. MSM recruited in this study had a relatively higher frequency of sexual behaviors compared to general MSM PrEP users in Taiwan. 8 Highly educated people were overrepresented in this study because these people could adopt innovative health technology more easily. The feasibility and usability data may not apply to those MSM who have much less frequent sex or to those who are less tech-savvy. Second, in terms of the small sample size in our study, the only inferential statistics in this paper is weighted kappa and the kappa can be calculated for fairly small sample sizes (e.g. five participants). 65 The results in our study still need to be interpreted with caution given the relatively small sample size. Third, one major strength of the app is that it was designed to integrate ED PrEP users' perspectives; however, routine daily PrEP users may be less motivated than ED PrEP users. Repeatedly logging daily pills may result in data entry fatigue. Fourth, we only measured DBS once in the study period. Multiple measurements may be needed to better examine the relationship between data of the number of pills used gathered from the app and the objective measurement of drug concentration.

Conclusion
This study demonstrated a user-centered designed app that accommodates both daily and ED PrEP use for MSM. We confirmed that complex behaviors such as different drugdosing regimens that are contingent on sexual behaviors could be incorporated into the design of a mobile app. Even though adherence monitoring in the app was less effective, the app engagement remained high during the 4-month study period, showing the potential for such an app to support MSM to improve PrEP adherence regardless of the dosing regimen. While COVID-19 impeded access to PrEP services and changed the MSM's sexual lifestyles, 13,66 the UPrEPU app can not only assist PrEP users to self-manage their PrEP adherence regardless of their dosing regimens but also reduce the burden for healthcare providers by offering users personalized reminders to take PrEP based on their sexual lives. This can prevent PrEP case managers from answering repetitive questions and messages to some degree. Even though the mood at sexual events was rated positive in our app, this may not reflect the overall mental health among participants.
Future work for the UPrEPU app can incorporate more focus on addressing potential syndemic problems that include mental health, sexual health, and substance use. For example, a just-in-time adaptive intervention on the mobile app that aimed to provide interventions to improve mental health and to monitor substance use for MSM may further include the PrEP component to be a more comprehensive harm reduction tool. 67 Only a few apps have been attempted, and even fewer have evaluated the feasibility and usability of the app. 68 Future work should also include strategies to increase interest in retention and adoption of the app. Economic evaluation of mHealth interventions are crucial to explore the sustainability and cost-effectiveness of mHealth interventions. 69,70 Future studies on economic evaluation of mHealth interventions are needed to optimize such technology tools for HIV prevention.
Acknowledgments: The authors acknowledge and thank all participants involved in the design, development, and pilot study of the UPrEPU app. The MOST and the Taiwan CDC had no role in the design of the study and collection, analysis, or interpretation of data or in writing the manuscript. Part of the results were presented as a poster at the 11th IAS conference on HIV science, Berlin, Germany (abstract number: A-IAS2021-00046).
Contributorship: CS, HW, and YY conceived, designed, and performed the study. HW and YY contributed to data handling and analysis. HW and CS wrote the manuscript. HW, SK, YT, CY, CL, PH, NK, PA, and CS contributed to manuscript revision. All authors reviewed and approved the final manuscript.
Declaration of conflicting interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.  Guarantor: CS.
Informed consent: All study participants provided written and verbal informed consent to participate in this study.