{'fraction_non_alphanumeric': 0.031179644025317532, 'fraction_numerical': 0.012340172465060957, 'mean_word_length': 5.189430787432628, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Digital Tracing during the COVID-19 Pandemic: User Appraisal, Emotion, and Continuance Intention

: This study explores how people appraise the use of contact tracing apps during the novel coronavirus (COVID-19) pandemic in South Korea. Despite increasing attention paid to digital tracing for health disasters, few studies have empirically examined user appraisal, emotion, and their continuance intention to use contact tracing apps for disaster management during an infectious disease outbreak. A mixed-method approach combining qualitative and quantitative inquiries was employed. In the qualitative study, by conducting interviews with 25 people who have used mobile apps for contact tracing, the way users appraise contact tracing apps for COVID-19 was explored. In the quantitative study, using data collected from 506 users of the apps, the interplay among cognitive appraisal (threats and opportunities) and its association with user emotion, and continuance intention was examined. The ﬁndings indicate that once users experience loss emotions, such as anger, frustration, and disgust, they are not willing to continue using the apps. App designers should consider providing technological affordances that enable users to have a sense of control over the technology so that they do not experience loss emotions. Public policymakers should also consider developing measures that can balance public health and personal privacy. be used for telemedicine, delivery, quarantine management, and decontamination. and feel that they have limited control over the expected consequences. Our qualitative study provides information on the factors that increase loss emotions: users are signiﬁcantly threatened by intrusions to their privacy.


Introduction
The novel coronavirus  outbreak that began in late 2019 has presented an unprecedented challenge to public health across the globe. COVID-19 is a viral respiratory disease caused by a coronavirus. It can cause a fever, cough, breathing difficulties, pneumonia, and even death, especially among older adults or people with underlying health conditions (e.g., chronic diseases and respiratory problems) [1]. As of 16 December 2020, the World Health Organization (WHO) reported that there were 71,581,532 confirmed cases of and 1,618,374 deaths from COVID-19 (https://COVID19.who.int/). Given the potential spread from asymptomatic patients, rapid testing and tracing for vulnerable populations is critical for the containment of the virus [2].
In South Korea, diverse mobile applications have been developed and are widely used by those who want to obtain information about the movement of infected people. To date, more than 10 mobile apps have been developed during the COVID-19 pandemic, and they provide information on infected people's movements along with demographic information (e.g., gender and age) and send notifications regarding new cases. By using the apps, people are able to quickly recognize whether they might have had any contact with an infected person, which helps them decide whether to take preventive actions (e.g., virus screening or self-isolation). The Corona 100 m (Co100) app is one example of mobile apps that have been widely adopted in South Korea. This application provides users with information about coronavirus-confirmed individuals' past movements and notifies them via an alarm when a location visited by a confirmed individual is within 100 m of the user. Although it has been reported that such contact tracing apps are beneficial for public health, the use of aggressive digital contact tracing based on global positioning system data from smartphones and credit cards has raised significant concerns regarding its potential to intrude on individuals' privacy.
Despite increasing attention paid to the use of digital tracing using smartphone apps for COVID-19 [3], little is known about how individuals evaluate the use of contact tracing apps and how their appraisals influence their emotions and continuance intentions with regard to the use of the apps. To fill these gaps in research, drawing on cognitive appraisal theory [4,5], this study seeks to answer the following questions: (1) What are the threats and opportunities users perceive while using contact tracing apps during the COVID-19 pandemic?; (2) How do the users react emotionally toward the technology; and (3) How do the users' appraisal and emotional reactions determine their continuance intention? The study contributes to the literature on public health disasters by: • contextualizing user appraisals of an information technology (IT) event (i.e., the use of contact tracing apps) in a pandemic situation; • verifying the role of emotions in determining their continuance intentions; and • providing insights into implementing a technology that is controversial in terms of the opportunities and threats to public health and individuals' civil liberties.

Information Technology for Disaster Management
The use of IT for disaster management is crucial because it enables the public to access timely and accurate information [6], thus helping control and prevent further damages and losses [2,7]. In particular, during an epidemic or pandemic, the use of IT that enables access to accurate information about the contagion [8] and the tracing of infected people to isolate them [9] are key to managing the health crisis [10]. For example, researchers found that during the severe acute respiratory syndrome-related coronavirus (SARS-CoV) epidemic in 2002-2003, geographic information systems that enabled the mapping of the spread of the disease and contact tracing were found to be effective in controlling the spread of the virus [11]. In contrast, some researchers have argued that digital tracing for COVID-19 has the potential to compromise individuals' civil liberties [3], and it may exacerbate individuals' incivility, aggravate social divide, and cause disruption [12]. While the role of IT is becoming increasingly important for public health disaster management [13,14], the question of how to balance public safety and individuals' liberties has yet to be fully discussed.
During the COVID-19 pandemic, mobile applications for contact tracing have played an important role in tracking and controlling the pandemic [15,16]. By using these mobile applications, an individual can be notified when he or she is close to a place that has been visited by a person with a confirmed infection, while if an individual gets infected, his or her mobile apps can be used to trace past movements. These mobile applications are developed by the current localization or communication technologies (i.e., cellular networks, Wi-Fi, GPS, and Bluetooth), which enable smartphones to constantly trace the user's location [16]. Currently, many countries have developed their mobile applications based on different technologies. For example, the "TraceTogether" in Singapore and the "COVIDSafe" in Australia rely on Bluetooth [17], whereas the "Kwarantanna domowa" in Poland and the "Live Corona Map" in South Korea use GPS [17,18]. These technologies show different abilities to track and trace. According to Hernández-Orallo et al. [16], most mobile applications rely on Bluetooth, but Bluetooth is only effective in detecting nearby devices of around 1-2 m away and that have the app installed. In contrast, although the precision of GPS can reach 10-15 m, the precision can be severely decreased when individuals are indoors [16]. Moreover, some people have concerns that the government would track them using GPS [17]. Therefore, it is important to take technology types (i.e., cellular networks, Wi-Fi, GPS, and Bluetooth) and human concerns into consideration when designing and developing mobile applications for contact tracing. Table 1 displays the role of IT during an epidemic or pandemic.

IT Category
Technology Applications Findings Reference

Geographic information system (GIS)
-HealthMap Using GIS-based digital dashboards helped the general public cope with health disasters and enabled the health workforce to make better decisions in risk assessment and emergency response operations during the H1N1 pandemic. [13] -Digital dashboard for mapping disease cases Using a digital dashboard for mapping disease cases during the COVID-19 pandemic was essential for timely and effective epidemic monitoring and response. [11] Social media -Twitter -Instagram Social media, such as Twitter and Instagram facilitated the detection of infection, and they helped respond to emerging infectious events. [19] -Blogs -Facebook -Twitter -YouTube The use of social media during the Middle East Respiratory Syndrome coronavirus (MERS-CoV) outbreak in South Korea was found to have a positive effect on negative emotions (e.g., fear and anger), which in turn increased the public's preventive behavior by enhancing their risk perception. [20] -Government social media account Developing and managing government social media accounts was crucial for government agencies to communicate with the medical workforce, public organizations, patients, and volunteers for effective disaster management. [21] Mobile technology -Mobile telemedicine Mobile telemedicine systems enabled immediate access to information for organizing medical resources, effective decision making, and improving patient care in the public health crisis. [8] -Mobile app for contact tracing A mobile contact tracing app used to identify infection movement had the potential to improve data access and quality to support evidence-based decision making for the Ebola response in Guinea. User literacy, government capacity for data utilization, and privacy concerns were found to be limitations of using the app. [22] Hospital information system -Incident command dashboard -Telehealth systems An enterprise healthcare system combining a hospital incident command system and site-based command system effectively supported patients, the medical workforce, and the community during the COVID-19 pandemic. [2] Robot-controlled technology -Robots There is an increasing need for the use of robots for coping with the COVID-19 pandemic. Robotics should be used for telemedicine, delivery, quarantine management, and decontamination. [23]

Cognitive Appraisal Theory
Given that IT for disaster management enables people to stay informed about the potential damages or losses in their physical and social environments, a user's appraisal of IT is associated with his or her emotional experiences while using the IT. Prior research has employed cognitive appraisal theory to explain how people evaluate the use of IT for disaster management (i.e., an IT event) and how people emotionally react to the event [20]. For example, the use of a contact tracing app during the COVID-19 pandemic might make people vigilant and cautious, and they may feel relieved because the information on the movements of infected people can help them quickly take the necessary actions to prevent infection. On the other hand, others might believe that such a digital tracing technology will ultimately have a detrimental impact on people's lives because some personal information on the infected individuals can be revealed to the public. If the expected consequences of the use of the contact tracing apps are positive for the user, then the expected consequences can be categorized as opportunities, whereas if the consequences are deemed to be negative, they can be categorized as threats [24]. Given that an IT event is multifaceted, user appraisals contain positive and negative consequences depending on the event's relevance and importance to the user [25].
Emotions in IT use reflect a user's mental state and whether he or she is ready for action, which leads to certain behavioral responses to IT [26]. Understanding users' emotional responses is important because emotions play a role as adaptive intermediaries between technological stimuli and behavioral outcomes [4]. In this sense, understanding how users emotionally respond to the use of contact tracing apps during COVID-19 helps researchers determine their continuance intention to use the apps.
Given the unusual research context (the use of contact tracing apps during the COVID-19 pandemic), we believed that either a qualitative or a quantitative method alone is Sustainability 2021, 13, 608 4 of 17 insufficient to address our research questions. Accordingly, we adopted a mixed-method approach, combining a qualitative study, and subsequently, a quantitative study. In the qualitative study, we identified threats and opportunities that users perceive in the use of contact tracing apps and their emotional experiences to contextualize cognitive appraisal theory in the situation of the COVID-19 pandemic. In the quantitative study, (1) we operationalized major constructs based on the qualitative study, and (2) drawing on cognitive appraisal theory, we developed and tested a model that explores the interplay among perceived threat and opportunities, emotions, and continuance intention. Figure 1 represents the workflow of our study combining qualitative and quantitative approaches.

Research Setting
South Korea is among the countries that have adopted mobile applications for contact tracing to control the spread of infection. It has been known that South Korea has been relatively successful in controlling the spread of the virus due to the use of contact tracing apps [18]. Users receive notifications on the updated situation (e.g., a total number and a daily number of infected cases) and on confirmed cases' movements. While the personal identity is anonymous, gender, age, and living area are disclosed. For example, an app called Coronavirus Map provides at-a-glance COVID-19-related information, such as quarantined places and the past movements of confirmed cases. The places that the confirmed cases have visited are marked and linked in a line to intuitively show the moving path of each case. Coronavirus Doctor (CoDoc) is an app that provides all information related to COVID-19 in one map. Most importantly, the location information of the confirmed cases as well as selected clinics and detention hospitals in each region are marked on the map. The app also provides a link to the webpage that contains the government's official responses to COVID-19-related questions. The users can select a certain confirmed case and view past movements of the individual. Apart from these examples, more than 10 applications have been widely adopted and well received as a smart way to avoid COVID-19 in South Korea. Most contact tracing applications are based on GPS technology.

Research Methods
We conducted 25 open-ended interviews comprised of nine office workers, seven college students, six housewives, and three freelancers who had used contact tracing apps for COVID-19. We determined the optimal number of interviewees by confirming theoretical saturation, which indicates that no new codes will emerge from additional data collection and analysis [27]. After conducting five interviews, we analyzed the interview data based on the coding approach. Having conducted interviews with 25 subjects, we could no longer find new codes. The average age of the interviewees was 40.1 years old, and there were 14 male interviewees (56%) and 11 female interviewees (44%). The interviews were conducted one-on-one using Skype and lasted 25-30 minutes each. Interviewees were asked about (1) their overall experiences in the use of mobile apps for contact tracing, (2) what they thought could be negative/positive consequences of the use of the apps, (3) their emotions while using the apps, and (4) their intention to continue using the apps.
The interviews were audiotaped and transcribed verbatim by the authors. Data were put into a qualitative software data analysis program, ATLAS.ti (cleverbridge, Inc. Cologne, Germany). A total of 144 responses from 25 interviewees were retrieved, among which 77 responses were categorized into appraisal, 47 into emotion, and 21 into continuance intention. An initial coding system was prepared based on Beaudry and Pinsonneault's [25] appraisal categories. Participants' emotional responses were coded based on Beaudry and Pinsonneault's [26] framework of emotions, which consists of four emotion classes (achievement, challenge, deterrence, and loss emotions). For the coding procedure, following Stein et al. [28], we grouped specific emotions into a category. For example, responses including emotions, such as satisfaction, enjoyment, and relief, were grouped into the class of achievement emotions. The final coding system consisted of four main categories (threats, opportunities, emotions, and continuance intention), which were further divided into 22 subcategories. The coding results were cross-checked with another researcher who holds a PhD degree in the management of information systems. The inter-rater agreement (IRA) scores were calculated. The IRA was 0.81, and Cohen's κ scores [29] averaged 0.80. The first author reconciled disagreements in codes when disagreements appeared in the coding results.

Results and Analysis
Following the procedure of qualitative data coding proposed by Braun and Clarke [30], we first generated initial codes from the responses. Next, we collated the initial codes into a broader level of themes. The coding results showed that individuals' appraisals of the use of contact tracing apps were mixed and often contradictory. People evaluated the use of contact tracing apps as either beneficial or detrimental, while some evaluated them as beneficial and detrimental simultaneously due to different aspects of the technology. Although people appraised the use of contact tracing apps as beneficial due to the expected positive consequences (e.g., speedy access to information, push notifications, public risk perception, prevention effectiveness), they also appraised them as detrimental due to the expected negative consequences (e.g., system quality, misinformation, privacy intrusion, mental distress). Speedy access to information was found to be most salient among the perceived opportunities, whereas privacy intrusion was found to be most salient among the perceived threats (see Figure 2). Diverse emotions were identified, such as anxiety, anger, worry, satisfaction, hope, interest, arousal, and disgust. All the identified emotions were grouped into four categories (i.e., achievement, challenge, deterrence, and loss emotions) based on the framework for classifying emotions proposed by Beaudry and Pinsonneault [26]. The results show that people experience positive emotions, such as achievement (e.g., relief) and challenge (e.g., hope); however, the results indicate that people also experience negative emotions, such as deterrence (e.g., fear) and loss (e.g., anger). Achievement and Diverse emotions were identified, such as anxiety, anger, worry, satisfaction, hope, interest, arousal, and disgust. All the identified emotions were grouped into four categories Sustainability 2021, 13, 608 6 of 17 (i.e., achievement, challenge, deterrence, and loss emotions) based on the framework for classifying emotions proposed by Beaudry and Pinsonneault [26]. The results show that people experience positive emotions, such as achievement (e.g., relief) and challenge (e.g., hope); however, the results indicate that people also experience negative emotions, such as deterrence (e.g., fear) and loss (e.g., anger). Achievement and deterrence emotions were most pervasive and loss emotions were least pervasive (see Figure 3). Our coding results reveal that users' emotional responses to the tracing apps were mixed. Some people have positive emotions and negative emotions simultaneously. As per intention to continue, 57% of responses were positive, whereas 43% were negative. Table 2   Diverse emotions were identified, such as anxiety, anger, worry, satisfactio interest, arousal, and disgust. All the identified emotions were grouped in categories (i.e., achievement, challenge, deterrence, and loss emotions) based framework for classifying emotions proposed by Beaudry and Pinsonneault [ results show that people experience positive emotions, such as achievement (e.g and challenge (e.g., hope); however, the results indicate that people also ex negative emotions, such as deterrence (e.g., fear) and loss (e.g., anger). Achievem deterrence emotions were most pervasive and loss emotions were least pervas Figure 3). Our coding results reveal that users' emotional responses to the traci were mixed. Some people have positive emotions and negative emotions simulta As per intention to continue, 57% of responses were positive, whereas 43% were n Table 2 presents respondents' sample quotes and their coding results.  System quality • Updates in many cases were delayed when the number of confirmed cases increased rapidly. 5

Misinformation
• I was shocked to see my acquaintances who used to caution themselves against fake news share unverified information without much thought.
• Fake news are too much in user communities. Disinformation, rumors, and various conspiracy theories are pervasive. 6 Mental distress

•
When the application said that the place that I visited earlier was a dangerous spot, I got too anxious. I thought I would rather not know all the news.

•
Sometimes it is annoying because my phone rang all day because of the notifications. 8

Privacy intrusion
• Although the identity of a confirmed individual was not disclosed, the public can identify who he/she is from some of the demographic information, such as the company's name, age, and gender. The greatest benefit of using tracking apps is to access information on the status of infection very quickly. I find information from the apps is much faster than that provided by mass media. 2 Push notification • While I go out or travel, the app sends me an alarm when a confirmed individual comes within 100 m of the user. 9

Perceived threat
System quality • Updates in many cases were delayed when the number of confirmed cases increased rapidly. 5 (6%)

Misinformation
• I was shocked to see my acquaintances who used to caution themselves against fake news share unverified information without much thought.
• Fake news are too much in user communities. Disinformation, rumors, and various conspiracy theories are pervasive.

(8%)
Mental distress • When the application said that the place that I visited earlier was a dangerous spot, I got too anxious. I thought I would rather not know all the news.
• Sometimes it is annoying because my phone rang all day because of the notifications.

(10%)
Privacy intrusion • Although the identity of a confirmed individual was not disclosed, the public can identify who he/she is from some of the demographic information, such as the company's name, age, and gender.
• I do not understand why the information about confirmed individuals' movements is disclosed by a time slot. Why not show the movement information of the confirmed cases by each region in sum?

Speedy access to information
• The greatest benefit of using tracking apps is to access information on the status of infection very quickly. I find information from the apps is much faster than that provided by mass media.

(26%)
Push notification • While I go out or travel, the app sends me an alarm when a confirmed individual comes within 100 m of the user.
• Even if I do not keep reading news, I can receive notifications from the apps, which alerts me to be cautious from visiting the places where new infected cases were confirmed.

(12%)
Prevention effectiveness • The use of tracing apps enables me to prevent getting infected effectively. If you know the movements of infected people, then you can quickly identify whether you have ever been in contact with those infected. In case you think you are potentially infected, you can quickly take a proper action to protect yourself for others. You should prevent others from getting infected from yourself, too!

(6%)
Risk perception • I think the use of tracking apps is really important to control the infection of COVID-19 because it raises public involvement and enhances risk perception.
• We are getting used to living in such a pandemic. Constant alarming notifications are needed to refresh our risk perception.

Theoretical Framework
Our qualitative research reveals that individuals' appraisals of the use of contact tracing apps for disaster management are mixed: some evaluate it as beneficial, while some view it as detrimental. It also indicates that users perceive threats and opportunities from different aspects of the technology, which results in mixed emotions. Some users experience positive emotions, and some negative, while some experience positive and negative emotions simultaneously. Thus, we propose that perceived threat and opportunity should not be conceptualized as two ends of a continuum, but rather as independent constructs. Furthermore, all classes of emotions should be examined simultaneously within a model. Figure 4 presents the research model we developed based on cognitive appraisal theory with the backdrop of our qualitative research.

Appraisal and Emotion
Cognitive appraisal theory has been regarded as the theory of emotions, given that it explains how individuals' emotions are elicited by appraisal [31]. Emotions are defined as "reactions to situational events that are appraised to be relevant to a person's needs, goals or concerns" [28] (p.371). In the context of IT use, cognitive appraisal theory holds that emotions are elicited by a user's appraisal of an IT event. That is, users experience certain emotions not because of the IT event itself but rather because of how they think about and assess the use of IT in a certain situation [32].
Diverse emotions have been classified in emotion frameworks, which apply to distinct contexts. Of these, our research draws on the emotion framework proposed by Beaudry and Pinsonneault [26]. The framework classifies IT users' specific emotions into four categories: loss, deterrence, challenge, and achievement.
Loss is a class of emotions that includes anger, frustration, dissatisfaction, and disappointment [26]. Loss emotions arise when a user perceives IT as a threat and feels that he or she cannot control its consequences [26]. Our qualitative study revealed that some users experienced anger, frustration, and dissatisfaction when they saw people suffering from public criticism because their identities had been disclosed. Although the users themselves were not infected and used contact tracing apps to prevent infection, they were sympathetic to those who had been infected and consequently had their personal information shared with the public. In this case, these users appraised contact tracing apps as a potential source of social stigma, which is a threat, and they experienced loss emotions. On the other hand, some of these users appraised the same IT stimuli as beneficial because they could identify trends and examine the infection status infection of certain local areas, which helped them to take preventive actions. Accordingly, a user's appraisal of contact tracing apps is ambivalent and it has a bi-directional influence on loss emotions. The greater a users' perceived threat, the greater the user would experience loss emotions. In contrast, the greater the user's perceived opportunity, the less the user would experience loss emotions. Hence, we propose the following hypotheses: Hypothesis 1. Perceived threat is positively associated with loss.

Hypothesis 2. Perceived opportunity is negatively associated with loss.
Deterrence is a class of emotions that contains anxiety, worry, fear, and distress [26]. Such emotions arise when a user perceives an IT event as a threat but he or she feels that they have some control over the consequences. In general deterrence emotions have a negative valence due to the IT stimuli. In our research context, frequent notifications and warning messages can be the causes for deterrence emotions. In our qualitative study, users reported that they were anxious, distressed, and worried when the contact tracing app constantly sent notifications about the new cases and warning messages about the locations that infected people had visited. The same users appraised contact tracing apps as beneficial because they could be alerted to infection risks and avoid them by using the provided information. Accordingly, the users assessed contact tracing apps for infection prevention in oppositional ways, thus having a bi-directional influence on deterrence emotions. We propose the following hypotheses: Hypothesis 3. Perceived threat is positively associated with deterrence.

Hypothesis 4. Perceived opportunity is negatively associated with deterrence.
Challenge is a class of emotions that contains excitement, hope, anticipation, arousal, and flow [26]. These emotions occur when a user appraises an IT event as an opportunity and feels that he or she has some control over its positive consequences. Our qualitative study found that some users were excited that contact tracing apps can potentially control the spread of the infection. They believed that IT use was key to managing public health disasters; however, some users considered IT for contact tracing a threat because it may stigmatize the infected people. Accordingly, we propose the following hypotheses: Hypothesis 5. Perceived threat is negatively associated with challenge.

Hypothesis 6. Perceived opportunity is positively associated with challenge
Achievement is a class of emotions that includes happiness, satisfaction, enjoyment, and pleasure, and these emotions arise when a user appraises an IT event as an opportunity that generates positive outcomes [26]. In our qualitative study, although some users felt that they could not influence the pandemic and only received information on its spread via contact tracing apps, they were satisfied with their use of the apps. In contrast, users who appraise contact tracing apps as detrimental would be less satisfied with the apps, and the use of the apps would not be enjoyable. Therefore, we present the following hypotheses: Hypothesis 7. Perceived threat is negatively associated with achievement.

Hypothesis 8.
Perceived opportunity is positively associated with achievement.

Emotions and Continuance Intention
Cognitive appraisal theory holds that emotions are elicited by cognition and result in actions to achieve certain behavioral intentions [33]. Researchers have found that emotions can powerfully influence an individual's behavioral intentions [34].
Elicited loss emotions have a negative valence, creating defensive mechanisms for users to avoid pressures and shift their attention away from a problem [35]. Researchers have empirically determined that loss emotions, such as disappointment, anger, regret, and dissatisfaction, are more likely to lead to negative IT outcomes, including complaints [36] and an unwillingness to continue to use [37]. As they trigger defensive mechanisms, loss emotions can also influence individuals to deemphasize the importance of IT [38]. It has also been found that users with loss emotions are more likely to avoid the negative affect by reducing or limiting the use of IT [39]. Accordingly, we propose the following hypothesis: Hypothesis 9. Loss is negatively associated with continuance intention.
Deterrence emotions are elicited when users anticipate negative outcomes from their IT use [38]. When people believe that they have some control over a situation, the negative valence caused by deterrence emotions results in constrained action tendencies. In the IT context, users who experience deterrence emotions are more likely to avoid or escape certain situations, thus leading to a decrease in IT engagement [26]. Accordingly, we put forward the following hypothesis:

Hypothesis 10. Deterrence is negatively associated with continuance intention.
Challenge emotions are likely to encourage individuals to explore the new capabilities provided by IT and thus maximize its benefits [25]. Researchers have suggested that challenge emotions are positively associated with the attitudes toward IT [40] and the use of IT [39]. In the COVID-19 situation, users with challenge emotions would feel that contact tracing apps are instrumental in controlling the spread of the virus. Accordingly, we propose the following hypothesis: Hypothesis 11. Challenge is positively associated with continuance intention.
Achievement emotions arise when a user perceives beneficial outcomes from the use of IT with limited control over the consequences [41]. Given that individuals fulfill their needs by using IT, they are happy, satisfied, and pleased. However, they do not further explore the benefits of IT because they have little control over the consequences. For example, users of contact tracing apps are happy, satisfied, and pleased when they feel that their need for information is fulfilled. On the other hand, they might feel as though they only receive information and cannot change the situation itself. In this case, their emotions are positive in general, leading them to the state in which they feel pleased with their IT use. Such emotions have been found to be positively associated with attitudes toward IT use [42]. Based on this information, we propose the following hypothesis: Hypothesis 12. Achievement is positively associated with continuance intention.

The Direct Effect of Appraisal on Continuance Intention
The main tenet of cognitive appraisal theory is that an individual's appraisal of an IT event elicits a certain emotional reaction. Researchers have argued that both consistent and contradictory appraisals of an IT event are possible [28], and our qualitative study supports the theory of mixed appraisals. Although a user may feel that contact tracing apps are beneficial for controlling the spread of infection, he or she can also appraise the use of such apps as detrimental because it encourages excessive government surveillance and causes significant privacy invasion. Thus, a user can appraise aspects of contact tracing in different ways. Therefore, because perceived threat and opportunity function as independent antecedents of continuance intention, we present the following hypotheses: Hypothesis 13. Perceived threat is negatively associated with continuance intention.

Hypothesis 14.
Perceived opportunity is positively associated with continuance intention.

Data Collection
We used a survey to collect empirical data from individuals who had used smartphone apps for contact tracing during the COVID-19 pandemic. To conduct the survey, we commissioned an online research institution with the largest panel size in South Korea. The survey was conducted from 22 to 30 April 2020. Among the 1000 questionnaires initially distributed to the panel, 567 questionnaires were returned. After removing 59 questionnaires that were incomplete or imprecise regarding the names of the tracing apps used, we included 506 questionnaires in the final data analysis. Table 3 displays the demographic characteristics of the respondents.

Measurement
To psychometrically assess perceived opportunity and threat, we adapted the scale developed by Bala and Venkatesh [43]. To measure user emotions, we adopted the classification of emotions proposed by Beaudry and Pinsonneault [26]. The respondents were asked to report the degree to which they felt each specific emotion. To measure the users' continuance intentions, we incorporated the scale presented by Bhattacherjee [44]. A sevenpoint Likert scale was used for all psychometric measures. To establish the robustness of our structural model, we used age, gender, education, tracing-app usage frequency, and innovation traits as control variables. The measurement items, scales, and their sources are presented in Appendix A.

Results and Analysis
We tested our model with the partial least squares (PLS) method, a structural equation modeling (SEM) form that enables researchers to examine several factors affecting multiple dependent variables [45]. Researchers recommend using PLS when the phenomenon is in the early stages of theoretical development [46], which is the case in this study. We used the software SmartPLS 3.0 (SmartPLS GmbH, Bönningstedt, Germany) for data analysis.

Measurement Model
Drawing on the emotion framework [26], we placed diverse emotions into four classes: loss, deterrence, challenge, and achievement. To test the validity of the discrete emotion classes, we conducted a principal axis factoring analysis using the varimax method [47]. Of the 21 emotion items, 18 were loaded on each distinct factor, and the factor loadings for all items were greater than 0.7 except for three items: satisfaction (0.627) from the achievement class and playfulness (0.598) and excitement (0.577) from the challenge class. Given the nature of the apps, which were used to prevent infection, the users may not have experienced these emotions. Accordingly, we removed these three items from further analyses. The results of the factor analysis showed that all item loadings exceeded the recommended value of 0.7 and were thus statistically significant. Table 4 contains the factor and cross loadings for each item. Next, we tested the reliability and validity of our measurement model. Table 4 shows that all the criteria for convergent validity were met, with CA values ranging from 0.883 to 0.942, CR values ranging from 0.910 to 0.954, and AVE values ranging from 0.679 to 0.796. To assess discriminant validity, we examined the AVE values of all constructs. For the discriminant validity of our measurement model to be ensured, the square root of a construct's AVE must exceed its correlation with every other construct [45]. As shown in Table 5, each construct's square root of AVE is greater than its correlation with all the other constructs, confirming the discriminant validity of the measurement model. As shown in Table 6, the users' emotional reactions toward the use of contact tracing apps were mixed. At the specific emotion level, the users experienced relatively greater levels of relief (mean = 4.39), hope (mean = 4.20), and worry (mean = 4.20). At the emotion class level, the users experienced positive emotions mixed with negative emotions: achievement (mean = 4.01) and deterrence (mean = 3.85) were the most salient emotions, followed by challenge (mean = 3.81) and loss (mean = 3.14).
Finally, we used Harman's one-factor analysis to examine whether data variance was due to the use of a common data source [48]. Common method bias (CMB), wherein a single factor accounts for most of the covariance in the research variables, can reduce the quality of a dataset. Therefore, we subjected all measures to a principal component analysis without factor rotation, and we determined that no single factor predominantly accounted for the variance in the data.  Figure 5 shows the results of the structural model test. The model accounted for 43% of the variance in continuance intention. The results indicated that perceived threat negatively influenced continuance intention (β = −0.129, p < 0.001), while perceived opportunity positively influenced continuance intention (β = 0.493, p < 0.001). In addition, the results showed that perceived threat significantly increased challenge (β = 0.133, p < 0.05), deterrence (β = 0.336, p < 0.001), and loss emotions (β = 0.366, p < 0.001). In contrast, perceived opportunity significantly increased achievement (β = 0.257, p < 0.001) and challenge emotions (β = 0.298, p < 0.001) and decreased loss emotions (β = −0.234, p < 0.001). The results also revealed that only two classes of emotions-challenge and loss-were significant factors that influenced continuance intention. Challenge increased continuance intention (β = 0.228, p < 0.001), whereas loss decreased continuance intention (β = −0.264, p < 0.001).

Discussion
The results demonstrate that although the respondents considered their use of contact tracing apps to be beneficial in general, they also evaluated them as detrimental due to privacy issues, misinformation, mental distress, and poor system quality. Such negative appraisals impacted the respondents' continuance intentions. The results also show that some users were more concerned about disclosing their personal information (e.g., past movements in detail by time slot) than becoming infected (see Table 2). Although specific personal identity information is not included in contact tracing apps, the public can potentially identify individuals based on the revealed information. If users are concerned about criticism from the public, they may be reluctant to be tested or hide the fact that they are infected. In this case, disclosing information on the confirmed individuals' movements can hinder control of the infection.
The results also indicate that user emotions play a significant role in determining continuance intention. Of the discrete emotions, loss was found to be the most salient class of emotions that decreased continuance intention. These results imply that if a person experiences loss emotions (anger, frustration, dissatisfaction, disappointment, disgust), he or she will be unwilling to continue using contact tracing apps. Cognitive appraisal theory explains that people experience loss emotions when they evaluate the use of a technology as detrimental and feel that they have limited control over the expected consequences. Our qualitative study provides information on the factors that increase loss emotions: users are significantly threatened by intrusions to their privacy.

Implications for Research
This study makes several key academic contributions. First, it fills a gap in the disaster management literature by empirically examining users' appraisals of technologies that rely on massive surveillance and tracing based on a centralized database. Such technologies are controversial due to the conflict between public safety and the invasion of privacy. This study explores the contradiction underlying user appraisals and emotions, which jointly determine users' continuance intentions. Our conceptualization of perceived threats and opportunities can help researchers develop, test, and revise models that consider the interplay between users' appraisals and their emotional responses to IT during a public health disaster.
Second, this study contextualizes users' perceived threats and opportunities while using contact tracing apps during the COVID-19 pandemic when rapid testing, tracing, and treating are crucial for preventing the spread of the virus. Although the public agrees that digital tracing is necessary during this public health crisis [3], our study emphasizes that individual users appraise the use of contact tracing as detrimental due to the expected harmful effects on their lives.
Studies examining the role of emotion in IT use are rare, despite calls to pay more explicit attention to this topic [28,49]. Our work contributes to emotion classification by empirically testing the utility of the emotion framework proposed by Beaudry and Pinsonneault [26]. Although other studies have previously adopted this framework, most represented each class with a single specific emotion. Few studies have examined the full range of emotions related to IT use. By successfully grouping specific emotions into different classes, this study provides a more comprehensive and nuanced understanding of the role of emotion in one's continuance intention for IT use.

Implications for Practice
Although information use is essential in a disaster response [9], our results indicate that it can be a double-edged sword for individuals during a crisis. IT artifacts that enable excessive information disclosure without providing users with control over the consequences of their technology use may cause loss emotions, such as anger, frustration, and disgust. Such emotions discourage users from continuing to use the technology. Therefore, app designers should create functions that give users a sense of control over their technology use. Designers of IT artifacts or tracing-app functions must avoid the excessive disclosure of personal information. Public policymakers should also consider developing measures that protect privacy.

Limitations and Future Research Directions
The contributions of our study should be interpreted in light of its limitations, which point to future research directions. First, the current study employed a cross-sectional survey method to collect data from tracing app users during the COVID-19 pandemic. Users' emotions and their continuance intention might have been influenced by the severity of infection in a specific period of time. Although our study focuses on the relationship between emotions and continuance intention, future studies might benefit from collecting data from multiple periods of time to ensure the generalizability of the findings. Second, our data were collected from a single source, and all research constructs were measured by respondent perceptions. Although our data analyses revealed that the CMB was not a serious concern for the current study, we encourage future studies to use objective data, such as usage data or log data, in the use of specific functions to better address the CMB issue.

Conclusions
Recent IT research has highlighted the role of IT in disaster management. Our model developed based on cognitive appraisal theory recognizes that contact tracing apps for infection prevention provide users with unique socio-technological environments that feature dual effects of IT artifacts that engender perceived threats and opportunities. Our study emphasizes that the way users assess contact tracing apps determines their emotional experience, by which their continuance intentions are influenced. Therefore, to sustain a user's engagement with contact tracing apps, proper campaigns for the use of technology and public policy that alleviate public concerns in relation to tracing technology should be developed.
Author Contributions: A.S. conducted the investigations and mainly wrote the manuscript. M.L. revised the manuscript and contributed to data presentation. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.

Institutional Review Board Statement:
The study was conducted according to the guidelines of the Declaration of Helsinki.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.

Conflicts of Interest:
The authors declare no conflict of interest. Table A1. Measurement items.

Appraisal
To what extent do you agree with the following statements? Seven-point Likert scale: 1 = strongly disagree; 7 = strongly agree Perceived threat [43]