{'fraction_non_alphanumeric': 0.028334183116031756, 'fraction_numerical': 0.017444824823366595, 'mean_word_length': 5.173336330935252, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
No Online Information Outbreak: A Quantitative Content Analysis of the CDC and USDA Websites for Available Information on Zoonotic Disease

Zoonotic diseases are a significant threat to human and animal health with the effects of a widespread epidemic impacting agricultural producers and consumers alike. Online information sources have the opportunity to widely distribute information, but, with a topic as complex as zoonotic disease, information sharing should be managed carefully. Risk communication and prepared responses for zoonotic disease can help communicate messages effectively. This study looked at how two federal websites, CDC and USDA, were communicating about zoonotic disease. The quantitative content analysis methodology was guided by the research objectives of 1) determine availability of information related to zoonotic disease, 2) describe the zoonotic diseases, impacts, and messages 3) determine the use of prepared responses in articles related to zoonotic disease, and 4) determine connectivity with other online resources on zoonotic disease. Results indicate information is difficult to find on both websites, and there is a lack of connectivity with other online resources. Prepared responses were used to varying degrees. Implications and recommendations from this work are that agricultural communicators and those with influence over federal communication on zoonotic disease should actively integrate prepared responses in communication and seek opportunities to connect to a larger network of those working in zoonotic disease.


Introduction and Problem Statement
It is estimated that three-fourths of emerging infectious diseases are zoonotic, meaning the disease can spread between humans and animals, such as West Nile virus and Lyme disease (Taylor, Latham, & Woolhouse, 2001). Zoonotic diseases pose a significant threat to public health, making it especially important for wildlife and health officials to communicate with the public about health and safety information and promote good decision-making (Clarke, 2009;Decker et al., 2009). In the event of a zoonotic disease outbreak, good communication is essential (Tabbaa, 2010). As wildlife and health officials learn more about how the public searches for information about zoonotic diseases and how that information influences people's decisions and behaviors, officials can develop more effective communication campaigns and messages (Clarke, 2009;Griffin, Dunwoody, & Neuwirth, 1999).
Online communication and new media have impacted the way people receive and seek information about issues, particularly during times of crisis, offering the public more power to engage with diverse information (Westerman, Spence, & Van der Heide, 2014) and government agencies in new ways (Mergel, 2016). The public will seek further information about particular issues once they have been exposed to a risk message, such as a message that communicates about a zoonotic disease outbreak (So, Kuang, & Cho, 2019). A variety of factors can determine if an individual will seek information about the risk message, such as fear, anxiety, perceived severity, and perceived susceptibility (So et al., 2019). As individuals seek information about messages, their perceptions of related issues are influenced and could potentially change over time (So et al., 2019).
When individuals are processing information related to risk, they will also consider the values and perceptions of trust in the entity that is communicating (Clarke, 2009). Often, the entity communicating about zoonotic disease outbreaks will be agencies associated with state and federal government, such as public health agencies or wildlife management agencies. Individuals want these agencies to reflect their own values related to personal health and the protection of other individuals and the animal(s) associated with the disease (Clarke, 2009). While the public will typically react to an outbreak based on their perception of the risk, the government and related institutions will make decisions based on the actual risk; meaning the way the government communicates information about a zoonotic disease outbreak is fundamental to how the larger public will react (Smith, 2006). Government organizations that are communicating about zoonotic disease outbreaks have the potential to lead the public's knowledge and understanding of the disease outbreak and give recommendations to people on what actions to take to protect themselves (van der Meer, 2018). Because of the hierarchical structure of government organizations, those communicating directly with the public are often employees in district or local offices (Blau, 1968;Mergel, 2016;Palttala, Boana, Lund, & Vos, 2012;Rogers, Burnside-Lawry, Dragisic, & Mills, 2016). The local and district offices rely on the national organization's websites and articles to communicate with the public about matters related to public health, like a disease outbreak (Goel, Belardo, Iwan, 2004;Mergel, 2016;Institute of Medicine, 2003). Research on understanding how federal agencies such as the CDC, Center for Disease Control, and USDA, United States Department of Agriculture, are communicating online about zoonotic disease would clarify the regional agency, and ultimately the public, response to and level of preparation for a zoonotic disease outbreak.

Theoretical and Conceptual Framework
The World Health Organization (2017) suggests that entities communicating about potential disease outbreaks follow basic principles of risk communication, including developing trust from the public, remaining transparent, and using proactive communication. Quality risk communication plays an integral role in response to and management of zoonotic diseases. When communication is good, it can "rally support, calm a nervous public, provide muchneeded information, encourage cooperative behaviors, and help save lives; bad communication can fan emotions, disrupt economies and undermine confidence" (Tabbaa, 2010, p. S81).
The model of risk information seeking and processing (Griffin et al., 1999) describes the many factors that impact how individuals seek and process information related to risk, including how perceived hazard characteristics lead to an effective response which then leads to information sufficiency that is impacted by subjective norms. Clarke (2009) contends that this model could be further extended when communicating about zoonotic disease risk to include values, value orientation, and opinion leadership. Public relations models of communication for engaging with the public in health communication efforts have categorized communication as one-way, two-way, or multidimensional (Avidar, Ariel, Malka, & Levy, 2015;Taylor & Kent, 2014). The opportunity for online, organizational communication with the public has opened doors for increased engagement and two-way communication. However, federal organizations have struggled to make this model work effectively. For example, in 2015 the CDC tried to use an online Twitter chat to respond in real-time to public health questions about an Ebola outbreak. While the CDC gained 80,000,000 Twitter impressions during the chat, it only responded to 15% of the tweets, which left many members of the public who were not able to engage with the organization during this time of public health crisis (Young, Tully, & Dalrymple, 2018). However, other chats like one about Zika were only attended by those involved in public health and not members of the public (Young et al., 2018), making online engagement complicated to understand. Federal agencies tend to prefer more traditional one-way communication channels (Avidar et al, 2015;Young et al., 2018), but scholars continue to recommend that entities work to communicate, engage, and provide agency for the public in a way that meets the public's needs (Reynolds & Seeger, 2005;Taylor & Kent, 2014), taking advantage of multiple platforms (Siwak, 2018;Taylor & Kent, 2014), and a variety of communication channels (Boehm, 2012) to increase opportunities for two-way communication (Avidar et al., 2015) about issues related to human and animal health. While the top-tier of government agencies are institutionalized and have more regulations on communication efforts, some of the associated agencies are able to take information from the centralized organization and share and engage with the public in a new way (Mergel, 2016), like a local USDA office sharing an article about zoonotic disease from the USDA website with followers on social media.
A society's response to risk is based on the quantity and quality of media coverage, the actions of risk management agencies, and the perception by the public of these agencies' ability to manage risk and education efforts of government agencies, nongovernmental organizations, and/or other actors (Decker et al., 2009). Response to a zoonotic disease outbreak can be broken down into 7 types of prepared responses (Levings, 2011). The present study investigated all seven: 1) Situational awareness (knowledge of the threats to appropriately prepare and the capabilities and priorities of other entities; awareness of the tools others are using and planning for their use), 2) Research (needed to be done for effective response, from identification of a pathogen and its antigens, elicited immune response or ecology to field application and commercialization (or management) of technologies), 3) Tool acquisition (selecting, ordering, and receiving of tools designed to aid in preparedness for the disease, 4) Modeling (a systematic description of an object or phenomenon that shares important characteristics with the object or phenomenon), 5) Training and exercises (prepares personnel for specific roles in responses and assures the organization and its partners of personnel skill levels and readiness), 6) Animal movement traceability (ability to follow an item or a group of items, like an animal from one point in the supply chain to another, either backwards or forwards), and 7) Policy development (guidelines and positions that influence the decisions made) (Levings, 2011). Coordination and integration among all agencies involved in the response to a zoonotic disease outbreak could improve if these recommended responses are applied.
Past research has evaluated the online content of government agencies that are communicating about environmental risks and have determined that different agencies employ a range of communication approaches. Eschenfelder (2007) determined that state wildlife agencies in four states were communicating about Chronic Wasting Disease, a possible zoonotic disease that impacts deer, through a private citizen approach, which assumes citizens involvement in policy decisions are very limited. Boehm (2012) concluded that federal agencies communicating about Colony Collapse Disorder were using outdated information, avoided controversial topics, and lacked variety in the content they were sharing. It was further determined that the information federal agencies were sharing online related to Colony Collapse Disorder was not relevant to individuals who were seeking information about the issue and that this could lead to a decline in public engagement of the issue (Boehm, 2007).

Purpose
The purpose of this study was to understand how two major government websites, CDC and USDA, made information available related to zoonotic disease. The broader purpose was to make strategic recommendations for federal agencies communicating zoonotic disease information in online environments. The specific objectives guiding the study were: RO1: Determine availability of information related to zoonotic disease. RO2: Describe the zoonotic diseases, impacts, and messages mentioned. RO3: Determine the use of prepared responses in articles related to zoonotic disease. RO4: Determine connectivity with other online resources on zoonotic disease.

Methods
A quantitative, content analysis was used to address the research objectives. This allows for objective and systematic quantitative description of content (Berelson, 1952;Krippendorff, 2013), often in an online environment. The codebook for this study was based on recommendations by Riffe et al. (2005) and Krippendorff (2013) and reviewed by a panel of experts for face content and validity. The codebook was comprised of 13 variables including clicks to get to an article about zoonotic disease from the homepage, names of diseases mentioned and major characteristics, prepared response strategy, additional resource, type of response, location, prevention, location of article. Four independent coders were used in the study. Coders were trained and determined initial reliability of the codebook using the EPA website so as not to invalidate the study sample. In this pilot, a Cohen's Kappa of .70 or higher on all independent variables was established with a total Cohen's Kappa of .85 for all variables combined. Next, the four coders moved to the true area of interest for this study, the CDC and USDA websites. When beginning the process, it was immediately clear that the first question on the initial codebook "how many clicks does it take to get to information on zoonotic disease from the homepage" would not work with the CDC and USDA websites. It was impossible to get to information on zoonotic disease through clicks alone. Coders had to search within the CDC and USDA websites to get information on zoonotic disease. Thus, this variable was removed in the code book for the true sample of the study and a new sampling frame was established.
The sampling frame was generated from an initial search on the USDA and CDC websites using the term "zoonotic disease" which resulted in ~ 36,000 and 48,737 articles, respectively. Researchers investigated the top 30 articles from each site, which was approximately the first three pages of search results for a more detailed analysis. This sample was determined based on data collected by Google about how much people interact with the pages within its search engine in 2019. On the first page of Google searches, 67% of the people who search for a topic click on searches that appear on the first page, while results from the next two pages account for 4% of the clicks, and the numbers for four and beyond represent less than 1%, as the other people typically do not click on any search results (Law, 2019). Similar patterns are seen in Amazon shoppers with 45% of shoppers reporting they do not scroll past the first page of results and two-thirds of product clicks coming from the first page of search results (MarketingCharts.com, 2019). No data was available specifically on how many pages people would scroll through looking for health-related information or disease-related information. Thus, the researchers in this study concluded that the first three pages of results included the majority of what people would be willing to scroll through and/or read, and these articles were pulled for more in-depth analysis in the present study.
Coders begin with coding the same 20% of the sample to again establish inter-rater reliability on the study sample using Cohen's Kappa (Riffe et al., 2005;Krippendorff, 2013). Data were collected within 24 hours of one another in Spring 2018 in order to reduce the chance of changes to the websites. Inter-rater reliability was achieved on each of the remaining 12 variables with a Kappa of .70 or higher as recommended by Riffe et al. (2005) and Krippendorff (2013). Inter-rater reliability for all variables combined was .92. After inter-rater reliability was achieved on the first 20%, the researchers divided the remaining articles and coded independently as recommended by Riffe et al. (2005) and Krippendorff (2013). Data were analyzed for descriptive statistics using IBM SPSS Statistics 20 (IBM Corp., Armonk, NY).

RO1: Determine availability of information related to zoonotic disease.
Availability of information on the CDC and USDA websites was difficult to find. In the codebook for the study, researchers explored how much information was available on the homepages and how many clicks it took to get to a page that mentioned zoonotic disease. Zero references to zoonotic diseases were on the home pages or any of the main pages from the navigation of the CDC and USDA sites. Researchers had to use the search feature within both sites to find resources related to zoonotic disease. Through searches on the USDA and CDC websites using the term "zoonotic disease" approximately 36,000 were found on the USDA website and 48,737 articles were found on the CDC website.

RO2: Describe the zoonotic diseases, impacts, and messages mentioned.
In order to determine how the zoonotic diseases were described on the CDC and USDA websites, coders recorded information related to the origin of the zoonotic disease and if no mention was made of the origin of the disease. The greatest origins mentioned on the CDC website was for a vector origin with 40% (n = 12) and the USDA website mentioned animal vectors the greatest number of times at 46.7% (n = 14). Eight of the articles on the CDC website didn't mention the disease origin at all, while all of the USDA articles mentioned an origin (Table 1).  The level of impact monitoring was also recorded. The largest number of impact monitoring mentioned for the CDC was national impact monitoring with three (10%) of the articles mentioning it. No other levels of impact monitoring were mentioned in any of the articles coded in this study. The USDA website had the largest numbered recorded for impact monitoring at the global level with four (13.3%) of the articles mentioning it followed by three (10%) articles each for national and regional impact monitoring (Table 3).

RO3: Determine the use of prepared responses in articles related to zoonotic disease.
The highest number of prepared responses for both websites was the situational awareness response with 14 (46.7%) of the CDC articles including this response and 23 (76.7%) of the USDA articles including this response. The next highest response for the CDC website was research with 9 (30%) of the articles mentioning it, while the next highest for the USDA was modeling with 19 (63.3%) of the articles mentioning it (Table 4).

RO4: Determine connectivity with other online resources on zoonotic disease.
To determine the connectivity of the CDC and USDA websites with other resources online, coders investigated which other references were linked to the articles available on these sites. Twenty-two (73.3%) CDC articles linked to other resources and 17 (56.7%) USDA articles did. The largest numbers of articles on both sites linked to resources available within their own website with 21 (70%) of the CDC articles linking to sources within their organization and 16 (53.3%) of the USDA articles linking to sources within its own website. One article from the CDC linked to the World Health Organization (3.3%) and one (3.3%) article linked to the Health Resources and Services Administration. The USDA had one (3.3%) article link to the World Organization for Animal Health. There were no links in 7 (23.3) CDC articles and 13 (43.3%) USDA articles (Table 5).

Conclusions, Discussion, and Recommendations
While an outbreak of a zoonotic disease can spread quickly to infect thousands of people and animals, no similar outbreak of online information was seen on the CDC and USDA websites. While there were a large number of articles available on both websites (CDC ~ 36,000; USDA = 48,737) about zoonotic disease, the information was difficult to find. With no references to zoonotic disease on the homepages or main navigation, an employee of a local or district office or a member of the public may think zoonotic disease is not a major concern for either the CDC or the USDA. Utilizing the basic principles in the model of risk information seeking and processing (Griffin et al., 1999), providing information in a more prominent location on the homepage regarding zoonotic diseases may increase awareness within individuals and assist with understanding and processing. An understanding of the perceived risk could better prepare individuals if/when an outbreak occurs (So et al., 2019). However, this data was collected during a time without a widespread epidemic of zoonotic disease; one could assume that during an outbreak this information would be more prominent on both the CDC and USDA websites. Agricultural communicators should be aware of the lack of information available to the public without a direct search of the term zoonotic disease. Since this term is not one used in everyday life for many consumers, more communication focused on the awareness and preparedness for a zoonotic disease is needed. Future research should explore the public's knowledge of, perceptions of, and preparedness for zoonotic disease. In addition, message testing on messages surrounding zoonotic disease to better understand what messages publics understand and which specifically resonate with them.
The CDC and USDA websites did not take advantage of the interactive nature of online communication. While keeping traffic inside the organizational website is understandable, there is an opportunity to connect to a variety of different information from credible sources including other government agencies and world health organizations. This lack of interactive information available and a lack of variety in the information provided aligns with previous work by Boehm (2012). All of the information found on the CDC and USDA websites was in the form of articles and very few links were provided to content outside of the entities' own websites (CDC = 2, 6.3% and USDA = 1, 3.3%). The opportunity for the internet to transform the way people engage with and learn about health information is stifled when the interactive nature of the web is not utilized. While a website is limited in its ability to directly engage with the public in the two-way communication model recommended by scholars (Reynolds & Seeger, 2005;Taylor & Kent, 2014), there are ways federal agencies could integrate more interactive elements into websites and available articles within the sites. Links are one simple way to increase interconnectivity, but improving search features, allowing comments or feedback, and linking to social media platforms or other new technology may be a way for federal agencies to engage more with stakeholders at all levels.
The use of preparedness responses evident in articles within this study varied, which was similar to previous work on government communication online (Boehm, 2007;Eschenfelder, 2007). The highest use was of situational awareness with 14 (46.7%) CDC articles and 23 (76.7%) of USDA articles. The USDA articles used more total preparedness responses than the CDC did. The low number of articles with tool acquisition and training may indicate a gap in the available tools and training for zoonotic disease, which is concerning as these are the responses in which people can actively engage in handling a zoonotic disease outbreak or preparing in advance of an outbreak (Levings, 2011), and may be the most useful tools for local and district government offices. It is recommended that federal agencies, and agricultural communicators in general, adopt and embrace the preparedness responses to better inform the public about zoonotic disease and how to prepare for, avoid, and respond to an outbreak. Communication efforts can include multiple responses to better prepare other government offices and the public. The public can be scared when hearing messages related to risk, but the inclusion of preparedness responses can empower people to know how to act, which can lessen the associated fear.
With 50% of articles on both the CDC and USDA websites having a message related to environmental impacts, this may connect with the values of some members of the public, as recommended by Clark (2009). The USDA website had more messages related to economic impacts (n = 10, 33.3%) than the CDC (n = 3, 10%), which may be a function of the value agricultural producers place on zoonotic disease, as this will affect their business bottom-line. However, the impact of zoonotic disease can be economically stressful for people who contract diseases too. Messages related to handling procedures were high on the USDA website (73.3%), which is a message stakeholders likely care about when seeking information related to zoonotic disease. However, information on handling procedures on the CDC website were at 30%, which aligns with Boehm (2007) that some federal agencies may lack important and valuable information for the general public on what action to take based on information provided. This would also be advantageous for government employees at local and district offices seeking to inform stakeholders in their region on how to take action once a person or animal is infected. Future research should investigate how individuals, including government employees, respond to these types of messages in an effort to understand the value people place on these specific messages.
The limited information available on the CDC and USDA websites about the origin of zoonotic disease was also a concern, as this is vital for individuals to know so they can avoid infection for themselves and/or their animals (Clarke, 2009). As people interact with information about zoonotic disease, their opinions and decisions could be positively influenced (So et al., 2019), and a prominent presence of this type of information could solidify opinions and decision making of the public and smaller agencies prior to a major or isolated zoonotic disease outbreak. Moreover, during times of zoonotic disease there is a wealth of inaccurate information available about disease spread, so federal agencies and local branches of offices need information readily available for people to fact-check other sources. Additionally, federal agencies will be pulled in multiple directions once an outbreak occurs, thus it is to their benefit to already have a solid presence in place that visitors to their site are familiar with and able to navigate with ease. Additionally, if a minor outbreak occurs, federal agencies may not launch a full-blown response, but local and regional offices will need readily available materials to distribute to local publics.
It is a limitation of this study that it is only a snapshot of what was happening at one particular time on the CDC and USDA websites. When data were collected, there was not a large outbreak of zoonotic disease or huge national news coverage of a specific or general zoonotic disease. Future research should compare available information on these sites at a later date and during a time of a large-scale U.S. outbreak of a specific zoonotic disease to see if available information and placement of this information on the sites changes. Another limitation in this study is the use of "zoonotic disease" as the search term to generate the articles in the sample. This was selected to cast a wide net for all articles in this category, and it is the common term used by the industry. However, the general public may not think to search for this particular term. Future research should explore the available articles related to specific zoonotic diseases, as this may be what the public would search for on these websites.