{'fraction_non_alphanumeric': 0.028135144883972587, 'fraction_numerical': 0.012324155344475172, 'mean_word_length': 5.160925925925926, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Beyond the Surveys: Qualitative Analysis from the Academic Reading Format International Study (ARFIS)

The Academic Reading Format International Study (ARFIS) collected data from more than 21,000 university students in 33 countries regarding their reading format (print or electronic) preferences and behaviors when engaging with academic texts during a three-year period. Quantitative analysis shows a consistent preference for print reading among most students worldwide. This paper presents new findings from our qualitative analysis of students’ survey comments, addressing the questions: Why do students prefer to read in print or electronically? What factors affect their decisions to read a text in their less preferred format? Are there behavioral patterns that will assist librarians and educators when deciding upon format policies?


Introduction
There are two major tracks of research on the topic of reading format, whether print or digital. First, are comparisons of cognitive performance when reading on print and electronic formats, which include tests of comprehension, memory and learning. The body of evidence from this track has made it increasingly clear that, in certain circumstances, print reading results in better comprehension than digital reading; but also, that the nature of the circumstances around the reading are important to outcomes and that not all circumstances result in this difference in comprehension performance.
Second are studies of readers' format preferences, behaviors, and attitudes. Most of these rely on surveys of various reader populations, and results consistently show that a large majority of readers prefer print. Commercial sales of electronic books (e-books) reflect the print preference of general readers and even show a decline in the percentage of e-book sales in the United States from 2014 through early 2019. 1 Even so, consistent minorities of readers express a preference for digital formats, and we know that digital formats continue to enjoy substantial, if in some cases declining, readership.
What are the differences between these groups of readers-those who say they prefer to read in print and those who say they prefer to read in electronic format? What factors drive their preferences, and what relationship do these preferences have to what we know about the influence of the medium on learning?
Schools and institutions of higher education continue to push for the use of digital technologies to offset the mounting costs of textbooks, to promote environmental sustainability, and as a natural outcome of technologically informed pedagogy. It is essential for us to understand better what drives learners' format preferences, particularly as they relate to learning, to make informed policy and practice decisions at the classroom and institutional levels and to shape the design and development of optimal reading and learning circumstances in tandem with the technology sector.
To that end, this study analyzes qualitative data taken from the largest international study of tertiary students' reading format preferences to date, the Academic Reading Format International Study (ARFIS), to elucidate the nuances and the variety of circumstances affecting their reading attitudes and behaviors. While quantitative data from this study was published in 2018, 2 the current analysis moves beyond the what of establishing definitively a global preference for print reading among tertiary learners and delves into the why behind these preferences.
The research questions are these: Why do students prefer to read in print or electronically? Under what circumstances will they read a text in their less preferred format? Are there patterns that can assist librarians and educators when deciding upon format policies?

Review of Selected Literature
Studies of students' digital reading behaviors from the early 2000s to present reflect the evolutions in technology, the proliferation of format choices, and increased access to digitally based academic texts and electronic books. Carol Tenopir summarized the findings of 200 studies published between 1995 and 2003 in her report for the Council on Library and Information Resources (CLIR). Among the findings relevant to this paper, she reports that students used electronic resources if they were "perceived as convenient, relevant, and time saving to their natural workflow"; that print books were preferred over e-books, whose usage was still in the early stages; and that most users printed out electronic articles they deemed useful. 3 In his review of research on students' attitudes and use of e-books, Bernd Becker considered 2010 as "Year Zero" for e-books because the iPad went mainstream that year, enabling easier access to e-books and other electronic sources. 4 In regard to format preferences, Becker states, "You can confidently expect to see the phrase 'students prefer print over digital' in the results section of any study conducted between 2010 and 2015, and likely for many more years to come." He notes that studies are just beginning to investigate the factors affecting preferences and behaviors.
Mizrachi's ethnographic study of 41 undergraduates 5 found an overwhelming preference for reading academic texts in print, but that "their behaviors did not necessarily reflect their preferences." She sought to explore this phenomenon further in two subsequent studies: her 2014 survey of nearly 400 undergrads on which ARFIS is based 6 and her case study of 61 first-year students in a class whose course readings were offered both electronically and in print. 7 Even when afforded the choice of accessing online readings for free, using print, or any combination thereof, more than 70 percent reported that they read all or most of their texts in print. The sample populations of Mizrachi's studies were all different cohorts of undergraduates at the University of California, Los Angeles, but results remained consistent. Generally, students believed they learned better with print because it is more conducive to interactive reading and engagement, but they liked the convenience of accessing materials online and its perceived lower cost and ecological friendliness. They preferred print for longer and important readings, but many were "okay" with reading shorter texts on screen, especially when they felt pressed for time.
The ARFIS project expanded into a collaboration of research partners from 38 institutions in 33 countries, the largest study of its kind. Data was collected from 21,266 college and university students at all levels regarding their academic reading format preferences and behaviors, whether print or electronic, from 2014 through 2017, the first half of which was analyzed and published in 2018. 8 This data shows that large majorities of tertiary students in every demographic group and country represented state that they prefer to read their academic texts in print and believe that they focus and remember information better in print, while a minority do state a preference for digital text. Differences between country-level responses were of small statistical significance and limited effect size, underscoring the consistency of these reports around the world.
The ARFIS project includes data from Mizrachi's 2014 study along with the data from 32 other countries (see the full list of ARFIS countries in appendix A). Among the several ARFIS partners who published results of their country's data, researchers in Finland, 9 Norway and Romania, 10 the United Kingdom, 11 Qatar, 12 China, 13 and Australia 14 analyzed their students' comments using qualitative methods to reach deeper understandings of their attitudes and behaviors. All findings are derived from data coding practices consistent with Mizrachi 15 and, like Mizrachi, they found that preferences do not always drive actual reading behavior. Many students will use either format depending on contexts and circumstances.
In 2017, Maura Smale performed a qualitative study of 30 undergraduate students from three City University of New York (CUNY) campuses to investigate questions on how, when, and where students access and do their assigned readings. 16 She found that most of the students preferred print because of the ease of highlighting and annotating the text, as well as the lack of distractions. CUNY campuses are commuter schools, however, so many students found electronic format more convenient during their commute time. They also considered cost when making format choices.
Among the many actions sponsored by the European Cooperation in Science and Technology (COST) is the Evolution of Reading in the Age of Digitisation (E-READ). E-READ involves nearly 200 scholars of reading and literacy from across Europe and seeks to "develop new research paradigms, and metrics for assessing the impact of digitization on reading." 17 In October 2018, this action resulted in the Stavanger Declaration, which lists among its summary of conclusions the consistency of print preference among readers "for longer single texts, especially when reading for deeper comprehension and retention, and that paper best supports long-form reading of informational texts. Reading long-form texts is invaluable for a number of cognitive achievements, such as concentration, vocabulary building and memory." 18 These decades of research present solid evidence that majorities of students prefer to use print for learning; that minorities prefer electronic formats; and that actual reading behaviors do not always match stated preferences. There is less clarity across the literature about what drives these preferences and behaviors. What are the variables that may influence these preferences, and what are the factors that may cause learners to compromise on their preferred format and use their less-preferred format for reading, particularly since broad national-level socioeconomic and technological development factors could not be identified as primary factors in prior research?
The current study analyzes a sample of English-language comments from 653 participants across 15 different countries included in the ARFIS study to elucidate why tertiary readers prefer the formats they do and under what conditions preferences and behaviors vary.

Methodologies
The Academic Reading Format International Study (ARFIS) questionnaire was developed, piloted, and validated in 2014, 19 with the intention of enabling wider, more consistent, and more comparable data gathering pertaining to learner reading format preferences across contexts and over time.
The ARFIS instrument contains 16 Likert-style statements, six demographic questions, one multiple-choice question, and an open remarks field. Every Likert-style question includes space for optional explanations of their response with open, free-text comments, and these comments, as well as the open remarks field, are the sources of qualitative data for this analysis. Not all questionnaire items pertain directly to format preferences; some ask about text engagement behaviors such as annotating and highlighting, or about reading in a second language (see all statements in appendix B). We coded and analyzed the participants' qualitative explanations from the eight statements that relate most directly to the research questions of this paper, concerning the reasons behind learners' stated format preferences: 1. I remember information from my course readings best when I read them from printed pages. 2. It is more convenient to read my assigned readings electronically than to read them in print. 3. I prefer to have all my course materials in print format. 4. I prefer electronic textbooks over print textbooks. 5. I prefer to read my course readings electronically. 6. If an assigned reading is 7 pages or more, I prefer to read it in print. 7. If an assigned reading is fewer than 7 pages, I prefer to read it electronically. 8. Optional additional comments To gather data from a large international sample, local research partners translated the original survey into more than 20 languages and distributed it electronically to student populations in 33 countries between 2014 and 2017.
For logistical reasons, we excluded from analysis a large number of qualitative responses in this data set in languages other than English-including French, Norwegian, Italian, Chinese, Arabic, Spanish, and others. These comments do merit additional study.
Participants' comments were coded using an iterative, inductive approach into categories that were not preconceived but, rather, developed based on the data. The coding was conducted by two researchers independently, who then discussed and agreed upon a final classification and parsimoniousness together. Ultimately, comments were coded as pertaining to: Affordability; Convenience/Accessibility; Learning Engagement; Learning Quality; Tactile Properties; Physical Effects; Ecological Friendliness; and Knowledge/Comfort/Habit of Format. Additional codes for comments-Both, Depends, and Contradictory-reflect circumstances when students might use or prefer either format. Many comments were coded into more than one category.
Based on their responses, participants have been classified for analysis as either electronicpreferring overall (E-students); print-preferring overall (P-students); or neutral. This classification was determined according to the following procedure.
Response choices to the Likert-style statements on the ARFIS survey range from 1 (Strongly disagree) to 5 (Strongly agree). The authors calculated the median scores for each respondent to categorize them as overall preferring print or overall preferring e-formats for academic reading. The scores were calculated by assigning each response a score from 1 to 5. For statements where respondents were asked how well they like or prefer print readings, responses were scored: Strongly Agree = 5; Agree = 4; Neither Agree nor Disagree = 3; Disagree = 2; Strongly Disagree = 1. For statements asking how well they like or prefer e-readings, statements were reverse-scored. The result of this reverse-scoring procedure is that, overall, scores closer to 5 always reflect print preference, and scores closer to 1 always reflect e-preference.
The median response across all 12 validated Likert-style scale items for each participant was used to generate a median scale score for each participant. Participants with scale scores of 1, 1.5, or 2 are considered e-students-indicating that at least half of their questionnaire responses showed agreement or strong agreement with electronic preference. Those with scale scores of 4, 4.5, or 5 are considered p-students-at least half of their questionnaire responses indicated agreement or strong agreement with print preference.
Those with scores of 2.5, 3, or 3.5 are considered neutral-meaning that, across the questionnaire, their responses were either neutral or fairly well balanced across print-preferring and electronic-preferring expressions.
Across all respondents (n = 21,266), the final categorizations are as follows: • e-students: 1,843 (~8.67%) • neutral: 4,309 (~20.27%) • p-students: 15,110 (~71.06%) Out of the 15,110 p-students, 2,005 from 23 countries provided comments in English. We used the case selection function in SPSS to draw a random sample of 328 of these (16.35%) for analysis and compared them to the 325 e-students who provided comments in English (100%).
Demographics of the p-student sample show 69.8 percent (n = 229) are female; 79.1 percent are undergraduates (n = 260), and 78.5 percent (n = 204) of those undergraduates are first-year students. Their ages range from 18 to over 40, but 78.3 percent (n = 257) of this sample are 24 years or younger. About 21 percent (n = 69) state they have a visual impairment. Table 1 lists the countries and their n in this random sample.
The e-students' genders are split evenly: 50.2 percent (n = 163) male and 49.8 percent (n = 162) female. More than 68 percent (n = 222) are undergrads, 60 percent of whom (n = 133) are first-year students. The age range is also 18 to over 40, but only 65.3 percent (n = 212) are 24 years or younger. One in 10 (10.2%, n = 33) claim a visual impairment. Table 2 displays the countries and numbers of respondents who provided English language comments on their electronic format preference (n = 325).

Advantages of Learning in Print
Those who prefer print largely do so for reasons related to perceived ease of learning, although health and ergonomics, ease of access, and convenience are also reasons for preferring print among this population.
Respondents ascribed a sense of better or easier learning and information retention to a variety of qualities and affordances found in print formats. We coded 517 relevant comments from p-students describing the reasons for their preference. Almost 29 percent discussed the ease in which they can interact with print format, specifically highlighting and annotating (coded as learning engagement), and another 22.4 percent remarked that they learn better using print (coded as learning quality): • "…you can [actively] read, e.g. highlight and mark pages, move quickly from page to page if you need to refer back." • "I am a person that learns by writing, and so when I read I like to take notes on the sidebars… I can't do this on digital media." • "More visual tags that I remember; i.e. where on the page what I read stood, if it was at the beginning or the back of the book etc." • "I can learn better from a hard copy textbook than from an e-book." Besides better learning engagement and quality, 5.5 percent of comments reflected a preference for the tangible experience of print over electronic. As one student wrote: "It takes a different energy to continually read information from your computer; it is more relaxing reading printed pages. You can hold them and quickly go back and forth, there is also something psychological [sic] better about turning pages than scrolling and clicking the computer. I have a different absorption-concentration when reading printed material, it is better." Only four comments mentioned price as a benefit of print (such as "not everyone can afford internet") and nobody noted ecological friendliness as a reason for preferring print, although this was listed by some p-students as a positive feature of digital. Five students found information in print more believable and permanent, with comments such as: "I feel like it actually exists and notes can't be deleted or corrupted like that of digital base"; "I take it more seriously." More than 11 percent of the comments stated that print was better for their physical health, specifically citing their experiences with ergonomic issues, eyestrain, backaches, and headaches when reading for a long time on screen. A total of 92 comments (17.8%) report issues of inconvenience with digital technology, such as: • "A hard copy [print] is efficient and convenient because you can access it at any time without waiting for it to load. No skills required to use a hardcopy." • "Printed pages are portable." • "You can actually carry a hard copy with you and need no internet connection." • "I am a kinetic learner and need to study while walking around but e-format makes it impossible for me to study as I cannot walk around with my laptop." Figure 1 illustrates the reasons for preferring print uncovered in this study by percentage.

Advantages of Learning in Electronic Format
Of the 509 comments coded as reasons for e-format preference, e-students most commonly noted the convenience and accessibility of electronic format (34%, n = 173). Comments include these: • I love electronic format because I can access it from anywhere at any time.
• Printed pages [can] get lost.
• They are more convenient, save time, and can be accessed by any student as long as they have internet access. These students also discussed how digital features enhance their learning engagement (14.3%, n = 73), especially their ability to search for specific information using the "find" features, the ease with which they can look up references and relevant links, and how they have mastered the electronic highlighting and annotating features. Exemplary comments include the following: • "Printed resources lack the interactive aspect of learning which makes the learning process challenging at times." • "It is much easier to search for keywords and highlight content in digital form."

FIGURE 1 Factors behind Print Preference
Other favorable aspects of electronic formats include: • Lower costs: "Printed books are over-priced for the amount of time they are used." • Environmental friendliness: "I want to protect the environment," "Save the trees!" • The inconvenience and possible health issues involved in carrying heavy books: "The weight alone will hinder my mobility," "Print textbooks are large and heavy." Only 25 comments (4.9%) reflect students' belief that they learn just as well or better with electronic than print.
One student's comment of why he/she prefers electronic format summarizes well the most common responses: "I don't like books because they are: heavy and bulky (most of the time); easy to forget/not pack into your bag (I never forget my electronic device); expensive when compared to most digital formats; no easy way of doing word searches for assignments. I can't damage/ruin [e-format], even if electronic device breaks, the digital book is still on the cloud. I can easily make notes on [digital], without having to worry about resale value." Factors for preferring electronic format mentioned by E-Students are illustrated by percentages in Figure 2 above.
To facilitate an easy comparison of the most prevalent reasons reported by students in this study for preferring print or electronic formats, we have created Table 3 below.
These findings are consistent with other studies of students' format preferences and support the universality of students' perspectives of using print and electronic formats. 20

Compromises
Qualitative analysis of comments from our sample of 328 p-students and 325 e-students also demonstrates that preferences do not always predict behaviors, nor are they without exceptions. Comments by students in both sample groups acknowledge positive aspects of their less preferred format, and many write of specific circumstances under which they will switch. Therefore, the real behavioral question is not "Do students prefer print OR electronic?" but "WHEN do they prefer print and WHEN electronic?" To explore our research question, "Under what circumstances do they read a text in their less preferred format?" we analyzed the comments in Both, Depends, and Contradictory categories. Fully 197 p-students' responses and 192 e-students' responses were coded into these categories.
Among the p-students, we find comments such as these: • "Depends on the course and content amount." • "There isn't really much difference." • "Some readings are better read in print while others are suited to electronic format." • "Electronic readings are very helpful and they save time and effort, but I still prefer my readings in print." • "I like to use both but I find it difficult to highlight important information when I am online." • "I like to have both options. I retain more information if I read the material in print but I use the time spent in trains and buses to read and revise using electronic copies." Students in this group are more open to using electronic format when considerations of time, convenience, and cost override considerations of learning engagement and desired outcomes of a specific learning task. Their responses to the questions regarding impact on the length of the reading show more format flexibility than the e-students; 47 percent expressed a willingness to read short texts electronically if they were of less importance, relevance, and difficulty. Some p-students described concerns for ecological issues as well.
E-students also included comments showing the belief that print can be better under some circumstances and for specific purposes. The most prevalent example is the preference of print textbooks over e-textbooks as illustrated by nearly 16 percent of the 76 students who commented on the statement "I prefer electronic textbooks." Typical comments include "I normally prefer electronic readings but textbooks are the exception." Another 17 responses to this statement were coded as Both or Depends. While this is the strongest example of when estudents would prefer print, other comments sprinkled throughout the survey indicate some flexibility and willingness to compromise.
For most e-students, the length of reading makes no difference on their preferred format. Only about 27 percent state conditions in which using print would be "okay" such as specific types of reading (such as handouts or slides), if the digital format is difficult to read (like a blurry pdf), or if it lacks learning engagement features (such as highlighting). Among this group are also students who state that shorter readings should be available in print and longer readings online.

What patterns found among the responses can assist librarians and educators when deciding upon format policies?
Two findings among the demographics of these random sample populations are particularly noteworthy. There are an equal number of female and male e-students, whereas nearly 70 percent of the p-students are female. The latter percentage is representative of the gender distribution among the entire ARFIS population, 21 indicating that males may be more prone to preferring electronic format than females. Male respondents in this study group are overrepresented in graduate/post-baccalaureate ranks, so the gender difference may appear because of academic level effects rather than any intrinsic gender differences related to learning or format preference. Additional study is recommended. There is an 11 percent difference between the numbers of undergraduates in each preference group: 79.1 percent of p-students and 68.3 percent of e-students. As reported in Mizrachi et al. 2018, 22 the association of format preference with academic rank is statistically significant, and linear, with higher academic ranks showing smaller majorities of print preferrers, although the effect size is small. We might reflect on whether the lower undergraduate years of tertiary study are more likely to feature types of reading that seem linked to print preference, such as the use of textbooks.
Analysis of the comments is consistent with earlier studies showing that students prefer print primarily because they believe print facilitates better learning engagement and outcomes with less detrimental physical effects such as eyestrain and headaches. Electronic format is preferred more for its perceived lower costs, greater accessibility, convenience, and environmental friendliness. Some p-students admit to compromising with electronic for the same reasons, and many demonstrate a willingness to read shorter texts online. We found it common for e-students to prefer electronic format for everything except textbooks.
In general, participants here prefer print for reasons related to learning, and prefer electronic formats for reasons related to cost and convenience. Comments from p-students show a higher priority for learning outcomes and engagement over convenience and cost when choosing a format. E-students did display enthusiasm for engaging with digital texts using the "find," "search," and other features not possible with print, but only a minority actually said they learn better with electronic, and many more stated that format does not make a difference in their learning. Logistical factors of convenience and cost appear to be the primary drivers of their digital format preference. One e-student stated, "If I can find it electronically, I read it electronically. If I can't, I often don't read it." We thus see a schism in which many students must choose between better learning tools or better economics and convenience. These differences carry a potential impact on individual learning outcomes and disrupt efforts for equal educational opportunities.
With the priority that some e-preferring participants place on cost and convenience, along with lesser attention to learning concerns, we might also be seeing evidence that these readers are electing formats that are "good enough" for the objective at hand. In their theory on student approaches to learning, Ference Marton and Roger Säljö originate the concepts of "deep" and "surface" approaches 23 -identifying that learners might approach a task with a goal of deep learning, or alternatively, with the sole purpose of meeting a measure, such as passing a test or getting a grade. Could a student's format preference be a reflection of their learning approach to a specific task, or perhaps a more generally strategic approach to academics? This could be an area for further study.
Related to this, we do not know these students' socioeconomic means, academic success levels as measured by course grades and assessments, nor personal circumstances outside the classroom, such as employment or family obligations, that may also affect their behaviors. Yet, we can see that these factors do influence format choices. One student wrote that they do not have internet at home, for example; another stated that they have a smartphone, but no personal computer or laptop. These logistical circumstances directly affect readers' format choices, sometimes resulting in compromise with the format that is preferred for learning.
Ergonomic issues also exist and do not affect all participants equally. More than one-fifth (21%) of p-students noted visual impairments, and approximately 10 percent specifically noted eyestrain, headaches, and other physiological reasons for preferring print over electronic. Estudents with visual impairments appreciated the ability to zoom or otherwise enhance digital text; and e-students generally noted the negative effects of carrying heavy books all day. These differences reflect the varying affordances of different software and hardware combinations for digital reading, as well as individualized differences in readers.
We must therefore consider the evidence that individual characteristics, task circumstances, and context-driven logistics such as cost and convenience weigh more heavily in determining format preferences and behaviors than do broad technological or sociocultural factors.
Results from the research at this time cannot presume to offer general guidelines to librarians and educators seeking to determine collection and format policies and practices. Instead, we must recommend that individual institutions look inward and explore their own unique learning cultures and the specific characteristics of their student populations to determine which approaches will serve them best. We know that learning from long-form, informational reading is better supported by print today; but this may be moot if learners have to travel long distances or pay great sums to acquire print. We know that, often, short-form, leisure, or quick-reference readings are perfectly well-supported by digital formats; but this may also be problematic if readers struggle with ergonomic issues related to the format and do not have access to optional printing services.
Does your institution serve distance students? Do learners have visual limitations? How expensive are local printing services? How are textbooks acquired and distributed, and who bears those costs? Are internet access and digital hardware widely available and affordable for the learner population? Are academic reading materials primarily long-form readings or short articles? The data show that localized, contextual considerations such as these are likely to provide practitioners better insight into the merits of print or digital reading for any given reading event than broad generalized recommendations.
Ane Landøy and Almuth Gastinger in Norway 24 and Xiao Peng, Pan Yantao, and Zhang Jiuzhen in China 25 describe their progress in this direction by conducting focus groups and more in-depth surveys of their locally situated students and staff. Our next research steps involve consolidating existing data on reading format preferences into a decision model that would facilitate the type of local population factor review necessary to inform classroom and collections decisions.
We must also consider results from the many studies investigating the effects of formats on learning and cognitive functions, which at this time primarily show print as being more effective. How do we resolve possible discrepancies between cost/convenience and the students' optimal learning experiences? How do we build inclusive educational environments that consider the learning needs of all our students? Slightly farther afield, how can such findings inform the development of better educational and reading technology, from both software and hardware ends? Understanding our students' preferences and behaviors is an essential start on this journey toward digitally responsive pedagogy, information practice, and technology development.