{'fraction_non_alphanumeric': 0.01977979148397155, 'fraction_numerical': 0.01091298840494982, 'mean_word_length': 5.477437677500789, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Writing a Literature Review Narrative

The purpose of this paper is to present an approach for translating the copious background material gathered from the professional literature into a coherent and compelling written document. The approach includes steps for weaving together summary information from each study in a way that offers the reader a logical and clear description of pertinent methodological and conceptual issues and the implications for further research. The step-by-step approach enables both the novice and veteran researcher to translate the details of each study into a cogent synthesis that leads to determining the base of knowledge about the topic.


Introduction
The first phase of any research study involves identifying an important and current problem. The next phase involves exploring the prior research on the problem. No study is worthwhile if it does not fill a gap, address an inconsistency, or extend on what is already known about the phenomenon of interest. The role of literature reviews lies in the necessity to situate a new study in the sweet spot between what is known and unknown about a problem.
Two major challenges face researchers who are confronted with the task of conducting a comprehensive review of the literature that is pertinent to the problem they plan to investigate. The first is making sense of the abundant number of published studies that are available on almost every contemporary problem. The second is presenting findings in a narrative form that both describes key issues in the body of research literature and makes a compelling case for the purpose and methodological approach of a new study. In a previous article, I provided an in-depth description of the first (Schirmer, 2018). In this article, I will present an approach for the second. Let me begin by first summarizing the technique I offered in the previous article for translating the copious background material gathered from the professional literature into a coherent and compelling written document.

Identifying Patterns and Trends in the Research Literature
The goal of the next phase of the literature review process is to characterize the patterns and trends in the research literature. The first step in attaining this goal involves identifying the key components of each research study. These include the purpose and research questions/hypotheses, methodological design, participants and setting, procedure, intervention (if appropriate), measures, data analysis approach, findings, conclusions, and implications. The second step is to compare each of the components across studies to identify commonalities. These commonalities offer patterns and trends for organizing the studies.
For example, Lockman and Schirmer (2020) found five patterns in their review of the research literature published between 2014-2019 on the practices associated with effective online learning. The patterns they identified were course design factors, student support, faculty pedagogy, student engagement, and student success factors. By arranging and rearranging the corpus of 104 studies according to similarities in addressing each of these patterns, the final organizational framework enabled Lockman and Schirmer to discuss and critique the 104 studies grouped by common features.

Applying an Organizational Framework for the Literature Review
Once the patterns are identified, the studies that fit into each of these are slotted into the appropriate ones. Each pattern comprises a heading under which the pertinent studies are discussed. The pattern is introduced in a few sentences, every study is summarized in its own paragraph, and a final paragraph synthesizes key findings across the studies for that pattern. For example, Figure 1 shows how Lockman and Schirmer (2020) introduced the pattern, summarized one of the studies, and then synthesized key findings for the pattern.

Introduction to Student Support Studies Pattern
A substantial body of the recent research literature has involved support for students within online coursework. Though regional accreditation of institutions of higher education in the U.S. requires that student in online coursework receive equivalent access to support services as those attending classes on campus, these studies have addressed the challenges of offering comparable support within virtual modes and the researchers have sought to identify the specialized support needed by students in fully online classes. (Lockman & Schirmer, 2020, p. 133) One Article Summary Yilmaz and Keser (2017) conducted a mixed methods qualitative and correlational study to investigate the impact of interactivity and metacognitive support on the academic achievement of 127 college freshmen enrolled in an online computer course. Measures included an achievement test that covered the information security content in the online course, self-perception scale of course interactivity, and survey of students' opinions of the online learning environment. Based upon ANCOVA and t test for the quantitative data and qualitative coding of the open-ended questions, no significance was found for interactivity or metacognitive support on course achievement.

Summary of Student Support Studies Pattern
Student support studies have involved identification of categories of support needed by online students and relationships between these factors and student success. The factors found to be important to student achievement in online coursework include orientation to online instruction prior to beginning coursework, quality of faculty-student interaction, academic support for students, and establishment of trust and a sense of community in the online course. Trust was identified as particularly important in one study that included the perceptions of students with learning disabilities. (Lockman & Schirmer, 2020, pp. 133-134).

Narrativizing the Summaries
If each pattern only included summaries of each study, it would read like an annotated bibliography. To create a narrative, the summaries must be weaved together in a way that offers the reader a description of pertinent methodological and conceptual issues and the implications for further research. The narrative must flow logically and lead to identification of the gaps, International Journal of Education ISSN 1948-5476 2021 inconsistencies, and flaws in the body of prior research.
To narrativize the summaries, between 2-5 studies that cohere are brought together into one paragraph. Key points are kept and unimportant details are discarded from the original summaries. In some cases, as the first example in Figure 2 demonstrates, studies are grouped within support citations for overarching statements. In other cases, shown in the second example in Figure 2, greater detail for individual studies is provided.
The voice used in the narrative should reflect the primacy of research findings, secondarily the informed opinions of the researchers about implications for further research and practice, and, lastly, the informed opinion of the authors of the literature review about what is known and not known about the phenomenon. The voice within literature reviews is achieved through phrases such as, "It was found" and "Results have shown," unlike comparable phrases in book and white papers that are designed to lecture the reader such as, "We know" and "Teachers should."

Example of Grouping Studies
Researchers have also examined the relationship between specific support factors and student performance in online courses. Motivational support, cognitive learning strategies, social support, and sense of community have been found to predict student engagement and course achievement, but no significance has been found for interactivity and metacognitive support (Park & Yun, 2017;Yilmaz & Keser, 2017;Vayre & Vonthron, 2017). Wang (2014) found that students' perceptions of trustworthiness in online coursework was influenced by prior positive experience and good reputation of the online learning system or instructor, good accessibility and usability of the online learning system, instructor responsiveness, sense of care and community created by the instructor, and understandable privacy and security policies.. The feedback from the students with disabilities showed that because of trust issues, the majority initially held reservations about requesting accommodations. (Lockman & Schirmer, 2020, p. 133)

Example of Greater Detail Per Study
Two studies have involved experimental investigations of support interventions. Fricker (2013) examined the effect of a dedicated graduate teaching assistant on student retention and course completion. The teaching assistant provided tutoring and monitoring on an allegedly 24/7 basis. Fricker reported that 19 of the 20 students completed the course with a C or better and credited the support provided by the teaching assistant. Taylor et al. (2015) investigated the influence of a video course orientation tutorial on online course completion rates. Their results showed significant improvement in grades and reduction in withdrawals for only some of the courses. (Lockman & Schirmer, 2020, p. 133).

Figure 2. Examples of Narrativizing
International Journal of Education ISSN 1948-5476 2021

Critiquing the Research Literature
Literature reviews are half done if only a synthesis is provided. It is also essential to critique the literature to pinpoint the weaknesses that future studies should address. Critique takes place at two levels, individual studies and the body of research. Individual studies are best critiqued within the discussion of each pattern, as shown in the first two examples in Figure 3. This critique should address weaknesses in methodology, articulation of theoretical frameworks and conceptual orientations, logic of methodological design for answering the research questions, relevance of the sample populations, pertinence and trustworthiness of the measures, sufficiency of data, appropriateness and thoroughness of the data analysis approaches, explicit and logical chain of reasoning in arriving at results, sufficiency of evidence for conclusions and implications, and thoroughness of descriptions for enabling replication of the study.
Critique for the body of research literature should address issues of trends in the application of methodological designs among the studies, the presence of replication studies to verify or dispute previous findings, and whether there is an obvious line of research inquiry in which a body of knowledge is developed from study to study. An example is provided in Figure 3.
To distinguish between critique aimed at individual studies and critique of the body of research literature, the heading "Methodological Considerations" is helpful for separating these two sets of critique. The importance of this section cannot be underestimated because it is overall critique that offers researchers with guidance about flaws in the body of research that should be addressed in future studies.

Example 1 of Critiquing Individual Studies
Zimbardi et al. (2017) examined the effect of students' use of feedback with subsequent performance on similar tasks. They found that the immediacy and usefulness of the feedback had the greatest impact on improvement with similar tasks, and students who showed high levels of interaction with the feedback made the greatest improvement. Alternatively, Webb and Moallem (2016) found little direct effect of feedback on student performance, though they concluded that instructor feedback should be frequent, timely, motivating, informative, and precise. They also recommended that feedback should be written, verbal, and dialogic. (Lockman & Schirmer, 2020, p. 135)

Example 2 of Critiquing Individual Studies
One focus of the research on student engagement has involved examining the effect of student interaction patterns on course satisfaction. Results have shown no differences between patterns of student interactions in face-to-face and online sections of the same course (Almeda et al., 2018), a significant relationship between interaction and student confidence in performing Internet-related tasks (Kuo, 2014), and significant correlations between student satisfaction and learner-learner interaction, learner-instructor interaction, learner-content interaction, and student social online presence (Alsadoon, 2018;Kuo, 2014), Unlike Kuo's results, however, Gameel (2017) found that only learner-content interaction positively predicted learner satisfaction and concluded that interaction with content was more important to students than their interaction with instructors or peers. (Lockman & International Journal of Education ISSN 1948-5476 2021 Schirmer, 2020, p. 137)

Example of Critiquing the Body of Research Literature
The methodological designs in the body of research literature on online teaching predominantly have involved the identification, comparison, and correlation of variables potentially pertinent to online learning effectiveness. A small body of studies involved the experimental manipulation of variables. Investigations using qualitative designs were usually part of mixed-methods research. Most studies addressed gaps and weaknesses in prior research though the designs were not always linked to a theoretical framework.
Essential elements for quality research were applied to the corpus of research studies to evaluate the extent to which the body of research on online instruction met standards established by pre-eminent professional organizations including the American Educational Research Association (2006), Council for Exceptional Children (2014), and Institute of Education Sciences and National Science Foundation (2013). Regardless of the methodological component we evaluated, we found that it was rare for the authors to provide enough information to enable replication of the study.
Participants were almost always selected from convenience samples and so diversity reflected the ages, genders, ethnicities, and countries of origin of the sample rather than an effort to seek diversity that represented the population at large. Our corpus included just one study in which the authors disaggregated data for students with disabilities and one study in which the sample was chosen explicitly to represent students at-risk academically. Authors rarely justified sample sizes with power analyses for quantitative studies or data saturation for qualitative studies. The context for the studies was infrequently described in detail and some authors offered no description of the setting. For studies employing a qualitative design, the role of the researchers in gathering data was seldom explained. Though descriptions of procedures are often abbreviated in published research studies because of journal page limitations, few studies included even brief detail on the chronology followed in carrying out the investigations.
Measures for the quantitative studies were almost always identified, described clearly, and appropriate for answering the research questions. For the many studies that utilized survey questionnaires as data sources, there was typically little information provided about trustworthiness to reliably and validly capture the constructs and phenomena of interest or assurance that the individuals who completed the questionnaires represented the targeted respondents. For the correlational studies, potentially intervening variables were not identified during data collection. Unlike the measures for the quantitative studies, the procedure for collecting interview and other data for the qualitative studies was rarely described in detail. Specifically, descriptions of qualitative data collection seldom included interview protocols or sample questions, observational protocols and schedules, use of recording devices or field notes, document analysis, or establishment of chain of evidence. Thus, the qualitative data in these studies were not obviously dependable, trustworthy, and adequate for answering the research questions. (Lockman & Schirmer, 2020, pp. 147-148) ISSN 1948-5476 2021

Writing Conclusions for Literature Reviews
Whether the literature review is part of the rationale section of a research study or a freestanding synthetic literature review, the conclusions offer the opportunity for the researchers to summarize what is known and still unknown about the problem, identify findings that offer compelling versus promising evidence, and describe weaknesses in prior research that undermine the applicability of outcomes for professional practice. Conclusions should lead logically to the purpose of new studies that will advance knowledge and practice on the problem.
For example, Lockman and Schirmer (2020) drew several conclusions about the research on the effectiveness of instructional practices for online courses in higher education. One was that most of the strategies with promising effectiveness in the online environment are the same ones that are considered to be effective in face-to-face classrooms. Another is the dearth of studies utilizing methodological designs from which claims of causality could be made or metaanalyses could be conducted. Instead, they found "a mélange of promising practices with insufficient research to support guidelines for online pedagogy, student academic support in online modes, faculty professional development, and future directions for the role of online instruction in higher education" (p. 148). They concluded that findings pointed to few implications for practice but many paths for future research such as pedagogies and learning resources that promoted critical thinking, problem solving, communication skills, self-efficacy, and creativity.

Differentiating the Features of Literature Reviews from Research Papers
For novice researchers, the distinctions between literature reviews and research papers may be unclear. The reason is that from elementary school through graduate school, the term "research" almost always refers to research papers. The quality indicators for research papers are substantially different than those for reviews of the research literature.
• Research papers begin with a thesis statement. It is a statement of the central argument that establishes the purpose and position of the paper. Though the thesis can be in the form of a question, it is not a research question but rather sets forth the focus of the paper and stance of the author. Literature reviews begin with a statement of the problem. When written as a freestanding synthetic literature review, the research question shapes the scope of the literature search.
• Research papers are organized by the points the author wants to put forward, with each point supported by primary and secondary sources that can reflect a mix of opinion and research. Literature reviews are organized by patterns and trends discerned in the body of research literature. Primary sources are used predominantly, and clear distinctions are made between findings supported by research evidence and those based on opinion.
supports the author's argument. The voice used in literature reviews is objective. Bias is avoided throughout, from selecting the studies to be reviewed to disclosing disparate and contradictory findings.
• The arc of a research paper involves the thesis statement, evidence for the thesis, and summary of the main points that support the thesis. The arc of a literature review involves a rationale for the importance of the topic and scope of the review, analysis of primary research studies, discussion of findings from similar studies, critique of the body of research literature, and implications for future research.

Conclusion
The approach presented in this article can support researchers in presenting literature review findings in a narrative form that both describes key issues in the body of research literature and makes a compelling case for the purpose and methodological approach of a new study. This article builds on the first set of activities that include synthesizing and critically analyzing what is almost always a copious number of research studies and identifying gaps and inconsistencies in the body of research that lead to the next logical steps of research inquiry into the topic. The step-by-step approach enables both the novice and veteran researcher to translate the details of each study into a coherent synthesis that leads to determining the next step for research about the topic.