{'fraction_non_alphanumeric': 0.02738043001316367, 'fraction_numerical': 0.010618692408951294, 'mean_word_length': 5.2615384615384615, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
An analysis of verbs used in the course outcomes of outcome-based integrated courses at a medical school based on the taxonomy of educational objectives

Purpose The purposes of this study were to analyze the course outcomes of integrated courses of a medical school that introduced an outcome-based curriculum and to suggest ways to improve techniques in the description of outcomes. Methods In order to investigate the course outcomes of 39 courses in the first and second grades at Kyungpook National University School of Medicine in Korea, verbs for the course outcome were classified according to Bloom’s taxonomy and frequency analysis was completed. The appropriateness of use of verbs in description of the outcomes was also analyzed. Results The total number of course outcomes for both grades was 497. The number of course outcomes per credit ranged from a minimum of 0.9 to a maximum of 22.0. Most of the course outcomes were classified as the comprehension domain according to Bloom’s taxonomy. The most frequently used verb was “explain,” accounting for 61.0% (n=303) of the entire course outcome verbs. Some verbs in the outcomes, however, were unmeasurable. Conclusion The major findings of this study include the followings. First, the number of course outcomes varied according to courses. Second, several course outcomes included unmeasurable verbs. Finally, most of the verbs used to describe course outcome belonged to the two lower levels of Bloom’s taxonomy—knowledge and comprehension. In order to improve the description of course outcomes, this study suggests that it is necessary to adjust the number of course outcomes, applicate overt behavior verbs, and elevate the level of course outcomes from the lower memorizing level to the higher application level.


Introduction
It is important to describe learning outcomes appropriately in an outcome-based curriculum. This is because the learning outcomes clearly state what learners should achieve at the end of the learning as well as what are to be evaluated [1,2]. To describe learning outcomes is an important education designing activity that determines education and instructional directions [1]. Therefore, a medical school that implements the outcome-based curriculum should first inform its members about its macroscopic educational directions by establishing exit and phase outcomes. Second, instructors should be provided with guidelines that help them to describe learning outcomes of their courses and lessons ap-propriately. Third, the instructors should be provided with appropriate education corresponding to the guidelines. It is also necessary for the school to review the learning outcomes described by the instructors and to provide feedback to them about the relevance of their descriptions [3].
In Korea, outcome-based education (OBE) began to be Since then, developing learning outcomes to operate OBE has become a mandatory task for all medical schools to meet the Accreditation Standards [3].
Based on this educational and social background, most medical schools in Korea developed exit outcomes, phase outcomes, course outcomes, and lesson outcomes [4].
Exit outcomes indicate final achievements that students at graduation should accomplish. Phase outcomes may differ between the schools. Achievements that students should accomplish at the end of a period, either classified according to grades or contents of education, are defined as phase outcomes. If periods were classified according to educational contents, for example, premedical, basic medical science, and clinical medical curriculum can be classified into phase 1, phase 2, and phase 3 outcomes, respectively. Course outcomes indicate achievements that learners should accomplish at the end of a course. Finally, lesson outcomes, or instructional learning outcomes, are the smallest outcomes that indicate achievements that learners should accomplish at the end of a class [3]. Developing such learning outcomes is the starting point of developing a successful outcome-based curriculum.
While developing learning outcomes, many instructors have been confused about how to describe learning outcomes that are distinct from learning objectives [4].
Although "learning outcomes" itself can be used as a substitute for learning objectives or lesson outcomes for practical use in development of outcome-based curriculum [4,5], several literatures point out the conceptual differences [4,5].
Conceptually, the learning objective refers to a collective statement that describes new abilities acquired by learners through a lesson [6]. It is a term mainly used in the era when schools operate "subject-centered" or "discipline-centered" curriculum. Learning outcomes began to replace learning objectives with the emergence of OBE in the 1970s and many universities in the United States started to emphasize "ability to perform" rather than "acquiring knowledge [2]." Learning outcomes are defined as the competencies of a learner to integrate knowledges, skills, and attitudes in an authentic context after achieving specific learning objectives [2,6]. In summary, the learning outcomes emphasize more comprehensive competencies of the learners to perform than learning objectives do and are used as a higher concept than learning objectives. Learning outcomes used in this study are defined as the term that can replace learning objectives for practical purposes and emphasized practical performance ability of learners more comprehensively than that of learning objectives.
Despite the difference in the concept of the two terms, the method of describing learning outcomes in medical schools has developed based on the method of describing learning objectives [1]. Different methods of describing the learning outcome were proposed by researchers including Tyler, Mager, Gagné, and Gronlund with different degrees of specificity of statements [7]. One of the simplest and most widely used methods is Tyler's method. This method expresses a learning objective with a statement composed of the subject as the learner, the object as the educational contents, and a measurable behavior verb as the learner's final behavior [7].
Kyungpook National University School of Medicine (KNUSOM) adopted Tyler's method in the development of its learning outcome guidelines. The most significant reason was that the simplest method was suitable to spread the concept of learning outcomes. Even with this simple method it is still possible to state the contents of learning outcomes sufficiently.
In this context, studies regarding what learning outcomes actually are and how to describe them have been conducted based on papers about the concept of competency [3], description of learning outcomes, and an association of learning outcomes and evaluation [8] in outcome-based curriculum. In other study fields, research has been conducted on the analysis of educational objectives according to taxonomy [9,10]. However, it has not been possible to find previous studies at medical schools that analyzed how appropriate the learning outcomes were written according to the guidelines although the description of an outcome statement is of paramount importance in outcome-based curriculum.
Therefore, this study aimed to suggest possible ways to improve strategies in the description of learning out- Analysis is the ability to disassemble the given material of a phenomenon into an organization, structure, or component and discover their interrelationships. Students will actually find patterns that they can use to analyze a problem. Synthesis indicates the ability to arrange, organize, and integrate the elements of a given material to establish new and creative forms, principles, relationships, and structures, or to make predictions. Evaluation is the highest level of Bloom's taxonomy which refers to the ability to assess information to reach to a conclusion or to judge values based on some criteria [7]. In this study, the reason why the categorization of learning outcomes was based only on the cognitive/ knowledge domain was because course outcomes of lecture-centered courses mainly involve contents of the cognitive/knowledge domain.
In addition, the reasons why the present study applied Bloom's taxonomy, which is treated as a traditional taxonomy of educational objectives [6,12], instead of the newly revised Bloom's taxonomy were as follows. First, Bloom's taxonomy of educational objectives is still being widely used in most educational fields [7,10,12]. Second, the relevance of the revised taxonomy is still controversial even if it supplemented the knowledge domain [10,12]. Third, although the revised taxonomy provided a more detailed high-level cognitive domain [6], the level of course outcomes analyzed in this study was rarely in the high-level cognitive domain so that it was unsuitable to apply the revised taxonomy as the criteria for the classification of course outcomes in this study.
Among the courses analyzed in this study, course outcomes of "patient doctor relationship," "medical

Number of course outcomes according to grades
This study identified 226 and 271 course outcomes in 16 first grade and 23 second grade courses, respectively.
The first grade curriculum included nine basic medical science, five clinical medical science, and two medical humanities courses and the number of course outcomes per course ranged from a minimum of three to a maximum of 25 ( Table 2). The number of course outcomes per credit ranged from 0.9 to 22.0 ( Table 2). The second grade curriculum included 21 clinical medical     (Tables 2, 3). The summary is shown in Table 1.

Classification of course outcomes according to Bloom's taxonomy
The classification of the total 497 course outcomes according to Bloom's taxonomy resulted in 33, 302, 58, 9, 24, and 71 outcomes in the "knowledge," "comprehension," "application," "analysis," "synthesis," and "evaluation" domains, respectively. The detailed results according to grades were showed in Table 1. Table 1

Analyzing types of verbs in the course outcomes
The frequency analysis of the verbs used in the description of 497 course outcomes revealed that 78 different Korean verbs were used and that eight of them were unmeasurable verbs ( Table 4). The top three types of verbs most frequently used were "understand (21 times)," "recognize (5 times)," and "acquaint (3 times)." Regarding course outcomes verbs, the most frequently used verb was "explain." Of the total 497 course outcome verbs, 54.9 % (n=273) used this verb.

Discussion
This study aimed to analyze how course outcomes were developed in the process of transforming the existing learning objectives into learning outcomes in the establishment of OBE. The primary goal of this study was to suggest ways to improve strategies in the description of learning outcomes and therefore help professors who are responsible for providing lectures to describe learning outcomes appropriately.
The major findings of this study include the follow- In further studies, it would be necessary to analyze the content of learning outcomes based on the results of this study. It is expected that the learning outcomes that are more objective and that are more related to exit competencies can be developed when the results of the present study are provided as feedback to the professors who actually develop learning outcomes.