{'fraction_non_alphanumeric': 0.02911617961511048, 'fraction_numerical': 0.012045616535994298, 'mean_word_length': 5.015219721329046, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
State of Research on Gamiﬁcation in Education: A Bibliometric Survey

: Recent years have brought a rapid growth of scientiﬁc output in the area of gamiﬁcation in education. In this paper, we try to identify its main characteristics using a bibliometric approach. Our preliminary analysis uses Google Scholar, Scopus, and Web of Science as data sources, whereas the main analysis is performed on 2517 records retrieved from Scopus. The results comprise the cross-coverage of databases, geographic distribution of research, forms of publication, addressed research areas and topics, preferred publishing venues, the most involved scientiﬁc institutions and researchers, collaboration among researchers, and research impact. The main conclusions underline the sustained growth of the research output in the area for at least seven years, the widespread interest in the area across countries and branches of science, and an effective research communication in the area documented by the number of citations and the map of co-citations.


Introduction
Since gamification rose to popularity in the early 2010s, it has become an object of interest for education researchers. A short but good explanation of this interest was provided by Karl M. Kapp, who not only called gamification "the ideal process for creating engaging learning environments" [1], but also defined it as "using game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems" [1], implicitly pointing out its key virtue: the ability to build engagement and motivation.
In 2013, Simone de Sousa Borges et al. retrieved a total of 357 publications querying five bibliographic databases with the keyword "gamification" alone [17]. At the moment of writing these words, Google Scholar alone returns 19,000 results for the very same keyword. With such a flood of new knowledge, it becomes more and more difficult for a researcher to keep track of the state of gamification research even in his/her particular area of interest, such as education. This explains the demand for various kinds of literature surveys, providing a more or less comprehensive picture of the field. Searching for the keyword "literature" within the results of our main query on gamification in education (see Sections 2 and 3), we were able to identify 22 publications in this vein that employed a systematic approach to data collection and covered more than 10 publications. They are listed in Table 1. The Scope column defines the subdomain to which a given study was restricted, the Items column provides the number of publications that were eventually analyzed (after filtering out the ones considered irrelevant), the Databases column lists the data sources used (for the sake of brevity, dataset details were omitted), and the Year column specifies the last year covered in a given survey. The table is sorted upon the number of covered publications. Table 1. Prior literature surveys in the area of gamification in education.
Considering the aim of the research, we state the following research questions: RQ1. How has the scientific output with regard to gamification in education developed over time?
RQ2. Which countries contribute the most to the scientific output with regard to gamification in education?
RQ3. What publication types are mostly used to convey new research results with regard to gamification in education?
RQ4. In the context of which research areas and topics are the problems of gamification in education addressed?
RQ5. What are the publishing venues through which the results of gamification in education research are most often disseminated?
RQ6. Which scientific institutions are involved in the research on gamification in education?
RQ7. Who are the most prolific authors contributing to the research on gamification in education?
RQ8. Is there a wide collaboration among researchers of gamification in education? RQ9. Are the results of gamification in education research widely acknowledged or do they only reach a small groups of followers?
In the following section, the methods and data sources used to answer the stated questions are described. The obtained results are presented in Section 3, and discussed in the final section.

Materials and Methods
In the context of both the stated research questions and the volume of data to be processed, we decided to apply the bibliometric approach to achieve our goal. Bibliometrics, according to Nicholas and Ritchie [39], is "the statistical or quantitative description of literature", with "literature" understood as "a group of related documents". The exemplary characteristics of literature that may be described are "subject, document form ( . . . ), language ( . . . ), date" [39]. Surveys based on the bibliometric approach have been proven feasible and effective in various research areas, including business studies [40], medicine [41], tourism research [42], and education sciences [43].
Three stages of the survey procedure have been defined: 1.
Survey planning: Selection of data sources and specification of search criteria.

2.
Preliminary analysis: Answering RQ1 and selecting the data source which provides the widest coverage of the scientific output with regard to gamification in education for further analysis.
The necessary planning of the survey consisted in the selection of data sources for preliminary analysis and specification of search criteria. All data sources listed in the Databases column of Table 1 were considered for inclusion in the survey. Eventually, three sources were selected: Scopus, Web of Science (Core Collection) and Google Scholar, for the following reasons: • they index the largest number of documents, far exceeding the other databases; • most of the content of commercial publisher databases (e.g., Springer Link and Sci-enceDirect) is indexed by at least one of the three selected data sources; • most of the other databases used in prior research (e.g., ACM Digital Library; AISeL, ASME Digital Collection, IEEE Xplore) are dedicated to specific topic areas (such as computer science, information systems, or mechanical engineering), and this survey was not restricted to any of these areas; • ERIC is a database dedicated to an area consistent with the scope of the survey (education); however, it does not provide citation data which automatically excludes its use in the main analysis stage. Note that a simple query for peer-reviewed publications on "education" and "gamification" resulted in 262 items, which, while a small fraction of the number of items retrieved from the other sources, makes it a source to consider in future literature surveys on gamification in education.
With the abundance of literature on the survey topic, we wanted more to avoid false positives in the search results rather than minimize the number of false negatives. Therefore, we decided to use just two keywords: "education" and "gamification". While we considered the inclusion of other terms, we abstained from doing so. In particular, "learning" was abandoned as denoting a wider concept than education, and "instruction" was abandoned as a term having multiple meanings, only one of them connected to education.
For the two bibliographic databases (Scopus and Web of Science), the following assumptions were taken into consideration: • a paper relevant to gamification should have the term "gamification" mentioned in its title, keywords, or abstract; • a paper relevant to education should either have the term "education" or "educational" mentioned in its title or keywords (we ignore abstracts here, as many papers on gamification in non-educational contexts mention education as an exemplary field of gamification in their abstracts), or be published in an educational context (in a journal, book, or proceedings of a conference dedicated to education).
The search was performed on a single day (29 December 2020). Having retrieved, respectively, 2820 items from Scopus and 1988 items from Web of Science, we defined additional constraints: • papers from 2021 should be excluded (to avoid the false impression that the results cover the scientific output from 2021, whereas only few publications with such a publication date were found); • papers not in English should be excluded (to avoid the false impression that the results cover scientific output in non-English languages, whereas only a small amount of such publications were found; a proper survey in this vein should include various national-level bibliographic databases); • only the publications in the type of book, chapter, journal article, and conference paper should be included (to exclude all publications such as reviews, editorials, etc. that do not convey new research results).
Consequently, the following search terms were used: As a result, 2517 items from Scopus and 1743 items from Web of Science were retrieved. The case of Google Scholar is different. First of all, we were not able to define search criteria that would strictly resemble the queries performed on the other two databases. This is a well-known problem in the bibliometric literature (see e.g., [44]). Trying to make the search criteria as similar as possible, we decided to use two keywords: "education" and "gamification" with the "allintitle" option. Note that the exact search procedure performed by Google Scholar is not known, but the reader should be aware that the mentioned option does not mean that among the 1040 documents retrieved using these criteria, there were only those exactly containing the given keywords in the title (in fact, many of them did not).
Then, additional constraints were added: patents were excluded and the maximum year of publication was set to 2020; it was also ensured that only English-language documents were retrieved. This resulted in the following search results link: "https://scholar.google.com/scholar? hl=pl&lr=lang_en&as_sdt=1%2C5&as_yhi=2020&q=allintitle%3Aeducation+gamification\T1 \textquotedblright. A total of 980 items were retrieved with it.

Preliminary Analysis
In order to address RQ1, first, the lists of publications retrieved from the three databases were combined using publication title and year as the matching key. Due to various notations of authors' names, we decided not to include them in the matching key, however as the chance of two same-year publications from two distinct authors having exactly the same title is low, the effect of this decision on the obtained results is negligible. Note however, that the omission of the other metadata in the matching key (for instance, the journal or book title) inflated the number of duplicates, as all publications having the same title and year are treated as one: while such a situation does not happen frequently, sometimes it does (e.g., a paper is published first in conference proceedings and later the same year in a post-conference book or as a journal article).
Moreover, as we were aware of notable differences in letter case and interpunction of titles obtained from respective databases, the titles were matched considering only the alphanumeric characters, all converted to lowercase. As a result, 3944 unique items were identified (out of 5240 total retrieved items). Table 2 shows the cross-coverage between three queried databases (given in the first column). For every row, corresponding to database D, the three columns (referred to as C2..4) list the share of publications retrieved from D that were also found in the database given in the respective header of C2..4 (note the denominator is the number of publications in D, hence the total of respective columns is not 100%). If the column denotes database D, the given number is the share of publications from D that were not found in either of the two other databases. The fifth column gives the share of items having duplicates in the same database. It should be reminded that the number of duplicates reported here results from the chosen data consolidation procedure (see the previous paragraph), and includes as duplicates many papers which are not actually duplicates. On the other hand, while the mentioned procedure helped to remove many actual duplicates, particularly from the Google Scholar results, we have found through manual examination of randomly chosen items that there still were multiple publications listed under several more or less modified titles (usually fragments of other metadata were included in the actual title, probably as a result of an imperfect automatic acquisition of metadata by Google Scholar). As we did not aim to ensure the quality of bibliographic data, we did not attempt to correct such issues. The values in columns C2..5 in each respective row sum up to 100%. The sixth column is not related directly to the preceding four, and shows the share of items retrieved from database D in the combined list of 3944 unique items.
Looking at the obtained results, it is quite surprising-in the context of prior results such as those reported in [45], which indicated similar coverage at least of Scopus and Web of Science-how large the share of publications unique to just one database is. Even if we look only at the two bibliographic databases which were searched using criteria as alike as possible, it reaches about half (Web of Science) or more than half (Scopus) of their respective content. This is an important observation for future literature surveys on the topic, indicating the significant differences in coverage of the databases. Figure 1 provides a visual answer to RQ1, showing the number of publications registered in respective databases in subsequent years. The items with no publication year indicated were omitted, as well as few publications having erroneous metadata listing year of publication before 2010 (even as early as 1982) whereas their actual year of publication was found to be much later (after 2010).
Looking at Figure 1, the earliest items retrieved from Google Scholar were published in 2010, from Scopus in 2011, and in Web of Science in 2012. The scientific output became considerable in 2013 and grew from that year on. According to data from both Scopus and Web of Science, the growth dynamics was high and continued until recently. The results obtained from Google Scholar paint a somewhat different picture: here, the growth was moderate and reached its peak in 2018.
Regarding the data for 2020, it must be taken into consideration that although the survey was performed at the end of 2020, there are many publications still in press which will have a publication year of 2020 (this applies to all three data sources), and there is a delay between when an item is published and when it is registered in a bibliographic database. This applies specifically to Scopus and Web of Science (especially the latter, having very long data processing periods, which explains the sharp drop in the number of publications for 2020). For the purpose of performing the main analysis, the largest data source, Scopus, was chosen, as it covers a sufficient share (over 60%) of relevant publications identified in any of the considered databases and provides the complete set of information needed to answer all stated research questions. It is also the only source which provides unique author identification numbers, helping to avoid mistreatment of a single author publishing under different names or name forms as well as multiple authors having the same surname and first name initials. To illustrate the advantages of such precise identification of authors, among the 24 most prolific authors listed in Section 3.2.6, if the counts obtained by assigning publications to authors by their surname and initials were used instead, only 13 authors would have their publication count unchanged, of which only nine would retain their rank, and four authors would not even make it to the list. Figure 2 provides a visual answer to RQ2, showing the geographic distribution of the research based on data retrieved from Scopus regarding location of institutions with which the publication authors were affiliated. Researchers from exactly 100 countries and territories contributed to the studies on gamification in education. While, predictably, the largest share of research (almost 13% of publications) comes from United States of America, Spain comes closely behind (almost 9%), followed by United Kingdom and Germany (both 5.4%) and then Brazil (4.2%), Portugal (3.3%), and Australia 3.1%. The last three countries that qualified to the top 10 are Italy (2.8%), Canada, and Malaysia (both 2.5%). The combined output of the top 10 countries constitutes 50.9% of the whole analyzed data set.  Figure 3 provides a visual answer to RQ3, showing the number of publications classified according to their types. As can be seen, over 63% of the analyzed items were conference papers, and a little less than 1/3rd were journal articles. Book chapters constitute less than 1/20th of the publications and books, about 1/180th of the whole set.

Addressed Research Areas and Topics
With regard to RQ4, Figure 4 shows the number of publications attributed to respective research areas according to the classification used by Scopus (note that the Social Sciences category includes education sciences). Only the areas with at least 10 publications were included in the chart, but there were publications found corresponding to every area defined by Scopus. Note that many publications were classified as belonging to more than one category.  Looking at Figure 4, Computer Science is the area presenting the most interest in gamification in education (about 1/3rd of the publications), followed by Social Sciences (about 1/4th), Engineering (about 1/7th), and Mathematics (about 1/14th). None of the remaining areas passed the threshold of 1/30th.
A more detailed information is provided in Figure 5, which addresses the question of specific topics covered by the research. It shows the map of co-occurrence of the most frequent keywords, generated using the VOS Viewer tool [46]. In Figure 5, the relative frequency of keywords is represented by their respective font size, the co-occurrence of keywords is represented with connecting lines, and the clusters of repeatedly co-occurring keywords are shown in the same color. Note that some keyword labels are missing for no other reason but the way VOS Viewer renders the graph.
Among the most frequent keywords, apart from gamification and education which were used in the query specification, four keywords passed the threshold of occurring in at least one percent of analyzed papers: motivation, serious games, game-based learning, and e-learning. Regarding the connections, apart from some easily predictable ones (e.g., learning-training, education-engagement, or gamified learning-intrinsic motivation) there are also some less obvious (e.g., virtual reality-medical education, blended learning-software engineering, or adaptation-computer science education).

Dissemination Channels
In response to RQ5, Figure 6 shows the seven publishing venues that were most often chosen by authors for dissemination of the research results on gamification in education. These include three conference proceedings series, three book series (also known to publish conference proceedings or post-proceedings) and only one journal. Together, these venues are responsible for 21% of the publications covered in the survey. The connected points depict the number of publications in each of these venues in respective years.

Most Involved Scientific Institutions
With regard to RQ6, altogether, there were 160 distinct institutions identified in the dataset retrieved from Scopus listed as an affiliation of at least one author. Table 3 lists the 20 most frequently encountered ones. Seven of them are based in Spain, which is quite a surprise even considering Figure 2. Similarly surprising is the fact than only one institution based in USA made it to the top-20 list.

Research Leaders
Addressing RQ7, Table 4 lists the authors of at least eight works covered in the query (their works combined constitute less than 1/3rd of the total analyzed output). There were 159 authors identified who authored at least three publications (this pertains to less than 3% of all contributors). While the most prolific authors are from Brazil, Portugal is the most represented country in the list with six authors.

Research Collaboration
Addressing RQ8, Figure 7 shows the map of the most prolific co-authors, generated using the VOS Viewer tool [46]. The number of publications is represented by the circle and font size of their respective author name, the co-authoring of publications is represented with connecting lines, and the largest clusters of authors working together are shown in the same color. Note that some author name labels are missing for no other reason but the way VOS Viewer renders the graph.  Several large clusters are easily distinguishable; only few of them are built around the most prolific authors, including (as shown on Figure 7): Isotani and Toda (pale red), Fonseca and Redondo (heather) and Antonaci (pale green); more such clusters include only authors who contributed a small number of publications, such as: Anastas and Brooks (red), Walsh and Jani (green), Whitehead and Briffa (blue), Osbourne and Whitman (iris), Hiltunen and Bauman (olive) or Walker and Lee (cerulean).

Research Impact
With regard to RQ9, Table 5 Figure 8 shows the map of co-citations for the 99 most prolific authors, generated using the VOS Viewer tool [46]. The number of 99 was fine-tuned to remove numerous outliers, with one large, one medium-sized, and just one small cluster remaining. The number of citations of an author is represented by the circle size, the co-citation of authors (a publication in which they are cited together) is represented with connecting lines, and the largest clusters of authors cited together are shown in the same color.

Discussion and Conclusions
By applying the bibliometric approach based on data retrieved from the Scopus database (and to a lesser extent, Web of Science and Google Scholar), and using simple visualization tools we were able to respond to all nine of the stated research questions.
Regarding RQ1, the presented results show that, since 2013, gamification in education is a vivid and quickly developing area of research, with hundreds of new relevant publications coming out every year. As the data for the last year (2020) are not reliable (many publications are still in the process of indexation in the databases), it cannot be said that interest in the topic has already passed its peak: the data from the preceding years argue for the contrary. This observation is supported with the prior literature surveys in the area mentioning an increasing number of publications (see Table 1 and the works listed therein).
As for RQ2, the presented results indicate that even though there are countries (USA, Spain, United Kingdom, and Germany) leading the research, it is not dominated neither by a single country or a group of them, and the interest in gamification in education is widespread as about half of countries and territories of the world contributed to the research in the area. These findings are consistent with the recent observations by Grosseck et al. [18].
With respect to RQ3, research results regarding gamification in education are mostly disseminated at conferences, and journals are the second choice of the authors. Book chapters and books other than conference proceedings are relatively scarce. Note that the domination of conference proceedings is much more notable compared to surveys covering a smaller number of publications, e.g. [19].
Considering RQ4, the notion of gamification in education has already spread to all the research areas defined by Scopus. The area most often dealing with it is Computer Science. This interest may be attributed to various reasons, such as:

•
Computer Science is a somewhat difficult subject of education, therefore the support for engagement and motivation offered by gamification is especially demanded (note also the high interest from the fields of Engineering and Mathematics); • Computer Science subjects are mostly taught using computers, making it easy to introduce gamification software in the process compared to subjects traditionally taught without the use of computers; • a considerable part of publications on gamification in education is devoted to presentation of new software tools, which, regardless of the area of education they are applied to, are often published in Computer-Science-related venues.
Regarding RQ5, consistently with the fact that the most preferred dissemination channel for the results of gamification in education research are conferences, the proceedings series are the dominating publishing venues for the area. The only journal that managed to attract a number of publications large enough to be listed among the conference proceeding series is "Computers and Education". This should serve as an indication for researchers coming to the area where to publish their results to reach the relevant audience.
With respect to RQ6, the interest in gamification in education is not limited to a handful of institutions, and those leading the field did not dominate it. As an illustration to the observed lack of dominance, the most frequent affiliation was one that none of the most prolific authors was affiliated with, and located in a country from outside the top 10 list (compare Tables 3 and 4 and Section 3.2.1).
Regarding RQ7, there is no huge gap in the number of publications between the leaders (the first spot is taken by S. Isotani who contributed with 21 publications) and the other authors, especially compared to other areas of research (consider e.g., the signalprocessing guru, Harold Vincent Poor's 1809 publications indexed in Scopus). This may stem from the fact that gamification in education is relatively new field of research. The fact that only less than 3% of authors contributed with at least three publications indicates that research on gamification in education is usually a short-time activity rather than an area of scientific specialization.
In the context of RQ8, the presented results show that a number of collaborating researcher groups of various size have been established, however inter-group collaboration has been scarce so far. This may be interpreted in the context of the results of RQ7: in a shorttime research there is little reason to reach out for collaboration outside of an established working group.
As for RQ9, both the high number of citations of the key works in the area and a large co-citation cluster involving multiple authors indicate that the research community acknowledges one another's work on gamification in education.
The results of RQ9 also support the correctness of the choice of search phrases: all the identified most-cited authors are actually contributors to the area of gamification in education. For a comparison, the list of top-cited authors in [18] lists such names as Deterding, Zichermann, Werbach, or even Deci, all of whom have important contributions to gamification research yet outside of the covered educational area.
In conclusion, we would like to underline the key observations from the performed survey: The fast growth of the publications in the area of gamification in education continuing for at least seven years. The worldwide interest in the area is indicated by the number of countries in which the contributing authors are based and the number of institutions to which they are affiliated. An effective research communication in the area is documented by the high number of citations and the large cluster of co-citations. We believe these findings can inspire new research in this field, both by confirming the ongoing interest in the area, and by revealing data such as the unexpected keyword connections, which allows new research questions to be pursued.