{'fraction_non_alphanumeric': 0.03279247673342104, 'fraction_numerical': 0.029235491887151, 'mean_word_length': 5.1819277108433734, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
BIBLIOMETRIC ANALYSIS OF ORIGINAL SCIENTIFIC PUBLICATIONS FROM THE INSTITUTO NACIONAL DE SALUD DEL PERU FROM 1998 TO 2018

Objetives: To measure the scientific production of the Instituto Nacional de Salud del Perú(INS) through bibliometric indicators of production, collaboration and impact. Materials and methods: Bibliometric study of original publications from journals indexed in Scopus and Scielo Peru in the period between 1998 and 2018. The production indicators were: the number of publications per year and per subperiod (1998-2008 and 20092018), by theme, and by journal. The collaboration indicators were: the co-authorship index, the number of institutional signatures, the national and international collaboration rate, the proportion of first-time authorship with INS affiliation. The impact indicators were: the number of citations per year, citation/document index, the proportion of documents ever cited and the citation speed. Results: A total of 618 publications had at least one author with INS affiliation. From these, 55.9% (346/618) were published in Peruvian scientific journals and 30 INS authors were major producers. From the publications total, 49.0% (303/618) had a first author with INS affiliation. Regarding the subject of investigation, 132 (21.4%) publications were on vector-borne diseases; 9.7% (60), on tuberculosis; and 9.5% (59), on zoonoses. The international collaboration rate was 38.8%. The ratio of citations/documents was 12.8 for the entire period. The H index of publications with INS participation was 37. Conclusions: In the last two decades, the contribution of the INS was focused on infectious diseases. On the other hand, it is required to strengthen the productivity indexes of INS authors, as well as consolidating the new thematic lines that they have developed in the last decade. This effort will allow the INS to improve its function as a public research institute.


INTRODUCTION
In recent decades, the scientific production of Peru's research centers has increased in all areas of knowledge (1) , including biomedical and health fields (2,3) . Bibliometric studies use statistical methods to quantify and describe the characteristics of scientific literature (4) . Bibliometrics have systematically characterized the scientific publications of Peruvian authors and institutions in defined time periods, by subject (5)(6)(7) and for the purpose of description, evaluation or monitoring (8,9) .
The Peruvian Instituto Nacional de Salud (INS) is a public research institute that is part of the Peruvian Ministry of Health (MINSA). Its mission is to promote and disseminate scientific research in the public health field, to improve population's living conditions. Some studies have quantified the scientific production of the INS. Between 2007 and 2011, the MINSA had 457 publications; however, it is not possible to identify in how many did INS participate (1) . An analysis in the Web of Science (WoS) database for the period 1997-2016 found 196 publications with INS participation (2) . Another study of Peruvian scientific production in medicine from journals indexed in WoS found 65 publications with INS affiliation between 2000 and 2009 (3) .

Motivation for the study:
The biomedical scientific production in Peru has been quantified, however, the production of the Instituto Nacional de Salud del Peru (INS) has not been evaluated.

Main findings:
The INS participated in 618 original publications. Over the years it has increased its production and has more international collaboration and citations, however, its leadership has decreased. The 3 most addressed topics were vector-borne diseases, tuberculosis, and zoonoses.
Implications: Peru needs a strengthened public health research institute with national and international leadership.

KEY MESSAGES
We did not find specific studies on the INS scientific production, so we lack information about INS publications, such as characteristics, trends over time and the impact of the citations. The two-decades analysis of these bibliometric indicators will allow to properly fill this knowledge gap (2). Due to the foregoing, we conducted a bibliometric study of the original scientific publications with the participation of the INS during 1998-2018.

Design and information source
A bibliometric study of the INS scientific production was carried out through the analysis of original publications in magazines indexed in Scopus and Scielo Perú during 1998-2018.
The Scopus database was selected because it includes all the journals indexed in Medline (4) and has an important collection of Latin American journals. It also facilitates the search by institutional affiliation and the identification of the authors' affiliations, which is necessary for the collaboration indicators (10) . Scielo Perú allowed the inclusion of publications in Peruvian journals not indexed in Scopus.
The search was conducted in October 2019. In Scopus, the search by affiliation to the "Instituto Nacional de Salud, Lima" was used (affiliation identifier: 60071247). It was filtered for the period 1998-2018; later the database was downloaded in .CSV format, according to the following variables: authors, document title, year, title of the source/journal, authors with affiliations, abstract, and document type.
Journals in the "Health Sciences" category from SciELO Perú were identified. At the time of the search, this base included 12 journals. In order to recover these publications, all the issues of the journals published in Scielo Peru's portal were reviewed. The digitized version of each publication was manually reviewed and those with at least one author with INS affiliation were obtained. This identification was carried out by two analysts with experience in bibliometric studies (DF and MC).

Analysis unit
The unit of analysis was the original publication. The following document types were considered for Scielo Perú's records: original article, short original article and short communication. Regarding Scopus, records were manually reviewed to validate them as publications that reported results of study a or research project, following the structure or not (introduction, methods, results and discussion).

Data base elaboration
664 publications were obtained from Scopus, 3 of them eliminated due to repetition, and 12 removed due to the lack of author with INS affiliation. After debugging, 649 records were left. Data from 621 publications was collected by the manual search of Scielo Perú.
On Scopus, the following indexed publications were found: 508 articles, 57 letters to the editor, 34 editorials, 33 reviews, 10  A consolidated spreadsheet was created for the original publications. Since the information exported from Scopus presented errors in the document classification, each record was manually verified, by reviewing the structure and information of the abstract or the full article, in order to validate it as an original publication. This process was carried out by two experienced collaborators in bibliometric studies. In case of discrepancies, the identification was made by the author.
The database of the original publications included the following variables: recovery database, name of scientific journal, year of publication, title of publication, abstract, authors, institutional signature, country signature.
The names of the authors were manually normalized due to incomplete data, spelling problems or different names for the same author. For the standardization of the institutional affiliation, the largest institution comprising a laboratory, management, institute, faculty or sub-unit was considered.
For the citation analysis, the number of citations per year for each record with at least one citation was collected in a spreadsheet. The citation count was performed only for the publications retrieved from Scopus through its Metrics tool. The publications retrieved from Scielo Perú were not considered because their citation-counting tool is still going through the development phase and it was confirmed that in many cases it does not provide accurate data.

Variables
The number of original publications per year of publication, scientific journal name and subject matter were the production indicators. The publication years were re-categorized into 2 sub-periods (1998-2008 and 2009-2018). The author assigned the thematic manually, by reviewing the title and abstract of each publication and taking as reference the National Priorities for Health Research in Peru 2019-2023 (11) . The productivity index was defined as the logarithm of the number of original publications per year.
The collaboration indicators were the following: co-authorship (number of authors/number of original publications), number of institutional signatures per document (number of institutional signatures/number of original publications), national collaboration rate (original publications with at least one institutional signature from Peru other than the INS/total number of original publications × 100%), and international collaboration rate (percentage of original publications with at least one foreign institution signature).
The number of authors was recategorized into single, two to five, six to ten, and more than ten. The number of institutional signatures was recategorized into only INS, two to three, four to five, and six or more. Journals were considered to be national (published in Peru) or foreign, based on their identification. The occurrence of a first author with INS affi-liation was used as an indicator of capacity for creation and initiative (12) , since it is the position with the greatest intellectual and operational contribution within the study (13,14) .
The following indicators are constructed from the number of citations per year: citations obtained per original publication, citation/original publication index, number of original publications cited at least once, and proportion of documents cited (4) . The speed of citation was defined as the time (in years) from publication to the first citation (10).

Data analysis
Bibliometric indicators will be presented for the entire study period and for the sub-periods 1998-2008 and 2009-2018. The bibliometric indicators of production, collaboration and impact will be presented in frequencies, percentages and central tendency measures. The characteristics associated to original publications with first authorship from the INS were evaluated. Proportions were compared with the Chi-square test, and the ratio of proportions was calculated with a 95% confidence interval. The H index was calculated for the INS production. The statistical analysis was carried out in Microsoft Excel and Epidat 3.1.

Production indicators
During the study period, 618 original publications with at least one author with INS affiliation were recovered from Scopus and Scielo Perú. A second-degree polynomial type growth was observed in the scientific production ( Figure 1). 55.9% (346/618) of the articles were published in Peruvian journals. Between 1998 and 2008, 26.6% (62/233) of the articles were published in international journals, and between 2009 and 2018, it was 54.5% (210/385). For the whole period, 136 different journals published articles; 77.8% of the publications were concentrated in 26 journals (Table 1).

Collaboration indicators
We identified 4,099 author signatures. For the entire study period the co-authorship index was 6.63. In addition, 80.3% of the publications had collaborations with authors from another institution. Between the two subperiods, the rate of international collaboration went from 27.9% to 45.5%, while national collaboration went from 65.2% to 71.2% (Table 3).         (8), other arboviruses (4), and acute febrile syndrome (4). c Includes subtopics: leptospirosis (14), rickettsiosis (10), echinococcosis (8), rabies (7), hydatidosis (5), plague (4), fascioliasis (3), brucellosis (2), toxocariasis (2), cysticercosis (2), Borrelia infections (1), and ehrlichiosis (1). d Includes gastroesophageal reflux, alternative medicine, climate change, zoology, vaccination, traffic accidents, clinical practice guidelines, genomics, pulmonary embolism, bioinformatics, microbioma. the purchase of goods and services for research activities (17) . The fewer studies generated, conducted and concluded, the fewer publications to disseminate their results. Approaches to INS scientific production were made in studies that measured national production. For example, between 2000 and 2009, 65 publications were identified in the ISI Web of Knowledge database (3) , while between 1997 and 2016, 196 publications with INS affiliation were found in WoS (2) . Both analyses recovered articles of diverse typology. In our study, taking into account only original publications,    we found four and three times the amounts reported for those periods, by the mentioned studies. This difference could be explained by the database used, WoS has selective criteria for the inclusion of journals. In 2014 the number of journals indexed in WoS was 12,000, while in Scopus it was 21,921, in addition, 11,738 journals were indexed in Scopus but not in WoS (10) . The productivity of INS authors follows Lotka's law. Almost half of them had low productivity, and 30 were the most productive, which means that a significant percentage of researchers worked on a research project and did not re-engage in the research-publication process; therefore, they did not continue the line of research initiated (15) . This characteristic does not allow the formation of experts in thematic lines nor does it encourage collaboration in these areas (16) . Another explanation would be the lack of an adequate publication culture. Between 2004 and 2008, only 18 of 129 INS research projects became a scientific publication (17) . Another study showed that 79 (26.7%) of 296 abstracts submitted to an international congress on tropical diseases culminated in scientific publication within 6 years (18) .

DISCUSSION
When it comes to biomedical research, authorship contribution is usually assigned in descending order: the first author is the one with the greatest contribution to the scientific activities of the study and exercises leadership during the project (12,19) . It was found that first authorship of INS-affiliated authors decreased in the last decade, which could be associated with the increase of the international collaboration ratio and a secondary role of INS authors. In collaborative studies, authors from low-and middle-income countries are often relegated in the order of authorship because their contribution is limited to recruitment of study subjects, data collection, sample referral, and other technical aspects (20,21) .
As is the case for other research institutions (22,23) , the INS international collaboration network expanded in the decade of 2009-2018. Institutional signatures were most frequently from the United States, followed by Brazil and Argentina. This trend has also been observed in similar institutions, such as the Fiocruz Institute in Brazil (16) . The frequent collaboration of Peruvian institutions with those from the United States has been reported in medical publications between 2000 and 2009; those that collaborated the most were Johns Hopkins University, the Center for Disease Control and the University of Washington (3) , the leadership of these U.S. institutions was also evident between 1997 and 2016 (2) .
International collaboration is key to scientific research in Latin America, including the field of public health. Between 1996 and 2011, scientific publications on public health and me-dicine had a 30% international collaboration rate, decreasing from 35.8% in 1996 to 24.5% in 2011 (24) . In our study, international collaboration was found to be close to 40%, with an increase in the 2009-2018 decade. International collaboration is variable in Peruvian institutions. In the 2006-2011 period, 74.4% of UPCH publications and 56.2% from UNMSM had international collaboration, percentages higher than the values (90%) seen in public research institutes (1) .
Articles where first authors are from the INS are published mainly in the RPMESP. This scenario is due to the fact that the INS is the publisher of this journal, it is published in Spanish and is the only Peruvian public health journal indexed in Scopus and Medline. A negative aspect of this preference is that the RPMESP is not indexed in WoS and does not regularly publish in English, which limits the citation possibility of its articles. In fact, the RPMESP achieved only 945 citations in the period 2010-2017 (9) .
Infectious diseases were the main topic, which is consistent with the two-thirds of the articles on this subject in publications with Peruvian participation between 1997-2006 (2) . In the field of infectious diseases, the growth of research on tuberculosis stands out, which production tripled in the studied subperiods. The large production in these topics respond to the fact that their authors came from the National Center of Public Health, a division of the INS dedicated to research on communicable diseases, where the greatest number of projects are produced (17) .
The INS scientific production covered most of the topics included in the national research priorities (25) and in the research agendas approved in 2009 and 2011, respectively. Despite this, there were some neglected topics, for example, maternal mortality did not have any specific research, or mental health which had 10 publications in 20 years, 9 of them in the last decade. The reduced scientific production on such topics is a problem in the country. An analysis of the national scientific production between 2011 and 2014 found maternal mortality and mental health, with 11 and 28 original articles respectively (26) , as the least studied topics.
As of 2011, there were over 300 citations per year, and 2017 had the highest number of citations for INS articles. This phenomenon could be explained by the indexing of the RPMESP in Scopus and by the increase of international collaboration and publications in international journals (27) . One third of the publications received at least one citation in the same year of publication, and almost half during the following year, this reflects a high speed of citation; that is, research results tend to be disseminated quickly to the scientific community and used in a short time (10) .
This study had some limitations: the citation analysis of all the original publications was not performed, only those obtained from Scopus, which would not reflect the real use and impact of these publications. This analysis supposes an appropriate attribution of authorship, which could be affected by the changes in the authorship criteria occurred in the two decades covered by this study. INS authors might have not recognized their institutional affiliation, and authors not related to the institution might have stated an INS affiliation (28) . The analysis of funding sources was not carried out, due to the heterogeneity of the reports, neither the analysis of indicators that evaluate dissemination, industrial production or the impact on health services or systems (29) . Finally, this analysis may not reflect all of INS's scientific activities, since near 85% of projects it undertakes are not published (17) .
The strength of this study is the analysis of original publications, which are the standard format for presenting results of scientific research about technologies, products, new concepts or hypotheses. Citation of these articles brings us closer to the effective impact of an institution's publications on the scientific progress (30) .
In conclusion, there is an increase of INS participation in original publications during the 2009-2018 decade, despite a drop in production in the last three years of observation. The contribution has been mainly on vector-borne diseases, tuberculosis and zoonoses. The growing international collaboration has generated more publications in foreign and increasingly cited journals, but it reduced the articles with first authors from the INS. It is recommended to consolidate international collaboration with greater INS leadership; to increase the amount of large producers; to consolidate the new thematic areas in which INS researchers are positioning themselves; and to increase the rate of collaboration at the national level, with equal care as at the international level.