{'fraction_non_alphanumeric': 0.03439011284255777, 'fraction_numerical': 0.016389038151531435, 'mean_word_length': 5.155733186328556, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Content Analysis of Research Articles in Information Systems (LIS) Journals

This article analyses the origins and data collection methodologies of research articles published in three Information System journals (Information Systems Research; Journal of the American Medical Informatics Association - JAMIA; and MIS Quarterly) between 2005 and 2007. A coding sheet was utilized to collect the relevant data for content analysis. Most articles were written by multiple authors based in the USA. Over 80% of the articles published in JAMIA were funded with over 40% for the other two journals. The data collection methodologies were diverse, but the most popular for all three journals was the questionnaire. A variety of statistical packages were used to analyze quantitative data while coding was the most popular method for analyzing qualitative data. These journals published more mixed methodological approach studies compared to previous research in this area.


Introduction
This paper is derived from the concept of Evidence Based Practice, particularly with relation to the type of evidence available. Eldredge (2000) suggests part of the Evidence Based Librarianship framework involves applying "the bestavailable evidence, whether based upon either quantitative or qualitative research methods". A problem with Evidence Based Practice (EBP) is the availability of high quality, non-contradictory, complete, relevant evidence. This study analyses the data collection methods used to produce the evidence and the analysis of the data by the authors presenting their results. The evidence should be of a high quality which means the research methodology should be sound. Articles that are derived from funded research provided by government agencies or departments, such as the NIH (National Institute of Health) were reviewed prior to the research being undertaken, as well as the article itself being reviewed before publication. This double-reviewing should ensure the research is valid and that the results The type of research methodology, data collection and analysis used depends on the research question being studied; the topic or subject area; the researcher's skills, experiences and background; as well as the expected or intended audience. There may be more than one valid method for investigating the same research question (Crumley & Koufogiannakis, 2002).
There has been extensive research on research methodologies in IS journals between 1977 and 2004, with a certain degree of overlap between the studies, as illustrated in Table 1 Avison et al. (2008) analysed 17 years of the Information Systems Journal, considering the origin of papers. Most authors were geographically located in North America (33%), followed by the United Kingdom (32.5%) and 20% from other European countries. Claver, González and Llopis (2000) found that 89% of authors published in MIS Quarterly were based at USA universities and the remaining 11% from Canada. Dwivedi and Kuljis (2008) considered the geographical source of articles published in the European Journal of Information Systems, which was perhaps unsurprisingly 57% from AIS (Association for Information Systems) region 2 (Europe, the Middle East and Africa), followed by AIS region I (America) with 28%. Avison et al. (2008) found most articles in Information Systems Journal were written by two authors (44%), single authored (32%) or by three authors (18%). Claver, González and Llopis (2000) found most articles in MIS Quarterly were written by several authors (76%).
There is broad diversity in IS research (Vessey, Ramesh and Glass, 2002), so unsurprisingly the most popular methodologies varied (Table II). Case studies and survey research are used in the real world that is in natural settings. Case studies are more suitable than surveys for understanding context, while survey research is appropriate for answering questions of what, how and why (Pinsonneault and Kraemer, 1993). Palvia et al. (2003) highlighted the growth in the reporting of case study research over the 1990s, particularly in the journals MIS Quarterly and Information & Management. Table 2 includes both case study and field study, though Palvia et al. (2003) note that the difference between these is not clear and the terms are used interchangeably. Laboratory experiments occur in controlled environments and enable a degree of control that is not possible in case studies (Palvia et al., 2003).   Chen and Hirschheim (2004). Avison et al. (2008) found only 6% of articles published in the Information Systems Journal used a mixed methods approach. Pinsonneault and Kraemer, (1993) advocated the use of more mixed methods research, particularly combining surveys with case studies and field observations. This would provide a wider range of data which could lead to a greater understanding of the results obtained.
Two studies analysed research funding. Research was funded 25% of the time; the vast majority (82%) from government agencies (Dimitroff, 1992). In the second study, research was funded 27%; with just over half (51%) from government agencies (Gore et al., 2009).

Research Questions
This research aimed to answer six research questions: • How many authors are credited in these research articles?
• In which country are the first authors based?
• How many of the articles published are from funded research and who provides that funding? • What are the data collection methodologies utilized in these research articles? • Is a single methodological approach or mixed methodological approach more common? • Which statistical packages were used and how was qualitative data analysed?

Methodology
A journal impact factor is the average number of times that articles in a particular journal are referenced by other articles. Thomson Scientific produces annual Journal Citation Reports (JCR) based on a two-year impact factor. The 2006 JCR Impact Factors were calculated by dividing the number of citations received in 2006 to articles (published in 2004 and 2005) in Journal A by the number of articles published in Journal A (as those in the same years). The citation data does not refer to all journal publications, but only those indexed by Thomson Scientific (Cross, 2008).
The journals to be analysed were selected by using the 2006 Journal Citation Report (JCR) Social Science from the ISI Web of Knowledge. These were the top three ranked information systems journals: • MIS Quarterly (MISQ) -IF 4.731; • Journal of the American Medical Informatics Association (JAMIA) -IF 3.979; • Information Systems Research (ISR) -IF 2.537. These journals stress their commitment to research. The ISR website states it "is a leading international journal of theory, research, and intellectual development, focused on information systems in organizations, institutions, the economy, and society" (http://www.informs.org/Journal/ISR). JAMIA includes articles which _ _ K. Davies 20 "describe key perspectives, novel applications, and leading-edge original investigations in the field" (http://jamia.bmj.com/site/about/index.xhtml).
ISR and MISQ both specialize in the study of IS, whereas JAMIA does not specialize in this area, but regularly publishes articles on this topic.
Three years worth of articles (2005 -2007) were collected for analysis. JAMIA research papers including those subtitled original JAMIA investigations were included in the analysis. All papers from ISR and MISQ were selected.
The whole article was examined to collect the required data as this was not always present in the abstracts. Only the terms explicitly used in the published articles were noted. Content analysis is a flexible research method that has been utilized in information systems research focused on research methodologies (Palvia et al., 2003;Palvia & Pinjani, 2007). A coding sheet was utilized to ensure standardization, consistency and that all relevant data was collected for the content analysis (Palvia et al., 2003).
Eldredge (2004); Palvia et al. (2003); and Crumley & Koufogiannakis (2002) all provided definitions and explanations of different research methodologies, of which the following are utilized when evaluating the research studies analyzed in this study: • Archive of E-Mails: accumulated email messages are indexed to enable quick, searchable access of archived messages. • Electronic Medical Records (EMRs) contain patient chart information, but due to their computerized format EMRs can be utilized by researchers to investigate discharge/transfer orders, pharmacy orders, radiology results and laboratory results or to focus on patients with a particular condition or treatment regime. • Experiments / Simulations: researchers specify and control the conditions and variables which create an artificial environment; • Focus Groups are small discussion groups, usually between six and twelve participants guided by a facilitator who gathers data on attitudes, opinions, perceptions and / or knowledge of a particular topic or subject; • Freely Available Data, including information from the Internet, that is nonconfidential and non-proprietary; • Interviews are usually structured (with predefined questions that are standardized for all interviewees) or unstructured (with no set questions resulting in a more participant driven interaction); • Log of User Activity are machine-generated records that include: search queries; user page requests; time online (users); time spent on a particular website; postings written on a discussion forum; number of visits to a website; entry / exit page of a website; and web browsers used. • Observation is a recorded surveillance of an object or subject, but with little or no control over the event observed; • Questionnaires contain predefined standardized questions with either answers to select or a free-text response; which after completion can be returned by fax, post, e-mail or submitted via the Internet; • Secondary Data -studies that utilize existing organizational and company data, such as accounting reports and company reports.
In the analysis the research methods were examined to determine the frequency with which a mixed methods approach had been utilized, but not been specifically mentioned in the published article.

Results
Two hundred and thirty seven articles were identified from the three journals; 56 from ISR, 65 from MISQ and 116 from JAMIA.    Eighty-four percent of the first authors of ISR articles are based in the USA; followed by Canada (9%), South Korea (4%), Hong Kong and Japan (1.5% each). Eighty-two percent of the first authors of JAMIA articles are based in the USA; followed by Canada and Europe (both with 5%), Australia (4%), South Korea (2%), Israel and Japan (1%). Seventy-two percent of the first authors of MISQ articles are based in the USA; followed by Canada (14%), Singapore (8%), Hong Kong, South Korea and UK (2% each).
The majority of first authors are from the USA, but there is international collaboration with second and in some cases third and fourth authors from other countries. Fourteen of the MISQ articles included an author based in a different country than the first author: five from Europe (Denmark two; France, Ireland and Spain); three from Australia; and two each from Canada, Hong Kong and Singapore. Nine ISR articles had authors from different countries; five of which were USA (with the first author not from the USA). The other countries were Australia (two), Hong Kong and Singapore. JAMIA had five authors from different countries to the first author; UK (twice), Canada, Israel and the USA. MISQ also had two articles with a third author from a different country, China and South Korea.
Funding was more frequently mentioned in JAMIA articles (81%), compared to ISR (45%) and MISQ (41.5%). Table 5 shows that grants from NIH (National Institutes of Health) / NSF (National Science Foundation) / AHRQ (Agency for Healthcare Research and Quality) were the most common source of funding for articles published in JAMIA (55%) and ISR (39%). The most common funding source for MISQ articles (39%) was from overseas.   Thirty articles from JAMIA, fifteen from ISR and nine from MISQ were not analyzed as the research did not involve data collection. These papers included framework or model development and reviews of a model or theory. Table 6 shows the most popular data collection method for all the journals was the questionnaire. The second most popular method varied: experiments / simulation for ISR and MISQ; with interviews and electronic medical records for JAMIA. The third most popular data collection method for MISQ was interviews and for ISR secondary data, from databases and company information.
Mixed Method 44% 34% 53% Single methodologies were more popular with ISR articles and JAMIA (Table 7). Slightly more MISQ articles were published using a mixed methods approach, than a single research methodology.
The majority of mixed method research involved a combination of two methods, but three methods were utilized by 17% (JAIMA), 22% (ISR) and 27.5% (MISQ). Four research methods were reported by 7% of JAMIA articles.
Mixed method research in MISQ and ISR was more likely to include the questionnaire as one of the methods. The second method was usually experiment (54% ISR and 35% MISQ), followed by interviews (25% MISQ and 15% ISR). More JAMIA articles used observation in combination with interviews (50%) and questionnaire (33%  Qualitative data was analyzed by coding (ten JAMIA articles, eight MISQ and two ISR); content analysis (JAMIA and MISQ two articles, ISR one) and thematic analysis (JAMIA three and MISQ two). MISQ had four articles which used cluster analysis. JAMIA had articles report the use of software packages to analyze the qualitative data; five articles used NVivo and two Atlas.ti.

Discussion
Claver, González and Llopis (2000) found that articles in MISQ were written by multiple authors (76%). In this research 92% of MISQ articles were written by two or more authors. No JAMIA articles were published from single authors and just 4% of those published in ISR were sole-authored. JAMIA articles were more likely to be written by three or more authors (90%) and over half (52%) written by five or more authors. This could be linked to the fact that over half the JAMIA articles were funded by NIH, NSF or AHRQ grants which encourage a multidisciplinary approach with a combination of authors bringing different experiences and skill sets to the project which ultimately leads to an article published by multiple authors.
Claver, González and Llopis (2000) found that 89% of authors published in MIS Quarterly were based in the USA and 11% from Canada. In this research 72% of MISQ first authors worked in the USA and 14% from Canada; but four other countries were represented. This suggests the authorship of MISQ articles is becoming more international.
The European Journal of Information Systems published more articles from authors based in Europe (Dwivedi and Kuljis, 2008). It is not surprising therefore that ISR, JAMIA and MISQ publish more articles from North American authors as these three journals are all produced and published in the USA.
Funding was more frequently mentioned in JAMIA articles (81%), compared to ISR (45%) and MISQ (41.5%). These are much higher than previous research, 25% (Dimitroff, 1992) and 27% (Gore et al., 2009). Often a prerequisite for funded research is that the results are widely disseminated through, for example, publication of the research findings in a respected journal in that field. Peffers et al. (2007-8) highlighted the importance of effectively communicating research to the appropriate audience. Grants from NIH (National Institutes of Health) / NSF (National Science Foundation) / AHRQ (Agency for Healthcare Research and Quality) were the most common source of funding for articles published in JAMIA. This is a medical informatics journal and is a logical publication for authors who work in healthcare research. The most common funding source for MISQ articles was from overseas which reflects the diverse range of countries in which the authors work.
All three of the journals in this research published more articles using questionnaires as the preferred data collection method, which confirms the findings from previous research as noted in the literature review. The questionnaire is an easy research tool to utilize, allowing the examination of large samples in a time-effective and cost-efficient manner. Statistical packages also enable the rapid analysis of the data collected. However there are negatives associated with this research method. Questionnaires are limited by the choice of answers and usually respondents are not given the opportunity to explain their responses. There is also no guaranteed method of ensuring that respondents interpret the question in the way the author envisaged. In an interview for example, questions can be restated or explained if the respondent is not really answering the question posed.
The second most popular research methodology varied in this research. ISR and MISQ articles utilized experiments and simulations (18% for both); which is similar to the 19% from Vessey, Ramesh and Glass (2002) and 15% of Alavia and Carlson (1992). JAMIA articles utilized interviews and Electronic Medical Records in joint second (16%); Dwivedi and Kuljis (2008) also found interviews ranked second in popularity in published articles (18%). JAMIA focuses on healthcare and medical informatics so Electronic Medical Records (EMR) as a popular data collection tool is not surprising. The EMR research covered many elements including: reviewing discharge summaries for specific terms related to adverse effects; reviewing charts of patients with particular conditions; analyzing outcomes of computerized notification of abnormal test results (alerts); assessing the impact of using an EMR on specialists' time; comparing electronic billing diagnoses to electronic antibiotic prescribing; evaluating the quality of the data recorded; and considering the issue of consent from patients for their records to be used for research purposes. Palvia et al. (2003) and Chen and Hirschheim (2004) found different research methodologies reported in MIS Quarterly to Information Systems Research. In this research the data collection methods were similar for both journals with questionnaire the most popular followed by experiments / simulations. Avison et al. (2008) found only 6% of articles published in the Information Systems Journal used a mixed methods approach. Mixed methods approaches were used more frequently in this research by all three journals. Pisonneault and Kraemer (1993) identified an over-reliance on single methods design in IS research as a major weakness. This research suggests that researchers are addressing this issue with an increased use of mixed methods research.
The most popular statistical packages reported in ISR and MISQ was PLS Graph, whereas the most popular in JAMIA was SAS. PLS Graph is used for structural equations modeling using the partial least squares approach (http://www.plsgraph.com/) and SAS provides a range of statistical software from traditional analysis of variance to exact methods and dynamic data visualization (http://www.sas.com/technologies/analytics/statistics/stat/index.html). There are several possible reasons prompting the selection of statistical package. Firstly the author's decision to select a particular software package can be affected by availability, that is which is readily available to them; as both of these require a subscription. The second reason relates more to the research itself and in particular to the type of data gathered as well as the statistical test required (based on the data).
Qualitative data was more frequently analyzed using coding. Coding organizes the data by categorizing and indexing words, sentences or paragraphs. This enables the data to be systematically analyzed which improves reliability. Two software packages were utilized to analyze qualitative data; NVivo and Atlas.ti. The selection of software package may again be driven by availability or previous experience with one of the packages.

Limitations
Only three of the information systems journals were analyzed in this research, though these were the top-ISI ranked journals. There are other information systems journals that could have been included in this research, including Communications of the ACM; Decision Sciences; European Journal of Information Systems; Information & Management; Information Systems Journal; Journal of Information Technology; Journal of Management Information Systems; and Management Science.
Each article was only analyzed by the author. However other studies such as the study by Palvia et al. (2003) also used one coder.

Conclusion
Prospective authors to these journals may consider some of these research findings, prior to submitting an article. There were considerably fewer research articles published annually in Information Systems Research (median 18) and MIS Quarterly (23) compared to JAMIA (37). Over 90% of articles published in these three journals were written by more than one author. These three journals are published in the USA and over 85% of the first authors of research articles worked in North America.
The number of research articles that utilized a mixed methods approach has increased compared to earlier studies. A range of data collection methods were used which provided depth to the evidence enabling integration of the results to provide a more complete picture. However, the questionnaire was the most popular data collection method for all three journals. There are many advantages to using questionnaires, including low financial cost; time-efficient; information can potentially be collected from a large population; and ease of analysis. However, there has been no research within Information Systems to determine which research methods are the most effective and relevant. Further work needs to be done to establish whether questionnaires are actually the most suitable method for Information Systems research.