{'fraction_non_alphanumeric': 0.02908781628747608, 'fraction_numerical': 0.014501382096534126, 'mean_word_length': 5.535853251806559, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Availability of results from clinical research: Failing policy efforts

Objectives: Trial registration has a great potential to increase research transparency and public access to research results. This study examined the availability of results either in journal publications or in the trial registry from all studies registered at ClinicalTrials.gov. Methods: All 137,612 records from ClinicalTrials.gov in December 2012 were merged with all 19,158 PubMed records containing registration numbers in the indexing field or in the abstracts. A multivariate analysis was conducted to examine the association between the availability of the results with study and participant characteristics available in registration records. Results: Fewer than 10% of the registered studies and 15% of the registered and completed studies had published results. The highest publication rate of 22.4% was for randomized trials completed between 2005 (starting year for structured indexing in PubMed of study registration) and 2010. For 86% of overall and 78% of completed registered studies, no results were available in ClinicalTrials.gov or in journal publications. Studies funded by industry vs. other funding sources and drug studies vs. all studies of other interventions were published less often after adjustment for study type, subject characteristics, or posting of results in ClinicalTrials.gov. Conclusion: Existing policy does not ensure availability of results from clinical research. International policy revisions should charge principal investigators with ensuring that the approved protocols and posted data elements are aligned and that results are available from all conducted studies.


Introduction
Clinical research aims to inform clinical and policy decision-making by providing valid evidence for treatment benefits and harms [1][2][3][4]. However, when conclusions about treatment benefits and harms are based on incomplete evidence, biased decisions and ineffective health care can result [5]. Bias in the publication of studies that show impressive results can exaggerate the benefits of examined treatments [6][7][8][9][10].
Several policy initiatives have tried to improve transparency and ensure wider availability of results from clinical research. The Food and Drug Administration Modernization Act of 1997 required that the NIH create a trial registry -ClinicalTrials.gov -for drug efficacy studies approved with Investigational New Drug applications. In 2000, the National Library of Medicine at the National Institutes of Health launched ClinicalTrials.gov and opened it to the public via the Internet, and Congress mandated the registration of all clinical trials of pharmacological treatments for serious or life-threatening diseases at the ClinicalTrials.gov online database [11][12][13][14]. Then in 2005, the International Committee of Medical Journal Editors (ICMJE) and the World Association of Medical Editors made registration a condition of publication for all clinical studies [15]. Finally, the International Clinical Trials Registry Platform (ICT-RP) was developed to include 13 primary registries which met the requirements of the ICMJE in providing 20 items with ''the minimum amount of trial information that must appear in a register in order for a given trial to be considered fully registered'' (see Supplementary Appendix 1, also available at http://www.who.int/ictrp/network/trds/en/ index.html).
Ensuring the public access to the results from clinical studies, the Food and Drug Administration Amendments Act (FDAAA) of 2007 mandated posting of the results from applicable clinical trials (e.g., interventional, non-phase I trials of drugs and devices subject to FDA regulation) on Clinical-Trials.gov within 1 or 2 years of study completion [16].
Nonetheless, publication of the results in journal articles remains voluntary. Less than half of the NIH-funded registered trials are published in a peer-reviewed journal within 30 months of trial completion [17]. Only 29% of completed registered studies involving children and 53% of NIH-funded trials have been published [18].
Previous research used time-consuming manual searchers of the publications in various subsets of registered studies (i.e., by source of funding [17], study participants [18], and specific health conditions) [19]. In contrast, this paper examines result availability from all studies registered with Clini-calTrials.gov to answer the question: Do existing policies in research registration, publication, and indexing guarantee access to the results? This study defines ''availability of the results'' as publication in the journals indexed on Medline or posting the results with ClinicalTrials.gov

Methods
For this study, ClinicalTrials.gov registration records were linked with Medline publication records by a unique registered study identifier (number of clinical trial or NCT). First, all records of the registered studies with no time restriction were downloaded from ClinicalTrials.gov from February 2000 to December 2012 (Appendix 2). All 20 fields required by the ICMJE were downloaded. The frequency of study types, design, funding, participant characteristics, and posting of the results were analyzed relying on information provided by the investigators in registration records [20]. Accuracy of the data in ClinicalTrials.gov can be confirmed only by comparing the posted data elements with those approved by the institutional review boards, which was beyond the scope of this study. For validation purposes, ambiguous data (e.g., enrollment values of more than 99,999 participants or negative publication time intervals when publications occurred before studies started subject recruitment) were excluded from the validated analyses.
In contrast with the previous research focusing on clinical trials only [21], all registered studies were analyzed irrespective of study design, funding, subject characteristics or market status of the examined treatments assuming that all clinical research evidence is important for decision-making (Appendix 2). The study design was categorized: as randomized trial when the study design field mentioned random allocation of participants into the treatment groups; as non-random studies when investigators did not explicitly mention randomization; and as unknown study design when investigators left this field blank. Interventions were categorized as drug, procedure, radiation, biologics, or behavioral according to the categories in ClinicalTrials.gov. Study findings were categorized into two categories: industry funding category included all studies funded by pharmaceutical or device companies exclusively or in combination with individuals, universities, or community-based orga-nizations; all other funding sources category included studies funded by the National Institutes of Health or other U.S. Federal agencies exclusively or in combination with industry funding, as well as studies funded by individuals, universities, or community-based organizations without any industry involvement. The length of studies was calculated as the time period between start and completion dates. The publication time was estimated as the time period between study completion and journal publication.
Medline publication records have unique study registration numbers in a specifically designated field with secondary study identification ([SI]) (detailed information about indexing of the registered studies is available in PubMed tutorial at http:// www.nlm.nih.gov/bsd/disted/pubmedtutorial/ 020_810.html) [22]. In compliance with ICMJEÕs recommendations from 2005, all publications should report a trial registration number in the article abstract. Since the specific field for indexing of registered studies (secondary study identification field) was introduced in 2005, Medline was searched for registration numbers in that designated [SI] field, but also in the article abstracts. A subgroup analysis was then conducted among the studies registered before and after 2005.
All retrieved references were imported into the reference manager Endnote and then were exported to an Excel worksheet compatible with the SAS statistical software.
When the article reported the results from more than one registered study, additional reference records were created for each NCT. When a registered study was published in more than one article, the fact of the publication was counted only once using the earliest available publication.
For validation purposes, in order to determine the accuracy of the Medline indexing of the registered studies (since some publications of the registered trials did not report study identification numbers in abstracts but may have mentioned the registration status in the full texts of the articles), this study examined whether the secondary study identification field identified all publications of the registered studies in the core clinical journal (Abridged Index Medicus list of core clinical journals is available in Appendix 3) that published the greatest number of registered RCTs. To carry out this task, trial registration numbers were searched for in all Medline fields and in the full texts of the articles published in that journal. If full text articles did not include study registration numbers, it was concluded that the published study was not registered [15].
The publication records from PubMed were then merged with the registration records from Clinical-Trials.gov by the unique NCT number and the availability of the results and factors associated with publication status were examined. Publication rates were calculated as percentages of the registered studies with linked publication records in PubMed. The percentages of the registered studies were calculated with disposition of the results within the trial registry. Speculative imputations for missing data were avoided. Multivariate odds ratios of publication were calculated by study type (observational vs. interventional), random allocation of participants, recruitment as reported in ClinicalTrials.gov, funding, examined interventions, posting of the results in ClinicalTrials.gov, and age and sex of enrolled participants. It was hypothesized that odds of publications differ by study funding, design, and examined interventions. Subgroup analyses were conducted of the publication odds among registered completed or terminated studies to test the hypothesis that completed studies would have greater odds of publication. Publication odds may depend on the year of study registration, for example, recently published studies would have lower publication odds compared with the studies registered more than 3 years ago. To address this issue survival analysis was also conducted to examine publication of the completed studies by study design and funding [23]. The detailed analysis of the factors associated with posting the results in ClinicalTrials.gov was beyond the scope of this study [24]. All hypotheses testing at 95% CL were performed using SAS 9.3.

Results
A total of 137,612 registered studies were identified in ClinicalTrials.gov and 19,158 publications of registered studies in PubMed in December 2012 ( Fig. 1). After deleting duplicates and merging the data from the trial registry with publication records from PubMed, 137,607 records of the registered studies and 12,938 publications of the registered studies were included in the analyses (analytical data file is available at https://netfiles.umn.edu/xythoswfs/webui/_xy-23709234_1-t_xCJavcvC).
The studies were published in 1332 journals; 55% of the studies were published in 32 journals. Each of these 32 journals published more than 100 registered studies. The completed ''Secondary Source ID'' field identified 97.3% of the publications of the registered studies and 2.7% of the articles had registration numbers in the full texts of the articles but not in the ''Secondary Source ID'' field.
Only 9.4% of all registered studies and 15% of registered and completed studies were published (registration records in ClinicalTrials.gov could be linked to publication records in Medline) ( Table 1). Publication rates differed by subject and study characteristics ( Table 1). The highest publication rate of 22.4% was among the randomized trials completed between 2005 and 2010 (Fig. 2). The second highest publication rate of 22% was among the registered studies with the results posted in ClinicalTrials.gov. For the majority of registered studies (86% overall and 78% of completed registered studies), no results were available in Clinical-Trials.gov or in journal publications.
Multivariate analysis demonstrated that participant age and gender, interventional study design, tested interventions, and industry funding were associated with greater odds of publications. The registered studies enrolling exclusively children published the results more often than the studies of all ages had (Fig. 3). Registered studies that enrolled men and women or exclusively women published results more often than studies that enrolled exclusively men.
Interventional studies had greater odds of publication compared with observational studies. Randomized trials had greater odds of publication compared with nonrandomized studies. Studies with results posted on ClinicalTrials.gov had greater odds of publication compared with the studies without posting the results. The registered studies examining drugs had lower odds of publication compared with all other studies.
The studies funded by industry published results less often compared with all other studies. The described associations were significant irrespective of the recruitment status (Fig. 3). Survival analysis also demonstrated lower publication rates of industry-funded studies (adjusted hazard rates ratio 0.87 95% CI 0.82-0.91) and the studies examining This research examined whether study completion modified the association between funding source and the publication. For instance, the studies funded by industry vs. not-for-profit sources published results less often irrespective of the recruitment status of both drug studies and all combined nondrug studies (Table 2). However, the type of the examined interventions modified the association between funding source and publication. In fact, industry funding was associated with lower rates of publication for all drug studies and completed studies examining drugs, devices, and procedures. Industry-funded genetic studies published the results more often than the studies with not-for-profit funding sources. The subject allocation also modified the association between funding source and publication. Industry funding was associated with a lower rate of publication  Fig. 3 Publication of the studies registered in ClinicalTrials.gov studies. * Odds ratios were adjusted to all listed variables: subject age, gender, randomized design, funding, intervention types, and posting of the results on ClinicalTrials.gov.
for randomized trials but not for nonrandomized studies (Table 2). Registered studies enrolling exclusively adults vs. exclusively children or vs. exclusively seniors had lower odds of publication irrespective of intervention type and recruitment status ( Table 2). The odds of publication did not differ among registered studies enrolling exclusively children vs. exclusively seniors irrespective of intervention type or recruitment status ( Table 2).
Because of low publication rates, the results were not available for 73% of the enrolled participants in the completed studies and for 75% of the enrolled subjects in terminated studies (Table 3). Thus, since many studies were not reported at all, the results for more than 70% of the enrolled participants are unknown. The time between study start and completion was shorter for industry-funded (1.8 years) vs. other studies (3.1 years) and for drug studies (2.2 years) vs. other interventions (3 years) (data not shown). The time periods between reporting of results in ClinicalTrials.gov and study completion and study publication in Pub-Med averaged around 2 years.

Discussion
Very low publication rates were found for studies registered in ClinicalTrials.gov. For the majority of registered studies (86% overall and 78% of completed registered studies), no results were available in ClinicalTrials.gov or in journal publications. These results are in concordance with previously published studies confirming that the results are not available for the public from the majority of clinical research [17,21,25]. These results were compared with the existing literature by the methods to detect publications of the registered studies. When the researchers searched for publications by the names of the principal investigators, study titles, or examined drugs in addition to registration numbers, they reported similar (much less than 50%) or only slightly higher publication rates [25,17]. When the researchers found registration numbers in the full texts of the articles retrieved for inclusion into systematic reviews, the rates were similar to this study [19,26]. These search findings could be affected by poor compliance of the authors and journal editors with reporting registration of published manuscripts [17,27,28]. Reported differences in the availability of the results from clinical research reflect several accountability gaps: (1) The trial registry does not have a single field in which to mark study applicability to the federal law to post the results; (2) Journal editors published unregistered trials or did not require consistent reporting of the valid registration status in the abstracts; (3) PubMed indexed trials without provided valid registration numbers.
The present study has limitations. No analysis was performed as to whether journals adhered to the policy of publishing only registered studies [15]. Only one trial registry, ClinicalTrials.gov, was analyzed since all U.S. studies should be registered in this registry [29]. Other registries do not allow posting the results which makes result availability analysis difficult. The authors of the articles were not contacted to request additional informa-tion about study registration [30]. Also, principal investigators of the registered studies were not contacted for additional information about study enrollment or missing information. Actual publication bias was not analyzed by examining greater odds of publication of the studies with favorable statistically significant results [8], nor were deviations from the protocols in the published articles [31], or selective outcome reporting examined [32]. It was not a goal of this study to compare publication rates by way of searching for publications associated with each registered study, because to do so would have been time-prohibitive and unfeasible for clinicians, healthcare consumers or policy makers. Nonetheless, the straightforward search methodology followed in this study should have resulted in easy identification of publications of the registered studies.
It was concluded that existing clinical research policies do not guarantee availability of research results in general. Odds of the publication also differed depending on participant age and gender as well as study funding and design. Studies funded by industry vs. other funding sources and drug studies vs. all other interventions were published less often after adjustment for study type, subject characteristics, or posting of results in ClinicalTrials.gov. Therefore, policy revisions should specifically address the studies with the lowest publication rates, namely industry-funded drug studies.
Several initiatives focus on improving result availability from clinical studies. The recently proposed Trial and Experimental Studies Transparency (TEST) Act (H.R. 6272) would expand the manda-    [33]. Under TEST, all American and foreign trials of the drugs or devices marketed in the United States must be registered and the results must be posted in ClinicalTrials.gov within 2 years after study completion (including trials of unapproved drugs or devices) [33]. Under TEST, the investigators will be required to post subject flow, baseline and posttreatment outcomes, and adverse effects. In addition, TEST requires posting of the deposition of consent and protocol documents approved by institutional review boards [33]. This legislation is timely and important. International AllTrials campaign (available at http://www.alltrials.net) calls for registration and results reporting for all clinical studies. Sharing clinical research data is supported by the World Health Organization (WHO), the US National Institutes of Health (NIH), the Cochrane collaboration, the Institute of Medicine and other respected international organizations [34]. All clinical studies should be registered before recruitment of the first participant [18,35] in trial registries that comply with the World Health Organization (WHO) minimum dataset [36]. Trial registration should be mandatory irrespective of trial phase or design, study completion, foreign participant residency, market status of the examined interventions, or voluntary publication status [18,35,37,38].
Better coordination between the Office of Human Research Protection and the Office of Research Integrity is necessary to guarantee human subject protection and availability of the results. Routine monitoring would be more efficient and cost-effective with better harmonization and consistent use of unique study identification numbers across all databases-including ClinicalTrials.gov, Medline, the NIH grant database, and the FDA database. TEST legislation should require: (1) registration of all human trials as a condition of enrollment approval by institutional review boards; (2) validation of the provided information; and (3) accountability of sponsors for complete, accurate, and timely updated information. Continuous harmonization between IRB electronic databases with approved protocols with the trial registry would preclude subject enrollment in unregistered studies and thus guarantee a registration of all human trials.
Potential trial participants should be advised to participate only in registered studies with publicly available protocols and written assurance of the result availability [39].
Modern information technologies enable inexpensive and effective monitoring of compliance with the policy as well as availability of the results. Trial registries should have analyzable fields for trial completeness, applicability to FDA regulations, enrollment status, and exact reasons for termination. Automated alerts could be used to inform principal investigators, institutional review boards, and regulatory agencies about deadlines to comply with policy, enrollment status, and availability of the results.
Principal investigators and sponsors should be accountable for data integrity during registration and publication processes (Table 4) [35]. This study could not explain why some publications included NCTs that never actually existed, or why the recruitment status was omitted or different in the published articles compared with the protocols posted in ClinicalTrials.gov [35].
Poor publication rates of the registered studies demonstrate a need for revisions in clinical research policy in order to guarantee availability of the results [33]. It points to some serious deficiencies in the current research enterprise that threatens the societal goal of evidence-based practice [18].