{'fraction_non_alphanumeric': 0.026835607537361923, 'fraction_numerical': 0.017543859649122806, 'mean_word_length': 4.942471042471042, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Tracking uptake of innovations from the European Union Public Health Programme.

BACKGROUND
The European Commission developed the Public Health Programme to enable cross-national innovation and transfer in fields of health information, health threats and health promotion. PHIRE (Public Health Innovation and Research in Europe), a collaboration of the European Public Health Association (EUPHA) with seven partners, addressed the uptake of these public health innovation projects at country level.


METHODS
EUPHA thematic sections lead on areas of public health practice and research and experts can choose to be section members. The section presidents of seven sections chose eight European public health projects, starting in the EU Public Health Programme in 2003-05, that provided new knowledge for practice and covered a majority of the EU countries. A web-based questionnaire recorded country informants' (CIs) perceptions of uptake, assessed as relevance and dissemination to a range of public and non-governmental organizations.


RESULTS
108 CIs individually described the eight innovations in an average of 14 (46%) of the 30 European countries. Three of the eight innovations were considered of high relevance by >60% of respondents and at least 70% of informants considered seven of the eight innovation projects as of high or moderate relevance. Dissemination was noted across governmental, professional and academic settings, with high impact on knowledge/awareness for at least 30% of CIs. Some projects had uptake within the policy cycle in particular countries and connected strongly with academics and professionals. Projects working at local level had less visibility nationally and some projects were unknown to national respondents.


CONCLUSIONS
European Union funding for public health can contribute to cross-national knowledge transfer and uptake of innovations. More attention is needed to classify, characterize and identify public health innovations and to demonstrate their direct contribution to European health and well-being.


Introduction
Innovation, built on research, is considered by the European Union as an important driver of economic growth. 2 Equally, health contributes to economic welfare, because the healthcare system is a significant part of national economies. 3 Innovations are distinguished from inventions and isolated experiments through their scaling-up and diffusion. 4 Social innovations 5 provide an important balance to industrial and commercial innovation. Social innovations have been described as 'new ideas (products, services, models) that meet social needs more effectively than others and create new social relationships or collaborations'. 6 The European Commission Directorate for Health and Consumers has, since 2003, through its health programme provided co-funding for cross-national projects and demonstration activities such as sharing, benchmarking and networking. The third multi-annual programme 2014-20, entitled Health for Growth, proposes a smaller range of fields for projects and greater focus on information and monitoring. The Commission's impact assessment says: '. . . the success of the Programme can only be defined in terms of the rate of implementation, participation by Member States and take-up of the outcome in the different Member States and stakeholder groups . . .'. 7 PHIRE examined the uptake across European countries of innovations supported by the Public Health Programme in 2003-05. 8 Both quantitative and qualitative methods have been used in studies of the use of innovations for public health policy, [9][10][11] uptake of research programmes, 12 knowledge translation of research into practice 13 and evaluation of health systems. 14 A retrospective longitudinal assessment of selected research grants was used to evaluate the impact of national health research funding in Ireland. 15 Gaining information across European countries through the EUPHA, PHIRE has used a longitudinal approach to investigate the uptake and impact of health programme projects.

Methods
PHIRE drew on the organizational structures of EUPHA, its national associations and its thematic sections as well as the European Public Health Conference held in November every year. PHIRE sought to engage individual members of EUPHA through its sections. The EUPHA sections are formed around public health themes, and arrange activities both at the above mentioned annual public health conferences and other wise. Each Section has about 600-800 members. The study focused on 30 countries-the 27 EU member states with Norway, Iceland and Switzerland-that were most closely associated with the Health Programme.

Questionnaire
At the start of PHIRE in 2010, EUPHA had 18 sections and a section council. The lead of each section (usually the section president) was sent a description of PHIRE. They were asked to consider whether their section might be a participant in the project and, if so, to propose up to three European innovations that they might track. Seven section presidents agreed to take PHIRE on. Through further discussion, six proposed one innovation each and one proposed two innovations-a total of eight projects. Criteria for choosing an innovation project were: (i) selected from the EU Public Health Programme in the period 2003-05 to ensure that the project had been completed and reported, because a project life could be up to 3 years; (ii) implemented in a majority of the 30 European countries (EU 27 plus Norway, Iceland and Switzerland) and (iii) have potential to provide new knowledge for public health. In practice, the sections also proposed projects that were currently still known in 2010. The innovations chosen, and their acronyms, are shown in Box 1.
The PHIRE partners and the section presidents together developed a questionnaire, to be accessed through the web, which would collect information on perceptions of the impact of the innovations. There should be a short core instrument, with general questions suitable for all projects and attractive for the country informants (CIs) to work with. Fields to be covered included a description of the expected ways the innovation could have had an impact, and how the innovations affected activities in the specific country. Data were input to a web-based questionnaire by the CIs.

Country informants
Recognizing the difficulty of complete reporting for all countries within a voluntary system, PHIRE set a target to gain responses from up to 20 of the 30 EEA countries for each innovation. Five of the seven section presidents started, by contacting members of their sections. Where there were insufficient members, they drew on known participants in the projects and others expressing interest to assist, while seeking to keep a balance across countries and based on their knowledge of likely responses.

Invited and responding informants and countries
There were 298 invited informants, average 37 per project and 108 CIs, average 16 (44%) per project (Table 1). On average, CIs were invited for 80% of countries across all eight projects: CIs were invited for all 30 (100%) countries for two projects, ranging down to 18 (60%) countries for one project. The target level of replies from 20 countries was achieved for one project, down to nine countries for the least frequent. The average was 14 countries (46%) across the eight projects.
Across all countries, the number of projects where informants were invited ranged from 4 to 8, average 5.8 (72%) and where there was a CI ranged from 0 to 7, average 3.3 (41%) ( Table 2). For an average 1.2 (15%) projects, the CI had been previously involved with the project.

Relevance
The health programme projects were chosen by EUPHA section presidents for relevance to their section theme. CIs generally considered the innovation projects to have been relevant. Three innovations, EAAD, HA and CHOB, were considered of high relevance by >60% of CIs ( Figure 1); and for 7 of 8 innovation projects, at least 70% of CIs considered them of either high or moderate  relevance. Up to 20% of the respondents reported that they did not know about the relevance of URHIS and ENHIS in their country. In contrast, HA and CSAP were considered relevant, at least to a limited extent, by all respondents. All CSAP respondents had been chosen through their earlier involvement in the project leader or partner, but this was not the case among respondents for HA.

Dissemination
CIs were asked in the questionnaire to assess whether the project had no/limited/'considerable or high' impact across stakeholder groups or organizations. Options included: government, national health authorities, health care providers, professional organizations, local/ regional authorities, universities, research organizations, non-governmental organizations (NGOs), general population and target population addressed in the project, mass media, trading and other authorities. At least five such organizations had been reached for >70% of organizations for CSAP, down to 10% for CHOB (Table 3). A quarter of reported projects within countries reached five or more stakeholder organizations and half reached two to four organizations. There was greatest dissemination to national health authorities, government and universities (Figure 2). For example, in EAAD, 46% of CIs reported considerable or high impact on knowledge/awareness by universities. Similarly, for CSAP, the CIs reported greatest impact on knowledge/awareness across governments, health service providers, professional organizations, NGOs, the general population and the target population.

Involvement in the original project
CIs who had been involved in the original project were more positive about impact through dissemination compared with those who were not within the project: of the involved CIs, 49% responded that the impact on government was 'considerable' or 'high' compared with 17% among those not involved (Table 4).
Similar proportions, 53% compared with 18%, were also found when reporting impact on knowledge/awareness of national health authorities, although effects for local/regional authorities were less (28% compared with 13%; Table 5).

Free text
The CI also provided free text responses within the questionnaire (Supplementary Table S1). The majority of CIs described the innovations as relevant, and saw their impacts at different levels. There were reports of projects having strong impacts within the policy cycle in certain countries (see example Box 2), while some projects were less visible at national level (e.g. URHIS, which worked with municipalities rather than national governments). Most projects connected strongly with both academics and professionals (e.g. VENICE and EUCID) and also with public and media (e.g. EAAD). In some cases, however, the respondent knew little of the specific project or knew that the project had not had any uptake or action in their own country.

Discussion
The 108 country informants across 30 countries gave broadly positive assessments of uptake-relevance and dissemination-of the innovations in eight projects supported by the EU Health Programme (2003-05). The innovations, ranging from national accident reports to protocols for vaccination schedules, already   8  3  2  Germany  7  5  1  Greece  7  3  1  Hungary  7  3  1  Ireland  7  4  3  Italy  8  7  1  Latvia  4  4  1  Lithuania  7  5  2  Luxembourg  4  1  1  Malta  6  3  1  The Netherlands 7  5  3  Poland  6  4  2  Portugal  8  6  1  Romania  7  5  1  Slovakia  6  3  0  Slovenia  7  3  1  Spain  8  7  2  Sweden  8  5  2  United Kingdom 8  5  3  Iceland  2  2  2  Norway  5  1  1  Switzerland  3  1     PHIRE drew on the expertise of the researchers, public health organizations and other national stakeholders around EUPHA. Most of the experts did not know directly about the specific innovation or project being discussed; this would lead to an under-estimation of impacts. In balance, the section presidents also invited responses from experts outside EUPHA sections: about a third of CIs had previously been involved in some way in the innovation projects. These informants probably had better knowledge about impacts, although their bias-'speaking up' or 'speaking down'-is unknown.
The eight innovations in this study were chosen from 198 projects funded in the period 2003-05. There is a strong need for an overview and coordination of these activities, which might be considered similar to the range of 'molecules' or pharmaceutical products, or diagnostics and equipment, developed for clinical research. But in comparison with the commercial sector, public health lacks the incentives and drivers to take innovations to market and scale-up. One example is the Canadian Health Council, which has a database of innovative health practices. 17 These are categorized as 'emerging', 'promising' or 'leading', based on four evaluation criteria: quality of evidence, impact, applicability and transferability and filed by their place within the health care system.
Uptake of innovations has been measured at structural, organizational and personal levels. 18 The model developed to structure the questionnaire needed to be relatively general, able to report on different types of innovations working at different structural levels and through various ways in society. The final web questionnaire was quite detailed and informants could not always provide answers across the areas of information requested. In some situations, the responses from more than one informant were combined. Some respondents replied saying that this was not their area of expertise or interest-they were an expert member of the section, but knew little about the specific area of the project. Although most respondents were answering the survey in a second language, the free text option was successfully used (shown in the PHIRE work package report). 16 PHIRE recorded relevance and dissemination as estimates of uptake. All eight innovations were still active as European collaborations seven years later, demonstrating the incremental spread of ideas and methods in public health across Europe. Even primarily descriptive innovations such as Healthy Ageing were considered to have achieved national impacts-and indeed, may have contributed towards prioritization of Active and Healthy Ageing for the EU's Health Programme in 2014-20. From a policy perspective, the study demonstrates the cross-national added-value of these small investments from the European Commission's Public Health Programme.

Conclusion
Innovation is regarded as a driver for competitiveness and profits in the commercial sector. Equally, in the not-for-profit areas of health, innovation can lead to both better, sometimes more cost-effective, practice. Tracking innovation in public health is important, both to maintain the commitment of public services in their task and also to value investment in research and development for practice. Although the EU Public Health Programme supports only a small proportion of European public health innovations, PHIRE selected only a small proportion of these, evidence for the impact of public health projects as innovation has not previously been reported systematically. The collaborative approach of PHIRE provides a model for future assessments of impact (and potentially also, health benefits) from innovations.