{'fraction_non_alphanumeric': 0.024074074074074074, 'fraction_numerical': 0.013804713804713804, 'mean_word_length': 5.550165380374862, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Striving for success in international forestry research

This second special issue of Australian Forestry presents some of the results from ten international forestry research projects that have been funded by the Australian Centre for International Agricultural Research (ACIAR). Since 2005, when the international community agreed to the Paris Declaration on Aid Effectiveness (OECD 2005), there has been a commitment to increase monitoring and evaluation efforts to enable periodic qualitative and quantitative assessments of aid effectiveness. While progress is being made on this commitment, these efforts focus on ten high-level indicators of aid effectiveness, rather than on the effectiveness of particular programs or projects (OECD/UNDP 2016). With forestry research projects, very often there is a long lag time before the desired impact from the research is achieved, which should be taken into account when evaluating effectiveness. The ACIAR currently spends about $115 000 000 annually on agricultural Research for Development (R4D) projects in partner countries (ACIAR 2017), including about $10 000 000 on bilateral forestry research projects that cover many different themes of research and are implemented in many different situations. How then should the effectiveness or success of these investments be evaluated and what do we mean by the terms ‘effectiveness’ and ‘success’? In the development literature ‘effectiveness’ is defined as the extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance (OECD 2002), whereas for ‘success’ there is no agreed definition. Most donor organisations, including ACIAR, commonly use economic impact assessments to estimate the economic benefits that result from such research investments. Certainly this approach can demonstrate impressive returns on investment from selected forest research, as evidenced from ACIAR’s investments in plantation forestry in Indonesia (Lindner 2011) or domestication and improvement of Australian trees in Vietnam (Fisher & Gordon 2007). However, as Lindner et al. (2013) have demonstrated, much of the aggregated economic impacts from ACIAR’s projects come from a small number of highly successful projects. Therefore, it is desirable to consider a range of criteria in determining success of forestry projects. To improve the understanding of what success means for ACIAR forestry projects and what factors affect project success, 90 scientists were interviewed (Bartlett 2018c). They were selected using a purposive strategy because they had worked as project leaders, in-country coordinators or as collaborating scientists on one or more of the 30 selected projects and could still be contacted. From their responses, the following working definition of a successful ACIAR forestry project was developed: a project that uses high quality but flexible scientific methods to achieve planned outputs; enhances the capacity of partners; generates knowledge or technologies that can improve the system under investigation; facilitates ongoing scientific relationships and networks; and results in tangible scientific impacts and benefits for project stakeholders and local communities. This definition recognises that, when judging the success of research projects, there are multiple dimensions to consider and individual projects may only result in partial or incremental improvements to the system being researched as part of a long-term program. While economic impact assessments have a useful role, understanding the cumulative economic benefits arising from research projects does not necessarily help those who fund or implement such projects to understand what causes differential success in research projects. As Stern et al. (2012) have advocated, evaluations should seek to understand what works, what does not work, and why, so that these lessons can be used to replicate, generalise and scale up the results from development interventions. However, there are challenges in trying to compare the results from large numbers of projects that cover many different research themes and that are implemented in many different countries under a wide range of implementation contexts. To help address these challenges and improve the understanding of what enables some research projects to be more successful than others, a new methodology for evaluating the relative success of multiple projects has been developed (Bartlett 2016). In this methodology, two dimensions of project success are considered: the extent to which planned research outputs are achieved and adopted (achievements) and the extent of the impacts resulting from wider adoption, typically outside of the project and beyond its life (impacts). Under each dimension, four different criteria are evaluated, with scores out of ten given for both the achievements and impacts. This approach produces four categories of project success: high achievements-high impacts, high achievements-low impacts, low achievements-low impacts and low achievements-high impacts. This methodology has been applied in three case studies of ACIAR forestry projects, covering ten projects from Vietnam, reported by Bartlett et al. (2017), ten projects from Indonesia, reported by Bartlett (2018a), and ten projects from Papua New Guinea, reported by Bartlett (2018b). About one quarter of the 30 forestry projects (seven) were evaluated as having high achievements and high impacts, with three of these implemented in Vietnam and four in Indonesia. About half (16) of all the projects were evaluated as high achievements and low impacts, with projects in this grouping occurring in all three countries. The remaining one quarter (seven) of the projects had low achievements and low impacts, with two of these occurring in Vietnam, one in Indonesia and the other four in Papua New Guinea. By graphing both of