{'fraction_non_alphanumeric': 0.03342105263157895, 'fraction_numerical': 0.013618421052631578, 'mean_word_length': 5.285458304617505, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
ASEAN benchmarking in terms of science, technology, and innovation from 1999 to 2009

This article provides an empirical assessment of the performance of the member states of the Association of Southeast Asian Nations in terms of science, technology, and innovation. This study is relevant because it employs a larger data set, examines more countries, and covers more years than previous studies. The results indicate that these countries had differing patterns of performance, and the pattern of growth among them was asymmetrical. Additional findings suggest that these countries performed idiosyncratically with respect to the six quantitative dimensions we examined. Our research includes a form of comparative policy evaluation that might assist the monitoring of the implementation of “Vision 2020”. The results simplify how we determine the relative strengths and weaknesses of national innovation systems and are relevant to policy discussions. In relation to transferability, the findings demonstrate similarities to the European Union with regard to performance and governance.


Introduction
Thirty years after the birth of the Association of Southeast Asian Nations (ASEAN), member states gathered in Kuala Lumpur in 1997 to chart a vision for ASEAN on the basis of contemporary realities and prospects in the decades leading to the Year 2020. That vision is of ASEAN as a concert of Southeast Asian nations, outward looking, living in peace, stability and prosperity, bonded together in partnership in dynamic development and in a community of caring societies (ASEAN 1997).
Science, technology, and innovation (STI) play an important role in prosperity and economic growth. One of the goals of ASEAN in its ''Vision 2020'' programme is to strengthen integration as a way of promoting progress, peace, and prosperity in the region. Among other measures, ASEAN aims to achieve this goal via cooperation in science and technology (S&T) and the advanced development of human resource capacities (APAST 2007). While ASEAN has promoted STI, little scholarly work has been performed on the results of those efforts (e.g. Dodgson 2002).
Some limited research efforts have been devoted to studying empirically STI in ASEAN countries (e.g. Dodgson 2000; Lai and Yap 2004;Remøe 2010;Sigurdson and Palonka 2002). Dodgson (2000) studied four ASEAN countries by assessing data on scientific publications, the number of researchers, research and development (R&D), and foreign direct investment (FDI), and by analysing the electronics sector. Lai and Yap (2004) studied two ASEAN countries by examining human capital, R&D, S&T parks, foreign technology transfer, and government research institutes. Remøe (2010) studied all ASEAN countries by assessing data on R&D, education, patents, and competitiveness. Sigurdson and Palonka (2002) studied five ASEAN countries by examining data on R&D, education, and FDI. However, there has been no analysis of all ASEAN members that engages in STI performance assessment and growth analysis.
The purposes of this article are twofold. First, it provides empirical evidence of STI performance in ASEAN countries at the national level. Second, it compares and contrasts previous findings regarding STI policies in ASEAN countries at the national level. This article studies the performance of ASEAN member states with regard to innovation over a period of 11 years (from 1999 to 2009). We chose this period because it follows the implementation of ''Vision 2020'' in 1997, and it excludes the Asian crisis that took place in 1997 and 1998. This window also reflects the fact that the most recent data available are from 2009. The underlying question is: What was the STI performance of ASEAN member states during the selected time period? We consider two hypotheses: 1. Overall STI performance is distributed normally across ASEAN member states. 2. STI performance dimensions are distributed normally across ASEAN member states. These hypotheses are tested by studying the pattern yielded by the Summary STI Index using growth analysis and by reference to composite indicators. Composite indicators are increasingly recognised as a useful tool for policymaking (Sajeva et al. 2005). Composite indicators are also useful because they can integrate large amounts of information into easily understood formats (OECD 2003(OECD , 2008. Regarding indicators, the article employs three categories (i.e. enablers, firm activities, and economic size) and six dimensions (i.e. human resources; open, excellent and attractive research systems; finance and support; business R&D expenditure; intellectual assets; economic size), which generate 15 indicators in total as a means of studying ASEAN member states' innovation performance and growth.
The findings of this article contribute to fashioning useful and analytically sound perspectives on the degree to which STI goals have been achieved at the national level. This article is relevant for two reasons. First, it provides evidence-based policy recommendations pertinent to the improvement of national performance in the field of STI. Second, it adds to the academic literature investigating STI in ASEAN countries through its examination of a larger set of data, countries, and years.
The remainder of this article is organised as follows. Section ''STI policy coherence at the ASEAN level'' outlines the framework of this article by presenting the three dimensions of policy coherence in ASEAN countries. Section ''Data'' refers to the data and indicators used for computing STI performance. Section ''Methodology'' details the technique for calculating composite indicators. Section ''Findings'' presents the findings of the research by answering the research questions and testing the hypotheses. Section ''Discussion'' brings a discussion about the research results. Finally, Sect. ''Policy recommendations and conclusion'' provides evidence-based policy recommendations and concludes the article.

STI policy coherence at the ASEAN level
One central assumption in economic integration is policy coherence. Policy coherence comprises three dimensions: horizontal, vertical, and temporal. We look at the three dimensions in this section. First, ''horizontal coherence ensures that individual, or sectoral, policies, build on each other and minimise inconsistencies in the case of conflicting goals'' (Remøe 2005). According to the ASEAN Plan of Action on Science and Technology (APAST) 2007-2011, the new APAST takes into account the guiding principles and directives provided by the ASEAN Leaders [through the declarations] and the ASEAN Ministers for Science and Technology (S&T) (ASEANSEC 2009). Furthermore, all action plans are guided by the directives and general principles outlined in the declarations, including economic, social, cultural, technological, scientific, and administrative matters.
Second, ''vertical coherence ensures that public outputs are consistent with the original intentions of policy makers'' (Remøe 2005). Due to the comitological structure of AS-EAN's S&T cooperation, objectives are identified at the highest level (the ASEAN Ministerial Meeting on Science and Technology [AMMST]), and passed downwards to the Committee on Science and Technology (COST) and its subcommittees. Several monitoring and evaluation processes are in place to ensure the proper implementation of regional programmes, projects, and plans.
Third, ''temporal coherence ensures that today's policies continue to be effective in the future by limiting potential incoherence and providing guidance for change'' (Remøe 2005). According to APAST 2007APAST -2011'the new APAST takes into account the guiding principle and directives provided by previous action plans on S&T, as well as national S&T plans and programmes'' (ASEANSEC 2009) to prevent policy incoherence.
In addition, we sketch the three dimensions of policy coherence that might be considered in fashioning a policy coherence perspective in the context of ASEAN. Horizontal coherence of STI policies underlines the necessity of governing and coordinating many and various policy domains as a means of enhancing STI policy results. Horizontal relations in ASEAN's area of STI are visible in the AMMST, in which the ASEAN members take decisions on economic, social, and political issues following the guidelines, principles, and aims laid down in the declarations and in accordance with the consultation procedures and the principle of consensus.
Vertical coherence of STI policies refers to the relationships among different layers of government institutions. Vertical relations are highlighted by the institutional structure of ASEAN regional cooperation in S&T (see Fig. 1). This structure is designed in a hierarchical manner, in which the AMMST directs the COST, and COST assigns work to the subcommittees, advisory bodies, working groups, and joint committees.
Temporal coherence of STI policies refers to the deadlines and follow-up programmes of policies distinguishing between a long-term STI policy path and a short-term STI policy without a follow-up programme. ASEAN formulated its overall goals in ''Vision 2020'', goals that are supposed to be reached via several small steps called action plans (see Table 1). However, this long-term vision dramatically contrasts with the national policy cycle, which for several member states covers 4-5 years.

Data
In ASEAN, STI policies are integrated at the supranational level, where governance is characterised by intergovernmental, comitological and OMC features. ''Vision 2020'' provided a cooperative context that triggered STI policies at the national level, but the primary goals of such policies varied across four categories: the financial goal of stimulating R&D, diffusion or technology transfer, managerial gaps in running businesses, and systems for facilitating change. In particular, the primary goals of national STI policies resulted generally in performance dimensions that did not usually correspond with such goals.
In this section, we provide the data sources and identify innovation indicators. We collected the quantitative data from various sources, as presented in Table 2. We utilised the public databases from the United Nations Education, Scientific and Cultural Innovation and firm theories underline the significance of dynamic capabilities (Teece and Pisano 1994). Dynamic capabilities are formed by two components. First, diffusion capability requires a substantial level of capability on the part of the firm in selecting, using, and developing technology, and enables firms efficiently to accumulate, assimilate, and adapt appropriate extant technology. Second, creating capability is the capacity to create new technologies based on R&D and directed at creating first-mover advantages in the market, intellectual property, and licensing income (Dodgson 2002). The forces that determine the supply of innovations are the enablers and the firm. In addition to the firm, enablers are major forces of innovation performance that consist of human resources; open, excellent and attractive research systems; and finance and support.   Firm-creating capabilities are also major forces of innovation performance that consist of business R&D expenditure and intellectual assets. Regarding indicators, we employed three categories (i.e. enablers, firm activities, and economic size), six dimensions (i.e. human resources; open, excellent and attractive research systems; finance and support; business R&D expenditure; intellectual assets; economic size), which generate 15 indicators in total (i.e. Graduates in science, Tertiary, Total; Graduates in engineering, manufacturing and construction, Tertiary, Total; Percentage of tertiary graduates in science; Percentage of tertiary graduates in engineering, manufacturing and construction; Percentage of researchers FTE ISCED 6; Percentage of researchers HC ISCED 6; International students or internationally mobile students, Total; Percentage of GERD financed by business; Percentage of GERD financed by government; Percentage of GERD financed by higher education; Percentage of GERD financed from abroad; Percentage of GERD financed by private non-profits; Patent applications by patent office, Total; Patent grants by patent office, Total; gross domestic product (GDP) per capita).
The first category of indicators is ''enablers'', which are the major forces of innovation performance other than the firm. This category is split into three dimensions: human resources; open, excellent and attractive research systems; and finance and support. The first dimension captures six indicators that determine the disposability of an educated and highly skilled labour force. The second dimension contains one indicator and estimates international competitiveness with regard to attractiveness and openness. The third dimension consists of four indicators and estimates the availability of government support programmes for innovation activities and finance schemes for innovation projects.
The second category of indicators is ''firm activities'', which highlight the innovation performance at the firm level. This category distinguishes between two dimensions: business R&D expenditure and intellectual assets. The first dimension contains one indicator for firm investment in research and development that drive innovation. The second dimension consists of two indicators and estimates intellectual property rights (IPR) efforts in the innovation process.
The third category of indicators is ''economic size'', which determine the association of research and innovation activities with the economy. This category is unidimensional in that it covers economic relatedness. In this respect, De Solla Price (1969) showed that the amount of scientific publication coming from a country is correlated positively with its economic size as measured by its GDP, and not with geographic area, population or any other parameters. In addition, Narin (1994) has found that the correlation holds not only for scientific publication, but also for patenting.
We explain the rationale and source for each innovation indicator as follows.

A. Enablers
A. For some countries, the trend of absolute graduates in science or engineering might have been in decline although the trend of share of tertiary graduates with degrees in science and engineering was in growth. This shows that the expansion of tertiary education has led to a large number of students with degrees in other disciplines than science or engineering. For others, the share of tertiary graduates with degrees in science or engineering might have been in decline although the trend of total number of science or engineering graduates was in growth. This shows that the expansion of tertiary education has led to a large number of students with degrees in science or engineering. Therefore, we use both absolute (A.1.1. and A.1.2.) and relative indicators (A.1.3. and A.1.4.) in order to capture the differences in the trend of graduates in science, engineering, and other disciplines.
Concern about shortages of scientists and engineers resurfaced many times. A shortage is defined as a shrinking of national supply, that is that the number of new science and engineering graduates falls and is insufficient to replace those exiting the workforce (e.g. as a result of retirement, emigration or death). Perhaps the central issue behind concerns about shortages of scientists and engineers is the realisation that the economic growth depends on investments in knowledge, including an ample supply of scientists and engineers (OECD 2004). This is show by the supply of science and engineering graduates at the tertiary level in relation to all other graduates at the tertiary level.
A.1.5 Percentage of Researchers FTE ISCED 6 Rationale: Researchers with an ISCED 6 are among the most important inputs in the innovation process with regard to the human resource dimension because they are university graduates prepared for faculty and research positions. They have achieved the second stage of tertiary education leading to an advanced research qualification, an accomplishment enabling them to engage in the creation and conception of innovations.  (Table 3).
Finally, we utilised the Worldwide Governance Indicators because they are significantly correlated with innovation performance as measured by the Summary Innovation Index (Hollanders and Arundel 2007). In this respect, Celikel Esser (2007) investigated the link between innovation and governance by exploring the relationship between six governance indicators from the World Bank and the innovation performance measures from the European Innovation Scoreboard and the Global Innovation Scoreboard (Hollanders and Arundel 2006). For three governance indicators-control of corruption, rule of law and government effectiveness, there is a strong link with innovation performance. For the other three-voice and accountability, political stability and regulatory quality, there is a moderate relation with innovation performance. These indicators consist of six composite indicators of broad dimensions of governance. The process by which governments are selected, monitored, and replaced is measured by two composite indicators: ''Voice and Accountability'', and ''Political Stability and Absence of Violence or Terrorism''. The capacity of a government effectively to formulate and implement sound policies is measured by ''Government Effectiveness'' and ''Regulatory Quality''. The respect of citizens and the state for the institutions that govern economic and social interactions is measured by ''Rule of Law'' and ''Control of Corruption''. Voice and Accountability captures perceptions of how extensively citizens are allowed to participate in selecting their government, as well as freedom of expression, freedom of association, and a free media. Political Stability and Absence of Violence or Terrorism captures perceptions of the likelihood that the government will be destabilised or overthrown by unconstitutional or violent means, including politically motivated violence and terrorism. Government Effectiveness captures perceptions of the quality of public services, the quality of the civil service and its independence from political pressures, the quality of policy formulation and implementation, and the credibility of the government's commitment to such policies. Regulatory Quality is captures perceptions of the ability of the government to formulate and implement sound policies and regulations that permit and promote private sector development. Rule of Law generally captures perceptions of the extent to which agents have confidence in and abide by the rules of society, particularly the quality of contract enforcement, property rights, the police, and the courts, as well as the likelihood of crime and violence. Control of Corruption indicates perceptions of the extent to which public power is exercised for private gain, including both petty and major forms of corruption, as well as the degree to which elites and private interests have captured the state (Kaufmann et al. 2010).

Methodology
We computed the STI performance of each ASEAN member state for the period 1999-2009 by means of a composite indicator [the Summary Science, Technology and Innovation Index (SSTII)] constructed in accordance with Hollanders and Tarantola (2011). We chose this period because it follows the implementation of ''Vision 2020'' in 1997, and it excludes the Asian crisis that took place in 1997 and 1998. This window also reflects the fact that the most recent data available are from 2009. The steps for computing the SSTII are as follows.

Data availability
The data were retrieved from the sources displayed in Table 2. The reference year for all indicators was 1999. The covered period includes the 11 years from 1999 to 2009. Country-by-country data availability for each year is presented in Table 4.

Identifying extreme values
An extreme value in statistical terms is an outlier. We find two kinds of outliers in this data set: positive and negative outliers. Negative outliers are those extreme values less than the mean minus two times the standard deviation. Positive outliers are those extreme values higher than the mean plus two times the standard deviation. The outliers are presented in Table 5.
Transforming data that have highly skewed distributions across countries After the identification of the extreme values, those values were replaced by the negative and positive limits. These limits are defined as the mean minus two times the standard deviation and as the mean plus two times the standard deviation, respectively.

Imputation of missing values
There were three scenarios involving missing data: missing data at the beginning, inbetween, and at the end of the time series. In the following discussion, these scenarios are described and explained. If data were missing for the first year under study, they were imputed with the next available data. For instance, if the value from 1999 was missing but  For example, if the score from 2009 was not available but the score from 2008 was available, then the 2009 score was equalised with the 2008 score. In some cases, the entire data series for an indicator was missing for a country, leaving no opportunity for imputation of missing values. In these cases, the composite score was calculated without that indicator.

Determining maximum and minimum values
The minimum value is represented by the lowest value available in the period of study for each indicator within all countries. Likewise, the maximum value is represented by the highest value available in the period of study for each indicator within all countries.

Normalising values
After the minimum and maximum values are determined for each country and for each indicator, the minimum-maximum normalisation approach can be used to find the normalised values. The normalised score is computed as follows. The minimum value is subtracted from each indicator observation and the result is divided by the difference between the maximum and the minimum values. Using this approach, all normalised values are in the range between 0 and 1. The minimum normalised value corresponds to 0, and the maximum normalised value corresponds to 1.
where Y i is country i's normalised value, x i is the country's value for the original scale, max(x) and min(x) are, respectively, the maximum and minimum values that have been attained for this indicator across all countries.

Calculating composite scores at the dimension level
To calculate composite scores at the dimension level (i.e. human resources; open, excellent, and attractive research system; finance and support; business R&D expenditure; intellectual assets; and economic size), we grouped all sub-indicators of each dimension for each country and for each year and computed the average of all sub-indicators for a given dimension for a given country and for a given year. For instance, the calculation of the dimension of finance and support for Thailand, (which includes the sub-indicators GERD financed by government; GERD financed by higher education; GERD financed by abroad; and GERD financed by private non-profit) for the year 2000 resulted in the composite score of 0.36094 at the dimension level.
where l is the mean of normalised sub-indicator Y for country i for year t and N is the number of normalised sub-indicators. In the weighting process, all normalised sub-indicators receive the same weight.

ASEAN benchmarking 561
Calculating composite innovation scores The composite innovation scores were computed for each year and each country by taking the average of all indicator dimensions for a given year. Each composite score at the dimension level was computed by averaging the sub-indicators normalised scores for a given year. For example, the composite innovation index 2009 for Thailand equalled the average of human resources, open, excellent and attractive research systems, finance and support, business R&D expenditure, intellectual assets, economic size. In other words, the SSTII was computed as a linear aggregation.
where M is the mean of all dimension indicators for country Y for year t and n is the number of normalised dimension indicators. In the weighting process, all normalised dimension indicators receive the same weight.

Growth analysis
To assess an individual country's innovation performance over a time series, we conducted a growth analysis. For this purpose, we arranged the indicators at the dimension level (i.e. human resources; open, excellent and attractive research systems; finance and support; business R&D expenditure; intellectual assets; and economic size) from 1999 to 2009 to observe changes and trends in each dimension. In addition to the dimensions level, we computed the growth rates for the SSTII from 1999 to 2009. Therefore, we calculated the average value between 1999 and 2009 for each dimension to obtain a general picture regarding the clustering of countries in leaders, followers, catching-up, and trailing countries. The second measure of the growth analysis was the annual average growth rate.
To compute this rate, we first calculated ''the annual growth rate of the composite indicator between two consecutive years t -1 and t'' (Tarantola 2008). In the second calculation, we defined the overall growth rate of the entire period, from year 0 to t, i.e. the change from 1999 to 2009. The third and last calculation we made was the annual average growth rate between 1999 and 2009. After this step, the maximum for each country and the maximum for each cluster were displayed.

Categorisation
To categorise the countries, we defined five groups (i.e. significant progress, slight progress, stagnant, slight regression, and significant regression) and computed the average of all positive and all negative overall growth rates. The average of the positive values corresponded to significant and slight progress. The average of the negative values corresponded to slight and significant regression. In more detail, the group that displayed ''significant progress'' included all countries that were performing above the positive average. Second, those countries that had a positive rate but were below the average were categorised as ''slight progress''. Third, ''stagnant'' related to those countries that did not record any changes within the time series. Fourth, the group of ''slight regression'' included those countries that had a negative overall growth rate but were above the negative average. Finally, the countries below the negative average were categorised in the ''significant regression'' group.

Findings
Our quantitative findings are based upon SSTII (Table 6) and growth analysis (Table 7).
To identify the pattern of national STI performance, composite indexes were computed for each year and each country by taking the average of all indicator dimensions for the period 1999-2009. Each indicator dimension was computed by averaging the sub-indicator normalised scores for a given year. After each national mean was calculated, we computed the overall mean to identify ''leader '', ''follower'', ''catching-up'', and ''trailing'' countries. The overall mean corresponds to the group of ''catching-up'' countries. Countries with scores above the overall mean 0.3 are considered to be ''follower'' or ''leader''. ''Leader'' is represented by the maximum 0.6. Countries with scores below the overall mean 0.3 are considered to be ''catching up'' or ''trailing''. ''Trailing'' is represented by the minimum 0.2. The result shows that positions at the top and at the bottom are unambiguous, and independent of the weighing vector. Altogether, this result seems rather close to the conclusion that the weighing regime is only of secondary importance, thus vindicating the decision to keep the weighing as simple as possible (Schibany and Streicher 2008).
As a result, the quantitative evidence suggests that ASEAN member states had differing patterns of STI performance. In particular, Brunei Darussalam, Cambodia, Lao PDR, Myanmar, and Vietnam were ''trailing''. Indonesia, the Philippines, and Thailand were ''catching up''. Malaysia was a ''follower''. Finally, Singapore was a ''leader''. In addition, the pattern of growth did not behave as a bell curve. In particular, Brunei Darussalam, Malaysia, the Philippines, and Thailand showed ''significant progress''. Indonesia, Cambodia, Lao PDR, Myanmar, Singapore, and Vietnam showed ''slight progress''. No AS-EAN countries showed ''stagnation'', ''slight regression'', or ''significant regression''.
Based upon our quantitative findings, we are able to reject the first null hypothesis, which was that the overall STI performance is distributed normally across ASEAN member states. Therefore, we propose the following alternative hypothesis: The overall STI performance is distributed in a skewed fashion across ASEAN member states, with one leader and many trailing countries and with progress in terms of growth.
Similar results have been reported by Nguyen and Pham (2011). Between 1991 and 2010, scientists from the ASEAN countries have published 0.5% of the world scientific output. Singapore led the region with the highest number of publications (accounting for 45% of the countries' total publications), followed by Thailand (21%), Malaysia (16%), Vietnam (6%), Indonesia and the Philippines (5% each). The number of scientific articles from those countries has increased by 13% per year, with the rate of increase being highest in Thailand and Malaysia, and lowest in Indonesia and the Philippines. Based on the relationship between scientific output and knowledge economy, Nguyen and Pham (2011) identified four clusters of countries: Singapore as the first group; Thailand and Malaysia in the second group; Vietnam, Indonesia and the Philippines in the third group; and Cambodia, Laos, Myanmar and Brunei in the fourth group.
The evidence provided in Table 8 suggests that ASEAN member states performed idiosyncratically with respect to finance and support; open, excellent and attractive research systems; business R&D expenditure; human resources, and intellectual assets. Brunei Darussalam and Lao PDR emphasised finance and support (financial goal). Singapore emphasised open, excellent and attractive research systems (diffusion goal). Lao PDR, Malaysia, the Philippines and Thailand emphasised business R&D expenditure (managerial goal). Indonesia and Vietnam emphasised human resources (systemic goal).   Table 7 Growth rate year t Based upon our quantitative findings, we are able to reject the second null hypothesis, which is that the STI performance dimensions are distributed normally across ASEAN member states. Therefore, we propose the following alternative hypothesis: The STI dimensions are distributed in a skewed fashion across ASEAN member states.
Human resources are crucial for innovation. From 1999 to 2009, ASEAN emphasised the possession of an educated and highly skilled labour force. In particular, BN and SG showed ''significant progress''. KH, MY, PH, TH, and VN showed ''slight progress''. However, LA showed ''significant regression''. MY led performance in terms of human resources. ID and MM were following. PH, SG, TH, and VN were catching up. BN, KH, and LA were trailing.
Open, excellent and attractive research systems are important for international competitiveness. From 1999 to 2009, ASEAN emphasised the importance of attracting international students. In particular, MY, TH, and VN showed ''significant progress''. BN, LA, and PH showed ''slight progress''. SG was leading in performance in terms of open, excellent and attractive research systems. MY was following. ID, PH, and TH were catching up. BN, KH, LA, MM, and VN were trailing.
Innovation calls for the availability of government support programmes for innovation activities and finance schemes for innovation projects. From 1999 to 2009, ASEAN did not sufficiently emphasise the role possessing adequate finance and support. In particular, PH showed ''significant progress''. ID showed ''slight progress''. However, MY showed ''slight regression''. BN, SG, TH showed ''significant regression''. KH led in terms of finance and support. BN and ID were following. LA, TH, and VN were catching up. MY, PH, and SG were trailing.
Business R&D expenditures are also critical for innovation. From 1999 to 2009, AS-EAN did not sufficiently emphasise the role of business R&D. In particular, MY showed ''significant progress''. SG and TH showed ''slight progress''. However, BN showed ''slight regression''. ID and PH showed ''significant regression''. Despite the regression, MY and PH led in terms of business R&D. SG was following. LA and TH were catching up. BN, ID, and VN were trailing. Finally, patent activity is a framework condition for innovation. From 1999 to 2009, ASEAN emphasised the role of patent applications and grants. In particular, BN, ID, TH, and VN showed ''significant progress''. MY, PH, and SG showed ''slight progress''. SG led performance in terms of intellectual assets. ID, MY, and TH were catching up. BN, PH, and VN were trailing. Lall (2000) argues that the process of technological change in ASEAN is more a matter of acquiring and improving technological capabilities than innovating at the forefront of knowledge. Our findings show that science is not a priority among economies in BN, KH, and LA. Moreover, BN, KH, LA, MM, and VN trailed in terms of international students. The major challenge facing ASEAN science is the limited size of its members' educated and highly skilled labour force. Therefore, ASEAN should focus on an idiosyncratic formulation of STI polices to increase the number of graduates in science, engineering, manufacturing, and construction and the number of researchers and international students.

Discussion
Comparing our results with that of Vinluan (2011), the Philippines ranked low in research productivity compared to Singapore, Thailand, and Malaysia, particularly starting in the 1990s.
Some ASEAN member states with low national R&D expenditures possessed ambitious development plans. Malaysia's 7th 5-year plan (1996)(1997)(1998)(1999)(2000) sought to increase GERD as a proportion of GNP to 1% from 0.4% and to increase the number of scientists and technicians to 1,000 per million of its population, as compared with 400 in 1992. However, the region as a whole did not sufficiently emphasise the role of public and private R&D. Therefore, ASEAN should foster R&D investment by asking FDI to localise R&D laboratories and facilities in catching-up and trailing countries.
STI has become central in ASEAN not only for countries that have achieved high income (e.g. Singapore), but also for some of the lower-middle income countries that are still in the catch-up phase (e.g. Indonesia and the Philippines) as shown in Table 10. We found that high-income economies such as Brunei Darussalam and Singapore do not show the same STI performance (trailing vs. leader) despite the fact they have a high governance score (Table 9) and a high development index. The gap, according to the data in Table 8, appears to be associated with the open, excellent, and attractive research systems, business R&D expenditures, and intellectual assets that Singapore possesses and Brunei Darussalam lacks. For Singapore, close to or at the technological frontier, the growth potential inherent in the manufacturing and service industries in which Singapore has acquired comparative advantage now rests on staying abreast of the latest technological developments elsewhere and producing a steady stream of innovations.
During the past 11 years, ASEAN countries experienced economic growth. However, the region did not do enough to move up the value chain or to increase its reliance on productivity-enhancing innovations based on S&T for its dominant source of growth. In particular, ID, KH, LA, and VN showed ''significant progress''. BN, MM, PH, SG, and TH showed ''slight progress''. SG led performance in terms of GDP per capita. BN was following. MM and TH were catching up. ID, KH, LA, MY, PH, and VN were trailing. ASEAN benchmarking 567

Policy recommendations and conclusion
The first goal of this article was accomplished by providing empirical evidence of the STI performance in ASEAN at the national level by comparing and contrasting previous findings regarding STI policies in ASEAN. As a result, we answered the research question and provided an alternative set of hypotheses. What was the STI performance of ASEAN member states?
The results of the quantitative analysis indicate that ASEAN member states had differing patterns of STI performance. In addition, the pattern of growth did not behave as a bell curve. Therefore, one alternative hypothesis suggests that the overall STI performance is distributed across ASEAN member states in a skewed fashion, with one leader and many trailing countries, and with progress in terms of growth. Further quantitative results indicate that ASEAN member states performed idiosyncratically with respect to the six quantitative dimensions. Therefore, another alternative hypothesis suggests that the STI dimensions are distributed across ASEAN member states in a skewed fashion.
We uncovered several interesting findings in addition to the main results. Lao PDR, Malaysia, the Philippines, and Thailand emphasised business R&D expenditure, which corresponds to a managerial goal. Furthermore, the Philippines, Thailand, and Malaysia were among the member states in the mid-position between catching-up and following.
Regarding a better or alternative explanation of the results, this research suggests that national emphasis on STI policies is not the only factor influencing STI performance; the development stage and history (colonial influence) of ASEAN member states, as suggested by Remøe (2010) and Dodgson (2002), may also be factors.
In relation to transferability, the findings of our study demonstrate similarities to the EU in reference to performance and governance. Both governance systems include inter-governmentalism, comitology, and the open method of coordination. Both regional organisations show a similar country distribution to the four groups in percentages. In ASEAN, the member states are distributed in percentages as follows: trailing 50%, catching-up 30%, follower 10%, and leader 10%. In the EU, the distribution is trailing 37%, catching-up 33%, follower 14.8%, and leader 14.8%. While this study is relevant for the reasons given above, there are limitations. First, this study has fewer indicators than the studies conducted in the EU context due to the unavailability of data for some ASEAN member states. Second, the results of this study should be tested in a comparative study of regional integration in developing countries. Third, the time scope of this study (1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009) included changes of governments and changed priorities in the field of STI, as well as other differing contexts relevant to ASEAN member states, e.g. the devastating tsunamis of 2004 and 2006. Due to the limitations and the descriptive character of the research, this study raises several issues for follow-up studies. To tackle the first limitation, future research should include a follow-up study with a similar methodology after the lapse of 10 years in order to include the period 2010-2020, or they should include additional indicators in order to reflect a broader picture of STI performance in ASEAN. To tackle the second limitation, future research should include a comparative study between ASEAN and Mercosur or between ASEAN and the African Union. To tackle the third limitation, future research should include in-depth case studies of ASEAN member states' STI performance and the achievement of ''Vision 2020'' or ASEAN country-to-country performance comparisons between leader and trailing countries.
In addition, our research identifies categories and performance in a heterogeneous regional organisation, namely, ASEAN. This heterogeneous character is reflected in several other regional organisations, i.e. EU, Mercosur, and the African Union. Therefore, our research might allow future comparisons in different parts of the world at the country and regional level if the same methodology is applied.
Our research includes a form of policy evaluation. In general, the study provides a broad picture of STI in ASEAN. This is important in assisting the monitoring of the implementation of ''Vision 2020'' with regard to STI through a comparative assessment of the STI performance of ASEAN member states. The results assist in the determination of relative weaknesses and strengths of national innovation systems and could raise policy discussions at the ASEAN and the national levels.
STI policies play an increasingly important role in the international economy. The most compelling evidence of this phenomenon in ASEAN is ''Vision 2020''. The premise was that STI are related to economic growth. Data were amassed for each of the affected countries to prove this case. These data led to the conclusion that ASEAN countries experienced economic growth between 1999 and 2009. However, the region did not do enough to move up the value chain and increase its reliance on productivity-enhancing innovations based on S&T as a dominant source of growth, despite the fact that ASEAN countries pursued the establishment of STI goals. How, then, do we begin to fashion policy recommendations at the national and regional levels? We can separate recommendations into two main sets.
The first set involves the norms themselves. The STI regime is expressed through norms. As far as the concept of norms is concerned, STI norms are social definitions regarding what is appropriate in achieving STI policy goals. At the national level, such norms define the legitimacy of a state's activities. At the ASEAN level, they define the legitimacy of ASEAN actors' activities. In the sphere of STI regulation, defining these norms defines ASEAN institutions, the state, their relation with each other, and their relation with those regulated. Following Donnelly (2010), two types of norms exist: constitutive and regulative. ''Constitutive norms'' define the actors and entities that are legitimately involved in the process of governance. ''Regulative norms'' define the interaction between actors and entities in the act of governing and reinforce the commitment to constitutive norms through social activities.
The connection between the constitutionalisation of STI governance in ASEAN depends on the perceptions of the state at the national level (either nanny state or laissezfaire state). Explicit norms at the ASEAN level require resonance with implicit or explicit norms at the national level. Regulative norms can be broken down analytically into archetypal narratives regarding the state and business or the market. The state can be seen as either a nanny state or state based on laissez-faire policies. The business or market can be seen as beneficial to society or opportunistic. Therefore, STI norms that are regulative reflect vertical and horizontal interactions.
In this respect, Ahrens (2002) analysed critical problems of policy implementation and sought to identify general principles that might be suitable as guideposts in making the state more effective regardless of the particularities of its STI policy. He argues that governments must assume a market-enhancing role and must enhance the state's capabilities and capacities for implementing public policies. Crafting public institutions that ensure accountability, transparency, and predictability of policymaking and involve the private sector in political decision-making processes is critical for successful policy implementation. In addition to institutional arrangements that help governments credibly pre-commit to policies, the quality and institutional design of public administration and public-private interfaces are crucial ingredients of an effective governance structure.
The second set of recommendations involves the type of system that should be developed for ensuring compliance with these norms. Which institutions should be involved and what methods should be authorised for use? If we look at regional innovation systems, the debates have implicated the issue of how to find a feasible balance between the need to respect diversity among member states and the unity of ''Vision 2020''. The open method of coordination is an oriented policy instrument that might address this balance. The main procedures of this method are common guidelines to be translated into national policy, combined with periodic monitoring, evaluation, and peer review organised as mutual learning processes, and accompanied by indicators and benchmarks as means of comparing best practice.
The open method of coordination might improve ASEAN integration by providing better tools for achieving integration and generating new solutions to STI coordination problems. Such a method formalises the process of integration by coordination, by which integration is no longer a transfer of competences to the supranational or regional level that requires subsequent mandatory adaptation of national institutions and rules but essentially a reconciliation of national policies that reflects the diversity and strong interdependency of ASEAN economies.