{'fraction_non_alphanumeric': 0.021452073772621007, 'fraction_numerical': 0.01231708373019309, 'mean_word_length': 5.1719438877755515, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Objective or Perception-Based: A Debate on the Ideal Measure of Corruption

In Kazakhstan, several institutions have developed new measures of corruption. This Article addresses the doubts that empirical analyses have raised as to whether and to what extent existing measures of corruption provide valid and reliable estimates of corruption levels in Kazakhstan. Domestic institutions decided to develop the new measures after exploring reasons international measures seemed to be failing to provide a proper assessment of corruption in the country, with the hope that they could generate better estimates of corruption levels across different regions, sectors, and time.


Introduction
Kazakhstan, usually regarded as a country plagued by high levels of corruption, has recently adopted several legislative and non-legislative measures to address the problem and hopefully to curb corruption. In order to assess the effectiveness of the anticorruption policies, several institutions-from the Institute of Public Policy of the Nur Otan party to the This Article is divided into five sections. In the first section, this Article provides a brief overview of why corruption matters and how it has been assessed in the past, paying attention to the fact that while some efforts have been made to generate fact-based or objective measures of corruption, the best-known measures of corruption are subjective or perception-based. In the course of the discussion, we try to present the main advantages and disadvantages of each of the methodologies.
After a brief review of the related literature in the second section, this Article presents the data collected from a variety of global datasets in the third section. In doing so, we demonstrate that no matter how the level of corruption is estimated, Kazakhstan is generally regarded as a highly corrupt polity. 5 Building on this discussion, we perform some correlation analyses that raise questions about the validity and the reliability of the estimated corruption level in Kazakhstan. We find that the levels of corruption estimated by the Corruption Perception Index ("CPI"), the World Governance Indicators ("WGI"), and the Global Competitiveness Index ("GCI") are weakly and insignificantly related to one another and are fairly unstable over time.
In the fourth section, this Article explores three explanations for why international measures of corruption are unable to assess and track the changes in the level of corruption in Kazakhstan. In doing so, we suggest that this anomaly can be explained on the basis of three different reasons, namely that Kazakhstan is so different from any other place that methodologies that work elsewhere do not work in the Kazakhstani context, that the methodologies employed to estimate the level of corruption are wrong, and that the information processed to estimate corruption levels in Kazakhstan is misleading. After reviewing the evidence in favor of and against each of these explanations, we conclude that the most compelling explanation for why international measures do not work well in the case of Kazakhstan is that the measures rely on misleading information. This finding is important because it can drive various institutional actors to gather better information to generate better estimates of corruption levels.
In the fifth section, this Article presents the three methodologies devised by institutional actors in Kazakhstan to measure corruption. The Article evaluates the merits and possible shortcomings of each of them. In the final section, the Article draws some tentative conclusions.

I. Related Literature
In the last few decades, corruption has been an increasingly popular topic in public policy research and other related areas of scientific analysis. Although scholars have yet to reach a consensus on the definition of corruption, scholarly research has focused on the causes and consequences of 5. See Dossym Satpayev, Corruption in Kazakhstan and the Quality of Governance 1-2 (Inst. for Developing Countries, Discussion Paper No. 475, 2014). corruption 6 on both theoretical 7 and empirical grounds. 8 While the research on corruption was largely descriptive and theoretical at the earlier stages, this was mainly due to the nature of the topic and the difficulties in establishing an objective measurement for corruption and quantifying corruption-related activities. 9 Empirical studies have flourished immensely, especially after the availability of wider datasets, and have extensively explored both the causes and the consequences of corruption. 10 These studies have used different theoretical perspectives to examine the causes and effects of corruption. In the case of Kazakhstan, researchers would focus narrowly on a particular aspect of corruption, such as rent-seeking behavior, and identify a wide range of factors that could be responsible for it. 11 Such factors include the lack of democratic culture, 12 a weak judicial system, 13 the limited power of civil society, 14 the "resource curse," 15 "political inertia," 16  13. On the relationship between corruption and the weakness of the judicial system, see S. V. Simonov, Retrospektivnyj analiz rassmotreniya terminov "korruptsiya" i "korruptsionnaya deyatel ' population's high level of tolerance for corruption, 18 and the limited access to information on public services. 19 The need to identify the causes, the consequences, and the possible developmental costs of corruption has generated a rich methodological debate on how corruption can be best assessed. One stream of research has devised objective measures that look at the actual activities related to corruption and is more reliable when the data is available. 20 The other stream of research has developed subjective, perception-based measures of corruption (such as the CPI by the Transparency International) whose emergence has made a significant contribution to the empirical research on corruption.
Neither objective nor subjective measures are perfect. Objective measurements appear ideal, but they are difficult to obtain, and above all, they make cross-country comparison problematic since corruption has different definitions and meanings in different countries, legal traditions, and cultures. Survey-based subjective measures have some intrinsic problems as they rely on perceptions rather than objective facts, and they are often calculated through a collection of other surveys. 21 Furthermore, they often fail to reflect the details of country-specific conditions, anti-corruption strategies, and implementation of these strategies. 22 Additionally, aside from region-specific sources of measurement errors, perception-based measures are hardly comparable across time even for the same country, 23 suggesting that more research is needed to identify country-specific conditions and more appropriate country-level corruption indicators. 24 Finally, subjective  25 They measure corruption more accurately in countries where more data is available from a wider range of sources, such as in the developed world. 26 In developing or less-developed countries, the indexes are based on a smaller number of less reliable sources. 27 As a result, subjective measures measure corruption more accurately when corruption is less prevalent and they are less reliable where they are needed the most. 28 At the same time, both the exact definition and the measurement of corruption, and differences due to country-specific factors (such as culture, tradition, etc.), still pose a challenge for researchers. Due to these limitations, researchers came up with a comparative studies analysis that is more meaningful for high-income countries than for low and middle-income countries. In other words, corruption estimates, that are quite precise in highincome countries, often have problems in terms of validity and reliability in developing countries. 29 In spite of the fact that scholars and practitioners have been aware of the shortcomings of these measures and the need for better measures, these perception-based estimates of corruption have been widely employed because it is generally believed that bad data is better than no data. 30 Efforts have, however, been made to cope, if not solve, some of the above-mentioned methodological problems. For instance, a new subjective or perception-based assessment of corruption was developed by asking respondents to indicate whether they had had direct experience of corruption (like the International Crime Victims Survey) instead of asking them to indicate how much corruption they believed existed in their respective countries. 31  surveys and found that corruption perceptions are biased due to respondent characteristics and country characteristics. 32 As a result, estimates of the level of corruption in a given country are not always precise, efficient, valid, or reliable-a problem that has been documented for several countries and that undermines policy makers' ability to assess whether and to what extent the anti-corruption policies that they design and implement are effective in curbing corruption. 33

II. Corruption Measures for Kazakhstan
The case of Kazakhstan is rather interesting in this regard. Kazakhstan is a young nation that became independent with the collapse of the Soviet Union. It has experienced a high rate of economic growth for nearly two decades and has set forth, in several strategic documents, rather ambitious economic and developmental objectives. 34 The Kazakhstan-2050 Strategy, for example, indicated that Kazakhstan's objective is to become one of the thirty developed countries in the world by 2050. 35 The Kazakhstan-2050 Strategy also made clear that this ambitious developmental goal should be achieved by improving the levels of education and public health, by diversifying the economy, by promoting good governance through proper institutional and administrative reforms, by attracting foreign direct investments, and by curbing corruption. 36 Given the importance of reducing corruption, Kazakhstani policy makers need precise analytical tools to track the level of corruption across regions, across sectors, and over time. Yet, as we will show in the rest of this section, some of the best-known international measures of corruption not only fail to provide any indication of corruption levels across regions and sectors, but often fail to provide proper nation-wide estimates of corruption that could be used in diachronic analyses.
The best-known international measures of corruption are the CPI by Transparency International, Control of Corruption by the WGI, and the Global Competitiveness Index's estimates of diversion of public funds, irregular payments and bribes, and favoritism in decisions of government officials. 37 32. Id. at 1. 33 The CPI is computed by aggregating the data collected from twelve data sources. It has been expressed on a ten-point scale from its creation until 2011, and on a 100-point scale from 2012 onwards. 38 Low values indicate high levels of, or total, corruption, while high values indicate low levels, or absence, of corruption. 39 The second measure of corruption that is widely used internationally is represented by the Control of Corruption variable. 40 This variable is one of the six dimensions of governance included in the WGI. 41 It has been computed from 1996 onward (except in 1997, 1999, and 2001), and it ranges from a minimum of -2.5 to a maximum of +2.5. 42 Positive scores indicate a considerable ability to curb corruption and a corresponding low level of corruption. Negative scores indicate that there is little, if any, ability to control corruption. 43 The third, fourth, and fifth measures of corruption used in the course of the present analyses are represented respectively by diversion of public funds, irregular payments and bribes, and favoritism in decisions of government officials. 44    corruption-related variables is expressed on a seven-point scale where the highest score means "best" and the lowest score means "worst." 46 The data presented in Table 1 provides what appears to be, at face value, a rather coherent picture of the level of corruption in the Republic of Kazakhstan. In fact, no matter how one measures corruption, the Republic of Kazakhstan seems to be characterized by high levels of corruption. In 2013, Kazakhstan scored only 26 of the 100 points on the CPI, and was assigned a score of -0.90 in terms of Control of Corruption by the WGI. In 2014 it was assigned a score of 3.0 in terms of favoritism in government decision by the GCI, 3.3 in terms of diversion of funds, and 4.1 in terms of irregular payments and bribes. 47 If the cutoff point-that is, what separates corrupt polities from non-corrupt ones-is 50 for CPI, 0 for WGI, and 3.5 for favoritism, diversion of funds, and irregular payment and bribes, three indicators place Kazakhstan in the camp of the corrupt countries, while the remaining two indicators barely place Kazakhstan in the camp of the non-corrupt ones. Hence, on the basis of these data, one could be legitimately tempted to conclude that corruption is a pervasive problem in Kazakhstan.
This conclusion would, in turn, be further corroborated by the fact that the rankings generated on the basis of each of these indicators reveal that a large number of countries are or at least are believed to be less corrupt than Kazakhstan. Specifically, 64 countries do better-that is, have less-than Kazakhstan in terms of diversion of funds and irregular payments and bribes, 76 countries do better in terms of favoritism in government decision, 125 countries do better in terms of CPI, and 166 countries do better in terms of  While these measures prima facie seem to depict a coherent picturethat there is a lot of corruption in Kazakhstan, or at least more than in many other countries-the quantitative analyses of these variables reveal that the picture is much fuzzier than the data presented in Tables 1 and 2 indicate and that the various corruption measures discussed here may have problems with validity and reliability. CPI, control of corruption, as well as the three indexes devised by the GCI are all believed to have some problems in terms of validity, reliability, and predictive power. The validity of each of these indexes can be tested by correlation analysis. 49 Specifically, each of these measures is validated if the correlation with another measure of corruption yields a positive, strong, and statistically significant coefficient. 50 The results of the correlation analyses, presented in Table 3, raise several doubts about the validity of these measures of corruption. Only two of the fifteen correlation coefficients are strong, positive, and statistically significant. Four of the remaining correlation coefficients are extremely weak, four of them are negative, and all of them fail to reach statistical significance.
The In order to provide more compelling evidence about the reliability/unreliability of each of these six indexes, we can correlate the estimates generated in a given year with a given index with the estimates generated with the same index in a different year. If the correlation analysis yields a positive, strong, and statistically significant coefficient, it demonstrates that the data is reliable. 51 The result of the correlation analysis presented in Table 4 sustains the claim that the five measures of corruption used to compile the global dataset are not all terribly reliable.
The correlations between CPI scores in a given year with CPI scores in each of the previous years never yield statistically significant coefficients and in three cases out of six it generates a negative coefficient. Hence, the results of the correlation analyses cast serious doubts as to whether CPI scores for Kazakhstan are reliable.
By correlating WGI's control of corruption score measured in a given year with the control of corruption scores for the previous years, we find that all correlation coefficients are positive but insignificant from a statistical point of view. This evidence once again casts serious doubts about the reliability of these estimates for Kazakhstan.
By correlating the scores pertaining to the diversion of funds in a given year with the scores for previous years, we find four of the six correlation coefficients are positive and relatively strong, but one is extremely weak and 49 one is negative. In other words, this statistical analysis does not provide much evidence in favor of the reliability of this data. The correlation of the scores of favoritism in a given year with scores generated in previous years never yield statistically significant coefficients and in three cases, the coefficient is negative.
Finally, since the collection of data for the bribes variable began several years after the data for diversion and favoritism started to be collected, it is possible to correlate the scores generated in one year only with the scores generated in the previous year or two before. Of the three correlation coefficients, two are positive and statistically insignificant while the third is significant but negative. This evidence suggests that even this series of data has major problems in terms of reliability. The results of the correlation analyses presented in Tables 3 and 4 suggest a simple conclusion: that the corruption perception index, control of corruption, as well as the three GCI indexes appear to be somewhat problematic both in terms of validity and reliability. This means that while these data can be employed to provide a general picture of the corruption level in Kazakhstan in a rather crude way-there is a lot of corruption-the data is remarkably less useful in precisely estimating how much corruption there actually is and in tracking how the level of corruption varies over time.

III. Three Explanations
Why do international measures of corruption fail to provide a precise assessment of the level of corruption in Kazakhstan? There appears to be three possible answers for this question. The first answer is that Kazakhstan is a somewhat exceptional case where a methodology, that is successfully used to estimate the level of corruption elsewhere, does not work. The second answer is that international measures of corruption have some inherent methodological flaws and misrepresent the level of corruption in Kazakhstan. The third answer is that the methodologies employed to estimate levels of corruption are sound but the information/data that the methodologies employ to generate the scores is inaccurate. Vol. 50 Thus, we have three basic explanations for why the international measures of corruption do not provide terribly valid and reliable estimates of the level of corruption in Kazakhstan. In the remainder of this Article, we will critically examine the evidence in favor and against each of these explanations.

IV. Kazakhstan: Is it Exceptional?
There are several reasons why one could think that Kazakhstan may be a somewhat exceptional case. Kazakhstan's colonial legacy, Soviet rule, size, and economic success make it quite peculiar. In the last decades, Kazakhstan has been engaged in a process of deep transformation, from a planned to a market-based economy and from a Soviet republic to a unitary, multiethnic sovereign state. 52 It is the ninth largest country in the world, but it has one of the world's lowest population densities. Kazakhstan is at the cross-roads between the East and the West as it links the large and fast-growing markets of the People's Republic of China and South Asia with the Russian Federation and Western Europe by road, rail, and ports on an internal sea (the Caspian Sea). 53 Furthermore, it has rich natural resources, particularly oil and gas reserves (the country ranks 12th in the world in terms of oil reserves and 19th for natural gas reserves), which attract an increasing flow of direct foreign investments. For the most part of its post-Soviet history, it has experienced rapid economic growth. 54 Finally, Kazakhstan managed to join the Organisation for Security and Co-operation in Europe in 2010 and has more recently joined the World Trade Organization ("WTO"). 55 Over the past decade, the country has made important policy strides, progressed towards developing a rule-driven fiscal framework, strengthened public management and its business climate, and allocated resources for improving social services and critical infrastructure to sustain growth. Between 2000 and 2010, Kazakhstan saw significant improvements in social development indicators, including the Gender Inequality Index, which increased Kazakhstan's score by more than 30% over a decade, and the Human Development Index, which ranked Kazakhstan 69th out of 187 countries in 2012 (among some developed countries), an improvement from 80th place in 2005. 57 These achievements and successes have been achieved in spite of the fact the country has been plagued, if international indicators are to be believed, by high levels of corruption.
However, analysts suggest that in spite of remarkable recent transformation, Kazakhstan is facing a number of challenges that undermine its sustainability and reputation. 58 The challenges include growing regional disparities in wealth distribution, a persistently high poverty rate (particularly in rural areas), limited human capital, uneven implementation of the rule of law and democratic processes, limited citizen participation in policy processes, and excessive corruption. 59 As such, implementing governance reforms to advance effective functioning of government institutions, strengthening the quality of democratic institutions and rule of law, and reducing corruption are critical for the country to implement its ambitious vision and objectives. 60 Given this wide range of challenges, Kazakhstan has made and continues to make efforts to modernize the public sector and to make a wider use of information and communication technologies ("ICT") for provision of services and inclusion. This is evidenced by the launch of the One Stop Shop ("OSS") (Public Service Centers), an innovative organization, which, since 2004, has provided services of different government bodies at one location. 61 For over a decade, access to information on public services has been significantly improved, and alternative service delivery channels have been introduced via face-to-face OSS, e-government, and mobile technologies. 62 According to the 2014 United Nations E-Government Survey, Kazakhstan has improved its ratings from the 81st position in 2008 to the 28th position, out of 192, in 2014. 63 Conditions were created that allow for the citizens and businesses to report corrupt practices through call centers and blogs of the government bodies. 64 In addition, adoption of the law, "On Public Services," in 2013 led to the reduction of the number of permits and licenses for various business activities and has improved access to public services. 65 More importantly, policy makers in the country have repeatedly indicated that fighting corruption is a priority. 66 President Nursultan Nazarbayev, upon ordering the adoption of the Anti-Corruption Strategy of the Republic of Kazakhstan before the end of 2014, made clear that "the state should create conditions under which it will be impossible to use official powers for personal gain." 67 The Kazakhstan-2050 Strategy defines "corruption" as a direct threat to national security, and appeals to the state and to society to collectively fight against this scourge. 68 The Kazakhstani government authorities recognize the challenge of corruption and have taken major steps to develop measures for reducing the level of corruption in the public sector. 69 Kazakhstan was one of the first countries among the CIS countries to adopt the "Law on Fighting Corruption" in 1998 and the "Civil Service Law" in 1999. 70  September 2016, the Ministry for Civil Service was reorganized back into the Agency for Civil Service and Anti-Corruption. 73 In view of changing values and attitudes in the society, Kazakhstan introduced the new Ethics Code in December 2015 and replaced the former Code of Honor of Civil Servants of Kazakhstan. 74 The new Code of Ethics extends the standards of conduct with situational conditions of activity of civil servants, and elaborates on the behavior of civil servants in cases of conflicts of interest. 75 In January 2016, Kazakhstan introduced the new position of the Ethical Commissioner, in part to monitor the compliance of the government officials with the Ethics Code, as well as to support anti-corruption measures and education. 76 The new Law on Civil Service adopted in November 2015 introduced open and competitive civil service recruitment, competency-based career development, and a performance-based remuneration system. 77 The Law on Civil Service aims to reduce conditions for protectionism, nepotism, and patronage in the civil service. 78 The new Law on Fighting Corruption introduces a new vision of the corruption offenses defined by the authorized body in the field of anticorruption. 79 The document discloses the concept of "corruption," "anticorruption policy," and "corruption risk," and also introduces new anticorruption measures via anti-corruption restrictions and anti-corruption monitoring. 80 Anti-corruption monitoring will be carried out by all actors  combating corruption, including the authorized body on anti-corruption and its territorial divisions, and public authorities. 81 The results of the monitoring of anti-corruption are the basis for the analysis of corruption risks. 82 The subject of the analysis is to identify corruption risks and the possibility of studying the causes and conditions conducive to the commission of offenses of corruption. The law also reinforced the measure countering corruption as financial control by the declaration of natural persons, the declaration of assets and liabilities, and income of the property. 83 The Anti-Corruption Programme for 2011-2015 states that "the most corrupted areas are . . . public procurement, use of mineral resources, land matters and construction, customs and taxation, where there has been a significant growth in the resulting damage over the recent years." 84 In his address on January 27, 2012, President Nazarbayev instructed the Kazakhstani government to draft a new comprehensive anti-corruption program. 85 The presidential address on January 17, 2014, "The way of Kazakhstan-2050," emphasized that continued efforts to develop and implement a new anti-corruption strategy was an urgent priority. 86 Kazakhstan developed two parallel anti-corruption programs following the President's instruction: the Anti-Corruption Strategy for 2015-2025, developed by the Agency for Civil Service and Anti-Corruption in cooperation with the representatives of key government stakeholders and international experts 87 ; and the Anti-Corruption Program of the Nur Otan Party for 2015-2025, developed in consultation with international experts and the public. 88 The key goals outlined by the Anti-Corruption Strategy for 2015-2025 are: 1) fighting corruption in civil service; 2) introducing a public monitoring institute; 3) reducing corruption in quasi-government and private sectors; 4) minimizing corruption in court and law enforcement bodies; 5) building anticorruption culture; and 6) developing international cooperation on the issues of fighting corruption. The improvement of the country's position in the international corruption ratings, including the CPI rating of Transparency International, is one of the key performance indicators of this program along with other indicators such as quality of public services, public trust to the government, and the level of legal literacy among the population. 89 The Concept of Legal Policy of the Republic of Kazakhstan for 2010-2020 calls for greater accountability for corrupt activities. 90 The new Criminal Executive Code introduced a lifetime ban from civil service for persons who have committed crimes related to corruption. 91 Anti-corruption legislation sets the norms for confiscation of property obtained by criminal means and imposes personal responsibility on the heads of the public organizations for anti-corruption cases that are within their organizations. 92 Government officials are regularly trained in anti-corruption. 93 The reputation of Astana EXPO-2017, the President's effort to showcase the country to the world, was undermined by the corruption scandal involving some of its top managers. 94 The former chairman, Talgat Yermegiyayev, of "Astana EXPO-2017," embezzled approximately $22.4 million from the state-owned joint stock company. 95 There have been many cases of top officials being convicted of corruption, including at the central level (in particular, the cases of the ex-Prime-Minister; the ex-chairmen of the Statistics and Antimonopoly Agencies; and the ex-vice-ministers of Agriculture, Defense, Education, and Environmental Protection) and regional level (in particular within the oil-rich area of Atyrau and Mangistau). Nevertheless, there are some concerns that some of these measures were taken for political reasons. 96 Punishments for corruption crimes were strengthened and the definition of a "government official" was extended to managers of companies in which the government holds more than a 35% stake under the Law on the Fight against Corruption in December 2009. 97 Furthermore, whistleblower protection was reaffirmed, and punishment of state officials who failed to report corruption cases was introduced. 98 Whistleblowers can access hotlines of different government bodies including the Agency of the Republic of Kazakhstan on Civil Service and Fighting Corruption to anonymously report acts of corruption. 99 However, in practice, whistleblowing is considered taboo and a breach of corporate loyalty. 100 Although there have been several anti-corruption programs enacted since 2001, 101 it is difficult to evaluate the effect of anti-corruption measures solely from the number of measures implemented, each of which have different impact weights on various areas. Despite numerous anti-corruption measures, protectionism, nepotism, and business protection are still flourishing in Kazakhstan according to some analysts. 102 On May 6, 2015, the President announced the nation's plan of "100 concrete steps to implement five institutional reforms" which includes "ensuring the rule of law" and "transparency and accountability of the state" among key priority areas. 103 Several steps are aimed to strengthen the fight against corruption such as the implementation of new standards through the development of a civil service code of ethics overseen by a special commissioner; establishment of a special unit in the Agency dealing with systemic prevention and measures against corruption; adoption of a new Civil Service Law which will be applicable to employees of all state agencies, including law enforcement bodies; and comprehensive performance reviews of all civil servants combined with a new result-based payment system. 104 In light of the foregoing, it should be clear that Kazakhstan is a very complex case. It is very unique, and one could argue that the reason why international measures do not seem to do a great job at tracking corruption in the country is simply because Kazakhstan's combination of history, legacy, economic success, and reforms is truly exceptional.
If Kazakhstan's exceptionalism were the reason why the international measures of corruption fail to provide valid and reliable estimates of how much corruption there is in the country, then the international measures should provide valid and reliable estimates of corruption levels in other jurisdictions.

V. Faulty Method or Faulty Data?
Analysis of the data shows that international measures of corruption can also be problematic in other national settings both in terms of validity and reliability.
With regard to the validity, the results of our correlation analyses reveal that the corruption levels estimated by the basis of CPI are only weakly and insignificantly related to corruption levels measured on the basis of the Control of Corruption Variable computed by the WGI.
As previously discussed, the problem of the validity of the corruption estimates can be observed in the case of Kazakhstan and in other successor states (Russia, Uzbekistan), but it can also be detected in countries with different legacies such as the Gambia and Ghana.
The correlation analysis reveals that, though properly signed, the correlation between CPI and Control of Corruption estimates failed to yield statistically significant coefficients with the exception of Moldova and Indonesia-which are the only cases in which these corruption scores are cross-validated.  With regard to the reliability of the data, the results of the correlation analysis presented in Table 5 make it quite clear that CPI estimates are somewhat unreliable in the Russian case, where three of the six correlation coefficients fail to reach statistical significance, and they are very unreliable in Moldova where all the correlation coefficients are statistically insignificant. While the unreliability of CPI estimates in Kazakhstan, Russia, and Uzbekistan-all of which are successor states-could lead one to believe that the problems that CPI estimates confront in the post-Soviet countries may be due to a common, regional effect, the fact that the CPI estimates are very unreliable in the Gambia indicates that the reliability problems affecting CPI estimates are not the result of a regional effect but rather (1) faulty methodologies in generating such estimates of corruption levels 105 or (2) bad data despite good methodologies.
If CPI estimates were unreliable in each country in the world, their global unreliability could sustain the claim for faulty methodology. Although CPI estimates are unreliable in the case of Kazakhstan, Gambia, Moldova, and, to a lesser extent, Russia, there are several other cases in which CPI estimates 105. Knack, supra note 24, at 264. are very reliable and stable over time. 106 The results of the correlation analyses presented in Table 6 reveal, in fact, that CPI estimates can be highly reliable both in countries with a Soviet legacy, such as Uzbekistan, as well as in countries, such as Ghana or Indonesia, that do not have such a legacy.
More importantly, the fact that CPI estimates are highly reliable for some countries fails to corroborate and falsifies the claim of faulty methodologies. If the claim of faulty methodologies were true, the CPI's corruption scores would be problematic in each and every case. However, this is not what our data analysis reveals. 107 International measures of corruption are problematic in terms of validity and reliability. However, the fact that in some cases, such as in Indonesia, CPI methodologies can generate both valid and reliable estimates suggests that the problem is not so much in the methodologies employed to generate such scores, but rather in the data or in the information that these methodologies process.
Our findings have three sets of implications. First of all, they suggest that in most countries international measures fail to provide a proper indication of whether and to what extent corruption levels have changed over time and, as a result, should not be used to judge whether and to what extent anti-corruption policies were actually successful in curbing corruption. Second, our findings suggest that in order to be able to properly assess corruption levels and changes thereof, it is essential to rely on more than one measure. Third, our findings suggest that analysts need to develop and/or get access to better, more reliable, and more precise data in order to generate good and credible estimates of corruption. And because we know that in some cases, international measures of corruption have problems of validity while in other cases they have problems of reliability, some attention will have to be paid to why the data/information processed to generate corruption scores is not valid in some settings and not reliable in others.

VI. Measuring Corruption: Three Kazakhstani Solutions
In order to bypass both possible and documented problems, various Kazakhstani institutions have developed new methodologies to measure and track corruption. In this section we review two such approaches. One approach is by the former Academy of Financial Police and attempts to measure corruption in objective terms. The second approach was developed by the NOPPI and attempts to measure the perceived level of corruption.
The first methodology, which is by the former Academy of Financial Police, is a new, objective measure of corruption and is estimated in the following way. 108 First, the analyst must quantify the total number of employees in a given state body. 109 This number, which is coded as "Po," indicates the total number of persons authorized to perform state functions. 110 The second step is to identify and count the number of persons that have committed corruption-related crimes, the number of persons involved in administrative offences, and the number of people subjected to disciplinary measures and penalties for corruption-related activities. 111 The number of people committing a corruption related crime is coded as "Ly," the number of people committing administrative offense is coded as "La," and the number of people subjected to disciplinary measures is coded as "Ld." 112 Since corruption related offenses are more serious or severe than administrative offenses and disciplinary matters, they receive a higher coefficient.
Specifically, corruption crimes receive a score of 3, administrative offences receive a score of 2, and disciplinary matters receive a score of 1. 113 After computing the percentage of corrupt individuals out of the total number of employees in a state body, the analyst multiplies these percentages by the coefficient. 114 The analyst then computes the overall index of corruption for a given state body ("Pk") by adding the three numbers generated in this way. This formula can be used to calculate the level of corruption of several state bodies to identify which of them are more exposed to corruption or, for the sake of simplicity, more corrupt.
113. Id. 114. Id. The former Rector of the Financial Police Academy computed the level of corruption on the basis of data collected for 2013. The evidence presented in Table 8, taken from the former Rector's presentation, illustrates that the number of corruption offences ranges from 3 in the Office of the Public Prosecutor to 341 in the Ministry of Internal Affairs. Once we account for the number of employees in each of these bodies and we compute the level of corruption, we find that the corruption index ranges from 0.08 in the Armed Forces to 3.0 in Customs. Specifically, the highest incidence of corruption is found in Customs, followed by the judiciary, the Tax Agency, the Ministry of Internal Affairs, the Financial Police, the Office of the Prosecutor, and finally, the Armed Forces.
By adding up the corruption scores for each sector, we find that the public sector for Kazakhstan in 2013 had an overall corruption coefficient of 8.84.
By performing a similar analysis for the following years, the Academy of Financial Police believes that it will be able to precisely track not only the overall level of corruption in the country, to assess whether and to what extent anti-corruption measures introduced by the government are effective in curbing corruption, but will also be able to monitor how corruption varies across sectors and over time. 115 On the other hand, the Public Policy Institute of the Nur Otan Party has adopted a rather different approach. Instead of measuring corruption on the basis of objective measures, the party devised an index of perceived corruption, or, as it was tentatively called in the early stages of the project, the National Corruption Perception Index. 116 This index is computed on the basis of survey data. It can be used to assess the perceived level of corruption across regions and sectors and can eventually be employed to track variations in the level of corruption over time. 117 The survey to collect the relevant information asks respondents to indicate: a) how much corruption they believe there is in the country/region, b) how much corruption they believe is in a variety of sectors, and c) how pervasive they believe various forms of corruption (nepotism/clientelism/favoritism; abuse of resources/embezzlement; and bribery) are. 118 The first set of data indicates the perceived level of corruption across the Republic of Kazakhstan's sixteen subnational units (fourteen regions and the cities of Astana and Almaty). The second set of data indicates the perceived level of corruption across various sectors. The third set of data indicates the pervasiveness of three distinct forms of corruption that closely resemble the indicators employed in the computation of the GCI.
After recoding the responses into five-point scales, the index is then computed based on the following root-mean-square formula and rescaled to a one-hundred-point scale: X1 is the average of the responses concerning the level of corruption in the regions. X2 is the average of the responses on the level of corruption in the various sectors. And X3 is the average of the responses on the incidence of three different forms of corruption. It is impossible, at the moment, to say whether and to what extent the methodologies devised by the Academy of Financial Police and by the Nur Otan Public Policy Institute will generate better estimates in terms of validity and reliability than those generated by the international organizations. Nevertheless, it is a positive development for a country that is often criticized for allegedly having high levels of corruption because it demonstrates a genuine commitment to estimating corruption, understanding its causes, identifying the areas in which corruption is more pervasive, and assessing the impact of the anti-corruption measures adopted by the government. Even more remarkable is the fact that in a country generally not known for its pluralism, there has been a vibrant and theoretically informed, methodological debate and a plurality of perspectives on how corruption levels could and should be estimated. 119

Conclusion
The most unambiguous conclusion of the present analysis is methodological: when the methodologies devised by international organizations such Transparency International or the World Bank are applied in the case of Kazakhstan to estimate the level of corruption in the country, the corruption estimates generated seem to provide a somewhat inadequate picture. In fact, the estimates generated in this way are somewhat inconsistent with one another and seem to indicate major upward and downward swings in corruption levels that may not adequately capture the real level of corruption in Kazakhstan. A second and, in our view, more important conclusion is that this analysis also identifies a previously neglected "explanation" for the failure of these estimates in assessing the proper level of corruption. These estimates "fail" not because Kazakhstan is an exception or because the methodologies devised by international organizations are inherently problematic, but because the data they analyze to estimate corruption are, in the case of Kazakhstan but possibly in other cases as well, rather problematic.
Given the problems that international measures of corruption seem to encounter in the case of Kazakhstan and given the origin of such problems, several institutions in Kazakhstan have developed new approaches to estimating corruption. The former Academy of Financial Police of Kazakhstan, using what was regarded as a "criminological" approach, created an index of corruption that is objective, or fact-based, in the sense that it infers the level of corruption of various sectors and subordinately of the whole set of government institutions from the incidence of criminal, administrative, and disciplinary sanctions punishing relevant violations. The Public Policy Institute of the Nur Otan Party, Kazakhstan's ruling party, using what the Academy regards as a "sociological" approach, created a National Corruption Perception Index to assess the incidence of various forms of corruption across regions and sectors and, subordinately, in the country as a whole.
There are at least four reasons why the methodological exercises undertaken by Kazakhstani institutions represent a positive development. First, they testify to the fact that Kazakhstani institutions are genuinely committed to studying, countering, and reducing corruption. Second, since these methodologies are locally produced with locally collected data, there is every reason to believe that the data they employ are more precise than those utilized by international organizations to estimate corruption levels. Third, the corruption levels estimated with the methodology devised by the former Academy of Financial Police and by the NOPPI could represent a valuable benchmark against which to test the validity of the international measures. Fourth, and in our view most importantly, the fact that Kazakhstani institutions have devised both fact-based (or objective) and perception-based (or subjective) measures of corruption will enable scholars and practitioners to explore whether and to what extent perceptions of corruption relate to or are affected by the real incidence of corruption. By exploring whether variation (across sectors, regions, and time) is a function of changes in the objective level of corruption or whether such variation should be ascribed to other (cultural, cognitive, normative, etc.) factors, it will be possible to develop a better understanding of the factors responsible for the perception of corruption. Such a contribution would be quite interesting not only for Kazakhstani policy makers, but also for a wide range of comparative politics specialists.