{'fraction_non_alphanumeric': 0.021242754135444648, 'fraction_numerical': 0.011416654884773081, 'mean_word_length': 5.139973958333333, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Does Quality Make a Difference for Higher Education Graduates in Colombia? Measuring Differences in Accredited and Non-accredited Institutions With Similar Financial Characteristics

In this paper we explore the difference of quality in higher education between accredited and non-accredited higher education institutions (HEIs) in Colombia. For that purpose we test if the difference of the National Exam of Student Performance (Saber Pro), which is our proxy for quality between institutional accredited and non-accredited institutions is statistically significant. The hypothesis is that indeed those HEIs that have been granted the institutional quality accreditation by the Colombian ministry of education should have better results in the national exam than those that not. We argue that for robustness of the results, it is necessary to control for confounding effects among comparable HEIs. For this purpose, we employ a propensity matching score approach based on common financial characteristics in order to avoid issues of selection bias. Our results find that the difference in performance between accredited and non-accredited institutions is positive and statistically significant. This difference between performances can be attributable to the substantial differences in the pattern of expenditures and asset investment of non-accredited HEIs. JEL Classification: I21, I22, I23


Introduction
Around the globe and particularly in the last decades, governments have been focused in improving the quality of higher education. Higher education has become an essential component of national policy in order to enable economic in the form of a highly skilled workforce [1]. Alexander [1] argues that higher education institutions nowadays face more demands for accountability in quality from national governments than ever before. The main reason is that national governments are still the major source of financing for higher education institutions (HEIs) through either direct transfer (public higher education institutions) or indirect ones through national loan programs (private institutions). In order to assign these resources efficiently, governments rely on systems that try to measure the quality of the education imparted by higher education institutions. Although these quality systems are criticized due to their utilitarian quantitative nature, the trend is that they will only continue to grow and that those HEIs that does not adapt are condemned to see their financial resources from government sources reduced drastically [2].
Although national quality systems for ensuring the quality of higher education are quite heterogeneous, there seems to be consensus around which are the common determinants of a successful quality system. Williams et al. [3], in their multi country survey of higher education quality systems, argue that the most common determinants are: resources, environment, connectivity and output. Resources refers to the extent in which teaching and research is either financed by the government and the private sector adjusted accordingly to the type of HEI (public or private) that predominates in a given country. Environment refers to the role of government regulation in the higher education sector, the conditions of academic employment and the extent of the mix in the supply of higher education between public and private HEIs. Connectivity refers to the extent of internationalization of the student body and in the number of research articles or activities conducted with other HEIs at both the national and international level. Finally, output refers to the impact of research, teaching and training. In the case of research, output is measured by the impact factor of all HEIs in a given country, and similarly teaching and training by the numbers of graduates and their reported employability at the national level.

Literature Review
In Latin America during the 1990s and 2000s the process of globalization led to an exponential growth in the offer of higher education services. This growth was led mainly by private sector universities with different levels of quality. As a common denominator in the region, much of the growth was achieved by offering programs of dubious quality. Therefore, in order to control for quality, many of the national governments created institutions for accrediting the quality of the programs offered at the national level [4] (Lamarra, 2003). However, it is important to mention that although there were problems with quality in the low end of the spectrum, at the high end of the spectrum many private sector universities in Latin America have been responsible for expanding participation rates among students. Many authors agree that a national system benefits from having both private and public institutions, and that the most effective form of government regulation is the one that sets the rules in terms of quality, effectively evaluates performance but allows for some degree of autonomy in the implementation of quality systems by the institutions that conform the national system [5], [6], [7], [8].
In this context, Colombia in 1992 was among the first countries to establish a National Accreditation Council [9] 2 . In Colombia this institution acts as an adjunct organism of the Ministry of Education and is the organism in charge of assessing the quality of the programs that compose the national HEIs system. The CNA is responsible for either granting the accreditation in quality to individual programs and at the institutional level. The accreditation at the institutional level is the hardest one to obtain, and more desirable than the one at the program level, because is grants the institution certain benefits such as reducing bureaucracy in the approval of new programs of study and greater access to government funding in the form of student financing [9]. As with any standard process of accreditation the process relies heavily on a peer review of the autoevaluation report, the recommendations by the peer committee, and the results of previous accreditation processes of programs at the national and international level.
One particular characteristic of Colombia is that along with Brazil is one of the few countries that have some direct measure of the determinant of output in terms of teaching in their national quality system. In Brazil as in Colombia, undergraduate students are required to present a national exam of student performance which in Brazil is known as ENADE and in Colombia as Saber Pro. In Colombia the Saber Pro is a mandatory requirement for graduation. The Saber Pro is a standard exam that is common to all programs and measures five competencies in critical reading, civic competencies, quantitative reasoning, English (as the foreign language) and written communication. In recent years, as happened with Brazil, the exam has become a quantitative tool for the assurance of the quality of education and the ranking of HEIs in the country [10], [11].
One of the main criticisms towards private HEIs is that some of them are driven just by financial motives rather than for the public good [12]. The argument is that in profit driven HEIs there is an incentive to overlook quality. Even though in Colombia by law HEIs have to be conformed as non-profit foundations, this does not act as a deterrent for the "founders" to engage in profit taking activities via associate supplier or other perks deriving from their job position at the institution.
Although national exams of student performance are far for being a perfect measure of graduate quality, it is a reasonable proxy quality indicator in which accreditation agencies can rank HEIs output in terms of teaching. In recent years, as happened with Brazil, the exam has become a quantitative tool for the assurance of the quality of education and the ranking of HEIs in the country [10], [11]. In Colombia, resources and the financial characteristics associated with it is one of the most important determinants for obtaining the institutional accreditation. The basic idea in the context of the Colombian accreditation system is that financial characteristics should not only reflect the strength of the balance sheet and the income statement, but instead how these resources are applied to other measures of quality such as students and academic personal quality conditions. In this context, certain financial indicators can be considered a proxy of quality as long as they relate to the HEIs mission of providing high quality education. In a recent legal mandate of 2014, page 13, the CNA [9] has made it clear that "… the net income surpluses should be reinvested in the core functions of the institution, if this is not the case, the institution cannot be considered as a candidate for accreditation, or in the case of being previously accredited, the failure to comply will result in the loss of the accreditation 3 " [9]. Basically, the idea is that those HEIs that comply with the accreditation guidelines would be the ones that will attract the highest number of student due to the fact that government financial aid will be limited to the HEIs with institutional accreditation. Therefore, the main objective of this paper is to test if the differences between accredited and nonaccredited institutions in selected determinants of quality are statistically significant when controlling for similar financial characteristics. The remainder of the paper is organized as follows: in section II we describe the data employed in our study, in section III we

Data
For this study we used the dataset employed in Cayon, Correa, and Sarmiento-Sabogal [13], in our previous study we merged three different datasets, the first dataset contained the financial information for the fiscal year 2013 of the country most important HEIs. The reports gives detailed information about the composition on revenues, assets and expenditures of participant HEIs as well as detailed information of academic and administrative personnel [14]. The second dataset are the results of the national exam of student performance (Saber Pro) for the years 2012 and 2013 for the same institutions of the first dataset. As explained in detail in Cayon, Correa, and Sarmiento-Sabogal [14], the dataset has discriminated about student performance in five areas of study which are: Critical reading, civic competencies, quantitative reasoning, English as a second language and written communication. The original source of information is form the Instituto Colombiano de Fomento a la Educación Superior (ICFES) which ranks the test in term of quintiles in order to compare results, the quintiles are in inverse order, where 5 is the highest one and 1 the lowest. The only two components that are rated by letters are English and written communication, the first is rated in terms of letters being A-the lowest level ad B+ the highest, and the second is ranked in 8 levels being N1 the lowest and N8 the highest [15]. Finally, the third dataset is the historical student dropout rate of all the HEIs under study as provided by the statistical system of the Colombian Ministry of Education [16]. Our final dataset consists of a total of 123 institutions where we have 82 private and 41 HEIs, from those 28 are accredited and 95 non-accredited. In Cayon, Correa, and Sarmiento-Sabogal [13], we used the same dataset to observe the differences between public and private institutions in this study we use the same information to study the difference between accredited and non-accredited institutions. From our dataset 4 , we can conclude that the average revenue of HEIs institutions in Colombia 2013 was in the order of 83,616 million Colombian pesos or approximately USD 45 million. Average operating expenses were in the order of 75,875 million pesos or USD 41 million and approximately 49% of them belong to teacher expenses and 39% to administrative expenses. Net income is in average 7,342 million pesos or USD 4 million. Average assets are in the order of 213,617 million pesos or USD 115 million. The average undergraduate population is in the order 8,515 students and the average income per student at the national level is 9,39 million pesos or roughly USD 5,000. The high difference between tuition revenue and total revenue is explained by direct public transfers to public universities which also explain the difference between income per student and tuition income. Finally, the average score of the Saber Pro in 10.10 with range 11,60 and 9,29, and an excellent exam is any score above 11, a very good exam between 11-10.7, a good exam between 10.6-10.3 and anything below 10.3 is below average [17].

Methodology
In order to estimate the impact of financial characteristics on the results of the Saber Pro we use the same base regression as in Cayon, Correa, and Sarmiento-Sabogal [13] which has the following specification: The definitions variables are defined as Cayon, Correa, and Sarmiento-Sabogal [13], p. 295 and we can see from table 1 that the mix of the balance sheet in terms of equity and liabilities, other income, and administrative expenses are the most significant variables in explaining the results obtained in the Saber Pro by a particular HEI. Furthermore, in this study, we disaggregate total revenue and operating expenses in order to observe the effect that each source of income or major item of expense has on the Saber Pro 2013 score. We also use the accounting identity (equity + liability = assets) in order to account for the effects of capital structure of the HEIs on the Saber Pro 2013 score.
In the next step we use the same variables of equation (1) in order to obtain the implied probability of being an accredited institution using the following logit for  is an indicator function that takes the value of (1) if the HEI in the sample has an institutional accreditation or zero (0) otherwise; , it X is a vector that contains all the explanatory variables of equation (1). Once we obtain the coefficients of interest and the predicted probabilities of the cumulative standard logistic distribution ( , Pr(D 1) accredited t  ) from equation (2), we can compute the fitted cumulative probability that the observation is either accredited (our 'treated' variable) or nonaccredited (our 'non-treated' variable): Once we have estimated these probability values for all the HEIs at each point in time, we implement a matching procedure that we will describe in the following paragraphs.
Our procedure for testing differences in spreads is based on the Average Treatment Effect on the Treated (ATET) framework. This procedure uses the probabilities obtained in equation (3) and the original data in Table 1 to make a selection of counterfactual values based on propensity score matching. This procedure has certain advantages over traditional sampling or predicted values difference testing since it effectively addresses the problem of selection bias of comparable sample groups drawn from the nonaccredited HEIs. One key advantage of this method is that we can compare the actual values of our descriptive data without forgoing the richness contained in the observable characteristics of a regression model. Moreover, with ATET it is possible to determine which non-accredited institutions are more closely related in terms of common financial characteristics to those that are accredited which can have important implications regarding policy making.
This method was originally developed by Rosenbaum and Rubin [18] in order to address the nonrandomness of treated vs. non-treated groups in medical trials, and since then has been applied to other areas of the social sciences such as labour economics, policy research and finance. In this paper, we modify the framework proposed by Nssah [19] on how to apply ATET to economic policy programs and reframe it for our specific context.
Here, the "treated" group is characterized by a dummy that represents the accredited institutions (D=1) and the "non-treated" which are represented by the non-accredited institutions (D=0). Therefore, by dividing the data in our study into two vectors that represent the data of accredited ({ accredited data }) and non-acredited ({ non accredited data  }) using the algorithm in equation (7) we have: Where X is the vector of common observable characteristics represented by the explanatory variables from equation (2) and the averages of represent respectively the mean of the "treated" and the counterfactual mean of the "non-treated" or, in our setup, the accredited and non-accredited period. ATET using propensity matching estimators represents an interesting framework for testing contagion because the method yields strong estimates under the assumption of conditional independence [20]. The assumption can be formally defined as: In other words, conditional on observable characteristics (X), participation (D) is independent of the potential outcomes of   , accredited non accredited data data  . In order to be coherent with the principle of conditional independence, the basic idea behind propensity matching is to randomly select a sample from the non-crisis (non-treated) HEIs that most closely resembles the characteristics of our sample in the crisis (treated) HEIs. In other words, conditional on the common variables, the counterfactual observations of the non-accredited institutions will be the one that more closely resembles in terms of conditional variance those observations during the crisis period. Since the counterfactual group is selected randomly based on the closest characteristics with a treated observation, any source of endogeneity due to selection bias is effectively addressed.
Using the probability values from equation (3) we can implement the algorithm in equation (7) Where matched, () t cp represents the vector of matched accredited and non-accredited spreads based on the nearest difference propensity scores which are simply the one minus the cumulative probabilities obtained using equation (3), where (paccredited) are the 5 Unit homogeneity refers to the fact that participants cannot choose to participate in the experiment, so the experimental cumulative probabilities for those observations that belong to accredited institutions and (pnon-accredited) are those of the non-accredited institutions. The vector that represents the non-accredited institutions ({ non accredited data  }) is constructed by selecting the non-accredited institutions data that match the corresponding data points of the pnon-accredited cumulative probabilities obtained with equation (7). Therefore, we can find evidence if there is difference in the data of accredited and non-accredited institutions by testing if the average of the matched vector   i g is statistically significant via a simple ANOVA test where the null of no differences in a certain vector versus the alternative is formally defined as: . In this way, we observe the impact of that an institutional accreditation has in the data to the data that most closely resemble similar financial characteristics of counterfactual of non-accredited institutions. This approach is not new to the literature of the subject and has been employed previously to explore the question of student performance and the impact of financial aid in private universities in Mexico [21].

Results and Discussion
In Table 1, we can observe that the significant financial characteristics in relation to the Saber Pro are those one related to capital structure with a positive sign. This means that independent of the source of financing in average a higher number of assets or capital expenditures by Colombian HEIs have a positive impact on the Saber Pro scores. The only component of revenue that is significant is other revenue with a positive sign. This can be explained by the fact that in the case of the largest private and public HEIs in Colombia another income is represented by academic consultancy, editorial income, and other diverse sources such as university hospitals which all are related to activities of social engagement. Therefore, a plausible explanation is that HEIs with higher social engagement attracts a student pool that has a better performance score on the exams. The only group is comprised of both volunteer and non-volunteers; there is no bias based on the willingness of the participants to be a part of a given experiment. significant expense variable with a negative sign is administrative expenses. This means that a higher investment in support or administrative staff which can be interpreted as higher bureaucracy has a negative impact on the student performance exam score, so expending should be geared towards teaching staff rather than administrative staff. In Tables 2 and 3 we present the results obtained from applying the process for obtaining propensity matching estimators as explained in the previous section, where we can observe the difference between accredited (treated) and non-accredited (non-treated) selected random counterfactuals HEIs in Colombia. In Table 3 we present the differences between all the components of the Saber Pro 2013 and in Table  4 the same differences of the Saber Pro for the year 2012:  Table 2 we present the results obtained from running equations (2) to (8). The column difference denotes the Average Effect on The Treated or the difference between accredited (treated) and non-accredited (non-treated) institutions that compose our sample paired by common financial characteristics. Q1 represents the worst performing students in the test as part of the total population that took the test in 2013 and Q5 the best performing. The writing quintiles and the English Levels also represent the students as a percentage of the population that took the test with N1 being the lowest and N8 the highest in writing and A-the lowest performers and B1 the highest performers in English. *** 99%. **95%, *90% significance level  Table 3 we present the results obtained from running equations (2) to (8 As we can observe from Panel A in Tables  2 and 3 accredited institutions had 55.26% and 56.61% of the students above the A2 (Sufficient) level as opposed to 16.31% and 18.05% of non-accredited institutions. One can argue, that socio economic variables such as income have a direct impact in the level of English, but we control for those unobservable effects by taking into account the fact that among accredited institutions, a large percentage belongs to public universities, whose majority of the student body comes from a more varied socio economic background that their private accredited counterparts. All the differences between accredited and non-accredited institutions are statistically significant with the exception of the A2 English level in 2013 were there is no significant difference between the performances of accredited and non-accredited institutions. In order to provide a more complete picture as to what may be the reasons behind the differences between the performance of accredited and non-accredited institutions, we did the same matching procedure for other quality indicators based on similar financial characteristics. In Table  4 we present the results of the differences among selected quality indicators:  Table 4 we present the results obtained from running equations (2) to (8 From table 4 we obtain some important results; in first place the difference between the average revenue growth of accredited HEIs and nonaccredited HEIs in the period comprehended between 2011 and 2013 is not statistically significant. This means that revenue growth is the same in both kinds of institutions so one can infer that proportionally to the student population they receive similar revenues. Also the difference between the number of full time professors and part time professors is not statistically significant. However, non-accredited institutions have a higher number of adjunct lecturers than accredited ones (approximately 204 more in average) and the difference is statistically significant. In the case of income per student and tuition per student accredited HEIs command a premium of 4.72 (USD 2500) and 2.80 (USD 1500) million Colombian pesos and the difference is statistically significant. In the case of operating expenses and teaching expense per student accredited HEIs expend in average 5.48 (USD 2900) and 2.43 (USD 1300) million Colombian pesos than nonaccredited HEIs and the difference is statistically significant. In the case of students per teacher and number of campuses the difference is not statistically significant between accredited and non-accredited institutions. Finally, when we compare asset per student which is a proxy measure of the physical resources devoted to students the difference between and accredited and non-accredited institution is in the order of 7.09 (USD 3800) million Colombian pesos. This means that even though accredited and nonaccredited institutions have similar revenues is basically in how they expend the revenues in search of quality. Although both kinds of HEIs have the same number of professors, professors in non-accredited HEIs probably get paid substantially less than their accredited counterparts. Also, there is evidence than nonaccredited HEIs rely more on adjunct lecturers than accredited ones, and can have a negative impact on student performance. One reason is that probably these adjunct lecturers from nonaccredited institutions have lesser opportunities for advancement, and lesser resources to engage in pertinent research which at the end translates in underachieving student performance. Another variable of concern is the statistically significant difference between asset per student of accredited and non-accredited HEIs. This is a clear sign that non-accredited institutions invest substantially less in the wellbeing of their students than their accredited counterparts. This means that maybe non-accredited institutions in Colombia are more concerned about reducing costs and maximizing profit which has been a recurrent issue of ethical debate in higher education systems around the globe [21].

Conclusions
By using a propensity matching estimator approach, we tested for significant statistically differences between accredited and nonaccredited HEIs in Colombia. The counterfactuals among non1-accredited HEIs are selected randomly based on similar financial characteristics in order to avoid selection bias. Our results show that indeed there is a positive difference in performance in the National Student Performance Exam (Saber Pro) between students in accredited and non-accredited institutions is statistically significant. The difference can be attributed to other statistically significant differences in quality indicators such as the pattern of expending and HEI investment per student. There is conclusive evidence that students from Colombian HEIs that have obtained the institutional accreditation perform consistently better than students from non-accredited HEIs. It is important to remember that the institutional accreditation does not generate significant differences in the growth of revenue, which can be considered also as a warning system for public policy as to why quality is not generating a differential in gross revenue growth. In the case of Colombia, these findings can serve as the basis for a more in-depth discussion as to how public resources are being distributed for the strengthening of the quality assurance system. These resources should be geared towards those HEIs that can demonstrate a major social impact, transparency in the management of financial resources and better results in graduate performance. This new policy should be focused on those HEIs that are still non-accredited in order to assure the quality of education imparted. Ultimately, it is also responsibility of the state to ensure quality, since education has been long considered a public good. One way to enforce quality is to ensure measures that effectively intervene or close those HEIs that are not willing to participate or to engage with the national quality assurance system. Even though the Colombian government has reinforced control measures in recent years, there is still room for improvement.