{'fraction_non_alphanumeric': 0.020615072659682324, 'fraction_numerical': 0.0037174721189591076, 'mean_word_length': 5.244725738396625, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The Politics of Evaluation and the Methodological Imagination

As Ernest House (1973) observed many years ago, evaluation is as much used as a resource for gaining legitimacy or mounting a defense or an attack as it is as a contribution to democratic deliberation and the basis of informed decision making. Mostly executive decision makers do not want to be told that things are complex and open to different interpretations and valuing; they know that. It is the way out of or around the complexity and the plurality of interests and values that they want help with. They want to use evaluation as a resource to solve problems, not pose or redefine them. Some of the problems they want to address are social problems. Other problems are creatures of the politics of government: avoiding embarrassment, displacing blame, deflecting criticism, maintaining reputation, legitimating action or inaction, reordering priorities, justifying budgets. To the governmental frame of mind, beset with accountability, other people’s autonomy is a problem. It is a source of contingency, ambiguity, and unpredictability and a potential for loose cannons. The increasing tendency of governments to prespecify the characteristics of good evaluation by providing guidelines and standards for their evaluation contracts stems from an understandable desire for greater predictability and control over the content and process of evaluation. It is a kind of security blanket. For those who see evaluation as a contracted service and evaluators as hired hands, it makes good sense. It is a necessary prelude to managing the transaction costs and risks associated with outsourced services. Perhaps there are other forces at work too. We might see the impulse to regulate evaluation as a process of bureaucratization, an almost inevitable adjunct to the institutionalization of evaluation in so many jurisdictions. The pernicious effects of the bureaucratization of social science were well described by C. Wright Mills (1959) in his now classic exposition of how social science should and should not be pursued. He argued that much “of the propaganda force of bureaucratic social science is due to its philosophical claims to Scientific Method” (p. 105) and noted the methodological inhibitions this created. The association of certain methodological approaches, experimentalism, for example, with strong designs and other traditions, naturalistic inquiry, for instance, with weak designs is not a position that withstands critical scrutiny. This all too common juxtaposition of strong (superior) quantitative and weak (inferior) qualitative designs is not simply tendentious, as Kushner says. It is mendacious. Evaluation designs may be weak or strong, but this is a property of particular designs in particular contexts, not a necessary consequence of methodological traditions. Fitness for purpose and context is the real test of whether an evaluation design is any good. Fitness