{'fraction_non_alphanumeric': 0.02527527031048507, 'fraction_numerical': 0.008987203650431505, 'mean_word_length': 5.440838231535906, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Monitoring for learning and developing capacities in the WASH sector

The water sector faces immense challenges, which are characterized by complex interactions between the social and ecological systems. Improving the linkages between sector-wide monitoring, learning and capacity development is of pivotal importance for coping with this complexity. For deciding where to invest, how to sustain and improve water and sanitation services and for understanding which policies and strategies work, both reliable data and critical joint reflection are crucial. In the water sanitation and hygiene (WASH) sector, two developments have the potential to contribute to increased sector capacities to reflect, analyse and act upon lessons learnt. First, the interest of national governments and donors in monitoring and evaluation, which is gradually shifting from counting infrastructure built, towards sustainable development and sector performance reviews. Second, participatory monitoring methods and multi-stakeholder processes are being used to increase transparency and accountability and to facilitate dialogue, learning and joint action. The demand for continuous learning and adaptive management based on sound monitoring data can be stimulated by incentives and supportive institutional settings. The supply – the mechanisms, tools and capacities for monitoring – must also be strengthened, especially the capacity to use monitoring data to take action. Learning-oriented monitoring processes can help identify capacity gaps, while the process of joint analysis, reflection and sharing lessons has the potential to build capacities. Commitment of stakeholders throughout the sector to do things better and differently is a critical element towards building a learning and adaptive sector.

The context of the water sector is characterized by rapid social and environmental change and considerable uncertainty. Mistakes are made, unplanned directions are taken and unforeseen developments unfold. Insights are gained in the course of the activities. Monitoring can detect early signs of problems and success and provide information needed for decision-making. Practitioners, managers and policymakers need the capacity to analyse, communicate and act on the findings, applying the lessons learnt.
When aggregated, information and feedback from monitoring provide information on the performance of the sector as a whole and on the mechanisms used to deliver water and sanitation services. Monitoring may identify areas for capacity development and institutional, organizational or technical innovation to improve service delivery (Figure 1).
Interest in monitoring has been fuelled by efforts towards generating data on infrastructure built, such as the Millennium Development Goals. Currently, there is a push towards ongoing sector-wide monitoring by government together with other sector players. The drive towards aid effectiveness has sparked additional efforts to improve monitoring as a means to increase accountability and effectiveness of sector investments. This has resulted in various country sector performance monitoring processes including Joint Sector Reviews (JSR) (IRC, 2013c).
Although many monitoring processes have tended to emphasize the information needs of donor agencies and central governments (reporting and accountability focus), there is also a shift towards strengthening stakeholders' capacity to monitor and to take corrective actions (learning and adaptive management focus). We discuss several sector performance monitoring initiatives and their potential for learning and capacity development.
Multi-stakeholder collaboration and participatory approaches are becoming more common in the water sector (Pahl-Wostl et al., 2007). Participatory monitoring and social accountability tools are used as a means of improving service delivery and governance in WASH and engage stakeholders in reflection and learning processes. Increasingly, such approaches and tools are used at scale to complement national monitoring. Sector working groups (Smits et al., 2012) and multi-stakeholder platforms are used to share lessons learnt and create consensus or buy-in for change. Participatory methodologies that engage a wider range of stakeholders in monitoring processes can contribute to the learning capacity of the sector. Multistakeholder collaboration is expected to stimulate learning about complex systems, establish social legitimacy for decision-making and build relationships that reinforce commitments to learning and adaptation in the long term (Pahl-Wostl et al., 2007).
Throughout the paper we present short cases that illustrate links between sector monitoring, learning and capacity development in the WASH sector.

Concepts
In the following section we will outline the main concepts which we use in this paper: monitoring, 'sector learning', adaptive management and capacity development. A short introduction to each of the terms is provided.

WASH sector monitoring
Monitoring is an ongoing process by which stakeholders obtain regular feedback on the progress being made towards achieving their goals and objectives (Donaldson, 2011). Monitoring is done to see whether an expected result is achieved, and to take action if what is observed deviates from what was expected (IRC, 2013c).
WASH monitoring can be limited to the scope of a project, a programme, an organization or encompass the whole sector. Systematic collection of data and joint analysis of success and failure can enhance the learning necessary to innovate and continually improve service delivery. Multi-stakeholder reflection and learning can support the process of turning monitoring information and sector experience into actionable lessons (see Figure 1). Involvement of a diverse group of stakeholders in monitoring processes can provide data, while also contributing to improving the management and sustainability of services and building capacities (Shordt, 2000: 7).
Monitoring information can be used for checking and control, for problem solving and planning, and also as tools for learning and adaptive management by community members and sector professionals. If it is done well, monitoring has a number of benefits: • It brings together otherwise disparate information in one place. • It is easy to reflect on progress over time.
• Good and poor practices can be identified relatively easily. • The causes of good or poor performance can be determined. • Institutional roles can be further developed. • There is a framework for capacity development strategies. • Information for assessing the effectiveness of water and sanitation policy and undertaking policy reviews is readily available.
• The government has a credible system for allocating resources within the sector. • A coherent case can be made for better resource allocation to the sector. • Further research needs can be determined (Danert & Ssozi, 2013).
For monitoring to make a useful contribution to improving the impact of development there must be a greater focus on learning and action. Capacity building is needed to ensure that each stakeholder can perform their role in the process of monitoring and learning: from data collection, storage and analysis to validation of the data, participatory analysis, communication and taking action or motivating others towards corrective action (Danert & Ssozi, 2013).
In this paper we will distinguish between: (1) government-led monitoring at national level which is strongly linked to planning and budgeting; and (2) participatory monitoring with a focus on multistakeholder learning, accountability and joint action. In the first category we will discuss the potential links between monitoring, learning and capacity in JSRs. In the second group we will focus mainly on qualitative information systems (QIS) and accountability tools.

Increased interest in monitoring for public sector management and governance
There is a growing recognition that a performance or results orientation is an important aspect of public sector management. Monitoring can support sound governance in several ways: • Monitoring information can be used in decision-making and setting (budget) priorities.
• It can provide inputs for planning; for example, information on the performance of sanitation infrastructure or on the organizations and enabling environment.
• It can contribute to better accountability of service providers to service authorities, citizens and donors.
The monitoring process itself can encourage those involved in WASH service delivery to engage in collaborative and continuous learning (World Bank, 2001: 2). In many cases, national monitoring systems are principally designed to meet donor data requirements (World Bank, 2006: 13), rather than focusing on learning and service delivery as outlined above. There is a danger that monitoring is seen as a tool for compliance reporting (a 'policing system') rather than as a collaborative process for critical reflection and pro-active managerial action (Camacho Tuckerman, 2007: 28;World Bank, 2010: 19). Simply having a monitoring system in place does not guarantee that the information collected will be used (World Bank, 2006: 6).

Sustainable services require continuous learning and adaptation
To ensure that water and sanitation hardware deliver ongoing services, a range of stakeholders working at different institutional levels must provide their inputs. A broad set of skills and capacities is needed: engineering skills, capacities to design policy and legislation, to set and enforce regulation, to design systems and institutions for contracting, planning and monitoring, capacities for community dialogue, hygiene promotion, marketing, interactive methods in conflict resolution, day-to-day problem solving to name a few.
Factors such as climate change, the socioeconomic development and globalization are increasing the degree of uncertainty faced. This implies that the capacity to learn, anticipate and adapt is critical. The adaptive manager needs new skills and capabilities, informal and flexible management structures, and access to both expert knowledge and local lay knowledge (Pahl-Wostl et al., 2007;Wals & Blaze Corcoran, 2012).
Learning is understood as the process of gaining knowledge or skills, or of developing a desired behavior through study, instruction, or (shared) experience. Learning is viewed as the ability to constantly improve the efficacy of action. To distinguish between different outcomes and levels of learning, the concept of single, double and triple loop learning is often used. Single loop learning is the improvement of already established actions; double loop learning means a change in the frame of initial reference and guiding assumptions (such as system boundaries); and triple loop learning means a transformation of the frame of reference and of the whole regime (Hargrove, 2002).

Towards a learning and adaptive sector
'In today's world […] solutions are often not immediately apparent and uncertainty abounds, moving forward requires a creative, responsive and adaptive outlook. The capacity for learning and innovation becomes paramount in these complex and dynamic processes. These are processes that often require a society-wide perspective on learning and change' (Woodhill, 2004: 2). A learning and adaptive water sector is one where stakeholders are supported and encouraged to systematically capture and share experiences with others in the sector and to develop joint capacity to reflect on lessons learnt in order to adapt and improve service delivery (IRC, 2012).
'Social learning', with an emphasis on collaboration, diversity, citizen-driven and emancipatory approaches, reflection and reflexivity, is essential for adaptive management (Folke et al., 2003;Pahl-Wostl et al., 2007;Wals & Blaze Corcoran, 2012). Social learning at sector level, or 'sector learning' can be described as a multi-scale process: it goes beyond individuals and organizations towards sector-wide concerted action and institutional change for better service delivery (Pahl-Wostl et al., 2007;IRC, 2012).
At sector level, continuous monitoring, joint reflection, analysis and adaptive action are key elements for improved performance, better coordination and harmonization of approaches (IRC, 2012). A sector with the capacity to learn has mechanisms to share and collect data and experience, facilitate reflection and analysis, identify and implement potential innovations. Formal and informal networks, multi-stakeholder platforms and communities of practice support the collaborative generation of knowledge and actions and provide valuable feedback loops (Swartling et al., 2010;IRC, 2012;Kahangire, 2013). A supportive culture for knowledge sharing, continuous learning and evidence-based decision-making is a third requirement for fostering continuous learning and adaptive management (Camacho Tuckerman, 2007;. Peer review among municipalities (see Box 1 below) are one approach to organizing feedback loops and joint learning for performance improvement.

Monitoring and learning: central to adaptive management
Ideally, adaptive management is an iterative process in which stakeholders plan, implement, monitor, learn from and respond to monitoring results. Continuous monitoring can provide information that informs decision-making and planning and can reveal insights that lead to change in attitudes or approaches, and may fuel innovation and change processes (Camacho Tuckerman, 2007). It starts with answering the questions: what is the evidence, why has there been success or failure, so what are the implications for our work, and now who will do what to improve the situation?
A learning approach to monitoring, such as outlined in Figure 1, entails a process of continuous critical reflection and innovation with the aim to increase the knowledge base and improve performance: to learn from and respond to monitoring results ) (Box 2). Box 2. Monitoring contributes to full coverage and learning how to make investments last: 'Everyone Forever' Everyone Forever, a joint initiative of IRC, Water for People and local governments, has compelled government and the private sector to move from a narrow focus on installing water systems and opening and closing grants/loans to focusing on full coverage and learning how to make these investments last. Reflection on monitoring results by local governments and the private sector has led to innovative ideas that are being tested. Where full coverage was achieved, but not sustained, monitoring has highlighted the problems quickly and clearly and has catalysed exploration of creative innovations to address the challenges identified. This has included experimentation with insurance schemes and new non-donor-supported financial models. Some districts in Bolivia, India, Rwanda, Honduras and Malawi have reached or moved dramatically towards full coverage, which has subsequently spurred demands for replication by neighboring districts and communities, and scaling up. This is happening because the onus for success and the credit for the programme sit with mayors and District Commissioners (DCs). A pattern is emerging: momentum is built towards, or full coverage is actually achieved and mayors/DCs brag. Other mayors decide they want a part of this, as delivering on a big development goal helps them politically. Source: Water For People and IRC (2012).

Box 1. South Africa: peer-monitoring among municipalities
In South Africa, municipalities are undertaking a process of peer review to improve their performance. The District Water Services Managers' Forum is committed to promoting best practice for management of water services in rural areas of South Africa. The national regulation strategy identifies understanding performance as one of the most important activities to create an environment for effective regulation and good performance. Key elements of learning by peer review are: • Sharing of knowledge and experience in a real practical context as an effective learning mechanism. • Lessons are documented and shared for use by others.
• Peer reviews are conducted through in-depth site visits to participating water service authorities and/or water service provider. • 'Critical friends' coming together to learn and advise rather than conduct an inspection.
• A municipal improvement plan is developed based on the review report.
• Lessons and plan are shared at the District Water Services Managers' Forum. Sources: WIN-SA (2006) and personal correspondence from Jean de La Harpe, IRC.

Capacity and capacity development
Capacities are the knowledge, skills, attitudes and values of individuals, and aggregated in organizations, communities and other arrangements that define individual and collective behavior (Alaerts et al., 1991). Sector capacity can be defined as the collective capability of a sector to identify and understand its development issues and goals, to act to address these, to learn from experience, generate and accumulate knowledge for the future.
Many local governments struggle to even carry out basic governance functions such as tendering, procurement, planning and monitoring. This is not limited to the WASH sector but is a much wider problem (Lockwood & Smits, 2011: 125). As illustrated in Box 3, many local authorities, service providers and support agencies need support to enable them to monitor, access sector information, reflect on and make sense of it, and take required action.

Sector performance reviews and Joint Sector Reviews
In this section we will elaborate on national sector reviews as an important mechanism in country processes of planning, budgeting and decision-making. Based on a desk review of sector performance reviews in eight countries in Asia and Africa 1 (IRC, 2013a, unpublished) we explore the link between sector monitoring and learning. Questions to be answered in this section include: Box 3. Uganda: water authorities have weak capacity for monitoring In Uganda, the performance of the water authorities and the private operators is monitored and managed by the Directorate of Water Development (DWD). The minister responsible for Water has a performance contract with the water authorities (invariably the Town Councils and LC III councils), which in turn have management contracts with the private operators. The monitoring and supervisory function by the water authorities is weak, due mainly to capacity constraints in the town councils. While there is an elaborate planning and reporting structure and there are Planning Guidelines issued by DWD, in practice the water authorities rely heavily on the hired private operator to develop a business plan and annual work plans, which they simply endorse with minimal review, if at all. A deliberate action is needed by DWD to strengthen the capacity of local governments with readily available skills or to consider a tripartite agreement to reflect the reality on the ground. Source: GTZ (2009).
• What is the potential of monitoring sector performance for prompting a larger demand for learning, innovation, and adaptation of policies and practices?
• What mechanisms need to be in place to strengthen the link between sector monitoring and better and more structured learning in the sector?
A sector performance review is a structured process of defining performance criteria for the sector, collecting performance data, assessing progress, decision-making on corrective measures and reporting on progress achieved. Sector performance reviews are led by national governments and the results are meant to feed into the planning cycle. At least 40 countries now hold annual or biennial reviews of rural drinking water performance (often combined with urban water and sanitation), and 13 countries are in the process of establishing such mechanisms (WHO, 2012).
Sector performance reviews often go hand in hand with a sector-wide approach, assessing progress and reporting on overall sector performance instead of reporting on project achievements to individual donors (IOB, 2009). Experience in Ethiopia illustrates increasing dialogue, coordination and harmonization as a result of multi-stakeholder dialogue (Box 4).
A Joint Sector Review is an annual event within a sector review process that bring all sector stakeholders together to assess sector performance and to formulate recommendations for the way forward. The case of Uganda shows that they can become a dynamic joint process of monitoring, reflection, learning and planning (Box 5).
Preparations for the JSR include the compilation of a sector performance report, financial reports and additional studies. In Uganda the monitoring process results in a comprehensive annual Sector Performance Measurement Report. Over 100 stakeholders participate in its preparation. All data are analysed, with the information presented in tabular and graphical formats to readily convey the desired message Box 4. Ethiopia: moving towards 'one plan, one budget and one report' Since 2005 there has been a growing discussion among sector stakeholders on the need for greater harmonization and integration. The EU Water Initiative helped to facilitate the first WASH Multi-Stakeholder Forum (MSF) in 2006. This has evolved into an important annual sector event and played a significant role in bringing various sector actors to one discussion platform. The dialogue has been particularly important in developing consensus on the main sector blockages and annual priority actions. A few donors have started coordinating their approach. The World Bank and Department for International Development have fully harmonized their implementation modality while the African Development Bank is adopting a major part of the procedures. Other multilateral and bilateral organizations have been slow to join this effort, because of variations in their aid regulations and procedures set out in their country-specific support strategies. The sector is now moving, albeit slowly, towards a sector-wide approach, and aims to gradually move to 'one plan, one budget and one report'. Source: Government of Ethiopia (2011). and facilitate decision-making. The report is presented and discussed in the annual JSR, where seven to 10 strategic actions are agreed.
A Joint Technical Review (JTR) involving government, donor, non-governmental organizations (NGOs) and private sector representatives is held every 6 months to monitor progress of implementation of the agreed key actions from the JSR (RWSN, 2012). The main output of the sector review process is a sector performance report, and in some countries, an additional shorter report on the JSR event (an aidememoire), including the main recommendations and lessons learnt. In Uganda, reports on the implementation of prioritized actions (so-called undertakings) or additional studies are also produced. Countries such as Malawi and Ethiopia have started to follow Uganda's model for reviewing sector progress.

National sector performance reviews: background and purpose
Sector-wide reviews that are led by national government and involve all major stakeholders are increasingly recognized as key to improving WASH coordination and planning. The UN-Water GLAAS report 2 (2012) signals an increase in periodic review processes that are used for planning in sanitation and drinking water (WHO, 2012). The launch in 2013 of a first national JSR in fragile states such as Liberia and Sierra Leone seems to further confirm the above trend. However, there are further steps needed towards implementation of a comprehensive sector monitoring process on a routine basis.
Sector performance monitoring requires clear leadership and dedicated resources. A special unit, mostly a Monitoring and Evaluation (M&E) unit, is set-up in the responsible line ministry (the case in Uganda) or in the sector coordinating body. The process is coordinated and led by the M&E unit but requires high level commitment and mobilization of a large number of human resources, professionals in the different sector organizations at national and decentralized levels. The M&E unit takes overall responsibility of the process and final compilation of the sector review document and where needed contracts external expertise. Assessment of progress and in-depth reflections takes place in the various working groups.

Box 5. JSR in Uganda
The JSR is an annual 3-day forum for sector performance assessment which meets every year, around October. The JSR brings together 200-400 people, principally from the government and development partners, including ministers, MPs, technical and political leaders from districts, representatives of civil society, development partners and officials from the Ministry of Water and Environment and from other line ministries and private sector service providers. This forum provides information on sector performance and accountability, and promotes transparency. Source: Kahangire (2013).
National sector review processes are considered to be an effective means to improve sector coordination, and as a mechanism for stakeholder participation and joint ownership for sector performance. The value of the JSR as a platform where the different stakeholders come together and can be held accountable for their contribution to achieving improved sector performance is also recognized (IOB, 2009).
However, with the exception of Uganda, very limited information is available about the effectiveness of the sector monitoring process to inform action, or the learning impact of monitoring sector performance. The potential for sector performance reviews to address learning needs in the sector is seldom made explicit. In Uganda the JSR leads to an identification of outputs and results/trends, recommendations and lessons learnt (Thomson et al., 2005;RWSN, 2012).
Similarly, Holvoet and Inberg's analysis of JSRs from the education sectors in Burkina Faso, Mali and Niger conclude that JSRs prioritize accountability over learning needs, focus more on implementation than on its results and largely neglect accountability to users and learning (IOB, 2009).

Main characteristics of sector performance reviews
Most sector review processes are designed to involve a wide range of stakeholders and to be carried out in a cyclic process with a duration of 1 year for each cycle. However, not all countries have the capacity to conclude the full cycle every year. An annual JSR event often takes place in the capital. The way the JSR is set up defines the opportunities for interaction, reflection and effective participation.
The way the JSR is organized and the inputs are prepared varies from country to country: • In Uganda, working groups with representatives of different stakeholders prepare the inputs for the performance report which is compiled under the leadership of the Ministry of Water and Environment. The Joint WASH Sector Review is expected to bring together about 116 National and International Organizations with a total of 240 participants. An array of participants will come from national government agencies and ministries, development superintendents from the 15 political sub-divisions, Civil Society Organizations, WASH Coordinators from the counties, the private sector and local/ international NGOs. Source: Koroma (2013).
It is a challenge to ensure that information from sector monitoring is available in the right formats for the right people. In Uganda, sector performance reports are available on the ministry's website. A popular version of the 2012 report is disseminated through local newspapers. In Ghana, 'WASH Reflections' produced by the Resource Centre Network make the results of the JSR accessible to a wider audience (RCN Ghana, 2011). Most sector performance processes seem to lack a communication strategy with clear messages on the main findings and targeted to specific audiences.

Sector performance review and opportunities for learning
Based on our desk review we support Holvoet and Ingberg's conclusion that many JSRs are principally aimed at producing evidence in terms of achieving sector targets and at improving coordination. In theory JSRs offer a good platform for dialogue and exchange of views between different stakeholders; however, in practice the way they are organized does not always encourage critical reflection on bottlenecks and the main factors influencing sector performance (IOB, 2009) (Box 7).
The main product of JSR is considered to be the sector performance report (or the aide-memoire) of the JSR meeting. In addition, recommendations are formulated for those areas that are reported to lag behind in relation to annual targets. In Uganda, unsatisfactory progress towards sector targets is an important trigger for formulation of so-called 'undertakings', special actions or studies to get to a more profound understanding of underlying factors responsible for insufficient progress.
Working groups, consisting of representatives of various state and non-state actors, are a potential mechanism to strengthen learning in the sector monitoring process (Smits et al., 2012). In Uganda, a few of the working groups have adopted the practice of organizing separate 'learning journeys' to further analyse good practices and innovations (Kahangire, 2013). Action research, where policymakers and implementers engage and collect evidence for what works in communities, provides the working groups with new insights and prompts recommendations for adjusting policies, targets or practices to improve sector performance.
While learning is not an explicit focus of the sector performance monitoring process in Uganda, the process offers various opportunities for joint reflection, analysis and learning. As such it provides a model for linking performance review and learning in the sector. Kahangire (2013) suggests that learning and reflection could be further strengthened in the JSR in Uganda by focusing on rigorous reviews of evidence of performance along the undertakings of the previous year, before drawing emerging issues

Box 7. JSR in Nepal
In Nepal during the first JSR in 2011, participants discussed the issues raised in the thematic working group position papers and the Sector Status Report (May 2011), in both plenary and group sessions. They identified and prioritized three to four key actions that were to be taken by all the relevant stakeholders in the year ahead. Participants further recommended that a JSR be held in 2012 to assess progress towards achieving the agreed key actions. Source: NGO Forum Nepal (2011). and setting new undertakings. Another opportunity exists in the parallel sessions held during the JSR. The sessions can devote more time to discuss and distil ideas within a framework of learning and reflection. Learning and reflection need to be well documented, and principal actors for taking agreed followup measures should be clear (Kahangire, 2013). Figure 2 visualizes an ideal monitoring and learning process in Uganda (Kahangire, 2013), building on existing process and platforms.
In Ghana, sector stakeholders came together to review the second WASH sector performance report in November 2011 during a meeting of the National Level Learning Alliance Platform (NLLAP) (Resource Centre Network Ghana, 2011). The NLLAP is a WASH sector multi-stakeholder platform with the overall goal of improving sector learning and dialogue. By putting the review of the sector performance report in the agenda of the NLLAP a clear emphasis on the learning value of the JSR was made. NLLAP meetings are organized and facilitated by the Ghana WASH Resource Centre Network and are open to all interested parties. The discussions of each NLLAP meeting are summarized and disseminated in the form of a communiqué after the meeting. This experience shows how the sector review process is taking advantage of an existing learning structure in the sector. In Ghana, NGOs play an interesting role in promotion of a learning culture in the sector. The Mole Conference 3 , organized by CONIWAS, is an annual meeting and brings together sector practitioners from NGOs, government, private operators, networks and civil society organizations to dialogue, learn and share knowledge and information on specific themes that affect the sector. The sector review process in Ghana would benefit by involving and including the Mole Conference more structurally in the annual sector review process.
The experiences in Ghana and Uganda are rather exceptional in connecting the sector performance review process and sector learning processes. While in Uganda the sector performance review process creates space and platforms for sector coordination, dialogue and more in-depth assessment and reflection on specific areas of concern, the sector review process in Ghana is embedded in the existing platforms for sector dialogue and learning. However, we may argue that in both cases the potential links between the sector performance review process on one hand, and on the other hand national learning processes, lesson learning and development of context specific sector knowledge, are not yet optimized. There is potential to optimize the opportunities for learning by involving existing multi-stakeholder platforms in the sector monitoring process, especially by building on learning platforms at decentralized levels and by linking with civil society learning initiatives (Box 8).
While not perfect, both Ghana and Uganda provide an interesting reference for other countries looking to improve the learning value of their sector performance monitoring processes. After the first JSR meeting in February this year in Liberia, the challenge is to design a structured process for systematic follow-up on the main findings of the first JSR. The Uganda and Ghana models provide ideas on how to close the monitoring-learning loop, from sector monitoring, to lesson learning, capacity development, adaptive management and policy influencing. It takes time and commitment to achieve a monitoring and learning culture (Danert & Ssozi, 2013).

Embedding sector performance monitoring and sector learning
In Uganda, the annual sector review process is embedded in the existing structure and daily work of all sector actors. The experience in Ghana shows that if that is not the case, it is more difficult to maintain momentum and to produce an annual performance report each year.

Box 8. Civil society organizations produce joint performance report in Ethiopia
In keeping with the principle of mutual accountability, the civil society organizations operating in Ethiopia produced an Annual Joint Report on WASH in 2010. This emanated from a commitment made at a multi-stakeholder forum in 2009 that WASH sector civil society organizations would produce an annual performance report that could be incorporated into the National WASH Report, produced by the National WASH Coordination Office. Source: Government of Ethiopia (2011).
Multi-stakeholder platforms at national and decentralized levels enable stakeholder participation and buyin and can enhance the quality and effectiveness of the monitoring process. The existing sector learning platforms in Ghana (including the Resource Centre Network (RCN), national learning platforms, regional learning platforms and learning events organized by civil society) provide mechanisms for enabling dialogue and critical reflection within the joint monitoring process and enhancing its value as a process for learning. In Uganda the joint monitoring process went hand in hand with the establishment of various multi-stakeholder coordination platforms at national and decentralized levels. Establishing coordination platforms at national level and in the districts will need to be a first step in countries such as Liberia, Sierra Leone and Nepal, in order to institutionalize a joint multi-stakeholder sector performance monitoring process.
Recommendations to strengthen learning in sector monitoring processes are to: • Make learning, reflection and innovation explicit goals of the process.
• Identify a learning agenda, case documentation and facilitation of learning and reflection, which can further boost learning potential.
• Consider practical issues around the design of platform meetings and their facilitation (for instance the number and kind of participants, the agenda for the platform meeting, their timing, location and duration) (Kahangire, 2013).
Establishing a structured process of sector monitoring, learning and adaptive management will require good assessment of institutional and individual capacity, and capacity needs. Capacities can be built up gradually, and as necessary, depending on the level of complexity of the sector and the monitoring framework (Danert & Ssozi, 2013).

Participatory monitoring to strengthen sector performance, accountability and transparency
Over the past 15 years participatory monitoring and social accountability tools have been increasingly used as a means of improving service delivery and governance in WASH. Accountability tools include participatory budgeting, public expenditure tracking, citizen report cards (CRCs), social audits, citizen charters, right to information acts, and community scorecards (CSCs) and a range of indices for benchmarking performance and planning.
The tools rely on engagement by the public to improve accountability from service providers and public authorities. In this paper we focus in particular on two accountability tools (CSCs and CRCs) and one participatory monitoring method (the qualitative information system -QIS) for quantifying qualitative change. The effectiveness of both accountability tools and other participatory methods is significantly enhanced when the information and findings are discussed in multi-stakeholder platforms and linked to formal planning and decision-making processes.

Community scorecards and citizen report cards
CSCs and CRCs are upward accountability tools, used by users of water services to monitor the provider's performance and provide feedback against collectively determined standards and indicators. They often track inputs/expenditures, monitor service quality, evaluate performance against benchmarks across facilities and districts, and generate direct feedback mechanisms between providers and users. Most of the documented experiences with CSCs and CRCs are related to urban water supply (Box 9). The availability and accessibility of information and the transparency of practices play a crucial role in increasing accountability and identifying low-performance areas. Importantly, transparency increases users' trust in the provider, creates a foundation for partnership and helps transform negative perceptions held by providers of poor individuals and neighborhoods (NETWAS, 2007;Nimanya, 2010;WaterAid, 2010).
Both tools survey the services to households and use standardized sampling methods, grading service providers on access, quality, affordability, willingness to pay, staff behavior, efficiency, reliability, adequacy of supplies and overall user satisfaction. When the scoring is carried out periodically, trendanalysis is possible (MWI, 2007).
One of the challenges is how to apply these tools at scale. Information and communications technology (ICT) is increasingly used for large-scale data collection and data management and to improve reliability of responses. 'MajiVoice' ('Water Voice') in Kenya enables water users to submit real-time feedback on service delivery using SMS, an online website, or unstructured supplementary service data (USSD)a service that facilitates two-way data exchange. Citizens can report corruption, submit a complaint, request explanations for their bill, or provide suggestions. Upon submission of feedback/a complaint the water user receives a reference number and can track the progress of the response to the complaint.
Instead of requiring Water Action Groups to manually collect feedback from local communities, water users can now directly send it to the Maji Voice system. Water Action Groups review the complaints, follow up first with the water companies, and if the issue is unresolved, then with the Water Services Boards and the regulator (World Bank, 2012).
Summarizing, CSCs and CRCs: • Aim to improve transparency about the water services and give consumers a tool to hold service providers and service authorities accountable.
• Use random sampling surveys for interviewing households. These have been mainly paper based, but increasingly benefit from ICT, which enables rapid feedback at scale.
Box 9. Social accountability tools in Wobulenzi, Uganda The Wobulenzi pilot of social accountability tools demonstrates their ability to produce significant operational results (improved performance, the introduction of corrective measures) as well as process outcomes (institutional, behavioral and relational changes). According to the evaluation conducted by NETWAS and World Bank Institute, based on surveys in 2008 and 2009, water provision in Wobulenzi has improved following the implementation of social accountability tools. Stakeholder satisfaction has improved significantly, water use has increased and the number of users with difficulties in accessing water as a result of competition for access at water points has decreased. Further, users reported that 'improved communication with service providers had increased the transparency of the costs of certain water services, such as connections to the piped system', and 'service providers began to adjust their practices to improve services in response to public feedback'. Source: WaterAid (2010).
• Use focus group discussions or multi-stakeholder meetings to create a dialogue between users, service provider and service authority for a better understanding of the issues the different stakeholders face and to agree on follow-up action to improve service delivery.

Qualitative information system
Participatory monitoring methods and tools are developed for the effective and efficient planning of new services and for the monitoring of existing ones, at community and district levels. One of these methods is QIS, a flexible system that enables quantification of people's perceptions of qualitative change over time. QIS works by quantifying qualitative indicators with the help of progressive scales called 'ladders'. These indicators can be process indicators, such as participation and inclusiveness, or outcome indicators, such as behavioral change. Each step on the ladder has a short description, called a mini-scenario, which describes the situation that signifies a particular score. Scoring is done jointly with respondents using participatory methods. Each scale ranges from the absence of the particular indicator at the lowest level (score 0) to the optimal mini-scenario at the highest level (score 4). The method allows for comparison across areas, groups and time.
Qualifying quantitative performance at scale in Bangladesh with QIS. The BRAC WASH programme in Bangladesh, implemented by BRAC, a large Bangladeshi NGO, spans almost half of the country (248 of the 502 Upazilas) and targets approximately 50 million people (Sybesma & Ahmed, 2013). In 2012 IRC supported BRAC to design a simplified QIS for monitoring programme outcomes to complement the existing input (e.g. the number of visits) and output (e.g. number and types of toilets built) monitoring systems. QIS enables the quantification of qualitative information about outcomesin terms of sanitation and hygiene behaviors, management and sanitation marketingin a way that allows both programme staff and programme beneficiaries to learn from the results. Household data are poor-specific, toilet use is scored gender-and age-specific. Figure 3 represents one of the scales used.
QIS will help households, village WASH committees and programme staff to measure progress on local services and hygiene behavior and to plan improvements. At programme level, QIS will be used by the Monitoring and Quality Control Unit of the BRAC WASH programme to report outcome progress to BRAC's general management, the Government of Bangladesh and programme funders. A yearly indepth study measures programme outcomes in a representative sample. This will feed into an annual event designed to foster learning and to inform adaptation of programme strategies (Sybesma & Ahmed, 2013). To increase efficiency of data collection and facilitate progress mapping, an app has been developed for smartphones which is used to collect the data and feed into a geo-referenced database (IRC, 2013b).
Summarizing, the QIS: • Monitors qualitative change over time using progressive scales, allowing for comparison across areas (e.g. districts) and stakeholders groups (e.g. service providers) and time.
• Uses random sampling surveys and focus group discussions.
• Monitoring data are discussed and validated with the involved stakeholders. Meetings aim at corrective action where needed.
• ICT applications, such as smartphones, allow for application at scale and rapid use of the data by programme management.
Which mechanisms and capacities are needed to ensure monitoring and learning are linked?
The shift towards decentralization in many countries has meant a shift in monitoring roles and capacities too. District government (or its equivalent), programme staff, the private sector and NGOs at these levels have greater responsibility for managing programmes and for monitoring them, but struggle to fulfil these roles (IRC, 2013c).
There need to be strong incentives for monitoring data and findings from sector evaluation to be used. In most governments, incentives or sanctions are generally focused on policy formulation, spending allocations and the early stages of implementation, rather than on results achieved or on the feedback loop from implementation experiences to new policies, strategies or budgets (World Bank, 2010: 17). A powerful 'champion'a minister or senior officialcan lead the push to institutionalize learningoriented monitoring and the need to devote sufficient resources to it.
Effectively linking monitoring and learning processes also requires a variety of capacities. We do not claim to present an exhaustive list, but building on Le Bay & Loquai (2008); Danert & Ssozi (2013);World Bank (2001) and discussions at UNESCO-IHE's symposium (2013) 4 , the following capacity needs for Monitoring and Evaluation at decentralized levels emerge: 1. Capacity to design and test methods and tools for M&E. 2. Capacity to use M&E data for improvement/decision-making: Fig. 3. Measuring village WASH committee performance (Sybesma & Ahmed, 2013). VWC, village WASH committee. 4 Monitoring for learning and building capacities in the WASH sector, http://www.irc.nl/page/80819. (a) Enhancing statistical literacy and understanding infographics. (b) Developing procedures and systems for exchange of information and data. (c) Ability to collaboratively access, collect, stock and update relevant information. (d) Capacities for self-reflection and collaborative learning. (e) Confidence to communicate monitoring results to different audiences. 3. Capacities of citizens to monitor actions of their water service provider, to voice criticism and to demand accountability.
In short, to make M&E more effective tools for WASH sector improvement requires both building demand for using monitoring results for evidence-based learning, and improving the supply side (mechanisms, tools and capacities for monitoring).

Discussion
Over the past 15 years there has been a growing awareness that the sector cannot improve itself without reliable and up to date information about sector performance and the reasons behind success and failure. This is illustrated by a range of monitoring initiatives at international, regional, national, decentralized and project level 5 . There are encouraging examplesnotably Ugandawhere sector-wide monitoring and planning have been implemented. There are also noteworthy innovations such as QIS, CSCs and the use of ICTs to empower users and increase accountability for service provision. However, there is still limited documentation on using monitoring to plan, adapt, learn and improve or of the water sector monitoring as part of sector capacity building efforts.
Often monitoring becomes synonymous with information management systems. However, the value of extensive data and information is limited if it is not reflected upon and used in decision-making. To ensure that services reach everyone there is a clear need to strengthen continuous collaborative monitoring processes and ensure that these contribute to the capacities of sector players to learn, adapt and innovate.
There is potential to link monitoring, learning and capacity building more systematically. For example, sector monitoring processes and JSR events can be better linked to multi-stakeholder platforms and better supported to enable structured reflection, lesson learning and adaptation of current policies and practices. Until recently, much of the participatory monitoring that seeks to contribute to more in-depth analyses of why certain elements or processes of service delivery are not functioning well, took place at relatively small (project) scale. Digital methods for data collection and storage have the potential to enable such methods to be scaled up. Multi-stakeholder mechanisms can support the process of turning monitoring information and sector experience into actionable lessons for the sector.
Capacities are needed for both monitoring and learning. To make monitoring a more effective tool for sector improvement requires building demand for evidence-based learning, building on monitoring data. At the same time, mechanisms, tools and capacities for monitoring need to be strengthened. A learning and adaptive sector requires mechanisms for sharing, critical reflection and sense-making to support decision-making at national level, and at decentralized levels.
Developing the required capacities for a structured process of sector monitoring, learning and adaptive management will require good assessment of institutional and individual capacity needs. These can then be built up gradually, as necessary, depending on the level of complexity of the sector and the monitoring framework in each country.