Governments can strategically introduce visual thinking as part of monitoring and evaluation to support information management, enhance understanding, ensure retention, ensure effective stakeholder engagement, and most importantly, use evaluative evidence. In this evidence story we share how EvaluVision, a visual thinking technique, increased evaluation use during a workshop hosted by the World Food Programme and the Development Monitoring and Evaluation Office (DMEO) in India.
Michael Quinn Patton in his book “Facilitating Evaluations” observes that there is a high price for poor facilitation and communication of M&E insights such as ignored participant needs, submerged stakeholder voices, and hiding underlying causes that may finally undermine impact of a programme.
Effective facilitation and communication increase the relevance, meaningfulness, credibility, and utility of evaluation insights.
EvaluVision, a combination of Evaluation and Visualization, aims to improve the utility of evaluation insights by organizing technical information into comprehensible graphics. It encourages engagement across all stakeholders and has been designed and tested at workshops in Asia and the Pacific Region.
A typical public programme generates huge volumes of data in a year. Insights are presented as narrative reports which are then summarized based on the needs of the audience.
At the centre of this process in India is the Development Monitoring and Evaluation Office (DMEO), NITI Aayog, which is the nodal M&E institution. Together with DMEO’s technical support, the central ministries and departments in India generate a large volume of evidence periodically – annual monitoring of key indicators for programmes every year (160 programmes in 2020-21 and 139 in 2021-22), followed by an independent evaluation every two-to-three years.
As a result, non-performing programmes are either discontinued, merged with others, or revised in consultation with key stakeholders. Use of visual-thinking tools and techniques can benefit the entire national M&E value chain, from planning and interpretation of results to its dissemination and uptake.
Planning is at the core of all M&E exercises and involves multiple stakeholders with varying capacity of M&E.
Visual thinking techniques, such as EvaluVision, combine simple yet compelling graphics and group facilitation exercises to help articulate the perspectives of each stakeholder more effectively. An immediate benefit of using EvaluVision is the development of comprehensive theory of change(s) built consultatively with all stakeholders/ministries, clearly describing the interlinkages, underlying assumptions, and timeline of change for a programme.
For instance, India implements one of the world’s largest public food security and nutrition schemes involving multiple ministries in charge of production, supply, and final distribution of food commodities to the most vulnerable population. Using today’s visual-thinking tools, a theory-of-change linking the food safety net with the final goal of Zero Hunger by 2030 can emerge, that describes the pathway of change envisioned by each implementing ministry. As a result, a ministry can ramp-up its M&E activities to generate critical evidence on challenging issues that may have gone undetected and clearly define their timeline of change and efforts required from a medium to long-term perspective.
As ministries move from solving simple and complicated to complex problems, the verticalized intervention or linear thinking would deliver limited impact and therefore a system-based understanding becomes more important. EvaluVision is best suited to facilitate better understanding of complex problems along with inter-relations between various parts for systemic approaches. Moreover, the complex problems based on systemic approaches require a flatter and matrix organization to address them, something which can be explained better through visual thinking, facilitation, and communication techniques.
Interpretation of results is currently a fragmented process, with each stakeholder zooming-in on parts of evidence rather than the larger picture.
For evaluators, the need of the hour is to include visual thinking techniques while preparing a data analysis plan. In usual practice, data analysis is given more emphasis than dissemination. While narrative reports and charts are comprehensive, in practice stakeholders have limited time to digest this voluminous information and arrive at a larger picture of the sector.
This leads to prioritization of some findings above others while missing out on the larger context of the scheme/programme. The space for engaging all stakeholders also gets constrained, and some stakeholders may feel left out. A “diagnostic map” can be developed using EvaluVision tools, that summarizes the performance of contributing activities and their key successes and bottlenecks. This map can act as a catalyst in reducing the verticalized approach of service delivery, increasing the emphasis on system-level understanding and helping to build consensus on the larger situation.
Insights from visual thinking products “stick” longer compared to narrative reports and stakeholders develop a consensus on what needs to be done going forward.
There are a number of reasons why visual communication of M&E findings is especially important in an administrative setup like India:
Products from EvaluVision can help retain institutional knowledge around the larger picture in a constantly changing senior leadership. In practice, the “diagnostic maps” will be very useful for new officers joining a ministry/department to get a quick self-explanatory overview of their implementation landscape and challenges that need tackling.
Today, users can choose from a wide range of collaboration tools and facilitation techniques to help implement visual thinking in evaluations, such as Mural, Miro, Google Jamboard, and Stormboard. WFP has developed a comprehensive guidance e-book on EvaluVision, which is a step-by-step introduction to implementing these techniques. This e-book comes with short introduction video.
To answer this, the World Food Programme (WFP) held a virtual workshop with the DMEO, NITI Aayog. Objectives of the workshop were to introduce the concepts of EvaluVision to DMEO, as well as understand the challenges and opportunities to implement EvaluVision in government settings.
The “EvaluVision Triangle” was used to visualize the preferred future where visual-thinking, facilitation and communication is used in the M&E process to improve quality and use of insights, and the factors that prevent in implementing such a methodology. Participants were asked to write their thoughts in three separate boxes titled – pull from the future, push from the present, and weight of the past. Mural platform was used for this exercise.
The pull from the future include better interpretation and utilization of M&E results, bridging the gap between evaluators and programme implementers, and the potential to co-create effective programmes with various stakeholders. The push from the present comprise of the need for quicker results by decision makers, need for innovative techniques to engage various stakeholders, and availability of new and online visual-thinking and facilitation tools and platforms.
The critical challenge or weight of the past that may slow down the use of visual thinking techniques is the fast-paced nature of policy cycle in India that leaves little time and space for innovative approaches. There is a gap in capacities among evaluators in terms of using visual-thinking techniques (“fear of getting it wrong”), and less space for experimenting with new dissemination tools due to the tight timelines. Stakeholder’s exposure to visual-thinking and group facilitation also remains limited.
There are three opportunities available for institutionalizing EvaluVision approaches within the government M&E system in India. First, as the conversation around use of visual-thinking tools and techniques gain prominence in government evaluations, the next challenge is to start embedding it into the evaluation cycle. Present opportunities include the joint studies planned by WFP and DMEO on food security and nutrition issues in India. This can then be expanded to other evaluation studies, which are proposed to be undertaken by the DMEO.
Second, the ongoing capacity building initiatives for officials at DMEO and other government stakeholders along with introduction of short-term and long-term courses in Universities. In both these cases, visual thinking will have to be embedded in the content, format, and communication aspects of M&E capacity building. The EvaluVision techniques will have to be taught as part of the M&E training or course. Moreover, Massive Open Online Courses (MOOCs) on M&E for explaining the theory and concept can be created using EvaluVision tools.
Third, in a federal set-up like India, where states have their independent M&E systems, an effort will have to be made to reach across to states like Karnataka and Tamil Nadu with more matured M&E systems, to start introducing EvaluVision concepts.
Visual thinking and visual communication were innate to the human civilization before the language was created. Hence, it is only a matter of time and a consistent effort that visual thinking and communication will become a reality in evaluation.