What we’re reading in September: Scaling, Privacy and Developmental Evaluation

Search on EES

Evaluation for impact at scale – a need for interventions along the pathway


Much media attention has been paid to the failure of the international community to live up to its grandiose commitments, from the 2030 Agenda for Sustainable Development to the Paris Climate Agreement. In most cases, strong initial actions have foundered upon attempts to consistently scale.  As this timely blog post acknowledges, scaling has featured too infrequently in evaluation frameworks despite its central importance to the most pressing issues of the age.  Rules of scaling are an evolving area of scientific inquiry (see Geoffrey West’s Scale) but, as Johannes Linn argues here, it is imperative that the particular challenges of scale are incorporated into monitoring and evaluation.  As he makes clear, the project-based nature of evaluation is ill-suited to measuring scaling, which is usually over a longer time frame than evaluators are commissioned to review. In response, Mr Linn proposes a “programmatic” approach, viewing projects in sequence across a “scaling pathway”. There are also some useful synergies with systems thinking, where an appreciation of systemic change can better conceptualise scale goals.

Data Privacy and Confidentiality


Privacy considerations have become an indispensable part of any evaluation, and a serious pitfall for any underprepared evaluator.  This piece acts as a valuable gateway to an often intimidating aspect of professional M&E, with practical advice and a clutch of important links.  For a subject noted for opaque legal guidelines, the blog is a readable introduction to the most pressing issues, such as informed consent in action, sensitive personal information, safe storage and monitoring.

Ten Years of USAID Developmental Evaluation https://usaidlearninglab.org/lab-notes/new-study-highlights-10-years-developmental-evaluation-usaid 

Evaluation, with its project focus, can be guilty of a certain short-sightedness.  As an antidote to that, such longer-term studies as this 10-year review of USAID’s Developmental Evaluation are of great value.  The review is also an interesting deep-dive into Developmental Evaluation itself, as a novel approach to utilisation-focussed evaluation and its rapid adoption at USAID (especially since 2016).  Although a highly technical and jargon-filled document, it could prove useful to those individuals and organisations looking to adopt Developmental Evaluation but wary of its cost and overlap with existing efforts and approaches.

Reflections from BetterEvaluation’s outgoing CEO, Patricia Rogers


 As its outgoing CEO, Patricia Rogers uses this blog to explain the origins of the online resource, which so many evaluators have come to rely on.  She also makes clear that the project was made possible by the consistent willingness of evaluation professionals to share their experience.  For a sector in which knowledge sharing between organisations remains underdeveloped relative to the intensity of work done internally, this is a reassuring point.  Patricia gives an extended list of acknowledgments, which include EES conferences. It is obvious from her account that curated meetings and collaborative spaces still provide an essential service to the evaluation community and information diffusion within it.

International Labor Organization, i-eval Flash News https://www.ilo.org/wcmsp5/groups/public/—ed_mas/—eval/documents/publication/wcms_819720.pdf 

i-eval flash is the ILO Evaluation office’s thrice-annual newsletter on evaluation at the organisation and is worth subscribing to.  In this edition, the authors describe a recent synthesis review of the ILO’s institutional response to the pandemic.  The study, one of several on ILO’s performance during the global health crisis, could be very valuable as a model of evaluating pandemic response in the context of an organisation’s underlying objectives.  The newsletter also includes summaries of other evaluative activities including new guidance on capacity development, a review of modality for clustered evaluations and impact evaluation quality assessment.  The ILO is a global organisation with evaluation activities taking place across the world; the i-eval flash also provides a useful overview of its knowledge sharing, workshops and collaborative work.