Category Archives: monitoring and complexity

Adjusting a Theory of Change midway

Leave a reply

I got a very good feedback on my blog post last week on complexity informed Theories of Change, it was shared widely on Twitter. But I also got some questions. One person pointed out the fact that the method I described in that post was mainly focusing on new programmes that are developing their first Theory of Change. But what about programmes that are in the middle of implementation? Programmes where the programme team sense that things are not going the way they are supposed to according to their Theory of Change, Logframe or project plan. Should the managers of such programmes just stop operations and go back to the drawing board to develop a complexity aware Theory of Change? This is in most cases not possible, unless things are really going badly. How can these programmes incorporate some of the ideas of complexity informed Theories of Change?
Continue reading →

Complexity informed Theory of Change

2 Replies

results chainWe know that current development challenges are complex. But not all elements of a development programme are necessarily complex. How does a theory of change look like that shows us which aspects of a programme are complex and which aren’t? How does this help managers to develop appropriate strategies for interventions and focus their attention?

I have been thinking quite a bit about monitoring and how to find a monitoring framework that works in programmes that are facing the complexities of the real world (and I blogged about it before here and here). More and more monitoring guides speak about the complexities programme managers and staff face ‘out there’ and some guides even venture as far as to say that change in the real world is not linear or predictable. The new BEAM Monitoring guidance, which I co-authored, for example, states that ‘a market systems programme is unlikely to achieve its goals in a simple, linear way. It may be difficult to fully understand (at least in advance) how causes and effects will work at a system-wide level. There may therefore be significant uncertainties about how the overall market system may be re-oriented to serve poor people better.’ Continue reading →

Some principles for measuring change in market systems

Leave a reply

CGAPThis blog post was originally published on the Website of CGAP as part of a series of blog posts on measuring change in market systems development under the title “New Funding Approaches Call for a New Way of Measuring Impact.” CGAP (the Consultative Group to Assist the Poor) is a global partnership of 34 leading organizations that seek to advance financial inclusion.

The focus of financial sector development is shifting. Development organizations funding financial inclusion now operate with a vision of sustainability, resilience and impact at scale, and their goals stretch beyond building individual institutions. Now, they aim to improve the whole ecosystem for financial services, taking a facilitative rather than a direct intervention approach. Continue reading →

Monitoring in complex systems: from spotlight and indicators to sensor networks and landscapes

4 Replies

I have been thinking a lot about how a monitoring framework could look like that takes into account the quirks of a complex system. One of the central things, I believe, is that we need to find alternatives to measuring only where we believe that change will happen towards an approach to gather data more broadly.
Continue reading →

Monitoring and Measuring Change in Systems: case study out now

Leave a reply

Systemic M&E Case StudyThe SEEP Network published a new paper, which I have co-authored together with Lucho Osorio and Margie Brand. The publication is part of SEEP’s Systemic M&E Initiative.

This paper evaluates the validity and usefulness of seven principles for appropriate design and management of systemic M&E frameworks. The principles were defined in an earlier publication (see here). This new paper now puts the principles to the test in the context of the Market Assistance Program (MAP) in Kenya and its institutional host, the Kenya Markets Trust (KMT).

You can download the publication from the SEEP Website.

Exploring narrative sensemaking

Leave a reply

I haven’t been posting for a while. The reason is that our first daughter was born in August and we are still overwhelmed with having a new person in our household. My work has been cut down to the minimum so we can cater to and at the same time hugely enjoy the new person in our lives.

Nevertheless, I have been doing some work. An interesting new avenue I am exploring is that of narrative sensemaking. Narrative inquiry has a long history and there are various branches to it. The branch I am exploring is based on the approach by Dave Snowden and his company Cognitive Edge, which attempts to collect metadata together with stories that can be analyzed statistically. This effectively adds a quantitative component to the otherwise purely qualitative nature of narrative inquiry.

As a first pilot we have added a narrative study to a larger thematic study on Regional Economic Development (RED). This thematic study is implemented by a consortium consisting of Mesopartner and SISTME for the Multilateral Investment Fund (MIF) of the Inter-American Development Bank. The goal of the narrative part of the study is to find factors that promote or hinder Local Economic Development initiatives to reach scale – either through effecting changes on policies or through a copying effect by other regions and actors.

Currently, we are collecting narratives from LED practitioners in Latin America. But we are also adding experiences from practitioners all around the world to get a richer picture and be able to compare the importance of the factors. We are using SurveyGizmo to collect the narratives. Although it does not allow for all the tricks as a specifically developed software like Cognitive Edge’s SenseMaker, we see it as a low-cost alternative to test the approach. We will know more about the suitability of the tool when we are done with the study. In any case we are eager to also use the more powerful SenseMaker Suite in upcoming projects and compare the functionality.

If you have made experiences in local and regional economic development that you would like to share, please fill out the questionnaire and share your story. You will have the chance to win a book voucher worth 75 USD. Here is the link to the three versions of the questionnaire we have: English, Spanish, Portuguese. 

Monitoring and Results Measurement: ideas for a new conceptual framework

Leave a reply

In this post I want to share with you an idea of a new conceptual framework for monitoring and results measurement (MRM) in market system development projects. To manage your expectations, I will not present a finished new framework, but a model I have been pondering with for a while. The model is still in an early state but it would be great to harness your feedback to further improve on it. Indeed, what is presented here is based on everything I have learned in recent years from a large number of practitioners that contributed to the discussion on Systemic M&E, my work in the field, but particularly also from the guest contributions here on my blog by Aly Miehlbradt and Daniel Ticehurst and the intense discussions on MaFI. The ideas are strongly based on the learning from the Systemic M&E Initiative and also apply the seven principles of Systemic M&E, although I am not doing this explicitly. Continue reading →

Results Measurement and the DCED Standard: a commitment to move forward

2 Replies

We now need to start a constructive discussion on how a truly systemic Monitoring and Results Measurement (MRM) framework could look like (as Evaluation does not play a big role in the current discussions, I am adopting the expression MRM and avoid the M&E). In this post, I will take up the discussions on MRM and the DCED Standard for Results Measurement from the two guest posts by Aly Miehlbradt and Daniel Ticehurst and will add from a discussion that runs in parallel on the very active forum of the Market Facilitation Initiative (MaFI). I will also add my own perspective suggesting that we need to find a new conceptual model to define causality in complex market system. Based on that, in my next post, I will try to outline a possible new conceptual model for MRM. Continue reading →

Guest Post: Daniel Ticehurst with a critical reply on the DCED Standard

7 Replies

Daniel TicehurstAfter submitting a long comment as a reply to Aly Miehlbradt’s post, I could win Daniel Ticehurst to instead write another guest post. Daniel’s perspective on the DCED Standard nicely contrasts with the one put forward by Aly and I invite you all to contribute with your own experiences to the discussion. This was not originally planned as a debate with multiple guest posts, but we all adapt to changing circumstances, right?

Dear Marcus and Aly, many thanks for the interesting blog posts on monitoring and results measurement, the DCED standard and what it says relating to the recent Synthesis on Monitoring and Measuring changes in market systems.

Continue reading →

Guest Post: Aly Miehlbradt on the DCED Standard and Systemic M&E

1 Reply

Aly MiehlbradtThis is a guest post by Aly Miehlbradt. Aly is sharing her thoughts and experiences on monitoring and results measurement in market systems development projects. She highlights the Donor Committee for Enterprise Development (DCED) Standard for Results Measurement and its inception as a bottom-up process and draws parallels between the Standard, her own experiences, and the recently published Synthesis Paper of the Systemic M&E Initiative.

In one of Marcus’s recent blog posts, he cites the SEEP Value Initiative paper, “Monitoring and Results Measurement in Value Chain Development: 10 Lessons from Experience” (download the paper here), as a good example of a bottom-up perspective that focuses on making results measurement more meaningful for programme managers and staff. Indeed the SEEP Value Initiative was a great learning experience, and is just one example of significant and on-going work among practitioners and donors aimed at improving monitoring and results measurement (MRM) to make it more useful and meaningful. The DCED Results Measurement Standard draws on and embodies much of this work and, also, promotes it. In fact, the lessons in MRM that emerged from the SEEP Value initiative came from applying the principles in the DCED Results Measurement Standard.

Continue reading →