“There is a great satisfaction in building good tools for other people to use“
- Freeman Dyson, Disturbing the Universe
Current clients (2013)
Overseas Development Institute (ODI), UK
- Complex systems tools: a PRF-funded research project – Working with Ben Ramalingam on exploring the use of network models for conceptualising and measuring change in DFID funded women and girls empowerment programmes,
AusAID Vietnam
- Building M&E capacity of Gov of Vietnam partners in phase 4 of the AID funded Human Rights Technical Cooperation Project, implemented with the assistance of the Australian Human Rights Commission. This follows on from my earlier involvement in review and design stages of phase 3 and 4 respectively, described below 2010 review of the first three phases of this project, described below.
Evaluation Department, DFID, in UK
- Synthesis of literature on evaluability assessments. Purpose:To produce a short practical note that summarises the literature on evaluability assessments, and highlights the main issues for consideration in commissioning an evaluability assessment
Recent clients (2012)
AusAID Vietnam
- Building M&E capacity of Gov of Vietnam partners in phase 4 of the AID funded Human Rights Technical Cooperation Project, implemented with the assistance of the Australian Human Rights Commission. This follows on from my earlier involvement in review and design stages of phase 3 and 4 respectively, described below 2010 review of the first three phases of this project, described below.
Evaluation Department, DFID, in UK
- Synthesis of literature on evaluability assessments. Purpose:To produce a short practical note that summarises the literature on evaluability assessments, and highlights the main issues for consideration in commissioning an evaluability assessment
Evaluation Department, DFID, in UK
- Lead adviser on the design and implementation of an evaluablity assessment for evaluations of DFID investments in ‘Empowerment and Accountability’ and DFID’s Strategic Vision for Girls and Women”. DFID has undertaken to evaluate the impact of and approach to implementation of two key policy areas, both agreed in 2011. Developing and implementing both of these macro-evaluations will be a challenging process, with multiple components, stakeholders and deliverables. DFID has opted to undertake the preparation and design work in a number of stages, to allow for reflection and consideration of different options.
Evaluation Department, DFID, in UK
- Core team member of an retrospective evaluation of the impact of DFID rural livelihoods investments in India. There are two principal objectives to this study: (i)To establish the impact(s), positive and negative, of DFID’s 20 year £250 million Rural Livelihoods investments in India, (ii)To establish (and provide evidence) to what degree development interventions can be evaluated rigorously ex-post.
Overseas Development Institute (ODI), UK
- Member of Advisory Group for the Policy Influence Monitoring Project – which will seek to monitor, evaluate and build the capacity of 3ie grantees as they seek to ensure that results of their impact evaluations are used by policy makers
Recent clients (2011)
AusAID Vietnam
- Developing an M&E Framework for phase 4 of the AID funded Human Rights Technical Cooperation Project, implemented with the assistance of the Australian Human Rights Commission, in partnership with several agencies of the Government of Vietnam and associated bodies. This follows on from our 2010 review of the first three phases of this project, described below. Because of my travel constraints this was a desk based piece of work, so the scale of consultation with other stakeholders was very limited. However, my further involvment via M&E capacity building inputs with project partners in 2012 may help redress this weakness.
Evaluation Department, DFID, in UK
- Working with a team led by Elliot Stern with the task of “Developing a broader range of rigorous designs and methods for impact evaluations” The Tors begin “There is an increasing demand to conduct impact evaluations in the field of development programmes; from DFID’s perspective we view these as studies which attempt to create a counterfactual and enable attribution of results to programmes. Traditionally , the randomised controlled trial has been regarded as the ‘gold standard’ for impact evaluations. These are possible in some development contexts (for example, in assessing the effectiveness of new drugs or vaccines, of health treatment methods, or the efficacy of new drought resistant seeds), but there is a wide range of development activities where this approach is inappropriate or just not viable”….”The present study is intended to identify these [other] approaches and explain how they can be rigorously quality assured”
World Vision, in UK and Canada
- Working with Tracey Delaney to review World Vision’s experience with the use of the Most Significant Change (MSC) technique. Somewhat to my surprise, many different World Vision offices and programmes have used MSC in a variety of ways, for monitoring purposes, as part of evaluations, and to generate impact stories for sponsors. The first challenge we have is to identify and make contact with all the people who have been associated with various uses of MSC. We are then gathering data via a combination of online surveys (short and long), skype interviews, country visits and email exchanges. Another challenge is how to open up the process to all participants, possibly via a dedicated wordpress.com site, yet complete the work with the modest amount of time that has been scheduled.
Unnamed client
- Carried out reviews of draft Terms of Reference for evaluations of work to be undertaken by their country offices. The aim was to help build the quality of these evaluations by allowing country offices to opt into this type of support mechanism. The feedback received from the participating country offices so far has been very positive. The challenge: In the process of providing advice one is inevitably developing a model of a desired approach to evaluations, albeit one based on existing guidance materials. That model needs some discussion. Ideally there would be annual meetings with other people in my role, plus the client, to discuss and cross-check our emerging models of what constitutes good practice.
Other current activities
Monitoring and Evaluation News
- Managing the website and moderating the associated email lists (2500+ members)
Journals
- Peer review of papers for publication in American Evaluation Journal, Evaluation and Program Planning and Development in Practice
19 August, 2009 at 1:16 am
Dear Rick
Found your site while grappling with an M&E function I need to fulfill of a child survival project that is based on communication for change… but there are so many other factors that influence the outcome – this is in rural UP and urban Maharashtra.
I will continue to come back to this website and your blog for inspiration
- i am an Indian, independent consultant with a doctorate in public health from Hopkins in the mid-1990s
20 December, 2009 at 5:57 pm
http://www.mefeedia.com/watch/24063573
You might like this connexion between aid monitoring and sensmaking tools. An idea that emerged at the World Water Forum with akvo.org after a UN monitoring workshop.
Best Regards
23 July, 2010 at 5:12 pm
Are there any examples of websites where this has been applied, so we can see what it looks like – click on the tags, see how the ratings work?
7 March, 2011 at 11:43 pm
Given your comment on http://blogs.dfid.gov.uk/2011/03/what-should-we-make-of-the-new-uk-aid/
, I hope the ‘unnamed client’ above is a large British national agency!
13 September, 2012 at 10:13 pm
Good to see some of the larger players are being exposed to evaluation techniques such as the MSC. Well done!
14 March, 2013 at 2:23 pm
Hi Rick,
Just to let you know.
After training by Theo Nabben, the MSC method is now (March 2013) in the early stages of being implemented in the EDF funded Rural Infrastructure Development Programme, which is working in 17 districts in Malawi.