“There is a great satisfaction in building good tools for other people to use“

- Freeman Dyson, Disturbing the Universe

---

Current clients (2011)

---

AusAID Vietnam

• Developing an M&E Framework for phase 4 of the AID funded Human Rights Technical Cooperation Project, implemented with the assistance of the Australian Human Rights Commission, in partnership with several agencies of the Government of Vietnam and associated bodies. This follows on from our 2010 review of the first three phases of this project, described below.  Because of my travel constraints this was a desk based piece of work, so the scale of consultation with other stakeholders was very limited. However, my further involvment via M&E capacity building inputs with project partners in 2012 may help redress this weakness.

Evaluation Department, DFID, in UK

• Working with a team led by Elliot Stern with the task of “Developing a broader range of rigorous designs and methods for impact evaluations” The Tors begin “There is an increasing demand to conduct impact evaluations in the field of development programmes; from DFID’s perspective we view these as studies which attempt to create a counterfactual and enable attribution of results to programmes. Traditionally , the randomised controlled trial has been regarded as the ‘gold standard’ for impact evaluations. These are possible in some development contexts (for example, in assessing the effectiveness of new drugs or vaccines, of health treatment methods, or the efficacy of new drought resistant seeds), but there is a wide range of development activities where this approach is inappropriate or just not viable”….”The present study is intended to identify these [other] approaches and explain how they can be rigorously quality assured”

World Vision, in UK and Canada

• Working with Tracey Delaney to review World Vision’s experience with the use of the Most Significant Change (MSC) technique. Somewhat to my surprise, many different World Vision offices and programmes have used MSC in a variety of ways, for monitoring purposes, as part of evaluations, and to generate impact stories for sponsors. The first challenge we have is to identify and make contact with all the people who have been associated with various uses of MSC. We are then gathering data via a combination of online surveys (short and long), skype interviews, country visits and email exchanges. Another challenge is how to open up the process to all participants, possibly via a dedicated wordpress.com site, yet complete the work with the modest amount of time that has been scheduled.

Unnamed client

• Carried out reviews of draft Terms of Reference for evaluations of work to be undertaken by their country offices. The aim was to help build the quality of these evaluations by allowing country offices to opt into this type of support mechanism. The feedback received from the participating country offices so far has been very positive. The challenge: In the process of providing advice one is inevitably developing a model of a desired approach to evaluations, albeit one based on existing guidance materials. That model needs some discussion. Ideally there would be annual meetings with other people in my role, plus the client, to discuss and cross-check our emerging models of what constitutes good practice.

---

Other current activities

Monitoring and Evaluation News

• Managing the website and moderating the associated email lists (2500+ members)

Journals

• Peer review of papers for publication in American Evaluation Journal, Evaluation and Program Planning and Development in Practice
• Reader of chapters in forthcoming book on the Evaluation of Research on and in Violently Divided Societies, being produced by IRDC and INCORE/ International Conflict Research Institute

Workshops

• Presentation on Social Frameworks, at a workshop in Bilbao organised by ECODE – CRITICAL STUDY OF LFA IN THE BASQUE COUNTRY
• Review and comment on ODI paper on the analysis of networks, via a webinar

And…

• Participation in the CES Learning and Innovation Prize Judging Panel

---

Recent clients (2010)

---

AusAID Vietnam

• Helped review the functioning of the AusAID funded Human Rights Technical Cooperation Project, implemented with the assistance of the Australian Human Rights Commission, in partnership with several agencies of the Government of Vietnam and associated bodies. This was done in association with a human rights specialist from the Danish Institute for Human Rights. There were two main challenges: the sensitivity of human rights as a topic of discussion with the Government of Vietnam and the limited access to information about a related activity, the bilateral Human Rights Dialogue between Australia and Vietnam. PS: I am pleased to say the work is now completed and some usefull recommendations were generated, which now appear to be on their way to being implemented.

DOEN Foundation, Netherlands

• Reviewed and commented on an evaluation of their program of support for cultural activities in Africa and Latin America. Then provided input into brainstorming session on how the lessons from this successful evaluation could be used to design a monitoring system for other parts of their international program. The main challenges here: How to monitor and evaluate a wide diversity of arts and cultural activities, taking place across many countries.

IRC International Water and Sanitation Centre, in Netherlands

• Assisting staff associated with the WASHCost project to develop skills in network visualisation and analysis that will enable them, and others in IRC, to analyse the performance of Learning Alliances at national and international levels. The development of Learning Alliances is seen as a major means by which the project will achieve its objectives i.e.  “To achieve measurable improvements in water, sanitation and hygiene (WASH) service delivery by improving access to accurate knowledge on disaggregated costs and by embedding improved decision-making processes in lead organisations in the WASH sector at intermediate, national and international levels” At this stage we are using Skype teleconferences combined with a wiki, Google Docs, and MS SharedView. Face to face meetings and training will take place later. The main challenges here: To obtain sufficient and relevant data from each country partner, and the lack of a theory of change of how the learning alliances are supposed to work. PS: Overall, I dont think we have been very successful.

Bill and Melinda Gates Foundation, in India

• Helping to review and revise the Avahan project’s Theory of Change; participating as a member of the project’s Community Mobilization and Structural Interventions Advisory Group; continuing to assist Praxis India in their measurement of community mobilisation activities within Avahan.

AusAID, in Indonesia

• Leading the Independent Monitoring and Evaluation Team, conducting annual reviews of four maternal and neonatal health programs in eastern Indonesia. A good opportunity to learn about what works and does not work when attempting to involve multiple donors and implementing agencies in annual reviews of their projects (in the spirit of the Paris Declaration). This work started in 2006 and will end in 2010.

---

Other recent activities

Journals

• Peer review of papers for publication in Evaluation, American Evaluation Journal, Journal of Development Effectiveness

Conferences

• Designed and delivered a half day workshop on Network Visualisation and Analysis: Tools for Evaluation at the Prague 2010 European Evaluation Society Conference

Workshops

• Presentation in the Evaluation of Storytelling Workshop, Derry, organised by the Irish Peace Centres (IPC) and the International Conflict Research Institute.

Supporting individual consultants

• I have started to provide online technical support to other consultants (in Sweden, Netherlands, New Zealand, UK) with whom I have had some previous working relationship. At present I use a combination of Skype and MS Shared View to communicate by voice, text chat and shared computer screens. Both programs can be downloaded free.

---