|
|
Archive of posts tagged Monitoring
Hello developers,
It’s done! Thanks to six 1%PROJECTS in Nairobi we’ve been able to test four M&E 2.0 tools: a smartphone application, participatory video, texting and blogging. In a pilot of just four days, between June 25th and June 28th, twelve highly committed participants have provided us with valuable insights into the (possible) value of M&E 2.0 and the challenges that still lie ahead.
Inspired by the thought that ‘change is good’, we’d hereby like to share our four most significant insights and changes of opinion based upon this pilot:
1. Using M&E 2.0 tools encourages enthusiasm for M&E in general.
Based upon the fact that a lot of our colleagues tend to see M&E as dull, boring and complicated, we were surprised to meet a group of motivated staff members, all eager to learn more about M&E. And the use of 2.0-oriented tools definitely increased their enthusiasm. In itself, being able to work with smartphones and flip-cams stimulated participants to carry out M&E, it actually became fun!
2. M&E 2.0 improves the appreciation and perception of organizations.
The pilot-organizations received a lot of positive feedback from actors around them, simply because of using our M&E 2.0 methodologies. In the case of Amani Kibera for instance, several community members were impressed by the fact that the organization was able to carry out a real sms campaign! Other CBO’s working in the same area approached staff members of the organization to find out how they had managed to organize this. Making use of these tools is regarded as something reserved to big NGO’s and corporations, not as a playing field for small CBO’s.
3. Availability of required M&E 2.0 technologies is still limited.
For successful implementation of the piloted M&E 2.0 tools in the long run the presence of technological hardware like smartphones, flipcams and laptops is required. This is a challenge for all of the involved organizations. The phones required for the use of texting as an M&E tool are readily available and widespread, However, this tool unfortunately has high percentages of errors and drop-outs in the process of data collection.
4. A lot of work remains to be done.
To enable organizations to keep on implementing M&E 2.0 a lot of work remains to be done increasing the usability and scalability of 2.0-tools. The front-end, but mainly the back-end of tools like a smartphone application and a texting campaign are still highly inaccessible for tech-dummies. It requires too much expertise and needs to be simplified. This will encourage project owners to continue using M&E 2.0.
As highlighted above he pilot in Nairobi has provided us with some first answers to the questions ‘Which tools (mobile, video, photo) can be used best for data collection and how?’ and ‘How can we motivate and engage project owners to use this tool?’. Now we need more input regarding the question ‘How can the input generated through these tools be visualized (e.g. a dashboard, through mapping) and which software is needed to do this?’.
Therefore, at this point we challenge you: programmers, communicators and designers, to come up with relevant solutions to this puzzle. Feel free to use our findings, analyze our raw data or ask for our opinion. We remain committed to developing M&E 2.0. At your service! 
Where would 1%CLUB be without co-creation session? That’s like cappuccino without milk, or like Lois without Clark. Therefore this April 23rd a group of Monitoring & Evaluation (M&E) innovators got together in Amsterdam to co-create the future of M&E 2.0 for development cooperation. Based upon the outline presented in our previous blog representatives of Akvo, Butterfly Works, Ventanaz, Frog Design, Text to Change, and Oxfam Novib all gathered to ‘develop the ideal M&E 2.0 tool that follows all M&E 2.0 principles’. The upcoming pilot in Kenya provides a short-term opportunity to test the outcomes of this co-creation in practice.
As Joost van der Made of Frog Design puts it in the opening session of the co-creation ‘need is the mother of all inventions’. Therefore a more in-depth dialogue about the value of M&E 2.0 is started. What’s the real need? What’s the question behind the question? A brainstorm provides the insight that one specific need does not exist, but that several target groups each have their own needs and expectations from M&E 2.0: project owners, local communities, 1%members, funders, other project owners, scientists, etc. The current challenge is to invent a 2.0 tool that benefits project owners and local communities, and also has potential value for the other target groups.
A number of ideas is brought forward to respond to this challenge. A gallery of short video clips in a catchy format can for instance transfer basic info about projects and project progress to several target groups. An Open MSC process or Open Knowledge platform can build upon existing knowledge and methodology, and increase openness, crowd participation and real time communication. Participants receive incentives for uploading new stories, videos and photos. An Open data tool can connect data of 1%Project owners to other existing data. This enables project owners to learn and to put their project in a bigger picture, with ‘Hans Rosling’-like presentations and countrywide benchmarks. Using an online Dashboard project owners can moreover reflect their ambitions and planning (‘begin with the end in mind’). This will make it easier for funders/investors to compare projects and make a contribution. In addition to this dashboard Storytelling can provide a way for realtime communication using videos, geo tag photos, blogs, sms and an apps on telephones. Project are able to share updates and news quick and easy through a ‘one click update’-system.
But, as the famous T.S. Eliot already stated ‘there falls a shadow between conception and creation’: execution! Out of all the ideas brought forward, the participants consider the following characteristics of a tool most valuable an relevant to be executed in the pilot:
− An open, 2.0 translation of MSC (Most Significant Change),
− that uses different tools for data collection: mobile, texting, video, photo,
− that connects data gathered in a project and ‘open data’,
− based upon a format of questions that combines both quantitative and qualitative  data and is easy to use, and
− tested on different types of projects: (1) projects that are still designing their activities, (2) projects that have just started and (3) project that have been implemented already.
To further develop these elements into a tool three critical questions remain for the last part of the co-creation:
1. Which tools (mobile, video, photo) can be used best for data collection and how?
2. How can the input generated through these tools be visualized (e.g. a dashboard, through mapping) and which software is needed to do this?
3. How can we motivate and engage project owners to use this tool?
In three separate groups specific input related to these questions is given. And with that, the co-creation M&E 2.0 comes to an end. Thank you Mark Tiele Westra (Akvo), Ineke Aquarius (Butterfly Works), Jaap van ‘t Kruis (Ventanaz), Joost van der Made (Frog Design), Arjen Swank (Text to Change) and Peter Huisman (Oxfam Novib)!
Now, allow us to take all the input related to these three last questions back to the drawing table first, before we share the provisional answers in our next blog. This also creates the opportunity for you to become part of this co-creation process and to come up with your suggestions. Be our guest, all input is appreciated…
Keep you posted!
How do you normally feel about monitoring and evaluation (M&E)? A drag? Boring? Complicated? A ‘have to’? You wouldn’t be the only one. It’s part of the job though. But it can definitely be more simple and useful. That’s why we are excited to announce the development of something new: an M&E 2.0 toolkit, that adds a pragmatic and more fun dimension to existing methods. Because we believe you can make it work!
So what’s new? Let’s sketch some differences between current M&E and M&E 2.0 from our viewpoint:
| M&E 1.0 |
M&E 2.0 |
| Top down (led by project managers) |
Bottom up (led by the participants) |
| Experts (experts provide opinion) |
Crowd (target groups provide multiple opinions) |
| Transparency |
Open |
| One time (after intervention finished) |
Real time (continuously throughout project) |
| Accountability (to donors) |
Open source (stakeholders use the data they need) |
| Broadcasting (one source sends) |
Conversation (multiple sources are in dialogue) |
| Focus on institutions |
Focus on people |
| Control (of people and project) |
Trust (among people) |
| Justification (to donors) |
Learning (continuous learning during intervention) |
| Single data entry |
Multiple data linked to each other |
Over the past years several M&E methods & tools have been developed that fully or partly adhere to the M&E 2.0 philosophy as described above. This is not to say that 1.0 is dead, it’s just time to add new dimensions and make it more bottom-up, fuelled by the crowd, real time and more a learning experience instead of a justification method.
1%CLUB and EyeOpener Works are in the process of selecting and testing new M&E 2.0 methodology. Additional criteria for this methodology, in addition to the ones mentioned above, are simplicity/applicability, scalability, open-source and multi-disciplinarily.
Three current M&E methods & tools that in our view best match these criteria for selection of M&E 2.0 methods & tools are:
- Most significant change (MSC)
- Participatory Video (PV)
- Mobile reporting
We’ve chosen to undertake the pilot in Kenya with six 1%CLUB projects. In collaboration with other experts and relevant stakeholders we hope to come up with new methodology that aims to combine the best elements of already existing tools. The next four months will be a process of continuous reflection on our endeavours, which we’ll gladly share with you. Join our learning curve!
More to come… Keep you posted!
|
|
