New Monitoring & Evaluation Site by UNICEF's Evaluation Office

UNICEF's Evaluation Office has developed a website called, MyMandE. As stated on the websites homepage, MyMandE "is an interactive WEB 2.0 platform to share knowledge on country-led M&E systems worldwide. In addition to being a learning resource, My M&E facilitates the strengthening of a global community, while identifying good practices and lessons learned about program monitoring and evaluation in general, and on country-led M&E systems in particular."

"While My M&E was founded by IOCE, UNICEF and DevInfo, it is managed by a consortium of partner organizations including IDEAS, IPDET, WHO/PAHO, UNIFEM, ReLAC, Preval, Agencia brasileira de Avaliacao, SLEvA and IPEN. If your organization wishes to join the consortium as a partner, please send an email to Marco Segone, UNICEF Evaluation Office, at msegone@unicef.org."

"My M&E is a collaborative website whose content can be modified continuously by users. To develop and strengthen a global community on country-led M&E systems, registered users have the facility to complete their own social profile and exchange experiences and knowledge through blogs, discussion forums, documents, webinars and videos."

Marco Segone discusses MyMandE in an presentation available online. MyMandE has the following sections:

• Wiki
• Community
• Webinars
• Videos
• How to
• Trainings
• Virtual Library
• Jobs
• Roster

Under Videos, my 3-part series on impact evaluation is listed! So, for those interested in international program/project evaluation this site provides many resources (and hopefully more will be added) and allows the international community of M&E people to contribute.

Standardized Tools for Measuring Child Abuse and Violence Against Children

In 2006, the World Report on Violence Against Children was published by the United Nations, authored by Paulo Sergio Pinheiro. This report  was the first comprehensive global attempt to describe the scale of
all forms of violence against children and highlighted the large  scale rate of violence against children at home, in schools, in their  communities, and by the state despite most countries having signed  the Convention of the Rights of the Child.

From a monitoring & evaluation perspective, why should I discuss child abuse and violence against children? One of the major findings of this study was that too many complicated and different tools were being used across countries which made it difficult to assess and compare child abuse rates and trends.

In response to this finding, recommendations were made to develop a approaches and standardized ways to measure voilence agaisnt children that would allow for cross-national comparisons. A set of tools were
developed by 122 experts and are discussed in an article published in 2009 in the journal, Child Abuse & Neglect. These tools are called the International Child Abuse Screening Tools (or ICAST). The ICAST tools were piloted in convenience samples in 7 countries representing all regions of the world. As stated in the journal article mentioned above, these tools were specifically designed to be used in cross-cultural, multi-national, multi-cultural studies so that comparisons could be made across time and countries.

There are 4 ICAST tools to measure child abuse: a) the parent report for young children, b) the child report, c) the institutionalized child report, and d) the young adult retrospective report. These 4 ICAST tools can be found on the International Child Abuse and Neglect website. They have been translated into the following languages: Spanish, Arabic, Icelandic, Hindi and Russian.

If your project or program is considering a baseline assessment of child abuse or violence, these tools may be useful.

Personal Digital Assistants (PDAs) and Child Protection

Recently, Save the Children (SC) received a grant from Google to test some innovative ideas. One of the innovative projects that SC is testing is the use of personal digital assistants (PDAs) in child protection. PDAs are hand-held computer that is a mobile information manager.

The use of PDAs for child protection is being field tested in the country of Azerbaijan. Once part of the Soviet Union, Azerbaijan-- like many other Soviet states-- institutionalized children for even minor mental or physical disabilities or even if the parents could not afford to support them. Since independence, international organizations have encourage the Azeri government to pass de-institutionalization reforms; that is, as much as possible returning children to their families, or foster homes, and supporting both the children and the families. One approach to meeting de-institutionalization of children in Azerbaijan is the community case management approach. Case management involves assessing, monitoring and evaluating each child and family. Community case management involves ensuring local referral and support services are available, accessible and used by the children and their families.

Currently, child case management involves the use of paper-based in-take assessment forms, child and family monitoring forms, and referral service follow-up. Collecting all this data requires time and money. In addition, the time to process all these data (data entry, data cleaning, data analysis, case reports produced) means that the children and families may not have immediate care or support they need.

With a current project in Azerbaijan, SC is conducting a comparative study of paper-based vs. PDA child case management. In this study, 10 of 60 child case management social workers have been randomly selected to participate in two groups. The first group will be 5 randomly selected social workers selected to use paper-based child case management forms and the second group will be 5 randomly selected social workers to use PDAs that have digital versions of the child case management forms.

After social workers in each group has completed case management forms for approximately 10 children and families, SC will examine the following:

• The time and cost differences between paper-based vs. PDAs for each child and family, which will determine the cost effectiveness of PDAs. That is, do PDAs reduce costs and if so, how much;
• Data quality, which will determine if the error rates are different. That is, does direct data entry and transmission to the central database reduce errors and if so which errors and by how much; and
• User satisfaction, which will determine if social workers are most satisfied with paper-based on PDAs as a case management data tool.

The pilot testing of PDAs for child protection is occurring in Azerbaijan as I write (end of Aug 2010) but should be completed in late September. Once the results are in I'll share them on this blog.

Admitting Project Failure

Buckminster Fuller said, "If I ran a school, I'd give the average grade to the ones who gave me all the right answers, for being good parrots. I'd give the top grades to those who made a lot of mistakes and told me about them, and then told me what they learned from them."

This quote characterizes the new website, FAILFaire, which provides an online forum to report and discuss project failures. As I have mentioned in a previous blog, all too often most project evaluations do not mention any project shortcomings or failures. The international development community often highlights success and files away failures. But this is a mistake!!! Only by "talking openly and seeing where we have failed may help us learn, make better decisions, and avoid making the same mistakes again." I believe it is no secret that many project just don't work...for various reasons.


Hopefully, though, most projects should be based on sound evidence that they will work prior to implementation. But, for those occasions when projects do fail they should be talk about for lessons learned. 

Besides the website, FAILFaire holds conferences. The 1st was held in New York by MobileActive and focused on technology and the 2nd was held in Washington DC in July by the World Bank.

One example reported at a FAILFaire conference was by UNICEF. It was the 5 Million Stories by
2010 Project. UNICEF Innovations’ Chris Fabian and Erica Kochi co-presented what they jokingly referred to as a “zombie project”, because despite the fact that the project couldn’t get off the ground, it kept being half-heartedly restarted over the years. “Our Stories” was designed to give children around the world the chance to tell their stories to be published online as part of a look at the global experience of childhood, with the ultimate goal of having 5 million stories posted by 2010.

Launched in 2007, Kochi and Fabian estimate the project had a .008 success rate, since it only gathered 400 stories. They say that this project was a failure of real world application, in that although the idea was good, there was no real desire for it among the community it targeted. As Kochi explained, “No one asked for this.” Other problems included using proprietary, non-open source code so that they couldn’t adjust when there were problems, a lack of ownership and commitment to the project by key stakeholders, and a long timeline that mean that resources never aligned with needs – in 2007 there was money for PR, in 2008 pro-bonoe design resources, in 2009 the software development. In 2010, they finally shelved the project.


So, if you have a failed project, please proudly post it on FAILFaire.