On Home Leave (to the US) Until July 2010

I will be taking my annual Home Leave to the US during the month of June, 2010. When I return home to Tbilisi, Georgia, in July 2010 I will resume my program/project design, monitoring and evaluation blog.

Competencies Needed by Evaluators

In 2001, the American Journal of Evaluation published an article by Jean A. King, Laurie Stevahn, Gail Ghere and Jane Minnema titled, "Toward a Taxonomy of Essential Evaluator Competencies," (22; 229). These authors conducted an exploratory study to determine the extent to which 31 evaluation professionals from diverse backgrounds and approaches could reach agreement on a proposed taxonomy of essential evaluator competencies.

Using the weighted scores, I entered the competencies into Wordle to get a graphic representation of these competencies. Those competencies in large font size were weighted as more important than those in small font, though all were considered important.

The most important competencies were Framing a Research Question, Research Methods and Research Design. Of comparatively less important were competencies of Training Others, Supervising, and Responding to RFPs (Request for Proposals).

Quotes Related to Evaluation

• True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information. Winston Churchill
  
• The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements and expect me to fit them. George Bernard Shaw
  
• Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. Albert Einstein
  
• We cannot discover what ought to be the case by examining what is the case. We must decide what ought to be the case. Paul Taylor
  
• The most serious mistakes are not being made as a result of wrong answers. The truly dangerous this is asking the wrong question. Peter Drucker
  
• One of the great mistakes is to judge policies and programs by their intentions rather than their results. Milton Friedman
  
• The pure and simple truth is rarely pure and never simple! Oscar Wilde
  
• First get your facts; then you can distort them at your leisure. Mark Twain
  
• My mind is made up, don't confuse me with the facts! Unknown
  
• A funeral eulogy is a belated plea for the defense delivered after the evidence is all in. Irvin S. Cobb
  
• There's a world of difference between truth and facts. Facts can obscure the truth. Maya Angelou
  
• It is easier to believe a lie that one has heard a thousand times than to believe a fact that no one has heard before. Unknown
  
• We want the facts to fit the preconceptions. When they don't it is easier to ignore the facts than to change the preconceptions. Jessamyn West
  
• If at first you don't succeed, destroy all evidence that you tried. Unknown
  
• Absence of evidence is not evidence of absence. Carl Sagan
  
• Evaluate what you want -- because what gets measured, gets produced. James Belasco
  
• Price is what you pay. Value is what you get. Warren Buffett
  
• For changes to be of any true value, they've got to be lasting and consistent. Tony Robbins
  
• Extraordinary claims require extraordinary evidence.Carl Sagan

Country-Level Monitoring & Evaluation Dashboard

This past week (2-7 May 2010) I worked with a great team in SC's Cairo office to sketch-out a system that would provide the Senior Management Team (SMT) a consolidated "dashboard" report of key project
indicators on a quarterly basis.

The goal of the dashboard report is to provide the SMT an at-a-glance view of the progress of all projects to meet 1) quarterly benchmarks on key indicators, 2) successes and challenges for that quarter, 3) the burn-rate of project funds, 4) as well as number of beneficiaries reached, thus allowing better informed decision-making by the SMT on project improvement BEFORE the end of the reporting year or the project.

Like the dashboard of a car, this Dashboard Report will carry only those indicators that the SMT consider "key" to measuring the accomplisment of the project NOT ALL indicators.

Example of a Dashboard Report

A Dashboard Report is primarily based on comparing quarterly achievements with quarterly benchmarks for activity-level indicators. The Dashboard Report will present result-level indicators on an annual basis. Therefore, if projects do not have end-of-project targets and established quarterly (or semi-annual) benchmarks then developing a meaningfuly Dashboard Report to show process is more difficult.

In addition, identifying over or under achievment of quarterly benchmarks, or annual targets, will only be the beginning of the process; if, for example, there is an under achievement of a quarterly benchmark, then the SMT can "drill-down" deeper with the Project Director as to possible reasons which can be discussed and hopefully resolved before the next quarter.

A Dashboard Report should be simple and easy to read; however, this takes a lot of work! Behind a good Dashboard Report is consensus of key indicators, systems to ensure data quality, adequate staff resources, long-term SMT committment, userfriendly IT tools, clearly defined roles and flow of data, and project with M&E plans that clearly state end-of-project targets to be achieved and quarterly or semi-annual bechmarks to track progress toward the targets.

The Egypt Country Office will be working on developing these systems and has planned the 1st draft of a Dashboard Report this summer. Once it is developed I will present it here.