The next set of project evaluation designs that I will presenting are considered weaker than the previous designs; weaker meaning that they can not provide good evidence that a project's interventions are directly attributable to the measured outcomes or to what degree the project contributed to the measured outcomes (net impact).
The pre-test with a project group combined with a post-test of both project and comparison group design is shown below. As the name implies, the pre-test (baseline study) is conducted only with the people involved in the project. There could be a number of reasons why this might occur: save money, project team didn't like the idea of a comparison group initially, or some technical reasons. However, later there may be the budget, interest or feasibility to include a comparsion group.
The advantages of this design are that it can assess reasonably well how a project is being implemented and whether intended outputs from activities were produced. IF the comparison group studied at the end of the project is quite similar to the project group in characteristics, and adequate mixed methods can demonstrate the comparison group were similar at the baseline as the project group on the outcomes that will be measured, then this design MIGHT demonstrate project effects.
The disadvantages are that is that it is difficult to conclusively determine if the differences between the project and comparison group at the end-line study are due to the project or other factors. Another weakness of this design is that local context events can effect outcomes in the comparison group. For example, an agricultural project compares agricultural output of its farmers to a comparison group of farmers without knowing that the the comparison group of farmers received more irrigated water than previous years thus increasing their output. To monitor local context events, retrospective information can be obtained from the comparison group.
