When people ask about “development impact” they are often referring to the impact of some set of development policies and projects; let’s call it a “development portfolio.”

The bulk of the new evaluative work going on today is assessing the impact of specific projects, one at a time. Each such project is only one component of the portfolio. (And each project may itself have multiple elements.) As evaluators we worry mainly about whether we are drawing valid conclusions about the impact of that project in its specific setting, including its policy environment. The fact that each project happens to be in some development portfolio gets surprisingly little attention.

There is both a demand side and a supply side to evaluations. On the demand side, the choices about what gets put up for an evaluation appear to be heavily decentralized and largely un-coordinated. Interaction effects are sometimes studied, but for the most part the components are looked at in isolation. Then we will have little option but to add up the results across multiple evaluations and hope that interaction effects are minimal.