I have just re-read Howard White's informative blog posting on "Using the causal chain to make sense of the numbers" which refers to what he calls "the funnel of attrition". I have reproduced a copy of his diagram here, one which represents the situation in an imaginary project. He uses the diagram to emphasis the need to do basic analyses of implementation data (informed by a theory of change) before launching into sophisticated analyses of relationships between outputs and impacts.
The same set of data can be represented using a Venn diagram, to show the relationship between these 8 sets of people, as shown in this truncated version below:
Venn diagrams like these can also be read as describing relationships of necessity and sufficiency. According to the above diagram, knowing about the interventions is a necessary condition of taking part in the intervention. There are no cases (in the above sets) where people have taken part without already knowing about the intervention.
However, it is conceivable that some people could be assigned to an intervention without knowing about it in advance and making their own choice. In that case the set relationships could look more like the diagram below (yellow being participants who were assigned without any prior knowledge). Here the key change is the overlap in their memberships, the actual numbers of people could well be the same.
Its possible to imagine other complexities to this model. For example, some people may change their behavior without necessarily changing their attitudes beforehand, because of compulsion or pressure of some kind. So the revised model might look more like this...(brown being participants changing their behavior due to compulsion)
In both these examples above, what was a necessary condition has becomes a sufficient condition. Knowing about an intervention is sufficient to enable a person to participate in the intervention, but it is not the only way. People can also be assigned to the intervention. Similarly, changing their attitudes is one means whereby a person will change their behavior but behavior may also be changed through other means e.g. compulsion.
The point of these two examples is that when monitoring implementation it is not good enough to simply record and compare the relative numbers who belong to each consecutive group in the "funnel of attrition" . Doing so implies the Theory of Change (or Theory of Action, as some people might prefer to call this) is the only (i.e. necessary) means by which a desired outcome can occur, which seems highly unlikely. Instead, what is needed is a comparison of the membership relationships between one set and the next, to identify whether other conditions might also be sufficient for the expected change to happen. This can be done using nothing more complicated than cross-tabulations in Excel.
But this view does have significant implications for how we monitor project interventions. It means it is not good enough to simply track numbers of people participating in various activities. In order to identify possible relationships of necessity and sufficiency between these events we need to know who participated in each activity, so we can identify the extent to which membership in one set overlapped with another. In my experience this level of implementation monitoring is not very common.
PS: For more reading on set relationships and concepts of sufficient and necessary causal conditions, I highly recommend:
PS: For more reading on set relationships and concepts of sufficient and necessary causal conditions, I highly recommend:
Goertz, Gary, and James Mahoney. 2012. A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences. Princeton University Press. http://www.amazon.co.uk/Tale-Two-Cultures-Qualitative-Quantitative/dp/0691149712/ref=sr_1_1?ie=UTF8&qid=1353850106&sr=8-1.
