tag:blogger.com,1999:blog-6719829.post113473459722931161..comments2024-03-27T12:04:05.897+00:00Comments on Rick On the Road: The "attribution problem" problemRick Davieshttp://www.blogger.com/profile/07028422984421301184noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-6719829.post-1136211808486589692006-01-02T14:23:00.000+00:002006-01-02T14:23:00.000+00:00Hola Rick,With the best for 2006 and in response t...Hola Rick,<BR/><BR/>With the best for 2006 and in response to yours of 23 December.<BR/><BR/>I agree...especially when the emphasis in monitoring and evaluation is on process rather than product, a big challenge in these times of results-based management. The context of development is so complex, open and dynamic that I am more and more convinced the planning challenge is one of addressing uncertainty. And if there is uncertainty about what you will be able to do, all the more uncertain is what you will achieve, even in terms of activities and outputs. And when we are talking, as we are, about end results, SMART predefinition of outcomes will tend to lead you away from engaging with other actors who aim for the desirable social change, instead of simply the possible. <BR/><BR/>For planning, I have been working with a strategic risk management approach that focuses on enabling an organisation to consciously and continually seize opportunities and run dangers and threats. The roles of other social actors (and external and internal factors) figure prominently in this methodology but you remind me that I must re-visit your network model. Thanks.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6719829.post-1135352706328629662005-12-23T15:45:00.000+00:002005-12-23T15:45:00.000+00:00Hi Ricardo, (responding to your second paragraph.....Hi Ricardo, <BR/><BR/>(responding to your second paragraph...)<BR/><BR/>The ethical dilemma of how to claim responsibility for results that others are likely to have contributed to arises, in my view, from problems at the planning stage. “M&E is the back-end of strategy” – that’s where a lot of M&E problems originate. If development agencies start by developing plans that acknowledged the role of other actors then expectations will be established that: (a) they will not be the only agent of influence, (b) they will have to generate information about the influence of those other actors, as well as themselves. Linear logic models (e.g. the Logical Framework) are not conducive to this type of contextualised planning (and learning). Network models are more conducive, especially those which describe actor networks.<BR/><BR/> I go back to an earlier point here. The end results that aid organisations should be looking is new knowledge about how change is taking place, not simply the reporting of change that has happened (and then making some claim on that). The former is usable by many others, the latter is much less so.Rick Davieshttps://www.blogger.com/profile/07028422984421301184noreply@blogger.comtag:blogger.com,1999:blog-6719829.post-1134905798365628122005-12-18T11:36:00.000+00:002005-12-18T11:36:00.000+00:00And mine on yours:- Attribution or contribution – ...And mine on yours:<BR/><BR/>- Attribution or contribution – What is the difference?: <BR/><BR/>Thanks for calling me on this. The difference is probably only of connotation. As you know, development agencies seek to hold their grantees accountable for results. In this process "attribution" has come to mean there is evidence that predefined results were achieved AND would not have been achieved without the activities that we funded. "Contribution", however, is more modest and realistic: there is evidence of a reasonable relationship between outcomes and the activities of a grantee. Also, it is more viable to hold grantees accountable, and for them to be demonstrate, for their contribution to development. <BR/><BR/>- Looking at the role of other actors: <BR/><BR/>The point is well taken. In my next evaluation assignment I will expand the question on evidence of contribution to that of other actors. It will be most interesting because my experience so far is that it is a major challenge, for funding agencies and NGO grantees alike, to agree on and communicate in a verifiable manner the outcomes to which they have contributed. But, part of the difficulty is an ethical dilemma--how to claim to be even partially responsible for results to which others have contributed as much or more? So, it occurs to me that tackling that challenge head-on would be productive.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6719829.post-1134843413131403542005-12-17T18:16:00.000+00:002005-12-17T18:16:00.000+00:00Hi RicardoThree comments on your comments:- I am n...Hi Ricardo<BR/><BR/>Three comments on your comments:<BR/>- I am not sure what the difference is between identifying the contribution versus attribution<BR/>- I am in favour of your more actor-oriented description of what an outcome is. This makes outcome descriptions more communicable, more comrehendable and more verifiable than descriptions of abstract events or processes (which are endemic)<BR/>- I dont feel comfortable with focusing in too exclusively on the contribution the development organisation/project has made to the outcome. A wider focus that also looked at the role of other actors, and the context, might help generate more usefull generalisable knowledge that would help make the outcomes more replicable elsewhere.<BR/><BR/>regards, rickRick Davieshttps://www.blogger.com/profile/07028422984421301184noreply@blogger.comtag:blogger.com,1999:blog-6719829.post-1134822573095744952005-12-17T12:29:00.000+00:002005-12-17T12:29:00.000+00:00Rick, I really appreciate your views on this issue...Rick, I really appreciate your views on this issue. In my own experience, however, there is a relatively simple way forward. For social change organisations operating in environments that are characterised by complex, dynamic and open systems, the solution is to identify contribution instead of attribution of outcomes. Easier said than done, of course. <BR/><BR/>The problem begins with agreeing on what we mean by “outcomes”. I find the general definition (see for example the OECD Glossary of Key Terms in Evaluation and Results Based Management, 2002) to be insufficient. OECD proposes: “The products, capital goods and services which result from a development intervention….” The devil is in the “resulting from” that leads to the despair over the difficulty, or impossibility, of attribution. (The OECD translation of outcome into Spanish as “efectos directos” emphasises the conceptual issue.) In contrast, I find the emphasis of the Evaluation Unit at IDRC on “development is accomplished through changes in the behaviour of people” to be more helpful. <BR/><BR/>Today, my generic, working definition of outcomes is “changes in the behaviour, relationships, activities, or actions of individuals, groups or organisations that are a direct or indirect, partial or total result of the activities or outputs of the development actor.” I rush to clarify that although generally these changes are the result of multiple actors and factors, there are some—a presidential decree for example—that may be attributed to one forceful, effective actor. <BR/><BR/>I have used this definition in three quite different evaluations this year—for a development agency’s multi-annual environmental programme in Latin America, a 36-member European network and a smaller Asian federation. First, I customise the outcome definition for each organisation. Then, I ask the organisations being evaluated to a) formulate the outcomes they have achieved and b) provide the evidence of their contribution. As evaluator, quickly my role becomes that of facilitator and even capacity-builder for these organisations as they identify and communicate concretely and rigorously the changes to which they have contributed. The results are inspiring—for the development actors, for those who commissioned the evaluations and for me as the evaluator.Anonymousnoreply@blogger.com