Assumptions, evidence and multiple stakeholders

Over the last few months I have been on the sidelines of a review of an NGO funding mechanism. The review report has been drafted, then re-written. But as yet, as far as I can see, three major issues have not been addressed. These are likely to be relevant to many other multi-donor NGO funding mechanisms.

Issue No. 1: The treatment of key assumptions

The first issue is core funding of NGOs (national and local). At the centre of the original project design was the belief that that provision of core funding will make an important difference to how NGOs work. The review team recognised this idea. But they did not then question or explore in any detail how the provision of core funding will lead to better development outcomes. Yet this was undoubtedly the potential killer assumption in the centre of the project design. In fact there are two linked assumptions here, that both needed examination, even if only at a desk level.

The first assumption is that core funding will increase the freedom and autonomy of NGOs. This assumption could have been explored by looking at the different NGOs that had been funded by the project, and then making some comparisons.

Firstly, by comparing NGOs where project provided core funding was a big versus small proportion of the NGO's overall budget. In the review there was no table showing such figures, though they were readily available, and though there were significant differences between NGOs in this respect. Is there any evidence of autonomy being greater where core funding was a bigger proportion of an NGO's income? Or are other factors more important in determining autonomy? An even bet, I suspect.

Secondly, by comparing the extent of the constraints imposed on NGOs by core funding mechanism, versus the constraints the NGOs experienced when using other sources of funding. Did the review team ask NGOs to compare the project (as core funder) to their other donors in terms of the constraints they imposed, and what did they find out about the differences? Complaints about funding procedures need a comparator.

The second assumption is that increased freedom and autonomy of NGOs will lead to better development outcomes.

Here it would be useful to compare the core funded NGOs’ performance against that of other NGOs who are more constrained by their donors (e.g. as a result of their project specific funding), but working on the same type of development outcomes. For example, where both NGOs work on education sector issues. At the very least it would have been possible to identify some of these cases through interviews with NGOs, and maybe even interview some of them, to at least get to the stage of developing some indicative hypotheses.

The reason for making such a comparison is that there are some good counter arguments in favour of constraint as necessary component of creativity, versus privileging freedom and autonomy. Biological evolution is the most creative process we know of, and that process works through the imposition of a very severe constraint: the need to be able to adapt to the current environment, or die. Architecture is another field where it is recognised that the presence of constraints can drive creative solutions. There have also been extensive research on the role of constraints in the fields of art and literature.

Issue No. 2: The use of evidence

The second issue was about the use of evidence. Although there had been an annual review earlier in the year, important lessons have not yet been learned from the experience. In that review there was extensive and selective use of unattributed comments by NGOs, with no information presented on how representative each of these views were. Understandably that caused major problems for the acceptance of that review. The first and second draft of the mid-term review seemed to continue that questionable tradition, albeit with a little more balance.

The alternative approach, which had been proposed before the most recent review, was that by default, all comments made by NGOs should be from identifiable sources. Exceptions could then be made where there were explainable reasons why identities had to be withheld. The assumptions behind this proposal were that:

* NGOs are mature organisations led by mature people who have a working relationship with the funder, which can withstand open expression of criticisms. Not the reverse.

* NGOs need to be confident and assertive, if they are to be effective advocates. If they cannot openly express their critical views to their own donor, how can they ever be effective advocates of critical views to less sympathetic audiences?

In contrast, the review made a brief and sympathetic reference to the earlier reviews use of evidence from NGOs, and then focused on the issue of whether results of the review interviews should be quantified or not. This was not the primary issue, and does not even need to be seen as an either/or choice. The primary issue with the both review methodologies was how transparent and trustworthy the process of data collection and analysis is. The continued selective use of unattributed comments weakened the value of both reviews.

Given there are only a small number of NGOs that had been funded it would have been quite easy to tabulate, using text not numbers, all the answers given to a number of key questions, using one table per question, and to use these in the respective relevant sections of the report.

Issue No. 3: Multi-stakeholder involvement

The third issue was multi-stakeholder involvement. The problematic nature of multi-stakeholder involvement in strategy design and evaluation was barely recognised. The project design required a common goal and convergent activities working towards that goal. Yet it involved working with NGOs who are autonomous, diverse and sometimes conflicting in their views of the world. The project's ability to find a common goal, or to mobilise people around a common goal, was in practice very very limited. Similar challenges face the whole issue of appropriate NGO representation on the project’s governing body.

Nevertheless, in this context the review proposed “Widening the dialogue on problem definition and strategy development, bringing together NGOs, government, donors and others, and using competitive funding to NGO consortia to channel demand and support the identified priorities”. And at the same time “Limit the role of the [management team] to administering grants and allied activities, avoiding other activities of a more interventionist type that might undermine the central aim”.

The review proposed that project’s strategy be defined via the proposed steering committee, and supported via “strategic issues” meetings involving a wider group of stakeholders. Although proposals were made re the inclusion of different categories of people, based largely on expertise categories, how they are chosen, and subsequently re-appointed and replaced was the more challenging question that was answered.

There is a counter argument that an independent review team could have given some attention to. That is, project’s strategy should be developed by a limited group of identifiable stakeholders with visible interests. And that NGOs using project funding should also be encouraged to seek funding from alternative sources. Overall, the project (or its donors) should be encouraging the development of plurality of funding sources, representing a diversity of strategies. Rather than trying to merge many conflicting interests within the strategy of one funding mechanism, in a non-transparent fudge that satisfies no one.