The context
A specialist agency is funding more than 40 different partner organisations, each working in a different part of the country but with the same overall objective of increasing people's levels of physical activity (because of the positive health consequences). These partners are often working with quite different communities, and all have substantial degree of independence about how they work towards the overall objective.
Some agency representatives have asked about the nature of the target that program as a whole is working towards, and have emphasised how essential it is that there be clarity in this area. By target they mean an actual number. Specifically the percentage of people self-reporting that they achieve a certain level of physical activity each week, as identified by an annual survey that is already underway and will be repeated in future.
Possible responses
In principle it would be possible to set a target for the proportion of the population reporting being physically active. Such as 75%. But it would be very hard to identify an optimal target percentage, given the diversity of partner localities, and the communities within these.
Relative targets may be more appropriate. Such as a 25% increase in reported activity levels. Especially if partners were each asked to identify what they think are achievable percentage increases in their own localities within the next survey period. This estimation would take place in a context where these partners already have experience working in those locations, identifying some of the things that work and dont work. My hypothesis, yet to be tested, would be that these partners will make quite conservative estimates. And if so, this might come as some surprise to the donor and perhaps some revision of their own expectations
Taking this idea further, partners could be periodically asked if they wanted to adjust their expectations upwards or downwards , of the change that could be achieved - in the time remaining in the interventions lifespan. Subject to being able to explain the rationale for doing so. My second hypothesis is that this number, and commentary, could be a valid and useful form of progress reporting in its own right.
Making sense of the responses
An assessment of overall progress over longer time scale would need to consider both the scale of ambitions and the extent of their achievement. These can't be combined into one number based on a simple formula because any such number could be achieved by adjustment of expectations and or performance. However it could be usefully represented by a scatterplot, with data points reflecting each of the partners, of the kind shown below.
The location of partners in different quadrants suggests different implications about how the different partners should be managed
- High ambition/low achievement: May need additional support, capacity building, or problem-solving
- Low ambition/low achievement: May need fundamental partnership restructuring or exit considerations
- High ambition/high achievement: Candidates for scaling, sharing learning, reduced support intensity
- Modest ambition/high achievement: Opportunities to stretch ambitions
This framework also provides plenty of potentially useful analytic questions
- Are ambitions increasing or decreasing?
- Is the gap between expected and actual narrowing or widening?
- For a given level of actual achievement did differences in expectations have any role or consequences
- For a given level of expected change what might explain the differences in the partners actual achievements
- How do individual partners positions within this matrix change over time? Are there distinct types of trajectories and how can these differences be explained?
In summary
A single numerical value based on the data in this matrix will provide a meaningless simplification.
In contrast, a scatterplot visualisation can generate multiple potentially useful perspectives.
It is more useful to see targets as necessarily malleable responses to changing conditions, than as unarguable reference points.
