Logo

PRIORITY 11:

Evidence and Evaluation

down-arrow

Our aim

Ensure evidence, data and insight inform the development of policy and practice to support active lives. Embed high quality evaluation into all Greater Manchester (GM) Moving work, to understand impact, learn and improve, and support change.

Why?

To ensure we are doing the right things with the right people to have the desired impact and understand how change happens. Therefore, we can be more effective and activities can be adapted based on emerging evidence of what works.

What have we done?

With the GM Moving, Local Pilot and GM Walking evaluation partner (Substance Consortium), we are using progressive forms for evaluation (a mix of realist, developmental and participatory approaches) to understand how change is happening, as well as what has changed.

As a result, we have learnt a lot about how change happens, and have developed five enablers of change. These are the conditions we know need to be in place in order for system change to happen. If we can identify the conditions, we can notice when they are happening, understand why, and then grow, scale and share what’s working.

The five enablers of change are:

  • Involving local people and growing assets
  • Effective work across and between sectors
  • Transforming governance and processes
  • Learning and adapting
  • Strategic leadership enabling collective leadership.

In addition, we have:

  • Created tools that make data easier to use. This has meant funding is better targeted, and therefore received by those who need it most
  • Changed how we view and measure our impact as an organisation, reflecting on our approach within GM Moving and the wider system
  • Recommissioned our evaluation partners (Substance Consortium) for another two years, with an added focus on building the skills within our workforce so the approach itself is embedded and more sustainable
  • Using the learning from GM Moving to translate into other workstreams, other agendas and places.

Case Study: A framework for capturing value

As we developed our understanding of evaluation method approaches within complex systems, it became clear that we cannot separate the actions of any one individual or organisation, nor can we show that one event or action directly leads to another. The work we do can be unpredictable, with many influencing factors.

We knew we had to develop a way to capture the value we were having and to better understand what was working and what wasn’t.

A community of practice event shared and understood all the different ways in which our partners capture value. Following this, we developed a framework, guided by the enablers of change, which sets out the changes we would expect to see if we embedded the enablers of change.

Our new way of capturing value involves understanding our impact from multiple perspectives, not just considering our own views. We take into account other types of evidence that show change: things that we see, hear and feel, and the connections we make e.g. strong relationships, trust and reciprocity.

We recognised that the ways in which we were asked to report on work were sometimes affecting what was measured and captured. Due to the work we reported on, we were finding it increasingly difficult to understand where we were having the greatest impact. For example, we may have built relationships with key partners across the system (which takes time, trust and persistence) and be working more collaboratively as a result. However, we could not capture this in our outputs, nor would we have understood any more about how to best build relationships for the future.

However, as we, and our board, are accountable for public funds, we had to ensure we made efficient use of resources. We needed to balance what we can measure as an organisation with what we can measure at a system level. This meant we established a clear rationale, and evidence base, to support the use of system indicators and demonstrate how system indicators would support the robust organisational indicators already in place. The new process helps shift what is valued in measurement and adopts new, emerging methods of evaluation, with a focus on learning and collective sense-making.

As a result, we:

  • Became better able to see where we’re adding value and where the gaps are
  • Strengthened our relationship with board as there were visible levels of trust
  • Can now accumulate evidence around enablers for change, so we have a more refined and nuanced understanding of how the enablers for change support system change
  • Can focus on the things that make a real difference in people’s lives rather than being driven by targets and KPIs.

Further details

For more information on our evaluation processes or findings contact Hazel@greatersport.co.uk.