Moving From An Evaluator Pipeline To An Evaluation Ecosystem
/If you’ve been paying attention to evaluation practice in philanthropy, you know there is work to do to evolve who we are and what we do. The majority of evaluators aren’t white by accident. The toolbox of evaluation isn’t discounting tools because they don’t work, but because they aren’t trusted or valued in favor of more “traditional” approaches. It’s time to step up and reclaim a broader array of traditions and examine more comprehensive sources of insight and evidence. If you know us, you know we care deeply about the Equitable Evaluation Framework - and it is going to take a lot of work to get there.
In order to evolve and strengthen the field of philanthropic evaluation, Engage R+D partnered with Equal Measure, Center for Evaluation Innovation, and the Luminare Group on a collective effort including three projects.
This summary report provides information about current evaluation pipeline efforts.
Broadly speaking, there are two ways people become evaluators; sometimes referred to as pipelines (or more recently pathways):
First, there are accidental evaluators – who come into evaluation with training in other disciplines and/or an understanding of philanthropy who learn evaluation while on the job.
The other route are the academic evaluators – who come into roles with extensive formal training in evaluation, but who may lack an understanding of the philanthropic context and other relevant content areas.
We found that diversifying the pipeline is the primary purpose for many of the internships and fellowships. A diverse pipeline is needed and important, but that is not the same as broadly promoting practices that enhance, reinforce, and embrace equity, as well as focus on relevance and use.
Recruiting diverse evaluators into contexts that have a strong white dominant culture will do little to advance equity and evolve evaluation. In fact, just diversifying the people doing evaluation is likely to do more harm than good. It can be an elevated form of tokenism.
It is not simply a matter of changing who is involved in the work, but the intent and the work itself.
For too long we have relied on assumptions and scripts that no longer serve us. It’s time to dig into our values and intentions and better align our practices to them. Evaluation can play a role in that, but cannot do so in isolation. There needs to be broader support for the evaluation role(s) within the organization as well as greater diffusion of evaluative thinking throughout organizations. Including evaluative thinking content in leadership development programs for philanthropy is another approach for creating a vibrant evaluation ecosystem and would likely yield benefits beyond just evaluation (e.g. better strategy development and implementation).
Conversely, efforts are needed to help evaluators grow as managers and as leaders as well as opportunities to help evaluators name their values and what they bring as evaluators as well as their limits and boundaries. This could also include support for determining when a request or proposed evaluation is not advisable and how to communicate concerns and, where possible, viable alternatives that are more appropriate.
Foundations are often turbulent contexts that require evaluators to have the ability to identify the shifts taking place and navigate the changes that means for evaluation. These changes can be significant because evaluation can serve different purposes in Foundations such as making better investment and funding decisions, identifying and maximizing impact, cresting transparency and accountability, and identifying and reducing risks and negative consequences. Understanding the role evaluation plays in the work of the foundation is critical. There is also increasing interest in looking at values alignment - looking at how the work is done, how decisions are made, and who is involved.
There is a growing sense that the evaluation pipeline needs to be reimagined and recalibrated.
As evaluation functions expand to include learning and strategy, and impact investing becomes a larger part of foundations’ portfolios; pipeline programs conceptually and as delivered may not be designed to meet this need. Efforts such as Expanding the Bench seek to reframe and better contextualize pipeline efforts. Shifting from ‘evaluator pipeline’ framing and language towards thinking about and moving towards an ‘evaluation ecosystem’ may help better reveal intentions and pathways.
It’s work that will take all of us. Are you in?
This piece originally appeared on the Luminare Group blog and follows “It’s Time to Let Go of Tired Narratives about Talent in Evaluation.”
