Impact, Influence, Leverage and Learning (I2L2)

[In November 2015 ORS Impact and I revised and updated the I2L2 framework document. The new link is I2L2 and below].

Years ago while working at the Annie E. Casey Foundation we struggled with how to organize, summarize and communicate very diverse grant making strategies and results (from direct service work to community change, from technical assistance and capacity building to policy and advocacy). We had plenty of numbers and examples without a common framework for communicating them. This triggered both a return to basics around an intentional outcomes focus (with Results Based Accountability) as well as common definitions for the “types” of outcomes and results we were aiming for. There was a lot of experience with naming and describing outcomes for child and family well-being (e.g., increased employment, improved school attendance) but the struggle was often with summarizing more developmental outcomes like organizational capacity, changes in attitudes and beliefs, and early policy and advocacy investments. Collective memories may be fuzzy but I credit Miriam Shark at Casey with advancing a set of 3 result categories:

  •  Impact – results that were intended to achieve direct change and impact on people
  • Influence – results that described intended change in organizations, beliefs and behaviors, contexts, and policies and practices
  • Leverage – changes in resources and funding, in this case, especially where the foundation investments influenced others to change how they invest in the same or similar strategies

We later discussed including a second “L” for Learning outcomes–especially where there was intentional strategies to acquire knowledge needed to inform other work. This list may seem simple (and even obvious now in retrospect) but it provoked some key thinking and behaviors.

First, it helped program officers and grantees organize and report results in all 4 categories which was especially helpful for activities and investments that could not measure community impact directly or within a short time period. The influence and leverage results could describe the early evidence that change was happening on a path to impact outcomes. In addition, it allowed for not only results from different strategies within a portfolio or across the foundation to be consolidated, it also helped people to look at all four types of impact, influence, leverage and learning outcomes for individual grants or activities.

Second, it provided a prompt to everyone to define their intended results (and intentional strategies) for all four categories at the beginning of the planning and work. Again, this is certainly obvious within most results and outcome based planning but often the focus is only on the long term impact and less upfront attention is given to the earlier and needed influence and behavior change outcomes needed to achieve changes in people and places. What often happens is that impact results are defined up front and measured but if not fully achieved both foundation and grantee fell back on narrative or bullet-form examples of “other changes” that occurred–often influence and leverage–defined and documented in retrospect.

Starting with this initial set of ideas I asked ORS-Impact to prepare a tool for Making Connections community change sites and grantees to understand how to define and measure influence and leverage. This initial guide helped get the concepts and early definitions to a limited audience of grantees and it provided examples of indicators and ways to measure both influence and leverage. Later in 2006, we focused on the policy and advocacy aspects of influence which took the form of other manuals and guides, and also contributed to the growing policy advocacy evaluation work.

I continued to use the initial framework with the inclusion of learning outcomes in different work with multiple organizations and ORS-Impact also went back to it in work with other clients. Deceptively simple but helpful as an organizing framework when the work or array of investments and strategies have different levels of focus and change and operate in different timeframes but are meant to relate and be complementary. Certainly deeper and more comprehensive theory of change exercises help to define these same elements in different ways but these can be challenging to summarize and communicate to audiences not immersed in the work (like board members or the general public).

So we decided to go back to the original ideas and publications and spend time documenting good case examples of how the framework has been used and what organizations have gained from it. Jane Reisman, Anne Gienapp, Sarah Stachowiak, Marshall Brumer, Paula Rowland, and the ORS-Impact team worked with current and past clients and colleagues to assemble these examples. We also shared the examples at the American Evaluation Association conference and other meetings which helped to develop the version you can read here as I2L2.

We continue to receive positive feedback especially around the I2L2 framework’s ability to help organize thinking and definitions of expected change and results. Again, this doesn’t replace theory of change and other in-depth planning but when community change strategies and their intended outcomes are complex and highly interrelated (sometimes without distinct sequencing), I2L2 helps groups to organize, define, document and communicate the results they are aiming for and achieve.

So where are we now? We have spent a lot of time and effort on defining terms and examples for influence and leverage. (Others have also contributed their work on these categories–see Jim Casey Youth Opportunity Initiative‘s Assessing Leverage guide. We would like to focus on how to help people and organizations focus on defining intentional and planned learning results and the strategies to get there. Here we want to define learning to not be only the lessons acquired from (usually) failing to achieve impact or successfully reach targets but more importantly the intentional agenda for acquiring needed knowledge. Defined at the beginning and evaluated along the way.

Do you have examples of work defining learning results? Learning outcomes? How have you evaluated learning?

We’d love to hear from you.

4 thoughts on “Impact, Influence, Leverage and Learning (I2L2)

  1. Trilby Smith

    Hi Tom. Thank you for this post. In my position at Vancouver Foundation, I am responsible for supporting evaluation and learning for our Youth Homelessness Initiative. A large part of my work has been to engage our grantees in the development of a learning agenda. This agenda is focused on the things that grantees want or need to know in order to be able to do their work better. We have an overarching learning question, and all of our learning work helps us to answer that question. I try to keep focused on the short-term and long-term simultaneously – trying to find ways to improve practice now at the same time that we are trying to inform changes in policy over the long-term. It is exciting, humbling and really fun work, and I’m glad that others out there are thinking about the learning piece as part of the evaluation picture.

    Reply
    1. Tomeval Post author

      Trilby, Thank you for responding and sharing your experience and example. Being proactive and prospective and intentional with the learning agenda is exactly what I hope to see more of. Instead of only looking backwards for learning/lessons in lieu of results. And your focus on intentional use of the learning/knowledge to improve practice in the short-term and long-term are great. Thanks and good luck, Tom

      Reply
  2. Audrey Jordan

    Tom,

    No surprise that we are in 100% agreement, and that I applaud your and ORS efforts. I think the guidance materials you have provided are sorely needed and ought to be broadly utilized.

    I would love to hear of examples where foundation funded, place-based, community change initiatives fully embrace the types of influence, leverage and learning results you describe. My experience continues to be that if impact results cannot be claimed (even if the actual efforts of the funders themselves are the influence and leverage results you describe and are in no way tied to direct services or intended beneficiaries of change), and in relatively short periods of time given the insidiousness and complexities of the population-level inequities such initiatives seek to address, then “no results” is the refrain. Any good examples, anyone?

    Reply
    1. Tomeval Post author

      Thanks, Audrey – What nonprofits commit to foundation staff who promise CEOs who promise boards often skip over the important step of thinking about systems and sequence. It’s too difficult to explain the complexity so we default to simple words that put everyone in a position of promising concrete outcomes in a very specific (usually short) period of time that neglects the bigger goals of helping communities help/grow themselves. And those changes take time and capacity and networks and system changes. I am encouraged by the “embedded” and hometown funders who know they are staying and in it for the long haul so recognize there are parallel (and interrelated) results of impact and systems change. Prue Brown with others at Chapin Hall documented some of these:
      http://www.chapinhall.org/research/report/embedded-funders-and-community-change

      Thanks, Tom

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *