Using Evaluation to Shape Strategy: How Can We Make It Better? Are We There Yet?

Publication Date: 4/19/2011

Evaluation is an important tool in our grantmaking program. We use it to review progress in a field, explore what needs to be done, and help us tweak strategies as needed along the way.We do this kind of evaluative work to some extent in all of our program areas, but in this month's "What We're Think

In our Environment & Communities program area, one of the four goals is to protect, conserve, and restore Greater Philadelphia’s water resources in the Delaware River Basin.  To achieve this goal, we have two objectives: promoting policies and practices that facilitate “green infrastructure” approaches to storm water management; and promoting regional policies and institutions that protect drinking water in the Delaware River Basin, particularly Philadelphia. 

So how did we select these priorities?

As with all of our current grantmaking, it starts with our 2001 Strategic Plan. “Protecting and restoring watersheds and related ecosystems” was one of three program priorities established in the Environment & Communities program area, building upon 15 years of grantmaking in the field, including two major efforts to protect and restore the Delaware and Schuylkill Rivers—waterways that historically have played important roles in shaping the growth and development of Greater Philadelphia and are major sources of drinking water. 

In conjunction with the Strategic Plan, we increased our evaluation efforts and commissioned strategy reviews in several of our major funding areas, including the watershed work. A strategy review looks at a cluster of grants, not to see how each grantee did, but to assess the Foundation’s choice of program goals and the work to meet those goals. This review, along with further analysis and discussions with practitioners in the field, led to refining our grantmaking priorities. We developed separate funding priorities for protecting water resources and for preserving open space, though both promoted an integrated approach to land use and water quality, and emphasized what we were going to accomplish (outcomes), not just what we were going to do. In addition, the E&C program piloted a new grant review process for its work in the Schuylkill River Watershed, using a Request for Proposals that emphasized our interest in collaborative work on policy and demonstration projects in the watershed.

Based on the revised grantmaking strategies, the Foundation made significant investments in the region toward protecting water resources; since 2005, we awarded 66 grants totaling over $14 million to advocate for state policy reform and promote innovative local, demonstration projects, with an emphasis on storm water policy/practice and drinking water for Greater Philadelphia. In addition, grants totaling over $10 million were awarded to 21 organizations as part of the Schuylkill River Watershed Cluster for projects including trails and river recreation, land conservation, water quality/quantity, and networking and capacity.

By 2009, Foundation staff and leadership wanted to know what we had accomplished and what were the salient issues going forward. For this we commissioned two assessments: an evaluation of the Schuylkill Cluster grants (Peter Szabo, 2010), including the Schuylkill Learning Community in which all grantee organizations participated; and a strategy review (ConservationStrategy LLC, 2010) of our overall work on water policy and protection. The strategy review included a significant exploration of current issues in the Greater Philadelphia region to help the Foundation focus its grantmaking.

These two reviews were instrumental in helping the Foundation define and focus its current grantmaking strategies (described earlier) to protect water resources in the region. We learned, among other things:

• Supporting groups working on water policy is very important to the region—but we needed to focus on specific issues to work on.
  
  
• If we want demonstration programs to lead to policy changes, then we need to be deliberate about that from the start, including understanding the pathway to change.
  
  
• We have more success in a specific geographic area when the projects funded are tightly aligned with our overall grantmaking strategies.
  
  
• There are critical water policy issues in our region right now, so we should focus on them in a timely and deliberate fashion. Thus our focus on green infrastructure for stormwater management and efforts to protect drinking water in the Delaware River Basin, starting with understanding the potential impact of natural gas extraction.

We try to strike a balance between assessing and refining our grantmaking strategies and giving them enough time to show results. We think we’ve managed that balance in the water policy work, and we are venturing forward with a more focused and targeted strategy aimed at critical, timely issues in the region.

Likely, in a few years, we’ll be assessing our results again!

_{Photo courtesy of Schuylkill Banks Development Corporation}