Friday, November 13, 2009

Measuring Advocacy and Policy Outcomes

One of the more challenging aspects of projects in Save the Children is measuring IR-4 performance (Enhanced Enabling Environment), especially if advocacy and policy outcomes are envisioned. This challenge was addressed by various authors and organizaitons in The Evaluation Exchange (Vol. XIII, No.1, Spring 2007), sponsored by Harvard Research Project.

In this issue of The Evaluation Exchange the authors attempted to define advocacy and policy change and how to evaluate this change. Excerpts from one article, which presents a illustrative menu of outcomes and strategies for various types of advocacy and policy objectives, are below.

Objective: Shifts in social norms. Social norms are the knowledge, attitudes, values, and behaviors that comprise the normative structure of culture and society. Advocacy and policy work and intervention increasingly has focused on this area because of the importance of aligning advocacy and policy goals with core and enduring social values and behaviors.

Examples of outcomes
Changes in awareness
+  Increased agreement about the definition of a problem
+  Changes in beliefs

+  Changes in attitudes
+  Changes in values
+  Changes in the salience of an issue
+  Increased alignment of campaign goal with core societal values
+  Changes in public behavior

Examples of strategies to achieve these outcomes
+  Framing issues
+  Media campaign
+  Message development (e.g., defining the problem, framing,
+  Development of trusted messengers and champions 

Objective: Strengthened organizational capacity. Organizational capacity is another name for the skill set, staffing and leadership, organizational structure and management systems, finances, and strategic planning of nonprofits and formal coalitions that do advocacy and policy work. Development of these core capacities is critical to advocacy and policy change efforts.

Examples of outcomes
Improved management of organizational capacity of organizations
   involved with advocacy and policy work

+  Improved strategic abilities of organizations involved with
    advocacy and policy work
+  Improved capacity to communicate and promote advocacy
    messages of organizations involved with advocacy and policy work
+  Improved stability of organizations involved with advocacy and
    policy work

Examples of strategies to achieve these outcomes
Leadership development
Organizational capacity building
+ Communication skill building
+ Strategic planning

Objective: Strengthened alliances. Alliances among advocacy partners vary in levels of coordination, collaboration, and mission alignment and can include nontraditional alliances such as bipartisan alliances or relationships between unlikely allies. Alliances bring about structural changes in community and institutional relationships and are essential to presenting common messages, pursuing common goals, enforcing policy changes, and protecting policy “wins.”

Examples of outcomes
+ Increased number of partners supporting an issue
+ Increased level of collaboration (e.g., coordination)
Improved alignment of partnership efforts (e.g., shared priorities,
   shared goals, common accountability system)
Strategic alliances with important partners (e.g., stronger or more
    powerful relationships and alliances)
+ Increased ability of coalitions working toward policy change to
   identify policy change process (e.g., venue of policy change, steps
   of policy change based on strong understanding of the issue and
   barriers, jurisdiction of policy change)

Examples of strategies to achieve these outcomes
Partnership development
+ Coalition development
+ Cross-sector campaigns
+ Joint campaigns
+ Building alliances among unlikely allies

Objective: Strengthened base of support. Nonprofits draw on grassroots, leadership, and institutional support in working for policy changes. The breadth, depth, and influence of support among the general public, interest groups, and opinion leaders for particular issues are a major structural condition for supporting
policy changes. This outcome category spans many layers of culture and societal engagement including increases in civic participation and activism, “allied voices” among informal and formal groups, the coalescence of dissimilar interest groups, actions of opinion leader champions, and positive media attention.

Examples of outcomes
Increased public involvement in an issue
+ Increased level of actions taken by champi ons of an issue
+ Increased voter registration
+ Changes in voting behavior
Increased breadth of partners supporting an issue (e.g., number
   of “unlikely allies” supporting an issue)

+ Increased media coverage (e.g., quantity, prioritization, extent
   of coverage, variety of media "beats,” message echoing)
+ Increased awareness of campaign principles and messages
   among selected groups (e.g., policymakers, general public,
   opinion leaders)
+ Increased visibility of the campaign message (e.g., engagement
   in debate, presence of campaign message in the media)
+ Changes in public will

Examples of strategies to achieve these outcomes
Community organizing
+ Media campaigns
+ Outreach
+ Public/grassroots engagement campaign
+ Voter registration campaign
+ Coalition development
+ Development of trusted messengers and champions
+ Policy analysis and debate
+ Policy impact statements

Objective: Improved policies. Change in the public policy arena occurs in stages—including policy development, policy proposals, demonstration of support (e.g., co-sponsorship), adoption, funding, and implementation. Advocacy and policy evaluation frequently focuses on this area as a measure of success. While and important focus, improved policies are rarely achieved without changes in the preconditions to policy change identified in other outcome categories.

Examples of outcomes
+ Policy development
+ Policy adoption (e.g., ordinance, ballot measure, legislation,
   legally binding agreements)
+ Policy implementation (e.g., equity, adequate funding, other
   resources for implementing policy)
+ Policy enforcement (e.g., holding the line on bedrock legislation)

Examples of strategies to acheive these outcomes
Scientific research
Development of “white papers”
Development of policy proposals
Pilots/demonstration programs
 Educational briefings of legislators

+ Watchdog function

Objective: Changes in impact. Changes in impact are the ultimate and long-term changes in social and physical lives and conditions (i.e., individuals, populations, and physical environments) that motivate policy change efforts. These changes are important to monitor and evaluate when grantmakers and advocacy organizations are partners in social change. Changes in impact are influenced by policy change but typically involve far more strategies, including direct interventions, community support, and personal and family behaviors.

Examples of outcomes
Improved social and physical conditions (e.g., poverty,
   habitat diversity, health, equality, democracy).

No comments:

Post a Comment