President's Council on Sustainable Development

This is historical material, "frozen in time."
The web site is no longer updated and links to external web sites and some internal pages will not work.

President's Council on Sustainable Development - Lessons Learned

Publications | PCSD Home

Lessons Learned
from Collaborative Approaches

NEW NATIONAL OPPORTUNITIES TASK FORCE

Final Draft
April 1997

Table of Contents

Introduction
General Findings
1. Stakeholders often realize significant benefits through collaboration
2. Collaboration is a process that is helping us learn how to solve society's complex problems, and evaluation is a key to learning
3. Evaluations of collaborative efforts are rarely conducted either during or after a project
4. Collaborations are becoming more complex and evaluation processes must adjust accordingly
5. Trust is essential and ownership of the process and outcomes fosters trust

Key Characteristics of Collaborative Processes

1. Characteritics of the Vision and Objectives
Shared Vision and Objectives
Measurable Outcomes
2. Characteristics of the Process
Process is Equally Managed by Stakeholders
Shared and Defined Decision Making Process
Up-Front Planning
Conflict Resolution
Open Communications Among Participants
3. Characteristics of the Participants
Balanced and Inclusive Stakeholder Participation
Strong Leadership
Create Capacity for Stakeholders to Understand Information
Facilitators May Help and Should Apply Similar Tools

Recommended Next Steps
Methodology
Summary of Projects Evaluated
Bibliography
Working Group Members

Introduction

It is increasingly common for businesses, government, citizens and non-governmental organizations to find themselves participating in many collaborative efforts to solve environmental, social, and economic problems. They are doing so because collaborative approaches, it is commonly believed, lead to more comprehensive and acceptable outcomes at reduced cost than traditional regulatory and litigation-oriented approaches. Indeed, the PCSD, which is itself a collaborative process, recommended collaborative approaches to reform the environmental regulatory system, create an alternative regulatory path, and solve community- and ecosystem-based problems. The PCSD remains committed to collaborative strategies.

Yet, the majority of participants in formal collaborative processes know that with this great promise comes great challenges. Success depends on many factors--some common sense, others less obvious, and many not always practiced or universally understood. For this reason, the New National Opportunities Task Force decided to examine some of the lessons that could be learned from a sampling of formal collaborative efforts now underway or recently completed. Specifically, the Task Force wanted to (1) document lessons learned from a diverse sample of collaborative processes; (2) identify when and under what circumstances collaborative approaches are useful and effective; (3) identify characteristics that are essential to successful collaborations and, conversely, the characteristics that cause collaborative processes to falter and fail; and (4) recommend next steps for evaluating collaborations.

The working group reviewed existing academic literature on collaborations, a variety of written project evaluations where they existed, and other published and unpublished background materials for a number of high profile projects. The findings and recommendations presented below are based on these materials and the expertise of the PCSD working group.

Cases Studies Evaluated

See Summary of Projects Evaluated

General Findings

1. Stakeholders often realize significant benefits through collaboration. Although not every problem can be solved through collaboration, people and organizations collaborate because it (1) allows them to advance their own self-interests in ways consistent with others' self-interests, (2) may result in equal or better environmental and social outcomes at lower costs than traditional, more adversarial approaches, (3) creates multi-stakeholder ownership of the process, outcomes and measures of success which can spur positive changes in policy and practice, and (4) may yield comprehensive geographic and sectoral solutions to complex societal problems by helping stakeholders understand each other's needs, recognize the needs of future generations and overcome institutional blind spots caused by narrow organizational missions, and traditional media-, pollutant-, or facility-specific approaches. For these reasons, PCSD remains committed to collaborative strategies.

2. Collaboration is a process that is helping us learn how to solve society's complex problems, and evaluation is a key to learning. Learning can and should take place during and after a collaborative process. During a collaboration, the parties can learn in real-time, track short-term milestones, and take proactive steps to ensure the success of the project or program. After a collaboration, the participants or others can look back to learn whether the overall program or subparts of it met their objectives, and how to improve future efforts. Both forms of evaluation are important, although we need to recognize the difference between proactive, real-time learning and retroactive, after-the-fact learning. In either case, stakeholders should be involved in the evaluation. In some instances, it may be useful to engage an outside expert or organization to help guide, direct and implement the data collection and analysis. If the parties to a collaboration have reached an impasse that causes a breakdown in the collaborative process, it may also be useful to seek the assistance of outside stakeholders to conduct the evaluation.

3. Evaluations of collaborative efforts are rarely conducted either during or after a project. When they have been conducted, they are not usually designed at the outset of the project, with adequate involvement of the diverse stakeholders who have an interest in the project's success or failure, or, as may be useful in some instances, with outside professional expertise to guide and assist participants in creating the evaluation. These shortcomings often lead to recommendations with little, if any, stakeholder buy-in. Because many of the collaborations reviewed are "works in progress" and the available evaluation materials were limited, it is not possible to determine whether the efforts had succeeded or failed. Despite these constraints, the existing evaluation materials are useful for identifying specific features of collaborations that can strengthen the process and results (See Finding 5).

4. Collaborations are becoming more complex and evaluation processes must adjust accordingly. In the past, a partnership was often viewed as a one-time collaboration such as a regulatory negotiation ("reg neg"), where a single party governed the process, many players participated, and the process had a discreet endpoint (e.g., writing a rule). Now references to collaborations often mean something different. The Environmental Protection Agency's (EPA's) Project XL, Common Sense Initiative (CSI), and National Environmental Performance Partnership System (NEPPS) are themselves long term collaborative efforts to create institutional frameworks that foster many simultaneous mini-collaborations and sequential negotiations or projects. For example, the EPA set up the XL framework to foster facility-specific agreements that would deliver more innovative and protective environmental solutions than would be expected under existing regulatory requirements. These new approaches indicate a fundamental change in how society solves its problems and requires evaluations at two levels--the overall framework and individual projects.

5. Trust is essential and ownership of the process and outcomes fosters trust. Stakeholders' trust in each other and in the "process" are the most important determinants of success for most projects. The most important determinant of trust is stakeholder ownership of the process, outcomes, and metrics of success. However, even when trust among stakeholders is difficult to obtain, collaboration can still be successful if the stakeholders trust the process. Although it is difficult to predict how much of an impact a particular problem will have on trust, it is certain that routine breakdowns in trust make it difficult to achieve the original goals of a project.

There are specific "characteristics" of a formal collaboration's objectives, process, and participants that will engender stakeholder trust and ownership. When present, they significantly increase the chances of success. A common theme underlying many of the characteristics is the importance of participants maintaining shared or equal power and influence. Not surprisingly, when collaborations first begin, influence and power are often unevenly distributed among the partners, with some partners having greater control over the process, resources and information. Success in collaboration seems closely linked to the ability of the participants to resolve or overcome these inequities.

Key Characteristics of Collaborative Processes

1. Characteristics of the Vision and Objectives

Shared Vision and Objectives. Dedicating time and energy early in the process to define a shared vision and develop mutually agreeable objectives can be a powerful unifying and motivating force for project ownership. Collaboration works best when there is a clear agreement on what the group intends to achieve. A review of the literature suggests that it is common for projects to suffer from vaguely defined objectives, although most projects examined involved a combination of generally stated environmental and cost-reduction or effectiveness objectives. For example, EPA declared the vision of the CSI to be "cleaner, cheaper, and smarter." It has been difficult for stakeholders to translate this broad view into specific objectives since these stakeholders often disagreed about whether all parts of the objective could or should be advanced as co-equal priorities. In contrast, stakeholders in the Great Printers Project, agreed at the initial project meeting that making pollution prevention a standard business practice among lithographic printers could harmonize economic and environmental objectives. This provided purposeful direction for the discussions which followed. In South Florida, a mutually agreed vision of a healthy ecosystem that coexists with, and mutually supports, a sustainable economy provided a foundation for agreement on the primary objective of restoring more natural flows of water to the Everglades and other habitats. Overall, the case studies suggest that stakeholders should strive to be as specific as possible about their objectives early in the process, within their broadly agreed upon vision. When specificity is missing, stakeholders should be prepared to revisit the objectives later in the process.

Measurable Outcomes. The case studies indicate a general consensus that (1) projects should focus on quantifiable environmental and economic results, (2) agreed upon measures can strengthen ownership, and (3) a monitoring system should allow stakeholders to easily track results. Yet, in many of the case studies, performance measures do not exist. Where they exist, it is uncertain whether they are being used. The lack of specific and agreed-upon measures in Project XL, the CSI and NEPPS undoubtedly is responsible for some of the disagreements that have arisen in these projects. There are some notable exceptions, including the South Florida Ecosystem Restoration project and the CSI Metal Finishing sector's strategic goals project. Note also that measures may be specific to an overall project--the NEPPS system as a whole--or a specific negotiated agreement--between EPA and a state environmental protection agency. Both are important. Despite the general lack of environmental and economic measures, all case studies examined (with the exception of the CSI and Project XL which cannot yet be fully evaluated) indicate a belief that they have had some successes. One of the lesser known benefits of stakeholder's focus on outcomes has been discussions about developing baselines, tracking results, and collecting and reporting data.

2. Characteristics of the Process

Process is Equally Managed by Stakeholders. Shared management and decision making authority is often critical to success. The Great Printers Project (GPP), viewed by stakeholders as a great success, involved a unique triumvirate management structure, where the Council of Great Lakes Governors, the Environmental Defense Fund, and the Printing Industries of America jointly managed the process. When one stakeholder or the convening party is viewed as having more control over the process or outcomes of a project than the others, difficulties sometimes arise. This issue has arisen in the context of EPA's Common Sense Initiative, where some participants perceive that EPA has a disproportionate share of control over the process. Since EPA is the agency responsible for making the regulatory changes that may be proposed under the CSI, the agency has a strong interest in the outcome and may unintentionally exercise disproportionate control of the process.

Shared and Defined Decision Making Process. Clearly defining both the decision rules and stakeholders' roles in decision making early in the process is another critical feature for engendering ownership. When implemented correctly, a "consensus" decision rule is perhaps the most effective way to create ownership because it allows participants to present their views honestly while maintaining sufficient power to protect their interests. Whatever the decision rules, participants should have a role in developing them and share the power of decision making. The more participants have at stake, the more critical it is to define clearly the decision-making processes.

Up-Front Planning. Organizations launching new collaborative efforts need advance plans for budgets and resources. Often, however, there is no such plan. The lack of planning, the reality that collaborative processes require more financial and human resources than many realize, and the routine delays in government procurement mean that it can take months or years to fund critical activities such as facilitation, research, travel, and consultants. The delays can deflate expectations and leave public interest organizations and small businesses without adequate resources to participate in the early stages of the process. The result may jeopardize stakeholder ownership. Public agencies often face these problems when they lead collaborative processes because new initiatives are typically launched quickly and with great fanfare, well after the agency's formal budget planning process for that year has concluded. All of the case studies with direct and important federal participation faced these dilemmas. Budget planning has also affected the foundation community, as it has scrambled to keep up with requests from non-governmental organizations for resources to participate in collaborative efforts.

Conflict Resolution. Conflict resolution has two important features that should also be clearly defined at the beginning of the collaborative process. First, a facilitated process with a professional facilitator or co-management team of stakeholders can avoid or resolve most conflicts before they escalate. Second, because all conflicts cannot be avoided, clearly articulated and collectively developed conflict management ground rules should be established. The GPP used small ad-hoc groups of representative stakeholders to resolve differences. A tiered approach that escalates issues to a higher level group or stakeholder-chosen decision making body may be a useful approach. A tiering approach will only work well, however, if decisions are dispatched quickly. The principal lesson is that when power is skewed, the process may collapse. When power is shared, the potential exists for outstanding results.

Open Communications Among Participants. Open and transparent formal and informal communication is critical for trust and ownership. Information that is withheld, or that is suspected of being withheld, threatens trust. Yet, overloading stakeholders with written material is not helpful and can shut down communication. Participants should periodically identify what information is available and/or needed by each of the participants. The importance of informal communications--by phone, in person--or the exchange of draft documents can be even more important in building trust than formal communication.

3. Characteristics of the Participants

Balanced and Inclusive Stakeholder Participation. Balanced and inclusive stakeholder participation is critical and can be accomplished through a variety of techniques. "Balance" refers to the perspectives and interests that are brought to the collaboration. "Inclusivity" refers to the openness and reach of the invitation to participate. The objective, of course, is to find the "right" stakeholders--stakeholders that have a substantial interest in the issue and/or a role in its resolution. The GPP is an example where an inclusive and committed core group systematically brought in a wide network of local stakeholders throughout the Great Lakes Basin, resulting in a satisfying process. The South Florida Ecosystem Restoration project similarly increased stakeholder participation when Governor Chiles created the "Governor's Commission for a Sustainable South Florida" to complement a federal interagency task force. While inclusive stakeholder involvement is desirable, it also can slow a process down considerably because it is more difficult to satisfy all interests. However, the cases reviewed seem to suggest that the greater the inclusivity at the beginning, the greater the potential for success, in spite of increases in time and energy required to create a more inclusive process. Failure to find creative ways to allow potential stakeholders to participate may cause a project to come to a halt when latent stakeholder issues re-emerge.

Strong Leadership. Strong leadership is also a key to a successful collaborative process. For an organization such as the EPA that launches a new framework for cleaner, cheaper and smarter environmental protection such as the Common Sense Initiative, it is important to designate a leader or champion who provides strategic direction and moral support, secures or helps others secure financial support, and is willing to take risks to resolve an impasse in negotiations quickly and definitively. In South Florida the Governor's Commission and the Interagency Task Force provided this leadership. In launching the NEPPS system, several state environmental commissioners played an important role in overcoming concerns that the NEPPS policy was developed by a small group of state and federal agency personnel. In this situation, the leaders of the effort emerged during the process.

Create Capacity for Stakeholders to Understand Information. Because not all stakeholders come to a collaborative effort with the same knowledge or experience, it is typical for some stakeholders to feel left out of the process. This issue most often arises for citizens, public interest organizations and small businesses. It is important for all stakeholders who need them to have access to adequate information and financial and human resources to help them fully participate. Without this access, stakeholders may conclude they are being mislead or excluded by the other stakeholders. Whether they are actually being misled is largely irrelevant. Their belief usually cannot be overcome by either assertions to the contrary or piles of paper with detailed technical information.

Facilitators May Help and Should Apply Similar Tools. In situations where stakeholders have significantly different perspectives, and do not begin a collaboration trusting one another or the process, a "third party" facilitator without a stake in the issue or debate can make a significant contribution to helping parties communicate and build trust. The key to a facilitator's success is independence and objectivity. Beyond independence and objectivity, successful facilitation involves helping participants establish a common vision and objectives, and develop clear decision making rules and conflict resolution processes at the beginning of the process. The facilitator must also be able to anticipate conflict, help the group to resolve conflict and bring them back to the over-arching vision and objectives. Among a variety of challenges facing a facilitator is the potential for partiality that may arise because he or she has been hired by one of the stakeholders, and may, in turn, consult more frequently with that stakeholder. To overcome this possible impression of impartiality, a facilitator must be sure to consult jointly and equally with all stakeholders.

Recommended Next Steps

Step 1. Create a guidebook based on the "Key Characteristics" identified in this summary report as well as other materials publicly available. Experience and the literature both strongly suggest that the single greatest determinant of success for a collaborative process is the extent to which it has engendered ownership and trust.

Step 2. Existing and future collaborative projects should establish ongoing processes for multi-stakeholder evaluations. The processes should include opportunities to make mid-course corrections and evaluate a project at its completion. The PCSD's search of the literature suggests that there is a shortage of comprehensive evaluations of specific collaborative processes conducted either during or after the collaboration. Even when they do exist, they have generally not been designed at the outset of the collaboration with input from all the stakeholders; nor have they clearly articulated interim milestones, final objectives, or feedback mechanisms to improve the process as it moves forward.

Methodology

To undertake this "lessons learned evaluation," the New National Opportunities Task Force established a multi-stakeholder working group (working group membership is listed further below). Participation on this working group included representatives from business, government and environmental organizations. All decisions were consensus based.

As originally envisioned, the working group's analysis was to include systematic evaluations, including interviews with stakeholders, of a select number of collaborative efforts. Given time and resource constraints, the working group raised concerns that adequate evaluations could not be done properly or in the allotted time. Based on this concern, the working group agreed to search the literature for evaluations of recent collaborative efforts. The literature review focused on recent unpublished or soon to be published studies rather than on the academic literature because most of the collaborative efforts of interest have not yet been completed or have been completed only recently. It will be some time before peer reviewed academic studies are completed. Simultaneously, the working group developed a protocol for conducting interviews with stakeholders and reviewing case materials to the extent that was possible. Both published and unpublished academic literature were used to develop the protocol.

To conduct the review of existing evaluations, working group and task force members nominated dozens of case studies of collaborative processes. Working group members then searched for available written material, developed a set of case selection criteria, screened the cases against those criteria, and developed a shorter list of case studies to examine.

The process of selecting case studies revealed that very few systematic studies of collaborative processes have been conducted. Two exceptions were the "Report of the Interagency Ecosystem Management Task Force" and the "Great Printer's Project" recently completed mid-term evaluation. There was no multi-stakeholder study of the U.S. Environmental Protection Agency's Project XL or Common Sense Initiative, although the Global Environmental Management Initiative had just completed its review of both programs. The materials reviewed varied in quality, comprehensiveness, purpose and possible biases. From these limitations arose the Task Force's initial recommendations to ensure that rigorous evaluations are undertaken in the future and that they are conducted with appropriate stakeholder input throughout a project. Despite the limitations of the available materials, there was information useful to the working group.

Time constraints, the volume of material, and the working group's intent to proceed by consensus raised several challenges in drawing useful and valid conclusions from the materials. The working group agreed to assign each member of the working group with responsibility for reviewing several case studies in depth and for becoming familiar with as many other case studies as possible. Working group members used the draft research protocol as a template for their reviews.

During several day-long, facilitated meetings, the working group systematically reviewed each case study identifying how the case materials addressed the following issues: (1) stakeholder participation, (2) clarity of objectives, (3) decision making processes, (4) conflict resolution processes, (5) communications among participants, (6) available information and resources (technical and financial), (7) outcomes (environmental results and cost-effectiveness) and (8) measurability of the outcomes, (9) indirect benefits, and (10) miscellaneous issues not captured in the other categories. In some cases, working group participants had personal experiences with the case studies and their personal experiences were noted. After lengthy discussions of the case studies the working group was able to reach concensus on overal conclusions about the case studies. These conclusions were presented to and approved by the full Task Force.

Summary of Projects Evaluated

Bibliography

Evaluation Protocol:

"The Ecosystem Approach: Healthy Ecosystems and Sustainable Economies," Vol. I Report ofthe Interagency Ecosystem Management Task Force (June 1995).

Environmental Defense Fund, Environmental Sustainability Kit (1996).

Global Environmental Management Initiative, Research proposal to study "new or existing, positive, flexible, mandatory or voluntary business/government initiatives in the US at the federal level" (1996).

Long, Frederick & Mathew Arnold, The Power of Environmental Partnerships (1995).

Meidinger, Errol, Roger Clarke, and Margaret Shannon, Research protocol from study of "Science and Policy in Natural Resources" (1996).

Case Study Materials:

Administrative Conference of the United States, "An Evaluation of Negotiated Rulemaking at the Environmental Protection Agency, Phase I" (September 1995) (examined woodstoves, asbestos in schools, hazardous waste underground injection, hazardous waste manifest, minor permit modifications under RCRA, coke ovens, fugitive emissions and clean fuels.

"The Ecosystem Approach: Healthy Ecosystems and Sustainable Economies," Vol. I - Overview, Report of the Interagency Ecosystem Management Task Force (June 1995).

"The Ecosystem Approach: Healthy Ecosystems and Sustainable Economies," Vol III - Case Studies, Chapter 7 - South Florida (March, 1996) p. 159. Evaluation of the Great Printers Project, Phase I and II (October 1996), Northwood Consultants.

Global Environmental Management Initiative, "Industry Incentives for Environmental Improvement: Evaluation of U.S. Federal Initiatives" (September 1996), (covers EPA's Project XL and Common Sense Initiative).

Hawkins, David, G. and Chris van Loben Sels, Memoranda to Fred Hansen Deputy Administrator, U.S. Environmental Protection Agency and Keith Laughlin, Associate Director for Sustainable Development, Council on Environmental Quality, (July 1 and July 3, 1996).

Pollution Prevention Pilot Project (4P), Advisory Council Meeting (February 28, 1996)

Pollution Prevention Pilot Project, Project Summary (June 6, 1994).

President's Council on Sustainable Development Eco-efficiency Task Force, Auto Team Report (March 1, 1996).

Steinzor, Rena I. "Regulatory Reinvention and Project X: Does the Emperor Have Any Clothes?" 26 Environmental Law Reporter 10527 (October 1996).

U.S. Environmental Protection Agency, Draft Final Report "Negotiated Rulemaking on Off-Road Vehicle use at Cape Cod National Seashore" (June 1996), Resolve, Inc.

U.S. Environmental Protection Agency, "A Review of the National Environmental Performance Partnership System Activity," (unpublished) U.S. EPA, Region IX.

U.S. Environmental Protection Agency, May 17, 1995, National Environmental Performance Partnership System.

U.S. Environmental Protection Agency, Project XL, 60 Federal Register 27282 (May 23, 1995).

U.S. Environmental Protection Agency, "Project XL Stakeholder Processes - Draft Interim Report of Observation Data" (August, 1996) (unpublished) Resolve, Inc.