Here we reflect on the successes and challenges of our effort to involve regional stakeholders in an academic research project. This reflection is partially informed by results from a formal assessment of the WW2100 stakeholder engagement process that wass the subject of an OSU master’s thesis (Ferguson, 2015). Ms. Ferguson conducted semi-structured interviews and a detailed online survey to characterize and assess expectations and outcomes from researchers and stakeholders who participated in the LAN and TAG.
We present this reflection in the context of three goals developed by the broader impacts team, the group of researchers that led stakeholder engagement activities. By involving regional water managers and users in the project, we sought to:
enhance advancement in science by placing a deliberate emphasis on societal relevance and adaptation,
provide a mechanism for regional and “boots-on-the-ground” project review,
support learning and information exchange among water users, managers, scientists, and educators in the Willamette River Basin (WRB).
Goal 1: enhance advancement in science by placing a deliberate emphasis on societal relevance and adaptation
LAN and TAG meetings did lead to meaningful discussions about regional water policy that influenced the scope and direction of the project. For example, a LAN meeting in May 2012 generated a list of specific water management policies that attendees felt were regionally relevant, and that they wanted to see incorporated into WW2100 modeling and scenarios. The research team then considered each suggestion and where possible adapted model design to accommodate the suggestions. The TAG process – where a group of representative stakeholders designed the constraints for two multivariable scenarios – is another example of how stakeholder knowledge informed the research process. The TAG designed the thematic direction and assumptions for the Extreme and Managed scenarios, and in the process also influenced the direction of many of the single variable human dimensions scenarios.
However, there was also tension among participants about the role that stakeholders should play in the research process. Some researchers came from traditions and disciplines that placed a strong emphasis on the value of local knowledge. Others raised concerns about the institutional bias that stakeholders might bring, or had little experience working on teams with non-academic partners. In interviews with researchers and stakeholders, Ferguson (2015) found that the most prevalent challenge mentioned during interviews was diversity of visions for the project. This manifested both within the research team and in the stakeholder engagement aspect of the project, as participants pursued different research or stakeholder engagement goals. Other project challenges mentioned by interviewees included: the logistical difficulties of coordinating so many people, the complexity of the modeling endeavor in and of itself, interpersonal differences among participants, and the challenge of merging research styles from different disciplines.
The technical challenge of modeling such a complex system led to frustration for some LAN and TAG participants. Development of the model took longer than expected, and this reduced the time and project emphasis placed on the evaluation of policy alternatives and adaptation strategies to mitigate future water scarcity. For example, stream water temperature played an important role in early LAN discussions, because it affects regulatory requirements to protect endangered fish. Yet by project year 5, the research team had not been able to incorporate stream temperature into the integrated model in the way they had hoped.
Analysis scale was another source of tension between researcher and stakeholder goals. The complexity of the model meant that much of the analysis and interpretation focused at the scale of the entire Willamette watershed, while stakeholder questions and interest often focused on the more localized scale where their management decisions played out. In TAG meetings, participants suggested that analyses at a sub-basin scale could be one direction for future research.
Despite these challenges, Ferguson’s interviews showed that LAN and TAG participants felt that they had contributed to building the model and that the completion of an integrated water model for the basin was a great success for the project. Her survey also showed that participation was positively correlated with participants’ perceptions of feeling heard (rs = .36, p < .001) and valuing the stakeholder engagement process (rs = .39, p < .001).
Goal 2: provide a mechanism for regional and “boots-on-the-ground” project review
We found that LAN workshops created useful checkpoints where researchers could present progress and receive feedback. This feedback sometimes led to explicit model adjustments, for example, changes in assumptions about farmer planting dates, and careful consideration about how to incorporate Bull Run, the large water source outside the WRB for the city of Portland (Ferguson et. al., 2014). But LAN engagement also sometimes led to frustration, when researchers could not or would not adjust assumptions or model components to match stakeholder experience. This frustration sometimes stemmed from the different perspective of researchers and stakeholders. For example, a sub-group of project researchers developed a model for urban water demand that was guided by the academic literature in economics and, based on this literature, identified water price and household income as the greatest determinants of future residential water demand. But some representatives from urban water providers felt that this model did not adequately represent the influence of water conservation education and technology, or the declines in per capita water demand they had observed in recent decades.
The complexity of Envision and its modeling components also hindered the regional review process. Pre- and post-event surveys showed that participants came away from LAN meetings with a much greater understanding of modeling components (IWW, 2012; Ferguson, 2015). But some participants felt that it was unrealistic for them to provide meaningful feedback within the one-day, large group format of the annual LAN meetings. Researchers responded to these concerns with follow-up correspondence and meetings, by developing workshop summaries, and by creating an online FAQ document (IWW, 2012; IWW, 2014; Ferguson et. al., 2014).
The format of the TAG process also helped address these concerns. By involving fewer participants (25) and more frequent (six roughly monthly) half-day meetings, it provided more opportunities for participants to ask questions and share their diverse perspectives. Questions about the limits of the WW2100 model, understanding of the larger Willamette water system, and what constituted Extreme versus Managed futures, prompted further inquiries and conversations between TAG meetings and related model refinements. These questions also led to what several TAG members described as, for them, among the most educational parts of the TAG process.
One of the benefits of the stakeholder engagement process is that it created awareness about the project among regional water managers. Ferguson’s survey showed that active participants in the WW2100 stakeholder engagement process understood the Willamette Envision model better and found the model to be more useful. In the survey, participation was positively correlated with survey respondents’ perception of the Willamette Envision model’s utility (rs = .32, p = .002) and understanding of the Willamette Envision model (rs = .42, p < .001). Interviewed stakeholders also expressed that through the process they gained an understanding of the model and other water users. They also found the Willamette Envision model more credible than they may have otherwise and implied that they could serve as project ambassadors, sharing the information with their colleagues.
Goal 3: Support learning and information exchange among water users, managers, scientists, and educators in the WRB.
In Ferguson’s analysis, the most frequently mentioned outcome of the stakeholder engagement was that participants were learning and had an overall positive experience. Interviewed stakeholders valued the process for the opportunity to build relationships with diverse water users, regulators, and researchers and to begin a constructive dialogue about planning now for possible water availability constraints in the future. Some example interview comments included, “I would say that the discussions and the relationship-building have been more beneficial to me than the actual nitty-gritty numbers that it produces,” and “I think just even having that dialogue amongst the users was probably one of the most successful parts of the project.”
We believe that two key factors enabled these learning successes. First, the goal to develop an integrated model of the Willamette water system created a forum for sharing different regional and sector perspectives. The topic of anticipating water scarcity drew the interest of individuals in many different water-related sectors, and provided an opportunity for different types of water users to learn about and consider how the water system is interconnected. The project’s broad geographic scale and long modeling horizon also freed participants to think about big picture connections. Second, because this was a research project, led by university partners, it created a neutral forum not tied to specific agency perspectives or a short-term management objective. Many stakeholders commented that they had enhanced their whole basin perspective by engaging in WW2100, and that they had few other professional opportunities to build that perspective.
As educators, we count the learning fostered by the stakeholder engagement process as among the greatest successes of the project, and we hope that it will enhance stewardship of water resources in the basin over the long term. WW2100 was an ambitious biophysical and socioeconomic modeling project that emphasized stakeholder participation as a central way of working. Stakeholder engagement in science is not a new concept. The engagement of researchers and stakeholders in discussions and decision making from project initiation to completion sets WW2100 apart in that the process is itself an experiment. Many examples can be cited on how stakeholder input was incorporated into the model. Examples can also be cited on how stakeholders’ thinking was enhanced by working with the researchers and engaging people from different sectors around a common theme.
The project revealed that there are needs and questions around water scarcity, climate change, people, policies, and land uses beyond the scope of the project. With many of the results and their interpretation coming only in the latter half of the fifth and final year of the project, we simply ran out of resources (time and funding) to address the many uses that researchers and stakeholders see as valuable from the project. A final stakeholders’ workshop served as a forum for elucidating, prioritizing, and discussing a future action plan. At this meeting, stakeholders identified such unfulfilled research needs such as integrating fish life history with a temperature model, new ways to explore and predict urban water demand that might include conservation and different infrastructure pricing, anticipating additional drivers to farming and irrigation practices, more in-depth analysis on the impacts of water policies, and perhaps even the role of changing water storage capacity in the basin.
Our project is among the first to describe and analyze the engagement process from project inception to completion. We were not perfect. However, our research of the process has revealed that engagement in science from inception to completion provides a societally relevant peer-review process, one that stimulates critical thinking on the applications and needs for water science.