Methods - Addressing Uncertainty
This section of the Methods addresses uncertainty; it is organized as follows:
Uncertainty and Use of a Scientific Approach
Uncertainty as Handled in this Report
Uncertainty and use of a Scientific Approach
The Framework utilizes two streams of science (sensu Holling 1996). The first stream (reductionist) is a science of parts whereby specific experiments are conducted to assess specific questions and processes that effect specific variables and to address null hypotheses with “either/or” outcomes. Information from this level is used (when data are available) to describe the ecological attributes (i.e., key environmental correlates) and change/trends in the Columbia Basin. The goal of this level of science is to narrow (e.g., using brief time frames and small areas) the focus of the experiments and resulting information to the point that uncertainty is reduced to an acceptable level and that most peers will agree on the resulting conclusions. A problem with using this focused approach within an ecological context is that once a piece (i.e., small area within the basin) is pulled out and studied and null hypotheses accepted or rejected, there is a tendency to extrapolate the findings to the entire basin without integrating findings from adjacent small areas.
The second stream of science used in the Framework is the integration of parts (Holling 1996). Scientific evaluation at this level occurs not by conducting specific experiments but by synthesizing information from unplanned as well as planned interventions in the whole system or by comparing and contrasting extreme examples. Challenges by peers are important at each step of the process (agreement among peers is probably the exception rather than the rule). These challenges are based on multiple lines of evidence (versus experimental results).
The purpose for using the second stream of science in the Framework process is to gain a basic understanding of how the ecological system functions and how it might respond to proposed alternatives. These are difficult to impossible to fully analyze for the entire array of environments in the Columbia Basin using only traditional experimental approaches.
These two streams of science relate across spatial scales (Figure III.D.1). The reductionist approach to science is practiced at the 6-HUC or smaller area. At this scale, environmental attribute data are most often described with a variance term using conventional statistical tools. Due to an initial lack of data (much less variance of data) across the basin, analyses are (at first) deterministic. The holistic approach of the second stream of science integrates information across landscapes, land ownerships, and subbasins, up to the basin or province, and synthesizes information from different sources, experts, and studies, where available. The second stream of science addresses questions and hypotheses that are related to patterns across the basin or province (e.g., how much habitat enhancement is required to improve chinook productivity at the province level by 10 percent?).
The Framework does not solely follow either stream of science. It is conceived and designed to address questions across the hierarchical levels simultaneously, so that knowledge of, and actions pertaining to, each level in the hierarchy is used in context of other levels. Thus data collection and analysis methods are coordinated with levels above and below. The process of coordinating data, analysis rules and language across levels is difficult. Resolving these difficulties takes time and effort to communicate. There are many possible benefits for addressing these difficulties. The benefits of taking an adaptive assessment approach to solve these difficulties (as illustrated in Figure III.D.1) are: (1) Increased system understanding passing through to the lower levels of the hierarchy and (2) Increased statistical rigor (i.e., attention to bias when estimating environmental attributes and to experimental error, sensu Karl et al. 2000) when collecting data at specific sites for testing basin-scale questions and hypotheses.
As the Framework is implemented, uncertainty associated with environmental parameters can be quantified and estimated as variances in their values in specific geographic and ecological contexts. Such estimates can be aggregated (i.e., step up the spatial hierarchy) to address hypotheses formulated during the Framework analyses presented in this report. Understanding of the whole system gained from the first Framework analysis will step down to provide an understanding of how the fuller system might function at the scale of smaller areas (6-HUCs) of the basin. This process of increasing statistical rigor at the basin and province scales, and increasing understanding of the system at the subbasin and 6-HUC scales, actually pertains to many activities that improve learning through time. This is part of adaptive management in the sense of Holling (1978) and Walters (1997).
Adaptive management in the Framework process addresses uncertainty for both levels of science by: (1) Defining questions and goals, (2) Stating working hypotheses via working models that clearly articulate assumptions and predictions, (3) Implementing management actions and research to address uncertainty, by devising management as science experiments, and (4) Monitoring and interpreting the results of management actions (Figure III.D.2). If this process determines that assumptions are met or addressed and world view analyses provide the explicit comparisons that contribute to the decision making process, the process ends with using the answers to the questions to reaffirm or revise current management direction. If monitoring cannot or does not allow assumptions to be adequately addressed or if worldview analyses are not explicit, re-evaluation will be necessary and one should enter the adaptive management evaluation process again (Figure III.D.2). Re-evaluation might involve collection of data to address uncertainty associated with environmental attributes and bio-rules, reformulating questions or modifying worldviews or alternatives. As the Framework process started we, as others (Karieva et al. 2000), realized data was lacking by which to parameterize a stochastic model for the whole basin. As a consequence, our model runs are designed to be deterministic. Results from our deterministic analysis are not meant to appear certain. They are meant to be the basis for formulating hypotheses about how the information from small 6-HUCs can be integrated to answer holistic questions.
In summary, we propose, in this Framework, that scientific uncertainty be addressed by using an adaptive assessment and management process. Uncertainty for both streams of science will be addressed as the Framework is applied so that when change is detected at the local level, it can be related to and understood in the context of the whole, evaluated, and turned into action to evaluate management guidelines designed to maintain or restore desirable ecosystem functions.
Uncertainty as Handled in this Report
The Ecological Work Group of the Framework determined that data were not available by which to parameterize a stochastic model to assess the proposed alternatives at the basin and province levels. The EDT and HCI models were run in a deterministic mode and as such do not explicitly address uncertainty (variability) for the first stream of science (that addresses local conditions. Uncertainty for the first stream of science will be addressed by incorporating more attention to variance and explicitly uncertainty as the Framework process moves forward. Uncertainty associated with the second stream of science – that is, uncertainty over how the entire system as a whole is thought to function – is addressed for each of three Framework alternatives by use of the concept of world views. The three Framework alternatives examined in this report describe three different future visions (based on three different world views) for fish and wildlife management in the Columbia River basin. The analyses of world views pertain to chinook issues and as such do not address wildlife and fish/wildlife analyses conducted for the alternative analyses. The wildlife analyses at this phase of the Framework are intended to be illustrative of how populations and functions could be addressed at the subbasin level.
Inherent in each of the world views is a set of assumptions about the way the world works. Because our knowledge of these assumptions is imperfect, there is uncertainty as to the overall fish and wildlife benefits each alternative may provide.
Our analyses examine the elements of uncertainty and risk by evaluating each alternative and determining the maximum benefit and risk resulting from its implementation. Benefits and risks vary by world view. Maximum benefit is achieved when all of the critical assumptions inherent in the world view represent correct guesses about how the biological systems truly operate and respond to human activities. The maximum risk is the outcome when these same assumptions are all wrong. The purpose of this section is to present an assessment of the uncertainty and risks inherent in the three alternatives. We describe how the alternatives perform under two opposing worldviews, which we refer to as Technology Optimistic and Technology Pessimistic. We present a third worldview, Moderate, which shows an intermediate position that reflects likely outcomes when only a portion of the assumptions inherent in the Technology Optimistic and Technology Pessimistic worldviews actually represent the true State of Nature (Table III.D.1).
In Table III.D.2 we show how each of the alternatives should perform in relation to the worldviews based on the assumptions inherent in each alternative. Each of the three alternatives were analyzed under each of these three world views, resulting in nine analysis outcomes. The range of analysis outcomes represents, in a sense, the spread of expected results under the various world views they represent, that is, the uncertainty of how the world and its biological systems operate.
Alternative 2 and 5 should perform best under the Technology Pessimistic worldview because they depend least on technology for successful performance, and Alternative 6 should perform best under the Technology Optimistic worldview because it depends most on technology for successful performance, where performance refers to future size and trend of chinook salmon populations.
However, our real interest in how we deal with uncertainty lies in how the alternatives perform when we guess incorrectly about how the world and its biological systems operate. For Alternatives 2 and 5, this occurs when we implement either alternative and discover later that the Technology Optimistic worldview was more accurate. For Alternative 6, the worst-case scenario results when we implement the alternative and eventually determine that the Technology Pessimistic view of the world was more accurate. As in life, we would like to choose an option that performs well even when we are wrong about a number of key assumptions.
In the following subsection we present the results of the worldview analysis.
The results of the worldview analysis presented in this section show chinook production potential (abundance) by alternative and worldview, at the basin scale (Figure III.D.3).
· The data presented in III.D.3 show that at the basin level Alternative 2 might be expected to outperform the other alternatives.
· As expected, Alternative 2 performs best under the Technology Pessimistic worldview and poorest under the Technology Optimistic view. However, even under the worst case condition (Technology Optimistic) Alternative 2 produces a larger increase in chinook abundance than any other alternative (within worldviews).
· Alternative 2 produces the highest benefits when it is assumed that juvenile transportation is ineffective, in-river survival rates are low, ocean nearshore survival is high, and hatchery fish fitness and post-release survival are low.
· Because the increase in chinook abundance for Alternative 2, under the worst case scenario is greater than the best-case scenario for the other alternatives, there is less risk and uncertainty associated with the selection of this alternative (at least with regard to producing more chinook). Under the best-case scenario (all assumptions are true) chinook abundance may increase by as much as 381 percent; under the worst case, 164 percent.
· The cost of implementing Alternative 2 has been estimated at ~$765 million a year. In contrast, the cost of implementing Alternative 5 and Alternative 6 has been estimated at $390 million and $210 million, respectively. Thus, to reduce uncertainty to the level shown for Alternative 2, the region must pay an additional $375–$555 million a year (CH2Mhill 2000).
· Note that all of the alternatives, regardless of worldview, increase overall chinook abundance by more than ~107 percent. Therefore, the implementation of any of the alternatives should result in a significant increase in chinook production over current. In the figures below we show that some alternatives achieve the increase through actions that emphasize natural production (Alternative 2); others through the use of hatcheries (Alternatives 5 and 6).
· Alternative 5 improves chinook production potential from 114 percent (Technology Optimistic) to 216 percent (Technology Pessimistic). This alternative performs best when it is assumed that transportation is relatively ineffective, in-river juvenile survival is low, nearshore ocean survival rates are high, and habitat restoration actions in the tributaries are effective. In fact, Alternative 5 requires the largest increase in freshwater habitat productivity of all the alternatives to produce the number of chinook shown in Figure III.D.3.
· Because dams are not removed in Alternative 5, the juvenile transportation program eliminated during the early spring and summer could be revived if research confirmed transportation survival benefits. This flexibility reduces the risk associated with guessing wrong about the transportation assumption (i.e. ineffective).
· Alternative 6 improves chinook performance by 107 percent (Technology Pessimistic) to 122 percent (Technology Optimistic). Alternative 6 is therefore relatively insensitive to the assumptions included in the worldviews. Most of the chinook production increase in this alternative is a result of improvements made in tributary habitat and hatchery fish fitness.
· Alternative 6 performs best when it is assumed that transportation is effective, ocean nearshore survival is low, hatchery fish fitness is high and habitat actions focused on the tributaries are effective in increasing freshwater productivity. In short, Alternative 6 assumes that we have, for the most part, mitigated for hydro impacts through transportation and juvenile bypass facilities, and therefore efforts should now be focused on improving tributary habitat.
· Alternative 6 relies on the least amount of improvement in freshwater habitat productivity to achieve its objectives. Thus, there is less risk associated with Alternative 6 in regards to meeting the habitat goals embedded in the alternative in comparison to the others.
· The transportation program could be eliminated in Alternative 6 if research shows this program to be ineffective. This flexibility reduces the risk of guessing incorrectly about the effectiveness of the transportation program.
· It should be noted that in all of the alternatives it is assumed that the actions were implemented as designed. This means that dams can be removed and habitat can be improved, in some cases dramatically on both public and private lands. There is considerable risk that in the non-modeling world (i.e. real world) that some actions may be politically impossible to implement or, over time, become socially unacceptable. Thus, attempting to implement an alternative that requires significant social change may pose greater risk than one that does not.
The data in Figure III.D.3 show the percent increase over current in chinook production potential for each of the alternatives. For clarity sake, we also present the estimated number of adult chinook produced by alternative and worldview in Figure III.D.4. From the data in Figure III.D.3 we conclude:
· Total chinook production potential is less than 1,000,000 adults for all alternatives under all worldviews.
· Under their respective best case scenarios Alternatives 2, 5, and 6 produce 992,000 728,000, and 755,000 chinook adults, respectively. In the worst case scenarios, Alternative 2, 5 and 6 chinook production decreases to 898,000, 652,000 and 428,000 chinook, respectively. The point to be made is that the difference between the best case (Alternative 2) and worst case (Alternative 6) is approximately 564,000 adults. This defines the maximum reward possible for choosing the right alternative and State of Nature. Because there is good deal of uncertainty around this estimate, it is up to resource managers to decide whether doubling or halving the number would have any impact on the selection of one approach over another.
Although total chinook production may weigh heavily in the selection of a preferred alternative, a second, and probably just as important criterion, in the selection process is each alternative’s reliance on natural versus hatchery production to achieve its objectives. The percent of the total chinook production that natural fish make up for each alternative is shown in Figure III.D.5.
The key points the reader should come away with from the data presented in Figure III.D.5 include:
· Alternative 2 actions result in a Columbia River system that emphasizes natural over hatchery production. The emphasis on natural production poses some risk however as it means that assumptions regarding our ability to improve and recover habitat become more critical. As habitat actions will require many decades to both implement and derive fish survival benefits, the pay-off of as to when the region could see the run sizes depicted for the alternative may be longer than the other alternatives which rely more heavily on hatcheries.
· The approach taken in Alternative 2 (e.g. recover mainstem habitat, increase habitat connectivity, restore ecosystem function) is more consistent with the Council’s Scientific Principles (see Appendix I). If our assumption is that by following these principles the region is much more likely to improve chinook performance, then there is less risk in selecting an approach like Alternative 2 in comparison to the others.
· Some of the actions included in Alternative 2 may not be internally consistent. The alternative emphasizes natural production yet still allows for the continuation of a large hatchery production program. Because there is still considerable debate (uncertainty) as to the impact hatchery fish have on wild stocks, either eliminating or severely curtailing the hatchery program could reduce this risk.
· The implementation of Alternatives 5 and 6 result in a Columbia River system heavily dependent on hatchery production to achieve their respective chinook performance objectives. This is especially true if the Technology Optimistic worldview best represents the correct State of Nature. A decision to place a large emphasis on hatchery production poses significant risk to natural (wild) fish through the mechanisms of competition, disease, genetic introgression and harvest. A major assumption, and therefore risk, inherent in both Alternative 5 and 6 is that the region can maintain a large-scale hatchery program and increase natural chinook abundance through aggressive habitat measures directed at the tributaries.
The level of natural production resulting from each alternative is also important for it has a direct effect on the regions ability to meet ESA requirements. These effects are best seen by looking at the impacts each alternative has on the listed ESU’s (Figure III.D.6). It should be noted that the data shown in Figure III.D.6 are for the Moderate worldview only. These results are sufficient to make the key points presented below.
· Alternative 2 substantially increases chinook abundance, productivity and life-history diversity in all ESU’s. Thus, there is less risk associated with this alternative in regards to recovering listed chinook stocks.
· It is evident from the data in Figure III.D.6 that each of the alternatives provides the least amount of benefit to upper-Columbia River ESU’s (12 and 13). This is especially true for the productivity parameter, which for ESU 12 is actually reduced under Alternatives 5 and 6. These data point to the fact that actions in all of the alternatives have been focused primarily on improving chinook performance in the Snake River (ESU’s 14 and 15). To reduce the extinction risk for stocks originating in the Upper-Columbia River, consideration should be given to implementing more actions in these ESU’s.
· The difference in chinook performance in Alternatives 5 and 6 indicate that performance in some ESU’s (e.g 12 and 13) could be improved by simply shifting habitat actions from public to private land or vice-versa. The poor response for ESU 13 under Alternative 6 is in a large part the result of implementing less effective (lower intensity) habitat actions on private lands in comparison to Alternative 5. To reduce the risk that Alternative 6 may actually reduce productivity in some ESU’s, more thought needs to be given to where habitat actions are implemented (public or private) and at what scale (intensity).
· The large improvement in chinook abundance for ESU 14 under Alternative 2 comes primarily from new spawning habitat created in the Snake River and John Day pool. In contrast, the majority of the chinook production for ESU 14 under Alternatives 5 and 6 results from increased production in the John Day, Deschutes and Umatilla Rivers. Alternative 2 basically assumes that dam removal will create two populations with greater abundance than the Hanford Reach fall chinook population. It will be difficult, if not impossible to test the validity of this assumption without actually removing a project. Removing a smaller dam on a tributary could possibly test this assumption.
To increase the performance of natural fish, each of the alternatives includes actions that improve freshwater habitat conditions. The amount of habitat improvement expected from each alternative is shown in Figure III.D.7.
Key points for Figure III.D.7 include:
· All of the alternatives require a substantial increase in freshwater productivity in order to increase chinook performance throughout the Columbia River Basin. Alternative 5 requires the most improvement in freshwater habitat, while Alternative 6 requires the least. At a minimum the alternatives assume that freshwater habitat productivity can be improved by 35 percent. This is a relatively large improvement that may not be achievable either as a result of social constraints or due to the ineffectiveness of habitat actions.
· The data in Figure III.D.7 should not be interpreted as requiring a 35 percent-66 percent improvement in freshwater habitat in all reaches of the Columbia River Basin. Instead, the correct interpretation is that the alternatives require that we eliminate 35 percent-66 percent of the identified habitat problems. These problems may be as simple as removing a small blockage or as complex as restoring late summer stream flows in a tributary dewatered as a result of agricultural practices. Regardless, under either interpretation there is still considerable risk that this range of improvement in freshwater habitat cannot be achieved. However, the exact scale of the effort, and thus probable success, will not be known until after a Diagnosis has been completed for all of the subbasins. It is envisioned that the Diagnosis would be performed as part of the Assessment and Subbasin Planning phases of the Council’s Framework program.
· Each of the alternatives assume a different level of habitat restoration effort (intensity) dependent on whether this habitat is located on private or public lands. Alternative 2 places equal effort in improving habitat on private (2) and public lands (2). Alternative 5 emphasizes habitat actions on public land (3) over private (2), while Alternative 6 requires the same amount of effort as Alternative 5 does for public lands (3) but significantly less on public lands (1). It is assumed that there is more risk associated with an alternative that requires substantial improvement in habitat on private lands in comparison to an alternative that relies on actions on public lands.
· The Technology Optimistic worldview assumes that less habitat improvement is needed because the quality of the habitat is better than assumed under the other worldviews. The risk in making such an assumption is that regional managers may underestimate the amount of effort (and costs) required to achieve habitat objectives.
In regards to hatcheries, the data in Figure III.D.8 show the overall expected adult return rate for hatchery fish by alternative and worldview. The main points from Figure III.D.8 include:
· The adult return rates in this figure should be considered a long-term average. There will be years when the adult return rate is higher and years when it is lower. Thus, resource managers should not expect to see these adult return rates in every year for every stock.
· All alternatives assume that hatchery fish survival can be improved through the use of innovative culture practices (NATURES, etc.) and that this improvement would result in a ~50 percent increase in hatchery fish survival. Studies underway in the Yakima and other basins should help determine the validity of this assumption in the next three years. For now then, there is considerable risk that these survival benefits cannot be achieved.
· The adult-return rates shown for yearlings under each of the worldviews appear to be very high in relation to the adult return rates observed for chinook juveniles originating from hatcheries located higher in the Columbia River system (Berggren and Basham 2000). There is therefore considerable risk that the adult return rates presented for each worldview may not be achieved on a long-term basis for upper basin stocks. It should be noted that we anticipate that these values will decrease significantly during the Assessment Phase of the Framework process as more information on the Pathogen attribute is developed.
· The smolt-to-adult return rates for Alternative 2 are considerably higher than those estimated for the other alternatives under all worldviews. This results primarily from improvements to mainstem Columbia and Snake River habitat that reduces mortality and decreases the amount of time required for juveniles to migrate from natal streams to the estuary. While the direct survival benefits from actions such as dam removal are relatively certain, increased survival from a decrease in travel-time is not. A major assumption (risk) in Alternative 2 is that there exists a strong flow survival relationship for juvenile migrants.
· Although smolt-to-adult return rates appear high for the Technology Optimistic viewpoint they are consistent with this worldview assumption that the low hatchery fish survival rates observed over the last 10 years reflect poor nearshore ocean survival conditions and that these conditions though cyclical, would improve over time. To realize the best-case scenario for Alternatives 6 requires this assumption to be true.
· The increase in subyearling hatchery fish survival under Alternative 6 in comparison to Alternative 5 for the Moderate and Technology Optimistic worldviews reflect the effect juvenile transportation assumptions have on model results. Alternative 6 maximizes transportation, while in Alternative 5 it is used only when river conditions deteriorate (high temperature, low flow). A major assumption inherent in Alternative 6 is that for subyearling chinook, transportation would provide significant survival benefits in comparison to in-river migration.
· For yearling hatchery chinook, Alternative 5 produces higher smolt-to-adult return rates than Alternative 6. This is due primarily to the transportation assumptions included in Alternative 6. Transportation survival for yearlings is less than ~50 percent for both the Technology Pessimistic and Moderate worldviews. In-river survival under Alternative 5 for juveniles migrating from Lower Granite Dam to below Bonneville Dam can be as high ~55 percent, dependent on time of year. Thus, in-river migration provides a significant survival advantage for upper river hatchery stocks in Alternative 5. If transport survival is high (Technology Optimistic) then the difference in the smolt-to-adult return rates for these alternatives narrows, but for hatchery stocks as a whole, Alternative 5 still has a higher return rate than Alternative 6. The point being that this uncertainty can be reduced significantly for yearling hatchery fish by simply placing production facilities lower in the basin where they would be less affected by the transportation and in-river survival assumptions.
· Placing hatchery facilities lower in the basin would also allow the region to more effectively separate hatchery fish from natural fish, thereby reducing the uncertainty associated with hatchery fish impacts on natural (wild) stocks originating from ESUs 13-15. Such an approach would require defining the purpose of each hatchery (i.e. mitigation, harvest, supplementation) and then defining the areas (provinces) where each type of facility would be allowed.