The ISAB commends the Council for commissioning the summary report Mainstem Passage Strategies in the Columbia River System: Transportation, Spill, and Flow Augmentation by A. Giorgi, M. Miller, and J. Stevenson of BioAnalysts, Inc. (Giorgi et al. 2002). The Giorgi et al. report was created for the Council as a background document in the context of the mainstem program amendments. The Giorgi et al. report summarizes existing information on three topics that the ISAB and its predecessor advisory groups have debated intensely (and advised the Council about) over the past decade or so.
The Council asked the ISAB to evaluate whether the Giorgi et al. report captures the full spectrum of scientific issues and knowledge related to mainstem passage, and specifically whether it adequately addresses the following questions:
- What does the scientific literature tell us regarding the benefits, shortcomings, or risks associated with each passage strategy as compared to other passage options?
- Which aspects of the scientific information are in dispute?
- What are the critical uncertainties attending each strategy?
- What is being, or could be done to reduce uncertainty and disputes?
In the context of the public comments, the Council asked how serious are the disputes raised in the public comments, and whether the various opinions are really very divergent.
The summary document by Giorgi et al. has helped update the ISAB's understanding of the issues, and we are confident that it will be useful to the Council and Council staff. The comments help round out the outstanding issues.
General Report Evaluation
Giorgi, Miller, and Stevenson have done a good job of reviewing mainstem passage strategies designed to improve survival of juvenile salmonids in the Columbia River system. Their focus is on review of key studies and analyses that have taken place under contemporary river operations, which were initiated in the early 1990s and formalized as guidelines in the 1995 and 2000 Federal Columbia River Power System Biological Opinions (BO). Their approach was to abstract the key findings of the studies and compare the results. We believe they have largely succeeded in this task, and the resulting report provides information that ought to form the basis of revisions in the Council's Fish and Wildlife Program, as well as guides for upcoming Mainstem and Systemwide project solicitations.
With respect to the ISAB's advisory role for both the Council and the National Marine Fisheries Service, we emphasize that there are many uncertainties in the information used as a basis for the hydropower mitigation actions in the FWP and for the Reasonable and Prudent Alternatives (RPAs) of the Biological Opinion. A concerted effort is needed to reduce the uncertainties. The Giorgi et al. report helps all parties to focus on these uncertainties.
The information reviewed by the Giorgi et al. report was summarized objectively. Findings and conclusions were specified where appropriate based on the scientific information available. In cases where the results were open to question, these were identified as critical uncertainties and "research opportunities" (often a euphemism for badly needed information). These uncertainties and research opportunities provide a valuable basis for soliciting new work under the Fish and Wildlife Program. The report cites important scientific literature that should be the foundation for new proposals.
Overall, the report was very conservative in drawing statistical conclusions. This fact needs to be understood for proper interpretation of the implications of the report. From a purely statistical standpoint, tests that fail to show statistical significance in data can be definitive in stating that no effect was found, yet these tests do not definitively prove the absence of effect. The report's statistical conservatism affects its value as a synthesis of current understanding. The report is not an "analysis" of the issues, but a compendium of findings to date. The ISAB considers this to be an important distinction. For example, when the report says no significant effect has been detected, this is technically a correct summary of the statistical analyses reported in the individual studies that were reviewed, but it should not be taken to preclude the possibility that an effect exists. The possibility remains that new studies or further rigorous analyses of existing data (in particular, analyses that combine data from several separate studies) would generate statistically significant results or a different answer. Future analyses should put more emphasis on statistical parameter estimation and power analysis, rather than just hypothesis testing.
General Evaluation of Comments
The comments from the public and agencies provided additional useful technical information, usually amplifying points made in the Giorgi et al. report, but often, it is our impression, from more of an advocacy position for certain management strategies. Contrary to what was suggested in some of the comments, we did not see any pervasive tendency for selective use of data or misinterpretation of results by authors of the Giorgi et al. report. Genuine technical disagreements occur over possible interpretations of available information for methods of analysis and implementation measures. Giorgi et al. were criticized for not providing the full range of scientific "opinion" on the key issues, yet the intent of the report was to summarize information, not opinion. However, as we noted ourselves above, the report does not summarize alternative hypotheses that have not yet been tested. To the extent that the commentors were seeking more attention to these alternative hypotheses, we concur. The comments can be used, along with the report, to frame the uncertainties and opportunities for additional research.
Conclusions
It is clear from both the Giorgi et al. report and the comments that the mainstem management issues tackled by the report (transportation, spill, and flow augmentation) are not yet resolved, although there is little or no dispute on many component topics (see appendix). Lack of resolution is largely due to a lack of critical information; further resolution requires additional research, experimental manipulation, adaptive management, monitoring, and analysis (evaluation).
