In December 2020, NOAA Fisheries asked the Independent Scientific Advisory Board (ISAB) to review the scientific findings and subsequent dialogue associated with two published papers (Faulkner et al. 2019, Storch et al. 2021) that investigated fish size selectivity in juvenile bypass systems and its implications for estimating and interpreting juvenile salmonid survival.
It has long been observed that juvenile salmonids that encounter multiple juvenile bypass systems during downstream migration return as adults, on average, at a lower rate than those that have fewer bypass encounters. Two, non-mutually exclusive, hypotheses have been put forth to explain this phenomenon: 1) bypass systems impart some sort of damage or stress that results in mortality, but not until the fish have completed passage through the hydropower system; 2) bypass systems select for individuals that are smaller or have other characteristics that result in a survival disadvantage regardless of passage routes at dams. Addressing the issue of effect of passage history on ocean mortality is important because the current management strategy of maximizing spill is designed to route fish away from bypass systems.
Faulkner et al. (2019) sought to investigate whether differences in length between fish using alternative passage routes might help explain differences in associated adult return rates. They found that smaller fish were more likely to enter juvenile bypass systems than larger fish and that smaller fish were less likely to return as adults. They also found that apparent effects of bypass passage on adult returns were diminished or disappeared when fish length was taken into account. In a comment to the journal, Storch et al. (2021) were critical of the data and approach adopted by Faulkner et al. (2019). In addition, the 2019 CSS report (McCann et al. 2019) had an appendix (Appendix G) that was also critical of Faulkner et al. (2019).
The ISAB considered the following review questions:
- Was the Faulkner et al. (2019) analysis scientifically sound, and were the data it used appropriate for addressing the question?
- Were the conclusions drawn by Faulkner et al. supported by their results?
- Does the ISAB have recommendations to improve the analysis?
- Are the criticisms raised by Storch et al. comment and the CSS report appendix valid and supported by the evidence and do any of those criticisms weaken Faulkner et al.’s results or conclusions?
- Was the Faulkner et al. (2021) response to the Storch et al. (2021) comment appropriate and were their criticisms of the Storch et al. methods valid?
- Is PITPH (an index of average cumulative powerhouse passage for groups of fish) an effective index of the powerhouse passage of individual fish, and is it valid to use it to draw causative inferences about effect of powerhouse passage on ocean survival?
One of the key reasons for the disagreements among these papers is a scientific problem known as the ecological fallacy.The ecological fallacy occurs when it is assumed that relationships observed for groups necessarily hold for individuals or vice versa. Faulkner et al. (2019) looked at the effect of fish length on survival within populations; Storch et al. (2021) looked at the effect of fish length on survival across populations. For example, considering salmonids in general, the larger individuals of smolts going to sea are more likely to return than are smaller ones of the same population in a given year. However, the average smolt length has little or no explanatory power for predicting the marine survival of that year’s cohort relative to smolts from other years, and the average marine survival observed among populations is not strongly associated with fish length either.
Faulkner et al. (2019) estimate individual-level effect of length on return probabilities while Storch et al. (2021) estimate the population-level effect of length on return probabilities. Faulkner et al. (2019) then go further and try to interpret why such a relationship between length and bypass probability may occur. Their discussion about possible size-selectivity of bypass structures is interesting but should be considered conjectural and a new hypothesis to test. Similarly, the Faulkner et al. (2019) discussion of the second finding of fish length affecting return probability is of great interest and yet more tenuous than the first finding. The time from recording length to the return of adult fish is now on the scale of years, which includes possible size-dependent mortality (predation), bioenergetics, and involves other habitats (including the ocean). Faulkner et al. (2019) accurately present these as possibilities in the Discussion (which is appropriate in a scientific publication), but the alternatives are not supported by the actual analyses.
Fisheries managers in the Columbia River Basin may be required to make decisions about management actions primarily intended to influence group-level survival (e.g., flow manipulations). In other situations, they face decisions about management actions that are designed primarily to improve within-group survival of individual fish (e.g., local habitat restoration projects). Managers should be cautious about incorrectly assuming that actions that influence population survival will similarly influence survival of individual fish. Likewise, it would be incorrect to assume that improvements that benefit the survival of individual fish will necessarily benefit the survival of the population. Researchers and managers should clearly identify the biological level (e.g., individual, population, metapopulation, community) of observations used in quantitative analyses and the appropriate biological level to which conclusions and recommendations apply.
The reviews raise important questions about the treatment of the data, questions asked, and analytical methods that require a coordinated (with original authors) or third-party comparative approach. Such follow-up analyses would likely add important insights to the data and the relationships of length with bypass encounters and return rates. The effects of the differences between the two analyses can be assessed and even resolved, which would lead to an even stronger set of findings. Without such an effort, the discussion and arguments will remain unresolved and allow for easy mis-interpretation of the results of each analysis. Faulkner et al. (2019) have raised a good set of questions and Storch et al. (2021) have provided a thoughtful response; it would be unfortunate and a missed opportunity not to pursue this further.
In conclusion, the original paper, the comment by Storch et al. (2021) (and Appendix G), and the response by Faulkner et al. (2021) provide an opportunity to make progress on the issue of the role of body length in how the fish use the bypass system and may clarify the effect of length on bypass usage and perhaps, return probability. If there is a size-selection effect on bypass probability, then there may also be an opposite effect on powerhouse passage probability (assuming this is not affected by spill passage probability), which suggests the need for yet another analysis.
