In early March 2004, the Council requested that the ISRP review the US Geological Survey's proposal, "Estimating the survival of sub-yearling Chinook salmon through Bonneville Dam during two spill operation scenarios using Radio-Telemetry: 2004." This proposal is closely tied to several radio-telemetry studies funded through the US Army Corps of Engineers' Anadromous Fish Evaluation Program (AFEP) and will be implemented through the AFEP; consequently, the ISRP conducted a review of this recently developed proposal in concert with its larger ongoing review of the AFEP. However, to help inform decisions on and further development of this proposal in advance of potential implementation, the ISRP provides this review before its final report on the AFEP review, which is scheduled for release before April 23, 2004.
Recommendation
In general, the proposal is well prepared by competent experienced principal investigators with a team including statistician, John Skalski. Good background is provided from previous spill and survival studies conducted in the Columbia River, and a fair case is made for the need for summer spill survival studies. However, the proposal needs to clearly specify the experimental design that would be subject to the statistical analysis. Specifically, the spill treatments need to be described in greater detail. These issues require a response or proposal revision before the ISRP can make a final determination on the proposal's technical adequacy. The ISRP understands this proposal has been slated for funding, and tags have been purchased. However, the ISRP is also aware this could be a multi-year project, and our comments and request for a response are intended to ensure a strong and clear experimental design for the project from the outset.
Comments
This study overlaps and expands an ongoing study by the USGS at Bonneville Dam, "Estimating the survival of migrant juvenile salmonids through Bonneville Dam using radio-telemetry: 2004 evaluations" (SPE-P-02-1) and is very similar to "Estimate the survival of migrant juvenile salmonids through the Dalles Dam using radio telemetry: 2004 evaluation" (SPE-P-00-8). The present study adds a treatment to the ongoing Bonneville study and focuses on subyearling chinook in contrast to the ongoing study, which includes yearling and subyearling chinook and steelhead.
While the ongoing study does not specify the dates to be covered by the study, this one specifies two periods: from June 20 to July 20 and July 1 to July 31 to be covered by the study.
Under Study Objective 1, the two spill conditions to be tested are clearly spelled out, but the rationale is not given. The details of how spill treatments will be varied per day within each period also need to be given. The rationale for the BiOp spill is a given (75 kcfs day spill and spill to the gas cap at night), but how was the 50 kcfs 24 hr spill decided on? Two other related questions:
(1) Is the spread between spill levels enough? There will only be a difference during the day (of 25 kcfs) because the spill volume to the gas cap at Bonneville at night averages ~ 50 kcfs. Maybe a test between 75 kcfs and 50 kcfs is sufficient but the proposal should include a summary of previous spill survival data to make the case.
(2) Why wasn't a 'spill on vs. spill off' test proposed?
Study Objective 2 is proposed to evaluate the spill test in Objective 1 for two periods, June 20 - July 20 and July 1 - July 31. It is clear that this objective is, in part, a feasibility study to determine if a radio-tag study can be done for sub-yearling chinook nearing the later summer when water temperatures are warming. This is a worthwhile goal. The overlap in periods to be evaluated is not adequately explained. The third set of estimates, generated for July 20- July 31 should also be better explained. In sum, the treatment alternating between these two or three overlapping periods should be clearly articulated.
The principal investigators should consider separating the analysis into four blocks, two times of day and two spill levels. The proposal makes no mention of this. This block design would extract more information and probably would require no more fish. They need to be able to separate day versus night as well as different spill levels.
The following comments apply to this proposal and the other two related radio-telemetry studies at Bonneville and the Dalles Dam.
We would like to see less emphasis on "hypothesis testing." The authors tend to lean toward "estimation" but we would suggest almost exclusive emphasis on point estimation with associated measures of precision and accuracy leading to confidence intervals with high probability (power) that the half-width is no more than the size of the effect to be detected.
Methods for estimation of required sample sizes should be given or referenced, e.g. it is unclear if one or two sample procedures are being used in the Section "Sample Size for the RSSM analysis." See, e.g. Zar, J.H. 1999. Biostatistical Analysis, Fourth Edition. Prentice Hall, NY. (Page 106 and page 132).
John Skalski presented an overview of release-recapture methods for estimation of survival of juveniles at the Portland District Corps of Engineers survival studies meeting held December 11, 2003. If we understood his presentation correctly, there was discussion of non-statistical bias in estimation of the route specific survivals with the design proposed in this study, leading him to suggest a design based on three releases with one release at the dam into a well defined route, e.g. ice and trash sluiceway. Apparently, this issue is to be investigated under Objective 2 of the proposal SPE-P-00-8 "Estimate the survival of migrant juvenile salmonids through the Dalles Dam using radio telemetry: 2004 evaluation" and Objective 1 of SPE-P-02-1. The issues should be discussed in the present proposal. Is the non-statistical bias of sufficient magnitude to require the three-release design? We assume the potential overlap in the studies will be eliminated.
Point estimates for parameters of interest are complicated ratios and ratios of ratios that typically have unknown statistical biases associated with them and no explicit formulas for variances. The formulas for reconstruction of dam survival and other parameters are more complex, e.g. some involve linear combinations of products of these ratios. Apparently, variances for dam survival and route specific survival estimates are approximated by the "delta" method with unknown accuracy, precision, and statistical bias. We believe that the properties of these approximations (including properties of formulas for target sample sizes) should be compared with bootstrapping of real data and Monte Carlo simulations before data from this project are analyzed and the next round of survival studies are designed.
Minor comment. The standard errors listed in Table 1 are best guesses based on the models, assumptions, and planned sample sizes. They are not guaranteed.
A final programmatic comment. Summer spill is currently a key issue, and this study starts to address some of the questions of the benefit of summer spill by providing real data, which provides rationale to move forward with this project. However, a primary issue the ISRP has identified in the review of the AFEP is the need for a long-term plan for fish studies related to the hydrosystem with proposals developed as far in advance of implementation as feasible to allow for adequate planning, and proposal development and review.
Review of response (ISRP 2004-5a)
In the above review, the ISRP found the proposal to be well prepared by competent experienced principal investigators. Good background information was provided from previous spill and survival studies conducted in the Columbia River, and a fair case was made for the need for summer spill survival studies. However, the ISRP commented that to make a final determination on the proposal's technical adequacy, a response was needed. The ISRP recommended that the proposal specify the experimental design and clarify some issues relative to the statistical analysis. Specifically, the spill treatments needed to be described in greater detail.
On April 21, a response was provided. Based on the proposal and response, we find the proposed experimental design to be sound. Regarding Dr. John Skalski's response on the design and statistical analyses, the ISRP was comfortable with his response to our specific questions about adequacy of the "two release" design and proposed statistical analyses. His approach to analyze the indices on juvenile survival using profile likelihood methods is not necessarily the approach reviewers would take, but the methods are sound and he provided a reference to a BPA report where he has reported some of the technical comparisons that the ISRP suggested were necessary.
In summary, the response adequately addressed the ISRP concerns and should be made part of the project proposal. However, based on the recent low flow projections, the ISRP wonders whether the study design will remain the same (i.e. spill conditions/treatments), and if the study is feasible at all. The ISRP should be kept informed of the project's status, and if the study design does substantially change, further ISRP review may be warranted.
