In 1998, the U.S. Congress' Senate-House conference report on the Fiscal Year 1999 Energy and Water Development Appropriations bill directed the Independent Scientific Review Panel to review the fish and wildlife projects, programs, or measures included in federal agency budgets that are reimbursed by the Bonneville Power Administration (Bonneville's "reimbursable" program). The ISRP is to use the same standards and make recommendations as in its review of the projects proposed to implement the Northwest Power and Conservation Council's Columbia River Basin Fish and Wildlife Program. Pursuant to the conference report, in June 2003, the Council requested the ISRP to focus its Fiscal Year 2004 reimbursable review effort on the United States Army Corps of Engineers' Anadromous Fish Evaluation Program (AFEP), which is one component of Bonneville's "reimbursable" program.
The AFEP's main purpose is to produce scientific information to assist the Corps in making engineering, design, and operations decisions for the eight mainstem Columbia and Snake River hydroelectric projects to provide safe, efficient passage of fish through the mainstem migration corridor. Funding for the AFEP is appropriated by Congress, expended by the Corps, and reimbursed by Bonneville. Unlike projects directly funded through the Columbia River Basin Fish and Wildlife Program, AFEP projects have not undergone ISRP review; consequently, this review provides the opportunity to ensure that AFEP proposals receive a similar level of scrutiny for scientific soundness as Fish and Wildlife Program proposals.
This is the first of two reports pertaining to the ISRP's review. In this report, the ISRP provides comments on each of the 52 pre-proposals submitted to meet the research needs for the AFEP in Fiscal Year 2004. The report is intended to aid the Corps in selecting among pre-proposals and assist the project sponsors in drafting final proposals. However, the ISRP comments are limited because the pre-proposals did not provide enough information for a complete technical review. The Council's request specified that the ISRP's technical review be at a pilot scale and the ISRP focus on subsets of AFEP proposals covering the estuary and fish survival at the Columbia River federal hydroelectric projects (project survival). In consultation with Council and Corps staff, the ISRP further defined the subset of project survival proposals as those for the John Day, Ice Harbor, and The Dalles dams. Consequently, the ISRP assigned at least three reviewers to examine the highlighted subsets of proposals. One reviewer, but usually two or more, also reviewed the other proposals so that the ISRP could grasp the full extent of the AFEP.
The review steps to this stage that have included an ISRP or Peer Review Group member are:
- participation in an AFEP overview presentation from the Corps,
- observation of Corps' meetings where one-page statements of needs were developed,
- observation of Study Review Work Group and System Configuration Team meetings for prioritizing statements of needs and proposed studies,
- review of pre-proposals in relation to the statements of needs,
- participation in the Corps' AFEP pre-proposals presentation and discussion meeting, and
- discussion of the pre-proposals with the full ISRP to draft consensus comments.
The ISRP's pre-proposal review is intended to engage the ISRP in the Corps' project selection process so that the ISRP can gather sufficient information to make project specific and programmatic assessments on the substance, scale, scope, and process of the AFEP, and to compare the AFEP with the Council's Fish and Wildlife Program. The timing of the ISRP's pre-proposal reviews was selected to be least disruptive of the Corps' normal, annual cycle of selecting AFEP projects. The AFEP schedule and process of setting priorities and selecting projects are significantly different from that of the Fish and Wildlife Program.
The ISRP will observe several important remaining steps in the AFEP process before its review is finished. These include the Corps' selection of pre-proposals for implementation or further consideration, the development of pre-proposals into statements of work, and the reporting of study results at the annual AFEP research workshop. In recognition of these remaining steps, the ISRP defers making overall programmatic comments and final recommendations on projects until the review is complete. The ISRP will issue a second report by mid-January 2004 that will provide programmatic comments and recommendations on the Anadromous Fish Evaluation Program and project selection process as a whole, and will specifically address the questions described in the section below.
One issue the ISRP intends to fully address in its final report is how to best incorporate scientific peer review in the Corps' process. Most of the pre-proposals reviewed by the ISRP are not well enough developed to be amenable to scientific review and, as written, do not meet ISRP criteria for scientific soundness. Detail on research methods and study variables is particularly lacking in the pre-proposals. For some pre-proposals this lack of detail is because the researchers are analyzing 2003 data, and/or 2004 hydro-operation plans are not finalized. These proposals (e.g. #s 13, 18, 20, 21, 22, 23, 24, 25, 28, etc) contain statements indicating that the 2004 study designs will not be finalized until: "the final set of objectives and hypotheses has been selected", "2003 data are completed", or "until the operational test designs are determined after further discussion this fall". It is understandable that study designs and objectives should be largely based on the most up-to-date data, but the specification of the variables to be tested for effects on survival is essential for the development of a complete, appropriate study plan. The variables chosen for analysis are significant determinants of whether the project will significantly contribute to problem solving, and these variables should be subject to review. Conducting a meaningful technical review without a detailed study design is not possible.
Other pre-proposals are not as constrained by hydro-operations or past years' data (e.g. estuary or avian predation pre-proposals) but still do not provide details that should (and could) be provided to enable scientific review and comparison between competing proposals.
To support a thorough and efficient scientific review, detail on project background and relation to passage problems should be documented and provided along with fully developed proposals. In the next stage of the review, the ISRP will review final study designs or statements of work for many of these proposals. Project details adequate to support scientific review may be provided at this stage. As the ISRP drafts its programmatic report it will address the questions below with specific attention given to whether the AFEP project development and funding process is sequenced so that an ISRP or other independent review can usefully influence the development of proposals, inform policy decisions surrounding the proposals, and ensure program accountability.
