Research, Monitoring, and Evaluation
We propose a way to monitor biological outcomes to track progress toward meeting basin and province goals. The role of research in this process is also addressed. The following topics are covered:
Conceptual Overview with Example
Role of Independent Science Advisory Board
Discussion and Elaboration on the Use of Biological Objectives
Conceptual Overview with Example
The portion of the analytical framework pertaining to setting biological objectives (Figure V.1) links the strategies to the vision. When strategies accomplish the biological objectives the vision will be achieved. Biological objectives are described in terms of attributes that characterize the ecological system and that comprise input to the EDT fish model. Output from the EDT fish model describes biological performance, in other words, productivity and abundance of the populations of interest. Unfortunately, because of variability and response delays, it may take several decades before a meaningful change in biological performance is detected empirically (Lichatowich and Cramer 1979).
Fortunately, changes in the environmental attributes often can be detected more quickly than can changes in population response. The reliability of the environmental attributes as indicators of progress toward biological performance objectives depends upon the veracity of the rules used to estimate the latter.
Environmental attributes describe the landscape at various scales of time and space. To the extent that the EDT expert system, as embodied in the rules (Bio-rules), is a valid representation of the relationship between the population of interest and its environment, the environmental attributes predict the biological performance response. A monitoring plan devised to (1) track the status of environmental attributes over time, and (2) determine the veracity of the EDT rules, is therefore a potentially very useful way to assess the degree to which an environmental attribute is moving toward the desired level of the biological objective, in the short term.
The Bio-rules are not site specific. They are applicable across the full landscape and at all levels in the spatial temporal hierarchy. Bio-rules are hypotheses about how the population will respond to its environment (e.g., Figure V.2). Some of these hypotheses are based on results of past studies, and others are based on expert opinions. In some cases, the uncertainty about the rule or hypothesis has an important effect on predicting the expected outcome of an action. Hypotheses that are critical to outcomes, that are uncertain, and that lend themselves to resolution through research, should be considered as priorities for future studies. The Bio-rules are a set of hypotheses that can be tested through research, whereas the environmental attributes—which vary over time and space— are parameters of the ecosystem that could be measured in a monitoring program.
Adaptive monitoring is comprised of three levels: implementation, effectiveness, and validation. These levels are discussed below, relative to the Multi-Species Framework structure and the EDT method.
Implementation monitoring is used to ensure that strategies and treatments are implemented in accord with stated management standards and guidelines. It is used to determine if the basic management directives are correctly followed.
This type of monitoring is used to evaluate the validity of the Bio-rules developed for estimating how a species will respond to changing environmental attributes. In short, this monitoring intends to confirm that the implemented strategy is having the predicted effect on the targeted environmental attribute. If not, then the Bio-rules will need to be changed to better fit the monitoring data. It is also used to determine if mid-course corrections to these strategies are needed due to the ineffectiveness of the strategy, changing environmental conditions, or real-world limitations (local species extinction, etc.).
An additional goal of effectiveness monitoring is to distinguish treatment effects from other environmental variations or perturbations. This is achieved by using two different types of effectiveness monitoring: active and passive. Active monitoring refers to the classic experiment whereby a scientist manipulates environmental variables to determine effects on species performance. In passive monitoring the scientist simply tracks (through measurement or literature) a set of physical or biological variables over time in an attempt to detect major changes in the environment or scientific understanding of how the environment works. Examples of active and passive monitoring would be the testing of a new fish passage facility and the tracking of ocean conditions, respectively.
Validation monitoring is used to confirm that as environmental attributes change, fish and wildlife populations respond as predicted by the EDT and or HCI models. In other words, validation monitoring tests the validity of the basic, underlying scientific assumptions, and tracks trends in population performance measures that imply that goals are being achieved. This might entail statistical trend analyses. Examples of this type of performance measure for salmon include adult returns, juvenile production, and harvest levels. In general, these are the types of data that can be used by policy makers to describe program effectiveness and progress to the general public, and to determine the veracity of the underlying assumptions used in the modeling process.
Role of Independent Science Advisory Board
A Scientific Advisory Board could be very useful to advise on multiple facets of the Council's plan. This board could be responsible for providing advice in the following areas:
1. review and approve the Bio-rules used to predict effects of different strategies on environmental attributes,
2. review and approve the rules used to predict the effects that changes in environmental attributes have on fish and wildlife performance,
3. change these Bio-rules,
4. develop performance measures,
5. select strategies for implementation,
6. evaluate the effectiveness of the implemented strategies,
7. review and approve province plans,
8. change strategies and program direction, and
9. develop and implement the Council's Monitoring and Evaluation program.
Discussion and Elaboration on the Use of Biological Objectives
We propose to define biological objectives based on management goals and on the amount that environmental attribute(s) must be changed to meet the goals. The amount of change required to meet a goal also depends on the set of Bio-rules used in the EDT model. The Bio-rules should be states as testable, explicit hypotheses. The hypotheses are derived from the scientific literature, research studies, and specific analyses using statistical or modeling tools. Examples of such tools potentially include h-VSP developed by the National Marine Fisheries Service, and the models developed within PATH. EDT could be used to evaluate subbasin plans for their contribution to the larger scale (province and basin) vision and biological objectives.
The Council could use EDT to set biological objectives to describe the environmental changes needed within a province to meet the overall goals. Subbasin plans could then detail the strategies and actions needed to make these changes across the province.
The biological objectives could also provide the basis for a regional monitoring and research and evaluation effort. When adopted, the biological objectives could be based on a set of working hypotheses contained within EDT. These could be tested and refined through research and evaluation that could lead to revision of the objectives at some future point. The biological objectives themselves could be measurable attributes that would provide a way to define progress and to track change through a monitoring program.
Our knowledge base about how management actions affect fish and wildlife is, and probably always will be, incomplete. Hence any action plan devised to meet a specified set of goals should be based on a working hypothesis, namely that the actions will result in specific, measurable environmental changes that, in turn, will help meet the goals. The biological rules of the EDT analysis explain the rationale behind the working hypotheses. The EDT analysis translates observable habitat conditions into abundance, productivity, and life history diversity potential for selected target species (e.g. chinook salmon).
EDT characterizes habitat in the basin in regard to some 45 environmental attributes that are described at the 6-HUC level. These attributes are related through Bio-rules (hypotheses) that are used to estimate life-stage survival. Integration of these life stage survivals across the life history trajectory of the target species (e.g., chinook salmon) provides an estimate of the abundance, productivity, and life history diversity of the species as a result of the habitat conditions.
Biological objectives would be based on three characterizations of the environment derived from the EDT hypotheses: (1) the Current Potential, (2) the adopted alternative, and (3) the Historic Potential. These characterizations would be developed at the basin and province scales based on information gathered at the 6-HUC level. The Current Potential describes the abundance, productivity and life history diversity of target species based on the habitat conditions, as they exist and on our knowledge represented in the Bio-rules. The Historic Potential is roughly equivalent to the historical condition and is a depiction of the basin and province based on regional climate, geology, and other parameters without the influence of large-scale human activities. It can be used as a reference point to describe change. The characterization of the adopted alternative will show how current environment would move toward the potential based on a set of management actions. The idea of the biological objectives is to use the environmental attributes to characterize this projected movement of the environment from the Current Potential to that of the adopted alternative, in the direction of the Historical Potential.
Figure
V.3 provides on example of how this movement might be
accomplished. The figure shows three
types of biological objectives that could be developed. The first is a set of attributes describing
aquatic habitats, the second describes terrestrial habitat types, and the third
describes biological performance using the abundance of three fish species as
indicators. EDT would be
used to describe the current state of the parameter (abundance, productivity,
or life history diversity) based on the rules embedded within the EDT model,
the potential for the parameter, and the target value associated with an
alternative (or the adopted alternative).
The biological objective describes the amount of change needed in the
attribute within a province. Change is
measured relative to the current state and in the direction of the potential or
goal.
The figure suggests
several parameters that could form biological objectives. While likely candidates, they are intended
to demonstrate the concept and are not by any means exhaustive or complete. However, we propose that objectives have
explicit criteria. We suggest that they could be:
· readily measurable,
· intuitive,
· capable of providing clear direction to smaller scale planning,
· effective in representing needed conditions,
· integrative across many human activities.
It may be useful to define biological objectives as emergent properties of the ecosystem that would be the target of a monitoring program. They would be qualities that we can readily observe or measure such as temperature, flow, or numbers of fish. Ecological processes and function that are the underlying mechanisms determining emergent properties are the basis for the rules that are contained within EDT. These would be based on research and on other tools such as the NMFS h-VSP analysis or the models developed by PATH and Species-Habitat Project (SHP). The rules (hypotheses) would be tested and refined by research and evaluation. Examples include survival rates, trophic relationships, ecological redundancy, and the like. Many of these are measurable only under controlled circumstances such as those in a research program. Once understood, they would lead to better rules within EDT but would not need to be routinely monitored (there may be exceptions to this such as some survival rates).
Monitoring Aquatic habitats. While all of the attributes used within EDT might be important, it is likely that a smaller subset could be identified that would most influence many other attributes and that would be most influenced by management actions. Figure V.3 shows three candidate attributes: summer low flow, summer high temperature, and the percent fine sediment. The figure summarizes the results of a single alternative (e.g., the adopted alternative) for a single province. The bars show the deviation of the province with regard to each attribute relative to the potential for the province. Those on the left of each group show the current deviation of the province while those on the right show the target situation for an alternative or for the adopted alternative. The arrows refer to the biological objectives: the amount of change in an attribute within a province needed to meet the program goal based on the set of hypotheses within the EDT model.
Monitoring Terrestrial Habitats. Terrestrial habitats and aquatic habitats are often described in different terms, reflecting different scientific traditions and perspectives. While the two sides are converging, differences remain that should be reflected in the biological objectives. Terrestrial biologists generally refer to habitat types that are described on the basis of vegetation coverages rather than on the habitat attributes that are used for aquatic habitats. EDT is being adapted to provide a uniform structure to describe both aquatic and terrestrial habitats. Figure V.3 demonstrates how terrestrial habitat types could be handled in much the same way as the aquatic habitat attributes. Again, the potential, current, and target habitats could be displayed to derive a biological objective as the change in a habitat type (e.g. shrub-steppe) within a province in the direction of the potential.
Monitoring Biological Performance. This refers to measures of abundance, productivity, and diversity of specific species. These are calculated within EDT on the basis of the rules and the habitat conditions. For many, these are the bottom line for performance of the program and are often directly translatable into the values and qualities referred to in the goal for the basin. EDT could be used to set biological objectives for the program in terms of biological performance that could be expected if the other objectives are met and if the rules or hypotheses within the model effectively represent the behavior of the Columbia River ecosystem. In many ways, these biological objectives serve as the final check on how accurately we can depict and predict the behavior of the system. These measures are meaningful only over relatively long time frames and are the result of complex and imperfectly understood ecological processes. Hence, while important, they may be less revealing about program effectiveness in the short term, relative to other parameters such as the habitat attributes.
Some measures of biological performance may be derived using other methods; for example, the responsible federal agencies may define criteria for delisting species listed under the Endangered Species Act. These could be incorporated and used as objectives. It is likely that the same procedure for monitoring and evaluation using EDT as the analytical framework could be used to track progress toward ESA recovery goals.
At the province level, the biological objectives would guide development of finer-scale objectives and strategies (management actions) at the subbasin level. For example, if the biological objective for a province were to reduce average water temperatures in late summer by 25 percent, the job at the subbasin level would be to devise a set of strategies to achieve this in the most biologically effective manner at the lowest cost while keeping with local values and so on. The objectives would not be prescriptive because the Council has no regulatory authority. Instead, the Council could be saying that it would fund actions that are consistent with the goal of changing conditions within a province as described by the biological objectives.
The biological objectives thus provide measurable attributes to track progress. Summer low flow, summer high water temperature, and fine sediments are relatively easily measured. Following adoption of the program, these, and perhaps others, could be monitored to update the EDT analysis. Periodically, a formal updating could occur to define progress. The five-year life of the Council’s Fish and Wildlife program stipulated in the Northwest Power Act provides a natural point to assess progress as well as the need to refine the biological objectives. The abundance of specific fish and wildlife populations (including their status relative to ESA delisting criteria) could be related to the biological objective to track progress as well. The rules within EDT that are the basis for derivation of the biological objectives would be the subject of evaluation efforts following adoption of a program. As better information is developed, the formal updating of the analysis could indicate the need to revise and update the biological objectives.
Several lines of modeling and assessment may prove fruitful in the near-term. Analyses of fish and wildlife interactions could explore the use of Bayesian belief network (BBN) models as an alternative to the HCI species habitat capacity models. The models may be used to assess habitat quality as well as proportional contributions for various species in functional diversity and redundancy analyses. Since USDA Forest Service has developed BBN models for many of the vertebrate species (fish and wildlife) in the basin (Raphael et al, 2001, Marcot et al, 2001), the Framework process could greatly benefit by coordinating modeling efforts with USDA Forest Service. The Ecological Work Group will also focus on applied aspects of the HCI, BBN, and KEF analyses by posing management hypotheses about which ecological functions may be in jeopardy if the wildlife species that perform such a function(s) are not maintained (Cederholm et al. 2000). Hypothesis-testing to assess management activities will be greatly facilitated by current efforts to integrate Framework Strategies with WHR management activities. Such work could quickly link management activities to KEF attributes for fish and wildlife using simple queries in the WHR database. The theoretical basis for the Framework methodology also provides for input of finer scale data to supplement the WHR database as one moves down the landscape hierarchy into the subbasin and watershed scales.