How the monitoring program works

NSW OEH Estuaries and Catchment Unit – Report Card Methods

Introduction

This report card gives our estuaries a score for ecological health.  A healthy estuary would have clear water and not too much algae and would be brimming with a wide range of plants and animals.  Ecological health does not refer to other environmental health issues such as drinking water quality, safety for swimming, heavy metal contamination, disease, bacteria, viruses or our ability to harvest shellfish or fish.

To come up with an ecological health score, scientists have used the condition of particular parts of the ecosystem as indicators. These indicators do not have to cover every part of the estuary, but just like the way that your body temperature is an indicator of something wrong with your own health, the indicators show if the whole estuary is out of balance. 

The three indicators are chlorophyll-a levels (the amount of algae), turbidity (the amount of sediment or dirt suspended in the water which measures water clarity) and seagrass depth range (a biological indicator of water clarity).  Sensors are used by scientists to collect information about these indicators. 

Algae

Algae are always present in waterways but if conditions change and are right for algal growth, blooms can occur.  Blooms are likely if there are a lot of nutrients in the water which can come from urban stormwater, fertiliser runoff from farms and gardens and seepage from septic tanks.  When blooms decay, the resulting bacterial activity can reduce dissolved oxygen concentrations in the water column, possibly leading to fish kills.  Chlorophyll-a is a good measure of the amount of algae as all algae have chlorophyll-a in their cells which give them the green colour. 

Chlorophyll a

"Chlorophyll a is a pigment found in photosynthetic organisms.  It is an essential molecule for the process of photosynthesis (the conversion of light energy to chemical energy resulting in the consumption of carbon dioxide and the production of oxygen).  In surface waters, chl a is present in phytoplankton such as cyanobacteria, diatoms and dinoflagellates.  Because chl a occurs in all phytoplankton it is commonly used as a measure of phytoplankton biomass." (Annual technical report Ecosystem health 2006-7)

Sediment

Too much sediment (turbidity) in the water is detrimental to seagrass which require light for growth.  Seagrass is critical for the health of estuaries as it provides essential habitat for water bugs and fish which support bird life and the local tourism and aquaculture industries.  Excess amounts of suspended particles can also smother benthic organisms such as sponges and seagrass, irritate the gills of fish and transport contaminants. The sediment comes from the land and is washed into the waterway when it rains. If land is not properly managed with trees and groundcover, large amounts of sediment can wash into waterways.  Sediment also comes from roads and pathways washing directly into the stormwater and then the estuary.

Turbidity

"Turbidity is the measure of light scattering by suspended particles in the water column, providing an indirect indication of light penetration. (Annual technical report Ecosystem health 2006-7)

Seagrass depth range

Seagrass growth depends on how deep light penetrates through the water and is a biological indicator of water clarity.

Methodology

The vision for our waterways is to maintain or improve their condition in order to protect biological diversity and maintain ecological processes.

The objectives for the Report Card are:

1. To report on ecological health 

2. To track progress on management actions.

These objectives are specifically achieved by:

Providing information to assist in the current and ongoing protection of “high conservation” areas that currently provide substantial water quality and biodiversity benefits to the rivers and estuaries.

Providing information following the remediation of areas that have high pollutant loads and highlight areas that may require further actions

Providing information to help protect against further declines in water quality

The ‘ideal situation’ of NSW estuaries includes; intact aquatic habitats like seagrass, macrophytes and riparian vegetation; minimal algal blooms; and minimal sediment inputs. Implemented management actions will hopefully assist in achieving these outcomes.

There are a number of steps involved in determining the zone score and report card grade which will assist with tracking how well the vision for our waterways and the environmental values are being met, these include:

1. Selecting the indicators

2. Identifying the trigger levels

3. Collecting the data

4. Calculating the zone score

5. Allocating the report card grade

Methodology for report card score development

1. Selecting the indicators

To establish a reporting system that will assist with identifying areas for future remediation and protection the indicators that have been selected have been shown to be responsive to catchment management actions. 

There are many different estuary reporting programs world-wide, these often utilise indicators that are specific to local conditions or problems, but there is also a subset of indicators that are common to many of the programs (Table 1)

Table 1  Indicators used in Estuary Health Studies. 

 

Chl a

Turb

DO

Riparian

SG

Benthos

N15

Qld EHMP

X

X

X

X

X

X

X

Chesapeake

X

X

X

 

X

X

 

Ecohealth

X

X

X

X

 

 

 

Great Lakes

X

X

F

F

F

 

 

NSW MER

X

X

 

 

X

 

 

* Qld EHMP also sample nitrogen and phosphorus concentrations; NSW MER samples fish in a limited number of sites

Chl a – chlorophyll a; Turb – turbidity; DO – dissolved oxygen; Riparian – riparian vegetation; SG – seagrass distribution; Benthos – abundance of benthic animals; N15 – tracking of sewage derived N15, F - future

The NSW MER, based on the findings of Scanes et al. (2007) concluded that measurement of chlorophyll a and turbidity would provide an effective measure of the short term response of estuary health in response to management.  Seagrass and other macrophytes provide a long-term integration of estuary health. 

Chlorophyll and turbidity are the only common components of all projects. While there have been additional indicators used in other Estuary Health Studies, it is not always logistically feasible to use them in all situations and systems. It is recognised and accepted that there is potential for more indicators to be added in the future.

Being common in all the above studies, chlorophyll and turbidity represent the minimum set of indicators suitable for monitoring ecological health over relatively short time spans (1 -2 years).   This makes it suitable for assessing the effects of catchment management actions such as filtering urban stormwater through Water Sensitive Urban Design to reduce total loads of sediments and nutrients reaching the estuary.  This reduction in loads can then be translated into an ecological response.

2. Identifying the trigger levels

Ecological health refers to a system which has normal ranges of diversity and function.  These 'normal' ranges have been established from extensive monitoring of estuaries across NSW as part of the NSW MER program.  To establish these ranges, sites that represent a variety of ecological conditions (from reference sites to highly degraded) have been sampled over a number of years.  The data from the monitoring program have informed the trigger values which are fundamental for determining where a site 'fits' in relation to its ecological condition.

The National Water Quality Management Strategy (ANZECC 2003) suggests that the suitable method for deriving Trigger Value is to determine the 80th percentile value (i.e. the value that is met 80% of the time) for an indicator at reference sites.  A Trigger Value is formally the value which indicates that a variable is outside the “normal range” and could trigger further investigation.  In our context, we have used the Trigger Value to indicate estuarine conditions which are not desirable for continued estuarine health.

A Trigger Value is specific to different types of estuary, so the data for reference sites from the MER program was used to calculate Trigger Values for small coastal lagoons, large coastal lakes and lower, mid and upper zones within river estuaries (Roper et al. 2011; Table 2).

Ecological health does not refer to environmental health issues such as drinking water quality, safety for swimming, heavy metal contamination, disease, bacteria, viruses or our ability to harvest shellfish or fish.

Table 2  Trigger Values for NSW Estuaries (from Roper et al. 2011)

 

Turbidity

Chlorophyll

Lake

6.7

2.5

River estuary (mid)

1.9

2.2

Small lagoon

2.2

1.9

3. Collecting the data

The basic reporting unit for this study is the “Reporting Zone”.  A zone is actually a broad area within the estuary rather than a discrete point and may be represented by a single sample or by multiple samples. 

Samples were collected during the summer – early autumn period.  This represents the part of the year when the highest chlorophyll concentrations are expected. 

At each of the sampling sites, samples were taken in accordance with the NSW MER protocols which are described in full in Roper et al. (2011).  In summary, at each of the “Lake” sites turbidity was measured using a calibrated probe suspended at a depth of 0.5 m for 5 min as the boat drifted or was motored for 5 min (generally covering a distance of at least 300 m).  The probe logged data every second.  The final value for the “site” sampled was the mean of all the logged data.  During the drift, at least 5 samples of the top 1m of the water column were collected and combined in a bucket.  At the end of the drift, a single 200 ml sample for chlorophyll analysis was taken from the composite in the bucket. 

In rivers, an “underway sampler” is used to pass water past the probe whilst the boat travels at speed along a transect upstream from the middle to the upper part of the estuary.  The turbidity is calculated as the mean of logged values for the transect.  At two sites along the transect, composite water samples are collected for chlorophyll analysis. 

Chlorophyll samples were immediately (within 1 hr) filtered under mild vacuum and the filter with retained chlorophyll was frozen until analysis.  Chlorophyll was extracted into acetone and chlorophyll concentration determine by spectrometry.

4. Calculating the zone score

The aim for the calculation of scores is to summarise measured values of all indicators into one value that can then be compared between different reporting zones.

In general terms, we have used two basic calculations to each indicator. These are:

• Non-compliance – are the indicator values non-compliant with the chosen benchmark?

• Distance from the benchmark – how far from the benchmark are the indicator values?

Compliance is commonly used to assess the ecological condition of a waterbody and is central to the analysis of many Report Card style products. The distance measure is a recognition that most guidelines are published as threshold values which only allow for two possible states, compliant and non-compliant. The distance measure provides for more sensitivity for ecological condition along the gradient from good to poor. 

Calculating the non-compliance score

The non-compliance score was simply calculated by taking the number of samples that are outside the 'trigger value' as a proportion of the total number of samples taken in the sampling period. The non-compliance score is then expressed as a value between 0 and 1 with 0 equal to none of the values being non-compliant (ie all compliant) and 1 equal to all values non-compliant.

Non-compliance = number of samples non-compliant with benchmark divided by the total number samples.

Calculating the distance from benchmark score

The distance score has been expressed as a proportion between 0 and 1 so that it is standardised with the non-compliance score. To do that the distance score is expressed as a proportion of the Worst Expected Value (WEV) with a score of 0 equal to the benchmark value and 1 equal to the worst expected value for each of the indicators.  If the WEV is too high, the distance score will always be very low, diluting any difference between reporting zones and between years making comparison difficult. If the WEV is too low, the distance scores will be high, meaning the combined score will also be high. Ideally, the WEV will be just high enough that the majority of the measured values for all of the reporting zones will fall beneath it.

The worst expected value was determined by examination of a data set for all of NSW.  The 98th percentile value was selected as the WEV (Table 3).  This resulted in a value that was close enough to the measured values so that small changes in distance were not swamped by a very large WEV, but was large enough that most values were less than the WEV.  In the small number (2 %) of circumstances where measured values were greater than WEV, then the measured value was replaced with the WEV so that the distance measure became 1 (which is the highest possible value).

Table 3  WEV for Condition Calculations

 

Turbidity WEV

Chlorophyll WEV

Lake

20

30

River (mid)

60

30

Lagoon

20

30

Distance of each non-compliant value = (measured value – trigger value) / (WEV – benchmark value)

The distance score is calculated as the mean distance from the trigger of those values that are non-compliant for the reporting period.

Once the non-compliance and distance score have been calculated, both scores need to be combined to arrive at a single score that can be used to assess the condition of each indicator in that zone. We used the geometric mean of both scores it is more statistically accurate. This is because the distance score is conditional on being above the guideline or non-compliant.

Final Score for indicator =    (non-compliance x distance score)

The final “zone score” for each reporting zone is then the simple average of the indicator scores.

5. Allocating the Report Card Grade

Defining the report card grade is an important step in the development of the Report Card. The grade definitions below are linked to the values and goals outlined above and are structured to allowing easy comparison between each system and over time.  It is important that the cut-off values for each Grade reflect the condition of each Zone in comparison to a broader scale of condition across all NSW estuaries.  This is because we want the “Excellent” grade to really represent an excellent condition for a NSW estuary.  We need to define our own cut-offs (rather than use ones from other report cards) because the scores are strongly affected by the specific trigger values and WEVs used in our calculations, which will differ from those used by others.  To assist with the derivation of cut-offs, scores were calculated for 130 zones across a wide range of NSW estuaries using the same triggers and WEVs as the Great Lakes analyses.  Cut-offs were then defined as representing a percentage of the scores for the state (Table 4).  For example, a zone score less than 0.07 defined the 20% of best zone scores in the state and this became our “Excellent Grade (see Table 5 for other cut-offs).  We did not use a score of 0 as excellent  because, as a consequence of how the trigger values are calculated, we expect that even pristine reference sites will exceed triggers 20% of the time.

Table 4: Percentile Values  (from all NSW scores, n=130) which define cut-offs for grade scores.  Note that higher scores (those approaching 1) represent worse condition than lower scores (those approaching 0).

Percentile

Cut-off zone score

95

0.60

80

0.44

50

0.27

20

0.07

 1  Relationships between Grades, Zone scores and state percentiles

 Office of Environment and Heritage relationship with grades diagram

 Table 5. Report Card results, definitions, descriptions and cut-off values.   and an example of how the final scores might be categorised into each grade.

Grade

Result

Definition

Description

A

Excellent

All environmental values met (The indicators measured meet all of the benchmark values for almost all of the year)

The best 20% of scores in the state

B

Good

Most environmental values met (The indicators measured meet all of the benchmark values for most of the year)

Next 30 % of good scores

C

Fair

Some of the environmental values met (The indicators measured meet some of the benchmark values for some of the year)

Middle 30% of scores

D

Poor

Few of the environmental values met (The indicators measured meet few of the benchmark values for some of the year)

Next 15 % of poorer scores

F

Very Poor

None of the environmental values met (The indicators measured meet none of the benchmark values for almost all of the year)

The worst 5 % of scores in the state

Summary of the process for calculating the zone score

In summary, the process for calculating the zone score involved:

Calculating the proportion of time that the measured values of the indicator are outside the adopted guideline limits (Trigger Values)

  • Calculating the distance/departure from the guidelines for that indicator - defined as the extent that the year’s worth of data extends past the guideline and approaches the worst measured scenario for that indicator Worst Expected Value (WEV).
  • Calculating the geometric mean of the non-compliance and distance scores to get a final score for that indicator for each zone
  • Averaging the scores for the two indicators at each sites – this gives the “Zone Score”
  • Grade the Zone based on the Zone Score as A, B, C, D, F