4 Sources of Error

 4.1 The ideal method for compliance monitoring is a procedure that:

• .1 detects organisms in the ballast water discharge;

• .2 has an appropriate limit of detection;

• .3 is precise;

• .4 is accurate;

• .5 is economical;

• .6 is quick;

• .7 can be carried out with minimal technical expertise; and

• .8 can be obtained in all parts of the world.

However, any result obtained would have to include confidence limits based on both the sampling error and analytical error.

4.2 Sources of error include, but are not limited to, errors arising within:

• .1 sampling, including:

  • .1 sample loss (e.g. during filtration);

  • .2 incorrect use of equipment;

  • .3 day-to-day variations in the conditions in which the sampling is taking place; and

  • .4 the experience of the technicians;

• .2 processing the sample, including:

  • .1 incorrect use of equipment;

  • .2 day-to-day variations in the conditions in which the sampling is taking place; and

  • .3 the experience [and fatigue] of the technicians;

• .3 analysis of the sample:

  • .1 incorrect use of equipment;

  • .2 the experience [and fatigue] of the technicians;

  • .3 day-to-day variations in the conditions in which the sampling is taking place;

  • .4 the number of organisms counted. The distribution of organisms in a range of samples usually follows the Poisson distribution and higher numbers of samples give a lower relative variation and sample error;

  • .5 the inherent variation and errors arising from the methods used for analysis. This is especially so when the evaluation of organism numbers in a sample is based on manual counting methods due to human error. For example, although the definition of the minimum dimension of an organism in Guidelines (G2) is quite detailed, analytical results may be influenced by practical issues. These include situations when the size of an organism is determined on a two dimensional microscope, which cannot view the organism "from all perspectives"; and

  • .6 poor harmonization between laboratories and quality control within the laboratory. In the field of chemical analysis, inter-laboratory calibration occurs and is tested. Inter-laboratory calibration of biological samples is also common practice, but the difficulty in the compliance monitoring context is that the viability of the organisms needs to be documented and the viability may be impaired by the mode and duration of sample shipments to different laboratories. Therefore, laboratories should be well managed, and uncertainty limits (the analysis variation) should be calculated for each laboratory. This should be achieved in conjunction with ISO 17025, which provides a standard for the general requirements needed by laboratories to prove they are competent to carry out tests and/or calibrations, including sampling.

4.3 The variation arising from sampling should be added to that from analysis to determine the confidence limits within which the true value of the organism number lies. This has an important bearing on how the result can be used for enforcement of the BWM Convention.

4.4 The sampling uncertainty can be obtained by setting up a null–hypothesis, that is a general or default position that is expected in the results, e.g. the average concentration of organisms is equal to the D-2 standard at a selected level of significance and then the data would be analysed using one of the following tests:

Ideally, an analysis of the distribution should be performed before the data are statistically evaluated.

4.5 There has been much discussion within the IMO on whether the results of the analysis should be averaged to assess compliance or that every result should have to meet the D-2 standard. This is a unique debate at IMO due to the biological nature of the subject matter being analysed, and different States have significantly different views on this issue. Therefore, it will be very difficult to arrive at a conclusion as in the case of non-compliance the results of the analysis are likely to be used in the legal jurisdictions of each IMO Member State, and each of those States may require different evidence to support any enforcement action.

4.6 If the results of detailed analysis are to be averaged, then both the sample variation and the analysis variation need to be calculated and applied to the result. However, some analysis of the sample variation may be needed, as it may be unacceptably high. For example, for five treated ballast water samples, viable organism number results of 9,9,9,9 and 9 will provide the same average as 0,0,0,0 and 45. Both systems would pass the D-2 standard, if averaged; however, the variation is considerably bigger for the second set of results and may prove to be unacceptable because of the one large value.

4.7 If each of the results is treated as an individual value that has to meet the D-2 standard, then again the confidence limits would have to be calculated from the sampling and analytical errors. Here if all results are less than the D-2 standard, then the sampling has proved that the BWMS is meeting the standard.

4.8 The basic difference between instantaneous and average approaches is that the results of the average approach describe the variations of the concentration of organisms during the de-ballasting event, whereas the results of the instantaneous approach describes the variation based on the assumptions of the Poisson distribution. However, the average approach, based on the results of a few samples, has the disadvantage that the variation may be too high, is unacceptable and needs to be improved, which could invalidate the evaluation and lead to inconclusive results.

4.9 The instantaneous approach has the disadvantage that variations in the organism levels at different times of the discharge are not taken into account, which should not be a problem if all the samples meet the D-2 standard. If the discharge is not always under the D-2 standard, the problem can be mitigated by using a flow-integrated sample over set periods of time, which, if taken properly, represents an average of the organisms in the treated ballast water over that time when presented with variance estimates and confidence intervals. This constitutes a better representation of the ballast water quality than separate samples. In addition, a lower variation should be obtained because a larger sample is being analysed. The average approach is likely to have the same disadvantages unless the samples are very large and collected over most of the discharge.

4.10 The differences between applying an instantaneous sampling regime or an average sampling regime to the result are less extreme when taking numerous flow-integrated samples. This is because for each discharge there will be a number of results arising from samples that have been averaged over a specific time.