1 Safety Assessment Process
1.1 Application
1.1.1 Safety assessment provides for a systematic examination of the craft functions and
craft systems associated with the safe performance of these functions. A safety
assessment should be conducted for each craft before entry into service.
1.1.2 For craft of the same design and having the same equipment, one safety assessment
for the lead craft will be sufficient but each of the craft should be subject to the
same trial programme.
1.1.3 If in the course of the service life of the craft changes are made to the design
or operation of the craft or its systems, the effect of these changes on the results of
the safety assessment should be examined, documented and reported to the Administration.
1.1.4 The safety assessment should be conducted for the craft itself and for systems
installed on the craft. The systems considered should include, but not be limited to:
-
.1 propulsion system;
-
.2 electrical system;
-
.3 auxiliary systems;
-
.4 control systems, including directional, altitude and trim control; and
-
.5 navigational equipment.
1.2 Assessment team
1.2.1 An assessment team should be established and should include the builder or
designer, experts having the necessary knowledge and experience in the design and/or
operation for the specific evaluation at hand, and a safety engineer familiar with the
different steps of the assessment process. Other members may include craft operators,
equipment manufacturers and human factor experts.
1.2.2 The level of expertise and experience that individuals should have to participate
in the team will vary depending on the system complexity and the type of analysis being
performed.
1.3 Assessment process
1.3.1 General
1.3.1.1 The basic principles described below are based on established procedures outside
the marine industry.footnote They provide the methods to evaluate the craft functions
and the design of systems performing these functions. The safety assessment process
should ensure that all relevant failure conditions are identified and that all
significant combinations of failures, which could cause those failures conditions, are
taken into account.
1.3.1.2 The safety assessment is conducted in parallel with the design and construction
of the craft. Accordingly, three phases may be distinguished:
-
.1 Generation of requirements
Depending on the criticality of functional failures at craft and system level,
safety objectives are assigned to the various failure conditions identified. These
safety objectives are expressed as probability levels and probability budgets that
should be met by the implemented system, item and hardware/software configuration.
-
.2 Design implementation
During implementation account should be taken of the failure rate budgets assigned
to hardware and software items.
-
.3 Verification
In the verification phase it should be demonstrated that the hardware and software
actually implemented meet the relevant safety requirements.
1.3.1.3 Different processes are employed in the phases of the development cycle as
illustrated in figure 1:
-
.1 Functional Hazard Assessment (FHA) in the concept development stage;
-
.2 Preliminary System Safety Assessment (PSSA) during the design phase; and
-
.3 System Safety Assessment (SSA) in the verification phase.
1.3.1.4 There is likely to be some overlap between the phases and the assessment process
is iterative in nature. Individual activities will hence be revisited as the design
evolves and becomes more defined.
1.3.1.5 The Functional Hazard Assessment (FHA) is conducted at the beginning of the
development cycle. It should clearly identify and classify failure conditions associated
with the craft's functions. These failure condition classifications establish the safety
objectives. In table 1, the failure condition classifications (category of effect) are
related to the safety objectives, expressed as levels of probability. The output of the
FHA forms the starting point for the Preliminary System Safety Assessment (PSSA).
1.3.1.6 The Preliminary System Safety Assessment (PSSA) is a systematic analysis of the
proposed system architecture. Its purpose is to show how failures at a lower
hierarchical level can lead to the functional hazards identified in the FHA. The PSSA
should provide the designer with all necessary safety requirements of the system and
demonstrate that the proposed architecture can meet the safety objectives identified by
the FHA.
1.3.1.7 The PSSA is an interactive process and conducted at different development
stages. At the lowest level, the PSSA determines the safety related design requirements
of hardware and software. The PSSA usually takes the form of a Fault Tree Analysis
(Dependence Diagram and Markov Analysis may also be used). It should also address safety
issues arising from common cause considerations.
1.3.1.8 The System Safety Assessment (SSA) is a systematic assessment of the actual
system to demonstrate that safety objectives from the FHA and derived safety
requirements from the PSSA are actually met. The SSA is usually based on the PSSA Fault
Tree Analysis.
1.3.1.9 Activities typically performed in the FHA, PSSA and SSA are described below
under the respective headings.
1.3.2 Functional Hazard Assessment (FHA)
1.3.2.1 Scope of analysis
1.3.2.1.1 The scope of a safety assessment varies depending on factors such as system
complexity, level of service experience, and criticality of system failures. Before
starting a detailed analysis of system failures it is therefore necessary to make a
preliminary assessment in order to establish the required depth of analysis.
1.3.2.1.2 An FHA is performed at two levels, i.e. at craft level and at system level.
1.3.2.1.3 The craft level FHA is a high level, qualitative assessment of the basic
functions of the craft. A craft level FHA should identify and classify the failure
conditions associated with the craft level functions. The classification of these
failure conditions establishes the safety objectives that a craft should meet (see table
1).
1.3.2.1.4 The system level FHA is also a qualitative assessment which is iterative in
nature and becomes more defined as the development progresses. It considers a failure or
combination of failures that affect a craft function. Lower level hardware or software
items are not assessed in the system level FHA.
1.3.2.1.5 The output of the craft level FHA is the starting point for craft level fault
trees, while the system level FHA is used to generate top level events for PSSA fault
trees. In both cases the fault trees (Dependence Diagrams may also be used) can be used
to derive lower level safety provisions.
1.3.2.2 Procedures for FHA
1.3.2.2.1 FHA carried out at craft and at system level use the same principles. The FHA
process is a top down approach for identifying functional failure conditions and
assessing their effects. This assessment is made following the steps listed below. A
description of the FHA method following these steps is provided in section 2 (Safety
assessment methods).
-
.1 identification of all craft and system functions;
-
.2 identification and description of failure conditions associated with these
functions;
-
.3 determination of the effects of the failure condition;
-
.4 classification of failure condition effects;
-
.5 assignment of safety objectives/probability requirements; and
-
.6 identification of means of compliance.
1.3.2.2.2 The analysis should take account of the environmental conditions the craft is
likely to encounter en route.
1.3.2.2.3 During the execution of the FHA a list should be compiled describing:
-
.1 the craft configuration following the loss of systems examined in the FHA;
-
.2 resulting operational limitations; and
-
.3 the action required of the crew.
1.3.2.2.4 The results of the FHA should be documented following the format given in
section 1.4.1 below. They represent the input data and information for the PSSA process.
1.3.3 Preliminary System Safety Assessment (PSSA)
1.3.3.1 Scope of analysis
1.3.3.1.1 For each significant failure condition identified in the FHA, a PSSA should be
performed. Significant failure conditions are those classified as catastrophic,
hazardous or major. Catastrophic and hazardous failure conditions should be subject to a
qualitative and quantitative analysis. For failure conditions identified as major a less
thorough qualitative analysis is sufficient when the systems are not complex or when
relevant service experience is available.
1.3.3.1.2 The PSSA process examines the proposed system architecture with a view to
identifying individual failures and combinations of failures that can cause the
functional hazards identified by the FHA. The main purpose of the PSSA is to determine
whether the chosen design can meet the safety objectives identified by the FHA and to
derive safety requirements for systems and equipment associated with the function under
consideration. The PSSA process is iterative in nature and continuous throughout the
design cycle.
1.3.3.1.3 Since each significant failure condition should be analysed by a PSSA, there
are likely to be several PSSAs performed for a craft.
1.3.3.2 Procedures for PSSA
1.3.3.2.1 The PSSA process should identify the sequence of events resulting from
individual failures or combinations of failures that can lead to the functional hazards
identified by the FHA. It should also show how the FHA requirements can be satisfied by
the chosen design. The process uses a top-down approach that seeks to identify all basic
events that contribute to the functional hazards.
1.3.3.2.2 The assessment draws on established risk assessment methods of which the
following should be applied in the PSSA process:
-
.1 Fault Tree Analysis (FTA) or Dependence Diagrams (DD);
-
.2 Failure Modes Effect and Criticality Analysis (FMECA);
-
.3 Failure Modes and Effects Summary (FMES); and
-
.4 Zonal Hazard Analysis (ZHA).
More information on these methods is provided in section 2.
1.3.3.2.3 FTA, or DD, is a top-down approach that allows the logical representation of
many basic events (e.g. failure modes from FMECA) that combine to produce events at
higher levels (e.g. failure conditions from FMES, ZHA or FHA). Its main purpose is to
derive safety requirements for the basic events.
1.3.3.2.4 An FMECA provides for a systematic examination of potential failure modes of
equipment. It seeks to identify causes, analyse effects on system operation, quantify
occurrence probabilities (failure rates), and identify corrective actions, i.e. design
modifications.
1.3.3.2.5 The FMES summarizes lower level failure modes with the same effect derived
from previously performed FMECAs.
1.3.3.2.6 The objective of a ZHA is to identify potential areas of risk arising from the
design of the installation (segregation, separation, protection, etc.) and the operation
(maintenance tasks, etc.).
1.3.3.2.7 The results of the PSSA should be documented following the format given in
section 1.4.2 below. The outputs from the PSSA are the inputs for the SSA process.
1.3.4 System Safety Assessment (SSA)
1.3.4.1 Scope of analysis
The SSA is the final step in the assessment process. It integrates results of the
previously performed FHA, PSSA and flight/performance tests. While FHA and PSSA are used
during the design process to derive safety requirements, an SSA is a verification tool
to show that the implemented design satisfies the requirements established by the FHA
and PSSA.
1.3.4.2 Procedures for SSA
1.3.4.2.1 For each PSSA there should be a corresponding SSA. The verification process
should be supported by data sheets for which an example is shown in table 2. In these
data sheets requirements for specific failure conditions generated in the FHA and PSSA
process are correlated with the results obtained in the SSA for the implemented design.
1.3.4.2.2 The results of the SSA should be documented following the format given in
section 1.4.3 below. The "inspection activities" referred to in the documentation
(section 1.4.3 below) relate to scope and intervals of safety related checks to be
performed by the operator and the Administration during the service life of the craft.
The documentation should include those activities (regular checks by the crew,
maintenance tasks, inspections) necessary to satisfy the safety requirements established
by the PSSA.
1.3.4.2.3 Means of verification include tests, analysis, demonstration and inspection.
1.4 Documentation
The results of the Safety Assessment should be documented in a report addressing the
three main elements of the assessment process: Functional Hazard Assessment, Preliminary
System Safety Assessment and System Safety Assessment. The report should provide the
following information so that there is traceability of the steps taken in developing the
analysis.
1.4.1 Functional Hazard Assessment (FHA)
1.4.2 Preliminary System Safety Assessment (PSSA)
-
.1 Planned compliance method with FHA requirements.
-
.2 List of failure conditions for further analysis.
-
.3 Fault trees or Dependence Diagrams.
-
.4 Lower level safety requirements.
-
.5 Updated list of verification methods and requirements.
-
.6 Operational requirements (maintenance tasks, checks, etc.).
1.4.3 System Safety Assessment (SSA)
-
.1 Updated failure condition list, including classifications.
-
.2 Fault trees or Dependence Diagrams showing compliance with safety requirements.
-
.3 Documentation showing how requirements for the design of the system items
installation (segregation, separation, protection, etc.) have been incorporated.
-
.4 Verification that safety requirements from the PSSA are incorporated into the
design and/or testing process.
-
.5 Results of the non-analytic verification process, for example tests,
simulations, demonstrations, inspection activities.
2 Safety Assessment Methods
The assessment process described in section 1 employs a number of standard risk
assessment techniques. The present section provides some guidance on how the different
types of analysis should be applied to WIG craft systems. As all methods are well
established, further background information can readily be found in the literature.
2.1 Functional Hazard Assessment (FHA)
2.1.1 The starting point for a FHA is a comprehensive description of the craft and its
systems. This includes a complete breakdown of all systems and subsystems. The FHA is a
function driven process that can be performed at an early design stage where system
knowledge is still incomplete and subject to change.
2.1.2 The FHA comprises six main steps as outlined below:
-
.1 Identification of all craft and system functions
A function list is created at craft and at system level, taking into account both
internal and external functions. Table 3 gives an example of an input function
data sheet.
-
.2 Identification and description of failure conditions associated with these
functions
Multiple failures should be considered, especially when the effect of a certain
failure depends on the availability of another system. Failure conditions to be
considered include:
-
.1 loss of function (detected/undetected);
-
.2 malfunction (detected/undetected);
-
.3 incorrect function;
-
.4 reduced performance;
-
.5 interrupted function; and
-
.6 inadvertent function.
-
.3 Determination of the effects of failure conditions
Failure conditions should be examined with respect to their effect at craft and
system level and with respect to the effect on the crew, occupants and the
environment. All operational modes, environmental conditions and emergency/
abnormal situations should be taken into account when evaluating the effect of
failure conditions. If effects cannot be determined by the analyst, the associated
failure condition should be further examined using simulation techniques, model
tests or full scale tests.
-
.4 Classification of failure condition effects
The effect of failure conditions is classified according to the following
categories: catastrophic, hazardous, major, minor, no safety effect (see section 2
of chapter 1 and table 1). Material used to support the classification should be
documented. The need for further supporting material (e.g. simulations or tests)
should be identified.
-
.5 Assignment of safety objectives/probability requirements
For each failure condition probability requirements (see table 1) and qualitative
design requirements should be assigned and documented. The design requirements may
relate to the craft, systems and items.
-
.6 Identification of means of compliance
For each failure condition, the measures foreseen to comply with the safety
objectives should be identified and documented.
2.2 Failure Modes Effect and Criticality Analysis (FMECA)
2.2.1 An FMECA is performed for components or items that contribute to functional
failures identified as hazardous or catastrophic. These are, for example, parts
associated with basic events in fault trees. Procedures for FMECA are documented in the
literature.footnote The level of detail should correspond to the level of
indenture in the system hierarchy at which functional failures are postulated. The
analysis is an iterative process that evolves as the design becomes more defined.
2.2.2 The FMECA process is facilitated by worksheets as shown in table 5. An important
aspect of an FMECA is concerned with obtaining reliable data for failure mode rates
under similar environmental and operational conditions to those envisaged for the system
being analysed. Failure rate data may be obtained from handbooks in the public
domain,footnote from industry sources or by computational methods.
2.3 Failure Modes and Effects Summary (FMES)
The FMES summarizes all failure modes with the same effect from previously performed
FMECAs. Its purpose is to combine into a single event all item failures with the same
effect on the system, thereby simplifying the fault tree. Compared to an FMECA, it is a
higher level type of analysis where the failure effects of the FMECAs are failure modes
for the FMES. The FMES failure rates are obtained by adding the individual failure rates
of contributing low level, independent, failure modes. The FMES process is facilitated
by worksheets as shown in table 6.
2.4 Fault Tree Analysis (FTA)
2.4.1 FTA is employed in the PSSA process to determine the causes leading to undesirable
top events identified in the FHA. It is a graphical representation of events, or more
often combinations of events, that contribute to the top event. It provides the link
between the different analysis methods described in the present section by:
-
.1 using failure conditions identified as hazardous or catastrophic in the FHA as
top event;
-
.2 generating basic events that may have to be further analysed in an FMECA;
-
.3 demonstrating how combinations of basic events lead to failure modes derived by
FMES and ZHA;
-
.4 quantifying failure rate budgets for basic or intermediate events; and
-
.5 deriving permissible failure rates for basic events.
2.4.2 Principles and procedures for FTA are well documented in the literature.footnote
2.4.3 Instead of FTA, Dependence Diagrams may also be used to achieve the same
objectives.
2.4.4 In the SSA process, FTA is used to demonstrate that the safety objectives for the
top events are satisfied by the actually implemented design.
2.5 Zonal Hazard Analysis (ZHA)
2.5.1 Starting point for a ZHA is the definition of specific zones within the craft that
are, for example, separated by bulkheads or other parts of the structure. The analysis
is performed initially based on design drawings and later on mock-ups or the final
craft. For each of the zones four aspects are addressed in the analysis:
-
.1 Compliance with installation rules
Compliance with the provisions in these Guidelines relating to equipment
installation should be demonstrated.
-
.2 Interaction between systems
The analysis should identify intrinsically hazardous items (e.g. fuel lines) and
show that failures (e.g. fuel leakage) do not cause cascade type failures in
neighbouring systems.
-
.3 Maintenance errors
Improper equipment installation may increase the likelihood of maintenance errors.
The analysis should identify such areas and recommend alternative designs.
-
.4 Environmental effects
Consideration should be given to the effect of environmental conditions such as
lightning strike, bird strike, water ingress, etc.
2.5.2 Details of the analysis technique are inter alia given in the SAE Aerospace
Recommended Practice (ARP) 4761. Results of the analysis should be recorded in data
sheets as shown in table 7.