1
Predictive Flight Data Analysis
Lukas Höhndorf
Florian Holzapfel, Ludwig Drees, Javensius Sembiring, Chong Wang,
Stefan Schiele, Christopher Zaglauer, Kurthan Kersch, Bernhard Katzer
Institute of Flight System Dynamics
Technische Universität München
Garching, Germany
SAGEM 9th Flight Data Monitoring Conference
January 28th, 2015, Barcelona, Spain
Regulatory Framework 2
2
Airlines are required to implement a safety management
system (SMS)
SMS requires operators also to define their own
1 Acceptable Level of Safety (ALoS).
ICAO DOC 9859 “The minimum level of safety performance […] of a service
provider, as defined in its safety management […] .”
Europe aims at a target accident rate of less than one
accident per ten million commercial flights
(i.e. accident probability of 10-7 per flight).
10 million
commercial flights
1 accident
2
Flightpath 2050
BUT: How to quantify the current level of safety?
Solution? 3
3
?
𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐y 𝑜𝑓 𝐼𝑛𝑐𝑖𝑑𝑒𝑛𝑡s ∗ 0
𝑷 𝑰𝒏𝒄𝒊𝒅𝒆𝒏𝒕 = 𝑷 𝑰𝒏𝒄𝒊𝒅𝒆𝒏𝒕 = =0
𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝐹𝑙𝑖𝑔ℎ𝑡𝑠 400 000
Classical statistical approach Runway overrun example
vs.
Classical statistical approach is inappropriate and unsuitable for rare events
*Serious incidents as defined in ICAO Annex 13
Background 4
4
Mission Statement
Predicting statistically valid accident probabilities for an individual
airline based on available evidence from accident-free operation.
Accounting for airline-specific factors such as operations, training, etc.
Predictive Analysis:
Making quantitative statements about the future state based
on previous experience and knowledge.
BUT: How to implement Predictive Analysis
for practical application?
Basic Hypothesis 5
5
Predictive Analysis:
Making quantitative statements about the future state based on:
previous experience
knowledge
previous experience
• recorded data
=
data/evidence driven • known accident types and their causes
• physical relation between contributing
knowledge factors and accident
• known cause-consequence-chains
Basic Hypothesis:
1. Accidents cannot be directly observed in daily operation, however, the
contributing factors still occur at high frequency so they can be measured or
observed with statistical significance.
2. The relation between the contributing factors and the accident can be described
by the laws of physics and cause-consequence-chains based on operational and
procedural knowledge.
Predictive Analysis Concept 6
6
Unknown Advanced
Contributing Factors Statistical
Methods
Incident
Contributing
Physical Model
Factors
Predictive Analysis on Runway Overrun 7
7
Contributing Factors
(Model Input)
Weight Potential
Outcomes
Wind Outcome 1
(e.g. hull loss)
Incident Model
Frequency
Speed Overrun Outcome 2
Model Transition
Output Probabilities Outcome 3
Flaps
...
Incident Probability Outcome n
i.e. “Overrun”
Start of
Braking
…
Predictive Analysis on Runway Overrun 8
8
Step 1 Incident metric
Runway overrun:
Stop margin < 0
Step 2 Functional relationships between contributing factors:
Physical relationships
+ + +
Aerodynamics Propulsion Brakes Gravitation
𝐺 𝐸𝐵 1 𝜌 2 𝜌 2
𝑢𝐾 = ⋅ −𝑚 ⋅ 𝑔 ⋅ 𝑠𝑖𝑛Θ + ⋅ 𝑉𝐴𝐴 ⋅ 𝑆 ⋅ −𝑐𝑜𝑠βA ⋅ 𝐶𝐷 −𝑐𝑜𝑠βA ⋅ 𝑠𝑖𝑛βA ⋅ 𝐶𝑄 + 𝑋𝑃𝐺 + 𝜇 ⋅ −𝑚 ⋅ 𝑔 ⋅ 𝑐𝑜𝑠Θ ⋅ 𝑐𝑜𝑠Φ − ⋅ 𝑉𝐴𝐴 ⋅ 𝑆 ⋅ −𝐶𝐿
𝐵 𝑚 2 𝐵 2
Operational relationships
Runway Condition Procedures A/BRK Selection
DRY
WET
DRY WET OFF LO MED
Incident Model – Causal Chains 9
9
• Reverser • No Go-around
• Hydraulics performed
• A/SKID • Over speed
• ... • Check failed
…
System Human Incident 1
Failures Factor
Incident 2
Incident 3
Environment
….
• Runway Slope
• Contaminated RWY
• Tailwind
• …
Making Data talk 10
10
Asking the right question can significantly increase the information we obtain.
Frequency
vs.
Yes No Altitude (AGL) of Gear Down
Gear Down at 2000 ft AGL ? AGL at Gear Down ?
Quality of statistical statements depend on how we look at the data.
0.018
Gaussian distribution
0.016
Non-Gaussian distribution
0.014
Frequency
0.012
0.01
Underestimation
0.008
of high values
0.006
0.004
0.002
0
320 340 360 380 400 420 440 460 480 500 520
D Contributing factor
Reading between Lines – Parameter Estimation 11
11
• Develop algorithms to extract non-measured contributing factors
• Estimation algorithms are applied to every single flight
Parameter Estimation Implementation during Ground Roll
Parameter Expected Standard Deviation
Value
𝑪𝑫,𝑮 … …
𝑪𝑫,𝑮𝑺 … …
𝝁𝒓𝒐𝒍𝒍 … …
𝝁𝒓𝒐𝒍𝒍+𝒃𝒓𝒂𝒌𝒆 … …
Reading between Lines – Parameter Estimation 12
12
Proof of Match
Measured and Predicted Deceleration During Ground Roll
0.0 0
measured
predicted
longitudinal acceleration (m/s2)
-0.5 -0.5
Predicted
(model)
-1.0 -1
f-x-tot-B [m/s2
-1.5 -1.5
Measured
(QAR Data)
-2.0 -2
-2.5 -2.5
1670 1675 1680 1685 1690 1695 1700 1705 1710 1715
time [sec]
time (45 s)
Quantifying Main Drivers 13
13
What are the main drivers behind the incident probabilities?
Postholder ENV
Influencable No Quantify the sensitivities
Wind of the contributing
… …
factors
Postholder TRA
Influencable Yes
Flaps … …
Postholder TRA
Influencable Yes
Speed …… …
Deviation
Postholder …
Influencable …
… …… …
Change Management 14
14
Contributing Factors
Distribution based on
(Model Input)
actual flight operation (FDM)
Distribution proposed by
Weight
Flight Safety Manager
Wind
Frequency
Incident Model Potential reduction
Speed Overrun Model
Output
Flaps Incident Metric
Incident
Probability
Start of
Braking
• Predictive analysis allows the assessment of the impact of
mitigation actions BEFORE implementing them
Touchdown
• Impact of mitigation actions to OTHER incidents automatically
considered (e.g. runway overrun vs. hard landing vs. tail strike)
Identifying the Unknown 15
15
Dependence Correlation Coefficient Copula
Only captures constant Dependence Capable of capturing
dependency between two variable (nonlinear)
parameters dependencies
between more than two
parameters
Interface to AGS 16
16
• Since we do not want airlines to change their FDM system, we aim to
write our results into existing software.
• Concept: Data
Running our
algorithms…
Results
• A special thank you to SAGEM AGS for providing us with a university
research license and to Mr. Bernhard Katzer for his support!
Summary 17
17
Predictive Analysis enables airlines:
To QUANTIFY airline-specific To IDENTIFY and QUANTIFY
incident and accident probabilities HIDDEN and UNKNOWN
BEFORE things go wrong. contributing factors.
PREDICTIVE
ANALYSIS
To QUANTIFY the main drivers To QUANTIFY the effectiveness
behind incidents. potential mitigation actions
BEFORE implementing them.
18
Professor
Florian Holzapfel (florian.holzapfel@tum.de)
Flight Safety Group
Ludwig Drees (ludwig.drees@tum.de)
Javensius Sembiring (javensius.sembiring@tum.de)
Lukas Höhndorf (lukas.hoehndorf@tum.de)
Chong Wang (chong.wang@tum.de)
Stefan Schiele (stefan.schiele@tum.de)
Christopher Zaglauer (christopher.zaglauer@tum.de)
Kurthan Kersch (kurthan.kersch@mytum.de)
External Support
Bernhard Katzer (bernhard.katzer@dhl.com)
Thank you!
