F-16 DATABASE TERRAIN CUEING

AN INVESTIGATION OF DISPLAY HANDLING QUALITIES

Maj Kevin Christensen, USAF (M)
Capt Gregory Weber, USAF
Mr. Michael Seelos, USAF
416th Flight Test Squadron
Edwards AFB, CA
Mr. Sean Gillen
Lockheed Martin Tactical Aircraft Systems
Ft. Worth, TX

Figure 1, F-16 Block SOD

Background
In response to several mishaps in which perfectly good F-16swere flown
into the ground, the F-16System Program Office decided to integrate the Digital
Terrain System (DTS) into all F-16s. Since DTS was an off-the-shelf purchase,
this integration effort was considered a low cost way to reduce controlled flight
into terrain mishaps, especially when compared with the cost of such accidents.
Although not the primary function of DTS for reducing controlled flight into
terrain mishaps, Database Terrain Cueing (DBTC), aided pilot awareness of the
surrounding terrain by providing a cue in the head up display (HUD).

118

The F-16 Combined Test Force at Edwards Air Force Base, California
recently completed Phase 1 flight testing of the DTS. During this testing, the DTS
team identified and corrected several handling qualities problems. This paper will
not only share the results of our DBTC testing, but more importantly offer
valuable lessons for other flight test programs to use.

Figure 2, DTS Data Transfer Cartridge

Test Article Description

The heart of DTS was a portable black box called the Data Transfer
Cartridge (DTC) built by Fairchild Industries (Figure 2). The improved
DTS/DTC was a form, fit, and function replacement for the standard F-16 DTC.
The DTSDTC differed from a standard DTC in that it had a new mass memory
for storing a terrain database and a new microprocessor to drive the DTS. The
actual DTS software was contained in TERPROM (TM) software developed by
British Aerospace Systems and Equipment Company.

119

The terrain database used by the F-16 DTS came from the Level 1 Digital
Terrain Elevation Data (DTED) developed by the National Imagery and Mapping
Agency (Figure 3). The raw DTED represented the terrain by storing elevations
on posts along a grid in latitude and longitude. In the F-16 DTS, these terrain
database posts were spaced apart by about 500-600 feet in our test area. The
terrain database also contained a Vertical Obstruction Database (VOD), which
stored the location and height of manually entered man-made obstacles such as
towers.

Figure 3, Digital Flight Map Database

The DTS consisted of five functions: terrain referenced navigation (TRN),

predictive ground collision avoidance system (PGCAS), database terrain cueing
(DBTC), obstacle warning and cueing (QW/C), and passive ranging (PR). Only
two of the five functions, TRN and DBTC, will be discussed further.

Terrain Referenced Navigation

The primary purpose of the terrain referenced navigation (TRN) function
of the DTS was to provide accurate registration of the aircrzft within the terrain
database. TRN determined aircraft position relative to the terrain database by
using radar altimeter and inertial navigation unit (NU) data to generate a terrain
profile of the actual terrain under the aircraft (Figure 4). This terrain profile was
then matched to the terrain database to determine estimates of corrections to INU
latitude, longitude, and elevation. As part of the TRN function, a Kalman filter
computed various uncertainties and error states associated with its estimates of
position corrections.

120

Figure 4, Terrain Referenced Navigation

Database Terrain Cueing

Database terrain cueing (DBTC) is a vertical steering guidance function
that aids the pilot in flying low level at a selected terrain clearance height (TCH).
DBTC improved a pilots awareness of the terrain by providing a cue in the head
up display. Using the TRN position relative to the database, the DBTC function
generated a worst case terrain profile by simulating a sensor scanning the terrain
ahead of the aircraft (Figure 5). This scan area was adjusted to account for
uncertainties in TRN position corrections. Additionally, the scan was shifted into
a turn based on turn rate. With this scan area of the database defined, the final
worst case profile was reduced to a two-dimensional representation of the terrain
contained in the scan area (Figure 6).
The DBTC function then computed the normal acceleration or g
command necessary to clear the terrain in the worst case profile by the pilotselected terrain clearance height (TCH) within the limits of +2.0 to -0.9
incremental gs. The end product of this DBTC calculation was a box displayed
to the pilot i n the head up display (Figure 7). The displacement of the DBTC box
from the flight path marker was proportional to the change in g needed to fly at
the TCH. Figure 7 shows an example in which the pilot must push about 0.5 g in
order to fly the flight path marker into the DBTC box. Aircraft g was displayed in
the upper left corner of the HUD.

121

Figure 5, Simulated Scan Pattern

Figure 6, Two-Dimensional Worst Case Profile

122

Terrain Cue

Flight Path Marker

Current g

25 mils = 1.0 g
Figure 7, Head Up Display DBTC Presentation

Test Methods

The DBTC function was first tested by two different ground simulations.
Lockheed-Martin Tactical Aircraft Systems (LMTAS) used an unmanned, off-line
simulation to characterize the DBTC algorithm and compare DBTC performance
to that of terrain following systems. This simulation used a pilot model and did
not address the issues involved with displaying the cue to the pilot. LMTAS
further tested DBTC on the ground by using their handling qualities simulator
(HQ Sim). This simulator was a man-in-the-loop simulation which used actual
avionics and flight control hardware and software. The aircraft was modeled by a
six degree of freedom high fidelity model that'had been used during F-16 Low
Altitude Navigation and Targeting Infrared for Night (LANTIRN) development
and flight test. This provided the first opportunity to assess the presentation of the
DBTC cue itself. Initial HQ Sim results highlighted the potential for a pilotinduced oscillation to develop while following the DBTC cue at high
groundspeeds.
Initial DBTC flight test consisted of flying low level routes over isolated
peaks, rolling terrain, and rough terrain. Test points were chosen across the F-16
speed envelope, from 340 to 590 knots groundspeed. The DBTC function was
exercised at all TCHs, from 100 to 1,000 feet. Finally, two aircraft loadings, one
123

light and one heavy, were used to characterize how changes in aircraft
performance and mass properties would affect DBTC performance (Figure 8).

Figure 8, Two Aircraft Loadings Flown

In addition to flight testing across the aircraft envelope, two separate

techniques for following the DBTC cue were initially chosen to aid in evaluating
the DBTC function.
The DBTC characterizatioii runs were accomplished to
validate the unmanned simulation performance results through a comparison with
actual flight test data. Pilots were tasked to follow the DBTC box as smoothly as
possible. This technique required the pilot to act as a compensator to smooth out
any abrupt movements of the DBTC box. O n the contrary, the DBTC
performance assessment runs were accomplished to allow pilots and engineers to
evaluate the acceptability of the DBTC cue under a more operationally realistic
scenario. Pilots were tasked to follow the DBTC box as closely as possible, as if
it were a manual terrain following cue.

Initial Test Results

The initial DBTC flight test effort took place from April through June of
1996. A total of 15 sorties and 27 hours were flown on a single Block 50 F-16D
test jet. This testing was a subset of the Phase 1 DTS flight test effort which also
evaluated the other four DTS functions.
Figure 9 is a two-dimensional representation of initial DBTC performance
while flying over rough terrain. The x-axis shows distance along the low level
route in nautical miles. The upper curve shows aircraft pressure altitude, while
the lower curve shows terrain elevation. For reference, the middle curve shows
the 500 foot terrain clearance height added to ground elevation. In comparing the
top and middle curves, one can see that the aircraft flew well above the selected
TCH. In fact the second peak crossing was nearly 500 feet high. Additionally, as
the aircraft approached the peak, a late commanded climb resulted in a large
positive flight path angle at peak crossing. This in turn resulted in the aircraft
continuing to climb for several hundred feet more after crossing the peak, known
124

as ballooning. From a military utility standpoint, these DBTC performance

problems could degrade the pilots ability to terrain mask at a specified TCH.

Over Roueh Terrain at 500 ft TCH

9000

8000

4-0

7000

I
+

6000

5000

4000

10

Distance (NM)
Figure 9, Initial DBTC Performance

In addition to these DBTC performance issues, pilots found the DBTC cue
to be jittery and too sensitive when they attempted to follow the cue as closely as
possible. From an operational perspective, the test team did not think that pilots
would try to fly the box as closely as the test pilots were. However, the jerky
movement of the box was a big distraction which could divert the pilots attention
away from the actual terrain. The pilot would be able to do a better job of terrain
masking by visually referencing the actual terrain. Most significantly, if a pilot
did try to follow the box closely, flight safety could be jeopardized by a divergent
pilot-induced oscillation (PIO) that developed at various airspeed and terrain
combinations. The test team determined that the potential PI0 should be further
investigated and characterized.

Handling Qualities Evaluation

In order to investigate the handling qualities .(HQ) of the terrain cue, the
DTS team developed an HQ evaluation with a tight tracking task to force high
pilot gains. This HQ evaluation allowed the team to quantify HQ trends with
125

flight conditions using the Cooper-Harper HQ rating scale and to identify any HQ
cliffs. Additionally, the team hoped that this HQ evaluation would generate
pilot comments in order to isolate and correct the specific HQ problems.
The defined HQ tracking task was to keep the flight path marker inside the
DBTC cue box while flying wings level at a particular airspeed. Each test point
specified an airspeed and a terrain route. Terrain routes were selected to sample
isolated peak, rolling terrain, and rough terrain types. To limit the scope of this
HQ evaluation, all test points were flown at 500 feet TCH. The desired criteria
for this HQ evaluation was defined as following the cue with at least half of the
flight path marker (FPM) in the DBTC box for the entire route with no more than
five excursions per minute. This represented the pilot following the DBTC cue to
within +I-0.2 g. The adequate criteria was defined as following the cue with the
flight path marker at least touching the DBTC box for the entire route with no
more than five excursions per minute. This represented the pilot following the
cue to within +/- 0.4 g. Since PIOs were anticipated, pilots were told that
overshoots while correcting back to the box were not to be counted as excursions.
A thorough postflight video review was used to determine task performance.
Following each HQ test point, pilots were asked to give a quantitative
Cooper-Harper rating. These ratings were based on the tracking performance that
was attained and the amount of pilot compensation required to meet this level of
tracking performance. Since some of the HQ deficiencies seemed related to
turbulence, pilots were also asked to give turbulence ratings following each run
based on the definitions from the Department of Defense Flight Information
Handbook. In general, pilots rated the severity of turbulence, as light, moderate,
or severe, based upon. its effect on aircraft attitude and altitude. Turbulence was
also rated as either intermittent or continuous based on its frequency. Perhaps
even more important than the quantitative HQ ratings, the pilot comments that
these HQ evaluations inspired helped considerably in isolating specific HQ
problems.

Handling Qualities Evaluation Test Results

The DBTC HQ evaluation consisted of 5 more sorties flown in June and
July of 1996. In analyzing the results of this evaluation, the test team found a
correlation in three areas: turbulence, terrain roughness, and groundspeed.
The left plot in Figure 10 shows the correlation between Cooper-Harper
ratings and turbulence while flying over an isolated peak at 590 knots.
Unfortunately, even the Air Force Flight Test Center couldnt control turbulence,
so there were not many data points. However, these points did show that handling
126

qualities degraded to Level 3 when moderate turbulence was encountered during

this test point. Pilots commented that as turbulence increased, so did jitter in the
DBTC box. On some test points, this jitter was so bad that it was nearly
impossible to track the box. In order to compensate for this jitter, pilots would fly
the average of the box position.

Figure 10, Initial Handling Qualities Results

The center plot in Figure IO shows the correlation between Cooper-Harper

ratings and terrain roughness while flying at 590 knots. Test points flown in
rough terrain resulted in Level 3 HQ ratings for all four pilots sampled. The pilots
commented that in rough terrain, the box moved quickly, sometimes in discrete
jumps, making it nearly impossible to follow. To compensate for this, pilots
anticipated box movements based on the terrain ahead.
The right plot in Figure I O shows the correlation between Cooper-Harper
ratings and groundspeed while flying over rough terrain. At the highest
groundspeed sampled, all four pilots agreed that DBTC handling qualities were
clearly Level 3. The pilots commented that at the high speed, any inputs to correct
back to the box caused it to swap sides, resulting in a PIO. To compensate for this
highly sensitive DBTC box, pilots had to lower their gains. If the P I 0 became
divergent, pilots had to completely back out of the loop before attempting to track
the box again.

Handling Qualities Improvements

One step in improving the DBTC handling qualities was to reduce or
eliminate discrete jumps in the DBTC cue. The Lockheed-Martin engineers
determined that one possible source might be two states of the TRN Kalman filter.
127

First, the TRN map error state was added to the height of terrain within the worst
case profile used by DBTC. Additionally, the TRN map error uncertainty was
added into the calculation of the aircraft worst case height. During a single
iteration of the DBTC function, the worst case profile may jump in altitude as
indicated by the dashed terrain profile in Figure 1 1. Similarly, the TRN estimate
of worst case height may jump as indicated by the white aircraft. After these
calculations are made, the DBTC function would generate a new g-command. In
the example in Figure 1 I , DBTC commanded the white aircraft back to the preset
TCH over the dashed terrain by generating a large positive discrete jump in the
DBTC box. By removing these two calculations, Lockheed-Martin engineers
hoped that more consistent worst case profiles and worst case aircraft heights
would reduce the magnitude and occurrence of discrete jumps in the DBTC box.

MaphrUncertaintyJ
,.

.......

-- .....-._._.

..._._.
._..
_.....
Map Error State Jlrmp
\

Figure 11, Discrete Jump Example

Another step in improving the DBTC handling qualities was to reduce the
jitter in the DBTC cue. Figure 12 shows a pilotized block diagram of the DBTC
control loop. The very center of this and all things in the world is the pilot
himself. The pilot directly responded to the HUD cue and produced an input to
the aircraft. The top box represents the DBTC algorithm generating the required g
commands. The bottom row represents the feedback of aircraft g. This signal
was smoothed out by filtering high frequency noise. The difference between
commanded and actual g was then used to generate the displacement of the DBTC
box in the HUD.

128

Figure 12, Simplified DBTC Control Diagram

In analyzing the DBTC jitter and PI0 problems, Lockheed-Martin

engineers looked closely at time slices taken from different paths of the block
diagram. The time slices in Figure 13 were taken from a 590 knot test point over
rough terrain. The top plot shows the DBTC steering command in the HUD. The
second curve shows the feedback loop signal, actual aircraft g smoothed out by
the low pass filter. Finally, the bottom curve shows the pilots stick force in
response to the HUD display. This was one of many examples of PI0 seen during
flight testing. Looking at the plot of stick force, it was apparent that these inputs
had a big effect on actual aircraft g. The feedback of this aircraft g to DBTC
steering caused the box to rapidly move opposite of the pilots input, resulting in a
PIO. This can clearly be seen by comparing the top and second curves. This led
LMTAS engineers to target the feedback loop rather than the DBTC algorithm in
their quest to reduce the P I 0 potential.
Since the P I 0 tendency had a definite correlation to increased
groundspeed, engineers added a velocity-dependent lag filter to the feedback loop
(Figure 14). Additionally, the existing low pass noise filter was modified in an
attempt to reduce the jitter problem coming from turbulence.

129

...........

Time (sec)

Figure 13, Pilot Induced Oscillation Time Slice

Figure 14, Improved DBTC Control Diagram

130

The left graph in Figure 15 shows how parameters in the low pass filter
were changed to reduce the magnitude of high frequency noise. The parameters
of interest were damping ratio (6) and natural frequency (0,).
The two curves
show the change from 0.7 to 1.1 damping ratio. The right graph in Figure 15
shows the velocity-dependent lag filter. At low airspeeds, the time lag (T) was
minimal, but at higher airspeeds, this time lag increased to slow down the
response of the DBTC box to changes in aircraft g. Thus, the inherent pitch
sensitivity of the F-16 at higher airspeeds was compensated by the lag in the
feedback loop.
The first step in optimizing the feedback filter parameters was to fly them
in the HQ simulator. This allowed engineers to bound the parameters, thus
limiting the number of .sorties required to come up with the optimum
configuration.

Figure 15, Noise Filter Bode Diagram and Velocity-Dependent Lag Parameter

Handling Qualities Improvement Flight Test

On each flight test sortie, the pilot was given three DTSDTCs, one
containing the baseline configuration for the flight, and the other two varying one
or two parameters from that baseline. The pilots were not told the parameters on
any DTC until the optimization process was complete. Thus, by direct blind
comparisons of the three DTSDTCs flown by a single pilot on a single sortie, a
best configuration was chosen. The parameters that changed from the baseline
to the best configuration were evaluated by the engineers. The configurations
flown in the next sortie were chosen based on the current best configuration and
the engineers predictions of which parameters to alter further. This process was
131

repcated until a final sct of threc DTSDTCs was flown to pick the optimum
configuration.

Handling Qualities Improvement Test Results

The additional six DBTC sorties flown in follow-up flight test was money
well spent by the F-16 program office. Figure 16 dcmonstrates that the
optimization process really worked. The final best set of parameters improved
handling qualities across the board. Although the final evaluations were only
flown in light intermittent turbulence, the jitter in the box was significantly
reduced and handling qualities while flying over an isolated peak at 590 knots
moved up to Level I . Furthermore, the sensitivity of the DBTC box at high
speeds was reduced, with no significant P I 0 tendencies noted. The third plot in
Figure 16 shows that at our most demanding test point, 590 knots over rough
terrain, handling qualities improved from Level 3 to Level 2. The problem which
kept this best set of parameters from having Level 1 handling qualities was that
the DBTC box still moved too fast for a pilot to follow. This is best illustrated by
the center plot in Figure 16 which shows the correlation between Cooper-Harper
rating and terrain while flying at 590 knots. Over rough terrain, only adequate
tracking criteria could be attained.

Figure 16, Final Handling Qualities Results

132

Over Rowh Terrain at 500 ft TCH

9000

8000
h
CI

7000

al

'g 6000

5000

4000
0

10

Distance (NM)
Figure 17, Improved DBTC Performance

The changes in DBTC also improved overall DBTC performance (Figure

17). The new DBTC configuration commanded a climb earlier than the original.
This resulted in a near level flight path at peak crossing, allowing the pilot to fly
much closer to the preset TCH.

Future Testing
Phase 2 of DTS testing is currently underway at Edwards in 1997. In this
phase, the test team hopes to find other ways to improve this system. Specifically,
the team plans to extend the DBTC HQ investigation down to lower altitudes to
see if there is any correlation between HQ and TCH. We also hope to expand the
types of turbulence encountered during our future HQ investigations.
The test team is also planning on doing a military utility evaluation of
DBTC while flying with night vision goggles (NVGs) (Figure 18). The test team
thinks that this is the area that pilots will find DBTC most useful. In high moon
illumination conditions, pilots can easily see and avoid terrain while flying at low
altitude with NVGs. However, current generation NVGs lack the depth
perception that pilots are accustomed to in the daytime. This makes it more
difficult to judge altitude above the ground. The DBTC system should
compliment NVGs well, since rather than having to look at and interpret the
digital radar altimeter readout, pilots will be able to cross-check the DBTC
133

symbol to determine where they are in relation to their desired TCH. This
increased terrain awareness should reduce pilot workload while flying at low
altitudes with NVGs.

Figure 18, F-16 Night Vision Goggles

Lessons Learned
This flight test effort not only provided an excuse to have fun flying low
and fast, but it also generated numerous lessons for the test team to pass on to our
fellow flight testers.
Our first big lesson was that even avionics flight test may call for a
handling qualities evaluation. This is especially true for new displays designed as
flight directors to help reduce pilot workload. Another excellent example is the
flight testing of helmet mounted displays, especially if used for sensor slaving.
Our test team found that the HQ evaluation using the Coopei-Harper rating scale
was an excellent tool. Not only did it allow us to correlate HQ with flight
conditions, but the pilot comments generated by our HQ evaluation allowed us to
isolate and fix specific HQ problems. Of course this method has proven valuable
for classical flying qualities evaluations over the years, but it also worked well to
wring out the HQ problems of our DBTC display.
134

In the flight test community, after watching several spectacular crash

videos, we have been programmed to think of a divergent P I 0 as a catastrophic
event equal to loss of aircraft control. This is not always the case. Although we
saw divergent PIOs in our flight testing, aircraft control was never in question.
Stopping the P I 0 was as easy as looking away from the DBTC box. Another big
misconception is that PIOs are caused by high gain pilots who just dont have
the right feel for their aircraft. The test pilots were humbled because they
caused a P I 0 while trying to follow the box without having to try hard at all. The
test pilots found that at the most severe flight condition, if they even looked at the
box, they were in a PIO. Even our low gain pilot found himself in a divergent
PIO. The big lesson from this is that when humbled by a PIO, you can always
blame the engineers. Another big lesson is to sample a cross section of pilot
gains, or you may never uncover that hidden P I 0 tendency.
Our test effort proved once again that PIOs can and will happen during
flight test, so have a plan to deal with them. Our encounters with PIOs called for
some creative test planning which stressed the flexibility of our flight test effort to
its limit. Luckily our program had time in the schedule and could afford to add
more sorties without impacting any production releases. Other programs may not
have that luxury.

In fixing our HQ problems, we came up with some other lessons. First,

test teams need to refrain from jumping to hasty conclusions that may delay
finding the real fix. In our case, when we first discovered our HQ problems, our
initial reaction was to blame the basic DBTC algorithm. With thorough data
review, however, our engineers were able to fix the problem with filters in the
feedback loop.
A lesson relearned was the importance of ground simulation. This enabled
our team to not only check the HQ fixes, but also to optimize them before we got
into the air. The teams ability to flight test several DBTC options on a single
sortie was another big key to our success, enabling pilots to conduct a direct blind
comparison of the new configurations. Other test teams should also look for ways
to flight test different options on the same sortie, significantly improving the flyfix-fly test approach.
Finally, teamwork between pilots and engineers was essential in correcting
the problems seen in flight test. Overall, through our teamwork, we were able to
take this off-the-shelf piece of equipment and turn it in to something useful to
the pilot in the field.

135