The PC-based WOMBAT-FC Situational Awareness and Stress Tolerance Test
is designed to embody several demands and constraints from the candidate.
The suffix FC refers to Flow Controllers because WOMBAT-FC
is specifically designed to assess people who monitor a dynamic flow
of information, such as Air Traffic Controllers, Train and Port Traffic
Supervisors, Control Room Operators, Energy Plant Controllers, Vehicles
Dispatchers, etc.The individual tasks involve target tracking, pattern recognition,
spatial orientation and short-term memory, and on each a testee can reach
his or her asymptotic performance level after a short practice period. In
a quadrant-location task, as each pattern of numbers is learned, it is replaced
by a more difficult pattern of greater scoring worth. A two-back serial
digit-canceling task, with no real-world counterpart, is both tediously
boring and frustrating.
These tasks comprise the menu of scoring alternatives available to the testee
on request. Each is relatively culture-free in that it has no real-world
counterpart, and each can be learned quickly by the apt testee. The attention
demands of the WOMBAT-FC test are expanded by the ever changing information
presented by peripheral indicators. To score well the testee must monitor
the peripheral indicators vigilantly to follow the shifting priorities
of the various activities as indicated by their potential scoring worths
and current scoring rates and to detect indications of failure modes that
may require immediate termination of one activity in favor of another.
The testees's performance is monitored continuously during the 90 minutes
(typical) of the WOMBAT-FC test. Two Worth-Performance indicators are always
visible in the top corners of the display and serve as guides to the testee
in making the proper choices, establishing the best strategy and monitoring
scoring progress. The left indicator relates to the primary task (target
tracking), and the right indicator shows the secondary task (or Bonus) performance
level.
The product of the Worth (W) and Performance (P) represents the current
effectiveness and is computed into an overall index of recent effectiveness
that is also continuously displayed for the testee.
An indication of total points now (current points) and a prediction of the
final score (the E mark), based on current points plus current effectiveness
extrapolated for the time remaining, complete the thermometer-shaped total
scoring display. While performing the WOMBAT-FC test, the testee receives
constant performance feed-back and extrapolated outcome based on his/her
previous choices. The testee is expected to make good use of this data in
determining the best course of action.
The overall goal is to score as many points as possible during the 90 minutes of testing. The primary task consists of tracking moving targets as they follow assigned routes on a 5 by 5 grid (in figure below, the moving targets are labeled T, A, B, and Q). Each target enters the grid at a specific waypoint, follows its assigned route and leaves the grid at another, pre-determined waypoint. The testee must keep track of all the targets (a maximum of 12 targets at the time) in case one disappears. All the targets move at the same speed.
When one target disappears from the grid (in the figure above, target
B disappeared), the testee must find it and restore positive tracking as
fast as he/she can in order to keep the tracking performance to a maximum.
The faster the testee is in finding a missing target, the more points he/she
earns. The computer uses the testee's efficiency at finding missing targets
to determine whether he/she can manage a greater number of moving targets
(thus earning more points) or a lesser number of moving targets (thus earning
less points). The testee must therefore keep a good mental picture
of the dynamic situation even when the tracking task is no more visible
on the computer screen because attending other tasks. The better is the
mental picture, the faster is the missing target recovery.
Each target has one of three bright colors (we never make reference to "flight
levels" or "altitude" to avoid ATC experience biases.) Two
targets of the same color will collide if the meet on the grid, producing
a "starburst" symbol at the collision site for 10 seconds and
a subsequent loss of tracking worth (the figure below shows before and after
a collision.) A candidate with higher levels of residual attention
may be able to predict a conflict and communicate his/her prediction to the
computer. A good prediction will earn the candidate extra points and a erroneous
prediction will cost some precious points.
The tracking task has an automatic mode called Autotrack that
can be engaged to free the testee to pursue other methods for earning points.
Autotrack, however, is prone to failures. Thus it must be monitored continuously
even though the testee is working on another task.
Normally, Autotrack will automatically keep track of all moving targets.
If one target disappeared from the Autotrack surveillance, the word "Missing"
will be displayed in a rectangle on top of the monitor, and the target identification
letter on the left side of the picture will turn to red. The performance
indicator on the top left corner of the picture will show a decreased performance
until the missing target is recovered. A missing target keeps moving along
its assigned route and may collide with another, visible target.
There is another source of activity for the testee. Known as the Bonus
pouch, it is the source of three side tasks, called secondary tasks,
the performance of which can yield rewards and penalties in various forms.
Either bonus task can be requested whenever the testee elects to move away
from the primary task into this "second" simultaneous level
of activity.
The tracking task is primary in the sense that it cannot be ignored
without serious penalty (the routine must be maintained), and the bonus
task in play is secondary in that the testee may at any time suspend
it and return to the tracking task for manual intervention without any penalty
in the bonus task when its play is resumed. However, the secondary tasks
from the bonus pouch provide the problems and opportunities for the testee
to demonstrate situational awareness, short-term memory, temporal and spatial
pattern recognition, and procedural compliance.
The following three selectable bonus tasks are chosen from the Bonus Menu (figure above) where each choice is presented to the testee as a pictogram to minimize the use of language, thus reducing the risk of culture-based biases.
A 3-D figure rotation and matching task adapted from Shepard and Metzler (1971), requiring spatial orientation and self-assessment of confidence (labeled 1 in the figure above).
A sequential quadrant-location task involving graphically presented temporal mazes requiring an early recognition that specific maze patterns recur and that each pattern follows specific rules that can be learned by careful observation (labeled 5 in the figure above).A two-back delayed sequential-digit-canceling test of short-term memory (choice labeled 9 in the figure above).
Vigilance is an important aspect of situational awareness in the operation
of any complex system, and vigilant time-sharing of attention is required
during performance of either bonus task to catch and respond to autopilot
failures. Let's have a quick look at each bonus tasks.
The Figure Rotation task display two 3-D figures, side-by-side. One
figure will be inside a solid square and the other figure will be inside
a dashed square (right on the figure below). The solid square (left on the
figure below) means that you can rotate the figure using the two sticks,
it is the active figure.
Using both sticks on the WOMBAT console, the testee rotate the active figure until he/she knows all the details of its construction. Then, the testee transfers the control over the other figure, and studies it as well. The goal is to find out, as soon as possible, whether the two figures are identical, mirror symmetric or completely different. Points are earned by giving the computer the correct answer, the sooner the better.
In the Quadrant-Location task, the numbers 1 to 32
are placed in groups of 8 in the four quadrants on the display. The task
is to find each number in ascending order and press the button on the keypad
that corresponds to the quadrant in which it lies, thereby canceling the
numbers in sequence, 1 through 32.
The quadrant designation number appears in a box in the center of each quadrant. The 3 button corresponds to the upper-left quadrant, the 4 button to the upper-right quadrant, the 7 button to the lower-left and the 8 button to the lower-right quadrant. The figure above shows the display quadrants and their respective keypad buttons.
The Two-Back Canceling Task briefly displays a single digit comprised
between 1 and 8 inside a square drawn in the center of the
computer display. Starting with the third digit displayed, as soon as each
new digit appears, the testee is required to press on the keypad matching
the digit displayed two back in the sequence. Once the answer has been given,
right or wrong, the next digit appears, leading to the answer from the testee
matching the next two-back digit in the sequence.
The testee must remember the digits as the appeared and disappeared, he/she must also remember if the answer was given if an interruption becomes necessary to cater for a Autotrack failure in the primary task. The figure above shows the action of answering digit 6, but only after 5 and 2 had appeared and disappeared.
All four tasks, one primary and three secondaries, are relatively culture-free
in that they have no real-world counterpart, and each can be learned
quickly by the apt testee. Typical group progress curves on the WOMBAT-CS
test initially show gradual improvement in the rate of scoring, with relatively
uniform scoring after the fourth of fifth 10-minute segment (figure
below).
The attention demands of the test are expanded by situational information
presented by peripheral indicators. To score well, the testee must monitor
the indicators vigilantly to follow the shifting priorities of the various
tasks as indicated by their potential scoring worths and current scoring
rates and to detect indications of failure modes that may require immediate
termination of one activity in favor of another.
The task structure outlined places a high premium on a rational attention
allocation strategy. The background tracking task with its unreliable
Autotrack generating a performance score on the appropriate indicator rewards
the allocation of some portion of one's attention to the known signals of
variability: the changing worths of the different tasks and the running
overall indices of current scoring and predicted final score.
Tolerance of frustration is tested in part by intervals in the test when the very best that can be done will produce a low rate of score accumulation. In the face of this kind of computer behavior, the testee must recognize that something is going on beyond control and accept it until the bad times pass, but for some the frustration might lead to a breakdown in performance.
