PRESENTING THE WOMBAT-CS
The PC-based WOMBAT-CS (Complex Systems Operators) Situational Awareness and Stress Tolerance Test (often called HUPEX in Europe) is designed to embody several demands and constraints from the candidate. The individual tasks involve target tracking, pattern recognition, spatial orientation and short-term memory, and on each a testee can reach his or her asymptotic performance level after a short practice period. In a quadrant-location task, as each pattern of numbers is learned, it is replaced by a more difficult pattern of greater scoring worth. A two-back serial digit-canceling task, with no real-world counterpart, is both tediously boring and frustrating.
These tasks comprise the menu of scoring alternatives available to the testee
on request. Each is relatively culture-free in that it has no real-world
counterpart, and each can be learned quickly by the apt testee. The attention
demands of the WOMBAT test are expanded by the ever changing information
presented by peripheral indicators. To score well the testee must monitor
the peripheral indicators vigilantly to follow the shifting priorities of
the various activities as indicated by their potential scoring worths and
current scoring rates and to detect indications of failure modes that may
require immediate termination of one activity in favor of another.
The testees's performance is monitored continuously during the 90 minutes
(typical) of the WOMBAT test. Two Worth-Performance indicators are always
visible in the top corners of the display and serve as guides to the testee
in making the proper choices, establishing the best strategy and monitoring
scoring progress. The left indicator relates to the primary task (target
tracking), and the right indicator shows the secondary task (or Bonus) performance
level.
The product of the Worth (W) and Performance (P) represents the current
effectiveness and is computed into an overall index of recent effectiveness
that is also continuously displayed for the testee.
An indication of total points now (current points) and a prediction of the
final score (the "E" mark), based on current points plus current
effectiveness extrapolated for the time remaining, complete the thermometer-shaped
total scoring display. While performing the WOMBAT test, the testee receives
constant performance feed-back and extrapolated outcome based on his/her
previous choices. The testee is expected to make good use of this data in
determining the best course of action.
The primary task: Target Tracking
The overall goal is to score as many points as possible during the 90
minutes (typical) of testing. The primary background task for other time-shared
activities consists of a single-axis non-stereotypical left-hand tracking
task and a dual-axis right-hand stereotypical tracking task, which operates
in one of two possible control modes.
The testee uses the left hand to control the separation of two pairs
of short vertical lines in two different ways: velocity control and acceleration
control. The task is to keep the two pairs of vertical bars on both sides
of some vertical segments (targets) that are part of a moving hexagon drawn
in the center of the computer display.
The right hand controls the position of a small cross on the screen using
the same modes, velocity or acceleration. Both sticks will always be in
the same control mode. The task is to keep the cross center inside the moving
circle. The combine performance of the left and the right hand produces
the Tracking Score and is depicted in the top-left corner of the display
by the Performance-Worth indicator.
WOMBAT's Autotrack
The tracking task has an automatic mode called "Autotrack"
that can be engaged to free the testee to pursue other methods for earning
points. Autotrack, however, is prone to failures that can vary in severity.
Thus it must be monitored continually even though the testee is working
on another task.
One possible Autotrack failure produces tracking performance only a little
less than can be achieved by hand, while in the other Autotrack fails completely
and the vehicle's motion continues without any regard for the location of
the target. The testee will have to decide whether it is necessary to take
manual control to avoid a serious loss of points in the event of a complete
Autotrack failure or to continue performing another task of sufficient worth
to offset the points lost from a minor deterioration of tracking performance.
At any time during manual tracking, the testee may encounter either of the
two modes of control and must quickly diagnose the situation and respond
accordingly. In velocity control, stick displacement controls the speed
of the symbol motion, and in acceleration control, stick displacement increases
or decreases the speed of symbol motion, a relatively difficult dynamic
relationship that requires concentration, patience, and delicate touch.
When the failed Autotrack is "repaired," it can be re-engaged
provided the manual tracking errors in all three dimensions are within specified
and indicated target limits.
The WOMBAT's Pouch: the secondary level
There is another source of activity for the testee. Known as the Bonus
pouch, it is the source of three side tasks, called secondary tasks,
the performance of which can yield rewards and penalties in various forms.
Either bonus task can be requested whenever the testee elects to move away
from the primary task into this "second" simultaneous level
of activity.
The tracking task is primary in the sense that it cannot be ignored
without serious penalty (the routine must be maintained), and the bonus
task in play is secondary in that the testee may at any time suspend
it and return to the tracking task for manual intervention without any penalty
in the bonus task when its play is resumed. However, the secondary tasks
from the bonus pouch provide the problems and opportunities for the testee
to demonstrate situational awareness, short-term memory, temporal and spatial
pattern recognition, and procedural compliance.
The following three selectable bonus tasks are chosen from the Bonus Menu (figure above) where each choice is presented to the testee as a pictogram to minimize the use of language, thus reducing the risk of culture-based biases.
A 3-D figure rotation and matching task adapted from Shepard and Metzler (1971), requiring spatial orientation and self-assessment of confidence (labeled 1 in the figure above).
A sequential quadrant-location task involving graphically presented temporal mazes requiring an early recognition that specific maze patterns recur and that each pattern follows specific rules that can be learned by careful observation (labeled 5 in the figure above).A two-back delayed sequential-digit-canceling test of short-term memory (choice labeled 9 in the figure above).
Vigilance is an important aspect of situational awareness in the operation
of any complex system, and vigilant time-sharing of attention is required
during performance of either bonus task to catch and respond to autopilot
failures. Let's have a quick look at each bonus tasks.
The Figure Rotation task display two 3-D figures, side-by-side. One
figure will be inside a solid square and the other figure will be inside
a dashed square (right on the figure below). The solid square (left on the
figure below) means that you can rotate the figure using the two sticks,
it is the active figure.
Using both sticks on the WOMBAT console, the testee rotate the active figure until he/she knows all the details of its construction. Then, the testee transfers the control over the other figure, and studies it as well. The goal is to find out, as soon as possible, whether the two figures are identical, mirror symmetric or completely different. Points are earned by giving the computer the correct answer, the sooner the better.
In the Quadrant-Location task, the numbers 1 to 32 are placed in groups of 8 in the four quadrants on the display. The task
is to find each number in ascending order and press the button on the keypad
that corresponds to the quadrant in which it lies, thereby canceling the
numbers in sequence, 1 through 32.
The quadrant designation number appears in a box in the center of each quadrant. The 3 button corresponds to the upper-left quadrant, the 4 button to the upper-right quadrant, the 7 button to the lower-left and the 8 button to the lower-right quadrant. The figure above shows the display quadrants and their respective keypad buttons.
The Two-Back Canceling Task briefly displays a single digit comprised
between 1 and 8 inside a square drawn in the center of the
computer display. Starting with the third digit displayed, as soon as each
new digit appears, the testee is required to press on the keypad matching
the digit displayed two back in the sequence. Once the answer has been given,
right or wrong, the next digit appears, leading to the answer from the testee
matching the next two-back digit in the sequence.
The testee must remember the digits as the appeared and disappeared, he/she must also remember if the answer was given if an interruption becomes necessary to cater for a Autotrack failure in the primary task. The figure above shows the action of answering digit 6, but only after 5 and 2 had appeared and disappeared.
Testing Strategy
All four tasks, one primary and three secondaries, are relatively culture-free
in that they have no real-world counterpart, and each can be learned
quickly by the apt testee. Typical group progress curves on the WOMBAT-CS
test initially show gradual improvement in the rate of scoring, with relatively uniform scoring after the fourth or fifth 10-minute segment (figure
below).
The attention demands of the test are expanded by situational information
presented by peripheral indicators. To score well, the testee must monitor
the indicators vigilantly to follow the shifting priorities of the various
tasks as indicated by their potential scoring worths and current scoring
rates and to detect indications of failure modes that may require immediate
termination of one activity in favor of another.
The task structure outlined places a high premium on a rational attention
allocation strategy. The background tracking task with its unreliable
Autotrack generating a performance score on the appropriate indicator rewards
the allocation of some portion of one's attention to the known signals of
variability: the changing worths of the different tasks and the running
overall indices of current scoring and predicted final score.
Tolerance of frustration is tested in part by intervals in the test when the very best that can be done will produce a low rate of score accumulation. In the face of this kind of computer behavior, the testee must recognize that something is going on beyond control and accept it until the bad times pass, but for some the frustration might lead to a breakdown in performance.
See some WOMBAT users here.