Issue 1/94
The Bulgarian Air Traffic Services Authority has received two WOMBAT-FC
systems following an order placed in February 1994. Earlier in 1993, a delegation
from Sofia attended the IATA Human Factors Conference in Montreal. Dr. Elena
B. Pentcheva Psy., from the Bulgarian Transport Medical Institute of Sofia,
headed the negociations with Aero Innovation to acquire two units immediately,
with an option for two additional unitsto be exercised within 1994.
Research Specialist Dr. Elena Pentcheva (center), heading the Bulgarian delegation at the IATA Human Factors Conference in Montreal, October 1994, with Ms. Elly Michova (right), Balkan Airline's Safety Officer and Mr. Jean LaRoche (left), President of Aero Innovation, Inc.
"WOMBAT is unique and we are very excited to start using this system",
declared Dr. Pentcheva when reached on the phone last week. At the Sofia-based
institute, research projects are already being modified to include the use
of the two WOMBAT-FC Situational Awareness tests.
Mr. Jean LaRoche, President of ÆRO INNOVATION INC. is expected in
Sofia during the first week of May for technical and operational training
with the scientists of the Transport Medical Institute.
More news from the Bulgarians in AeroNews
2/94.
Montreal - Tokyo based Nozaki & Co., Ltd and Montreal-based Aero
Innovation Inc. have announced a joint effort to market WOMBAT in Asia.
Nozaki & Co. has been active in the aerospace industry since 1952, representing
leading aircraft and engine manufacturers with annual sales of U$1.2 Billion
throughout Asia. Aero Innovation, established in 1988, is well known for
the WOMBAT Aptitude Computerised Tests, which rapidly became the new world
standard in the selection of aviation personnel. The PC-based test presents
the candidate with culture-free pictograms in a multitask environment, requiring
judgment, decision making, stress tolerance, short-term memory, diagnostic
skills and spatial representation skills. Recent studies in the USA have
demonstrated an unprecedented correlation between WOMBAT-CS and the cost
of training ab initio pilots, making WOMBAT-CS the most predictive pilot
selection system in history. World-class airlines have purchased WOMBAT-CS
systems, along with the FAA and ICAO. WOMBAT is presently being evaluated
in the USA, Canada, Saudi Arabia, Koweit, Jordan, Japan, China, Singapore,
Australia and the Tcheck Republic.
Mr. Shinji Naka from
Tokyo's Nozaki & Co.
WOMBAT turn-key systems are reportedly sold for U$30 000 per unit. WOMBAT-FC,
for selecting air traffic controllers, was recently introduced with the
Bulgarian ATS. Four WOMBAT-FCs are also being used by Canada's Ministry
of Transport in Cornwall, Ontario for validation study purposes.
Typically, pilot selection tests account for about 25 percent of the variance in measures of success in pilot training and bear no evident relation to operational performance. Beyond basic intelligence and motor skills, pilot performance is believed to depend largely on "situational awareness," the overarching ability to attend to multiple information sources, evaluate alternatives, establish priorities, and work on the alternative with the highest momentary priority. Crew duties add a social dimension to individual performance that is now addressed by training in crew resource management. The solo WOMBAT test was designed to measure the situational awareness and stress tolerance of individual crew members, and the DuoWOMBAT addresses the abilities of crews to manage their resources.
by Dr. Stanley N. Roscoe
V.-P. Research & Development
Psychologists have developed accurate measures of human intelligence and
somewhat less precise but nonetheless useful instruments for describing
human personality factors. Unfortunately we have been less successful in
assessing human aptitudes for controlling air and surface traffic or deciding
what to do on a final approach following a midair collision between planes
landing and taking off over the same airport. The military have invested
huge sums developing and validating selection batteries that account for
no more than about 25 percent of the variance in training success and have
no evident correlation with operational performance.
The long-range prediction of who will rise to the surface in an operational
crisis is based on a foundation of psychological quicksand. As can happen
with flight crews and air traffic controller teams, the individuals involved
may have all the skills and knowledge normally required but be unable to
put them together in the confusion following the hypothetical midair collision
over the airport. To cope with the problems encountered by crew members
working with other crew members, most of the world's airlines have instituted
training in crew resource management, despite the fact that the evaluation
of CRM's effectiveness has been almost entirely anecdotal. By consensus,
CRM has high "face validity."
The costs of mistakes in selecting and evaluating
crew members are not limited to those resulting from mismanagement of critical
operations. It is also costly to train individuals who can pass all training
tests but then are unable to stand up under the stress of operational chaos.
Although certain personality tests are believed by some to reflect traits
conducive to effective and harmonious interactions among crew members, until
recently there has been no objective test specifically designed to call
for the working exercise of those traits. The PC-based DuoWOMBAT Crew Resource
Management Test has been designed to meet that need.
Historically aptitude tests have been crippled by the notion that performance
of complex team operations depends on a collection of individually simple
abilities. Consistent with this idea, batteries have been developed to test
reaction time, manual dexterity, short- and long-term memory, spatial orientation,
and the like. The fact that such batteries account for only about 25 percent
of the variance in training success depends in part on the correlations
among the so-called factors measured by the individual tests. Any one or
two of the tests provides almost as much predictive power as the entire
battery. Administering the rest of the battery is a waste.
David Hopkin (1993) of the Royal Air Force Institute of Aviation Medicine
has advanced the notion that situational awareness is a unitary though complex
human quality, above and beyond the sum of the component abilities currently
measured by psychomotor test batteries. The original WOMBAT Situational
Awareness and Stress Tolerance Test was designed to address the complexity
of this overarching unitary quality by creating a situational scenario of
similar complexity. The automatically adaptive scenario calls on a testee
to scan multiple information sources, evaluate alternatives, establish priorities,
and select and work on the alternative that has the highest priority at
the moment.
The solo WOMBAT test was designed to assess the operational aptitude of
individuals working in complex situations without regard to their interactions
with other individuals in a team or crew relationship. The latter situation
calls for additional personal attributes, primarily social in nature, that
have gained the attention of airline management and government regulators,
leading to worldwide formal training in CRM. To assess the actual benefits
of CRM training, an objective measuring instrument is needed.
To measure how well crew resources are managed, the solo WOMBAT has been
expanded into the DuoWOMBAT Crew Resource Management Test. Two testees,
sitting side-by-side at two networked WOMBAT terminals, work out joint strategies
for trading off duties to maximize the team's combined score. The 90-minute
test consists of two 30-minute phases of dual performance of the various
subtasks sandwiched between three 10-minute solo phases (10-30-10-30-10).
The three solo phases provide a learning curve for each individual to serve
as a basis against which the team's CRM performance is evaluated.
While situational complexity is necessary to test situational awareness
and crew coordination, it is not sufficient. To avoid confounding basic
aptitude with the effect of prior training in specific tasks such as flying
airplanes, the elements that comprise the test must be unlike any real-world
activities-unlike operating computers, unlike controlling any specific vehicle.
Furthermore, the individual subtasks must be simple enough to allow their
mastery in a short practice period before combining them in the test situation.
Sufficient situational complexity is achieved by the manner in which the
individually simple subtasks are combined in an adaptive scenario involving
multiple sources of information and multiple response alternatives.
The WOMBAT test is a game designed to embody all these
demands and constraints. The individual tasks involve pursuit tracking,
pattern recognition, short-term memory, and spatial orientation, and on
each a testee can reach an asymptotic performance level after a short practice
period. The 3-D tracking task is unlike anything called for in real-world
vehicle control. In a quadrant-location task, as each pattern of numbers
is learned, it is replaced by a more difficult pattern of greater scoring
worth. A two-back serial digit-cancelling task, with no real-world counterpart,
is both tedious and frustrating. The 3-D figure rotation and matching task
is a game that requires spatial orientation and calls for a self-assessment
of confidence.
These four tasks comprise the menu of scoring alternatives available to
each testee on request. Each task is relatively culture-free in that it
has no real-world counterpart, and each can be learned quickly by the apt
testee. The attention demands of the WOMBAT game are expanded by the ever
changing information presented by peripheral indicators. To score well the
testees must monitor the peripheral indicators vigilantly to follow the
shifting priorities of the various activities as indicated by their potential
scoring worths and current scoring rates and to detect indications of failure
modes that may require immediate termination of one activity in favor of
another.
The difficulty of developing tests of high predictive validity for complex
operational aptitude involves several factors, the first of which is the
usual clouding of operational performance criteria against which to validate
any such test. If measures of complex operational performance are unreliable,
as they typically are, there is no way that the high predictive validity
of a test can be shown statistically. The pass-fail criterion is virtually
useless when all operational personnel are given whatever amount of simulator
refreshment is needed for periodic recertification, and rating scales are
no better when almost everyone receives the same rating. Like CRM training,
the evaluation of CRM testing initially must be a matter of face validity.
However, the objective evaluation of CRM training and testing is not beyond
reach. By consensus, the airlines' CRM training programs have an observable
effect in the desired direction on crew behavior in the cockpit. If that
is indeed the case, the change in the individual crew member's behavior
should be reflected by improved performance on the DuoWOMBAT test (beyond
that attributable solely to taking and retaking the test). Conversely, continued
practice on the DuoWOMBAT would be expected to develop team behavioral strategies
that would readily transfer to the operational situation.
Hopkin, V. D. (1993). Situational awareness in air traffic control. Paper
presented at a conference on Situational Awareness in Complex Systems,
held in Orlando, FL, February 1-3 by the Center for Applied Human Factors
in Aviation, a joint venture of the University of Central Florida and Embry-Riddle
University.
Ottawa, ONT. - Transport Canada has announced the certification of the
world's first multi-crew, all-virtual touch screen simulator. Earlier this
year, the Ministry granted the HAWKER PILOT TRAINER a Level 2 FTD Certification,
allowing the users to log the training time on the HPT, in accordance with
the Personnel Licensing Handbook, vol. 1.
The HPT is manufactured by Hawker deHavilland Ltd. of Melbourne, Australia
and was developed in conjunction with the Royal Melbourne Institute of Technology
(RMIT) and Aero Innovation Inc. of Montreal. It is designed around proven
principles of positive transfer from research carried out at NASA AMES and
the Ohio State University.
The HPT is a fully reconfigurable simulator. It comes standard with five
generic flight models:
Each HPT is factory loaded with all flight models in one box. Other customized models can be added at any time. Each flight model has its own specific instruments and systems displayed on 8 ultra-high resolution CRTs. The instructor selects the desired aircraft model from an adaptive touch-screen menu and the on-board 486 computer loads the required cockpit configuration in 3 minutes.
The HPT offers training aids never seen before in any simulator. Named
Adaptive Augmentation System, the training system displays intelligent visual
cues to ab-initio students as well as real-time vector depiction for ground
school-flight training integration.
"It brings the classroom into the cockpit in the most efficient way,"
said Captain Kenneth Hiebert from Boeing Aircraft Group of Seatle WA, during
a demonstration in St-Hubert Airport, Canada.
"We have been able to teach new systems, such as the turbo-prop operation.
Our students can also experiment with jet speeds before they leave the college.
They reach the market much better prepared than before because of the HPT,"
said Captain Paul Mathieu, Director of the CQFA (Centre
québécois de formation aéronautique) in Chicoutimi,
Quebec. The school, the largest in Canada, operates 5 HPTs since 1992 and
their systems include an additional custom-made Beech Baron BE-55 model.
The HPT is also in use in the USA, Australia, New-Zealand, India and the
Netherlands.
The HPT is reportedly priced at U$200,000, which is, all things considered,
approximatively 70% less than the most affordable alternative.
When asked about an equivalent certification in the USA, Captain Paul Ray
of the FAA in Atlanta GA reportedly said that he believed "the touch-screen
technology had no benefit in flight training and could not be accommodate
in the actual classification the FAA is using." Captain Ray has since
retired.
For more information on the HPT, click here
to see the HPT page.
