Issue 1/97
Dunedin, New Zealand. A new study on situational awareness is expected
to be published soon. Conducted by University of Otago Senior Lecturer in
Psychology Dr. David O'Hare, the study took advantage of an unusual opportunity
in January 1995 when pilots from around the world participated in the Omarama
Cup Soaring Competition in the New Zealand Alps at Omarama.
Dr. David O'Hare, Senior Lecturer, University of Otago
O'Hare's stratified experimental plan tested independent groups representing the various stages in the sequence of selection, training, and increasingly complex operations. Dr. O'Hare recruited fourteen of the competing pilots to demonstrate their situational awareness on the WOMBAT-CS test. From this group, eight participants were classified as "elite" pilots on the basis of their consistently superior performances in gliding competitions at national and international levels; some had distinguished careers as professional pilots with both military and test flying experience. The other six participants were highly experienced but without notable competitive honors. Finally, a group of twelve male nonpilots was closely matched with the pilots in age and occupational achievement to serve as experimental controls based on the New Zealand Standard Classification of Occupations.
Prior to testing the glider pilots, O'Hare investigated the relationships between WOMBAT scores of 24 nonpilot males, varying widely in age, occupational status, and their scores on four tests from the Walter Reed Performance Assessment Battery. O'Hare's findings support the original premise that WOMBAT's individual cognitive tasks and the underlying abilities they are designed to measure are relatively unimportant in the context of situational awareness. Rather, what is important is how these relatively easy and largely culture-free tasks are managed according to rules, both learned and inferred, to maximize overall performance. The elite pilots had higher WOMBAT scores than the other highly experienced but less successful pilots, and both groups scored higher than the demographically matched nonpilot controls.
Beautiful University of Otago, South Island, New Zealand
O'Hare's experiment, Cognitive Ability Determinants of Elite PIlot Performance, to be presented at OSU's Ninth International Symposium on Aviation Psychology, Columbus, Ohio (April 97, For Information & Registration, phone 1-614-292-2405, fax 1-614-292-1014 or email), provides the best evidence to date of the validity of the WOMBAT-CS in predicting distant future success as a pilot.
March 1998 Update: Dr. O'Hare's study has now been published in Human Factors, 1997, 39(4), 540-552. Click here to read the abstract.
Dr. O'Hare has begun a new series of testing on CRM issues using the
DuoWOMBAT-CS. To find more details on Dr. O'Hare's work including the study's
publication date, browse on Aero Innovation's WWW site at www.aero.ca or
reach Dr. O'Hare by eMail at
OHARE@rivendell.otago.ac.nz.
Guest Writer: Ruth Heron Ph.D., F.Erg.S., CPE
The formal birth of ergonomics was heralded by a United States Air Force technical report presenting the findings of Fitts and Jones (1947) who analyzed reports of 460 "pilot error" experiences. The use of quotes around the term "pilot error" was meant by the authors to underscore its inappropriateness as an explanation of aviation incidents and accidents; they found these to be related to the design of the interface, rather than to the pilots. For example, some controls were easily confused with one another or, because of the illogical grouping, not easily identified and, therefore, often not activated quickly enough; others were difficult to reach; still others required movement in a direction opposite to that of the population stereotype (i.e. the direction expected by the majority of the population).
Ergonomists are people who apply rigorous methodologies to the systematic analysis and evaluation of human/machine systems and their environments. Basically, they are on the hunt for design features that are incompatible with human capacities and limitations and, therefore, have potential for decreasing system efficiency. Their mission is to catch these before they become associated with costly mishaps. Back in the forties, with the Fitts and Jones report at hand, one could only conclude that aviation technology during World War II had become so complex that it was outstripping the ability of pilots to use it. A recent ergonomic examination of pilot/system interaction in highly automated aircraft, which I carried out as part of a Transport Canada study of pilot fatigue, made me think history was repeating itself.
The purpose of the study was to generate information about multitasking requirements of aircraft operation, the performance of which might suffer under conditions of pilot fatigue arising from jet lag and/or build-up of sleep debt and resulting in disruption of circadian bodily rhythms. The information sought was to serve as a basis for conceptualizing the design of laboratory studies meant to throw light on individual differences in information-processing capacity during such periods of flight. In this article, I will use it to make some points about the use of ergonomics.
Data for the study were obtained through a form of task analysis in which the investigator synthesizes outcomes of various activities. In this case, the activities comprised: review of accident and other literature; study of a videotape focussing on the pilot during a two-leg trip in an A320; study of flight deck layouts and Standard Operating Procedures (SOPs); semi-structured interviews with sixteen pilots; and flight deck observation (i) during several B767 and A320 flights, (ii) in a stationary A320, and (iii) in an A320 simulator during a day of Line Oriented Flight Training (LOFT).
An important outcome of this study is confirmation of the immense complexity of piloting today's highly automated aircraft, involving as it does heavy demands for processing of simultaneous inputs to the brain, as well as skills in computer operation, communication, planning, problem solving, operational procedures, and interpersonal relations. This multitasking nature of aircraft operation requires a well developed mental model that allows for quick, virtually automatic detection and diagnosis of failures as well as for general problem solving. Without any doubt, the last segment of flight is the most difficult in this respect, especially if the ATC calls for a different plan.
In such a case, the Flight Management System (FMS) must be reprogrammed on the basis of calculations made on the spot, while air traffic is increasing and radio channels are most active. In situations of this kind, certain aspects of the overall task (perhaps certain motor responses) must be sufficiently automated at lower brain levels to leave upper brain levels free not only for such cognitive tasks as memory retrieval, computation, problem solving, and decision-making, but also for heavy intracrew and crew/ATC communication. The fact that these activities must be carried out in a perceptually complex environment that is aurally and visually noisy compounds the difficulty.
The task analysis report I prepared pointed to the situation described above as one worthy of simulation in the laboratory because, clearly, its multitasking properties are such as to invite impairment of performance from fatigue, especially in light of the fact that the design of the crew/autopilot interface can lead to mode confusion. The term in italics has been coined to cover what pilots experience when they do not know what mode the autopilot is in, why the aircraft does not respond to their inputs, or why the autopilot has taken control.
Numerous incidents and accidents have been attributed to this phenomenon. Notable among them are (a) the 1991 Moscow incident in which the crew of an A310 struggled roller-coaster fashion over a 6-minute period through five cycles of pitch excursions, but fortunately, if not to say miraculously, recovered control, and (b) the 1994 Nagoya, Japan, crash of an A300-600 that kept pitching up to maintain an unintentionally triggered go-around and eventually stalled as crew kept applying full down elevator in attempts to get back on the glide slope.
Much has been written about mode confusion and other interface problems that inhere in highly automated aircraft. The U.S. National Transportation Safety Board (NTSB) has acknowledged their serious nature and advocated application of the well known ergonomic principle of redundancy in design. Others recommend more intensive training and/or more sensitive selection processes that weed out airline pilot candidates who do not have an adequate grasp of digital electronic technology.
The Bombardier-Canadair Regional Jet CL-65: the most recent attempt in redesigning the human-machine interface. Would this please Leonardo Da Vinci, the Master in Ergonomics?
In the January 1995 issue of Aviation Week & Space Technology, NTSB Chief Safety Inspector Smart is said to have endorsed a "more scientific approach to testing new cockpit layouts." By "more scientific approach" he may have been referring to ergonomics. But, if so, that part of the quote is incompatible with the remainder, implying as it does that ergonomics would be applied only after a new cockpit layout had been developed. Nothing fits more uncomfortably with ergonomics than the notion that the time to apply it is at the end of product development when it is too late to influence the design process and retrofitting would be too costly! The best use of an ergonomist's store of knowledge and methodological skills is made when he/she is brought in at the conceptual stage of design and is retained on an ongoing basis through drawing, mockup, simulation, prototype, field evaluation, and even manufacturing stages.
Recently, Abbott, Slotte, and Stimson (1995) reported that the FAA's Transport Airplane Directorate had chartered a human factors team to carry out an in-depth examination of ways in which design, training, crew qualifications, and operations are related to crew/autopilot interface problems of the mode-confusion kind manifested in the Nagoya accident and many others. As part of the task, the team is also mandated to make relevant recommendations including those with respect to regulatory policy and processes. A laudable effort to be sure, but, alas, with lamentable overtones. Despite what was learned from the Fitts and Jones study fifty years ago, and despite enormous advances in what is known about the discipline of ergonomics, it is technology not the needs of pilots that has again dictated the design of the interface, with intensive ergonomic study after the fact instead of during the design and development process.
References on request. Dr. Ruth Heron, a specialist in aviation ergonomics, is President of Heron Ergonomics Inc., in Vancouver BC, Canada. Contact Dr. Heron by email.
Since 1994, the Operational Control Center (OCC) of the Société de Transport de la Communauté urbaine de Montréal (STCUM) has been successfully selecting its subway controllers using WOMBAT-CS. However, it did not take long before other departments from the Société heard of the cost-reducing experiment. The WOMBAT test has since been successfully implemented with the STCUM police force's unionized communication agents.
During Spring 1996, Lt. Michel Dupuis and Mr. Denis Masson based their use of WOMBAT-CS on a comparison of a pool of applicants with a group of known, experienced communication agents who could be evaluated by both their peers and superiors.
Lieutenant Michel Dupuis, STCUM Police Force
Subway policemen traditionally seek the Communication Agent promotion for the challenge of working at the OCC. The job primarily consists of rapidly dispatching fellow policemen to incident scenes and acting as liaison officer with the surface Montréal Police Department.
To our knowledge, it is the first time WOMBAT has been accepted by a labor union as a means of assessing members for promotion eligibility. The administration has involved the union during score normalization and establishment of threshold scores, thereby guaranteeing complete transparency and mutual understanding.
The STCUM has always had its own police force inside the underground metro. The 150 agents patrol by foot the 65 stations of the metro, armed only with a baton because of the high risk of colateral casualties when firearms are discharged inside concrete tunnels. A close working relationship is maintained with the surface Montréal Police Department which is called on the scene to complete the arrests.
The STCUM's Police Communication Agents work in teams of three during day and evening shifts, and teams of two during the night, silent hours. Situational Awareness is a crucial skill for the agents, who can quickly find themselves managing incidents involving thousands of commuters, employees and police force members.
Read more about the STCUM Operational Control Center and WOMBAT-CS in AeroNews 2/94.
|| Home page || AeroNews Index ||