NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

This project had the following two specific aims:

1. Develop a prototype display for robotics operations that integrated the electronic procedures with the displays for performing robotic tasks. The design process began with a hierarchical task analysis approach to drive out functional and information requirements for the display. Additionally, critical system states were identified using “Object-Process Methodology,” a task modeling approach that extends the task analysis to a computable level. We augmented our prototype design with the capability for automated execution of procedural steps. This prototype was integrated into the MIT (Massachusetts Institute of Technology) ISS robotic workstation simulation that has previously been used in several National Space Biomedical Research Institute (NSBRI) projects.

2. Perform a human-in-the-loop study that provided empirical evidence about the effects of the prototype system architecture, including varying levels of automation, on task performance, operator situation awareness, and mental workload. The study investigated how the allocation of procedural step execution between human operator and automation affects situation awareness (SA), mental workload, and task performance. Shifting more procedure steps to the automation may ease workload and speed task completion, but may also degrade the operator’s SA and ability to detect and resolve errant conditions. The experiment scenario included interrupting events that required switching between different procedures.

The project results provide a potential design method, implementation guidelines, and supporting empirical evidence for designing electronic checklists for other tasks.

Our initial efforts focused on developing a prototype electronic procedure system with new automated capabilities and integrating this capapbility into our robotic workstation simulator. First, informed by actual robotic arm procedures conducted on the International Space Station, we defined and developed a set of robotic task scenarios that includes system setup, support of a series of simulated spacewalk activities, and system shutdown. Within these scenarios, the possibility for a variety of failures and associated failure recovery steps were designed and built into the simulation so that we can test responses to non-normal situations. We used tools including Hierarchical Task Analysis (HTA) and Object-Process Methodology (OPM) to break down each task goal into its components and necessary actions and to define system states throughout the flow of the simulated robotic arm operation.

Secondly, we used the HTA and OPM analyses to design the computer code that creates the simulated integrated control panel and procedure checklist system. In doing so, we created a new type of human-machine electronic procedure interface that integrates automation and system information into an electronic display. By integrating a functional electronic control panel, procedure, and checklist into our simulator, we created a system that presents information equivalent to two separate systems and laptops that are currently used on the International Space Station during robotic operations. This, in turn, reduces the required number of displays and equipment, an important feature for future space missions that will have strict mass and power constraints.

Following the development of our prototype automated electronic procedure system, we completed a human-in-the-loop experiment to investigate how the allocation of procedural step execution between the human operator and automation, or level of automation, would affect task performance, situation awareness, and mental workload. The key independent variable in our experiment was the level of automation used during task completion. The simulator was capable of allocating the automation to perform all or a subset of the procedural steps including state selection, identification verification, and confirmation. Three levels of automation were tested--Full Auto, Full Manual, and Auto Set/ Manual Verify. If the automation was tasked with doing all of the procedure steps, then the system is in Full Auto mode in which we expected low subject mental workload and high performance but also low situation awareness. In Full Manual mode, all steps were allocated to the human operator, and we expected high situation awareness but also high workload and possibly lower task performance. In the intermediate automation allocation, Auto Set/ Manual Verify, the automation was tasked with setting the states while the human operator verified the state change. In this intermediate automation level, we expected high task performance and situation awareness, and intermediate workload.

Our results are largely consistent with our initial hypotheses, that Full Automation mode results in faster task completion times with practically no errors and low subject mental workload, measured objectively with a visual secondary task and subjectively using the NASA Task Load Index (NASA-TLX). The reduction in completion time comes primarily from the automatic selection of cameras and entry of arm parameters for Space Station Remote Manipulator System (SSRMS) movement performed by the automation, rather than manually. The elimination of manual control tasks reduces the visual demands of the task and frees attentional resources for monitoring system state and task progress. This behavior was observed in the improvement of the operator’s ability to detect system events and states in the Full Manual condition. However, the use of automation also degraded the operator’s ability to comprehend of current system state and predict future states compared to Full Manual mode. The intermediate automation mode, Auto Set/Manual Verify, was generally preferred by the subjects as the best compromise to speed up task completion while maintain their situation awareness of the task and also resulted in the best overall performance across time, workload, and awareness. We believe that the findings of this research will be beneficial in the design and use of future automatable systems to be used in long duration, limited resource, and time-delayed spaceflight missions.

This project has the following two specific aims:

1. Develop a prototype display for robotics operations that integrates the electronic procedures with the displays for performing robotic tasks. The design process will begin with a hierarchical task analysis approach to drive out functional and information requirements for the display. Additionally, critical system states will be identified using a “Object-Process Methodology,” a task modeling approach that extends the task analysis to a computable level. Lessons learned from the development of aviation electronic checklists will also be considered in the design. We will augment our prototype design with the capability for automated execution of procedural steps. This prototype will be built on the MIT (Massachusetts Institute of Technology) ISS robotic workstation simulation that has previously been used in several National Space Biomedical Research Institute (NSBRI) projects.

2. Perform a human-in-the-loop study that investigates the following questions concerning design choices for the integrated display:

a. Does the prototype electronic checklist enable the same or better situation awareness during task execution while minimizing mental workload when compared to current practice?

b. What procedural steps should be allocated to human operators or to the automation for both nominal operations and off-nominal, time-critical operations? How does the reliability of the automation affect the ideal allocation of steps?

c. Does the use of automated procedural step execution increase or decrease the human operator’s information requirements when executing multiple procedures?

The project results will provide a design method, implementation guidelines, and supporting empirical evidence for designing electronic checklists for other tasks.

Secondly, we used the HTA and OPM analyses to design the computer code that creates the simulated integrated control panel and procedure checklist system. In doing so, we created a new type of human-machine electronic procedure interface that integrates automation and system information into an electronic display. By integrating a functional electronic control panel, procedure, and checklist into our simulator, we created a system that presents information equivalent to two separate systems and laptops that are currently used on the International Space Station during robotic operations. This, in turn, reduces the required number of displays and equipment, an important feature for future space missions that will have strict mass and power constraints.

The proposed project will have the following two specific aims:

1. Develop a prototype display for supporting robotics operations that integrates the electronic procedures with the displays for performing robotics tasks. The design process will begin with a hierarchical task analysis approach to drive out functional and information requirements for the display. Lessons learned from the development of aviation electronic checklists will also be considered in the design. We will also augmented our prototype design with the capability for automated execution of the procedural steps. This prototype will be built on the MIT (Massachusetts Institute of Technology) ISS robotics simulation that has previously been used in several National Space Biomedical Research Institute (NSBRI) projects.

2. Complete human-in-the-loop studies that investigate the following questions concerning design choices for the integrated display:

a. Does the prototype electronic checklist enable the same or better situation awareness during task execution while minimizing mental workload when compared to current practice?

b. What is an appropriate allocation for procedural step execution between human operator and automation for both nominal operations and off-nominal time-critical operations? How does the reliability of the automation affect the ideal allocation of steps?

c. Does the use of automated procedural step execution increase or decrease the information requirements when executing multiple procedures?

The project results will provide a design method, implementation guidelines, and supporting empirical evidence for designing electronic checklists for other tasks.