Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Electronic Procedures for Crewed Missions Beyond Low Earth Orbit (LEO) Reduce
Fiscal Year: FY 2018 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/01/2015  
End Date: 09/30/2018  
Task Last Updated: 11/28/2018 
Download report in PDF pdf
Principal Investigator/Affiliation:   Holden, Kritina  Ph.D. / Leidos Corporation at NASA Johnson Space Center 
Address:  2101 NASA Pkwy/SF3 
Mail Code: C46 
Houston , TX 77058-3607 
Email: kritina.l.holden@nasa.gov 
Phone: 281-483-8829  
Congressional District: 22 
Web:  
Organization Type: NASA CENTER 
Organization Name: Leidos Corporation at NASA Johnson Space Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Morin, Lee  M.D., Ph.D. NASA Johnson Space Center 
Schreckenghost, Debra  M.E.E. TRACLabs, Inc. 
Greene, Maya  Ph.D. KBRwyle at NASA Johnson Space Center 
Lancaster, Jeff  Ph.D. Honeywell 
Cross, E. Vincent  Ph.D. Leidos at NASA Johnson Space Center 
Stephenson, Jerri  M.S. NASA Johnson Space Center 
Key Personnel Changes / Previous PI: December 2018 report: Jerri Stephenson joined the project in 2018 as Co-investigator in charge of eye tracker data collection and analysis. August 2017: Dr. Jeff Lancaster replaced Dr. Chris Hamblin as the Honeywell co-investigator. and E. Vincent Cross (Leidos) joined as a co-investigator. February 2017 report: Dr. Maya Greene, KBRwyle at NASA Johnson Space Center, joined the project as CoInvestigator with statistics expertise.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Solicitation / Funding Source: 2014-15 HERO NNJ14ZSA001N-Crew Health (FLAGSHIP & NSBRI) 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
(2) HCI:Risk of Inadequate Human-Computer Interaction
(3) Task:Risk of Inadequate Critical Task Design (IRP Rev E)
Human Research Program Gaps: (1) ExMC 4.01:We do not have the capability to provide a guided medical procedure system that integrates with the medical system during exploration missions (IRP Rev E)
(2) SHFE-HCI-06:We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously (IRP Rev D)
(3) SHFE-TASK-02:What model-based HF Tools can assist with the design and evaluations of spacecraft systems and task procedures?
Flight Assignment/Project Notes: NOTE: End date changed to 9/30/2018 per E. Connell (JSC HRP)--Ed., 6/25/18

NOTE: End date changed to 2/1/2018 per PI (Ed., 7/13/17)

NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/18/17)

Task Description: The concept of operations in today’s spacecraft cockpit is one in which virtually all tasks are driven by procedures. In addition, crewmembers have near constant access to ground resources and information in the performance of their tasks. As NASA once again prepares for crewed spaceflight beyond low Earth orbit (LEO), future spacecraft will require automated systems that will allow the crew to perform procedures without assistance from the ground. This change threatens to increase astronaut workload, decrease efficiency, and increase the risk of inadequate task execution if electronic procedures are not designed with proper research-based guidance.

Two ground-based investigations will be completed, leading to guidelines for designing and using electronic procedures. We will leverage and extend existing electronic procedure software and spacecraft simulations for these studies, to include the PRocedure Integrated Development Environment-PRIDE procedure authoring and execution software developed to model International Space Station (ISS) procedures, and an Orion-like electronic procedures prototype system.

The proposed work will provide electronic procedures guidelines to contribute to the Space Human Factors Engineering (SHFE) gap SHFE-HCI-06 closure via the following specific aims:

Aim 1: Determine the effect of level of automation of procedure step execution on Situation Awareness, and other human-system performance metrics.

Aim 2: In a complex, multiple-procedure scenario, determine the effect of procedure management aids (e.g., availability of task allocation information) on Situation Awareness and other human-system performance metrics.

Aim 3: Determine the effect of the level of integration of system and procedural information on Situation Awareness and other human-system performance metrics.

In study 1, we will implement and evaluate electronic procedures with three levels of automation: 1) no automation, 2) mixed automation, and 3) high automation. Subjects will perform representative system tasks using prototype displays and the electronic procedures prototype systems. Particular attention will be given to critical measures of human-automation interaction including: Situation Awareness, Usability, Workload, and Trust. The procedures will be performed using two different management aid designs.

In Study 2, levels of integration between the procedures and systems displays will be manipulated and compared experimentally; levels will range from no integration (procedures-only), to some integration (procedures and relevant system displays available only under certain conditions), to high integration (procedures and relevant systems displays always visible on the same monitor). Key metrics in this study will include Situation Awareness, Usability, Workload, and Trust. Eye-tracking data and data from a prototype functional Near-Infrared Spectroscopy sensor will also be used to assess performance.

Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, habitats, oil and gas refineries, and power plants. Candidate guidelines will also be submitted to appropriate NASA documents; for example, the Orion Display Format Standards document, NASA Human Integration Design Handbook (HIDH), and NASA-STD-3001, as applicable.

Research Impact/Earth Benefits: Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, aircraft, habitats, oil and gas refineries, and power plants. The project includes a university collaborator who will serve as liaison to an industry consortium focused on studying and improving procedure design.

Task Progress & Bibliography Information FY2018 
Task Progress: In future missions, NASA plans to venture beyond Low Earth Orbit (LEO). Communication between the spacecraft and Earth will be subject to longer delays, and in some cases, will be unavailable. These spacecraft will require systems that will allow the crew to perform procedures without assistance from Mission Control. Crewmembers will rely much more heavily on the automated, computer-based tools available within their vehicle or habitat. Given the increased autonomy from the ground, missions will likely be even more dependent on procedures for day-to-day operations and survival. Crewmembers will need to be able to independently operate procedures without missing important steps, keep track of who is working each procedure (i.e., their crewmate, or the system automation), and maintain a view into the vehicle or habitat system’s health and status, along with the effects of their actions on those systems.

Current state-of-the-art electronic procedures, such as those used in the aviation domain, can potentially offer some of the support previously provided by Mission Control. This includes assisting the crew in keeping track of the current step, allowing for some automated execution of steps, providing reminder features when multiple procedures are being worked, and providing some level of integration with system telemetry to assist the crewmember in monitoring and timely decision making. Automating functions within these systems has the potential to make crew more autonomous from Mission Control. If not implemented carefully, however, this change could increase astronaut workload, decrease efficiency and situation awareness (SA), and increase the risk of suboptimal task execution (Parasuraman & Manzey, 2010; Parasuraman, Malloy, & Singh, 1993; Metzger & Parasuraman, 2005; Singh, Sharma, Parasuraman, 2001).

Two key questions addressed in the present research are: 1) When incorporating automation into electronic procedures, how much automation should be provided? 2) What is the importance of a relevant graphical system display in the execution of electronic procedures, and how important is the visual integration of that display with electronic procedures? Of primary interest is the effect of these different procedure configurations on human performance, namely SA and workload. The three aims of this research are as follows:

Aim 1: Determine the effect of level of automation of procedure step execution on SA, and other human-system performance metrics.

Aim 2: In a complex, multiple-procedure scenario, determine the effect of procedure management aids (e.g., availability of task allocation information) on SA and other human-system performance metrics.

Aim 3: Determine the effect of the level of integration of system and procedural information on SA and other human-system performance metrics.

Two studies were completed as part of this research project: Study 1 addressing level of automation of electronic procedures (Aims 1 and 2 above), and Study 2 addressing procedure/display integration (Aim 3). One of the key measures of interest in both studies was Situation Awareness, which was measured using the Situation Presence Assessment Method (SPAM; Durso, Hackworth, Truitt, Crutchfield, and Manning, 1998). SPAM uses time to answer a query about the system as a measure of situation awareness. Accuracy, workload, trust, usability and subjective preferences were also measured in both studies.

Study 1 involved twenty-seven crew-like subjects learning and performing tasks with PRocedure Integrated Development Environment (PRIDE) procedures and a habitat simulation system developed by TRACLabs. Subjects completed the scenario-based procedures while seated at a workstation intended to simulate a Habitat Control Station. The station consisted of two computers: the Procedures computer used to complete electronic procedures using PRIDE, and the Mission Control Center (MCC) computer where subjects answered computer-based questions from a mock Mission Control Center throughout the test session. MCC questions were Situation Awareness queries or simple progress questions (distractors). For the within-subjects study, subjects completed the procedures under conditions of: 1) no automation, 2) mixed automation, and 3) high automation. Another experimental factor was the location of a procedure management aid (always visible or on request).

Study 2 involved twenty crew-like subjects learning and performing tasks with an electronic procedures system and a flight control simulation. Subjects completed the scenario-based procedures, which included responding to alarms, performing malfunction procedures, and monitoring spacecraft launches, while seated at a workstation intended to simulate a future space vehicle control station. The station consisted of two computers: the Procedures computer used to complete electronic procedures using Orion-like procedures developed by the Crew Interface Rapid Prototyping Laboratory, and a Mission Control Center (MCC) computer where the subjects answered computer-based questions from a mock Mission Control Center throughout the test session. MCC questions were Situation Awareness queries or Bedford workload rating scales. For the within-subjects study, subjects completed the procedures under one of the following configurations: 1) procedures-only (no system display, telemetry only), 2) serial procedures (procedure was shown with an associated system display in toggle fashion, only for commands or on request), and 3) simultaneous procedures (procedure and related system display were shown on the same monitor). In addition to accuracy, workload, trust, usability and subjective preferences, performance data were collected from an eyetracker and a prototype functional Near Infrared Spectroscopy (fNIRS) sensor.

Results have implications for the design of electronic procedures systems, including use of automation and the role of systems displays. Results from both studies are currently being prepared for submittal to a technical journal.

References

Parasuraman, R., & Manzey, D. H. (2010). Complacency and bias in human use of automation: An attentional integration. Human factors, 52(3), 381-410.

Parasuraman, R., Molloy, R., & Singh, I. L. (1993). Performance consequences of automation-induced 'complacency'. The International Journal of Aviation Psychology, 3(1), 1-23.

Metzger, U., & Parasuraman, R. (2005). Automation in future air traffic management: Effects of decision aid reliability on controller performance and mental workload. Human Factors, 47(1), 35-49.

Singh, I. L., Sharma, H. O., & Parasuraman, R. (2001). Effects of manual training and automation reliability on automation induced complacency in flight simulation task. Psychological Studies-University of Calicut, 46(1/2), 21-27.

Durso, F.T., Hackworth, C.A., Truitt, T., Crutchfield, J., and Manning, C.A. (1998). Situation awareness as a predictor of performance in en route air traffic controllers, Air Traffic Quarterly, 6, pp. 1-20.

Bibliography Type: Description: (Last Updated: 08/31/2018) 

Show Cumulative Bibliography Listing
 
 None in FY 2018
Project Title:  Electronic Procedures for Crewed Missions Beyond Low Earth Orbit (LEO) Reduce
Fiscal Year: FY 2017 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/01/2015  
End Date: 09/30/2018  
Task Last Updated: 02/03/2017 
Download report in PDF pdf
Principal Investigator/Affiliation:   Holden, Kritina  Ph.D. / Leidos Corporation at NASA Johnson Space Center 
Address:  2101 NASA Pkwy/SF3 
Mail Code: C46 
Houston , TX 77058-3607 
Email: kritina.l.holden@nasa.gov 
Phone: 281-483-8829  
Congressional District: 22 
Web:  
Organization Type: NASA CENTER 
Organization Name: Leidos Corporation at NASA Johnson Space Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Hamblin, Christopher  Ph.D. Honeywell International 
Morin, Lee  M.D., Ph.D. NASA Johnson Space Center 
Schreckenghost, Debra  M.E.E. TRACLabs, Inc. 
Greene, Maya Ravid Ph.D. KBRwyle at NASA Johnson Space Center 
Key Personnel Changes / Previous PI: February 2017 report: Dr. Maya Greene, KBRwyle at NASA Johnson Space Center, joined the project as CoInvestigator with statistics expertise.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Solicitation / Funding Source: 2014-15 HERO NNJ14ZSA001N-Crew Health (FLAGSHIP & NSBRI) 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
(2) HCI:Risk of Inadequate Human-Computer Interaction
(3) Task:Risk of Inadequate Critical Task Design (IRP Rev E)
Human Research Program Gaps: (1) ExMC 4.01:We do not have the capability to provide a guided medical procedure system that integrates with the medical system during exploration missions (IRP Rev E)
(2) SHFE-HCI-06:We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously (IRP Rev D)
(3) SHFE-TASK-02:What model-based HF Tools can assist with the design and evaluations of spacecraft systems and task procedures?
Flight Assignment/Project Notes: NOTE: End date changed to 9/30/2018 per E. Connell (JSC HRP)--Ed., 6/25/18

NOTE: End date changed to 2/1/2018 per PI (Ed., 7/13/17)

NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/18/17)

Task Description: The concept of operations in today’s spacecraft cockpit is one in which virtually all tasks are driven by procedures. In addition, crewmembers have near constant access to ground resources and information in the performance of their tasks. As NASA once again prepares for manned spaceflight beyond low Earth orbit (LEO), future spacecraft will require automated systems that will allow the crew to perform procedures without assistance from the ground. This change threatens to increase astronaut workload, decrease efficiency, and increase the risk of inadequate task execution if electronic procedures are not designed with proper research-based guidance.

Two ground-based investigations will be completed, leading to guidelines for designing and using electronic procedures. We will leverage and extend existing electronic procedure software and spacecraft simulations for these studies, to include the PRocedure Integrated Development Environment-PRIDE procedure authoring and execution software developed to model International Space Station (ISS) procedures, and an Orion-like electronic procedures prototype system.

The proposed work will provide electronic procedures guidelines to contribute to the Space Human Factors Engineering (SHFE) gap SHFE-HCI-06 closure via the following specific aims:

Aim 1: Determine the effect of level of automation of procedure step execution on Situation Awareness, and other human-system performance metrics.

Aim 2: In a complex, multiple-procedure scenario, determine the effect of procedure management aids (e.g., availability of task allocation information) on Situation Awareness and other human-system performance metrics.

Aim 3: Determine the effect of the level of integration of system and procedural information on Situation Awareness and other human-system performance metrics.

In study 1, we will implement and evaluate electronic procedures with three levels of automation: 1) no automation, 2) low automation, and 3) high automation. Subjects will perform representative system tasks using prototype displays and the electronic procedures prototype systems. Particular attention will be given to critical measures of human-automation interaction including: Situation Awareness, Usability, Workload, and Trust. Subjects will perform the task in 2 sessions: session 1 with a single procedure, and session 2 with multiple procedures. Training is provided as part of session 1. The procedures will be performed using two different management aid designs.

In Study 2, levels of integration between the procedures and systems displays will be manipulated and compared experimentally; levels will range from no integration (separate displays and procedures), to high integration (relevant sections or excerpts of the display integrated with procedures). Key metrics in this study will include Situation Awareness, Usability, Workload, and Trust. Eye-tracking data will also be collected to assess eye movements.

Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, habitats, oil and gas refineries, and power plants. Candidate guidelines will also be submitted to appropriate NASA documents; for example, the Orion Display Format Standards document, NASA Human Integration Design Handbook (HIDH), and NASA-STD-3001, as applicable.

Research Impact/Earth Benefits: Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, habitats, oil and gas refineries, and power plants. The project includes a university collaborator who will serve as liaison to an industry consortium focused on studying and improving procedure design.

Task Progress & Bibliography Information FY2017 
Task Progress: Software and procedures for the first session of Study 1 have been developed, and the first session of Study 1 has been run. Session 1 served as training, and was used as an additional experimental opportunity to examine the effect of "Send" command button location on Situation Awareness. Twenty-four participants have completed Session 1, and are ready to progress to the first experimental session (Session 2). The testing protocol for future sessions is currently being developed. The team has also received a sanitized version of the Orion procedure system to begin conceptualization of Study 2.

Bibliography Type: Description: (Last Updated: 08/31/2018) 

Show Cumulative Bibliography Listing
 
 None in FY 2017
Project Title:  Electronic Procedures for Crewed Missions Beyond Low Earth Orbit (LEO) Reduce
Fiscal Year: FY 2016 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/01/2015  
End Date: 10/01/2017  
Task Last Updated: 10/06/2015 
Download report in PDF pdf
Principal Investigator/Affiliation:   Holden, Kritina  Ph.D. / Leidos Corporation at NASA Johnson Space Center 
Address:  2101 NASA Pkwy/SF3 
Mail Code: C46 
Houston , TX 77058-3607 
Email: kritina.l.holden@nasa.gov 
Phone: 281-483-8829  
Congressional District: 22 
Web:  
Organization Type: NASA CENTER 
Organization Name: Leidos Corporation at NASA Johnson Space Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Hamblin, Christopher  Ph.D. Honeywell International 
Morin, Lee  M.D., Ph.D. NASA Johnson Space Center 
Schreckenghost, Debra  M.E.E. TRACLabs, Inc. 
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Whitmore, Mihriban  
Center Contact: 281-244-1004 
mihriban.whitmore-1@nasa.gov 
Solicitation / Funding Source: 2014-15 HERO NNJ14ZSA001N-Crew Health (FLAGSHIP & NSBRI) 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) Bmed:Risk of Adverse Behavioral Conditions and Psychiatric Disorders
(2) HCI:Risk of Inadequate Human-Computer Interaction
(3) Task:Risk of Inadequate Critical Task Design (IRP Rev E)
Human Research Program Gaps: (1) ExMC 4.01:We do not have the capability to provide a guided medical procedure system that integrates with the medical system during exploration missions (IRP Rev E)
(2) SHFE-HCI-06:We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously (IRP Rev D)
(3) SHFE-TASK-02:What model-based HF Tools can assist with the design and evaluations of spacecraft systems and task procedures?
Task Description: The concept of operations in today’s spacecraft cockpit is one in which virtually all tasks are driven by procedures. In addition, crewmembers have near constant access to ground resources and information in the performance of their tasks. As NASA once again prepares for manned spaceflight beyond low Earth orbit (LEO), future spacecraft will require automated systems that will allow the crew to perform procedures without assistance from the ground. This change threatens to increase astronaut workload, decrease efficiency, and increase the risk of inadequate task execution if electronic procedures are not designed with proper research-based guidance.

We propose 2 ground-based investigations that will lead to guidelines for designing and using electronic procedures. We will leverage and extend existing electronic procedure software and spacecraft simulations for these studies, to include the PRocedure Integrated Development Environment-PRIDE procedure authoring and execution software developed to model ISS procedures, and the Orion electronic procedures prototype system.

The proposed work will provide electronic procedures guidelines to contribute to the Space Human Factors Engineering (SHFE) gap SHFE-HCI-06 closure via the following specific aims:

Aim 1: Determine the effect of level of automation of procedure step execution on Situation Awareness, and other human-system performance metrics.

Aim 2: In a complex, multiple-procedure scenario, determine the effect of procedure management aids (e.g., availability of task allocation information) on Situation Awareness and other human-system performance metrics.

Aim 3: Determine the effect of the level of integration of system and procedural information on Situation Awareness and other human-system performance metrics.

Two studies will be completed as part of the proposed work. In study 1, we will implement and evaluate electronic procedures with three levels of automation: 1) no automation, 2) low automation, and 3) high automation. Subjects will perform representative system tasks, including nominal and off-nominal time-critical scenarios, using prototype displays and the electronic procedures prototype systems. Particular attention will be given to critical measures of human-automation interaction including: Situation Awareness, Usability, Workload, and Trust. Subjects will perform the task in 2 sessions: session 1 with a single procedure, and session 2 with multiple procedures. They will perform using two different management aid designs (list of current procedures and agents performing the procedures on a separate display or continuously visible).

In Study 2, levels of integration between the procedures and systems displays will be manipulated and compared experimentally; levels will range from no integration (separate displays and procedures), to high integration (relevant sections or excerpts of the display integrated with procedures). Key metrics in this study will include Situation Awareness, Usability, Workload, and Trust. Eye-tracking data will also be collected to assess eye movements.

Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, habitats, oil and gas refineries, and power plants. The project includes a university collaborator who will serve as liaison to an industry consortium focused on studying and improving procedure design. Candidate guidelines will also be submitted to appropriate NASA documents; for example, the Orion Display Format Standards document, NASA Human Integration Design Handbook (HIDH), and NASA-STD-3001, as applicable. The prototypes will be made available to the NASA Crew Interface Rapid Prototyping Laboratory for demonstration/consideration in the design of future electronic procedures capabilities for Orion.

Research Impact/Earth Benefits: Results from these studies will be applicable to a variety of domains that use electronic procedures, including space vehicles, habitats, oil and gas refineries, and power plants. The project includes a university collaborator who will serve as liaison to an industry consortium focused on studying and improving procedure design.

Task Progress & Bibliography Information FY2016 
Task Progress: New project for FY2016.

Bibliography Type: Description: (Last Updated: 08/31/2018) 

Show Cumulative Bibliography Listing
 
 None in FY 2016