Menu

 

The NASA Task Book
Advanced Search     

Project Title:  HCAAM VNSCOR: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasure Reduce
Images: icon  Fiscal Year: FY 2021 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/17/2021 
Download report in PDF pdf
Principal Investigator/Affiliation:   Robinson, Stephen K. Ph.D. / University of California, Davis 
Address:  Department of Mechanical & Aerospace Engineering 
1 Shields Avenue, 2132 Bainer Hall 
Davis , CA 95616-5270 
Email: stephen.k.robinson@ucdavis.edu 
Phone: 530-752-9541  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: University of California, Davis 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Karasinski, John  M.S. NASA Ames Research Center 
Marquez, Jessica  Ph.D. NASA Ames Research Center 
Key Personnel Changes / Previous PI: February 2021 report: Jessica Marquez, Ph.D., is now CoInvestigator (CoI). Steven Hillenius and Richard Joyce are no longer CoIs on the project.
Project Information: Grant/Contract No. 80NSSC19K0657 
Responsible Center: NASA JSC 
Grant Monitor: Whitmire, Alexandra  
Center Contact:  
alexandra.m.whitmire@nasa.gov 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: 80NSSC19K0657 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HSIA:Risk of Adverse Outcome Due to Inadequate Human Systems Integration Architecture (IRP Rev L)
(2) Team:Risk of Performance and Behavioral Health Decrements Due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team (IRP Rev F)
Human Research Program Gaps: (1) HSIA-201:We need to evaluate the demands of future exploration habitat/vehicle systems and mission scenarios (e.g. increased automation, multi-modal communication) on individuals and teams, and determine the risks these demands pose to crew health and performance (IRP Rev L)
(2) HSIA-401:We need to determine how HSI can be applied in the vehicle/habitat and computer interface Design Phase to mitigate potential decrements in operationally-relevant performance (e.g. problem-solving, execution procedures), during increasingly earth-independent, future exploration missions (including in-mission and at landing) (IRP Rev L)
(3) HSIA-501:We need to determine how HSI will be used in the development of dynamic and adaptive mission procedures and processes, to mitigate individual and team performance decrements during increasingly earth-independent, future exploration missions (including in-mission and at landing) (IRP Rev L)
(4) Team-105:We need to identify a set of countermeasures to support team function and enable multiple distributed teams to manage shifting levels of autonomy for all phases of increasingly earth independent, long duration exploration missions (IRP Rev L)
Flight Assignment/Project Notes: NOTE: End date changed to 4/14/2023 per NSSC information (Ed., 1/26/21)

NOTE: End date changed to 3/14/2020 per NSSC information (Ed., 1/22/2020)

Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

Future long duration exploration missions (LDEM) conducted by NASA will have an increased need for crew autonomy during routine and emergency procedures, due to the increased distance from Earth causing time delays in communications. Presently, many procedures are completed with constant communication between the crewmembers and mission control personnel. This need for increased autonomy will lead to a need for more information being stored on board and accessed by crewmembers in a timely and context appropriate manner during procedural execution. Emergent technologies in multimodal interaction such as augmented reality (AR) visual displays, spatial audio, and tactile feedback are likely to play a role in mitigating this need, leading to what we define as "enhanced electronic procedures." In this proposal we outline a research study which will use a multimodal enhanced electronic procedure to determine the best tasks and cues to pair with sensory channels for procedural execution tasks. Past efforts by our group have investigated procedural tasks using new technologies such as augmented reality and haptic cues. A ground-based research study will determine the effects of crew performance, situational awareness, and trust with the use of multimodal enhanced electronic procedures compared to traditional unimodal electronic procedures. The results of the ground-based study will lead to deployment in an analog mission for validation in a flight-like environment. From the lab and analog results, we will formulate recommendations for updated standards and guidelines for multimodal interaction and electronic procedures.

Research Impact/Earth Benefits: This research aims to re-define the meaning of "procedures" for astronauts performing complex task in space.

Traditionally, procedures are static, non-responsive documents that serve as passive instructions or recipes for astronauts to follow while performing a pre-planned task. The current research adds sensors and responsive procedure-viewer technology to allow a dynamic feedback loop to aid the astronaut in being certain that the procedures are being executed correctly.

Since procedures are very common in safety-critical tasks here on Earth (operating rooms, nuclear power plants, airliners, etc.), the results of this NASA research are likely to benefit a broad range of society on Earth.

Task Progress & Bibliography Information FY2021 
Task Progress: Materials, Methods, and Accomplishments: The HCAAM Enhanced Procedures research team has spent Year 1&2 preparing for Human Exploration Research Analog (HERA) Campaign 6, now scheduled to launch in September 2021. Accomplishments include:

• We are utilizing a COTS (commercial off the shelf) electrical generator as a spacecraft subsystem analog which subjects are asked to partially disassemble and then reassemble using both traditional and experimentally-enhanced procedures. During Year 1&2, we finalized our experiment system architecture for serving, displaying, and interacting with the enhanced procedure viewer. Raspberry Pi and sensors are now built into the customized Honda generator to monitor the state of the system and feed step-specific data to the enhanced procedures.

• Enhanced Procedure Viewer -- We have developed a novel Enhanced Procedure Viewer (EPV). This EPV integrates dynamic data from sensors directly into the procedure in real time, providing the crewmember enhanced situational awareness and progress tracking. The green and red sensor data statements allow the participant to know when a step has been completed correctly, avoiding the need for them to go back later to fix a missed step. Other enhancements include on-when-needed laser indicators to highlight items on the physical generator, caution/warning statements in the procedures, and videos on complex steps to provide additional context and support for the crewmember. Navigation of the procedure via up and down arrow buttons and a moving blue step indicator help to keep track of procedure progress and reduce errors from missed steps.

• Despite severe pandemic limitations for working in team settings or traveling, Year 1&2 has seen the Multimodal team and the NASA HERA support team work closely together to prepare for Campaign 6. At this point, two fully-modified generators have been shipped to NASA Johnson Space Center (JSC), have been assembled per a detailed assembly guide that we produced, and have been tested by HERA support engineers while on zoom calls with University of California (UC) Davis research staff. We also have a third modified generator at UC Davis ready to be shipped to HERA after planned pilot studies are complete, prior to Campaign 6.

Results: A series of small-scale pilot studies have been conducted at UC Davis during the development of the sensorized spaceflight analogs and the new Enhanced Procedure Viewer. As an example of the type of data output from the EPV execution log, we can plot the elapsed time spent on each step of the procedure.

Discussion: Pandemic-related schedule delays for the start of HERA Campaign 6 have actually been beneficial to the anticipated scientific value of the Multimodal HCAAM experiment. We have been able to use the extra time to run additional cycles of the design/test process for the IoT sensor system and the Enhanced Procedure Viewer.

Next Steps: Year 3 will see additional preparation steps plus execution of the first two missions of HERA Campaign 6, starting in September 2021. These steps include:

• Since our HERA experiment design combines four different factors in the enhancement of the generator procedures, we will conduct four human-subject experiment pilot studies at UC Davis to consider the relative influence of each of these factors alone, plus a control study with no procedure enhancements. Each study will include 16-20 subjects and will be conducted at UC Davis using detailed no-contact experimental protocols that we have developed for Covid safety. We anticipate that the control experiment plus two of the four single-effect experiments will be completed during Year 3. (These experiments are not required to be complete prior to Campaign 6, but will be necessary to analyze crew data collected in HERA.)

• Although we will have three identical experimental spacecraft analogs (modified generators) at HERA during Campaign 6, we are developing a plan to replace, repair, and re-introduce generators as sensor systems fail. This plan will utilize NASA JSC interns as repair volunteers, trained and supervised by current UC Davis students.

• With the recent announcement of the slip of Campaign 6 to September 2021, we are rebuilding our logistics support plan for funding, travel, and balancing student research responsibilities with the demands of coursework and required academic projects.

• Of course, our primary objective during Year 3 is the collecting and analyzing C6 experimental data.

Bibliography Type: Description: (Last Updated: ) 

Show Cumulative Bibliography Listing
 
 None in FY 2021
Project Title:  HCAAM VNSCOR: Enabling Autonomous Crew Task Performance with Multimodal Electronic Procedure Countermeasure Reduce
Images: icon  Fiscal Year: FY 2019 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 01/14/2020  
Task Last Updated: 03/29/2019 
Download report in PDF pdf
Principal Investigator/Affiliation:   Robinson, Stephen K. Ph.D. / University of California, Davis 
Address:  Department of Mechanical & Aerospace Engineering 
1 Shields Avenue, 2132 Bainer Hall 
Davis , CA 95616-5270 
Email: stephen.k.robinson@ucdavis.edu 
Phone: 530-752-9541  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: University of California, Davis 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Hillenius, Steven  M.S. NASA Ames Research Center 
Joyce, Richard  Ph.D. San Jose State University Research Foundation 
Karasinski, John  M.S. NASA Ames Research Center 
Project Information: Grant/Contract No. 80NSSC19K0657 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: 80NSSC19K0657 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HSIA:Risk of Adverse Outcome Due to Inadequate Human Systems Integration Architecture (IRP Rev L)
(2) Team:Risk of Performance and Behavioral Health Decrements Due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team (IRP Rev F)
Human Research Program Gaps: (1) HSIA-201:We need to evaluate the demands of future exploration habitat/vehicle systems and mission scenarios (e.g. increased automation, multi-modal communication) on individuals and teams, and determine the risks these demands pose to crew health and performance (IRP Rev L)
(2) HSIA-401:We need to determine how HSI can be applied in the vehicle/habitat and computer interface Design Phase to mitigate potential decrements in operationally-relevant performance (e.g. problem-solving, execution procedures), during increasingly earth-independent, future exploration missions (including in-mission and at landing) (IRP Rev L)
(3) HSIA-501:We need to determine how HSI will be used in the development of dynamic and adaptive mission procedures and processes, to mitigate individual and team performance decrements during increasingly earth-independent, future exploration missions (including in-mission and at landing) (IRP Rev L)
(4) Team-105:We need to identify a set of countermeasures to support team function and enable multiple distributed teams to manage shifting levels of autonomy for all phases of increasingly earth independent, long duration exploration missions (IRP Rev L)
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

Future long duration exploration missions (LDEM) conducted by NASA will have an increased need for crew autonomy during routine and emergency procedures, due to the increased distance from Earth causing time delays in communications. Presently, many procedures are completed with constant communication between the crewmembers and mission control personnel. This need for increased autonomy will lead to a need for more information being stored on board and accessed by crewmembers in a timely and context appropriate manner during procedural execution. Emergent technologies in multimodal interaction such as augmented reality (AR) visual displays, spatial audio, and tactile feedback are likely to play a role in mitigating this need, leading to what we define as "enhanced electronic procedures." In this proposal we outline a research study which will use a multimodal enhanced electronic procedure to determine the best tasks and cues to pair with sensory channels for procedural execution tasks. Past efforts by our group have investigated procedural tasks using new technologies such as augmented reality and haptic cues. A ground-based research study will determine the effects of crew performance, situational awareness, and trust with the use of multimodal enhanced electronic procedures compared to traditional unimodal electronic procedures. The results of the ground-based study will lead to deployment in an analog mission for validation in a flight-like environment. From the lab and analog results, we will formulate recommendations for updated standards and guidelines for multimodal interaction and electronic procedures.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2019 
Task Progress: New project for FY2019.

Bibliography Type: Description: (Last Updated: ) 

Show Cumulative Bibliography Listing
 
 None in FY 2019