Menu

 


Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2020 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/28/2020 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Tamsyn works at NASA Ames as part of San Jose University Research Foundation.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2017 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HFBP HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration (IRP Rev J)
(2) HFBP HCI:Risk of Inadequate Human-Computer Interaction (IRP Rev J)
(3) HFBP Team:Risk of Performance and Behavioral Health Decrements Due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team (IRP Rev J)
(4) MPTASK:Risk of Inadequate Mission, Process and Task Design (IRP Rev H)
Human Research Program Gaps: (1) HFBP-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies. (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?; IRP Rev G name change to HARI-02 from previous designation SHFE-HARI-02, which was IRP Rev F)
(2) HFBP-HCI-03:We need HCI guidelines (e.g., display configuration, screen-navigation) to mitigate the performance decrements and operational conditions of long duration spaceflight. (Previous title: SHFE-HCI-03) (IRP Rev H)
(3) HFBP-HCI-06:We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously. (Previous title: SHFE-HCI-06) (IRP Rev H)
(4) MPTASK-01:We need methods and tools to collect measures of missions, process, and task performance. (Previous title: SHFE-TASK-01) (IRP Rev H)
(5) MPTASK-03:We need methods and tools for planning and dynamic re-planning of crew schedules. (Previous title: SHFE-TASK-03) (IRP Rev H)
(6) Team Gap 06:We need to identify methods to support and enable multiple distributed teams to manage shifting levels of autonomy during long duration and/or distance exploration missions (IRP Rev E)
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2020 
Task Progress: As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for MCC to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 1 (4/2019 – 4/2020), this research completed a first pilot study evaluating human scheduling performance, completed a usability test assessing human scheduling strategies, and developed an analog experiment for Human Exploration Research Analog (HERA) Campaign 6. The pilot study (9 subjects) evaluated the scheduling performance on naïve users using Playbook as the scheduling platform. This is the first controlled experiment of its kind on this platform and significant effort was undertaken in order to support it (as Playbook is an operational software tool, not a research platform). Participants were naive to scheduling tasks and Playbook scheduling software. The sample was self-selected as participants volunteered to take part. The study utilized a 3x3 design. The independent variables were the number of activities to be scheduled and percentage of activities to be scheduled that had constraints. Participants received four training sessions, totaling to around 30 minutes that focused on how to use Playbook prior to experimental trials. Participants were then tasked to complete nine self-scheduling trials on an iPad. For each trial, participants answered Situation Awareness questions and completed a subjective workload assessment (NASA TLX). At the end of the experiment, participants completed a trust and usability survey (UEQ). Throughout the trial, data was collected in order to measure human performance. Initial findings were presented at the Human Research Program (HRP) Investigator Workshop (1/2020). Results suggest that increasing the number of activities decreases human performance and increases workload. Percentage of activities with constraints also significantly contributed to scheduling efficiency.

In collaboration with San Jose State Research Foundation and Center for Human Factors in Advanced Aeronautics Technology (at California State University, Long Beach), we conducted a usability test whose objective was to assess if different scheduling strategies emerged from Playbook naive users. Leveraging previously generated scheduling problems, we gave ten users three scheduling tasks, increasing in difficulty. The design of the user test was a modified Think Aloud Protocol, adapted to collect data on strategies implemented specifically. Users were not guided to select or develop a particular scheduling strategy. Seven scheduling strategies emerged from just these three scheduling problems. This is more diverse than expected. Users stuck with strategies even when aspects of the task had changed which make the current strategy no longer effective. Users with some background in scheduling (e.g., experienced registered nurse and project manager) tended to perform better in terms of completion time. Lessons learned will be incorporated into the future experiment.

During the Face-to-Face, VNSCOR HCAAM team agreed to participate in the HERA Campaign 6. As part of this requirement, we have requested and received NASA Johnson Space Center (JSC) Institutional Review Board (IRB) approval. We have also developed a research plan and experiment design for HERA analog as well as submitted the Science Requirement Document (SRD), software deliveries, and required procedures.

Bibliography Type: Description: (Last Updated: 07/02/2020)  Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Marquez JJ, Lee C, Torr A, Edwards T, Bresina J, Gregory K. "Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies." 2020 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 27-30, 2020.

Abstracts. 2020 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 27-30, 2020. , Jan-2020

Articles in Peer-reviewed Journals Marquez JJ, Hillenius S, Zheng J, Deliz I, Kanefsky B, Gale J. "Designing for astronaut-centric planning and scheduling aids." Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2019 Sep;63(1):468-9. (63rd International Annual Meeting of the Human Factors and Ergonomics Society, Seattle, WA, October 2019.) https://doi.org/10.1177/1071181319631386 , Sep-2019
Papers from Meeting Proceedings Marquez JJ, Hillenius S, Healy M, Silva-Martinez J. "Lessons Learned from International Space Station Crew Autonomous Scheduling Test." 11th International Workshop for Planning and Scheduling for Space, Berkeley, CA, July 8-10, 2019.

11th International Workshop for Planning and Scheduling for Space, Berkeley, CA, July 8-10, 2019. Paper ARC-E-DAA-TN70121. , Jul-2019

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2019 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/05/2020 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Hillenius, Steven  M.S. NASA Ames Research Center 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2017 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HFBP HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration (IRP Rev J)
(2) HFBP HCI:Risk of Inadequate Human-Computer Interaction (IRP Rev J)
(3) HFBP Team:Risk of Performance and Behavioral Health Decrements Due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team (IRP Rev J)
(4) MPTASK:Risk of Inadequate Mission, Process and Task Design (IRP Rev H)
Human Research Program Gaps: (1) HFBP-HARI-02:We need to develop design guidelines for effective human-automation-robotic systems in operational environments that may include distributed, non-colocated adaptive mixed-agent teams with variable transmission latencies. (Previously: How can performance, efficiency, and safety guidelines be developed for effective information sharing between humans and automation, such that appropriate trust and situation awareness is maintained?; IRP Rev G name change to HARI-02 from previous designation SHFE-HARI-02, which was IRP Rev F)
(2) HFBP-HCI-03:We need HCI guidelines (e.g., display configuration, screen-navigation) to mitigate the performance decrements and operational conditions of long duration spaceflight. (Previous title: SHFE-HCI-03) (IRP Rev H)
(3) HFBP-HCI-06:We need guidelines to ensure crewmembers receive all of the information required to accomplish necessary tasks in a timely fashion, even when operating autonomously. (Previous title: SHFE-HCI-06) (IRP Rev H)
(4) MPTASK-01:We need methods and tools to collect measures of missions, process, and task performance. (Previous title: SHFE-TASK-01) (IRP Rev H)
(5) MPTASK-03:We need methods and tools for planning and dynamic re-planning of crew schedules. (Previous title: SHFE-TASK-03) (IRP Rev H)
(6) Team Gap 06:We need to identify methods to support and enable multiple distributed teams to manage shifting levels of autonomy during long duration and/or distance exploration missions (IRP Rev E)
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2019 
Task Progress: New project for FY2019.

[Note added to Task Book in February 2020 when received period of performance information]

Bibliography Type: Description: (Last Updated: 07/02/2020)  Show Cumulative Bibliography Listing
 
 None in FY 2019