This website could be intermittent Saturday Mar 30, 2024 starting 7PM until next day 11AM Eastern Time due to server/facility maintenance. We apologize for any inconvenience.

 

Menu

 

The NASA Task Book
Advanced Search     

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2024 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 09/30/2025  
Task Last Updated: 02/27/2024 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Karasinski, John  Ph.D. NASA Ames Research Center 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Edwards works at NASA Ames as part of San Jose University Research Foundation. February 2021 report: Dr. John Karasinski is now a Co-I. February 2022 report: Dr. Edwards has left NASA. March 2023 report: Dr. Karasinski has changed affiliations. February 2024 report: Dr. Karasinski has changed affiliations.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Whitmire, Alexandra  
Center Contact:  
alexandra.m.whitmire@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:
No. of Master's Candidates:  
No. of Bachelor's Candidates:
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Flight Assignment/Project Notes: NOTE: End date changed to 09/30/2025 per A. Beitman/JSC (Ed., 2/21/23)

Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule, i.e., reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, novice human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2024 
Task Progress: As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight how to best manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule—that is, reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance has been characterized, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs, and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 5 (4/2023 – 4/2024), our research team focused on NASA Human Exploration Research Analog Campaign 6 (HERA C6) research task, where we are assessing the countermeasures developed in previous years. As part of HERA C6 work, this past year we finished data collection, completed post-processing of data, and conducted statistical analysis. In previous years, we had started transcribing voice recordings to gain insight into timeline preference meetings and self-scheduling sessions. Our analysis revealed several interesting insights: 1) no usability issue of concern was encountered by the crew, 2) time spent discussing scheduling preferences decreased over time, 3) the crew collaborated and discussed preferences during self-scheduling, and 4) increased off-topic discussions while self-scheduling correlated to lower workload. With respect to human performance, we did not find significant differences due to the presence of countermeasure aid in time on task, number of violations created, or workload. However, in missions where the countermeasure was available, the trend indicates a decrease in time on task and the number of violations created, suggesting that the countermeasure aids provide assistance during self-scheduling. Evaluation of a novel, psychometric measure “plan goodness” was completed and the survey was adapted. A portion of Year 5 has also been spent preparing for HERA Campaign 7 (C7). Like HERA C6, participants will conduct self-scheduling. Notably, participants will be self-scheduling two days each, resulting in 18% of the mission being scheduled by the crew. We also have added trust measures and used our improved plan goodness survey.

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
Abstracts for Journals and Proceedings Marquez JJ, Zheng J, Shelat S, Karasinski JA, Bresina J. "Crew autonomy through self-scheduling: guidelines for crew scheduling performance envelope and mitigation strategies." 2024 NASA Human Research Program Investigators' Workshop, Galveston, Texas, February 13-16, 2024.

Presentation. 2024 NASA Human Research Program Investigators' Workshop, Galveston, Texas, February 13-16, 2024. , Feb-2024

Articles in Peer-reviewed Journals Marquez JJ, Landon LB, Salas E. "The next giant leap for space human factors: The opportunities." Hum Factors. 2023 Sep;65(6):1279-88. https://doi.org/10.1177/00187208231174955 ; PubMed PMID: 37246369 , Sep-2023
Articles in Peer-reviewed Journals Landon LB, Marquez JJ, Salas E. "Human factors in spaceflight: New progress on a long journey." Hum Factors. 2023 May 15:187208231170276. https://doi.org/10.1177/00187208231170276 ; PubMed PMID: 37183683 , May-2023
Articles in Peer-reviewed Journals Saint-Guillain M, Vanderdonckt J, Burny N, Pletser V, Vaquero T, Chien S, Karl A, Marquez J, Wain C, Common A, Casla IS, Jacobs, J, Meert J, Chamart C, Drouet S, Manon J. "Enabling astronaut self-scheduling using a robust advanced modelling and scheduling system: An assessment during a Mars analogue mission." Advances in Space Research. 2023 Aug 15;72(4). https://doi.org/10.1016/j.asr.2023.03.045 , Aug-2023
Papers from Meeting Proceedings Zheng J, Shelat S, and Marquez JJ. "Facilitating crew-computer collaboration during mixed-initiative space mission planning." SpaceCHI 3.0, Cambridge, MA, June 22-23, 2023.

SpaceCHI 3.0, Cambridge, MA, June 22-23, 2023. , Jun-2023

Papers from Meeting Proceedings Marquez JJ, Shivang S, Zheng J, Karasinski JA. "Inferring Collaboration Strategies and Usability from Remote Observations in a Spaceflight Analog Environment." 14th International Conference on Applied Human Factors and Ergonomics (AHFE), San Francisco, CA, July 20-24, 2023. .

Abstracts. 14th International Conference on Applied Human Factors and Ergonomics (AHFE), San Francisco, CA, July 20-24, 2023. , Jul-2023

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2023 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 09/30/2025  
Task Last Updated: 03/16/2023 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Karasinski, John  Ph.D. NASA Ames Research Center 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Edwards works at NASA Ames as part of San Jose University Research Foundation. February 2021 report: Dr. John Karasinski is now a Co-I. February 2022 report: Dr. Edwards has left NASA. March 2023 report: Dr. Karasinski has changed affiliations.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Whitmire, Alexandra  
Center Contact:  
alexandra.m.whitmire@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:  
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Flight Assignment/Project Notes: NOTE: End date changed to 09/30/2025 per A. Beitman/JSC (Ed., 2/21/23)

Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule, i.e., reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, novice human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2023 
Task Progress: As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule—that is, reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify the crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance has been characterized, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs, and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 4 (4/2022 – 4/2023), our research team continued analyses for our controlled lab experiment on human self-scheduling performance (completed in Year 2). In conjunction with the usability self-scheduling strategy study we did in Year 1, we identified self-scheduling strategies and heuristics used by participants to address scheduling problems. We found that, even when participants are instructed to follow a given strategy, they rapidly develop their own self-scheduling heuristics as they learn to successfully complete the task. Based on this research, future scheduling tools should support and enable the variety of emergent self-scheduling strategies. This will ensure that future astronauts may conduct self-scheduling, managing the nuanced changes in schedule and priority as they see fit. Future experiments may focus on training participants using different strategies in order to determine how these strategies quantitatively impact human performance in the scheduling task.

Based on our controlled lab experiment, we were also able to define a concrete, quantitative usability testing framework for future iterations of Playbook, our self-scheduling software platform. The dataset from this experiment can be used as a satisfaction baseline that future Playbook designs can be compared to, in order to preserve the positive user experience. Furthermore, we can now compare usability and satisfaction results from the NASA Human Exploration Research Analog (HERA) in the context of a controlled dataset.

During Year 3, we designed three countermeasure aids to support self-scheduling. In Year 4, we completed the implementation of the mixed-initiative scheduling aids and integrated them into our HERA Campaign 6 (C6) study. We deployed novel interface countermeasures and aids (“No-Go Zones” and “Suggested Fixes”) to evaluate effects on self-scheduling performance, workload, and usability in an analog setting.

By the end of Year 4, we will have collected all the data from our participation in HERA C6. Initial work has focused on post-processing data collected from Missions 1, 2, and 3. By far, the largest effort has been to analyze voice recordings. We transcribed and analyzed voice recordings during Campaign 6 to gain insight into timeline preference meetings and self-scheduling sessions. We are observing collaboration occurring during self-scheduling sessions, which was unexpected. Additionally, we are evaluating the novel psychometric called the “Plan Goodness” questionnaire, readying ourselves for refinements for HERA Campaign 7. Once all the data is collected, we will finish post-processing and analyzing HERA C6 data while preparing for HERA C7, where we will have the opportunity to increase our number of participants.

Finally, we have started the process of compiling necessary guidelines for mixed-initiative scheduling during long-duration exploration missions based on what we have learned from four years of research.

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
Abstracts for Journals and Proceedings Marquez JJ, Sullivan D, Karasinski J. "The role of trust and usability to enable spaceflight crew autonomy." 2022 ASCEND (Accelerating Space Commerce, Exploration, and New Discovery), AIAA, Las Vegas, Nevada, October 24-26, 2022.

Abstracts. 2022 ASCEND (Accelerating Space Commerce, Exploration, and New Discovery), AIAA, Las Vegas, Nevada, October 24-26, 2022. , Oct-2022

Abstracts for Journals and Proceedings Marquez JJ. "Promoting astronaut autonomy in human spaceflight missions." NASA 2022 Summer Series, Mountain View, California, August 4, 2022.

Abstracts. NASA 2022 Summer Series, Mountain View, California, August 4, 2022. https://www.nasa.gov/summerseries/2022/jessicamarquez , Aug-2022

Articles in Other Journals or Periodicals Saint-Guillain M, Vanderdonckt J, Burny N, Pletser V, Vaquero T, Chien S, Karl A, Marquez J, Karasinski J, Wain C, Comein A, Casla IS, Jacobs J, Meert J, Chamart C, Drouet S, Manon J. "Enabling astronaut self-scheduling using a robust advanced modelling and scheduling system: An assessment during a Mars analogue mission." arXiv preprint server. Posted January 14, 2023. https://doi.org/10.48550/arXiv.2301.08248 , Jan-2023
Books/Book Chapters Landon LB, Marquez JJ, Flynn-Evans E. "Spaceflight human factors: Enter the cosmos." in "Human Factors in Aviation and Aerospace." Ed. J.R. Keebler, E.L. Lazzara, K.A.Wilson, E.L. Blickensderfer. Cambridge, MA: Academic Press, 2023. p. 517-42. https://doi.org/10.1016/B978-0-12-420139-2.00010-1 , Jan-2023
Papers from Meeting Proceedings Marquez JJ, Karasinski JA, Zheng J, Bresina J, Shelat S. "Crew autonomy through self-scheduling: Guidelines for crew scheduling performance envelope and mitigation strategies." 2023 NASA Human Research Program Investigators’ Workshop, “To the Moon: The Next Golden Age of Human Spaceflight”, Galveston, TX, February 7-9, 2023.

Abstracts. 2023 NASA Human Research Program Investigators’ Workshop, “To the Moon: The Next Golden Age of Human Spaceflight”, Galveston, TX, February 7-9, 2023. , Feb-2023

Papers from Meeting Proceedings Karasinski J, Shyr MC, Torr A, Marquez JJ. "Exploring self-scheduling strategies and heuristics in novice schedulers." AIAA SciTech Forum 2023 (AIAA SciTech), National Harbor, MD, January 23-27, 2023.

Abstracts. AIAA SciTech Forum 2023 (AIAA SciTech), National Harbor, MD, January 23-27, 2023. https://doi.org/10.2514/6.2023-1067 , Jan-2023

Papers from Meeting Proceedings Shelat, S., Karasinski, J. A., Flynn-Evans, E. E., & Marquez, J. J. "Evaluation of user experience of self-scheduling software for astronauts: Defining a satisfaction baseline." 24th Human-Computer Interaction International Conference (HCII 2022), Virtual, June 26-July 1, 2022.

HCII 2022. Lecture Notes in Computer Science. In: Harris D, Li WC. eds. Engineering Psychology and Cognitive Ergonomics. Cham: Springer International Publishing, 2022. p. 433-45. https://doi.org/10.1007/978-3-031-06086-1_34 , Jun-2022

Papers from Meeting Proceedings Karasinski JA, Zheng J, Yashar M, Marquez, JJ. "Integrating mission timelines and procedures to enhance situational awareness in human spaceflight operations." SpaceCHI 2.0: Advancing Human-Computer Interaction for Space Exploration, ACM Human-Computer Interaction (CHI) 2022 Workshop, Cambridge, MA, April 30-May 5, 2022.

Abstracts. SpaceCHI 2.0: Advancing Human-Computer Interaction for Space Exploration, ACM Human-Computer Interaction (CHI) 2022 Workshop, Cambridge, MA, April 30-May 5, 2022. , May-2022

Papers from Meeting Proceedings Marquez JJ, Shelat S, Karasinski J. "Promoting crew autonomy in a human spaceflight Earth analog mission through self-scheduling." 2022 ASCEND (Accelerating Space Commerce, Exploration, and New Discovery), AIAA, Las Vegas, Nevada, October 24-26, 2022.

Abstracts. 2022 ASCEND (Accelerating Space Commerce, Exploration, and New Discovery), AIAA, Las Vegas, Nevada, October 24-26, 2022. , Oct-2022

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2022 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/15/2022 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Karasinski, John  Ph.D. NASA Ames Research Center 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Edwards works at NASA Ames as part of San Jose University Research Foundation. February 2021 report: Dr. John Karasinski is now a Co-I. February 2022 report: Dr. Edwards has left NASA.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Whitmire, Alexandra  
Center Contact:  
alexandra.m.whitmire@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:  
No. of Master's Degrees:
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, novice human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2022 
Task Progress: Task Progress Summary Year 3

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously, as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight into how to best manage their own schedules, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule—that is reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify the crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance has been characterized, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs, and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 3 (4/2021 – 4/2022), our research team completed data post-processing and analysis for our lab-based, controlled experiment evaluating human self-scheduling performance. During Year 2, data collection was completed for 31 participants for a mixed factorial design 4 x 2 x 2 experiment design. One between-subject factor is type of task (scheduling and rescheduling), while there are two within-subject factors: number of constraints (low and high) and type of constraints (Requires, Time Range, Claims, and Ordering constraints). Dependent variables include: efficiency (e.g., time on task), effectiveness (e.g., number of violations), situation awareness, workload, trust, and usability. The results point to main effects due to number of constraints and type of constraint as key factors in scheduling task complexity. However, the type of constraint affects performance in various, different manners; there is no one type of constraint that is most difficult for novice schedulers to self-schedule. Type of task (scheduling or rescheduling) had an effect but not across all performance measures. We have published one journal paper and two conference papers on some of the results from this experiment. We also have one accepted and submitted conference paper.

As part of our efforts, in Year 2 we developed and implemented an algorithm to evaluate the self-scheduling tasks in order to rank and compare against human performance measures. We refined this method in Year 3 and have validated a method for ranking difficulty of scheduling problems. Metrics generated from modeling human self-scheduling accurately ranked the difficulty of scheduling problems in a human-in-the-loop experiment. We have published one conference paper on this result.

With respect to countermeasure aid development, various aids were brainstormed based on the experimental results. From these, three aids were selected to be developed. The first aid is a visualization imbedded in the Timeline that shows where activities can be scheduled without causing violations. The second aid is a set of suggested rescheduling options that would assist schedulers in resolving any activity violations. The third aid is a mixed-initiative scheduling aid that allows for certain interconnected activities to be moved without causing violations. By the end of Year 3, all three aids will have been designed and two will be implemented.

Finally, Year 3 marked the start of Human Exploration Research Analog (HERA) Campaign 6, which was postponed for several months due to the pandemic. Our experiment in HERA focuses on validating countermeasure aids in a spaceflight analog. Each analog crewmember is assigned a day in the mission to self-schedule for the entire team. Activities and groups of activities have many diverse sets of constraints, driven by the operational needs of the analog. Aside from collecting self-scheduling performance data, including workload, crew are also asked for their subjective assessment on the quality of the timeline created for their team. HERA Campaign 6 is focused on crew autonomy, and as such, all crew are given the opportunity to conduct self-scheduling for most of the mission. By the end of Year 3, data collection for two (of four) missions will be completed. Previous research in HERA was published at a conference this year.

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
Abstracts for Journals and Proceedings Marquez JJ, Edwards T, Karasinski J, Shyr M, Bresina J, Sullivan D, Shelat S, Brandt S. "Crew autonomy through self-scheduling: Guidelines for crew scheduling performance envelope and mitigation strategies." 2022 NASA Human Research Program Investigators’ Workshop, Virtual, February 7-10, 2022.

Abstracts. 2022 NASA Human Research Program Investigators’ Workshop, Virtual, February 7-10, 2022 (Abstract #1133-000379). , Feb-2022

Articles in Peer-reviewed Journals Marquez JJ, Edwards T, Karasinski JA, Lee CN, Shyr MC, Miller CL, Brandt SL. "Human performance of novice schedulers for complex spaceflight operations timelines." Hum Factors. 2021 Dec 20. https://doi.org/10.1177/00187208211058913 ; PMID: 34886710 , Dec-2021
Papers from Meeting Proceedings Karasinski J, Bresina J, Kanefsky B, Shyr M, Marquez J. "Towards a Characterization of Scheduling Task Complexity." AIAA SCITECH 2022 Forum, San Diego, CA, January 3-7, 2022.

AIAA SCITECH 2022 Forum, San Diego, CA, January 3-7, 2022. AIAA paper AIAA 2022-1412. https://doi.org/10.2514/6.2022-1412 , Jan-2022

Papers from Meeting Proceedings Shyr M, Edwards T, Brandt S, Marquez J. "The path to crew autonomy—situational awareness in scheduling and rescheduling tasks for novice schedulers." 72nd International Astronautical Congress (IAC), Dubai, United Arab Emirates, October 25-29, 2021.

Abstracts. 72nd International Astronautical Congress (IAC), Dubai, United Arab Emirates, October 25-29, 2021 (Abstract IAC-21,B3,4-B6.4,12,x66895). , Oct-2021

Papers from Meeting Proceedings Gale JW, Yashar M, Karasinski J, Marquez J. "Evaluation of self-scheduling exercises completed by analog crewmembers in NASA’s Human Exploration Research Analog (HERA)." ASCEND 2021, Las Vegas, Nevada, November 15-17, 2021.

ASCEND 2021, Las Vegas, Nevada, November 15-17, 2021 (Abstract # AIAA-2021-4076). https://doi.org/10.2514/6.2021-4076 , Nov-2021

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2021 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/12/2021 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Karasinski, John  Ph.D. NASA Ames Research Center 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Tamsyn works at NASA Ames as part of San Jose University Research Foundation. February 2021 report: Dr. John Karasinski is now a Co-I.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Whitmire, Alexandra  
Center Contact:  
alexandra.m.whitmire@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2021 
Task Progress: TASK PROGRESS SUMMARY YEAR 2

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight how to best manage their own schedule, minimizing idle time as they wait for MCC to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule—that is reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 2 (4/2020 – 4/2021), we completed interviews with human spaceflight planning and scheduling subject matter experts (SME), designed a larger human-in-the-loop experiment, started data collection for said experiment, re-evaluated previous analog scheduling research, and continued preparations for an analog experiment for Human Exploration Research Analog (HERA) Campaign 6. Our team has published pilot study results from Year 1, preliminary results from the current experiment, presented in the Human Research Program Investigator Workshop, and contributed to two book chapters.

Seven SMEs were interviewed in order to acquire subjective assessment on what makes a plan complex to schedule and what makes a good plan for astronauts. The SMEs were experienced human spaceflight flight controllers, mostly current or recently former planners from International Space Station and Shuttle programs. Based on these interviews, our team prioritized experimental design factors for scheduling task complexity and received insight into appropriate measures for situation awareness as well as “plan goodness.”

Our main focus for Year 2 is the completion of a large human-in-the-loop experiment evaluating human scheduling performance as a function of scheduling task complexity. An experiment was designed and software was developed in order to conduct this experiment completely remotely (as required by the on-going pandemic circumstances). Our custom experimental testing platform called ESSEX (Environment for Self-Scheduling Experiment) enabled our research team to collect objective and subjective data on remote participants while completing experimental tasks on Playbook (scheduling software platform). Power analysis showed we need 30 participants for a mixed factorial design 2 x 2 x 4 experiment design. One between-subject factor is type of task (scheduling and rescheduling), while there are two within-subject factors: percent of activities with constraints (low and high) and type of activity constraints. Dependent variables include: efficiency (e.g., time on task), effectiveness (e.g., number of violations), situation awareness, workload, trust, and usability. Additionally, we have developed and implemented an algorithm to evaluate the scheduling tasks in order to rank and compare against human performance measures. While recruiting remote participants is slower than anticipated, data collection will be completed by end of Year 2.

Due to the pandemic, data collection for the experiment in HERA Campaign 6 has been postponed. Limited work on this experiment focused on: completing data sharing agreements, developing “plan goodness” survey for analog participants, and improving our data collection methods for analog environment. In lieu of this research, we conducted a retrospective, exploratory analysis of scheduling tasks performed by HERA Campaign 3 analog crewmembers. Because previous research focused on usability only, this exploratory analysis emphasized performance. Several factors have limited our analysis: inconsistent use of scheduling task complexity factors, data log inconsistencies, and missing data logs. However, we were able to identify at least three measures: time on task, number of violations left in the timeline, and subjective ratings of difficulty. We have learned that while subjective ratings are useful, they do not necessarily correlate with objective scheduling performance measures. Both are critical for the evaluation of human performance in an analog environment. Subjective ratings tend to be aligned with time on task, though there is insufficient data for statistical analysis.

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
Abstracts for Journals and Proceedings Marquez JJ, Edwards T, Lee CN, Miller C, Bresina J, Karasinski J, Brandt S. "Crew autonomy through self-scheduling: guidelines for crew scheduling performance envelope and mitigation strategies." 2021 NASA Human Research Program Investigators’ Workshop, Virtual, February 1-4, 2021.

Abstracts. 2021 NASA Human Research Program Investigators’ Workshop, Virtual, February 1-4, 2021. , Feb-2021

Articles in Peer-reviewed Journals Hambuchen K, Marquez J, Fong T. "A review of NASA human-robot interaction in space." Current Robotics Reports. 2021 Aug 7;2(3):265-72. https://doi.org/10.1007/s43154-021-00062-5 , Aug-2021
Books/Book Chapters Holden K, Marquez JJ, Vos G, Cross EV 2nd. "Human Interaction with Space-Based Systems." in "Psychology and Human Performance in Space Programs, Volume 1: Research at the Frontier." Ed. L.B. Landon, K.J. Slack, E. Salas. Boca Raton, FL: CRC Press, 2020. p. 259-94. Book doi: https://doi.org/10.1201/9780429440878 , Oct-2020
Books/Book Chapters Holden K, Vos G, Marquez JJ. "The Human Factors of Design for Spaceflight." in "Psychology and Human Performance in Space Programs, Volume 2: Extreme Application." Ed. L.B. Landon, K.J. Slack, E. Salas. Boca Raton, FL: CRC Press, 2020. p. 205-24. Book doi: https://doi.org/10.1201/9780429440854 , Oct-2020
Books/Book Chapters Edwards T, Brandt SL, Marquez JJ. "Towards a measure of situation awareness for space mission schedulers." in "Advances in Neuroergonomics and Cognitive Engineering: Proceedings of the AHFE 2021 Virtual Conferences on Neuroergonomics and Cognitive Engineering, Industrial Cognitive Ergonomics and Engineering Psychology, and Cognitive Computing and Internet of Things." Ed. H. Ayaz, U. Asgher, L. Paletta. Cham, Switzerland: Springer International Publishing, 2021. p. 39-45. https://doi.org/10.1007/978-3-030-80285-1_5 , Jul-2021
Papers from Meeting Proceedings Lee CN, Marquez JJ, Edwards T. "Crew Autonomy through Self-Scheduling: Scheduling Performance Pilot Study." AIAA SciTech Forum, Virtual Event, January 11-21, 2021.

AIAA SciTech Forum, Virtual Event, January 11-21, 2021. , Jan-2021

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2020 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/28/2020 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Edwards, Tamsyn  Ph.D. San Jose State University Research Foundation 
Key Personnel Changes / Previous PI: February 2020 report: Mr. Steven Hillenius (Co-Investigator) left NASA. Dr. Tamsyn Edwards is replacing Mr. Hillenius as Co-I. Dr. Tamsyn works at NASA Ames as part of San Jose University Research Foundation.
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits: NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. As a result of this research, we will quantify the user performance envelope for the task of planning and scheduling, which impacts many jobs both on Earth and in spaceflight. The knowledge gained from our research can be generalized to benefit our understanding on how to improve roles that require planning and scheduling, such as project planning, personnel scheduling, and operational management. Our research will also contribute to developing the next generation of planning, scheduling, and execution software tools for NASA.

Task Progress & Bibliography Information FY2020 
Task Progress: As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for MCC to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this research is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and scheduling task complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

For Year 1 (4/2019 – 4/2020), this research completed a first pilot study evaluating human scheduling performance, completed a usability test assessing human scheduling strategies, and developed an analog experiment for Human Exploration Research Analog (HERA) Campaign 6. The pilot study (9 subjects) evaluated the scheduling performance on naïve users using Playbook as the scheduling platform. This is the first controlled experiment of its kind on this platform and significant effort was undertaken in order to support it (as Playbook is an operational software tool, not a research platform). Participants were naive to scheduling tasks and Playbook scheduling software. The sample was self-selected as participants volunteered to take part. The study utilized a 3x3 design. The independent variables were the number of activities to be scheduled and percentage of activities to be scheduled that had constraints. Participants received four training sessions, totaling to around 30 minutes that focused on how to use Playbook prior to experimental trials. Participants were then tasked to complete nine self-scheduling trials on an iPad. For each trial, participants answered Situation Awareness questions and completed a subjective workload assessment (NASA TLX). At the end of the experiment, participants completed a trust and usability survey (UEQ). Throughout the trial, data was collected in order to measure human performance. Initial findings were presented at the Human Research Program (HRP) Investigator Workshop (1/2020). Results suggest that increasing the number of activities decreases human performance and increases workload. Percentage of activities with constraints also significantly contributed to scheduling efficiency.

In collaboration with San Jose State Research Foundation and Center for Human Factors in Advanced Aeronautics Technology (at California State University, Long Beach), we conducted a usability test whose objective was to assess if different scheduling strategies emerged from Playbook naive users. Leveraging previously generated scheduling problems, we gave ten users three scheduling tasks, increasing in difficulty. The design of the user test was a modified Think Aloud Protocol, adapted to collect data on strategies implemented specifically. Users were not guided to select or develop a particular scheduling strategy. Seven scheduling strategies emerged from just these three scheduling problems. This is more diverse than expected. Users stuck with strategies even when aspects of the task had changed which make the current strategy no longer effective. Users with some background in scheduling (e.g., experienced registered nurse and project manager) tended to perform better in terms of completion time. Lessons learned will be incorporated into the future experiment.

During the Face-to-Face, VNSCOR HCAAM team agreed to participate in the HERA Campaign 6. As part of this requirement, we have requested and received NASA Johnson Space Center (JSC) Institutional Review Board (IRB) approval. We have also developed a research plan and experiment design for HERA analog as well as submitted the Science Requirement Document (SRD), software deliveries, and required procedures.

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
Abstracts for Journals and Proceedings Marquez JJ, Lee C, Torr A, Edwards T, Bresina J, Gregory K. "Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies." 2020 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 27-30, 2020.

Abstracts. 2020 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 27-30, 2020. , Jan-2020

Articles in Peer-reviewed Journals Marquez JJ, Hillenius S, Zheng J, Deliz I, Kanefsky B, Gale J. "Designing for astronaut-centric planning and scheduling aids." Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2019 Sep;63(1):468-9. (63rd International Annual Meeting of the Human Factors and Ergonomics Society, Seattle, WA, October 2019.) https://doi.org/10.1177/1071181319631386 , Sep-2019
Papers from Meeting Proceedings Marquez JJ, Hillenius S, Healy M, Silva-Martinez J. "Lessons Learned from International Space Station Crew Autonomous Scheduling Test." 11th International Workshop for Planning and Scheduling for Space, Berkeley, CA, July 8-10, 2019.

11th International Workshop for Planning and Scheduling for Space, Berkeley, CA, July 8-10, 2019. Paper ARC-E-DAA-TN70121. , Jul-2019

Project Title:  HCAAM VNSCOR: Crew Autonomy through Self-Scheduling: Guidelines for Crew Scheduling Performance Envelope and Mitigation Strategies Reduce
Images: icon  Fiscal Year: FY 2019 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 04/15/2019  
End Date: 04/14/2023  
Task Last Updated: 02/05/2020 
Download report in PDF pdf
Principal Investigator/Affiliation:   Marquez, Jessica J. Ph.D. / NASA Ames Research Center 
Address:  Mail Stop 262-2, Building 262, Room 132 
Human Systems Integration Division 
Moffett Field , CA 94035 
Email: jessica.j.marquez@nasa.gov 
Phone: 650-604-6364  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Bresina, John  Ph.D. NASA Ames Research Center 
Gregory, Kevin  M.S. San Jose State University Research Foundation 
Hillenius, Steven  M.S. NASA Ames Research Center 
Zheng, Jimin  M.S. San Jose State University Research Foundation 
Project Information: Grant/Contract No. Internal Project 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Unique ID: 12720 
Solicitation / Funding Source: 2017-2018 HERO 80JSC017N0001-BPBA Topics in Biological, Physiological, and Behavioral Adaptations to Spaceflight. Appendix C 
Grant/Contract No.: Internal Project 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: None
Human Research Program Gaps: None
Task Description: This task is part of the Human Capabilities Assessments for Autonomous Missions (HCAAM) Virtual NASA Specialized Center of Research (VNSCOR).

As NASA considers long-duration exploration missions (LDEMs), it is envisioned that crew will behave more autonomously as compared to low-Earth orbit missions. In this space environment, crew will have better and more timely insight as to how best to manage their own schedule, minimizing idle time as they wait for Mission Control Center (MCC) to respond or react to a delay in activity execution. Moreover, crew must be able to self-schedule: reschedule their own timeline without creating violations. NASA currently has not characterized crew performance for self-scheduling; specifically, non-expert human performance for the task of planning and scheduling has not been characterized experimentally. The focus of this proposal is to quantify crew performance envelope for the task of planning and scheduling as a function of plan complexity, and develop mitigations that are aimed at improving performance in the face of complex planning requirements. With regards to crew performance, we will study the relationship between planning efficiency, effectiveness, crew situation awareness, trust in planning software, and plan complexity. Once a performance envelope has been identified, we will shift our research emphasis to develop and evaluate countermeasures that mitigate adverse effects on performance. These mitigations will be evaluated in analogs and recommended countermeasures will be put forward if crew performance improves as compared to the baseline. Finally, based on research results, we will recommend corresponding standards and guidelines appropriate for autonomous crew in LDEMs.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2019 
Task Progress: New project for FY2019.

[Note added to Task Book in February 2020 when received period of performance information]

Bibliography: Description: (Last Updated: 03/21/2024) 

Show Cumulative Bibliography
 
 None in FY 2019