Menu

 


Project Title:  Objective Function Allocation Method for Human-Automation/Robotic Interaction using Work Models that Compute Reduce
Images: icon  Fiscal Year: FY 2020 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/07/2016  
End Date: 10/06/2020  
Task Last Updated: 07/30/2019 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feigh, Karen  Ph.D. / Georgia Institute of Technology 
Address:  School of Aerospace Engineering 
270 Ferst Drive 
Atlanta , GA 30332-0150 
Email: karen.feigh@gatech.edu 
Phone: 404-385-7686  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Georgia Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Pritchett, Amy  Sc.D. Pennsylvania State University 
Key Personnel Changes / Previous PI: July 2019 report: We added an additional PhD student and graduated the two previous PhD students.
Project Information: Grant/Contract No. NNX17AB08G 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NNX17AB08G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions. (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?; IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F)
Flight Assignment/Project Notes: NOTE: Changed end date to 10/06/2020 per NSSC information (Ed., 10/4/19)

Task Description: To develop effective Human-Automation/Robotic (HAR) systems, NASA requires the development of methods and tools to inform the decisions regarding function allocation between robots and crew members that are able to objectively assess the implications of the assignment of these roles for the human-system performance trade space. This research will establish a validated method for the evaluation of function allocation between robots and automated systems and their human crew mates for use in deep space exploration missions. It will further produce computational models of different possible combinations of a three person human crew and various classes of robots for a variety of tasks which can be used as-is for additional analysis or modified for future concepts of operation. The method for function allocation will apply fast-time simulation, which will be validated by ground-based human-in-the-loop experimentation. It may also include human-in-the-loop simulation in an analog environment.

The proposed research addresses three main research questions: First, how should roles and responsibilities be optimally assigned to robots and humans based on a combination of task demands, robotic capabilities, and available crew resources, with special attention to the capabilities inherent to classes of robots? Second, what is the human-robot system performance trade-space that serves as the basis for the allocation? Third, how can this function allocation method be validated as creating appropriate function allocation for both nominal and off-nominal operations?

We propose a three year effort to address these questions. In the first year we propose to model the function allocation design space that exists between humans and robots in deep space exploration missions. We will use a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In the second year we will explore the design space, deeply investigating each metric such as taskload, authority-responsibility mismatch, coherency, etc., while beginning the validation process through the use of human-in-the-loop experiments with simulated robots. In the final year we will move from exploring each metric individually to looking at their combined effects as we vary the design space constraints, the tasks, crew stress levels, and function allocation options. We will continue our validation efforts using human-in-the-loop experiments using a combination of simulated robots and/or real robots. These experiments will systematically explore a large number of conditions such that they serve not only to demonstrate the function allocation chosen by the method, but also to validate the method.

Research Impact/Earth Benefits: This research has the potential to impact several fields including computational modeling of function allocation, cognitive engineering methods, and the field human-robot teaming.

First, this project uses current-day computational methods to model and simulate the human-robot teams at work. We are expanding on existing methods used in aeronautics to advance the field of computational simulation of function allocation for the improvement of crewed space exploration where we encounter additional challenges of agents with differential capabilities, time delay of communication, and the need to represent limitations in resources which might be both physical (say a wrench or oxygen) as well as informational (say the current CO2 levels). The capability to simulate how human-robot teams work together will help systems designers understand the interaction between humans and space robotics to allow for robust and effective as well as efficient teamwork across missions and different crew-robot complements. In turn, human-robot teams not only become better at doing their taskwork, but also expand the capacity of what human-robot teams can accomplish. Human-robot teams may then go on to accomplish the numerous tasks that will expand humanity's knowledge of space exploration.

Second, our research also impacts the growing field of human-robot teaming, as robots continue to advance technically and become less like tools for humans and more like peers and teammates. The computational framework and capabilities we are creating and demonstrating advance the field of cognitive engineering to investigate robot-human teaming, which is a research area applicable to domains beyond space exploration including manufacturing, healthcare, transportation, and agriculture.

Task Progress & Bibliography Information FY2020 
Task Progress: In the first year of performance we have modelled the function allocation design space that exists between humans and robots in future space exploration missions. We have extended a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers.

In the second year of performance we created models of representative multi-human/multi-robot function allocations for prototypical EVA (extravehicular activity) missions. We then applied these models to demonstrate key implications in how various modes of human-robot interaction, including the implicit requirements for monitoring inherent to leaving human agents responsible for the outcome of tasks performed by robots, the implications of different human-robot control modes, and the idling time resulting from different distribution of tasks.

In this past year, we validated our simulation results by conducting a human-in-the-loop experiment of an on-orbit maintenance scenario. We then further modeled an additional case study involving an autonomous lunar rover with kinematic and dynamic motion. Finally, we applied work strategies to both these scenarios to identify how to select function allocations.

Bibliography Type: Description: (Last Updated: 10/04/2019)  Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals IJtsma M, Ma LM, Feigh KM, Pritchett AR. "Demonstration of the “Work Models that Compute” simulation framework for objective function allocation." Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2018 Sep;62(1):321-4. (62nd Annual Meeting of the Human Factors and Ergonomics Society, Philadelphia, Pennsylvania, October 1–5, 2018.) https://doi.org/10.1177/1541931218621074 , Sep-2018
Articles in Peer-reviewed Journals IJtsma M, Ma LM, Pritchett AR, Feigh KM. "Computational methodology for the allocation of work and interaction in human-robot teams." Journal of Cognitive Engineering and Decision Making. First published online August 30, 2019. https://doi.org/10.1177/1555343419869484 [note will be part of Special Issue on Human-Machine Teaming] , Aug-2019
Papers from Meeting Proceedings IJtsma M, Ye S, Feigh KM, Pritchett AR. "Simulating Human-Robot Teamwork Dynamics for Evaluation of Work Strategies in Human-Robot Teams." Paper presented at the 20th International Symposium on Aviation Psychology, Dayton, OH, May 7-10, 2019.

20th International Symposium on Aviation Psychology, Dayton, OH, May 7-10, 2019. , May-2019

Project Title:  Objective Function Allocation Method for Human-Automation/Robotic Interaction using Work Models that Compute Reduce
Images: icon  Fiscal Year: FY 2019 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/07/2016  
End Date: 10/06/2019  
Task Last Updated: 08/16/2018 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feigh, Karen  Ph.D. / Georgia Institute of Technology 
Address:  School of Aerospace Engineering 
270 Ferst Drive 
Atlanta , GA 30332-0150 
Email: karen.feigh@gatech.edu 
Phone: 404-385-7686  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Georgia Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Pritchett, Amy  Sc.D. Pennsylvania State University 
Key Personnel Changes / Previous PI: August 2018 report: No changes to personnel.
Project Information: Grant/Contract No. NNX17AB08G 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NNX17AB08G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions. (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?; IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F)
Task Description: To develop effective Human-Automation/Robotic (HAR) systems, NASA requires the development of methods and tools to inform the decisions regarding function allocation between robots and crew members that are able to objectively assess the implications of the assignment of these roles for the human-system performance trade space. This research will establish a validated method for the evaluation of function allocation between robots and automated systems and their human crew mates for use in deep space exploration missions. It will further produce computational models of different possible combinations of a three person human crew and various classes of robots for a variety of tasks which can be used as-is for additional analysis or modified for future concepts of operation. The method for function allocation will apply fast-time simulation, which will be validated by ground-based human-in-the-loop experimentation. It may also include human-in-the-loop simulation in an analog environment.

The proposed research addresses three main research questions: First, how should roles and responsibilities be optimally assigned to robots and humans based on a combination of task demands, robotic capabilities, and available crew resources, with special attention to the capabilities inherent to classes of robots? Second, what is the human-robot system performance trade-space that serves as the basis for the allocation? Third, how can this function allocation method be validated as creating appropriate function allocation for both nominal and off-nominal operations?

We propose a three year effort to address these questions. In the first year we propose to model the function allocation design space that exists between humans and robots in deep space exploration missions. We will use a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In the second year we will explore the design space, deeply investigating each metric such as taskload, authority-responsibility mismatch, coherency, etc., while beginning the validation process through the use of human-in-the-loop experiments with simulated robots. In the final year we will move from exploring each metric individually to looking at their combined effects as we vary the design space constraints, the tasks, crew stress levels, and function allocation options. We will continue our validation efforts using human-in-the-loop experiments using a combination of simulated robots and/or real robots. These experiments will systematically explore a large number of conditions such that they serve not only to demonstrate the function allocation chosen by the method, but also to validate the method.

Research Impact/Earth Benefits: This research has the potential to impact several fields including computational modeling of function allocation, cognitive engineering methods, and the field human-robot teaming.

First, this project uses current-day computational methods to model and simulate the human-robot teams at work. We are expanding on existing methods used in aeronautics to advance the field of computational simulation of function allocation for the improvement of crewed space exploration where we encounter additional challenges of agents with differential capabilities, time delay of communication, and the need to represent limitations in resources which might be both physical (say a wrench or oxygen) as well as informational (say the current CO2 levels). The capability to simulate how human-robot teams work together will help systems designers understand the interaction between humans and space robotics to allow for robust and effective as well as efficient teamwork across missions and different crew-robot complements. In turn, human-robot teams not only become better at doing their taskwork, but also expand the capacity of what human-robot teams can accomplish. Human-robot teams may then go on to accomplish the numerous tasks that will expand humanities knowledge of space exploration.

Second, our research also impacts the growing field of human-robot teaming, as robots continue to advance technically and become less like tools for humans and more like peers and teammates. The computational framework and capabilities we are creating and demonstrating advance the field of cognitive engineering to investigate robot-human teaming, which is a research area applicable to domains beyond space exploration including manufacturing, healthcare, transportation, and agriculture.

Task Progress & Bibliography Information FY2019 
Task Progress: In the first year of performance we have modelled the function allocation design space that exists between humans and robots in future space exploration missions. We have extended a computational framework called Work Models That Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In this year of performance we have created models of representative multi-human/multi-robot function allocations for prototypical EVA (extravehicular activity) missions. We have then applied these models to demonstrate key implications in how various modes of human-robot interaction, including the implicit requirements for monitoring inherent to leaving human agents responsible for the outcome of tasks performed by robots, the implications of different human-robot control modes, and the idling time resulting from different distribution of tasks.

This second year of performance we continued our exploration of the function allocation design space, and more deeply investigated metrics of effective function allocation and human-robot teaming. Our first research effort was an elaborate study of taskload in relation to function allocation, where we studied function allocations under different imposed taskload constraints for each agent. The results highlighted how the taskload associated with teamwork requirements impacts idle time and mission duration, and it can be a driving factor in a function allocation’s effectiveness.

Our second research effort this year was the modeling and analysis of robotic failures. We have modeled robotic failures and the required response from human agents to resolve these failures. For different function allocations, we studied how the effects of failures ripple through the rest of a scenario. The results from this study showed that the effects of failures are different depending on whether the robot is aware of the failure. When the robot detects the failure, and notifies a responsible (human) agent to take action, the failed action could quickly be re-allocated to a nearby and capable agent and the resulting impact on the mission was minimal. On the other hand, if the failure is only discovered when the human operator confirms the robot’s action (thus the robot was not aware), the action needs to be re-done completely, and the impact on the mission are more apparent.

It was also clear the impact of the failure depends on how the failed action relates to others and how the actions were allocated across the agents. For example, if a failed action has many follow-up actions, they need to be delayed until the failure is resolved. These delays then have a particularly large impact if they are allocated to different agents, because all of these agents need to wait (idle time). If a failed action has no follow-up actions, other processes and actions can continue while the failure is resolved, thus resulting in minimal impact on the mission.

The third research effort was focused on incorporating human-robot teaming metrics in our methodology and simulation environment. This resulted in several new function allocation metrics associated with teamwork, including the spatial proximity of agents as they are working together and the coherency of function allocations (measured through required interaction and communication). We took an interaction model (the Scholtz model) that is widely used in the robotics community as a basis for modeling joint activity, in which two agents work together on a single task. This model allows us to evaluate communication requirements at different levels of cognition (goals, intent, perception, and evaluation). These new metrics, together with our existing metrics, have been incorporated in a postprocessing analysis script that creates reports for cross-comparing different function allocations.

Our fourth research effort is the validation of the simulation framework, which is currently ongoing. We have created an experiment plan for conducting a human-in-the-loop study, wherein we will have participants working on a simulated extra-vehicular mission. The participants will be working together with a simple robot, the Turtlebot2, which is able to perform inspections and fetch required tools. We have a total of four conditions, which vary in the function allocation between participant and robot, as well as the types of control input and monitoring that are supported by the robot. For each condition, we run a nominal case and a case wherein the robot fails one of the actions. The data gathered from this experiment will be compared to the output of our simulation framework to test whether it accurately captures the different aspects of function allocation and human-robot teaming.

Throughout these research efforts we have conducted several case studies, each extending our simulated extra-vehicular maintenance scenario. In this scenario, several panels exterior to the spacecraft need to be inspected and – if in bad condition – replaced. The aims of these case studies are to illustrate the use of our simulation framework for modeling and analyzing human-robot interaction in function allocation, including the metrics it can assess and the resulting insights that it can provide. The scenario includes six agents that can be deployed to execute the work: two human astronauts, one extra-vehicular (EV) and one intra-vehicular (IV), an RMS (remote manipulator system), a humanoid robot (Hum) that can operate inside and – at a more notional level – outside the spacecraft, a free-flying robot that can fetch tools, and the Mission Control Center (MCC). The robotic agents might need to be controlled by a human operator, depending on their capabilities per action, as well as the desired specifications of the function allocation. The scenario has been adapted for our validation experiment to include only one robot (the Turtlebot), working with the extra-vehicular and intra-vehicular astronauts.

A model of the work for this mission was created in the computational simulation framework Work Models That Compute. The work model consists of high-level function that each contain detailed descriptions of the work in the form of actions. These actions can be assigned to any agent involved in the scenario. The taskwork for the scenario includes actions associated with overhead tasks, locomotion outside the vehicle, inspection, replacement of a broken panel, and tool handling.

Bibliography Type: Description: (Last Updated: 10/04/2019)  Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Feigh KM, IJtsma M, Prichett AR, Ma L. "Computational Models and Measures of Human Robot Teaming for Space Exploration and Beyond." Robotics: Science and Systems (RSS) Workshop on Space Robotics, Pittsburgh, Pennsylvania, June 26-30, 2018.

Robotics: Science and Systems (RSS) Workshop on Space Robotics, Pittsburgh, Pennsylvania, June 26-30, 2018. , Jun-2018

Papers from Meeting Proceedings Ma LM, Ijtsma M, Feigh MK, Paladugu A, Pritchett AR. "Modelling and evaluating failures in human-robot teaming using simulation." 2018 IEEE Aerospace Conference, Big Sky, MT, March 3-10, 2018.

In: 2018 IEEE Aerospace Conference. 16 p. https://doi.org/10.1109/AERO.2018.8396581 , Mar-2018

Project Title:  Objective Function Allocation Method for Human-Automation/Robotic Interaction using Work Models that Compute Reduce
Images: icon  Fiscal Year: FY 2018 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/07/2016  
End Date: 10/06/2019  
Task Last Updated: 08/08/2017 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feigh, Karen  Ph.D. / Georgia Institute of Technology 
Address:  School of Aerospace Engineering 
270 Ferst Drive 
Atlanta , GA 30332-0150 
Email: karen.feigh@gatech.edu 
Phone: 404-385-7686  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Georgia Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Pritchett, Amy  Sc.D. Georgia Institute of Technology 
Key Personnel Changes / Previous PI: August 2017 report: No changes to personnel. One of the Co-PIs, Amy Pritchett, has relocated from Georgia Tech to Penn State where she will serve as the Head of their Aerospace Engineering Department.
Project Information: Grant/Contract No. NNX17AB08G 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NNX17AB08G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions. (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?; IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F)
Task Description: To develop effective Human-Automation/Robotic (HAR) systems, NASA requires the development of methods and tools to inform the decisions regarding function allocation between robots and crew members that are able to objectively assess the implications of the assignment of these roles for the human-system performance trade space. This research will establish a validated method for the evaluation of function allocation between robots and automated systems and their human crew mates for use in deep space exploration missions. It will further produce computational models of different possible combinations of a three person human crew and various classes of robots for a variety of tasks which can be used as-is for additional analysis or modified for future concepts of operation. The method for function allocation will apply fast-time simulation, which will be validated by ground-based human-in-the-loop experimentation. It may also include human-in-the-loop simulation in an analog environment.

The proposed research addresses three main research questions: First, how should roles and responsibilities be optimally assigned to robots and humans based on a combination of task demands, robotic capabilities and available crew resources, with special attention to the capabilities inherent to classes of robots? Second, what is the human-robot system performance trade-space that serves as the basis for the allocation? Third, how can this function allocation method be validated as creating appropriate function allocation for both nominal and off-nominal operations?

We propose a three year effort to address these questions. In the first year we propose to model the function allocation design space that exists between humans and robots in deep space exploration missions. We will use a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In the second year we will explore the design space, deeply investigating each metric such as taskload, authority-responsibility mismatch, coherency, etc., while beginning the validation process through the use of human-in-the-loop experiments with simulated robots. In the final year we will move from exploring each metric individually to looking at their combined effects as we vary the design space constraints, the tasks, crew stress levels, and function allocation options. We will continue our validation efforts using human-in-the-loop experiments using a combination of simulated robots and/or real robots. These experiments will systematically explore a large number of conditions such that they serve not only to demonstrate the function allocation chosen by the method, but also to validate the method.

Research Impact/Earth Benefits: This research has the potential to impact several fields including computational modeling of function allocation, cognitive engineering methods, and the field human-robot teaming.

First, this project uses current-day computational methods to model and simulate the human-robot teams at work. We are expanding on existing methods used in aeronautics to advance the field of computational simulation of function allocation for the improvement of manned space exploration where we encounter additional challenges of agents with differential capabilities, time delay of communication, and the need to represent limitations in resources which might be both physical (say a wrench or oxygen) as well as informational (say the current CO2 levels). The capability to simulate how human-robot teams work together will help systems designers understand the interaction between humans and space robotics to allow for robust and effective as well as efficient teamwork across missions and different crew-robot complements. In turn, human-robot teams to not only become better at doing their taskwork, but also expand the capacity of what human-robot teams can accomplish. Human-robot teams may then go on to accomplish the numerous tasks that will expand humanities knowledge of space exploration.

Second, our research also impacts the growing field of human-robot teaming, as robots continue to advance technically and become less like tools for humans and more like peers and teammates. The computational framework and capabilities we are creating and demonstrating advance the field of cognitive engineering to investigate robot-human teaming, which is a research area applicable to domains beyond space exploration including manufacturing, healthcare, transportation, and agriculture.

Task Progress & Bibliography Information FY2018 
Task Progress: In this first year of performance we have modelled the function allocation design space that exists between humans and robots in future space exploration missions. We have extended a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system), and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In this year of performance we have created models of representative multi-human/multi-robot function allocations for prototypical EVA (extravehicular activity) missions. We have then applied these models to demonstrate key implications in how various modes of human-robot interaction, including the implicit requirements for monitoring inherent to leaving human agents responsible for the outcome of tasks performed by robots, the implications of different human-robot control modes, and the idling time resulting from different distribution of tasks.

As a case study, consider an on-orbit maintenance scenario in which three panels exterior to the spacecraft need to be inspected and – if in bad condition – replaced. The aim of this case study is to illustrate the use of our simulation framework for modeling and analyzing human-robot interaction in function allocation, including the metrics it can assess and the resulting insights that it can provide. The scenario includes six agents that can be deployed to execute the work: two human astronauts, one extra-vehicular (EV) and one intra-vehicular (IV), an RMS, two humanoid robots (Hum) that can operate inside and – at a more notional level – outside the spacecraft, and the Mission Control Center (MCC). The robotic agents might need to be controlled by a human operator, depending on their capabilities per action, as well as the desired specifications of the function allocation.

The IV astronaut has access to a datalink with the humanoid robot, a datalink with RMS, and a radio-connection with the EV astronaut and the MCC. The EV astronaut has access to the same radio-link, and can additionally directly interact with the humanoid when they are working shoulder-to-shoulder. The RMS is in connection with the IV astronaut, as well as with the humanoid. Finally, MCC can talk directly to the both astronauts over the radio link. Depending on which agents are involved in the function allocation and the required information transfer, communication needs to occur over these channels.

The taskwork for the scenario includes actions associated with overhead tasks, locomotion outside the vehicle, inspection, replacement of a broken panel, and tool handling.

We tested three potential function allocations, each with different distributions of the work and different requirements for control modes and monitoring and confirmation needs. FA1 is a function allocation in which the EV astronaut directly controls a humanoid robot, the two of them working shoulder-to-shoulder to conduct the inspection and replacement of broken panels. This human-robot team is assisted by the RMS, which is being manually controlled by the IV astronaut.

For FA2, the humanoid takes over the tasks of the EV astronaut. Most actions need humans to tele-operate the robots, either through direct tele-operation or command sequencing. MCC directs the humanoid, whereas the IV astronaut can directly operate the RMS. Furthermore, we have denoted five critical actions that need to be confirmed instead of monitored: the “apply inspection tools” and the four actions associated with the replacement of a broken panel.

Finally, FA3 is a more notional function allocation, in which it is assumed that the humanoid is capable of executing tasks more independently from human operators. Thus, two humanoids, one performing the inspection and one doing the replacement of bad panels, are intermittently commanded by the IV astronaut and MCC, respectively. Humanoid I is continuously being monitored by the IV astronaut. MCC is responsible for Humanoid II, but because there might not be real-time data available for MCC, all actions of the Humanoid II are confirmed as opposed to monitored.

Results from FA1: Here we have an EV astronaut who is working shoulder-to-shoulder with a humanoid. The astronaut at times needs to directly control this humanoid to execute inspection tasks. Replacement of broken panels is conducted by the astronaut, in collaboration with the RMS, which is being controlled and monitored by an IV astronaut. The time traces show that the RMS, IV astronaut, and humanoid often need to wait for the EV astronaut to complete his/her task before they can continue their own operations.

Results from FA2: It shows MCC has a high taskload in controlling, monitoring, and confirming the operations of the humanoid. Furthermore, the IV astronaut has long periods of idling time and only occasionally needs to control and confirm the operations of the RMS. We additionally observe that the humanoid needs to occasionally wait for the MCC to provide commands. Likewise, the RMS is sometimes idling while the intra-vehicular astronaut is confirming the correct execution of the previous action.

Results from FA3: Here, two humanoids are together performing the mission, and are assumed to only occasionally need commands from human operators. The IV astronaut is responsible for the actions of Humanoid I and thus has a continuous monitoring load. MCC is confirming every action of Humanoid II, and together with the waiting for commands, this causes long idling times. Additionally, because the replacement of panels is now only executed by a single agent, the mission duration is extended.

The total number of information transfer requirements increases when moving from FA1 to FA3. The use of different communication channels is seen for the direct interaction between the EV astronaut and the humanoid, the Datalink1 for the humanoid, and Datalink2 for RMS. For FA1 we see some of the interaction associated with control and monitoring takes place through direct interaction between EV astronaut and the humanoid. For FA3, all communication takes place over the datalink channel with the two humanoids on it.

Bibliography Type: Description: (Last Updated: 10/04/2019)  Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Ma LM, Ijtsma M, Feigh MK, Pritchett RA. "Objective Function Allocation for Human-Robotic Interaction." 2017 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 23-26, 2017.

2017 NASA Human Research Program Investigators’ Workshop, Galveston, TX, January 23-26, 2017. , Jan-2017

Abstracts for Journals and Proceedings Ijtsma M, Ma LM, Pritchett RA, Feigh MK. "Work Dynamics of Taskwork and Teamwork in Function Allocation for Manned Spaceflight Operations." 19th International Symposium on Aviation Psychology, Dayton, OH, May 8-11, 2017.

19th International Symposium on Aviation Psychology, Dayton, OH, May 8-11, 2017. , May-2017

Abstracts for Journals and Proceedings Ijtsma M, Pritchett RA, Ma LM, Feigh MK. "Modeling Human-Robot Interaction to Inform Function Allocation in Manned Spaceflight Operations, Robotics: Science and Systems." Robotics: Science and Systems (RSS), Cambridge, MA, July 12-16, 2017.

Robotics: Science and Systems (RSS), Cambridge, MA, July 12-16, 2017. , Jul-2017

Papers from Meeting Proceedings Ma LM, Fong T. "Human Robot Teaming for Space Exploration." FSR 2017, 11th Conference on Field and Service Robotics, Zurich, Switzerland, September 2017.

FSR 2017, 11th Conference on Field and Service Robotics, Zurich, Switzerland, September 2017. , Sep-2017

Project Title:  Objective Function Allocation Method for Human-Automation/Robotic Interaction using Work Models that Compute Reduce
Images: icon  Fiscal Year: FY 2017 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 10/07/2016  
End Date: 10/06/2019  
Task Last Updated: 12/14/2016 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feigh, Karen  Ph.D. / Georgia Institute of Technology 
Address:  School of Aerospace Engineering 
270 Ferst Drive 
Atlanta , GA 30332-0150 
Email: karen.feigh@gatech.edu 
Phone: 404-385-7686  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Georgia Institute of Technology 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Pritchett, Amy  Sc.D. Georgia Institute of Technology 
Project Information: Grant/Contract No. NNX17AB08G 
Responsible Center: NASA JSC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.williams-1@nasa.gov 
Solicitation: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NNX17AB08G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions. (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?; IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F)
Task Description: To develop effective Human-Automation/Robotic (HAR) systems, NASA requires the development of methods and tools to inform the decisions regarding function allocation between robots and crew members that are able to objectively assess the implications of the assignment of these roles for the human-system performance trade space. This research will establish a validated method for the evaluation of function allocation between robots and automated systems and their human crew mates for use in deep space exploration missions. It will further produce computational models of different possible combinations of a three person human crew and various classes of robots for a variety of tasks which can be used as-is for additional analysis or modified for future concepts of operation. The method for function allocation will apply fast-time simulation, which will be validated by ground-based human-in-the-loop experimentation. It may also include human-in-the-loop simulation in an analog environment.

The proposed research addresses three main research questions: First, how should roles and responsibilities be optimally assigned to robots and humans based on a combination of task demands, robotic capabilities and available crew resources, with special attention to the capabilities inherent to classes of robots? Second, what is the human-robot system performance trade-space that serves as the basis for the allocation? Third, how can this function allocation method be validated as creating appropriate function allocation for both nominal and off-nominal operations?

We propose a three year effort to address these questions. In the first year we propose to model the function allocation design space that exists between humans and robots in deep space exploration missions. We will use a computational framework called Work Models that Compute (WMC), which allows us to model dynamical systems (such as space vehicles and robots), automated systems (such as the automated rendezvous and docking system) and human agents working together to achieve common goals. WMC was custom designed to model function allocation and to measure eight metrics of function allocation previously established by the proposers. In the second year we will explore the design space, deeply investigating each metric such as taskload, authority- responsibility mismatch, coherency, etc., while beginning the validation process through the use of human-in-the-loop experiments with simulated robots. In the final year we will move from exploring each metric individually to looking at their combined effects as we vary the design space constraints, the tasks, crew stress levels, and function allocation options. We will continue our validation efforts using human-in-the-loop experiments using a combination of simulated robots and/or real robots. These experiments will systematically explore a large number of conditions such that they serve not only to demonstrate the function allocation chosen by the method, but also to validate the method.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2017 
Task Progress: New project for FY2017.

Bibliography Type: Description: (Last Updated: 10/04/2019)  Show Cumulative Bibliography Listing
 
 None in FY 2017