Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Needs Assessment and Work Allocation Tools for Mission Operations and Procedures Reduce
Fiscal Year: FY 2015 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 07/24/2012  
End Date: 01/31/2015  
Task Last Updated: 02/03/2015 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feary, Michael  Ph.D. / NASA Ames Research Center 
Address:  Mail Stop: 262-4 
 
Moffett Field , CA 94035 
Email: Michael.S.Feary@nasa.gov, erin.s.connell@nasa.gov 
Phone: 650.604.0203  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Billman, Dorrit  San Jose State University Foundation at NASA Ames 
Project Information: Grant/Contract No. Directed Research 
Responsible Center: NASA JSC 
Grant Monitor: Whitmore, Mihriban  
Center Contact: 281-244-1004 
mihriban.whitmore-1@nasa.gov 
Solicitation / Funding Source: Directed Research 
Grant/Contract No.: Directed Research 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) SHFE-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions (IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F) (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?
Flight Assignment/Project Notes: NOTE: End date is now 1/31/2015 (previously 9/30/2014) per E. Connell/JSC (Ed., 9/9/14)

NOTE: End date changed to 9/30/2014 (from 7/31/2016) per M. Whitmore/JSC (Ed., 3/24/14)

Task Description: NASA has recognized that future long-duration spaceflight missions are at risk from inadequately designed Human-Automation/Robotics systems (Human Research Roadmap Risk of Inadequate Design of Automation/Robotic Integration, http://humanresearchroadmap.nasa.gov/Risks ). Further, methods are needed for identifying information needs and function allocation for work supported by Human-Automation/Robotic Integration (HARI) systems (SHFE-HARI_01, http://humanresearchroadmap.nasa.gov/Gaps/gap.aspx?i=326 ). Historically, introduction of novel technology in safety critical domains, and specifically novel automation-robotic systems, has resulted in high accident rates, most notably in aviation (e.g., National Transportation Safety Board (NTSB), 2008; Airbus, 2014). Given the extraordinary costs and risks of long-distance spaceflight, it is critical to prevent recurrence of this historical trend. The objective of this work is to develop methods to ensure effective system design of human-automation/robotic systems. Specifically, it is critical to develop methods for ensuring that technology is designed to provide the functionality needed for the work it is intended to support. Systematic methods are particularly critical where novel technology is involved and design cannot rely on copying successful solutions of the past. This task is intended to develop methods for identifying the relevant work functions, the information needed for particular functions, and how work functions should be distributed among humans and automation/robotics.

The research has an analytic and empirical strand. The purpose of the analytic strand is to provide methods and tools for measuring automation-to-work (ATW) alignment, for use guiding development and evaluation of HARI designs. The purpose of the empirical strand is to assess whether measured ATW alignment of HARI designs predicts the learnability of those designs, an important aspect of robustness. By ATW alignment we mean the correspondence between the elements and structure of interaction with the elements and structure of the work. Needs analysis identifies elements and structure of the work. In the analytic strand we will develop a scoring method for measuring alignment. This approach draws on and integrates a wide set of observations and proposals in HARI, Human Computer Interaction (HCI), Work Domain Analysis (WDA), and related disciplines. In the empirical strand we test the prediction that HARI designs that align with work more strongly will be easier to learn, particularly, easier to master using the automation for novel problem solving. We test this hypothesis by identifying and measuring designs that differ in ATW alignment and then comparing the designs with high versus low scores for how easily they are learned and how flexibly they can be used. The development of methods, tools, and data is intended to guide design and evaluation of HARI systems to ensure that such systems are fit-for-purpose, that is, it solves the correct problem. The research approach is based on identifying work needs in a manner that can guide design and evaluation, and consists of the following inter-related strands:

• Representation & Analysis Method. The team developed a method that represents a work domain in terms of the required functions within that domain, and which enables evaluating technology with respect to how well and in what respects the technology supports that body of work.

• Case Study. The team developed the method in concert with applying the analysis to a safety critical, highly automated work domain.

• Tool Prototypes. The team developed initial prototype tools to aid application of the method.

• Test environment. The team developed a medium fidelity test environment that was capable of performing the empirical assessment. The test environment (simulator is rapidly changeable and system design may be guided by those user preferences that are excessively shaped by familiarity and historical methods for accomplishing work rather than rigorous analysis of the current work needs).

• Preliminary Empirical Assessment: The team conducted an empirical assessment, which produced supporting though preliminary data supporting our approach.

The Needs Analysis method represents work as a matrix, which identifies the work functions in the domain of interest and the variables that affect or are affected by those work functions. The work functions in a work domain specify a high-level task or goal to be accomplished, in terms of the variables that are needed as input (such as information or resources) and the variables, which are affected as output of the function. For example, checking a proposed route might be a work function in supervisory control of a robot; input variables might include the proposed route, data about characteristics of the route such as terrain and distance, characteristics about the robot such as its loads and power, and context information such as other tasks slated for the robot; output variables might include modifications to the route, setting time and approval for the robot to begin the trip, and setting check points for robot to verify continuing or human to monitor robot performance.

It is expected that for many complex domains there will be clusters of variables that support a set of work functions and conversely clusters of work functions related because they draw on a related set of variables. Input and output variables provide a common language that can be used to represent device affordances as well as the work domain. Technology components or devices can also be represented in terms of these same variables: the display variables provide input to the human (input needed by the work function) and the control variables enable action by the human (output the result of the work function). Because both work and technology components can be represented in the “common language” of these variables, this enables evaluating alternative technology designs or implemented systems with respect to how well it supports the work functions in the target work domain.

Rationale for HRP Directed Research: This research is directed because it contains highly constrained research, which requires focused and constrained data gathering and analysis that is more appropriately obtained through a non-competitive proposal.

Research Impact/Earth Benefits: Reliance on automation is increasing in virtually every sphere of activity and design of how automation is integrated with the humans it serves is of critical importance. Methods we are developing for NASA to ensure a match between the automation and the work it is intended to support will be widely useful across the many other domains in which automation and robotics play an increasing role.

Task Progress & Bibliography Information FY2015 
Task Progress: In FY2014, the investigation team built a work representation of aircraft automation covering the domain of a nominal domestic passenger flight by a highly automated aircraft. Aircraft automation is an extremely useful test case for a new method because aviation allows control to range from highly automatic to highly human, is safety critical, and makes deep expertise available both from domain experts (pilots) and prior research. This domain provides a strong analog to spaceflight domains. The team built a medium fidelity flight simulator (a full cockpit, but it did have motion) a specific representation of work functions related to Air Traffic Control clearances, and alternative designs for mode control panels (MCPs), a critical part of the cockpit for autoflight aircraft. The team compared how well aligned these MCP designs were to the work domain.

As the team developed the method and applied it to the case study, the simple template work matrices were built in a spreadsheet. However, the process was not well supported by using a spreadsheet. As a result, the team developed a tool using an SQL database with a custom browser interface, which was “above and beyond” the year one deliverable specified in the research plan. The tool is designed to enable a domain expert to enter the work functions, the work variables, and their mutual relevance more easily. A second tool “above and beyond” development project concerned identifying the clustering structure implicit in a work matrix to make that structure more systematically identified and more directly presented to a user. The team also extended the bi-clustering statistical methods developed in genomics and also recruited and extended visualization methods developed there. These above and beyond tool development projects were “bonuses,” drew heavily on contributed time, and greatly benefited from collaborators, and have not been validated at this date.

The investigation team predicted that technology that is better aligned with the work domain will provide performance benefits. For example, technology that is more closely aligned with work should show better performance (e.g., easier learning), than technology that is not well aligned with work. The team tested this hypothesis in a preliminary experiment comparing learning to use two alternative designs of MCP, by pilots who were knowledgeable about much of the work domain and selected to be unfamiliar with either design. Due to a reduction in budget, the team was only able to run six participants; however, the data from these users did seem to confirm the hypothesis by showing faster learning for the better aligned MCP, and better transfer to similar but new situations.

Assessing alignment of technology with the work it is intended to support, or its fitness-for-purpose is critical to ensure sound HARI (Risk of Inadequate Design of Human and Automation/Robotic Integration) design. The method developed in this task for representations in work matrices and device matrices, using a “common language” of input and output variables has promise. The approach can be applied early in the design process, e.g., in requirement specification, and carried through to evaluation. The preliminary results provide suggestive empirical support.

Bibliography Type: Description: (Last Updated: 07/22/2015) 

Show Cumulative Bibliography Listing
 
Dissertations and Theses Stewart MJ. "Effects of flight tasks on approach minimums identification." Masters Thesis, San Jose State University, June 2014. , Jun-2014
Project Title:  Needs Assessment and Work Allocation Tools for Mission Operations and Procedures Reduce
Fiscal Year: FY 2013 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 07/24/2012  
End Date: 01/31/2015  
Task Last Updated: 05/24/2013 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feary, Michael  Ph.D. / NASA Ames Research Center 
Address:  Mail Stop: 262-4 
 
Moffett Field , CA 94035 
Email: Michael.S.Feary@nasa.gov, erin.s.connell@nasa.gov 
Phone: 650.604.0203  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Billman, Dorrit  San Jose State University Foundation at NASA Ames 
Project Information: Grant/Contract No. Directed Research 
Responsible Center: NASA JSC 
Grant Monitor: Whitmore, Mihriban  
Center Contact: 281-244-1004 
mihriban.whitmore-1@nasa.gov 
Solicitation / Funding Source: Directed Research 
Grant/Contract No.: Directed Research 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) SHFE-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions (IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F) (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?
Flight Assignment/Project Notes: NOTE: End date is now 1/31/2015 (previously 9/30/2014) per E. Connell/JSC (Ed., 9/9/14)

NOTE: End date changed to 9/30/2014 (from 7/31/2016) per M. Whitmore/JSC (Ed., 3/24/14)

Task Description: Increased system automation is critical for enabling more independent operation on long-duration and farther-distance space missions. Effective integration of human and automation elements is critical to success; this is demonstrated by the widespread involvement of poor Human-Automation Integration (HAI) design as a contributor to accidents and incidents in the sister application domain of aviation, where more data are available. While much is known about HAI design from research in aviation, long-duration space flight adds demands for robustness that are not well understood. HAI designs must be robust in supporting a variety of tasks playing out in circumstances that were impossible to fully anticipate, executed long after any training on these functions, by operators who must be competent in many types of work rather than specialists in any one.

The proposed research has an analytic and empirical strand. The purpose of the analytic strand is to provide methods and tools for measuring automation-to-work (ATW) alignment, for use guiding development and evaluation of HAI designs. The purpose of the empirical strand is to assess whether measured ATW alignment of HAI designs predicts the learnability of those designs, an important aspect of robustness. By ATW alignment we mean the correspondence between the elements and structure of interaction with the elements and structure of the work. Needs analysis identifies elements and structure of the work. In the analytic strand we will develop a scoring method for measuring alignment. Our approach draws on and integrates a wide set of observations and proposals in HAI, Human Computer Interaction (HCI), Work Domain Analysis (WDA), and related disciplines. In the empirical strand we test the prediction that HAI designs that align with work more strongly will be easier to learn, particularly, easier to master using the automation for novel problem solving. We test this hypothesis by identifying and measuring designs that differ in ATW alignment and then comparing the designs with high versus low scores for how easily they are learned, how well retained, and how flexibly they can be used.

Rationale for HRP Directed Research: This research is directed because it contains highly constrained research, which requires focused and constrained data gathering and analysis that is more appropriately obtained through a non-competitive proposal.

Research Impact/Earth Benefits: 0

Task Progress & Bibliography Information FY2013 
Task Progress: In FY2013, the investigation team developed the Automation to Work Alignment matrix for an analogous domain in aviation. The aviation domain was chosen because the team had access to different interface concepts that are designed to accomplish the same tasks with access to expert users and validated metrics.

The process of developing the matrix has produced a need for database and statistical analysis development to deal with the complexity of mapping description of work activities to descriptions of the behavior of specific technologies for completing the work. In particular the team has worked on the development of bi-clustering methods that allow sorting and grouping of related work functions and technologies. The team is currently preparing an empirical study to test the ability to use the method to assess alignment between two proposed automation interfaces.

Bibliography Type: Description: (Last Updated: 07/22/2015) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Feary M, Billman D, Chen X, Howes A, Lewis R, Sherry L, Singh S. "Linking Context to Evaluation in the Design of Safety Critical Interfaces." 15th International Conference on Human-Computer Interaction, Las Vegas, NV, July 21-26, 2013.

15th International Conference on Human-Computer Interaction, Las Vegas, NV, July 21-26, 2013. , Jul-2013

Books/Book Chapters Feary M, Billman D, Chen X, Howes A, Lewis R, Sherry L, Singh S. "Linking Context to Evaluation in the Design of Safety Critical Interfaces." in "Human-Computer Interaction. Human-Centred Design Approaches, Methods, Tools, and Environments. Part I, HCII 2013." Ed. M. Kurosu. Heidelberg : Springer, 2013. Lecture Notes in Computer Science (LNCS) series 8004, p. 193-202. http://dx.doi.org/10.1007/978-3-642-39232-0_22 , Jul-2013
Project Title:  Needs Assessment and Work Allocation Tools for Mission Operations and Procedures Reduce
Fiscal Year: FY 2012 
Division: Human Research 
Research Discipline/Element:
HRP SHFH:Space Human Factors & Habitability (archival in 2017)
Start Date: 07/24/2012  
End Date: 09/30/2014  
Task Last Updated: 08/08/2012 
Download report in PDF pdf
Principal Investigator/Affiliation:   Feary, Michael  Ph.D. / NASA Ames Research Center 
Address:  Mail Stop: 262-4 
 
Moffett Field , CA 94035 
Email: Michael.S.Feary@nasa.gov, erin.s.connell@nasa.gov 
Phone: 650.604.0203  
Congressional District: 18 
Web:  
Organization Type: NASA CENTER 
Organization Name: NASA Ames Research Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Billman, Dorrit  San Jose State University Foundation at NASA Ames 
Project Information: Grant/Contract No. Directed Research 
Responsible Center: NASA JSC 
Grant Monitor: Perchonok, Michele  
Center Contact: (281) 483-7632 
michele.perchonok22@nasa.gov 
Solicitation / Funding Source: Directed Research 
Grant/Contract No.: Directed Research 
Project Type: GROUND 
Flight Program:  
TechPort: No 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) SHFH:Space Human Factors & Habitability (archival in 2017)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) SHFE-HARI-01:We need to evaluate, develop, and validate methods and guidelines for identifying human-automation/robot task information needs, function allocation, and team composition for future long duration, long distance space missions (IRP Rev G name change to HARI-01 from previous designation SHFE-HARI-01, which was IRP Rev F) (Previously: What guidelines and tools can we develop to enable system designers and mission planners to conduct systematic task/needs analyses at the appropriate level of detail to allocate work among appropriate agents (human and automation)?
Flight Assignment/Project Notes: NOTE: End date changed to 9/30/2014 (from 7/31/2016) per M. Whitmore/JSC (Ed., 3/24/14)

Task Description: Increased system automation is critical for enabling more independent operation on long-duration and farther-distance space missions. Effective integration of human and automation elements is critical to success; this is demonstrated by the widespread involvement of poor Human-Automation Integration (HAI) design as a contributor to accidents and incidents in the sister application domain of aviation, where more data are available. While much is known about HAI design from research in aviation, long-duration space flight adds demands for robustness that are not well understood. HAI designs must be robust in supporting a variety of tasks playing out in circumstances that were impossible to fully anticipate, executed long after any training on these functions, by operators who must be competent in many types of work rather than specialists in any one.

The proposed research has an analytic and empirical strand. The purpose of the analytic strand is to provide methods and tools for measuring automation-to-work (ATW) alignment, for use guiding development and evaluation of HAI designs. The purpose of the empirical strand is to assess whether measured ATW alignment of HAI designs predicts the learnability of those designs, an important aspect of robustness. By ATW alignment we mean the correspondence between the elements and structure of interaction with the elements and structure of the work. Needs analysis identifies elements and structure of the work. In the analytic strand we will develop a scoring method for measuring alignment. Our approach draws on and integrates a wide set of observations and proposals in HAI, Human Computer Interaction (HCI), Work Domain Analysis (WDA), and related disciplines. In the empirical strand we test the prediction that HAI designs that align with work more strongly will be easier to learn, particularly, easier to master using the automation for novel problem solving. We test this hypothesis by identifying and measuring designs that differ in ATW alignment and then comparing the designs with high versus low scores for how easily they are learned, how well retained, and how flexibly they can be used.

Rationale for HRP Directed Research: This research is directed because it contains highly constrained research, which requires focused and constrained data gathering and analysis that is more appropriately obtained through a non-competitive proposal.

Research Impact/Earth Benefits: 0

Task Progress & Bibliography Information FY2012 
Task Progress: New project for FY2012.

Bibliography Type: Description: (Last Updated: 07/22/2015) 

Show Cumulative Bibliography Listing
 
 None in FY 2012