Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems--NNX15AR28G Reduce
Fiscal Year: FY 2018 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 08/01/2015  
End Date: 06/30/2018  
Task Last Updated: 09/26/2018 
Download report in PDF pdf
Principal Investigator/Affiliation:   Salas, Eduardo  Ph.D. / Rice University 
Address:  Department of Psychology 
6100 Main Street MS25 
Houston , TX 77005 
Email: eduardo.salas@rice.edu 
Phone: 713-348-3917  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Rice University 
Joint Agency:  
Comments: NOTE: Previous affiliation was University of Central Florida, until mid-2015  
Project Information: Grant/Contract No. NNX15AR28G 
Responsible Center: NASA ARC 
Grant Monitor: Gore, Brian  
Center Contact: 650.604.2542 
brian.f.gore@nasa.gov 
Solicitation / Funding Source: 2012 Crew Health NNJ12ZSA002N 
Grant/Contract No.: NNX15AR28G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:  
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-03:We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations (IRP Rev G name change to HARI-03 from previous designation SHFE-HARI-03 (which was IRP Rev F) (Previously: How can performance, efficiency and safety guidelines be developed for appropriate task automation and the effective allocation of tasks between humans and automation?)
Flight Assignment/Project Notes: NOTE: End date change to 6/30/2018 per NSSC information and E. Connell/HRP (Ed., 5/3/18)

NOTE: End date change to 4/30/2018 per NSSC information (Ed., 8/9/17)

NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

Task Description: NOTE: Continuation of "Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems," grant #NNX13AO51G with Principal Investigator (PI) Eduardo Salas, Ph.D., due to PI move in mid-2015 to Rice University from University of Central Florida.

Specific aims of this proposal are threefold: (1) develop a framework for human-systems integration requirements, (2) identify and develop a metrics criteria in which safety and efficiency can be characterized in human-automation teams, and (3) design, develop, and validate a theoretically-driven, empirically-based metrics toolkit that characterizes the safety and efficiency of human automation interactions. This proposal meets NASA goals and objectives by mitigating the risk of inadequate design of human and automation/robotic integration through the development of safety and efficiency metrics for human-automation systems. The proposal is divided into three primary phases. Phase 1 will consist of synthesizing and translating findings from the extant literature relevant to human automation/robotic integration. The result of this effort will be the development of objective metrics generalizable to individual and team levels that characterize the safety and efficiency of a human automation interaction. The final outcome of Phase 1 will be the development of a human automation interaction metrics (HAIM) toolkit. Phase 2 will involve in-depth preparation for scientifically sound experiments. Phase 2 ensures adequate time and methodology for meaningful outcomes for Phase 3. The central outcome of Phase 2 will be the final development of the experimental testbed and experimental protocol. Phase 3 will involve preparation for, and execution of, experiments. This will include the design and execution of a set of multi-level empirical studies aimed at validating the metrics toolkit. The validation studies will focus on testing different aspects of human automation interaction (e.g., levels of automation, task complexity, and the number and configuration of system operators). The outcome of the proposed effort will provide NASA a set of evidence-based, empirically-validated guidelines and a measurement toolkit for mitigating the risk of inadequate design of human and automation/robotic integration as it pertains to the development of safety and efficiency metrics for human automation systems.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2018 
Task Progress: Phase 1: The purpose of this phase was to synthesize and translate findings from the extant literature relevant to human automation/robotic integration in order to determine specific metrics that characterize the safety and efficiency of a human-automation interaction. The final outcome of Phase 1 was the development of the content for human automation interaction metrics (HAIM) toolkit. This content was then packaged inside an automated online tool that went a step beyond our original aim for this project by providing designers with a tool that not only contained content, but also provided information, tips, and advice for how to apply this content to their own measurement situations.

Phase 2: The original goal for this second phase was to complete in-depth preparation for scientifically sound experiments. This detailed preparation time in phase 2 was meant to ensure adequate time and methodology for meaningful outcomes for Phase 3. The intended outcome of Phase 2 was be the final development of the experimental testbed and experimental protocol. However, as more was learned throughout phase 1 about Human-Automation measurement and as the toolkit development was progressing, it was determined that Phase 2 should be adjusted to better meet the goals of the overall project. Therefore, Phase 2 was adjusted to be preparation for Subject Matter Expert (SME) interviews and iterative usability evaluations of the HAIM Toolkit instead of validation experiments to test the theoretical framework. This proved to be a very fruitful decision as the SME interviews provided valuable feedback and data that was able to be incorporated into the final design of the toolkit.

Phase 3: Preparation for and execution of, experiments. This final phase of the project was originally planned to include the design and execution of a set of multi-level empirical studies aimed at validating the theoretical framework and metrics toolkit. The focus of the validation studies was to be on testing different aspects of human automation interaction (e.g., levels of automation, task complexity, and the number and configuration of system operators). The intended outcome of the proposed effort was to provide NASA a set of evidence-based, empirically-validated guidelines and a measurement toolkit for mitigating the risk of inadequate design of human and automation/robotic integration as it pertains to the development of safety and efficiency metrics for human automation systems. Phase 3 was adjusted to focus on validation of the automated HAIM Toolkit through execution of usability evaluations by potential end users as this was deemed more valuable than the originally planned validation studies. The usability studies that were conducted were designed to evaluate both usability of the automated HAIM Toolkit as well as the value of the information that the toolkit provided. Multiple rounds of testing took place along with subsequent periods or toolkit redesign to incorporate feedback gleaned during the usability testing. The final outcome of this effort along with the research conducted in Phase 1 is a set of principles, guidelines, and tips for measurement that have been submitted to NASA in white papers and are in the process of being transformed into multiple manuscripts currently under preparation. Additionally, the final outcome of this phase is the development of the HAIM Toolkit, which is complete and will be delivered to NASA with the final report.

Manuscripts in various stages:

Hughes, A.M., Marlow, S.L., Hancock, G.M., Oglesby, J.M., Stowers, K. & Salas, E. (Under Revision). Physiological assessment of workload: A meta-analysis. Human Factors

Stowers, K., Iwig, C., Salas, E. (In preparation). Measurement in the Second Machine Age: Considerations for Future Human-Machine Systems. To be submitted to Human Factors.

Stowers, K., Sonesh, S., Iwig, C., Salas, E. (In preparation). Technology and Teams: The Impact of Technological Evolution on Team Performance. To be submitted to Group and Organization Management.

Stowers, K., Sonesh, S.C., Iwig, C., Salas, E. (In preparation.) Capturing Human Machine System Safety in Spaceflight. To be submitted to Aerospace Medicine and Human Performance.

Bibliography Type: Description: (Last Updated: 06/10/2021) 

Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals Hughes AM, Hancock GM, Marlow SL, Stowers K, Salas E. "Cardiac measures of cognitive workload: A meta-analysis." Human Factors. 2019 May;61(3):393-414. Epub 2019 Mar 1. https://doi.org/10.1177/0018720819830553 ; PubMed PMID: 30822151 , May-2019
Project Title:  Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems--NNX15AR28G Reduce
Fiscal Year: FY 2017 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 08/01/2015  
End Date: 06/30/2018  
Task Last Updated: 06/02/2017 
Download report in PDF pdf
Principal Investigator/Affiliation:   Salas, Eduardo  Ph.D. / Rice University 
Address:  Department of Psychology 
6100 Main Street MS25 
Houston , TX 77005 
Email: eduardo.salas@rice.edu 
Phone: 713-348-3917  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Rice University 
Joint Agency:  
Comments: NOTE: Previous affiliation was University of Central Florida, until mid-2015  
Project Information: Grant/Contract No. NNX15AR28G 
Responsible Center: NASA ARC 
Grant Monitor: Gore, Brian  
Center Contact: 650.604.2542 
brian.f.gore@nasa.gov 
Solicitation / Funding Source: 2012 Crew Health NNJ12ZSA002N 
Grant/Contract No.: NNX15AR28G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:  
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-03:We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations (IRP Rev G name change to HARI-03 from previous designation SHFE-HARI-03 (which was IRP Rev F) (Previously: How can performance, efficiency and safety guidelines be developed for appropriate task automation and the effective allocation of tasks between humans and automation?)
Flight Assignment/Project Notes: NOTE: End date change to 6/30/2018 per NSSC information and E. Connell/HRP (Ed., 5/3/18)

NOTE: End date change to 4/30/2018 per NSSC information (Ed., 8/9/17)

NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

Task Description: NOTE: Continuation of "Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems," grant #NNX13AO51G with Principal Investigator (PI) Eduardo Salas, Ph.D., due to PI move in mid-2015 to Rice University from University of Central Florida.

Specific aims of this proposal are threefold: (1) develop a framework for human-systems integration requirements, (2) identify and develop a metrics criteria in which safety and efficiency can be characterized in human-automation teams, and (3) design, develop, and validate a theoretically-driven, empirically-based metrics toolkit that characterizes the safety and efficiency of human automation interactions. This proposal meets NASA goals and objectives by mitigating the risk of inadequate design of human and automation/robotic integration through the development of safety and efficiency metrics for human-automation systems. The proposal is divided into three primary phases. Phase 1 will consist of synthesizing and translating findings from the extant literature relevant to human automation/robotic integration. The result of this effort will be the development of objective metrics generalizable to individual and team levels that characterize the safety and efficiency of a human automation interaction. The final outcome of Phase 1 will be the development of a human automation interaction metrics (HAIM) toolkit. Phase 2 will involve in-depth preparation for scientifically sound experiments. Phase 2 ensures adequate time and methodology for meaningful outcomes for Phase 3. The central outcome of Phase 2 will be the final development of the experimental testbed and experimental protocol. Phase 3 will involve preparation for, and execution of, experiments. This will include the design and execution of a set of multi-level empirical studies aimed at validating the metrics toolkit. The validation studies will focus on testing different aspects of human automation interaction (e.g., levels of automation, task complexity, and the number and configuration of system operators). The outcome of the proposed effort will provide NASA a set of evidence-based, empirically-validated guidelines and a measurement toolkit for mitigating the risk of inadequate design of human and automation/robotic integration as it pertains to the development of safety and efficiency metrics for human automation systems.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2017 
Task Progress: The Rice University research team continues to work to provide a tool guiding the evaluation of human-automation systems. Research and design efforts throughout the life-cycle of this project have focused on developing best practices for human-automation system measurement and incorporating these best practices into a web-based decision aid tool for providing system designers with practical guidelines and recommendations for how to evaluate their systems. Recent progress has focused on development of this online tool, called HASMAT: Human Automation System Measurement and Assessment Toolkit.

Development of HASMAT has followed an iterative design and development process including multiple phases of prototype design, testing, and revision. Major accomplishments during this year have included (1) transitioning the toolkit decision tree architecture, which was developed in the previous year, into the design of the first HASMAT prototype, (2) testing the prototype using validated usability evaluation techniques, (3) developing a second HASMAT prototype based on results from initial evaluations, (4) testing of the second prototype in a structured laboratory based usability study utilizing NASA SMEs (subject matter experts), (5) development of a vast database of measurement information, guidelines, and tips to incorporate into the toolkit (in-progress), and (6) development of the final, fully automated, HASMAT prototype (in-progress).

Research efforts this year began with the refinement of the toolkit decision tree architecture, which was developed during the previous year. Once refinement of this architecture was complete, it was used as a basis of the first toolkit prototype. This initial prototype was created using Axure prototyping software. The main goal for this initial prototype was to incorporate all decision tree questions, determine the most appropriate visual delivery method for the questions, and develop and ideal organization and flow of the questions within the decision tree. Once this initial prototype was complete, initial evaluation techniques were conducted by the Rice research team to evaluate all aspects of the initial prototype's design, flow, and support features. The results of these evaluations informed revisions to the toolkit.

The second HASMAT prototype incorporated all recommendations for improvement that resulted from evaluations of the initial prototype. The purpose of this prototype was to provide a semi-functional prototype that could be tested with potential end-users. This prototype was still lacking automated measure selection features, but the decision tree architecture had been refined in this prototype and other features were added. Therefore, usability testing of this prototype with NASA SMEs focused on obtaining input as to how the decision tree architecture could be improved as well as the content that had already been incorporated into this version of HASMAT.

Results from the structured usability studies aimed at evaluating the second HASMAT prototype were very enlightening and led to an extensive overhaul of the entire toolkit design. Additionally, this extensive update of HASMAT, which is currently in progress, is also aimed at adding in full functionality including automation of the dynamic measure selection feature that allows the toolkit to provide customized measurement recommendations based on system and contextual information provided by the toolkit’s users during their completion of the decision tree questions. Once complete, this final HASMAT prototype will be tested in a final round of usability testing using NASA SMEs, followed by completion of necessary revisions, and resulting in delivery of the final HASMAT prototype to NASA. Overall, major progress on the development of the final HASMAT prototype has been made in year 4 and will continue to be made through the end of July.

Bibliography Type: Description: (Last Updated: 06/10/2021) 

Show Cumulative Bibliography Listing
 
Articles in Peer-reviewed Journals Stowers K, Oglesby J, Sonesh S, Leyva K, Iwig C, Salas E. "A framework to guide the assessment of human-machine systems." Hum Factors. 2017 Mar;59(2):172-88. https://doi.org/10.1177/0018720817695077 ; PubMed PMID: 28324673 , Mar-2017
Articles in Peer-reviewed Journals Woods A, Iwig C, Dinh J, Salas E. "Informing the development of a safety and performance metric selection toolkit: Subject matter experts weight in." Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2016 Sep;60(1):1354-8. (2016 Annual Meeting of the Human Factors and Ergonomics Society, Washington, D.C., September 19-23, 2016.) https://doi.org/10.1177/1541931213601312 , Sep-2016
Project Title:  Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems--NNX15AR28G Reduce
Fiscal Year: FY 2016 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 08/01/2015  
End Date: 07/31/2017  
Task Last Updated: 06/13/2016 
Download report in PDF pdf
Principal Investigator/Affiliation:   Salas, Eduardo  Ph.D. / Rice University 
Address:  Department of Psychology 
6100 Main Street MS25 
Houston , TX 77005 
Email: eduardo.salas@rice.edu 
Phone: 713-348-3917  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Rice University 
Joint Agency:  
Comments: NOTE: Previous affiliation was University of Central Florida, until mid-2015  
Project Information: Grant/Contract No. NNX15AR28G 
Responsible Center: NASA ARC 
Grant Monitor: Williams, Thomas  
Center Contact: 281-483-8773 
thomas.j.will1@nasa.gov 
Solicitation / Funding Source: 2012 Crew Health NNJ12ZSA002N 
Grant/Contract No.: NNX15AR28G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:  
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-03:We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations (IRP Rev G name change to HARI-03 from previous designation SHFE-HARI-03 (which was IRP Rev F) (Previously: How can performance, efficiency and safety guidelines be developed for appropriate task automation and the effective allocation of tasks between humans and automation?)
Flight Assignment/Project Notes: NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

Task Description: NOTE: Continuation of "Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems," grant #NNX13AO51G with Principal Investigator Eduardo Salas, Ph.D., due to PI move in mid-2015 to Rice University from University of Central Florida.

Specific aims of this proposal are threefold: (1) develop a framework for human-systems integration requirements, (2) identify and develop a metrics criteria in which safety and efficiency can be characterized in human-automation teams, and (3) design, develop, and validate a theoretically-driven, empirically-based metrics toolkit that characterizes the safety and efficiency of human automation interactions. This proposal meets NASA goals and objectives by mitigating the risk of inadequate design of human and automation/robotic integration through the development of safety and efficiency metrics for human-automation systems. The proposal is divided into three primary phases. Phase 1 will consist of synthesizing and translating findings from the extant literature relevant to human automation/robotic integration. The result of this effort will be the development of objective metrics generalizable to individual and team levels that characterize the safety and efficiency of a human automation interaction. The final outcome of Phase 1 will be the development of a human automation interaction metrics (HAIM) toolkit. Phase 2 will involve in-depth preparation for scientifically sound experiments. Phase 2 ensures adequate time and methodology for meaningful outcomes for Phase 3. The central outcome of Phase 2 will be the final development of the experimental testbed and experimental protocol. Phase 3 will involve preparation for, and execution of, experiments. This will include the design and execution of a set of multi-level empirical studies aimed at validating the metrics toolkit. The validation studies will focus on testing different aspects of human automation interaction (e.g., levels of automation, task complexity, and the number and configuration of system operators). The outcome of the proposed effort will provide NASA a set of evidence-based, empirically-validated guidelines and a measurement toolkit for mitigating the risk of inadequate design of human and automation/robotic integration as it pertains to the development of safety and efficiency metrics for human automation systems.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2016 
Task Progress: JUNE 2016 PROGRESS REPORT

The third year of the project focused on completion of the subject matter expert (SME) interviews in order to determine best practices for measurement of human-automation systems within the specific context of spaceflight. The information gained during these interviews served as a supplement to the data gained through the qualitative literature review and during the 2014 HFES (Human Factors and Ergonomics Society) Measurement Workshop by providing a greater understanding of the specific context of interest -- spaceflight. Additionally, during Year 3, greater focus was placed on actual development of the toolkit as a web-based decision tree tool for guiding users in selecting metrics and applying appropriate measurement techniques. Currently, work is continuing on the development of the metrics toolkit prototype and preparation for the usability studies have begun. Usability studies will be conducted through the summer and will be followed by completion of final revisions to the toolkit.

Bibliography Type: Description: (Last Updated: 06/10/2021) 

Show Cumulative Bibliography Listing
 
Abstracts for Journals and Proceedings Iwig C, Oglesby J, Woods A, Dinh J, Salas E. "Development of an Evidence-Based Measurement Toolkit for Assessing Human-Automation System Safety and Performance." Presented at the 2016 NASA Human Research Program Investigators’ Workshop, Galveston, TX, February 8-11, 2016.

2016 NASA Human Research Program Investigators’ Workshop, Galveston, TX, February 8-11, 2016. , Feb-2016

Abstracts for Journals and Proceedings Woods A, Iwig C, Salas E. "Informing the Development of a Safety and Performance Metric Selection Toolkit: Subject Matter Experts Weigh In." Poster to be Presented at the 2016 International Meeting of the Human Factors and Ergonomics Society, Washington, D.C., September 19-23, 2016.

To be published in the 2016 Human Factors and Ergonomics Society Conference Proceedings. In press as of June 2016. , Jun-2016

Articles in Peer-reviewed Journals Iwig C, Oglesby J, Shimono M, Stowers K, Leyva K, Salas E. "Space flight task contexts for long distance and duration exploration missions: applications to measurement of human-automation interaction." Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2015 Sep;59(1):941-5. (59th Annual Meeting of the Human Factors and Ergonomics Society, Los Angeles, CA, October 26–30, 2015.) http://dx.doi.org/10.1177/1541931215591273 , Sep-2015
Project Title:  Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems--NNX15AR28G Reduce
Fiscal Year: FY 2015 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 08/01/2015  
End Date: 07/31/2017  
Task Last Updated: 04/13/2016 
Download report in PDF pdf
Principal Investigator/Affiliation:   Salas, Eduardo  Ph.D. / Rice University 
Address:  Department of Psychology 
6100 Main Street MS25 
Houston , TX 77005 
Email: eduardo.salas@rice.edu 
Phone: 713-348-3917  
Congressional District:
Web:  
Organization Type: UNIVERSITY 
Organization Name: Rice University 
Joint Agency:  
Comments: NOTE: Previous affiliation was University of Central Florida, until mid-2015  
Project Information: Grant/Contract No. NNX15AR28G 
Responsible Center: NASA ARC 
Grant Monitor: Gore, Brian  
Center Contact: 650.604.2542 
brian.f.gore@nasa.gov 
Solicitation / Funding Source: 2012 Crew Health NNJ12ZSA002N 
Grant/Contract No.: NNX15AR28G 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) HARI:Risk of Inadequate Design of Human and Automation/Robotic Integration
Human Research Program Gaps: (1) HFBP-HARI-03:We do not know how to quantify overall human-automation-robotic system performance to inform and evaluate system designs to ensure safe and efficient space mission operations (IRP Rev G name change to HARI-03 from previous designation SHFE-HARI-03 (which was IRP Rev F) (Previously: How can performance, efficiency and safety guidelines be developed for appropriate task automation and the effective allocation of tasks between humans and automation?)
Task Description: NOTE: Continuation of "Evidence-based Metrics Toolkit for Measuring Safety and Efficiency in Human-Automation Systems," grant #NNX13AO51G with Principal Investigator Eduardo Salas, Ph.D., due to PI move in mid-2015 to Rice University from University of Central Florida.

Specific aims of this proposal are threefold: (1) develop a framework for human-systems integration requirements, (2) identify and develop a metrics criteria in which safety and efficiency can be characterized in human-automation teams, and (3) design, develop, and validate a theoretically-driven, empirically-based metrics toolkit that characterizes the safety and efficiency of human automation interactions. This proposal meets NASA goals and objectives by mitigating the risk of inadequate design of human and automation/robotic integration through the development of safety and efficiency metrics for human-automation systems. The proposal is divided into three primary phases. Phase 1 will consist of synthesizing and translating findings from the extant literature relevant to human automation/robotic integration. The result of this effort will be the development of objective metrics generalizable to individual and team levels that characterize the safety and efficiency of a human automation interaction. The final outcome of Phase 1 will be the development of a human automation interaction metrics (HAIM) toolkit. Phase 2 will involve in-depth preparation for scientifically sound experiments. Phase 2 ensures adequate time and methodology for meaningful outcomes for Phase 3. The central outcome of Phase 2 will be the final development of the experimental testbed and experimental protocol. Phase 3 will involve preparation for, and execution of, experiments. This will include the design and execution of a set of multi-level empirical studies aimed at validating the metrics toolkit. The validation studies will focus on testing different aspects of human automation interaction (e.g., levels of automation, task complexity, and the number and configuration of system operators). The outcome of the proposed effort will provide NASA a set of evidence-based, empirically-validated guidelines and a measurement toolkit for mitigating the risk of inadequate design of human and automation/robotic integration as it pertains to the development of safety and efficiency metrics for human automation systems.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2015 
Task Progress: New project for FY2015. Continuation of project with the same title and Principal Investigator, grant NNX13AO51G, due to PI move in fall 2015 to Rice University from University of Central Florida.

Bibliography Type: Description: (Last Updated: 06/10/2021) 

Show Cumulative Bibliography Listing
 
 None in FY 2015