Menu

 

The NASA Task Book
Advanced Search     

Project Title:  Autonomous Diagnostic Imaging Performed by Untrained Operators Using Augmented Reality as a Form of 'Just-in-Time' Training Reduce
Fiscal Year: FY 2017 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 06/01/2016  
End Date: 05/31/2017  
Task Last Updated: 02/05/2018 
Download report in PDF pdf
Principal Investigator/Affiliation:   Martin, David S. M.S. / Wyle Laboratories, Inc./NASA Johnson Space Center 
Address:  Cadiovascular & Vision Laboratory 
2101 NASA Parkway, #1 
Houston , TX 77058-3607 
Email: david.s.martin@nasa.gov 
Phone: 281-483-9462  
Congressional District: 36 
Web:  
Organization Type: NASA CENTER 
Organization Name: Wyle Laboratories, Inc./NASA Johnson Space Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Laurie, Steven  Ph.D. Wyle Laboratories, Inc. 
Lee, Stuart  Ph.D. Wyle Integrated Science and Engineering/NASA Johnson Space Center 
Stenger, Michael  Ph.D. Wyle Laboratories, Inc./NASA Johnson Space Center 
Wang, Lui  Ph.D. NASA Johnson Space Center 
Project Information: Grant/Contract No. NCC 9-58-HFP04503 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NCC 9-58-HFP04503 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:
No. of PhD Candidates:
No. of Master's Candidates:
No. of Bachelor's Candidates:
No. of PhD Degrees:
No. of Master's Degrees:
No. of Bachelor's Degrees:
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) Train:Risk of Performance Errors Due to Training Deficiencies
Human Research Program Gaps: (1) SHFE-TRAIN-03: We need to develop guidelines for effective onboard training systems that provide training traditionally assumed for pre-flight. (IRP Rev F. Previously: How can onboard training systems be designed to address Just in Time (JIT) and recurrent training needs for nominal and off nominal scenarios?)
Flight Assignment/Project Notes: NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

Task Description: Introduction

Imaging technologies are key to the diagnosis and treatment of medical conditions that astronauts on an exploration mission might encounter and to research activities that characterize and understand the adaptations to micro- and partial gravity environments. Key limitations of the currently available imaging capabilities include the complexity and operator dependency of the procedures to acquire high quality results. Currently, two imaging procedures critical to space medicine and research, ultrasound and optical coherence tomography (OCT), are performed on the International Space Station (ISS) by astronauts with the assistance of real-time communication with experts on the ground using a method called remote guidance. With this methodology astronauts have acquired diagnostic and research quality data that are central to our understanding of the physiological consequences of weightlessness, including altered cardiac function, muscle atrophy, and the spaceflight-associated neuro-ocular syndrome (SANS). However, with the time delay in communications that will be inherent in exploration missions while traveling great distances from Earth, remote guidance will no longer be practical, particularly when one-way transmissions may take up to 10 minutes or more. To fill this gap, we developed advanced audio-visual training modules, a form of just-in-time (JIT) training, to acquire medically necessary and research-relevant images. Building on our extensive experience with remote guidance of ISS astronauts and demonstrated success with just-in-time training, we employed an augmented reality (AR) system (Microsoft HoloLens) that included three-dimensional graphics of relevant anatomy, step-by-step audio instruction, reference images demonstrating adequate and inadequate quality, and troubleshooting guides. The ability of untrained subject-operators to acquire high-quality ultrasound and OCT images was evaluated by expert reviewers and compared to current JIT techniques for quality and time efficiency.

Methods

Tutorials to acquire ultrasound and OCT images were adapted for use as a PowerPoint presentation viewed either on a laptop computer or using an augmented reality platform with a heads-up display. Instructional material presented by the two different training modalities was identical, except that the augmented reality tutorial provided additional spatial guidance to complete the scanning protocols. Additional guidance included arrows superimposed on the ultrasound and OCT keyboards when specific controls were needed and virtual green dots superimposed on the body over the ultrasound targets' approximate locations to guide ultrasound probe placement. Twenty subjects attempted to acquire ultrasound and OCT images using the tutorials provided viewed on a laptop computer, and 20 subjects attempted the same diagnostic imaging procedures with instructions viewed using the AR system. No subject had prior experience performing these procedures or using this imaging hardware, and no subject participated in this study as a member of both groups. Subjects used guidance from the tutorials to acquire five ultrasound targets that constitute a subset of images in a trauma-induced injury assessment protocol (images of the heart, lungs, liver, kidney and spleen) and a single thirteen-line, vertical raster scan centered on the macula of the left eye using OCT. Time limits were imposed upon the subjects to acquire each of the ultrasound or OCT targets. This is not dissimilar to the environment on ISS or anticipated on future exploration missions in which communication windows or mission objectives can impact the available time to complete imaging procedures.

After the data collection session, subjects completed a survey to capture their thought about the equipment, procedures, instructional material, and future improvements. Images were stored by the subjects and evaluated off-line in a blinded fashion by imaging professionals. Four radiologists from Vanderbilt University with training in ultrasound analyses evaluated the ultrasound images. Four trained evaluators at the Doheny Image Reading Center (University of California, Los Angeles), with previous experience using the grading techniques utilized in this study, evaluated the OCT images. Somers' D was used to assess the effect of the use of either laptop or HoloLens on the resulting image scores.

Results and Discussion

Nearly 70% of the ultrasound images and 53% of OCT scans that these previously untrained subjects acquired were considered to be diagnostically adequate. The quality scores for OCT images and 3 of the 5 ultrasound targets were not different between groups, but the laptop group performed better for the other 2 ultrasound targets. The time to acquire ultrasound images was, on average, shorter for the AR group (52 compared to 57 minutes). Interestingly, the survey results indicated that the AR group did not find the instructional material as helpful as the laptop group (p= 0.042), even though the content was the same for both. While it is evident that both the augmented reality and laptop groups struggled with the complexity of both ultrasound and OCT, there was a subset of subjects who appeared to embrace the technological challenge and performed well. This might be related to the education level, learning styles, or comfort level with technology, but these factors were not specifically evaluated. While some subjects embraced the technology, some were overwhelmed by the task of familiarizing themselves with a new technology (i.e., AR) while concurrently attempting complex, operator-dependent imaging which also was new to them. The inclusion of subjects not necessarily representative of the astronaut corps represents a limitation of the study. Additionally, the degree of familiarity with laptop computers and PowerPoint may have created a bias for the laptop group.

Research Impact/Earth Benefits: This study sought to evaluate a technique to help facilitate autonomous use of operator-dependent, non-invasive medical imaging devices in a subject population without prior ultrasound or optical coherence tomography (OCT) training. While the imaging protocols chosen for this study were based upon the health and medical risks identified as concerns for long-duration spaceflight, particularly exploration missions, these imaging protocols also are routinely used on Earth. Through this work we have developed and tested an augmented reality (AR) tutorial to guide inexperienced or untrained operators through complex, operator-dependent procedures for specific ultrasound and OCT applications. While originally developed for astronaut use beyond the range of timely communication with Earth, AR could enable more general uses in any remote or extreme environment lacking trained professionals. Ultrasound has been used in remote and extreme environments, including special forces military units in Afghanistan, Mount Everest expeditions, and rural locations without adequate medical expertise. Lessons learned from this project will facilitate the development of AR programs for similar applications. It should be noted that this type of training is not preferable to a study performed by a sonographer or other qualified medical professional trained in ultrasound. However, similar to defibrillators, if the equipment and adequate instruction is available, AR instruction could be used when equipment is available, but trained professionals are not.

Task Progress & Bibliography Information FY2017 
Task Progress: This research project did not receive formal funding until November, 2016. The entire project, from planning to initiation of testing to completion of data collection and analysis, was conducted using a compressed schedule of 6 months.

Bibliography Type: Description: (Last Updated: ) 

Show Cumulative Bibliography Listing
 
 None in FY 2017
Project Title:  Autonomous Diagnostic Imaging Performed by Untrained Operators Using Augmented Reality as a Form of ‘Just-in-Time’ Training Reduce
Fiscal Year: FY 2016 
Division: Human Research 
Research Discipline/Element:
HRP HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Start Date: 06/01/2016  
End Date: 05/31/2017  
Task Last Updated: 08/19/2016 
Download report in PDF pdf
Principal Investigator/Affiliation:   Martin, David S. M.S. / Wyle Laboratories, Inc./NASA Johnson Space Center 
Address:  Cadiovascular & Vision Laboratory 
2101 NASA Parkway, #1 
Houston , TX 77058-3607 
Email: david.s.martin@nasa.gov 
Phone: 281-483-9462  
Congressional District: 36 
Web:  
Organization Type: NASA CENTER 
Organization Name: Wyle Laboratories, Inc./NASA Johnson Space Center 
Joint Agency:  
Comments:  
Co-Investigator(s)
Affiliation: 
Laurie, Steven  Ph.D. Wyle Laboratories, Inc. 
Lee, Stuart  Ph.D. Wyle Integrated Science and Engineering/NASA Johnson Space Center 
Stenger, Michael  Ph.D. Wyle Laboratories, Inc./NASA Johnson Space Center 
Wang, Lui  Ph.D. NASA Johnson Space Center 
Project Information: Grant/Contract No. NCC 9-58-HFP04503 
Responsible Center: NSBRI 
Grant Monitor:  
Center Contact:   
Solicitation / Funding Source: 2015-16 HERO NNJ15ZSA001N-Crew Health (FLAGSHIP, NSBRI, OMNIBUS). Appendix A-Crew Health, Appendix B-NSBRI, Appendix C-Omnibus 
Grant/Contract No.: NCC 9-58-HFP04503 
Project Type: GROUND 
Flight Program:  
TechPort: Yes 
No. of Post Docs:  
No. of PhD Candidates:  
No. of Master's Candidates:  
No. of Bachelor's Candidates:  
No. of PhD Degrees:  
No. of Master's Degrees:  
No. of Bachelor's Degrees:  
Human Research Program Elements: (1) HFBP:Human Factors & Behavioral Performance (IRP Rev H)
Human Research Program Risks: (1) Train:Risk of Performance Errors Due to Training Deficiencies
Human Research Program Gaps: (1) SHFE-TRAIN-03: We need to develop guidelines for effective onboard training systems that provide training traditionally assumed for pre-flight. (IRP Rev F. Previously: How can onboard training systems be designed to address Just in Time (JIT) and recurrent training needs for nominal and off nominal scenarios?)
Flight Assignment/Project Notes: NOTE: Element change to Human Factors & Behavioral Performance; previously Space Human Factors & Habitability (Ed., 1/19/17)

Task Description: Imaging technologies are key to diagnosis and treatment of medical conditions, which astronauts on an exploration mission might encounter and in research activities to characterize and understand the adaptations to micro- and partial gravity environments. A key limitation of currently available imaging capabilities is the complexity and operator dependency of the procedures to acquire high quality results. Currently, two imaging procedures critical to space medicine and research, ultrasound and optical coherence tomography (OCT), are performed on the International Space Station (ISS) by astronauts with the assistance of real-time communication with experts on the ground using a procedure called remote guidance. With this methodology astronauts have acquired diagnostic and research quality data that are central to our understanding of the physiological consequences of weightlessness, including altered cardiac function, muscle atrophy, and the newly-described vision impairment and intracranial pressure (VIIP) syndrome. However, with time delay in communications inherent in exploration missions while traveling great distances from Earth, remote guidance will no longer be a practical approach, particularly when one-way transmissions may take up to 10 minutes. To fill this gap, we propose the development of advanced audio-visual training modules, a form of “just-in-time” (JIT) training, to address medically necessary and research-relevant issues. Building on our extensive experience with remote guidance of ISS astronauts and previously demonstrated successful work with just-in-time training, we will employ an augmented reality system (Hololens), three-dimensional graphics of relevant anatomy, step-by-step audio instruction, reference adequate and inadequate images, and troubleshooting guides. The ability of untrained sonographers and OCT operators to acquire diagnostically adequate, high-quality images will be evaluated by expert reviewers and compared to current JIT techniques for quality and time efficiency. At the completion of this project we will deliver a JIT training system that can improve quality of medical procedures performed during spaceflight or when performing unfamiliar procedures on Earth.

Research Impact/Earth Benefits:

Task Progress & Bibliography Information FY2016 
Task Progress: New project for FY2016.

Bibliography Type: Description: (Last Updated: ) 

Show Cumulative Bibliography Listing
 
 None in FY 2016