Effective spaceflight medical training must be combined with in-flight support tools to ensure crew competence in management of medical events and caring for sick astronauts. Collectively called Augmented Clinical Tools (ACT), these include technologies and applications to assist medical decision-making and action. Mixed Reality (MR) -- the ability to place virtual and photo-realistic items into the field of view using holograms -- provides an immersive, realistic user experience that has also proven feasible for training and guidance during technical non-routine tasks.

We propose to utilize existing technology to develop MR software that provides realistic training scenarios for astronauts, and combine medical education with real-time clinical support for some probable medical events in deep space. This includes a “SMART checklist” which guides astronauts through managing medical events in real-time. MR allows us to create lifelike space environments for astronauts to practice their skills. We will involve a wide range of stakeholders in software development and testing for usability, engagement, and performance. The project will take two years to complete and we will provide innovative products and guidance that can be incorporated into astronaut training to ensure that they have the knowledge, skills, and support to manage the expected and unexpected challenges on successful deep space missions.

In this study, we proposed to utilize existing technology to develop Extended Reality (XR) software that provides immersive, interactive, realistic training scenarios for astronauts and combines medical education with real-time clinical support for some probable medical events in deep space. This includes a "smart checklist" application deployed as an augmented reality (AR) coach (AR-Coach) that guides astronauts through managing medical events in real-time. XR technology allowed us to create lifelike space environments for astronauts to practice their skills. We involved a wide range of stakeholders in software development and testing for usability, engagement, and performance. This project will provide innovative products and guidance that can be incorporated into astronaut training to ensure that they have the knowledge, skills, and support to manage the expected and unexpected challenges on successful deep space missions. The specific aims of this project were:

Aim 1: Identify the necessary features and functionalities of Mixed Reality (MR) medical education for long-duration exploration missions (LDEMs).

Aim 2: Adapt an existing validated MR platform to deliver immersive medical education modules incorporating a real-time care-delivery guidance tool (SMART checklist) to support autonomous medical event management on LDEMs.

Aim 3: Implement a data-driven integrative approach to evaluate the MR medical education platform.

In the first year of this project, we convened a multidisciplinary expert panel composed of 45 panelists. During online meetings and through surveys between meetings, we applied a Delphi Method to get a consensus on functionalities that are essential for both XR medical education and clinical guidance during LDEM. The expert panel has also provided recommendations on specific medical events that would be suitable for an XR platform, in addition to relevant clinical competencies and instructional design considerations for simulation scenarios and Augmented Clinical Tools (ACT) development. Based on these findings, we listed a total of 89 distinct XR functionalities, from which 13 were removed based on the level of essentiality, assessed by experts after 4 rounds of the Delphi Method. Based on the expert panel rating, we have also selected Tension Pneumothorax and Smoke Inhalation as the medical events that will be featured in the XR scenarios. Preliminary data from the expert panel on emergency medicine XR scenarios were presented as a poster presentation at the 2021 Society for Academic Emergency Medicine (SAEM) Annual Conference. A final manuscript reporting the results of the expert panel and proposing a framework for the design and development of XR-based Space Health applications, entitled "Using Extended Reality (XR) for Medical Training and Real-Time Clinical Support during Deep Space Missions" is under peer review in the Applied Ergonomics Journal. We have also completed a Systematic Review of literature on XR application for Space Health, and a manuscript entitled: "Applications of Extended Reality (XR) for Space Health: A Systematic Review" is under review in the Aerospace Medicine and Human Performance Journal.

A prototype of the AR-Coach system was developed to provide procedural guidance through a HoloLens mixed reality device to astronauts during ultrasound procedures in space. This work was published in the peer-reviewed proceedings of the International Conference on Applied Human Factors and Ergonomics (AHFE).

In the second year of this project, we used professional XR film production and volumetric video techniques to create two interactive XR scenarios, deployed in three different modalities: screen-based virtual reality (VR desktop), fully immersive VR (head-mounted device), and mobile augmented reality (AR through a tablet). After design, development, and testing, we conducted usability studies and, subsequently, a randomized trial, recruiting astronaut-like participants and randomizing them to participate in medical training and assessment using one of the three XR modalities. Several metrics, including demographics, technology acceptance, sense of presence, learning outcomes, and digital physiological biomarkers of cognitive load were captured and analyzed. The findings of this study showed that the AR modality produces the highest sense of presence, involvement, and experienced realism when compared with desktop and VR modalities. There was no statistically significant difference between the three modalities regarding the system usability score (SUS), although the AR modality presented a greater score with a trend toward statistical significance. By investigating the cognitive load of subjects, there was no statistically significant difference in the NASA Task Load Index (TLX) overall score; however, the "physical demand" domain was scored greater in the AR modality compared to desktop and VR. Among several physiological metrics, low frequency/high frequency (LF/HF) ratio – a biomarker of cognitive load – was higher in subjects using the AR modality. Performance efficiency, measured by the time in minutes to complete the scenarios, did not differ between groups. Interestingly, the medical knowledge test showed that subjects in the AR modality presented a lower score compared to the Desktop modality, but no difference compared with the VR modality.

Effective spaceflight medical training must be combined with in-flight support tools to ensure crew competence in management of medical events and caring for sick astronauts. Collectively called Augmented Clinical Tools (ACT), these include technologies and applications to assist medical decision-making and action. Mixed Reality (MR) -- the ability to place virtual and photo-realistic items into the field of view using holograms -- provides an immersive, realistic user experience that has also proven feasible for training and guidance during technical non-routine tasks.

We propose to utilize existing technology to develop MR software that provides realistic training scenarios for astronauts, and combine medical education with real-time clinical support for some probable medical events in deep space. This includes a “SMART checklist” which guides astronauts through managing medical events in real-time. MR allows us to create lifelike space environments for astronauts to practice their skills. We will involve a wide range of stakeholders in software development and testing for usability, engagement, and performance. The project will take two years to complete and we will provide innovative products and guidance that can be incorporated into astronaut training to ensure that they have the knowledge, skills, and support to manage the expected and unexpected challenges on successful deep space missions.

This project will foster effective in-flight medical training for both on-demand and continued medical education.

This project will provide clinical decision support tools for astronauts managing in-flight medical emergencies under varied levels of autonomy.

The deliverables from this project have the potential to mitigate the risks related to Adverse Health Outcomes & Decrements in Performance due to inflight Medical Conditions during long-duration exploration missions (LDEM).

Impact for Earth

Similar to LDEM, many terrestrial settings require medical education and clinical guidance tools for autonomous training and clinical care. The deliverables from this project can be used to support healthcare professionals who work in austere environments. Even in resourceful environments, like hospitals, medical trainees practice medicine under varied levels of autonomy through their clinical training. The project's deliverables can also be used in this environment to support medical trainees during clinical training (e.g., medical residency) when they do not have immediate access to their supervisors (e.g., during shifts when attending is on call).

Effective spaceflight medical training must be combined with in-flight support tools to ensure crew competence in the management of medical events and caring for sick astronauts. Collectively called Augmented Clinical Tools (ACT), these include technologies and applications to assist medical decision-making and action. Extended Reality (XR) - the ability to place virtual and photo-realistic items into the field of view using holograms - provides an immersive, realistic user experience that has also proven feasible for training and guidance during technical non-routine tasks.

We propose to utilize existing technology to develop XR software that provides realistic training scenarios for astronauts and combines medical education with real-time clinical support for some probable medical events in deep space. This includes a SMART checklist which guides astronauts through managing medical events in real-time. XR allows us to create lifelike space environments for astronauts to practice their skills. We will involve a wide range of stakeholders in software development and testing for usability, engagement, and performance. The project will take two years to complete and we will provide innovative products and guidance that can be incorporated into astronaut training to ensure that they have the knowledge, skills, and support to manage the expected and unexpected challenges on successful deep space missions.

In the first year of this project, we completed a total of 5 monthly videoconferences with a multidisciplinary expert panel composed of 45 panelists. During the meetings and through surveys between meetings, we applied a Delphi method to get consensus on functionalities that are essential for both XR medical education and clinical guidance during LDEM. The expert panel has also provided recommendations on specific medical events that would be suitable for an XR platform in addition to relevant clinical competencies and instructional design considerations for simulation scenarios and ACT development. Based on these findings, we listed a total of 89 distinct XR functionalities, from which 13 were removed based on the level of essentiality assessed by experts after 4 rounds of the Delphi Method. Based on the expert panel rating, we have also selected Tension Pneumothorax and Smoke Inhalation as the medical events that will be featured in the XR scenarios. Preliminary data from the expert panel on emergency medicine XR scenarios was submitted and accepted for poster presentation at the 2021 Society for Academic Emergency Medicine (SAEM) Annual Conference.

In the second year of this project, we will develop two simulated scenarios for three different modalities: screen-based virtual reality (VR), immersive VR, and augmented reality (AR). After development, we will run an experiment by recruiting astronaut-like participants and randomizing them in one of these modalities. A series of learning outcomes and engagement metrics, including learners' physiological signals, will be captured and analyzed.

Effective spaceflight medical training must be combined with in-flight support tools to ensure crew competence in management of medical events and caring for sick astronauts. Collectively called Augmented Clinical Tools (ACT), these include technologies and applications to assist medical decision-making and action. Mixed Reality (MR) -- the ability to place virtual and photo-realistic items into the field of view using holograms -- provides an immersive, realistic user experience that has also proven feasible for training and guidance during technical non-routine tasks.

We propose to utilize existing technology to develop MR software that provides realistic training scenarios for astronauts, and combine medical education with real-time clinical support for some probable medical events in deep space. This includes a “SMART checklist” which guides astronauts through managing medical events in real-time. MR allows us to create lifelike space environments for astronauts to practice their skills. We will involve a wide range of stakeholders in software development and testing for usability, engagement, and performance. The project will take two years to complete and we will provide innovative products and guidance that can be incorporated into astronaut training to ensure that they have the knowledge, skills, and support to manage the expected and unexpected challenges on successful deep space missions.