NOTE: End date changed to 8/31/2020 per NSSC information (Ed., 7/6/2020)

NOTE: End date changed to 3/31/2020 per NSSC information (Ed., 9/25/19)

The research team has completed a series of interacting research tasks: identify stressors during spaceflight emergencies, develop adaptation and training module logic, implementing the prototype training system, human in the loop experiments, and establishing guidelines for implementation.

1. Identify Stressors for Training. The research team and panel of experts reviewed and identified the stressors associated with emergency spaceflight situations of fire on the International Space Station (ISS) with respect to task and effect on users. Procedural information for virtual reality (VR) training was detailed, including the events likely to occur in the environment, the types of stressors encountered, NASA emergency procedures, resources to respond during the event, and the performance effects the stressors may have. Stress training requires procedural information to be included in the scenarios and conveyed to the user, so the stressors do not interfere with skill/knowledge acquisition. The stressors were categorized by manipulation type (environment or task manipulation), which indicates if a stressor may affect the task and deviate from the selected emergency procedure. From the environmental stressors, a simplified fire procedure was created based on the existing NASA ISS emergency procedures for experimental use. This information was used in later stages of developing VR scenarios, adaptation rules, training material, and training protocols.

2. Develop Adaptation and Training Module Logic. The adaptations used implicit biofeedback, consisting of physiological signals exceeding a threshold “zone” and triggering a change in simulated stressors. The Training Module design was informed by past research on Stress Inoculation Training and Graduated Stress Exposure to implement a stress inoculation training strategy over five training sessions. The adaptation and training module rules taken together was used to produce a training pedagogy for use within the system.

3. Implement Prototype Training System. The research team designed and implemented the system architecture for a VR-ISS with a emergency fire simulation. The VR environment was developed for the stressors and emergency procedure selected in Task 2. This environment was designed so that users can complete a NASA emergency fire procedure and locate the source of a fire. The procedure requires emergency equipment including a portable breathing apparatus (PBA), portable fire extinguisher (PFE), and Compound Specific Analyzer–Combustion Products (CSA-CP).

Three different stressor levels were created in the simulation, involving a combination of stressors of varying intensity, including smoke density, fire alarms, and module power-outages that flickered the lights. The stressor levels were validated through objective physiological measures and subjective stress questionnaires as participants conducted the emergency procedure to locate the fire source. Results showed that the three-stressor levels implemented in a VR environment result in significantly different participant stress, workload, and distress.

The adaptive nature of the adaptive VR training system required that the trainee’s stress level could be measured in real-time. Detecting stress during training can be used to make concurrent adjustments to the training to build resilience in preparation for hazardous operations. A stress detection system was created with machine learning methods. Physiological signals are calculated from sensors which include electrocardiogram (ECG), respiration (RSP), electrodermal activity (EDA), and non-invasive blood pressure (NIBP). The physiological signals were streamed in real-time to apply preprocessing, high-dimensional feature extraction, feature selection, and classification using supervised machine learning approaches.

Current limitations with supervised machine learning approaches propelled the development of a statistical classification method by our research team, known as Approximate Bayes (ABayes). ABayes uses kernel density probability estimates to find conditional class probabilities for three stress states: low, medium, high. The advantage of ABayes is the direct and transparent calculation of the class probabilities. ABayes was evaluated on its accuracy of predicting stress using cross-validation and hold-out on unseen data.

4. Humans-in-the-loop (HITL) experiments. Three human-in-the-loop experiments were performed to test the components of the Adaptive VR turning system in isolation and as a system.

Study 1 assessed the ability to manipulate a person’s stress level in a virtual environment. Specifically, an evaluation was preformed to assess the extent to which operationally relevant VR stressor levels (i.e., low, medium, high) derived from existing emergency spaceflight procedures could evoke a reliable stress response.

Study 2 assessed the ability to reliably detect the stress level of a person in VR using a combination of biophysical sensors and machine learning to correctly classify low, medium, or high stress.

Study 3 assessed if the closed-loop VR training system can successfully inoculate participant’s stress over a series of training trials with graduated stress exposure. Specifically, the study tested the hypothesis that receiving stressor adaptation compared to no adaptation during ISS emergency procedure task training will result in relatively improved stress responses.

5. Establish guidelines for implementation. Based on the tasks above, implementation guidelines for the training module and adaptation manager are presented.

ANNUAL REPORTING--AUGUST 2020

The research team has made considerable progress on the research tasks. This project has several interacting research task, including: identify stressors during spaceflight emergencies, develop a virtual environment to train participants to NASA fire procedures, validate VR manipulations that evoke different levels of stress in human participants, create a real-time stress detection system, and create an adaptations manager to determine when and how to adapt stressors during training.

1. Identify stressors

The workshop panel of experts identified a series of potential stressors from International Space Station (ISS) emergency procedures, how they mapped onto the procedures, and what effects the stressors might have. The stressors were categorized by manipulation type (environment or task manipulation), which indicates if a stressor may affect the task and deviate from the selected emergency procedure. From the environmental stressors, a simplified fire procedure was created based on the existing NASA ISS emergency procedures for experimental use.

2. Develop a virtual environment & validate stressor manipulations

The VR-ISS developed for use in this experiment has three modules: Node 1, US Lab, and Node 2. A number of dynamic interactions were included in the VR-ISS to aid detection and location of the source of a fire. Atmospheric contaminant levels rose as a function of time and distance from the fire source. Virtual smoke changed in density as a function of time and spread in a uniform pattern consistent with a microgravity environment. Participants were able to conduct the procedure with simulated NASA equipment including Portable Fire Extinguishers (PFE), Portable Breathing Apparatus (PBA), and Compound Specific Analyzer-Combustion Products (CSA-CP).

The virtual environment also includes a number of tools for researchers including the ability to change the stressor levels uses, playback recording of the participant movements, and a spreadsheet of participant performance metrics (e.g., number of fire extinguishers used, time to completion).

Three different stressor levels were created in the simulation, each with a fire location randomized across module. Each stress level is a combination of stressors of varying intensity, including fire alarm noise, smoke density, and flashing module lights. The stressor levels were validated through objective physiological measures and subjective stress questionnaires, which show that the three-stressor levels implemented in a VR environment result in significantly different participant stress, workload, and distress.

3. Create a real-time stress detection system & validate prediction accuracy

A stress detection system was created with machine learning methods, has been modified to be used in real-time, and is currently undergoing validation tests. Physiological signals are calculated from sensors with a BIOPAC MP150, which include electrocardiogram (ECG), respiration (RSP), electrodermal activity (EDA), and non-invasive blood pressure (NIBP). The signals are streamed to a continuous buffer in a MATLAB code that applies preprocessing, high-dimensional feature extraction, feature selection, and classification using supervised machine learning approaches.

Current limitations with machine learning approaches propelled the development of a statistical classification method by our research team, known as Approximate Bayes (ABayes). ABayes uses probability densities to find conditional class probabilities for three stress states: low, medium, high. This is a more direct approximation than standard machine learning algorithms that approximate the class probabilities with predetermined rules (e.g., hyperplanes to divide groupings of data, decision trees). The advantage of ABayes is the direct and transparent calculation of the class probabilities.

The classification algorithm used in the real-time stress detection was evaluated “offline” by recording physiological data from 47 participants during a stressful ISS emergency fire procedure and classifying the data post-hoc for three varying stressful scenarios (low, medium, high). Machine Learning cross-validation analysis was performed on the physiological data. Accuracy of the 3-class data with ABayes compared against standard machine learning algorithms of random forest, decision tree, and support vector machine. The cross validation accuracy of predicting the correct stress level using ABayes was 76%. This ABayes classifier performance was comparable to the other classifiers. However, the ability of ABayes to output conditional class probabilities provides a means of measuring confidence in the class predictions, which is a distinct advantage over other classifiers.

4. Create Adaptations manager to adapt stressors during training

Initial work has begun on developing a set of IF-THEN rules to decide which stressor combination are appropriate for increasing or decreasing stressfulness of the environment. The actual strength of the stressor effects and the rules to determine how they are applied will be determined through pilot testing. This research task is still in-progress.