NOTE: End date changed to 08/19/2022 per L. Barnes-Moten/JSC and NSSC information (Ed., 8/2/21)

Stuster, J, Adolf J, Byrne V, Greene M. (2018). Human exploration of Mars: Preliminary lists of crew tasks. NASA/CR-2018-220043. https://ntrs.nasa.gov/citations/20190001401

We built a remotely deployable virtual reality environment that could be experienced on an Android phone with a cardboard virtual reality (VR) headset. This lightweight, inexpensive, and instrumented environment allowed us to test the effects of our Diminished Reality (DR) aids remotely. Creating this software required work with 3-D models, networking, and sending signals to and from the environment. The environment could be controlled by experimenters on a computer while the participant was immersed in the environment, and visuals/sounds could be triggered in real-time by the study staff. This “Wizard of Oz” methodology made it possible to move beyond the typical difficulties in training participants to use controllers or input devices.

Another benefit from this new software testbed is the ability to change the environment and run other experiments. The software is already being adapted for two studies funded by the NC Space Grant that answer new questions about training with cognitive aids in place.

Two experiments were conducted to understand the effects of universal and selective (context-aware) diminishment of distraction on performance, situational awareness, and subjective experience of workload in a novel assembly task. In both experiments, participants used procedural instructions to assemble a medical ventilator for use during a simulated emergency in virtual reality. In both experiments, the type of diminishment was manipulated within participants, so that each participant experienced universal attenuation of visual and auditory distraction, context-aware diminishment of the same, and a control condition with all distractions present. In the first experiment, the participants were STEM graduate students at a large state university. In the second experiment, small changes were made to the task and context-aware diminishment and then run remotely with participants who were employees at Johnson Space Center.

Our hypotheses were as follows: Performance would be supported in the DR conditions compared to a control condition. The workload would be lower in the DR conditions. Situational awareness would be lower in the DR conditions. Of the two DR conditions, the DR condition with universal attenuation would show the poorest situational awareness.

The main difference between experiments was the type of sample. Both groups were highly educated; however, the JSC participants also had work experience in science and technology. The main difference in the results between the experiments was the lack of performance differences by condition for the JSC participants, while those STEM graduate programs accomplished more steps in the Context-Aware condition compared to Control and had lower errors in the Universal DR condition compared to control. The same was true for situational awareness measures: diminishment harmed situational awareness for label locations and knowledge of the external concurrent emergency for the STEM graduate students, but not for those at JSC. One explanation might be the smaller sample size of JSC participants in Experiment 2, though the within-subject design meant that both experiments were adequately powered. However, even when ignoring significance, there were no trends toward performance differences in Experiment 2. A better explanation comes from the measures of workload.

In Experiment 1, workload was perceived to be lower in the Universal DR condition compared to the control. In Experiment 2, the JSC participants perceived lower workload in both DR conditions compared to control. In many studies, perceived workload changes when performance does not because participants are able to put forth the effort needed to perform (even when the task is extremely difficult, see Hancock and Matthews (2019) for a review of the varying relationships between performance and workload). The JSC participants, with their longer work histories and experience in problem-solving for ambiguous tasks, were likely able to keep up their performance even in the control condition, though they were also sensitive to how much harder they had to work to do so.

Last, it is worth noting that participants in both studies were highly educated and accomplished. Their performance in the study is likely higher than would be expected from a random sample, and their workload measures are likely lower. Any benefit from DR would also likely be higher for a random sample; however, this should be investigated empirically.

NOTE: End date changed to 08/19/2022 per L. Barnes-Moten/JSC and NSSC information (Ed., 8/2/21)

Stuster, J, Adolf J, Byrne V, Greene M. (2018). Human exploration of Mars: Preliminary lists of crew tasks. NASA/CR-2018-220043. https://ntrs.nasa.gov/citations/20190001401

In the area of protocol development, we finalized the measures we planned to use in the experiments and studies. These were all put into electronic formats to allow for distance testing. The measures included tests of situational awareness and other questionnaires. We also developed the performance assessment protocol for when participants complete the ventilator assembly task in virtual reality (VR). Last, we finalized the protocol for experimenter interactions with participants during the study, both in their formal interactions and how the experimenter is allowed to react when implementing the "Wizard of Oz" control of the VR environment.

In the area of software development, we added flexible networking capabilities to the VR environment, created the experimenter interface to control interactions in the VR environment, and tested the methods of visual and auditory diminishment. We developed the automated counterbalance scheme for participants and tested all software on a variety of Android phones.

In the area of participant recruitment, we gained Institutional Review Board (IRB) approval for recruiting graduate students at North Carolina State University (NC State) and are in the process of gaining IRB approval to run participants associated with NASA Johnson Space Center (JSC) in Houston.

Stuster, J, Adolf J, Byrne V, Greene M. (2018). Human exploration of Mars: Preliminary lists of crew tasks. NASA/CR-2018-220043. https://ntrs.nasa.gov/citations/20190001401

Other activities in the past year include completing a final research design with all variables specified and the methods of randomization and counterbalancing confirmed. One of the measures needed for the project was a scenario-specific situation awareness questionnaire that had to be created from the scenarios built for the study. To build these scenarios, we needed to gather background research on stimuli present in medical emergencies and on the International Space Station (ISS). To do this, we conducted in depth task analyses. Each of these was then created in the VR environment. To place the participant in a "scenario," we also had to build that scenario, which meant writing and revising a script with enough tasks and stimuli outside of the main task that we could ensure it was sufficiently distracting (but also informative). Once the script was written, it was recorded by actors on the research team and VR models were either added or built to populate the scenarios. We also had to ensure that each scenario had the same amount of time and events to make them comparable.

Stuster, J, Adolf J, Byrne V, Greene M. (2018). Human exploration of Mars: Preliminary lists of crew tasks. NASA/CR-2018-220043. https://ntrs.nasa.gov/citations/20190001401