NOTE: End date changed to 03/07/2024 per NSSC (Ed., 3/8/23)

In this project, we will answer the following research questions: - How does the magnitude of spatial disorientation change in relation to different magnitudes of gravitational cues provided through Earth, Martian, Lunar and 0g analogs? - Will vibrotactile feedback during our disorienting dynamic orientation task be useful? - How does the effectiveness of vibrotactile feedback change in relation to different magnitudes of gravitational cues? - When disoriented, a pilot’s own internal sensory feedback may be misleading. Will vibrotactile feedback be able to correct this misperception of orientation? - If vibrotactile feedback is unable to correct the misperception, will this create confusion? Will pilots be able to rely on and trust the vibrotactors during highly stressful and disorienting conditions? - Will participants learn to use the vibrotactile feedback better with greater exposure to the 0g analog condition? - What types of training can enhance the ability to use vibrotactile feedback to mitigate spatial disorientation? - Does vibrotactile feedback create dependence?

However, the vibrotactile feedback was unable to restore performance, in Martian, Lunar and 0g, to the Earth analog level. This was because participants experienced a feeling of conflict between their perception of orientation and their actual orientation indicated by the vibrotactors. Knowledge of being disoriented and high levels of trust in the vibrotactile feedback were not enough to allow continued learning in the spaceflight analog condition, suggesting that in stressful disorienting conditions that demand fast reactions, trust and knowledge are not enough to ensure reliance on vibrotactile feedback. Instead, a training program, in the Earth analog condition, where participants had to disengage from aligning with gravitational vertical while focusing on vibrotactile feedback was needed to acquire much better performance and sustained learning in the spaceflight analog condition. The training program did not reduce the feeling of conflict however it allowed participants to overcome it. This suggests that as we think about human and sensory augmentation through technology as a countermeasure for spatial disorientation and other problems, we must think about the nature of the training program. Our research shows that for space related tasks where there is a novel environment, the training program should teach astronauts how to disengage from their erroneous sensory system while focusing on the sensory augmentation device.

Our work contributes to a broad effort to enable space exploration with vibrotactile feedback. For example, it could be useful for recognizing alerts (Salzer, Oron-Gilad, Ronen and Parmet, 2011) such as during flight or egress. Vibrotactors could be used as a sensory augmentation device, enhancing performance of manual control tasks (Clément et al.,2018) such as during extravehicular activity (Bakke and Fairburn, 2019). After landing on the surface of a planet or the Moon or returning to Earth, vibrotactile feedback could likely help with postural instability (Sienko et al., 2013; Wall, 2010), and later on, with navigation while exploring the surface (Erp et al., 2005). During flight, vibrotactile feedback could be useful as an aid for maneuvers like sustained hovering (Raj et al., 2000; Lawson and Rupert, 2014). Our research shows that vibrotactile feedback will also be a useful countermeasure for spatial disorientation however will require specialized training. Finally, our work extends the sensory substitution literature (Bach-y-Rita and Kercel, 2003; Bertram and Stafford, 2016) where individuals are usually trained and tested to use a feedback device in the same environment. In our paradigm, individuals are trained to use vibrotactors in one environment (Earth analog) and then tested in a novel environment (spaceflight analog). We find that effective use of the vibrotactors requires not only free exploration (active sensing) but also specialized training that teaches individuals to disengage from one sense while focusing on the vibrotactile feedback. This could be relevant for other work where body systems or environment can change significantly, such as in rehabilitation (Alahakone and Senanayake, 2009; Wall, 2010; Sienko et al., 2013; De Angelis et al., 2021), sports (van Breda et al., 2017), virtual, augmented and mixed realities (Islam and Lim, 2022), human enhancement and augmentation (Raisamo et al., 2019).

In our 2023 study, blindfolded participants were strapped into a Multi-Axis Rotation System (MARS) device that was programmed to behave like an inverted pendulum in the roll plane. Participants used an attached joystick to keep themselves stabilized around the balance point. The direction of balance, in this experiment, was always set to the center (0 deg in the roll plane). In the Earth analog condition (vertical roll plane), participants could use full gravitational cues to determine their angular position. In the Martial analog condition (pitched back by 68 deg) and Lunar analog condition (pitched back by 80 degrees), participants had partial gravitational cues to determine their angular position. In the 0g analog condition, participants could not use gravitational cues to determine their angular position from the balance point. In the experiment, participants first received 8 100-second long trials in the Earth analog condition to learn the task. Then, they received 3 blocks of 4 trials where they had a randomized order of Earth, Martial, Lunar, and 0g analogs. We found that as the magnitude of gravitational cues decreased, performance worsened across a variety of metrics (e.g., number of crashes, standard deviation of angular position, positional drifting, joystick magnitude). As the magnitude of gravitational cues decreased, we also found that participants' self-reported levels of confusion increased when sensing their angular position and, surprisingly, also their angular velocity.

2. Does the effectiveness of vibrotactile feedback change in relation to different magnitudes of gravitational cues?

In our 2023 study, along with the Control group (described above) we also had a Vibrotactile group who had 4 vibrotactors placed on each arm from the shoulder to the wrist. The first vibrotactor (near the shoulder) activated when the MARS deviated 1 deg from the direction of balance, the second at 7 deg, the third at 15 deg, and the fourth (near the wrist) at 31 deg. We found significant differences between the Control and Vibrotactile group for all conditions except the Earth condition. This suggests that the vibrotactile cues are not very useful when full (Earth analog) gravitational cues are provided in a non-disorienting condition; however, they are useful in partial gravity. However, similar to the Control group, the Vibrotactile group's performance worsened as the gravitational levels decreased from Earth to Martian to Lunar to 0g. These results suggest that, without extensive training, the vibrotacile feedback most likely won’t be able to restore performance to Earth levels when exposed to partial-g. Additionally, we found significant increases in self-reported confusion for angular position and velocity in both groups; however, no difference between groups. This means that even at partial-g levels, with considerable gravitational cues (e.g., Martian), the vibrotacile feedback did not correct perception. Instead, participants still felt disoriented and confused; however, they were able to use vibrotactile feedback to perform better than the Control group. All of this suggests that astronauts will be much more susceptible to spatial disorientation in partial-g than in Earth conditions and this susceptibility will worsen as the gravitational level decreases.

3. Does a specialized training program lead to better usage of vibrotactile feedback in the 0g analog condition?

In our 2022 study, we examined learning and training and the findings have been published: Vimal, Vivekanand Pandey, et al. "Vibrotactile feedback as a countermeasure for spatial disorientation." Frontiers in physiology 14 (2023): 1249962. [Ed. Note: See Bibliography.]

All groups in this study did the stabilization task in the Earth analog condition on the first day, and the 0g condition on the second day. The Vibrotactile+Training group received a specialized training program that taught participants how to disengage from relying on gravitational cues while focusing on the vibrotactile feedback. The Vibrotactile group did not receive any training; however, did have exposure and practice on day 1 to become familiarized with the vibrotactors. The OnlyTraining group received the specialized training program; however, without any vibrotactile feedback.

Compared to the OnlyTraining group who received the same training, the Vibrotactile+Training group performed significantly better than the Vibrotactile group in the first block, across a greater number of metrics and with a greater magnitude of significance. They had better positional and velocity based control, fewer crashes, better joystick control, and less positional drifting, which is a characteristic feature of balancing in the spaceflight condition likely caused by poor angular path integration in the absence of gravitational cues. The Vibrotactile+Training group also performed statistically better when compared to the Vibrotactile group. These results show that the training program was effective for the Vibrotactile + Training group and resulted in significantly better performance in early exposure to the disorienting condition (Block 1). Nevertheless, in Block 1 of the spaceflight analog condition, the Vibrotactile + Training group did not perform as well as they had in the final block of the Earth analog condition on Day 1 and showed elevated levels of crashing. Importantly, 90% of both the Vibrotactile and Vibrotactile+Training groups reported confusion and a conflict in which they perceived their orientation to be different from what the vibrotactors were indicating. Therefore, the training did not reduce the feeling of conflict; but it did help the participants overcome this conflict.

4. Do participants learn to use the vibrotactile feedback better with greater exposure to the 0g analog condition?

While participants were informed that they would be in the 0g analog condition on the second day, they were not told that they may experience spatial disorientation. Would participants perform better, after Block 1, once they knew that they were disoriented and that their internal perception of orientation was incorrect? Surprisingly, the Vibrotactile group showed minimal learning on Day 2. By contrast, the Vibrotactile+Training group showed significant learning by improving positional, velocity, and joystick control and reducing the number of crashes. By the fourth block on Day 2, the difference between the Vibrotactile+Training and the Vibrotactile group significantly widened on most measures. Both vibrotactile groups by the end of trial 1 on Day 2 expressed awareness that they were disoriented and that a conflict existed between the perception of their orientation and what the vibrotactors were indicating. There was no statistical difference in their ratings of trust of the vibrotactile feedback between the Earth analog condition and the spaceflight analog condition (84%–92% trust). It is important to emphasize that the Vibrotactile group had both knowledge that they were disoriented and high levels of trust in the vibrotactile feedback, and yet they were unable to rely on the vibrotactile feedback. Why was the Vibrotactile group unable to continue learning even though they knew that they were disoriented and trusted the vibrotactile feedback? One possibility is that they were unable to build an association between their orientation and the vibrotactile feedback. In the sensory substitution literature, effective training often requires free exploration (active sensing) to build a strong association with the new sensory feedback device (Bach-y-Rita and Kercel, 2003; Bertram and Stafford, 2016). Our Day 1 exposure allowed the Vibrotactile group to have this free exploration with the vibrotactors; however, they most likely relied on the gravitational cues to complete the task and not the vibrotactile cues. This is reflected in their responses to the survey, where they did not report any increase in the usefulness of the vibrotactors across the trials; nor did they feel like the device became an extension of themselves on Day 1 or 2, whereas the Vibrotactile + Training group did show a significant increase in their report of usefulness by the end of Day 1, and an increase in the feeling that the device became an extension of themselves on Day 2. These results suggest that to build association between human and device, especially where one is trained and tested in different environments, one must give participants a training condition where the task demands that they exclusively use the device.

5. Does vibrotactile feedback create dependence?

In the final block of Day 2 in the spaceflight analog condition, we deactivated the vibrotactors to determine whether performance would become significantly worse than for the OnlyTraining group. If so, this would signify that the vibrotactors created a negative dependence. We found that in the final block, both the Vibrotactile and the Vibrotactile + Training groups did not perform worse than the OnlyTraining group, and instead showed a slight improvement by having lower mean squared displacements. These results indicate that the vibrotactors did not create a negative dependence, and instead helped the participants acquire a similar level of improvement and learning as the OnlyTraining group who showed significant learning, across blocks, as reflected in decreasing the frequency of crashes and the percentage of destabilizing joystick deflections.

Our work has relevance to other research on vibrotacile feedback where the system or environment can change significantly, such as in rehabilitation (Alahakone and Senanayake, 2009;Wall III, 2010;Sienko et al., 2013;De Angelis et al., 2021), sports (van Breda et al., 2017), virtual, augmented and mixed realities (Islam and Lim, 2022). Our work also has relevance to the larger fields of sensory substitution (Bach-y-Rita and Kercel, 2003;Bertram and Stafford, 2016) and human enhancement and augmentation (Raisamo et al., 2019) and provides insights into how to make stronger connections between feedback devices and the human, especially in novel environments that have not been experienced before.

We create a disorienting spaceflight analog task by placing blindfolded participants into our Multi-axis Rotation System Device (MARS) that is programmed with inverted pendulum dynamics (Figure 1) (Panic et al., 2015). Participants use an attached joystick to stabilize themselves around the direction of balance. When the MARS is oriented in the vertical roll plane (Earth analog condition), participants can use gravitational cues detected by their otolith organs and somatosensory shear forces detected by their skin to determine their angular position relative to the balance point (Vimal et al., 2016). By contrast, when the MARS is oriented in the horizontal roll plane (spaceflight analog condition), they do not tilt relative to the gravitational vertical, and as a result, they cannot use gravity-dependent otolith and somatosensory shear forces to provide a sense of angular position in relation to the direction of balance (Panic et al., 2017; Vimal et al., 2017). They can only use motion cues detected by the semicircular canals and somatosensory receptors. In this condition, as a group, participants show minimal learning, poor performance, and a very high rate of losing control (Vimal et al., 2017; Vimal et al., 2018). Ninety percent of participants report feeling disoriented, and all participants show a characteristic pattern of positional drifting.

In our present study, participants in the Vibrotactile group had 4 vibrotactors placed on each arm from the shoulder to the wrist. The first vibrotactor (near the shoulder) activated when the MARS deviated 1 deg from the direction of balance, the second at 7 deg, the third at 15 deg, and the fourth (near the wrist) at 31 deg. On the first day of experimentation, participants balanced in the vertical roll plane with the vibrotactors (i.e., they trained with natural terrestrial gravitational cues augmented by vibrotactors), and on the second day they were placed in the horizontal roll plane with the vibrotactors (i.e., they were tested in the spaceflight analog condition with vibrotactile cue replacement of missing gravitational cues). We hypothesize that the Vibrotactile group will perform better and show greater learning than the Control+Training group (who received training (see below) but have no vibrotactors) in the spaceflight analog task. However, we hypothesize that the Vibrotactile group will not perform as well in the spaceflight analog condition as they did in the Earth analog condition because the exposure in the Earth analog condition would not be sufficient to teach participants how to rely on the vibrotactors. This is because, in the Earth analog condition, participants primarily rely on gravitational-based cues (Vimal et al., 2017) and would most likely not pay attention to the redundant vibrotactile cues.

In the sensory substitution literature, effective training often includes free exploration (active sensing) with the device to build the appropriate associations between the new sensory feedback and the task (Bach-y-Rita and Kercel, 2003; Bertram and Stafford, 2016). We propose that in addition to free exploration, one also needs to create conditions during training where participants have to rely on the new sensory feedback. The Vibrotactile+Training group received a specialized training program based on our prior work (Vimal et al., 2019), where on Day 1, participants balanced in the Earth analog condition using vibrotactors. Unlike the Vibrotactile group, participants in the Vibrotactile+Training group did not know the location of the balance point, which was randomized and never at the gravitational vertical. Therefore, to complete the task successfully, participants had to disengage from aligning with gravitational vertical and instead had to rely on vibrotactile feedback and motion cues. We hypothesize that the Vibrotactile+Training group will perform better than the Control+Training and Vibrotactile groups in the spaceflight analog condition on Day 2. Finally, to determine whether a negative dependence on the vibrotactors would form, we disengaged the vibrotactors in the last block of the experiment in the spaceflight analog condition, and we hypothesize that performance will worsen however, will not be worse than the Control+Training group.

We found that the Vibrotactile group performed significantly better than the Control+Training group. These findings show that vibrotactile feedback can enhance stabilization performance in a spaceflight analog condition where participants cannot rely on gravitational cues and where they become spatially disoriented. When comparing the final block on Day 1 in the Earth analog condition (vertical roll plane), where participants could use gravitational cues, to the first block on Day 2 of the spaceflight analog condition (horizontal roll plane), we found that all groups performed significantly worse across the majority of the metrics. Why were both vibrotactile groups in the spaceflight analog condition unable to completely recover performance? When asked to report their magnitude of confusion about their self-orientation, all groups reported an average of 300-370% increase in their confusion between Day 1 (Earth analog) and Day 2 (spaceflight analog). When questioned at the end of Block 1 on Day 2, 90% of vibrotactile users from both groups reported that their perception of self-orientation did not match what the vibrotactors were indicating. In another words, when the participants were in the spaceflight analog condition (horizontal roll plane) and experienced disorientation, the vibrotactors led to a feeling of confusion and conflict where participants had to determine whether to follow their inner sense of orientation or use the vibrotactors. These new findings show, for the first time, that during disorienting and high-stress conditions where each participant’s perception of their orientation can be vastly different (Vimal et al., 2022) and where very large errors in perception occur, vibrotactile feedback may not be intuitively and immediately useful.

Perhaps one reason why the Vibrotactile group did not show as significant of an improvement in the spaceflight analog condition was that their exposure to vibrotactors on Day 1 was not enough. Our prior work (Vimal et al., 2017) shows that there are two dissociable components to balance control (i.e., alignment to gravitational vertical and dynamic stabilization), and in the Earth analog condition (vertical roll plane), participants primarily rely on using gravitational cues to align to gravitational vertical. Therefore, it is likely that participants in the Vibrotactile group primarily focused on gravitational cues sensed by their otoliths and touch receptors on their skin and did not pay significant attention to the vibrotactors on Day 1 since they provided a redundant cue. Using as motivation the training program from Vimal et al. (2019), on Day 1 the Vibrotactile+Training group’s task required them to disengage from aligning to gravitational vertical while relying on vibrotactile and motion cues to successfully perform the task. We did this by randomizing the location of non-vertical balance points in the Earth analog condition. Participants did not know the location of the balance point and had to search for them and then stabilize around them. For example, if the balance point was set at 10 degrees from the gravitational vertical, a greater number of vibrotactors would activate as one deviated from an angular position of 10 degrees. In this way, participants had to disengage from their sense of gravitational vertical and focus on the vibrotactors to find the balance point.

Compared to the Control+Training group (who also received the same training), the Vibrotactile+Training performed significantly better in the first block. The Vibrotactile+Training group also performed significantly better in the first block when compared to the Control+Training group on measures that the Vibrotactile group did not. These results show that the training program was effective and resulted in significantly better performance in early exposure to the disorienting condition. Nevertheless, in Block 1 of the spaceflight analog condition, the Vibrotactile+Training group did not perform as well as they did in the Earth analog condition and still showed elevated levels of crashing. Similar to the Vibrotactile group, 90% of the Vibrotactile+Training group reported confusion and conflict where they perceived their orientation differently than what the vibrotactors were indicating. Therefore, the training did not reduce the feeling of conflict, but it did help the participants overcome this conflict.

While participants were informed that they would be in the horizontal roll plane on the second day, they were not told that they might experience spatial disorientation. Would participants perform better once they knew that they were disoriented and that their internal perception of orientation was incorrect? Surprisingly, the Vibrotactile Group showed minimal learning on Day 2, only learning to reduce the frequency of Crashes with a marginal significance. By contrast, the Vibrotactile+Training group showed significant learning across the majority of the metrics. By the fourth block, the difference between the Vibrotactile+Training and the Vibrotactile group widened.

By the end of trial 1 on Day 2, both groups expressed awareness that they were disoriented and that a conflict existed between the perception of their orientation and what the vibrotactors were indicating. At the end of trial 1, there was no statistical difference in their rating of trust in the reliability of the vibrotactors between the Earth analog condition and the spaceflight analog condition (84% - 92% trust). Why was the Vibrotactile group unable to continue learning even though they knew that they were disoriented and that the vibrotactors were provided reliable information? One possibility is that they were unable to build an association between their orientation and the vibrotactile feedback. In the sensory substitution literature, effective training often needs free exploration (active sensing) to build a strong association with the new sensory feedback device (Bach-y-Rita and Kercel, 2003; Bertram and Stafford, 2016). Our Day 1 exposure allowed the Vibrotactile group to have this free exploration with the vibrotactors, however they most likely only used their gravitational cues to complete the task and disregarded the vibrotactile cues. This is reflected in their responses to the survey, where they did not report any increase in the usefulness of the vibrotactors across the trials, nor did they feel like the device became an extension of themselves on Day 1 or 2, whereas the Vibrotactile+Training group did show a significant increase in their report of usefulness by the end of Day 1 and an increase in the feeling that the device became an extension of themselves on Day 2. These results suggest that to build an association between human and device, especially where one is trained and tested in different environments one must give participants a training condition where the task demands that they exclusively use the device.

In the final block of Day 2 in the spaceflight analog condition, we deactivated the vibrotactors to determine whether the performance would become significantly worse than the Control+Training group, which would signify that the vibrotactors created a negative dependence. We found that in the final block, both the Vibrotactile and the Vibrotactile+Training groups did not perform worse than the Control+Training group and instead showed a slight improvement by having less mean-squared displacement. These results signify that the vibrotactors did not create a negative dependence and instead helped the participants acquire a similar level of improvement and learning as the Control+Training group who showed significant learning, across blocks, in decreasing the frequency of crashes and the percentage of destabilizing joystick deflections.