The goal of this sensorimotor/human factors project was to develop a virtual reality (VR) based training method for astronauts aboard International Space Station (ISS) or a Mars mission vehicle as a countermeasure of inflight spatial disorientation and navigation. These problems have been frequently reported by crews of Space Shuttle, Mir, and ISS as complicating responses to emergencies. The three-dimensional (3D) architecture and inconsistency of the visual vertical of adjacent quarters and modules, combined with the limited visual experience of crewmembers is the major cause of the problem, identified as a significant risk by NASA. Astronauts normally see the interior of a spacecraft from a variety of body orientations and viewpoints that cannot be simulated on the ground. It requires cognitive skills to interrelate cues perceived in a body centered (egocentric) frame of reference built up directly through navigation and also in an overall (allocentric) frame of reference defined by the spacecraft. Astronauts can either learn this interrelationship inflight, or develop the required cognitive knowledge prior to flight via VR simulation. As a member of NSBRI's Sensorimotor team led by Dr. Oman, we have conducted a series of experiments of 3D spatial orientation and navigation performance in a virtual space station using simulated emergency egress tasks. In the first experiment in a fully immersive virtual environment with a head mounted display, we showed that individual 3D spatial abilities (e.g. mental rotation and perspective taking skills), relative orientation to the environment, and the configuration of the environment influence performance. Subjects trained locally visually upright developed landmark and route knowledge, whereas those who maintained a constant orientation with respect to the entire station during training enhanced sense of direction and 3D cognitive map, and therefore performance in low visibility in a simulated smoke condition. This result suggests that training initially should be performed locally upright, followed training in a constant station orientation, and then trainees should be challenged by trials in randomized orientation. This could be customized based on individual spatial ability and task performance. This study, published in the Aviation, Space and Environmental Medicine, was awarded the 2007 ASMA Space Medicine Branch Young Investigator Award among 177 nominees. In the second experiment, it was shown that most 3D navigation performance measures for this egress task were similar in the immersive and non-immersive VR systems. Subjects pointed out that this egress task was mainly "done in your head", and that vestibular cues were not critical. This finding is important, since it suggests that laptop trainers (analogous to DOUG for EVA training) could be used for preflight (or even inflight) emergency egress navigation training. Based on these results, this project intended to clarify whether VR training can help to develop cognitive skills and to learn retention, improvement, and limitation of 3D human spatial orientation and navigation for long-term training. In the experiment, we demonstrated that "see-through walls" and a miniature 3D model of the environment by VR technology features were useful. Subjects trained with those VR tools showed better performance than those without at the training day, but same in both groups in one month later. This result showed the effectiveness of preflight spatial orientation and navigation training, especially in early stage of learning. Taken together, these studies provide solid laboratory validation for a preflight VR based navigation training countermeasure at the CRL 7 level. The next step is CRL 8 validation with human subjects in spaceflight to demonstrate operational feasibility and efficacy.

By gaining better cognitive map of the environment, motion sickness and Visual Reorientation Illusions could be reduced.

The simulation tool could be used to train other profession such as firefighters and submariners, as well as occupants of high-story buildings.

Results also support deep understanding in human from the viewpoint of brain and cognitive science. Our results also pertain to environmental and architectural design and pre/post-occupancy evaluation of buildings, underground, and cities.

We also continued an experiment to compare 3D spatial orientation and navigation performance with immersive and non-immersive VR simulation tools. Although immersive displays probably better simulate the vestibular and haptic cues required for spatial orientation, the subjects showed almost same performance using non-immersive desktop display.

The goal of this sensorimotor/human factors project is to develop a virtual reality (VR) based training method for astronauts aboard International Space Station (ISS) or a Mars mission vehicle as a countermeasure of inflight spatial disorientation and navigation. These problems have been frequently reported by crews of Space Shuttle, Mir, and ISS as complicating responses to emergencies. The 3D architecture and inconsistency of the visual vertical of adjacent quarters and modules, combined with the limited visual experience of crewmembers is the major cause of the problem, identified as a significant risk by NASA. Astronauts normally see the interior of a spacecraft from a variety of body orientations and viewpoints that cannot be simulated on the ground. It requires cognitive skills to interrelate cues perceived in a body centered (egocentric) frame of reference built up directly through navigation and also in an overall (allocentric) frame of reference defined by the spacecraft. Astronauts can either learn this interrelationship inflight, or develop the required cognitive knowledge prior to flight via VR simulation. This study intends to clarify whether VR training can help to integrate egocentric and allocentric frame of reference and to understand retention, learning, and the limitations of 3D human spatial orientation and navigation for long-term training. In the experiment, two groups (Control, Treatment) of subjects explore a virtual ISS while wearing a head-mounted display with head tracker. In Training, two groups are trained in a different manner but have the same total training time. The control group learns each module separately, while the treatment group learns the whole ISS at once. A virtual 3D space station model is also available to the treatment group. In Testing, the subjects are told their destination and are asked to point there. The visibility is sometimes obstructed by smoke. Upon arrival at the destination they point back to the start point and reproduce the experienced route using a virtual scale model. Correct answers for the pointing and route reproduction tasks are provided as feedback only for the treatment group. Testing is also done 1, 7 and 30 days later, where only the treatment group is told error types they made in the previous testing. The treatment group should show quantitatively superior spatial knowledge and navigation skills. The results should help define procedures for actual astronaut preflight spatial disorientation and navigation training.

The space station 3D model we have made for the previous experiments was similar to ISS, and consisted of seven rectangular modules (Destiny, JEM, Columbus, CAM, Zvezda (Service Module), and Soyuz), three cubic modules (Node1, 2, and a Russian node), and a PMA. The pictures of the ISS modules in orbit, ground mockups, and illustrations were used for the interior texture. The module size, shape, and location were, however, modified for experimental purposes. Now we are developing the 3D models to be closer to the ISS. With help of Drs. Edna Fiedler of NSBRI and Barbara Woolford of JSC, we have better ISS interior/hardware photos. A better 3D model of ISS was provided by Jeffery Murch and Patrick Troutman at NASA Langley Research Center.

Four new functions have been installed into the VR tool for the next experiment.

i) Outside-view map

ii) See-through (virtual X-ray vision) function

iii) Background sound

iv) Fog on/off

2. ISS emergency training

We observed ISS emergency training for Expedition 15 crewmembers on July 14, 2006. The training was performed by ISS Environmental Control Group (JSC-DT4) led by David Hudson. They simulated various situations such as decompression, toxic gas leak, and fire with smoke. Obviously, due to the physical conditions of the mockup trainers in Building 9 (not the same as the flight configuration), the training focus on the procedures at particular places in a module and less on inter-module activities. During and after the NSBRI's Summer Institute program, Dr. Aoki discussed with them how to define and incorporate a reasonable scenario involving 3D intermodule spatial activities into the training tool we have been developing. We are working to finalize some of the scenarios and include them in the tool. As soon as a scenario is installed in the tool, it will be evaluated by Hudson's group and hopefully ultimately by some astronauts.

3. A portable immersive VR system was developed with a laptop, a lightweight HMD and a 6 DOF head motion sensor. This system could be run only from laptop battery power supply. This can be used for demonstration and future experiments on an air-bearing bed and/or parabolic flight.

The goal of this sensorimotor/human factors project is to develop a virtual reality (VR) based training method for astronauts aboard International Space Station (ISS) or a Mars mission vehicle as a countermeasure of inflight spatial disorientation and navigation. These problems have been frequently reported by crews of Space Shuttle, Mir, and ISS as complicating responses to emergencies. The 3D architecture and inconsistency of the visual vertical of adjacent quarters and modules, combined with the limited visual experience of crewmembers is the major cause of the problem, identified as a significant risk by NASA. Astronauts normally see the interior of a spacecraft from a variety of body orientations and viewpoints that cannot be simulated on the ground. It requires cognitive skills to interrelate cues perceived in a body centered (egocentric) frame of reference built up directly through navigation and also in an overall (allocentric) frame of reference defined by the spacecraft. Astronauts can either learn this interrelationship inflight, or develop the required cognitive knowledge prior to flight via VR simulation. This study intends to clarify whether VR training can help to integrate egocentric and allocentric frame of reference and to understand retention, learning, and the limitations of 3D human spatial orientation and navigation for long-term training. In the experiment, two groups (Control, Treatment) of subjects explore a virtual ISS while wearing a head-mounted display with head tracker. In Training, two groups are trained in a different manner but have the same total training time. The control group learns each module separately, while the treatment group learns the whole ISS at once. A virtual 3D space station model is also available to the treatment group. In Testing, the subjects are told their destination and are asked to point there. The visibility is sometimes obstructed by smoke. Upon arrival at the destination they point back to the start point and reproduce the experienced route using a virtual scale model. Correct answers for the pointing and route reproduction tasks are provided as feedback only for the treatment group. Testing is also done 1, 7 and 30 days later, where only the treatment group is told error types they made in the previous testing. The treatment group should show quantitatively superior spatial knowledge and navigation skills. The results should help define procedures for actual astronaut preflight spatial disorientation and navigation training.

[Ed. note: FY2006 record created in December 2010 when discovered missing; needed for statistical purposes]