Since the last year’s Task Book report, we have completed two full studies without experimentally added time delay. Results from these zero-latency studies have allowed us to develop a geometrical model to assess and compare operator performance and new analytical methods to quantify operator performance during the teleoperation task. These results have been essential for identifying suitable task difficulty (rotation parameter) levels and latency ranges that we will use in our forthcoming time-delay experiment, for which pilot studies have been completed.
To better understand teleoperation system users’ capabilities under long time delays and, ultimately, to enable examination of compensation techniques for these time delays, we are analyzing the robustness of the classical finding that for delays above ~250 ms operators can no longer maintain smooth coordinated manual control but instead resort to a move-and-wait strategy. To test the generality of this finding and, in particular, to investigate whether the critical latency associated with the onset of move-and-wait behavior is indeed invariant, we have found a quantitatively modulated stressor that directly impacts the difficulty of a manually driven experimental teleoperation task.
The specific task we have selected is a three-dimensional Fitts-like task in which the participants move a cursor from a central starting location to touch a variably sized sphere target. The experiments were conducted in a virtual environment simulation presented in a stereoscopic head-tracked head-mounted display (HMD). The task itself is inspired by the NASA Human Exploration Telerobotics Project’s SPHERES (for Synchronized Position Hold Engage Re-orient Experimental Satellites) element.
The experimental stressor entails interposing a three-axis rotational misalignment between the visual display and manual input coordinates. We have chosen rotational misalignment as a stressor because it represents a problem commonly encountered in teleoperation: that of the remote camera not being optimally aligned with the operator’s input coordinates.
In the zero-latency studies reported here, our primary goals have been to
1) Examine the utility of this stressor in modulating task difficulty level across a broad range, spanning from almost subliminal to levels that make smoothly coordinated three-dimensional manual control almost impossible;
2) Cross-validate several metrics that we will use to quantify the degree of performance disturbance arising from application of the stressor; and
3) Formulate an analytic computational theory that takes into account both the specific geometry of the misalignment and the location of targets to predict the degree of disturbance arising from application of the stressor.
In the previous literature, the problem of telerobot input-to-display rotation has typically been addressed for misalignments about the user’s control yaw axis, but only rarely for pitch or roll misalignment. Moreover, there have been no systematic studies comparing the pattern of the disturbance between these three canonical misalignment axes. The comparison between control-display misalignment axes was a main objective of these initial empirical studies.
We use the term Misalignment Disturbance Function (MDF) for metrics such as movement time and path length that describe the disruption in user performance due to input-to-display rotation. To our knowledge, there has heretofore been no computational theory predicting the MDF based on geometric relation between rotation and target location for this type of task. We believe that the functional form of the MDF will not only be important for developing a theory of user response to control-display misalignment, but also for demarcating the effects that will be attributable to time delay once non-zero latencies are introduced into this specific teleoperation task.
In our first full study with zero latency, we collected data from 20 participants (12 M, 18 F) in order to characterize the control disturbance introduced by our input-to-display misalignment stressor. Each task trial began when the response cursor driven by the participant’s dominant hand was positioned at the central start point and a button press was applied with the nondominant hand. Participants were instructed to “make a smooth coordinated movement as quickly and comfortably as possible” from the start point to the target. Upon cursor contact with the target, the initial center starting point would reappear and the process was repeated for the next trial. Participants practiced this task during an initial familiarization and training phase both with zero input-to-display rotation and with misalignments. The complete movement trajectory for each trial was recorded and time stamped, providing direct measurements of overall movement time and trajectory path length.
The following three independent variables were used in this first study: 1) axis or misalignment rotation in either pitch, yaw or roll with respect to fixed laboratory coordinates; 2) rotation angle about the respective axis at ten discrete levels between zero and 180 degrees with participant-specific randomization; and 3) the rotation axis presentation sequence to balance for the six possible pitch-yaw-roll rotation axis orders. (Two extra participants, yielding the final total of 20, were later included as their data did not affect the planned statistical analyses.) A set of ten trials, each with a different target location, was completed for each rotation angle and axis. All ten rotation angles were completed for one rotation axis before commencing the next axis. Only data from the last seven trials in each set were used; the first three in each set served as practice during which participants could explore the rotation condition. The ratio of target size to distance from the starting point was held constant to maintain a uniform Fitts Index of Difficulty for the seven data trials.
Movement time and trajectory data from all twenty participants, pooled across all experiment conditions, were significantly positively correlated, indicative of a speed-accuracy tradeoff in their performance. Analyses of Variance (ANOVAs) conducted separately on path length and movement time (transformed to correct for data skew) produced identical results. These ANOVAs showed a significant effect of rotation angle and an interaction between rotation angle and axis, but did not reveal a significant effect of the sequence of rotation axis. General features of the data indicate that the MDF, whether expressed in terms of movement time or path length, reaches a peak at or near 120 degrees for all axes and that the peak’s magnitude was significantly greater for rotation about the roll axis.
The results from this study indicate that the MDF, i.e., the effect of the control-display rotation, is not isotropic with respect to axis of rotation: roll produces a distinctly larger nonlinearity and peak magnitude. We conjecture this is due to the displacement arising from the roll misalignment being projecting into the participant’s two key body reference axes, i.e., the lateral (left-right) axis and the vertical axis (usually aligned with gravity), which contrasts with pitch and yaw disturbances being projected into only one of these axes in the frontal plane. Our results for generalized, isotropic three-dimensional motion confirm earlier suggestions (Smith & Smith, 1962) that roll misalignments may be distinctly more difficult.
In addition to the amount of rotation, the orientation of the rotation axis could a key model parameter to predict behavioral consequences of rotations in general. In a second zero-latency experiment, we carefully controlled target location. Data from this experiment (still being analyzed) provide evidence that target direction with respect to the rotation axis predicts aspects of performance.
To estimate spatial discontinuities in recorded movement trajectories, we are developing computational tools to detect “change points” at which spatial and temporal derivatives change abruptly, indicating that participants were not able to locally adhere to our instruction for smooth movement. This discontinuity metric also correlated significantly with movement time and path length MDFs in our experiments.
We have derived a straightforward geometric description of the spiral movement paths experimentally observed for large rotational misalignments. Under the assumption that individual trial movement trajectories lie in a plane perpendicular to the axis of rotation, this geometrically founded model indicates that the MDF is well characterized by a simple secant function for rotations up to ~65 degrees.
The two investigations conducted this year have enabled us to isolate a set of multi-axis rotations and target locations for the design of a non-zero latency version of the experiment that is tractable in terms of expected participant time commitment and effort, while still offering a sufficient span of task difficulty. Pilot tests with these difficulty levels have been completed with three participants, indicating that a small number of discrete latencies settings below one second will enable us to examine with sufficient resolution the hypothesized impacts of interactions between latency and task difficulty in terms of the resultant MDF.