Interactive computing systems frequently use pointing as an input modality, while also supporting other forms of input. We focus on pointing and investigate the effects of variations, i.e. jitter, in the input device latency, as well as dropouts, on 2D pointing speed and accuracy. First, we characterize the latency, latency jitter, and dropouts in several common input technologies. Then we present an experiment, where we systematically explore combinations of dropouts, latency, and latency jitter on a desktop mouse. The results indicate that latency and dropouts have a strong effect on human performance; moderate amounts of jitter in latency do not change performance in a significant way in most cases.
Latency, or lag, is the delay in device position updates [
We present two empirical studies that systematically investigate the effects of dropouts and latency jitter on human performance. The studies employ Fitts’ law, a well-established model of pointing device performance. In our experiments, we used a mouse as an exemplary low-latency, low-jitter device, and artificially added latency and latency jitter to it, to match the range of latencies and jitter present in other commonly used devices, as well as in computer networks. We also varied the number of samples the system was omitting (“dropping”) and the periodicity of such omissions (Experiment 1), or the number and the percentage of the omitted samples (Experiment 2). The main goal was to determine, all else being equal, the effects of dropouts and latency jitter on device performance at varying mean latencies.
As one can often trade some latency for a decrease in latency jitter, typically through time-domain filtering, and extrapolate the missing and delayed samples, knowing the interrelationships between the factors allows a designer to make an informed decision in choosing an appropriate filter and its parameters.
Latency is the time from when the device is physically moved to
the time the corresponding update appears on the screen. For
technical reasons, it is hard to avoid latency. And it is known that
latency adversely affects human performance in both 2D pointing
[
Spatial jitter is caused either by hand tremor or noise in the device signal or both. Some devices also exhibit additional noise during movements. Hand jitter only exacerbates this problem, especially in devices used in free-space. Temporal jitter, or latency jitter, refers to changes in lag with respect to time.
To measure the latency, a video camera simultaneously filmed the motion of both the mouse and the cursor. The average delay of the mouse cursor motion relative to motion of the mouse was determined to be 8 ± 2.8 ms at the centre of the screen. More than 99.5% of the updates happened within 8–11 ms of the previous sample. Practically all of the remaining samples followed within 5–8 ms. We never observed a dropout in a mouse.
Optical sensing method employed by the mouse appears to filter the spatial jitter in hardware. Likewise, hand jitter, or hand tremor, does not appear to be an issue in our experiments, as resting the mouse on a physical surface largely eliminates tremor. Based on our measurements and the fact that our participants were young, we assume the input had no significant jitter of either kind.
The first experiment compares effective throughputs under various magnitudes of latency, time jitter, and dropouts. Twelve students participated in the experiment. The study lasted 40–50 minutes. The software, implemented a standard Fitts’ 2D task of 13 targets in a circle. The experiment was within subjects, and the order in which the various combinations of the factors were presented was randomized (without replacement), to compensate for asymmetric learning transfer effects. Each participant completed 100 “rounds” with different latencies, latency jitters, dropout durations, and dropout intervals, as described below. 0 ms dropouts 125 ms dropouts 250 ms dropouts 500 ms dropouts 5.0 4.0 1.0 0.0 0 50 100
150
The experiment had four independent variables in a (1×1 + 1×2 + 1×3 + 1×4) × (3×3 + 1) = 10×10 arrangement: • Latency (constant part): 10*, 40, 100, and 160 ms; • Dropout duration: 0*, 125, 250, 500 ms; • Intervals between dropouts: 0*, 500, 1000, 2000 ms. • Latency jitter (normally distributed, in addition to the constant value above): σ = 0* ms for 10 ms latency, 0, ±20 ms for 40 ms latency, 0, ±20, ±40 ms for 100 ms latency, σ = 0, ±20, ±40, ±60 ms for 160 ms latency; In the above list, * denotes the baseline condition, i.e., minimum latency, no latency jitter, and no dropouts. We chose a Poisson distribution for dropouts, as it is often used to model independent events, i.e., the time an event occurs does not depend on the previous occurrence. The indices of difficulty (ID), ranged evenly from 2.44 to 5.76 bits.
The dependent variable was effective device throughput.
The effect of latency on throughput was significant, F3,33 = 200.43, p < .0001. The interaction between the latency and dropout duration was also significant, F9,99 = 11.59, p < .0001. Figure 1 illustrates the results.
In this experiment investigate the effect of lower dropout percentages more thoroughly, to determine whether infrequent dropouts still have a measurable effect on throughput. Also, we aim to determine if there is a threshold for dropout duration, after which the throughput starts to drop progressively.
This experiment had three independent variables in a 4 × (5×5 + 1) = 4×26 arrangement, for a total of 104 combinations. In the following list, * denotes the baseline condition, i.e., minimum latency, no latency jitter, and no dropouts. The dependent variable was effective device throughput (in bits per second). All other aspects were similar to the preceding experiment • Latency (constant): 10*, 40, 100, and 160 ms; • Dropout duration: 0*, 10, 20, 40, 80, 160 ms; • Dropout percentage: 0*, 1, 2, 5, 10, 20%.
The effect of latency on throughput was significant, F3,33 = 359.40, p < .0001. No other significant interactions were observed. Figure 2 illustrates the results.
The effect of dropout duration on the throughput was significant, F5,55 = 3.08, p < .05. According to a Tukey-Kramer test, only the 160 ms condition was different from the others. The effect of dropout percentage on the throughput was significant, F5,55 = 16.55, p < .0001. According to a Tukey-Kramer test, no statistically significant difference exists between the 0, 1, 2, and 5% conditions. The interaction between the dropout percentage and duration was significant, F16,176 = 2.18, p < .01.
For low latencies, below approximately 40ms, we observed no significant differences in throughput, consistent with the first experiment and a previous study [5]. The significant interaction between latency and dropout percentages seems to be due to the 20% dropout condition, which has a significant drop of performance, F1,11 = 8.17, p < .05, even at low latencies, whereas the lower dropout conditions don’t have such behaviour, F1,11 = 0.09, ns; see Figure 2.
For dropout durations of up to 80ms, there seems to be no significant effect on throughput, F4,44 = 0.48, ns. For dropout percentages up to 5% we observe no significant drop in performance, relative to the no-dropout condition. Looking at dropout durations of 160 ms we see a significant drop in performance above 5%, F1,11 = 24.54, p < .0001, and no drop before that, F2,22 = 1.93, p = 0.16. However, for lower dropout durations this transition happens at higher percentages, e.g., after 10%, for 80 ms-long dropouts, as can be observed in Figure 2.
One of the surprising findings (Exp. 1) was that latency jitter, that is, variations of latency with time, had little effect on performance, resulting in the worst case in an 8.5% drop in performance at 100 ms base latency and jitter with σ = 40 ms. Compared to the dramatic drops with increasing latency or dropouts, such a small drop is likely to be of little practical significance. Moreover, we can hypothesize that a higher, yet constant, latency could result in worse performance compared to just keeping the latency variations at their original level. 5.0 4.0
For small dropout durations (up to 40 ms), dropout percentages can be relatively large (up to 20%), without noticeable effects on performance. On the other hand, longer dropouts (e.g. 160 ms) have significant effects even at low percentages (5% and more).
While long dropouts have a dramatic impact on performance, they are encountered in fewer situations, and, overall, their impact on performance is either similar to, or lighter than the impact of frequently encountered latency levels. Initial indications exist that interpolating dropouts by filtering may be of little or no use: for short intervals – because short dropouts have little effect on performance, and for large dropouts – due to this not being feasible. To summarize, while both latency and dropouts have detrimental effect on pointing performance, normally distributed latency jitter seems to have no noticeable effects. Filtering in order to combat latency jitter may actually be harmful, as the filter-added latency may outweigh any potential advantages.
Finally, we estimate that both latency and dropout duration are multiplicative factors for predicting the throughput. This suggests incorporating them into a homogeneous model for estimating the human pointing performance in the presence of latency and dropouts. This is a subject of future research. [5] Pavlovych, A. and Stuerzlinger, W. 2009. The tradeoff between spatial jitter and latency in pointing tasks. In Proc.
EICS '09. ACM Press , 187-196. [6] So, R. H. Y., and Chung, G. K. M. 2005. Sensory Motor Responses in Virtual Environments: Studying the Effects of Image Latencies for Target-directed Hand Movement. IEEE Engineering in Medicine and Biology Society, 5006-5008. [7] Software as a service.
http://en.wikipedia.org/wiki/Software_as_a_Service [8] Teather, R., Pavlovych, A., Stuerzlinger, W. and MacKenzie, S. 2009. Effects of tracking technology, latency, and spatial jitter on object movement, IEEE 3DUI 2009, 43-50. [9] Ware C., and Balakrishnan, R. 1994. Reaching for objects in VR displays: lag and frame rate. ACM TOCHI 1, 4, 331-356.