| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Department of Biophysics, University of Nijmegen, 6500 HB Nijmegen, The Netherlands
Submitted 5 May 2003; accepted in final form 15 June 2003
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Most studies on pointing toward remembered visual targets have in common that they showed that the distribution of finger positions for pointing to a remembered target is characterized by an ellipsoid with the long axis of the distribution oriented toward the subject. This has been interpreted as evidence for the hypothesis that the CNS specifies the parameters of the endpoint of the movement separately for distance and direction (Flanders et al. 1992
; Georgopoulos et al. 1984; Gordon et al. 1994), that the movement is planned in terms of displacement from the initial position (Messier and Kalaska 1997; Vindras et al. 1998), or as evidence for movement planning in a viewer-centered frame of reference (e.g., McIntyre et al. 1997). Other studies, however, interpreted errors in pointing as evidence for movement planning in a shoulder-centered frame of reference (Berkinblit et al. 1995; Soechting and Flanders 1989), or in both a shoulder-centered and a head-centered frame of reference (McIntyre et al. 1998; Soechting et al. 1990). The different results regarding the variability in the orientation of the ellipses in these studies are most likely explained by differences in the experimental conditions with which different studies approached the issue: in some studies, subjects had no visual information whatsoever on the environment, nor on their arm (McIntyre et al. 1997), whereas in other studies subjects had feedback on their arm (Berkinblit et al. 1995) or on both the arm and the visual environment (McIntyre et al. 1998; Soechting and Flanders 1989). However, because none of these studies tested subjects for pointing in all these different experimental conditions, it is not clear to what extent the different contributions of information about finger position and about target position relative to the visual environment could explain the different results.
To investigate the effect of vision of the finger and the environment on the pointing errors and in particular the effect of visual feedback of the finger, we measured pointing movements in 3 visual conditions: 1) complete darkness with visual feedback of the finger position; 2) complete darkness without visual feedback on the position of the index finger; and 3) vision of the index finger along with vision of a well-defined visual environment. A previous study by Van Beers et al. (2002) demonstrated that the contributions of visual and proprioceptive information may vary, depending on the experimental conditions. The first aim of this study was therefore to compare the distributions of pointing errors in each of the 3 conditions, to see whether the different results of previous studies could be explained by different contributions of visual and proprioceptive information in different experimental conditions. In particular, we tested whether the error distributions in each condition were oriented toward a single point, which might be interpreted as the center of some frame of reference for pointing (e.g., viewer-centered or shoulder-centered) and whether differences in the experimental conditions would change the location of any such point.
Several studies (Bock 1986; Enright 1995; Henriques et al. 1998; Medendorp and Crawford 2002a) have stressed the importance of fixation of gaze to a target on pointing accuracy by demonstrating that pointing errors increase when gaze deviates from the target position. This observation adds another complicating factor to the interpretation of the error ellipses that have been obtained in previous studies. Eye position was not measured in most of the studies on pointing to remembered targets and fixation might very well have been different in conditions with and without visual feedback about the environment. If pointing errors depend on gaze and if gaze is different with and without visual feedback on the finger and the environment, different orientations of error ellipses might also be attributed to differences in gaze.
Therefore the second aim of this study was to compare the errors in pointing and in gaze as a function of time in the period from target onset until completion of the pointing movement, and to look for the presence (or absence) of a correlation between constant and variable errors of 3-D gaze and of 3-D finger position during pointing.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Fifteen subjects (aged 21–49 yr) participated in these experiments. Ten subjects participated in the first experiment and 6 subjects participated in the second experiment. One subject (RK) participated in both experiments. All subjects had normal or corrected-to-normal vision and all gave informed consent. The experimental protocol was approved by the Medical Ethical Committee of the University of Nijmegen. None of the subjects had any known history of neurological sensory or motor disorders. All subjects were right-handed, except for subject MA, who participated only in the second experiment. Pointing movements were performed with the right hand, unless explicitly indicated otherwise. Three subjects (MA, NK, and SG) were familiar with the aim of this study. Their results were not different from those of the other subjects.
Experimental paradigm
Subjects were standing in a completely dark room. An L-shaped obstacle was attached on the floor to offer the subject a reference to maintain the correct location in the otherwise dark room in all 3 test conditions. Seven red light-emitting diodes (LEDs) were attached on the vertices of two 30 x 30 x 30 cm3 cubes, next to each other, about 25 cm in front of the subject (see Fig. 1). Each of these 7 LEDs served as a target for pointing movements in the first experiment. Targets 1, 4, and 5 were used in the second experiment.
|
The onset of the target marked the start of a trial. After 1 s the target LED was switched off and the cubes with targets were canted away. Two seconds after target disappearance an auditory signal notified the subject to start the pointing movement to the remembered target. Subjects were instructed to wait for the auditory signal before positioning their index finger at the remembered target position, and to keep it at the position of the remembered target for at least 0.5 s. Subjects could freely move their head and eyes and no explicit instruction was given about where to direct gaze.
Three visual feedback conditions were tested: pointing in complete darkness (DARK), pointing with feedback by means of a red LED on the tip of the index finger that was visible at all times (FINGER), and pointing in the presence of an illuminated cubic frame with a continuously lit red LED attached on the tip of the index finger (FRAME). In the latter condition, a well-defined visual environment was shown to the subject by means of illuminated optic fibers along the edges of a cubic frame of 90 x 90 x 90 cm3 (see Fig. 1). The surface of the optic fibers was roughened by sandpaper and red LEDs at the long ends of the optic fibers gave the optic fibers a red color. The frame was visible at all times in the FRAME condition. All targets were within this illuminated cubic frame, well within reaching distance from the subject.
In the first experiment, we investigated pointing movements without measuring gaze, and tested pointing movements to the 7 targets in each of the 3 visual feedback conditions. The visual feedback conditions were tested in pairs (DARK–FINGER and FINGER–FRAME). Measuring 2 feedback conditions took about 1 h, and measuring all 3 feedback conditions in one experimental session would exceed the maximum amount of time that the subjects could remain concentrated. Four subjects participated in both pairs of conditions, and 4 other subjects participated in either the DARK–FINGER pair or the FINGER–FRAME pair. As a result, we tested 6 subjects in the DARK and in the FRAME conditions and all 8 subjects were tested in the FINGER condition.
In this experiment, we also examined the influence of the effector arm (left or right) for pointing to targets in the same workspace relative to the shoulder of the pointing arm. Therefore we also had subjects to point with the left arm in the FINGER condition. Targets were presented at mirror-symmetric locations relative to the left shoulder. Four subjects performed pointing movements with the left and right arms (LEFT–RIGHT pair) and 2 subjects performed pointing movements with the left arm only (LEFT ONLY). This resulted in a total of 6 subjects performing pointing movements with the left arm in the FINGER condition. Two of the subjects who pointed in the LEFT–RIGHT condition pair had not participated in the DARK–FINGER or FINGER–FRAME pairs. Therefore the total number of subjects that performed pointing movements with the right arm in the FINGER condition increased to 10. Six subjects were tested twice in the FINGER condition for pointing with the right arm (see Table 1).
|
The targets were presented in a randomized order in 16 blocks of 20 trials each. In each block, only one visual feedback condition was tested. For each of the visual feedback conditions, each target was presented 20 times, except for the target that was closest to the subject's eyes (target 4), which was presented 40 times. A block with 20 trials typically lasted about 3 min, and after each block, room lights were switched on for about 1 min to avoid dark adaptation.
In the second experiment we measured pointing movements and binocular eye movements using the search-coil technique. In this experiment, subjects were tested in each of the 3 conditions (DARK, FINGER and FRAME) in one experimental session. Because the duration of these experiments had to be restricted to 45 min (because of the limited time available to wear the search coils without discomfort), only targets 1, 4, and 5 were used, which were presented 
13 times each in each condition. All 3 targets were at eye level. Targets were presented in a randomized order in 6 blocks of 20 trials each. In each block only one visual feedback condition was tested. Blocks with different visual feedback conditions were tested in randomized order.
Experimental setup
The position of several segments of the subject's body and the position of the targets were measured with an OPTOTRAK 3020 system (Northern Digital), which measures the 3-D position of infrared light–emitting diodes (IREDs) with a resolution better than 0.2 mm within a range of about 1.5 m3. The OPTOTRAK system was mounted on the ceiling above the subject at a distance of bout 2.5 m to the right of the subject, tilted downward at an angle of 30° relative to the ceiling. When pointing movements with the left arm were measured, the subject and the framework with targets were rotated 180°, for better visibility of the pointing arm to the OPTOTRAK system. The position of IREDs was measured with a sampling frequency of 100 Hz.
IREDs were placed on the subject's shoulder (acromion) and elbow (epicondylus lateralis). The position of the tip of the index finger was measured by means of an IRED attached to a thimble on the index finger. This thimble also contained a visible red LED that provided the subject with feedback on finger position in the FINGER and FRAME conditions. When gaze was measured, subjects were wearing a helmet with 6 IREDs, which were attached in such a way that at least 3 IREDs were visible for the OPTOTRAK system at all possible head orientations. This was necessary to calculate 3-D head orientation at all times, such that eye position could be reconstructed from head position (see following text).
Gaze was measured using the scleral search-coil technique (Collewijn et al. 1975) in a large magnetic field system (Remmel Labs). This system consists of a cubic frame of welded aluminum of 3 x 3 x 3 m3, which produced 3 orthogonal magnetic fields at frequencies of 48, 60, and 80 kHz. During these experiments subjects were tested such that the search coils were close to the center of the large magnetic field system. Care was taken that the calibration of the eye coil signals was performed in the same region of the magnetic field where the actual measurements took place. During the calibration procedure subjects fixated a series of red LEDs attached to a board at a distance of 90 cm in front of the subject. The LEDs were arranged at 3 circles of different radius (15, 27.5, and 37.5°), concentric around the straight-ahead direction. With this setup, calibration errors (defined as twice the SD) were typically about 0.5° in azimuth and 1° in elevation on average; resolution was <0.04°. As a result, the errors in the 3-D gaze position—resulting from calibration errors in the orientation of the 2 eyes in space—were on average about 0.6 and 1.1° in azimuth and elevation, respectively, and 3 cm in radial distance from the cyclopean eye.
Two PCs controlled the experiment, one of which was equipped with hardware and software for the collection of the search-coil data and with software for the stimulus presentation. The second PC contained hardware and software to collect the IRED data from the OPTOTRAK system, and was controlled by the first PC to synchronize the IRED data collection (second PC) with the collection of the search-coil data (first PC). Coil signals were sampled at 500 Hz. The OPTOTRAK system collected position data from the IREDs at a sample frequency of 100 Hz. In off-line analyses, the coil signals were resampled at a 100-Hz frequency by cubic spline interpolation.
Data analysis
We distinguish 2 types of pointing errors: the constant error, which is the distance between the LED position of a target and the average of all pointing positions toward that target, and the variable error, which reflects the distribution of the pointing positions toward a target relative to the average pointing position to that target. Pointing position is defined as the position of the IRED on the tip of the index finger at the end of the pointing movement toward the target. The distribution of the pointing positions for a target i is described by the 3-D covariance matrix Si. The 3 orthogonal eigenvectors of the covariance matrix Si describe the orientations of the variable errors. The corresponding eigenvalues of the matrix give the size of the variable error along the eigenvectors. These eigenvalues of the covariance matrix Si can be scaled to compute the limits that contain 95% of the data (see McIntyre et al. 1998 and Morrison 1976). A 
2 test was used to decide whether the 3 eigenvalues of the covariance matrix were statistically different (see Barlow 1989). In all figures we display only contours of the 95% confidence ellipses when one of the eigenvalues is significantly larger than the other 2 (P < 0.05). The eigenvector that corresponds to the largest eigenvalue will be referred to as the main axis of the distribution. We derive the accuracy of the main axis by means of a bootstrap method (see e.g., Mooney and Duval 1993). From the 20 data points for each target, we drew a random sample of 1,000 data points. For this artificial data set of 1,000 points we calculate the 3-D covariance matrix and determined the corresponding main axis. This procedure was repeated 500 times for each target, which resulted in 500 main axes per target. From the distribution of the 500 main axes we estimated the accuracy of the orientation of the ellipsoid.
Intersection point of confidence ellipsoids
Assuming that the ellipsoids for all targets are oriented toward one single point in space, one can find this point by estimating the intersection point of the main axes of the ellipsoids. We determined the accuracy of the orientation of the main axis of each ellipse by a bootstrap method. Because the orientation of the main axes can be determined only up to a certain accuracy, there will hardly ever be one single position in 3-D at which all main axes intersect exactly. Therefore we used a maximum log-likelihood method to determine the most likely position of the hypothetical intersection point. Finding the most probable intersection point in 3-D space given the pointing data set (D) corresponds to maximizing 
, where represents the probability that the intersection point is at position 
, given the data set D. Finding the most probable intersection point for multiple data sets Di, corresponding to the targets i, is equivalent to maximizing the product of the probabilities, according to
 |
). For a normal distribution, 
is proportional to
 |
 |
corresponds to the distance of position relative to the main axis of the ith distribution 
. 
i is the covariance matrix describing the SD of the data distribution for target i. When the data sets Di are normally distributed, maximizing the product of probabilities thus corresponds to minimizing
 |
is known as the Mahalanobis distance (see Duda and Hart 1973). The most probable intersection point given the data corresponds to the minimum of the sum of Mahalanobis distances. The basic idea behind this method is schematically displayed in 2-D in Fig. 2: For 3 targets we show hypothetical 2-D error ellipses. For each error ellipse, the probability of finding an intersection point decreases with distance relative to the main axis for that error ellipse. This probability distribution, orthogonal to the main axis, corresponds to a normal distribution. The SD of this normal distribution depends on the ratio between the largest eigenvalue of the covariance matrix Si and the smaller eigenvalue, and on the distance along the main axis relative to the center of the error ellipse. The most probable location for the intersection point corresponding to the minimum of the sum of Mahalanobis distances is indicated by a star.
|
To study the relation between gaze and pointing, we use the covariance between pointing position at the end of the pointing movement and gaze, which changes as a function of time
 |
represents gaze in trial i for target j as a function of time. 
represents the pointing position for trial i for target j. 
and 
represent the mean gaze as a function of time for target j and the mean pointing position for target j, respectively. Note, that gaze is a function of time, whereas pointing position p is not. Therefore any variations in the covariance between gaze and pointing in time are a consequence of changes in gaze as a function of time. |
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Figure 3 shows the main results for a typical subject (MV) when pointing with visual feedback of the fingertip (FINGER condition, Fig. 3, A and B), in the absence of visual feedback (DARK condition, Fig. 3, C and D), and with both vision of the environment and feedback of the fingertip (FRAME condition, Fig. 3, E and F). The top panels show top views on the 3-D position of the fingertip (Fig. 3, A, C, and E), and the lower panels show side views (Fig. 3, B, D, and F). In each panel we have also drawn a fictive subject to indicate the position of the subject relative to the targets.
|
As described in METHODS, we distinguish between a constant error and a variable error. Figure 3, A and B, shows that with vision of the tip of the index finger (FINGER condition) the constant errors are on average about 5 cm (range 3 to 8 cm for the different target positions). These constant errors are representative for all subjects: averaged over all subjects and all targets, the constant error is 5 cm (SD = 2 cm).
Figure 3, A and B, shows that the variable error is in general largest along an axis that is oriented toward the subject. Variable errors along the 2 minor axes of the ellipse are about the same in size and are much smaller than errors along the main axis. As a result, the distribution of pointing positions has a significant orientation for most targets, and the distribution is indicated by an ellipse (see METHODS).
Lack of vision of the tip of the index finger (DARK condition) leads to larger pointing errors than in the FINGER condition. Both constant and variable errors increase compared to that in the FINGER condition (compare Fig. 3, C and D and A and B, respectively). The average constant error in Fig. 3, C and D is about 7.5 cm (range 4 to 11 cm for different target positions). The constant error is mainly in the radial direction from the subject toward the target. Over all subjects, the average constant error is about 9 cm (SD = 4 cm), but pointing errors up to about 15 cm were observed. Averaged over all subjects, the constant error is significantly larger in the DARK condition than in the FINGER condition (t = 5.79; P < 0.05).
Similarly, the variable error is significantly larger in the DARK condition than in the FINGER condition (t = 6.05; P < 0.05). This is mainly caused by a large increase of errors in azimuth and elevation direction. Errors in distance are not significantly different from those in the FINGER condition. Because of the increased error in azimuth and elevation, the variable error is about the same in all directions for most target positions in the DARK condition. Fitting an ellipse to the data did not usually produce an ellipse with a clear orientation. For the data in Fig. 3, C and D, the orientation for the variable error is significant only for the leftmost target.
Figure 3, E and F, shows data for pointing toward remembered targets with vision of the index finger and with vision of an external frame. In this condition (FRAME) the average constant error for this subject is about 4 cm, ranging from 3 to 6 cm. Averaged over all subjects, the constant error in the FRAME condition is 4 cm (SD = 2 cm). This is significantly smaller than in the other 2 conditions (t = 7.08 and t = 2.48; P < 0.05, for DARK and FINGER, respectively).
The variable error is also smaller than that in any of the other 2 conditions. This effect is significant across subjects (t = 5.91 and t = 3.00; P < 0.05, compared with DARK and FINGER, respectively). The decrease in variable error is found especially along the axes in which the variability was already smallest (azimuth and elevation). Because the variability along the long axis of the distribution decreases relatively little, this results in a more pronounced orientation tuning of the 95% confidence ellipses.
Thus constant errors decrease when visual feedback of the finger is provided and decrease even further when additional feedback of the environment is presented by means of the illuminating frame. Providing visual feedback also results in a decrease in variable errors, mainly in azimuth and elevation direction, and hardly in radial distance.
Frames of reference for pointing movements
Figure 3 shows pointing responses toward 7 targets for one subject in 3 conditions. For the DARK condition most ellipses do not deviate significantly from a spherical distribution. For the FINGER and FRAME conditions, most ellipses do have a long axis with a clear orientation, usually oriented toward the subject. Therefore the analysis to test various hypotheses regarding the frame of reference for pointing, based on the search for a common origin of the main orientations of the variable error distributions, was limited to the FINGER and FRAME conditions.
Using the maximum likelihood estimation procedure (see METHODS) we determined the location of the most likely intersection point of all long axes of the error ellipses for each subject. The results of this analysis are shown in Fig. 4. For each of the subjects, the point that most likely serves as the intersection point of all ellipses is indicated by a star. The circle indicates the most likely intersection point for the data shown in Fig. 3. We tested 6 subjects in the FRAME condition (Fig. 4, A and B), and 10 in the FINGER condition (Fig. 4, C and D). Although the intersubject variability in the location of the intersection point is rather large, all subjects seem to show a most likely intersection point close to or in front of the eyes for both conditions.
|
Some studies in the past have suggested that the distributions of pointing errors are directed toward the shoulder (Soechting and Flanders 1989) or toward a position between the head and the shoulder (Soechting et al. 1990). The left and central panels of Fig. 4 show that the most likely intersection points of the subjects in our study do not lie close to the right shoulder in either condition. To test whether the location of the intersection point is affected by the pointing arm, we asked 6 subjects to point with their left arm instead of the right arm in the FINGER condition. Figure 4 (right panels) shows a top view and a side view of the most likely intersection points for these 6 subjects. Clearly, the intersection points lie close to the subjects' head and the locations of the intersection points for pointing with the right versus the left arm are not very different.
Table 2 shows the coordinates of the most likely intersection points relative to the cyclopean eye for all subjects who pointed with either the left or the right arm in the FINGER condition. A statistical analysis showed that the sideward location of the intersection point was not significantly different for pointing with the left or the right arm (t = 1.36, P = 0.19). For both arms the intersection point seems to lie just in front of the cyclopean eye.
|
The relation between gaze and pointing position
To investigate to what extent pointing and gaze are related, binocular gaze was measured. In these experiments only a subset of the targets was tested in each of the 3 feedback conditions because of the limited time available to wear the search coils. Nevertheless, the results provide evidence for a relation between pointing and gaze as a function of time, as will be illustrated below.
Figure 5 shows gaze at 3 different moments in time relative to target onset for the 3 visual conditions, FINGER (Fig. 5, A–C), DARK (Fig. 5, D–F), and FRAME (Fig. 5, G–I), for subject BB. The data are shown at 0.9 s after target onset (when the subject fixates the visible target, left panels); 2.9 s after target onset, which corresponds to 1.9 s after target offset (just before movement onset, middle panels); and 4.9 s after target onset (when the fingertip points at the remembered target position, right panels).
|
Figure 5, A, D, and G, shows that subjects fixate close to the target in almost all trials, when the target has been visible for 0.9 s. In the FRAME condition, subjects sometimes fixate at a point between the target and the back plane of the visible frame. At the end of the delay period, 1.9 s after target disappearance (Fig. 5, B, E, and H), gaze has drifted away from the target in a radial direction to a larger distance from the subject in all 3 conditions, but most clearly in the FRAME condition. During this drift period, the direction of gaze remains almost the same (i.e., any changes in azimuth and elevation are small). The amplitude of the drift at the end of the delay period depends on visual feedback of the environment. The amount of drift is about the same in the FINGER and DARK conditions, but is considerably larger in the FRAME condition (compare Fig. 5, B and E and Fig. 5H).
Figure 5, C, F, and I (right side), shows gaze at the time when the subject is pointing to the remembered target position. In these panels, the variability and location of the corresponding pointing positions are indicated by the error ellipses, which capture 95% of the pointing positions. Figure 5, C, F, and I, shows that gaze is much more variable at the time of pointing than at the time of fixation to the visible target (Fig. 5, A, D, and G). Gaze and pointing seem to overlap quite well for the FINGER condition, but less so for the DARK and FRAME conditions (Fig. 5, F and I), where gaze locations at the time of pointing are farther away from the subject than the corresponding pointing positions.
In Fig. 5 we showed that gaze does not remain fixated to the target position throughout the trial, but changes during the delay period. To test whether errors in gaze (at some period in time) and pointing position are related, we analyzed the constant and variable errors in gaze as a function of time in relation to the constant and variable errors in pointing.
Figure 6 shows the constant errors in gaze averaged over all subjects, for each of the 3 targets. Similar to the constant error in pointing position (see METHODS), we define the constant error in gaze as the deviation of mean gaze position from the target position. Because the average gaze position changes during a trial, the constant error in gaze also changes in time. The constant error in pointing position does not change during one trial, given that by definition the pointing position is the mean position of the tip of the index finger at the end of each pointing movement. The constant pointing error is indicated by a horizontal line to simplify a comparison with the constant error in gaze.
|
Figure 6 shows that the constant gaze errors in elevation (right column) show a more or less constant offset, slightly above (target 1) or below (targets 4 and 5) the target. These small deviations of about 2 deg or less may represent incorrect fixation to the target by the subjects, but they could also be attributed to small errors in calibration of 3-D gaze position (see METHODS). The most interesting effects are found for radial distance relative to the cyclopean eye (R) and azimuth angle (
), which are displayed in the left and middle columns, respectively.
In all conditions, the average radial distance of gaze at the end of target presentation lies within 2 cm from the target, except for the most distant target (target 1), for which gaze falls short by about 5 cm. After target disappearance gaze distance increases for all 3 targets and in all conditions, compatible with the drift in gaze described earlier in Fig. 5. The amount of drift away from the subject and the duration of this gaze drift are different in the 3 conditions.
In complete darkness (FINGER and DARK conditions, Fig. 6, A and D) gaze distance at the end of target presentation matches the target's radial distance well. During the 2-s delay period, gaze slightly drifts away from the subject by about 3 to 10 cm. When the finger is visible during pointing (FINGER condition), gaze returns back to the radial distance of the initial gaze position when the pointing movement starts, such that gaze and pointing position match quite closely when the subject points at the target (compare the traces for gaze error and the corresponding horizontal line for pointing error in Fig. 6A).
In the DARK condition when the finger is not visible during pointing (Fig. 6D), gaze distance relative to the cyclopean eye does not return to the target distance, nor to the pointing distance (flat lines in Fig. 6D). The average pointing error in radial distance in the DARK condition is much smaller than the average error in gaze at the time of pointing (on average about 2 vs. 7 cm, respectively).
Figure 6G shows the constant errors in radial distance for the FRAME condition: gaze distance corresponds well to the target distance at the end of target presentation. When the target disappears, gaze rapidly drifts away from the subject in the radial direction. Figure 6G shows that gaze moves back toward the target position after the cue to start the pointing movement. However, at the time of pointing the decrease in gaze distance does not completely compensate for the drift in the delay period (as it did for the FINGER condition), and thus gaze does not correspond to the target distance, nor to the pointing distance, which is close to that of the target.
For constant errors in azimuth (Fig. 6, middle columns), there is a clear distinction between targets 1 and 4 and target 5. For targets 1 and 4, which lie at eye level almost straight in front of the subject (see Fig. 1), gaze errors in azimuth direction are small (less than 2° from the target) in all 3 conditions (Fig. 6, B, E, and H). Gaze azimuth remains almost constant from target offset until the end of pointing. For target 5, however, which lies about 30° eccentric to the left at eye level (see Fig. 1) there are clear differences between the 3 feedback conditions: at the end of target presentation, gaze azimuth corresponds within 2° from the target position in all 3 conditions. After target disappearance, however, gaze drifts by about 3° to the right of target 5 (decrease in azimuth, corresponding to a more straight-ahead direction) in the FINGER and DARK conditions (Fig. 6, B and E). Similar to the effects described above for gaze distance, drifts in azimuth are compensated in the FINGER condition at the time of pointing, such that gaze direction returns to the target direction. In the DARK condition, gaze direction remains to the right of the target. In the FRAME condition, the visual feedback of the environment seems to prevent large drifts in azimuth for all targets (see Fig. 6H).
To summarize, as long as the target is visible, gaze is directed toward the target. After target disappearance gaze tends to drift away from the subject in the radial direction. In the DARK and FRAME conditions gaze remains too far from the cyclopean eye, relative to the target, whereas in the FINGER condition gaze almost completely returns at the time of pointing. In the FINGER condition, pointing errors correspond closely to gaze errors, which is not surprising given that the finger is visible during pointing. In the DARK condition, pointing errors are smaller than gaze errors, mainly because gaze errors are primarily attributed to drift from the target position. In the FRAME condition pointing errors are small, but gaze is directed to a position between the pointing position and the visual background.
To study the effect of variable errors in gaze on the variability of the pointing positions or vice versa, we compared gaze in time with the corresponding pointing position. There is only one pointing position per trial, but gaze may vary in time. Therefore we tested whether there is a moment in time when the variability in gaze is closest related to the variability in pointing position (see METHODS).
Figure 7 shows the covariance between the pointing position and gaze position as a function of time, averaged over all 6 subjects. The average covariance for radial distance (R), azimuth (), and elevation (
) is shown in the left, middle, and right panels, respectively. We tested whether the time when the highest covariance was reached was related to a specific stage in the delayed pointing task. Therefore we focus on 2 time intervals: the interval of target presentation (from 0 to 1 s) and the interval during which the finger points to the target (on average from 4.3 to 5.4 s after target onset). These intervals are indicated by gray bars in Fig. 7.
|
All 3 feedback conditions show a similar increase in covariance from target offset toward the time of pointing (see the bold lines in Fig. 7, indicating the values of the covariance averaged over all subjects). This increase toward the time of pointing was found for all subjects. The peak value of the covariance was reached at slightly different times in the interval between 4.3 and 5.4 s after target onset for different subjects. Therefore the peak value of the average covariance shown in Fig. 7 is about 25% smaller than the average of the peak values of all subjects. To overcome this problem of intersubject timing differences, we counted the number of subjects that show a significant covariance (P < 0.05) somewhere within the interval of target presentation (0–1 s) and the number of subjects having a significant covariance within the pointing interval (4.3 to 5.4 s). These numbers are displayed in the gray bars for each of the specific time intervals. Note that the number of subjects showing a significant covariance in the pointing interval is always at least equal to, but in general larger than, the number of subjects with a significant covariance during target presentation. We found that for almost all subjects, the largest covariance is reached in the pointing interval.
In the FINGER condition (Fig. 7, A–C), the average covariance exceeds a 5% significance level for all 3 coordinates (radial distance, azimuth, and elevation) at the time of pointing. Moreover, Fig. 7, A–C shows that the covariance increases gradually toward the time of pointing, indicating that the variability in pointing resembles the variability in gaze at a time, well before the time of pointing. This means that the variability of the pointing position in radial, azimuth, and elevation direction can be explained (at least partly) from the variability in gaze at the time of pointing. When the subjects are considered individually, all subjects show a significant covariance in all 3 coordinates in the FINGER condition at the time of pointing (P < 0.05), except for one subject, who shows a significant covariance for the 2 directional components (azimuth and elevation, P < 0.05), but not for radial distance (P = 0.18).
For the DARK and FRAME conditions (Fig. 7, D–F and G–I, respectively), the covariance between pointing position and gaze is less pronounced: at the time of pointing the average covariance is significant for azimuth and elevation (P < 0.05). The average covariance for radial distance, however, increases toward the time of pointing, but does not reach a significant value (P = 0.06 and P = 0.11 for DARK and FRAME, respectively).
As indicated by the numbers in the gray bars in Fig. 7, D–F (DARK condition), all 6 subjects show a significant covariance at the time of pointing for the azimuth direction, but only 4 subjects also have a significant covariance for radial distance and elevation at the time of pointing. The high correlation between gaze and pointing in the FINGER condition is not surprising because of visual feedback. However, all subjects also show a significant covariance between pointing and gaze at the time of pointing in the DARK condition in at least 2 of the 3 spatial parameters. Thus covariance at the time of pointing is also present without visual feedback of the finger during pointing.
For the FRAME condition (Fig. 7, G–I) 3 subjects show a significant covariance for all 3 coordinates at the time of pointing (P < 0.05). The remaining 3 subjects show a significant covariance in azimuth (P < 0.05), but only one of them also shows a significant covariance in radial distance (P < 0.05).
In all conditions, we found subjects that have a significant covariance between pointing and gaze at the time of target presentation (see the numbers in the gray bars that indicate the time of target presentation). In most of these cases, the covariance was smaller at the time of target presentation than it was at the time of pointing.
In summary, Fig. 7 shows that the variability in pointing position is related to the variability in gaze at the time of pointing and often already at the time of target presentation. This relation is most prominent for the FINGER condition, but is also present in the DARK and FRAME conditions.
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONACKNOWLEDGMENTSREFERENCES |
|---|
Gaze versus pointing
We found a significant correlation between the variable errors in pointing and in gaze at the time of pointing. This covariance between gaze and pointing could suggest 3 possible explanations. The first explanation could be a common command signal to drive gaze and pointing toward the same target position. Other explanations might be that gaze affects pointing or that pointing serves as a target for gaze. Obviously, these explanations do not exclude each other. We will consider the implications of each of these hypothetical explanations in the context of our experimental results to investigate which explanations are consistent with the data.
The 3 possible explanations are schematically illustrated in Fig. 8. At the time of target presentation, the orientations of the head in space (Hs), the eyes in the head (Eh), and the target on the retina (Tr) are available to calculate the perceived target position, which is stored during the delay period [Internal Target Representation (ITR)]. This ITR is used as target for the pointing movement (path A) and can also be used to guide the eyes to keep gaze at the remembered target position (path B). Evidence for such a common command signal for the eyes and the hand has been presented before by several studies, which reported that eye and hand movements show similar characteristics in tasks like choosing between 2 targets (Gielen et al. 1984) or anticipating target displacements (Frens and Erkelens 1991). Our data provide additional evidence in favor of a common drive of gaze and pointing because of the—for many subjects—significant correlation between pointing position and gaze, when the target is visible (see Fig. 7).
|
If the covariance between gaze and pointing were attributable only to a common command signal to the motor systems for gaze and pointing, one would expect that the gradual drift in gaze in the delay period should deteriorate the covariance between pointing and gaze. This is obviously not the case, as is shown in Fig. 7, which shows that the covariance increases, rather than decreases, with time. A possible explanation might be that the gradual drift in gaze reflects a drift of the ITR, which then should result in a constant error in pointing, proportional to the drift of gaze. Figure 6 clearly shows that this effect is not found at the end of the delay period: In general the constant error in gaze is much larger than the constant error in pointing. Therefore a common drive cannot be the only explanation for the results reported in this study.
Because various studies have shown that deviations of gaze from the target will affect pointing accuracy (e.g., Biguer et al. 1984; Bock 1986; Enright 1995; Henriques et al. 1998; Van Donkelaar and Staub 2000) one could argue that gaze accuracy affects the accuracy of pointing. Recently Neggers and Bekkering (2001) demonstrated a strong linkage between eye movements and pointing movements. Subjects were instructed to make a pointing movement and a saccade toward the same target. When the saccade had reached the target (but when the corresponding pointing movement was not yet completed), a new saccade target was presented. Subjects had to initiate a second saccade toward this new target, but the pointing movement had to stay directed toward the initial target. Neggers and Bekkering showed that the second saccade, away from the pointing target, was delayed until the pointing movement was nearly completed. These results were interpreted as evidence that gaze is used to improve accuracy of the ongoing pointing movement, in addition to a common command signal for eye and arm movements. This is in line with the conclusion obtained by Soechting et al. (2001), who reported that gaze position provides the target signal for hand movements to targets moving behind a moving background (Duncker Illusion).
Other evidence for gaze defining the target for pointing movements was presented by Kröller et al. (1999), who tested whether adaptive changes in saccadic amplitude influence pointing accuracy of the unseen hand, in a double-step adaptation paradigm. In the adaptation session, subjects had to make a saccadic eye movement toward a visual target on a 2-D table. During the saccade, the visual target jumped either backward or onward, thus shortening or lengthening the amplitude. This introduced an artificial postsaccadic error, which led to a corrective saccade. After the adaptation period, the saccadic system incorporated the corrective saccade in the first saccadic movement. When saccadic adaptation was achieved, subjects were asked to perform movements toward the same visual targets with the unseen hand instead of with gaze. The adaptation transfer from the saccadic system to the hand-pointing movement was most prominent, though not complete, when accompanying eye movements were allowed, but only when the adaptation concerned shortening of the saccadic amplitude.
The third alternative explanation, that gaze depends on pointing, is less likely: subjects can quite well look at remembered targets irrespective of finger position.
Evidence in favor of a role of gaze at the time of pointing on pointing accuracy was found in various studies, in which gaze and pointing position were dissociated. Bock (1986) and Enright (1995) showed that the pointing movement tends to overshoot the target distance relative to the gaze location, when gaze is not directed toward the location of the remembered target for the reaching movement. A similar finding was reported by Henriques et al. (1998, 2000), who studied pointing errors toward a remembered target, which was presented while gaze was directed in various horizontal and vertical peripheral positions not coinciding with the target. Subjects were found to overshoot the magnitude of the retinal eccentricity of the target, both in horizontal and vertical directions. Medendorp and Crawford (2002a) showed that a similar overshoot occurs when subjects are allowed to foveate a visual target but have to make a saccade in the delay period before pointing. In this paradigm, the pointing responses still indicated an overestimation of the retinal eccentricity of the target relative to gaze at the time of pointing. Therefore the authors concluded that the target position is updated for gaze at the time of pointing, which was interpreted as evidence that target position is stored in retinocentric coordinates. A similar question was addressed by Pouget et al. (2002),who investigated whether the remembered position of a reach target is also stored in a retinocentric frame of reference for targets of other modalities (i.e., auditory, proprioceptive and imaginary targets). They found that when gaze was not directed to the target position, subjects largely overshot the retinal eccentricity of the target, irrespective of target modality.
If fixation away from the target leads to overshoot of the retinal position of the target, one would expect a negative correlation between variability in gaze and pointing in the present study, where gaze drifts away from the target in the delay period. This is not in agreement with our data, which clearly show a positive correlation between pointing and gaze. Positive correlations have also been reported in other studies (see e.g., Flanders et al. 1999 and Soechting et al. 2001). One of the main differences between these studies and the ones that report overshoot of the target and a corresponding negative correlation is that in the latter studies, subjects deliberately fixated away from the target, whereas in our study (as in the studies by Flanders et al. 1999 and Soechting et al. 2001), subjects were not aware of the off-target fixation. None of our subjects was consciously aware of the drift in the delay period. One subject explicitly mentioned that he tried to use the strategy to rigidly fixate the remembered target in the DARK and FINGER conditions, to "anchor" the visual target in the otherwise dark environment. The results of this subject were not different from those of the other subjects, which indicates that the drift in gaze occurs unconsciously, even when a strategy is adopted to maintain gaze at the remembered target. When the remembered target position is stored relative to gaze, as suggested by, for example, Henriques et al. (1998), and when subjects are not consciously aware of the drift in gaze, the remembered target position is presumably not updated for the drifted gaze position at the time of pointing. Consequently, one will find a positive correlation between gaze and pointing, which is what we found.
Gaze drifts during delay period
From previous studies it is known that when subjects are left in complete darkness for a few minutes, gaze tends to shift toward a preferred distance, which varies between subjects from about 40 to 80 cm relative to the cyclopean eye, and slightly changes with gaze direction (dark vergence, Heuer et al. 1989). Gaze shifts in complete darkness, which occur already a few seconds after the disappearance of a visual target, were also described by Medendorp et al. (2002b). These authors tested the stability of gaze to a remembered target during active head movements. They found that the correspondence between target position and gaze gradually deteriorates after the visual target disappears, both for direction and for radial distance. Medendorp et al. presented a target at 20 cm from the cyclopean eye, and found that gaze distance starts to increase almost immediately after disappearance of the visual target, which is in agreement with our observations. We conclude that subjects do try to maintain gaze on the target, but fail to do so.
In the FRAME condition we found larger gaze drifts in radial distance than in the DARK and FINGER conditions, and hardly any drift in direction (see Figs. 5 and 6, middle columns). For the FRAME condition one might expect that it would be easier to maintain fixation at the remembered target position: the visual environment provides a reference that might assist the subject to correct for any unintended gaze drift. On the other hand, one might argue that because the visual environment provides a reference frame to store the remembered target position, precise fixation may not be that important, given that the visual frame serves as an "anchoring point" for the remembered target. Our results provide evidence for the latter because at the time of pointing gaze has not returned completely to the target position and, nevertheless, the pointing performance is more accurate than without the frame. More evidence for the latter interpretation was provided by Blouin et al. (2002), who showed that the definition of gaze direction after several saccades in the dark is more accurate when there is visual stimulation of the retina, than when there is no visual information whatsoever. This effect is irrespective of whether the visual stimulation caries spatial information. In the FINGER condition we found that vision of the index finger resulted in a correction of the radial drift in gaze. Following the reasoning of Blouin et al., this corrective movement of gaze may be the response to a more accurate definition of gaze direction attributed to visual stimulation of the retina by the tip of the index finger.
Frames of reference
Many authors have studied pointing movements toward remembered targets. However, the conclusions of these studies have not always been congruent. Some studies concluded that subjects make pointing movements to remembered visual targets in an illuminated environment in a viewer-centered frame of reference, usually with respect to the head or the cyclopean eye (e.g., McIntyre et al. 1997; Soechting et al. 1990). Other studies suggested that pointing movements are executed in a shoulder-centered frame of reference (e.g., Soechting and Flanders 1989), or both a shoulder-centered and a head-centered frame of reference, for pointing movements in the dark (McIntyre et al. 1998; Soechting et al. 1990). These studies all describe pointing movements to remembered visual targets, but they tested subjects under different visual conditions. The question arises whether these different conditions can explain the different observations.
Previous work on the effect of vision on the accuracy of pointing movements on a 2-D table was reported by Carlton (1981), who showed that vision of the pointer (a handheld stylus) was the most important requirement for accurate pointing, irrespective of visibility of the environment or the target. When the hand is not visible, continuous vision of the target improves the pointing performance of the hand, indicating that the CNS is able to correct pointing movements of an unseen hand during the execution (Adamovich et al. 2001; Prablanc et al. 1986).
Elliott and Madalena (1987) showed that subjects are able to accurately use a visual representation of the target in the control of aiming movements only shortly after visual occlusion. For pointing movements after delays of 2 s or longer, the pointing errors increase significantly, attributed to the decay of such visual representation.
Soechting et al. (1990) tested pointing movements toward remembered targets in the Dark. Unlike in our study, the room lights were on when the target was presented in the dark condition of their study. They tested the average orientation of the total error (constant and variable error taken together) between pointing position and target position, under the assumption that errors are largest in radial distance, and much smaller in direction. For pointing in the dark, they found that the total errors are best described relative to an origin located between the head and the shoulder. They also analyzed pointing movements when the room lights remained on after target presentation. In this Light condition they found that total pointing errors were best described relative to a position close to the subject's eyes. From these results they concluded that there exists both a head-centered and a shoulder-centered representation of target location in the CNS, and that for pointing in the dark, the CNS includes the shoulder-centered representation.
The FRAME condition in the present study is similar to the Light condition in the study by Soechting and colleagues. In this condition we find that orientations of the variable errors are best described as originating from a center close to the subject's head. However, in the DARK setup, we did not find that the variable errors originate from a distinct origin at or near the head or the shoulder or in between. Differences between the results of Soechting and Flanders and the findings of our study may well result from differences in the approach to estimate the orientation of the pointing errors. The analysis by Soechting and Flanders was based on the total error, whereas we analyzed the variable and constant errors separately. The 2 approaches will lead to the same conclusions, when the constant errors are mainly in radial distance. In our study, however, we found moderate constant errors also in direction, especially in the DARK condition (2° in the FINGER and FRAME conditions and 4° in the DARK condition), and these may result in different origins for the variable errors and for the total errors.
McIntyre et al. (1998) tested movements toward remembered targets performed in the dark and in a dimly lit room, apparently similar to our DARK and FRAME conditions. They found that, with vision of the environment and the arm, the distributions of variable pointing errors are oriented toward the subject's head. McIntyre and colleagues also tested the local distortion, which described the fidelity with which the relative spatial organization of targets within a small workspace region (on a sphere of 22-mm radius) is maintained in the configuration of final pointing positions. They found that the pointing positions reflected a contraction of the target configuration along an axis that was oriented toward a position between the subject's head and the shoulder. They interpreted the orientation of the variable errors and that of the local contraction as indicative for the use of both a viewer-centered and a shoulder-centered frame of reference. However, there may be alternative explanations.
Carrozzo et al. (2002) showed that the configuration of the targets influences the orientation of variable pointing errors, even though the targets were never presented simultaneously. Gentilucci et al. (1996) showed that the manifestation of such allocentric effects is strengthened by a delay between the visual stimulus and the motor response. In our study we used a delay period of 2 s, which suffices for allocentric effects to occur. Nevertheless, we found no effects on the variable errors related to our target configuration. This may be attributed to the fact that our target configuration was more complex than the configuration used by Carrozzo and colleagues, in which the targets were located on a straight line. In the setup in which the subjects were provided with the most allocentric information (FRAME), one might expect to find variable errors related to the illuminating frame. However, we found that the orientation of the pointing variability in the FRAME condition was largely related to gaze, instea
