INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Motion provides powerful information for recovering three-dimensional (3D) structure from the two-dimensional (2D) retinal image. In computational terms, it has been shown that only three snapshots of a rotating rigid object defined by four noncoplanar points are sufficient for recovering its 3D structure (Ullman 1979). Actually, humans and macaques can perceive the structure of a moving object solely using its motion information (Siegel and Andersen 1988). This phenomenon is called structure-from-motion (SFM) perception. But the neural mechanisms underlying SFM have not been fully clarified.

In the primate, selective responses to motion first emerge in primary visual cortex (V1). Directionally selective V1 neurons preferentially respond to motion in a particular direction within a small receptive field (RF). Because single V1 neurons can only deal with motion signals through small apertures, it is unlikely that they process SFM in an explicit fashion. V1 sends its outputs to several extrastriate areas including middle temporal area (MT), which is believed to be specialized for motion processing. Several electrophysiological studies of the macaque monkey have examined the contribution of area MT to the processing for SFM. Bradley et al. (1998) employed a moving random-dot pattern that is perceived as a rotating cylinder and found that the responses of MT neurons changed with the change in the perceived structure of the motion stimulus (also see Dodd et al. 2001). Lesions in area MT were shown to prevent the perception of SFM (Andersen et al. 1996). These reports suggest the involvement of MT in the processing of SFM.

MT neurons, however, have relatively small RFs and preferentially respond to locally presented translational motion stimuli. This response property might be suitable for some kinds of structural processing, such as the depth-order assignment of motion-transparent surfaces (Bradley et al. 1998; Qian and Andersen 1994). MT neurons have also been found to be selective for local speed gradients (Treue and Andersen 1996; Xiao et al. 1997). However, it is doubtful that a single MT neuron can code more complex structures, such as those with multiple motion gradients in various directions. Because even simple SFM stimuli like a rotating cylinder are actually composed of complex patterns of motion gradients, spatial integration of local motion signals seems essential for the processing for SFM. It has been proposed that the processing of SFM consists of multiple stages (Hildreth et al. 1995), and the integration of local motion signals may take place beyond area MT.

The dorsal division of the medial superior temporal area (MSTd), which is located in the upper bank of the superior temporal sulcus (STS), receives a direct projection from MT (Maunsell and Van Essen 1983; Ungerleider and Desimone 1986). Neurons in MSTd have large RFs and respond selectively to complex stimuli such as expansion, contraction, and rotation (Duffy and Wurtz 1991; Graziano et al. 1994; Lagae et al. 1994; Raiguel et al. 1997; Saito et al. 1986). Recently, a functional magnetic resonance imaging (fMRI) study in human subjects reported that responses related to SFM perception occurred in MT+, which is considered to be homologous to macaque areas MT and MST (Orban et al. 1999).

In the present study, we recorded the activity of MSTd neurons and studied its relation to SFM processing. To relate neural responses to SFM processing, the response selectivity to the simulated 3D structure of a surface was examined. We employed a rotating plane, inclined in a particular 3D orientation (plane stimulus), as a SFM stimulus. The plane stimulus was composed of random dots having a limited lifetime and was perceived vividly as a 3D-oriented surface in spite of the absence of depth cues other than motion. We employed this stimulus because it is well established that MSTd neurons respond to rotating frontoparallel planes, and we thought that MSTd neurons might also respond to the rotation of variously 3D-oriented planes. Furthermore, this stimulus is appropriate for a quantitative study because the structure of this stimulus can be defined by only two parameters, namely, tilt and slant. If area MSTd is involved in SFM processing, MSTd neurons should exhibit selectivity to these structural parameters.

In the current study, we find that many MSTd neurons have selectivity for tilt and/or slant of the rotating plane stimulus. The selectivity is position-invariant as well as speed-invariant. These results indicate that the selectivity of MSTd neurons to the rotating plane cannot be explained simply as responses to the local motion components of the plane stimulus, but rather as responses to the global stimulus. Thus MSTd neurons can code the 3D structure of rotating planes, and this suggests that area MSTd is involved in SFM processing.

A brief report of these experiments has appeared elsewhere (Sugihara et al. 1998).

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Behavioral task

Recordings were made from three awake, Japanese monkeys (Macaca fuscata). All procedures for animal care and experiments were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals (1996) and were approved by the animal experiment committee of the Okazaki National Research Institutes.

During the experiments, each monkey sat in a primate chair and looked at the screen binocularly. The screen was placed approximately 32 cm in front of the monkey so that 1 deg corresponded to 15 pixels (approximately 0.55 cm), and the screen covered 68.3 × 51.2 deg. (Hereafter, "deg" will be used to refer to the degree of visual angle.) Each monkey was trained to perform a fixation task. A trial started when a small fixation spot appeared on the screen. The monkeys were required to foveate the fixation spot within 500 ms and to maintain its gaze within 1 × 1 deg (or occasionally 4 × 4 deg) window. At the end of a successful trial, a drop of water was delivered as a reward, the fixation spot was turned off, and a 2-s intertrial interval was initiated. Eye position was monitored using the magnetic search coil technique (Robinson 1963). If the monkey's eye deviated beyond the window during a trial, the trial was automatically terminated without a reward, and the intertrial interval was initiated. During the period of fixation, a visual stimulus was presented. Data collection, events for the fixation task, and stimulus presentation were controlled by computer.

Surgery and recording

A stainless steel recording chamber and a head holder were fixed to the skull under general anesthesia and sterile surgical conditions. A search coil was placed in the eye and was connected to a plug on the top of the skull. After surgery, the monkey was allowed to recover for 1 wk before the experiment began. During this period, antibiotic (Cefazolin sodium) was given every 12 h.

Single-neuron activity was recorded from MSTd. The recording chamber was placed over the occipital cortex for one monkey and over the parietal cortex for two monkeys. A glass-coated Elgiloy microelectrode or varnish-coated tungsten microelectrode was advanced through the dura or inside a stainless steel guide tube that was advanced manually through the dura. Extracellular action potentials were amplified, and single neuron activity was isolated with a time-amplitude discriminator. Spike times were then converted to pulse sequences. MSTd was identified based on the following criteria: 1) depth below the dura, 2) location relative to area MT, 3) selectivity for optic flow, and 4) RF size.

The RF of an MSTd neuron was roughly mapped by a stimulus of the basic stimulus set (see following text). The RF typically contained the foveal region and usually covered the contralateral half of the screen and extended into the ipsilateral side to a considerable extent.

Visual stimuli and selectivity test

Every neuron was tested with two sets of stimuli. Stimuli were presented at the center of the RF. Within each set, visual stimuli were presented in a pseudorandom interleaved fashion, one stimulus per trial, and each stimulus was repeated at least four times, usually more than five times. Each stimulus consisted of 60 frames of moving random dots. Each frame of the stimulus was generated during the intertrial interval, stored in the computer memory, and presented in sequence during stimulus presentation. The positions of the dots varied across trials. During testing, the movies were presented at a frame rate of 60 Hz. Each stimulus had a duration of 1 s. The neuron's baseline activity was measured during 400-0 ms before the stimulus presentation. The visual response was defined as the mean discharge rate during stimulus presentation minus the baseline activity. The response was judged to be significant if the difference between the discharge rate during stimulus presentation and the baseline activity was statistically significant (t-test, P < 0.05).

BASIC STIMULUS SET. The first set consisted of eight stimuli to test selectivity for basic optic flow patterns: expansion, contraction, clockwise rotation, counterclockwise rotation, and the four directions of translation (up, down, right, and left). Each stimulus was composed of 314 dots that were displayed within a circular window (26.7 deg diam). Each dot moved for a 150-ms lifetime, disappeared, and then appeared at a new random location within the circle, and was given a trajectory and speed appropriate to its new location. The dots were relocated asynchronously, to avoid a coherent flickering of the stimulus. This constant reshuffling virtually eliminated pattern artifacts because the pattern of the dots was constantly and randomly changing. The reshuffling also eliminated density artifacts, since each local region in the display had approximately the same number of dots at any time. As a result, the mean luminance was also constant across the display. The translational motion stimuli moved at 20 deg/s, which is equal to the average dot speed in the expanding, contracting, and rotating stimuli.

PLANE STIMULUS SET. The second stimulus set consisted of rotating planes in various 3D orientations (Fig. 1). Each stimulus in this set was also composed of random dots, but their velocity field simulated a rotating plane. A simulated 3D orientation was defined by two parameters, tilt and slant. The tilt is defined as the orientation of the projection of the surface normal on the frontoparallel plane. In the present study, 0° of tilt corresponds to rightward and 90° of tilt correspond to upward (Fig. 1C). The slant is defined as the angle between the surface normal and the line of sight. In mathematical terms, different orientations of the plane stimuli can be represented as combinations of rotation and deformation with various ratios (Koenderink 1986). Such a description may be more neutral than the description using terms like "tilt" and "slant" that are intimately related to the three-dimensionality of an object. However, the description of the stimuli using such mathematical terms is less intuitive. Thus in the following, we will use the terms tilt and slant to characterize different stimuli for the sake of simplicity. We used a set of four tilts, namely, 0, 45, 90, and 135°, and a set of four slants, namely, 0, 20, 40, and 60° (Fig. 1C). Zero degrees of slant corresponds to a plane rotating on the frontoparallel plane, and the tilt cannot be defined. The plane stimulus set consisted of the combinations of each slant and tilt, thus a total of 13 stimuli (4 tilts × 3 slants +0°-slant stimulus). The pattern was rotated about the surface normal vector passing through the center, at 28 revolutions per minute (rpm). The direction of rotation that elicited the better response in the basic stimulus set was employed for the plane stimulus set. The opposite direction of rotation was also examined in many neurons. For the stimulus with a slant equal to 0°, the average dot speed was 20 deg/s, which was equal to the average speed in the basic stimulus set. To avoid a change in the spatial extent of the stimulus pattern accompanying the change in simulated 3D orientation, a circular aperture of 26.7 deg diam was used (Fig. 1A). So all the stimuli in the plane stimulus set had the same spatial extent as those in the basic stimulus set. This made the average speed larger for stimuli having steeper slants. Positions of the dots were calculated using orthographic projection to remove perspective information; no disparity information was added. Each stimulus in the plane stimulus set had a specific structure of the velocity field that corresponds to a particular slant and tilt as exemplified in Fig. 1B. Because of the orthographic projection, the stimuli had an ambiguity with respect to tilt such that two stimuli having a tilt difference of 180° were identical to each other. Other properties of the stimuli in the plane stimulus set were the same as those of the basic stimulus set.

View larger version (34K): [in this window] [in a new window]   Fig. 1. The plane stimulus set. A: schematic illustration of the rotating plane stimulus. The plane stimulus was composed of random dots and simulated a rotating plane with a particular 3-dimensional (3D) orientation. Small solid squares indicate moving random dots. The pattern is rotated about the surface normal vector passing through the center. The direction of rotation in this figure is anticlockwise and is indicated by arrows. Positions of the dots are calculated by using orthographic projection to remove perspective information. The circle within the shaded rectangle illustrates the aperture we employed for the plane stimulus set to avoid a change in the spatial extent of the stimulus pattern accompanying with the change in simulated 3D orientation. Only the unshaded part is presented as a visual stimulus. B: 3 examples of the velocity fields in the plane stimulus set. Arrows indicate the directions and speeds of moving dots. C: schematic illustration of plane stimuli with different simulated 3D orientations. A simulated 3D orientation is defined by 2 parameters; namely, tilt and slant. Each ellipse indicates a stimulus and is plotted in the tilt-slant space at the location corresponding to its simulated 3D orientation. The shape of the ellipse indicates schematically the simulated 3D orientation of the stimulus, although actual spatial extents of the stimuli are the same because of the aperture. We used a set of 4 tilts, namely, 0, 45, 90, and 135°, and a set of 4 slants, namely, 0, 20, 40, and 60°. At 0° slant, the stimulus is on the frontoparallel plane, and tilt cannot be defined. The plane stimulus set consists of the combination of each tilt and slant; thus a total of 13 stimuli (4 tilts × 3 slants + 0°-slant stimulus) are in this set. Because of the orthographic projection, the stimuli have an ambiguity with respect to tilt such that 2 stimuli having the tilt difference of 180° are identical to each other. Such identical pairs of stimuli are illustrated in black and gray, respectively.

CONTROL 1. POSITION INVARIANCE. The first control experiment tested the effect of the direction of local translational motion. The best stimulus in the plane stimulus set was presented at five retinal locations within the RF (Fig. 9). The stimulus size was the same as those of the main experiment. Five stimulus positions lay in an overlapping cloverleaf arrangement (Fig. 9A) and, as a whole, covered 53.4 deg (51.2 deg vertical, due to the limit of the screen size). The central position was the same position as that used for the plane stimulus set. In the regions where different stimuli overlapped, the direction of local motion reversed even if the entire extents of the stimuli were rotating in the same direction. Therefore if a neuron responded in the same way at all five positions, the response cannot be explained by tuning to the direction of local translational motion. In addition to the best stimulus, a stimulus with the same slant and tilt as the best stimulus but with the opposite direction of rotation was also used. Thus each neuron was given a total of 10 different stimuli (5 positions × 2 rotations). To examine the position invariance of the responses, we compared selectivity for direction of rotation across the five positions (Graziano et al. 1994).

CONTROL 2. ROTATION SPEED. The second control experiment tested the effect of the speed of the moving dots. Because we employed the same circular aperture for every stimulus in the plane stimulus set, the maximum or average speed as well as the magnitude of the speed gradient contained in the stimuli increased with the increase of slant. This might cause an apparent selectivity for slant if the neuron examined was sensitive to dot speed. To examine this possibility, we tested slant selectivity using three rotation speeds: slow (19 rpm), standard (28 rpm), and fast (42 rpm). Every stimulus had the same tilt as the best stimulus in the plane stimulus set. If the selectivity to slant does not change with the change in rotation speed, selectivity for the plane stimulus set cannot be explained by sensitivity to speed.

CONTROL 3. SHUFFLED PLANE STIMULI. The third control experiment tested whether the neurons were really sensitive to the structure of the velocity field. An alternative possibility was that the neurons might simply respond to the distribution of velocities regardless of their spatial structure. As a control for this possibility, we prepared another stimulus set (a shuffled stimulus set) by randomly shuffling the locations of dots from the original plane stimulus while preserving their velocities (speeds and directions). Thus the shuffled stimuli contained the same distribution of velocities as the original plane stimulus, but did not conform to the structure of a plane oriented in 3D space. If the neurons were really sensitive to slant and/or tilt, it was expected that the neurons would lose selectivity to the stimulus set or would not respond at all.

Histology

On completion of the last recording session, monkeys were killed under deep anesthesia with pentobarbital sodium and perfused through the heart with saline followed by 4% paraformaldehyde. The brain was then removed from the skull and was sectioned (50 µm in thickness) in the parasagittal plane. For two monkeys, the brain sections were stained with cresyl violet. Damage from the insertion of the guide tubes was identified at the anterior bank of the STS, and the locations corresponded well with the location of MSTd as described previously (Komatsu and Wurtz 1988). For another monkey, electrical markings were made at the last recording sessions, and the brain sections were stained with a modified silver stain for myelinated fibers (Gallyas 1979). The positions of electrical markings were identified within the densely myelinated zone (DMZ) in the anterior bank of the STS and this confirmed that the recordings were made from MSTd (Desimone and Ungerleider 1986; Komatsu and Wurtz 1988).

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

We recorded from 114 MSTd neurons that responded significantly (t-test; P < 0.05) to at least 1 stimulus in the basic stimulus set. Next, we examined the responses of these neurons to the plane stimulus set. Ninety-seven of these exhibited significant responses to at least one of the plane stimulus set that were more than one-half of the maximum response to the basic stimulus set. Further analysis of the responses to the plane stimulus set was conducted for these 97 neurons.

Examples of responses

To examine whether MSTd neurons could code a 3D orientation of the rotating plane, we tested responses to the plane stimulus set and analyzed the selectivity to tilt and slant.

One example of the neurons that responded selectively to the plane stimulus set is shown in Fig. 2A. The responses of this neuron increased as slant increased while the tilt was kept constant at 135°. On the contrary, with the constant tilt of 45°, the responses decreased as slant increased. Apparently, the responses of this neuron to the plane stimulus set changed depending on both tilt and slant. The bubble plots of the same responses are shown in Fig. 2B. Again, the large responses were obtained for the stimuli with 0 and 135° of tilt and with steeper slants. Due to the orthographic projection, the stimuli with 180° of tilt and those with 0° of tilt are indistinguishable. Hence the preferred tilt of this neuron is between 135 and 180° of tilt (or between 45 and 0° of tilt). Other examples of the neurons that responded selectively to both tilt and slant are shown in Fig. 7.

View larger version (22K): [in this window] [in a new window]   Fig. 2. Responses of an example neuron selective to both tilt and slant of the rotating plane stimulus set. A: each peristimulus time histogram indicates the responses of a neuron to a stimulus with certain tilt and slant. The histogram is placed at a position representing each stimulus in the tilt-slant space. The histograms are aligned at the stimulus onset (vertical line). The vertical calibration line on the left of each histogram indicates 100 spikes/s. The horizontal line below each histogram indicates the stimulus duration (1 s). B: the same responses shown in A are replotted as a bubble plot. The diameter of the circle represents the response amplitude. The scale is at the bottom left. The mirror image of the responses is also shown in gray. The arrow at the bottom right indicates the direction of rotation of the stimuli (in this case, clockwise).

Figure 3 shows two examples of the neurons that had different types of selectivity to the plane stimulus set. The neuron in Fig. 3A responded maximally to the stimulus with 90° of tilt and 40° of slant. It responded well to the stimuli with 45 and 90° of tilt, but slant did not clearly affect the responses. Another neuron, shown in Fig. 3B, responded well to the stimuli with shallower slants, irrespective of tilt. The maximum response was to the stimulus with 0° of tilt and 20° of slant.

View larger version (23K): [in this window] [in a new window]   Fig. 3. Two examples of neurons exhibiting different types of selectivity to the plane stimulus set. A: a neuron responding well to the stimuli with 45 and 90° of tilt. Slant did not clearly affect the responses. B: a neuron responding well to the stimuli with shallower slants irrespective of tilt. Conventions are as in Fig. 2B.

Quantification of selectivity

To examine how the neurons that were selective to tilt and/or slant represent the 3D orientation of the rotating plane, we analyzed selectivity quantitatively as described in METHODS. To quantify the selectivity, we computed a selectivity index for tilt and slant using two sets of responses to the stimuli that shared the same slant or tilt with the best stimulus. Figure 4 plots the same responses as shown in Fig. 2 as a function of tilt (Fig. 4A) and slant (Fig. 4B). The best stimulus in Fig. 4 is the one with 135° of tilt and 60° of slant. So the selectivity indexes for tilt and slant were computed using the responses as indicated by solid symbols in Fig. 4, A and B, respectively. For these responses, the selectivity index for tilt was 0.94, and that for slant was 0.60. The responses were classified as selective when the variation of the responses was significant (P < 0.05, 1-way ANOVA). The neuron whose responses are shown in Figs. 2 and 4 is selective to both tilt and slant.

View larger version (16K): [in this window] [in a new window]   Fig. 4. Replot of the responses of the neuron as shown in Fig. 2 as a function of tilt (A) and slant (B). The abscissa indicates tilt (A) and slant (B), and the ordinates indicate the amplitude of the response. Different symbols represent the responses to the stimuli with different slants (A) and tilts (B). "×" in B indicates the response to the stimulus with 0° of slant. Error bars are SDs. The largest response was obtained when tilt was 135° and slant was 60° (best stimulus). Solid squares indicate the responses to the stimuli that share the same slant (A) and tilt (B) with the best stimulus. We calculated the selectivity indexes for tilt (A) and slant (B) from these responses. The selectivity index of this neuron was 0.94 for tilt and 0.60 for slant and was significantly tuned for tilt and slant by 1-way ANOVA (P < 0.05).

Figure 5 shows the scatter diagram of the selectivity indexes of the 97 neurons that responded significantly to the plane stimulus set. The presence or absence of selectivity (in light of the above criteria) to tilt and/or slant is also indicated by different symbols. The selectivity index is continuously distributed. Of the neurons that responded significantly to the plane stimulus set, about two-thirds (65/97, 67%) exhibited selectivity to at least either tilt or slant. Of these, 17 were selective only to tilt, 23 only to slant, and 25 to both.

View larger version (29K): [in this window] [in a new window]   Fig. 5. Distribution of the selectivity indexes. The bottom left panel shows the scatter diagram of the selectivity indexes of 97 neurons that responded significantly to the plane stimulus set. Each symbol represents a neuron, and different symbols indicate the presence or absence of the selectivity to tilt and/or slant as shown in the inset. The top and right panels show the distribution of the selectivity index for tilt and slant, respectively. Filled bars in each histogram indicate the neurons that exhibited significant responses, and open bars indicate those that did not.

The distributions of preferred tilt and slant are shown in Fig. 6. In the distribution of preferred tilt, there is a weak but significant bias to 90° (Rayleigh test, P = 0.014). In the distribution of preferred slant, there are two peaks at 20 and 60°. Neurons with different slant selectivity had different tilt selectivity. The solid bars represent neurons selective to both tilt and slant, whereas the open bars represent neurons selective only to tilt (A) or slant (B). Almost all neurons preferring the steeper slant were also selective to tilt, whereas those preferring the shallower slant were not selective to tilt. This result seems reasonable when considering the distances between stimuli in the tilt-slant space (Fig. 1). That is, the distance between two stimuli with steeper slants is larger compared with the distance between two stimuli with shallower slants for the same tilt difference. This may result in a larger difference in the responses to two stimuli differing by the same amount of tilt when the slant is steeper, but a smaller difference in the responses when the slant is shallower.

View larger version (13K): [in this window] [in a new window]   Fig. 6. Distributions of the preferred tilt (A) and slant (B). The abscissa indicates tilt (A) and slant (B) that induced the maximum response, and the ordinates indicate the number of neurons. Only the neurons that were selective to either tilt (A) or slant (B) contributed to these histograms. Filled bars represent, in total, 25 neurons selective to both tilt and slant, and open bars represent neurons selective only to tilt (A) or slant (B). In A, a weak but significant bias to 90° is observed (Rayleigh test, P = 0.014). In B, there are 2 peaks at 20 and 60° in the preferred slant distribution. See the text for more details.

Selectivity to the basic and plane stimulus sets

Area MSTd has been characterized by the presence of neurons that are selective to optic flow patterns similar to the stimuli in the basic stimulus set (Duffy and Wurtz 1991; Lagae et al. 1994; Saito et al. 1986; Tanaka and Saito 1989). Thus it would be important to know how the neurons examined in the present study responded to such optic flow stimuli. The relationship between the selectivity to the basic stimulus set and selectivity to the plane stimulus set is summarized in Table 1. The stimuli in the basic stimulus set can be classified into three stimulus types (Duffy and Wurtz 1991): planar (4 translations), radial (expansion and contraction), and circular (2 rotations). As was shown previously (Duffy and Wurtz 1991; Graziano et al. 1994; Lagae et al. 1994), some MSTd neurons responded to only one stimulus type, but some others responded to more than one stimulus type. We classified neurons according to the number of stimulus types in the basic stimulus set that evoked responses greater than one-half of the maximum response. Neurons that responded to only one type were classified as single-component, those that responded to two types were classified as double-component, and those that responded to three types as triple-component. Of the 97 neurons examined, 42 neurons were classified as single-component, 30 as double-component, and 25 as triple-component.

Relationships exist between the selectivity to the basic stimulus set and to the plane stimulus set in two respects. The first one is between the neurons that responded to the circular motion stimuli in the basic stimulus set and the neurons preferring shallower-slant stimuli. The rotating plane stimuli employed in the present study obviously include rotation as an element of the stimulus. This is particularly clear for stimuli with shallower slants such as 0 and 20°, which are nearly identical to the circular motion stimuli in the basic stimulus set. Thus we can expect that neurons responsive to the stimuli with shallower slants would be sensitive to the circular motion stimulus in the basic stimulus set. Actually, 20 of 21 neurons preferring the stimuli with 0 or 20° of slant responded to the circular motion stimuli in the basic stimulus set, and 19 of these neurons exhibited the maximal response to the circular motion stimuli. Another relationship is that most of the neurons preferring the stimuli with steeper slants such as 60°, which usually were also selective to tilt (Fig. 6), were classified as double- or triple-component (Table 1). No other clear relationship was found.

Responses to the nonpreferred rotation

One conspicuous feature of the rotation-sensitive neurons in area MSTd is their selectivity to the direction of rotation (Saito et al. 1986). As the 3D surface orientation of the rotating plane employed in the present study is defined independently of the direction of rotation, each plane with a particular combination of tilt and slant could rotate either clockwise or anticlockwise. Therefore it is of interest to know whether neurons selective to tilt and/or slant of the rotating plane were also selective for the direction of rotation. For 48 neurons selective to either tilt or slant, we compared the responses to the plane stimuli in the preferred direction of rotation (preferred rotation) and the opposite direction of rotation (nonpreferred rotation). Figure 7 shows three examples of such neurons. The neuron in Fig. 7A responded well to the anticlockwise rotating stimuli and exhibited selectivity to both tilt and slant (preferred rotation, left panel). This neuron did not show any clear response to clockwise rotating stimuli (nonpreferred rotation, right panel). Likewise, of the neurons tested in both directions of rotation, 24 neurons (24/48, 50%) showed rotation-direction-selective responses in which the best response in the preferred rotation was more than twice as strong as that in the nonpreferred rotation. Among the remaining neurons, some responded equally in both directions of rotation (Fig. 7B). However, most neurons, when tested in the nonpreferred rotation, responded only to a subset of the stimuli that caused responses when tested in the preferred rotation. An example of such responses is shown in Fig. 7C.

View larger version (27K): [in this window] [in a new window]   Fig. 7. Comparison of the responses to the plane stimulus sets between different directions of rotation. Three examples of neurons exhibiting different response patterns are shown. The left column indicates the responses to the stimuli in the preferred rotation, and the right column indicates the responses to the stimuli in the opposite (nonpreferred) rotation. The scale for both these responses is at the bottom left in each row. , excitatory response; , inhibitory response. Other conventions are as in Fig. 2B. A: this neuron responded well to the stimuli in the preferred rotation but not to the stimuli in the nonpreferred rotation. B: this neuron responded equally to both directions of rotation. C: this neuron responded to the stimuli in the nonpreferred rotation, but the response was limited only to a subset of the stimuli that caused responses in the preferred rotation.

To compare the overall responsiveness in the preferred rotation direction with that in the nonpreferred rotation direction, all the responses to the stimuli in the plane stimulus set for each rotation were summed, and the sum for the nonpreferred rotation was divided by the sum for the preferred rotation. In 33 of the 48 neurons tested (69%), the computed ratio was <0.5. Thus a majority of neurons had selectivity not only to tilt and/or slant of the plane stimuli, but also to the direction of rotation of the plane stimuli.

We then compared the optimal slant and tilt of the plane stimuli between different rotations for 24 neurons in which the best response of the nonpreferred rotation stimuli was more than one-half of the best response to the preferred rotation stimuli. Figure 8 shows distributions of the differences in the optimum tilt and slant for these 24 neurons. All these neurons had selectivity to either tilt or slant in the preferred rotation. Neurons that also exhibited selectivity to tilt or slant to the stimuli in the nonpreferred rotation are shown with filled bars. Both the distributions of the differences in the optimum tilt and slant had peaks at 0°. Thus many neurons responded maximally to the stimuli with the same tilt and slant between the preferred and the nonpreferred rotations.

View larger version (15K): [in this window] [in a new window]   Fig. 8. A: distribution of the difference between the preferred tilt obtained from the clockwise rotating stimuli and that obtained from the anticlockwise rotating stimuli. The abscissa indicates difference in the preferred tilt, and the ordinate indicates the number of neurons. B: distribution of the difference between the preferred slant obtained from the clockwise rotating stimuli and that from the anticlockwise rotating stimuli. In A and B, only neurons whose best response of all the nonpreferred-rotation stimuli was more than 1/2 of the best response of all the preferred-rotation stimuli, and that exhibited selectivity in either of the rotations are included. Number of such neurons is indicated at the top right corner in each panel. Numbers in the parentheses are for neurons exhibiting significant selectivity in both rotations (). Both of the distributions in A and B have the peak at 0°, indicating that many neurons preferred the same tilt and slant in both rotations.

These results indicate that many MSTd neurons that are selective to tilt and/or slant of the plane stimuli are also selective for the direction of rotation, at least to some extent, but some neurons may encode tilt and/or slant independent of the direction of rotation.

Position invariance

Is the selectivity we observed really for tilt and/or slant? With the change in the tilt and/or slant, other stimulus elements in the plane stimuli also changed. The direction of local translational motion was one such stimulus element and if the neuron tested was a detector of a translational motion direction, it might also show an apparent selectivity to tilt and/or slant. To examine the possibility that the neurons responded not to the entire stimulus pattern, but merely responded to the direction of local translational motion, position invariance was tested in 24 neurons, of which one neuron was selective only to tilt, 11 only to slant, and 12 to both. At five positions within the RF, the best stimulus in the plane stimulus set, and a stimulus with the same slant and tilt as the best stimulus but with the opposite direction of rotation, was presented. Of these five positions, one was at the center of the RF, and the remaining four positions were at the peripheries within the RF (Fig. 9A). All 24 neurons tested exhibited a significant difference (t-test, P < 0.05) in the responses to the stimuli in different directions of rotation at the central position. When the stimulus was moved to the peripheral position, the direction of local translational motion was reversed even though the stimuli were rotating in the same direction (Fig. 9A). If the neurons were responding to the direction of local translational motion, the relative magnitude of the response between two directions of rotation should change, depending on its stimulus position. Figure 9B shows an example of the results of this test for one neuron. The responses to the preferred rotation were significantly stronger than those to the nonpreferred rotation at every position tested. Such position invariance was observed in most neurons. To evaluate the degree of position invariance, we calculated a position invariance index for each neuron (Graziano et al. 1994). First, we calculated the direction selectivity index at each of the five positions as 1  (response to the nonpreferred rotation)/(response to the preferred rotation). Note that the "preferred rotation" means the direction of rotation the neuron preferred when presented at the central position and, hence, the index could go negative in the periphery if the neuron showed the opposite preference there. Then a position invariance index was computed by dividing the direction selectivity index obtained at each peripheral position by that obtained at the central position. Thus four position invariance indexes were calculated for each neuron. If the preferred direction of rotation was the same at the central and peripheral positions, the position invariance index is positive. If the direction selectivity indexes were equivalent, then the ratio is unity. Finally, if the preferred direction of rotation changed, then the position invariance index is negative.

View larger version (18K): [in this window] [in a new window]   Fig. 9. A: schematic illustration of the alignment of the stimuli in the position invariance test. FP, fixation point; RF, receptive field. At 5 positions within the RF, we compared response to the optimum stimulus in the preferred rotation and response to the stimulus with the same tilt and slant in the nonpreferred rotation. B: example of responses of a neuron to the stimuli of the position invariance test. Filled and open bars indicate responses to the stimuli in clockwise and anticlockwise rotation, respectively. C, clockwise; A, anticlockwise. The height of bar indicates the response amplitude. Positions of bars correspond to the 5 different positions of the stimulus presentation. The response of this neuron was much stronger to the clockwise stimulus than to the anticlockwise stimulus at every position tested. C: distribution of position invariance index for 24 neurons. The position invariance index was calculated by dividing direction selectivity index obtained at each peripheral position by that obtained at the central position. Because there are 4 pairs of a central position and peripheral positions for each neuron, 4 data from each of the 24 neurons contributed to this graph. Three pairs from 3 neurons were excluded from this analysis because no clear response was obtained to both the preferred and nonpreferred rotation stimuli in these peripheral presentations presumably because the stimuli were laid outside the RF. So, a total of 93 pairs of responses is included in this analysis. The position invariance index was distributed around unity indicating that most neurons exhibited position invariance. See the text for more details.

The distribution of 93 position invariance indexes for 24 neurons exhibited a peak at unity, and a majority had positive values (Fig. 9C). Seventy-six of 93 position invariance indexes were between 0.5 and 1.5. These results indicate that the selectivity to the direction of rotation did not change significantly within the RF. Thus the results suggest that the selectivity to tilt and/or slant of the plane stimuli was not due to the direction of local translational motion.

Rotation speed

Another stimulus element that changed together with the change in the tilt and/or slant of the plane stimuli was the speed of the motion. It has been shown that some MST neurons are sensitive to stimulus speed (Duffy and Wurtz 1997; Orban et al. 1995). Since we employed an aperture for the plane stimuli, the maximum speed in the stimuli changed with the change of slant. Thus the slant selectivity we observed might reflect sensitivity to local speed. To examine this possibility, we investigated the selectivity to slant of the stimuli with three different rotation speeds for 29 slant-selective neurons. Note that local speed gradients in the plane stimuli, which itself may cause depth perception (Braunstein 1968; Harris et al. 1992), also changed with the change in rotation speed. Thus in this control experiment, the effects of both local speed and local speed gradients were examined.

The effect of speed was statistically evaluated by two-way ANOVA with slant and speed as the main factors. Of 29 neurons examined, the rotation speed had no significant effect on 21 neurons (P > 0.01). One example of such neurons is shown in Fig. 10A. In the left column, the responses are plotted against slant. The data showed a very good agreement across rotation speeds, despite the fact that local speeds and speed gradients contained in the stimuli at these three rotation speeds are quite different. In the right column, the set of responses is replotted against the maximum speed in the stimulus. If this neuron was responding to speed, the degree of response overlap should be greater when the responses were sorted according to speed. However, this was clearly not the case, and the degree of response overlap was much more prominent when the responses were sorted according to slant. Seven of the remaining neurons showed a significant effect (P < 0.01) of the rotation speed. One of such neurons is shown in Fig. 10B. This neuron exhibited stronger responses to steeper-slant stimuli regardless of rotation speed. Because the maximum speed in the stimulus was larger for steeper-slant stimuli, there was the possibility that this neuron might be more sensitive to stimuli with faster speeds. However, this was not the case because this neuron responded more strongly to more slowly rotating stimuli than to faster rotating stimuli. Like this example, in five of these seven neurons, the sensitivity to slant was not simply explained by the sensitivity to the rotation speed. Two other neurons exhibited response changes that were consistent with the prediction from their slant selectivity. These neurons exhibited stronger responses to steeper-slant stimuli and responded more strongly to stimuli rotating faster. The slant selectivity of these neurons may be explained by the sensitivity to local speed or speed gradient. Only one neuron exhibited significant interaction between slant and rotation speed. Taken together, we concluded that, for nearly all neurons examined, the slant selectivity cannot be attributed to local speed or local speed gradients in the stimuli.

View larger version (34K): [in this window] [in a new window]   Fig. 10. Responses of 2 slant-selective neurons to the stimuli with 3 different rotation speeds. To examine the possibility that the slant selectivity is simply due to the sensitivity to speed or speed gradients, slant selectivity was tested using 3 different rotation speeds. The abscissa indicates the slant of the stimulus (left column) or the maximum speed in the stimulus (right column), and the ordinates indicate the amplitude of the response. Different symbols represent different rotation speeds as indicated in the inset. Error bars are SDs. A: this neuron exhibited similar responses to the stimuli at any rotation speed (left panel). B: this neuron exhibited a shift of the offset of the responses depending on rotation speed. The pattern of the slant selectivity was almost the same across different rotation speeds (left panel). Although the response gain or offset changed depending on rotation speed, this change could not be explained solely by the change in the speed or the speed gradients in the stimuli (right column).

Shuffled plane stimulus

Finally, we examined whether the recorded neurons were really sensitive to the structure of the velocity field in the stimulus. Each plane stimulus had a specific distribution of velocities. Thus if the neurons simply responded to the distribution of velocities regardless of their spatial configuration, the neurons would show apparent selectivity to tilt and slant. To examine this possibility, we recorded the responses of 15 neurons, which exhibited selectivity to tilt and/or slant, to shuffled stimuli that were identical to the plane stimuli except that their component dots were randomly repositioned while preserving their speeds and directions (see METHODS and Fig. 11). These stimuli are not perceived as 3D planes, although the distribution of velocities in each stimulus is the same as the original plane stimulus.

View larger version (29K): [in this window] [in a new window]   Fig. 11. An example of the control experiment using shuffled stimuli. The left column indicates the responses to the plane stimuli, whereas the right column indicates the responses to the shuffled stimuli. A common scale as indicated at the bottom is used for all the charts. Other conventions are as in Fig. 7. This neuron responded well to the plane stimuli and showed significant selectivity to tilt, but it did not respond to the shuffled stimuli.

Figure 11 shows responses of a neuron to the plane stimuli (left column) and the shuffled stimuli (right column). This neuron clearly responded to the plane stimulus set and exhibited tilt selectivity. However, this neuron did not significantly respond to any of the stimuli in the shuffled stimulus set. Similarly, most of the neurons tested showed much weaker responses to the shuffled stimuli compared with the plane stimuli. In 13 of 15 neurons tested, the maximum response to the shuffled stimuli was significantly weaker than that to the plane stimuli (t-test, P < 0.05), and in 10 of these, the relative magnitude of the response was <0.5. These results indicate that the sensitivity to slant and/or tilt of the recorded neurons cannot be attributed to mere sensitivity to the velocity distributions. We conclude that these neurons were responding to the overall structure of the velocity field of the stimuli.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

To examine the role of area MSTd in SFM processing, we assessed the selectivity of MSTd neurons to the 2D visual stimuli that simulated 3D-oriented rotating planes. Two-thirds of the neurons that responded to these stimuli had selectivity for at least one of the stimulus parameters, namely tilt and/or slant, that can define the simulated 3D surface orientation of the rotating plane. This selectivity could not be attributed to the direction of local translational motion, local speed, local speed gradients or distribution of velocities in the stimuli. The preferred tilt and slant of MSTd neurons were distributed across the whole range of the stimuli used (Fig. 6). Thus these results suggest that area MSTd can code the 3D surface orientation and that this area is involved in SFM processing.

Representation of surface orientation in area MSTd

We showed that MSTd neurons have selectivity to tilt and/or slant of rotating planes. How, then, do these neurons represent the surface orientation in area MSTd? The selectivity to slant and tilt varied among MSTd neurons. Some neurons were selective to steeper slants with tilt selectivity, some others were selective to shallower slants without tilt selectivity, and still others were selective only to tilt. To represent the whole range of slant, it should be necessary to sum the signals from neurons with different preferred slants with various weights. The tilt of the stimulus may be represented by the activities of tilt-sensitive neurons.

Seyama et al. (2000) reported a psychophysical study employing stimuli similar to those used in the present study. They found that the visual stimulation by the rotating random dot plane caused a slant aftereffect, and that this slant aftereffect had a tilt dependency. They suggested that the processing of tilt and slant are not independent in humans and proposed that two types of detectors (tilt-sensitive slant detectors and tilt detectors) may be involved in surface orientation processing. The first type of detector in the model might correspond to the weighted summation of the signals from the neurons selective to tilt as well as to slant and those selective only to slant, and the second type of detector might correspond to neurons selective to tilt.

With regard to tilt selectivity, we found a weak but significant bias toward 90° in the preferred tilt distribution (Fig. 6A). Tanaka and Saito (1989) have examined the sensitivity of MST neurons to shear stimuli that are closely related to our plane stimuli, in particular, those with higher slants. Their results indicate that there is some bias toward the horizontal axis in the distribution of the preferred axis of the shear stimuli. In their Fig. 15, 6 of 14 neurons that were sensitive to the axis of the shear stimulus preferred the horizontal axis. Thus the bias of the preferred tilt observed in the present results seems consistent with their results.

Elements in the rotating plane for recovery of 3D structure

Numerous stimulus elements vary with the change in the orientation of the rotating plane; namely, local translational motion, local speed, local speed gradients, shearing motion, and the orbits of the moving individual dots. These elements can be potential clues for recovering the 3D orientation. Our control experiments showed that the selectivity to the plane stimuli cannot be attributed merely to the selectivity to local translational motion or local speed in the stimuli.

Psychophysical studies have demonstrated that humans can perceive depth from the speed gradients, meaning that the speed gradient is an important clue for recovering the 3D structure from the 2D image (Braunstein 1968; Harris et al. 1992). Some electrophysiological studies demonstrated that MT neurons are sensitive to the speed gradients in planar motion (Treue and Andersen 1996; Xiao et al. 1997). Thus it is possible that the MSTd neurons we examined in the present study receive signals of local speed gradients in the plane stimuli from area MT. However, our control experiment employing various rotation speeds suggests that the slant selectivity cannot be explained by local speed gradients in the stimuli. In this control experiment, although local speed gradients changed together with the change in rotation speed, the slant selectivity did not change in a manner consistent with that of local speed gradients.

In our rotating plane stimuli, the magnitude of the speed gradient along the direction of tilt is maximal, and that along the direction orthogonal to it is minimal. The ratio between these two values uniquely corresponds to the slant and is invariant with the change in the speed of the stimulus. Signals about local speed gradients extracted in area MT may be compared across different regions in the plane stimuli in area MSTd to compute the surface orientation in a manner independent of the absolute values of local speed.

The pattern of the orbit of each moving dot in the plane stimuli also provides a clue to recover the 3D orientation of the rotating plane. Although each dot in our stimuli had a limited lifetime so that each dot did not establish the complete elliptic orbit, MSTd neurons might exploit the information about orbit curvature if they could interpolate the orbits of the moving dots across time and space. Psychophysical experiments have suggested that space-time interpolation of motion of dots with limited lifetimes occurs in the processing of SFM perception (Treue et al. 1991). Sakata et al. (1994) found that neurons in the STS near area MSTd had selectivity to the rotation-in-depth of single dots. These STS neurons seem to have information about orbit. Thus it may be possible that MSTd neurons also exploit information about the orbit of the moving dots, although MSTd neurons do not respond well to the motion of a single dot.

In terms of four basic elements of optical flow, namely, translation, expansion/contraction, rotation, and deformation, our plane stimulus set is made of combinations of rotation and deformation in various ratios. It is shown that deformation provides information about the structure of the environment in a manner independent of self-motion (Koenderink 1986). Previously, only Lagae et al. (1994) have systematically studied the selectivity of MST neurons to deformation. They found that neurons sensitive to deformation tended to also be sensitive to other optical flow patterns, although neurons selective only to deformation were rare. As our neurons were sensitive to the combination of rotation and deformation, the present results seem consistent with their results.

Cue invariant representation of surface orientation

Area MSTd is believed to be involved in higher motion processing. Sensitivity of MSTd neurons recorded in the present study to the direction of rotation as well as to the simulated 3D orientation of the rotating plane suggests that these neurons specifically encode surface orientation defined by motion cues. A question arises whether surface orientation is represented in a cue-invariant manner in some other cortical areas. An fMRI study about 3D motion found activation of many areas in the intraparietal sulcus (IPS) in addition to MT+, the putative homologue of areas MT and MST in the macaque (Orban et al. 1999). In the macaque, neurons in the caudal area in the intraparietal sulcus (CIP) are shown to have selectivity to a surface orientation defined by disparity as well as texture gradients (Taira et al. 2000; Tsutsui et al. 1999). This result suggests that information about surface orientation defined by different cues such as disparity and texture gradients are integrated in CIP. MSTd neurons project to this area (Boussaoud et al. 1990). Although it is unknown whether CIP neurons are also selective to a surface orientation defined by motion, there is a possibility that the surface orientation extracted from motion information may reach CIP and that CIP neurons represent surface orientation in a visual cue-independent manner.