| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
1University Laboratory of Physiology, Oxford OX1 3PT, United Kingdom; and 2Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland 20892
Submitted 2 September 2003; accepted in final form 18 April 2004
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
for review; Adams and Zeki 2001; DeAngelis and Newsome 1999; Janssen et al. 2000). However, not all of these neurons contribute directly to binocular depth perception. A number of different experimental paradigms have been developed to ascertain whether neurons in these various visual areas carry perceptual signals about binocular depth in addition to stimulus-related signals about binocular disparity (Parker 2004). In most of these paradigms, a single experiment is commonly conducted within one visual area, and a summary of responses over a sample of neurons is presented. No previous work has compared systematically two different perceptual criteria within the same group of neurons from a single cortical area.
For example, some V1 neurons in the macaque are individually almost as sensitive as the monkey itself when tested on a task that requires the discrimination of stimuli on the basis of binocular disparity (Prince et al. 2000). The presence of these highly sensitive neurons suggests that they are specialized for the processing of binocular disparity. But in the same visual area, many cells do not signal the global stereo percept for stimuli like anti-correlated stereograms; these neurons are effectively driven by local binocular correlations that do not necessarily form part of the global stereo match (Cumming and Parker 1997). Moreover, V1 neurons do not explicitly signal relative disparity between nearby visual features (Cumming and Parker 1999) and respond consistently to false matches for windowed sinusoids (Cumming and Parker 2000). Overall, neurons in V1 are thought not to underlie the stereo depth percept directly. However, the relationship between these various tests has never been systematically examined: for example, we do not know whether the most sensitive V1 neurons identified in the discrimination task also respond in characteristic ways to anti-correlated stereograms.
There is growing evidence that primate area V5/MT may have a role in stereoscopic depth perception. Electrical micro-stimulation of small groups of disparity-selective neurons can bias the binocular depth percept reported by awake, behaving macaques (DeAngelis et al. 1998). Another line of evidence for the involvement of single V5/MT neurons in binocular depth perception is their response to random-dot structure-from-motion stimuli portraying rotating cylinders. On the computer screen, these cylinders consist of two planes of dots that move in opposite directions with a velocity that varies sinusoidally with position on the screen. Such a cylinder, in which all dots have the same binocular disparity, is inherently ambiguous because the direction of rotation could equally well be clockwise or anticlockwise (Ullman 1979). The direction of rotation can be rendered unambiguous by adding a binocular disparity that separates in stereoscopic depth the dots that are moving in opposite directions. Many V5/MT neurons are tuned to the direction of rotation defined by the sign of binocular disparity. In the case of an ambiguous, zero disparity cylinder, trial-by-trial variation in the firing of single V5/MT neurons is correlated with the monkey's reported percept of the direction of rotation (Dodd et al. 2001). Specifically, neurons that have a preferred direction of rotation, as defined by binocular disparity, also tend to increase their firing when the monkey reports that an ambiguous cylinder appears to be rotating in the preferred direction of the neuron (Dodd et al. 2001).
It has been argued that these perceptual signals, often measured as choice probabilities (CP), are a significant indicator that a neuron is involved in the formation of a perceptual decision rather than merely being responsive to the incoming sensory information (see Parker and Newsome 1998 for a review). If a positive CP indicates that a neuron is a member of a pool that determines the perceptual decision in that task, then any manipulation that generates an excitation in such neurons should necessarily affect the reported percept during the performance of that task. Underlying this argument is a basic assumption that the output of action potentials by neurons simply reflects the fact that the neuron has been excited in some way. The action potentials contain no information that could unambiguously identify the route by which the neuron has been excited.
For example, electrical micro-stimulation of groups of V5/MT neurons with similar direction preference can affect judgments of the direction of motion of stimuli in which small numbers of dots move consistently within a field of randomly moving dots (Ditterich et al. 2003; Salzman et al. 1990). Neurons in the same cortical area with the same visual properties also show measurable CPs for the same task (Britten et al. 1996). However, at this stage, there is no certainty that the electrical stimulation applied by Salzman et al. (1990) actually excited a particular group of neurons with measurable CPs.
Rather than using electrical stimulation, here we exploit the capacity of certain artificial visual stimuli to excite V5/MT neurons. The stimuli we use are anti-correlated random-dot patterns, which have formed the basis of earlier work that rejected a direct perceptual role for V1 neurons in binocular depth perception. For the same neurons, we can quantify the size of a CP and assess its statistical significance. In this paper, we are able to apply both these tests directly to single neurons in V5/MT and directly relate the measured size of the CP to the pattern of excitation brought about by binocularly anti-correlated disparities.
The responses of neurons to binocularly anti-correlated disparities have been examined in V1 with random-dot stereograms (Cumming and Parker 1997). In binocularly correlated stereograms, randomly located dots in one eye are matched with dots in the other eye that have the same luminance; the difference in relative location of the matching dots in the two eyes gives rise to binocular disparity and hence the perception of stereoscopic depth. In an anti-correlated random-dot stereogram, the dots in one eye are matched geometrically with dots of opposite contrast in the other eye. Under most conditions, humans and monkeys cannot see a coherent depth percept in binocularly anti-correlated random-dot patterns (Cogan et al. 1993; Cumming and Parker 1997; Cumming et al. 1998; Read and Eagle 2000). Nevertheless, anti-correlated stereograms produce disparity-selective responses in V1 because V1 neurons respond to false local matches regardless of the presence of a global match for the whole dot pattern (Cumming and Parker 1997). Neurons that underlie stereo perception should discount false matches and signal the global match, i.e., they should have solved the correspondence problem. At a much higher level of visual processing, in inferior temporal cortex, neurons selective for three-dimensional (3-D) shape seem to reflect this because their responses are modulated by correlated but not anti-correlated disparities (Janssen et al. 2003).
The aim of this paper is to make a specific test of the hypothesis that single V5/MT neurons have a general role in the perception of binocular depth (Cumming and DeAngelis 2001). We applied a systematic comparison of the responses of single neurons in two conditions: first when the monkey was making a perceptual judgment about the direction of rotation of binocular cylinder stimuli and second when the monkey's visual system is challenged with anti-correlated disparities. The first test identified neurons that have strong CPs. We reasoned that these neurons might have some special privileged role in binocular depth judgments. In this case, whatever the means by which these neurons are excited, the outcome should be a reliable change in the binocular depth percept reported by the monkey. We already know from previous work (and confirm again in this paper) that certain configurations of binocularly anti-correlated stimuli do not lead to globally consistent percepts of binocular depth. For this reason, our working hypothesis was that the neurons with strong CPs, the activity of which is most tightly linked with perceptual reports in a binocular depth task, should fail to respond in any consistent way when the disparity of binocularly anti-correlated patterns is changed. To the best of our knowledge, this is the first time that two independent tests of whether a neuron might be directly involved in perceptual judgments have been systematically compared on the same single neuron in any brain area.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
We used tungsten microelectrodes (impedance: 0.6–1.2 M
, Micro Probe) to record extracellularly from single neurons in cortical area V5/MT in two awake monkeys (Macaca mulatta). All procedures were carried out in accordance with UK Home Office Guidelines. Individual spike waveforms were stored to computer disk with a time-stamp. Isolation of spikes was checked off-line by inspecting the waveforms and re-selecting individual clusters of waveform shapes. The binocular eye position, psychophysical response of the monkey, and information about the stimulus were also stored with time-stamps. V5/MT was first identified through well-established physiological criteria, including the depth of the cortical sites at which responses were recorded, a high incidence of directionally selective neurons, and a retinotopic map consistent with previous findings about V5/MT in the macaque. Eccentricity of recorded receptive field positions ranged between 0 and 20°. At the end of the experiments, the recording sites in one monkey were confirmed histologically to be within V5/MT (see Dodd et al. 2001).
Stimuli and task
Stereograms were made up of binocularly presented black and white dots (maximum available contrast: 99%) shown at random locations within the stimulus on a mid-gray background (42 cd/m2). Stimuli were matched to the size of the minimum response field. The dot patterns moved in the neuron's preferred direction and at its best speed; in the case of the cylinder stimulus, one plane was moving in the preferred and one in the null direction and the rotation speed of the cylinder was optimized. Dot size was usually 0.25° and dot density 25% (10–15% for RF sizes >8 x 8°). A Wheatstone stereoscope was used to display pairs of dot patterns separately to the two eyes. The monitors were Eizo FlexScan 78 with a frame rate of 72 Hz. Horizontal binocular disparities were introduced by shifting the location of the dots in the two eyes. To avoid potential artifacts that might arise if changes of binocular disparity were to move the stimulus off the monocular receptive field, the random-dot stereogram (RDS) pattern that was matched to the size of the receptive field was surrounded by an annulus of dots at zero disparity. In anti-correlated RDS, dots in one eye were paired with dots of the opposite contrast in the other eye.
Rotating cylinders portrayed by structure-from-motion consisted of two transparent surfaces of random dots that moved in opposite directions (Dodd et al. 2001). The dots moved with a velocity that varied sinusoidally with position on the screen. The disparity of the dots separates the two surfaces and thus defines the direction of rotation. Two stimuli with the same separation in depth but opposite signs of disparity have opposite directions of rotations. Anti-correlated stimuli were always randomly interleaved with correlated stimuli in the stimulus sequence. Between seven and nine different disparities were shown for each condition.
The positions of both eyes were monitored with surgically implanted scleral search coils. After the monkey acquired the fixation point, a stimulus appeared in the receptive field for 2 s. The animals received a fluid reward either for maintaining fixation or, in the case of CP measurements (with the binocularly correlated cylinders), for making a correct saccade to one of two targets that appeared after the stimulus and fixation point were removed. In the case of the ambiguous zero disparity cylinder, the monkey was rewarded in 50% of the trials at random. Detailed experimental methods and analysis procedures for the CP experiments have been described previously (Dodd et al. 2001).
Data analysis
To analyze the pattern of anti-correlated tuning, a Gabor function was fitted to the mean firing rate as a function of disparity for each neuron by nonlinear regression (using the square roots of the mean firing rates, Prince et al. 2002)
 |
and 
are the spatial frequency and phase of the cosine component, 
is the SD of the Gaussian, d is the disparity, D is a position offset, and B is the baseline firing rate. To allow pairwise comparisons, only phase () and amplitude (A) were allowed to differ between correlated and anti-correlated data for a particular cell (Cumming and Parker 1997). Other parameters, like baseline, were fixed for each pair of tuning curves. Analysis of CP has been described in detail elsewhere (Britten et al. 1996; Dodd et al. 2001). |
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
We begin by examining the responses of single V5/MT neurons to a binocularly anti-correlated version of the cylinder stimulus. The cylinder was matched in size, direction tuning and speed to the receptive field. The parameters for the dots (dot density, size, and refresh rate) that make up the cylinder were chosen so that an anti-correlated single-plane RDS would not yield a coherent depth percept. The responses of 164 disparity-selective V5/MT cells were measured as a function of the disparity of anti-correlated cylinder stimuli. The minimum range of disparities tested was –0.6 to +0.6°, and the anti-correlated stimuli were presented randomly interleaved with a set of binocularly correlated stimuli over the same disparity range. Of these 164 neurons, 156 were tuned to binocular disparities in the correlated versions of these cylinders (ANOVA, P < 0.05) and were further analyzed.
In response to the anti-correlated cylinders, we saw three tuning patterns. In comparison with previous data from V1 and MST (Cumming and Parker 1997; Takemura et al. 2001), the most unusual pattern is illustrated in Fig. 1A. This neuron has a tuning curve for anti-correlated cylinders that has the same peak and general shape as its tuning curve for correlated cylinders. The second pattern is shown in Fig. 1B, where the anti-correlated response for cylinders is inverted relative to the correlated tuning curve. This behavior is common for V1 neurons with planar random-dot patterns (Cumming and Parker 1997). The third case, also observed in V1, is illustrated in Fig. 1C, where the cell shows little modulation as a function of disparity in the anti-correlated version of the cylinder stimulus.
|
). They were specifically required to have the same value of baseline firing and the same modulation frequency (see METHODS for equation). This procedure was applied for all of the 156 neurons that showed statistically significant tuning to correlated cylinders (ANOVA, P < 0.05). Thus Fig. 1A depicts a cell with a phase difference of 
0° and an amplitude ratio of 1.10, whereas the curves in Fig. 1B show a phase difference of approximately 
radians (180°) and an amplitude ratio of 0.49. The amplitude ratio for the cell in Fig. 1C is 0.11.
In total, 88/156 neurons (56%) were selective for the disparity in the anti-correlated cylinder (ANOVA, P < 0.05). When we analyzed the Gabor functions to the tuning curves of these 88 neurons, about half displayed a phase shift between anti-correlated and correlated responses that was close to , whereas the other half showed zero phase shift (Fig. 1D). There was no difference between the data from the two animals. As it was not possible to assign a meaningful phase shift to neurons that did not show significant modulation to the anti-correlated cylinders, such neurons are excluded from Fig. 1D.
On average there was little reduction in baseline firing for the anti-correlated versus the correlated disparities. For instance at zero disparity, the average raw firing rate for the correlated cylinder was 27.8 spikes/s and for the anti-correlated cylinder 26.1 spikes/s (for 154 disparity-selective cells). So across a population of disparity-selective cells, the raw firing rates alone give very little information about the type of stimulus presented.
The data suggest two patterns of response to anti-correlated cylinder stimuli, one of which shows an inversion when comparing correlated and anti-correlated responses and the other for which the responses are in phase. Neither group is likely to underlie the cylinder percept: neither monkeys nor humans can utilize anti-correlated disparities perceptually at these dot densities, yet both these groups of neurons show consistent tuning for the disparity of anti-correlated cylinders.
Cylinder psychophysics
We need to examine in greater detail the perception of binocular depth with cylinder stimuli made from anti-correlated dots to ascertain that neither of the two neuronal response profiles described in the preceding text could underlie judgements of direction of rotation of anti-correlated cylinders. With a sufficiently high dot density, neither human nor monkey observers can extract a depth cue from an anti-correlated dot figure consisting of a central circle or square on an annular background (Cogan et al. 1993; Cumming and Parker 1997; Cumming et al. 1998; Julesz 1971). When human observers view anti-correlated cylinders with non-zero disparities, they do not report a consistent global direction of rotation similar to that produced by correlated cylinders.
To check separately that monkeys cannot use anti-correlated disparities to make judgements about the rotation of cylinders, we attempted to train one monkey intensively. The animal was one of the two in this study and was fully trained with correlated cylinders, for which it could achieve psychometric thresholds of 0.0062° at an eccentricity of 1.68°. Across 55 correlated cylinder experiments, this monkey achieved a mean threshold of 0.02° for eccentricities ranging from 0 to 19.6° (mean 9.7°). Because it is unclear whether non-zero anti-correlated disparities might induce a rotation in the same direction or the opposite direction compared with correlated disparities, we tried different reward schedules. First, anti-correlated stimuli were interleaved with correlated cylinder stimuli. In this case, correct discrimination of the direction of rotation for the correlated cylinders was rewarded, whereas any responses to the interleaved anti-correlated cylinders were rewarded in 50% of the cases at random. A reward schedule was then chosen according to the choices that the monkey had made when anti-correlated disparities were added to the cylinder surfaces. This choice was not always consistent from day to day or even between one block of trials and the next. When we finally settled on rewarding the monkey to anti-correlated disparities in the same way as for correlated disparities, the monkey showed some consistent behavioral responses to the anti-correlated cylinders but never achieved >70% correct at any disparity (tested in the range ± 0.15°, which is >7 times threshold for correlated stimuli). Despite regular training sessions of 2–3 h/day over 3 months and testing >40,000 anti-correlated trials, the monkey appeared to have no consistent access to the information in V5/MT about these patterns. This behavioral result is also in agreement with that obtained by Janssen et al. (2003) for curvature discrimination in macaques with binocularly anti-correlated random-dot patterns.
The psychophysical result is significant in another respect. It is difficult to measure CPs for the anti-correlated cylinders in the same way as for correlated cylinders because no direction of rotation could be assigned according to the specific sign of binocular disparity. It is therefore unclear whether a neuron that has a stimulus preference for clockwise rotation may be expected to increase or decrease its firing when the monkey makes a perceptual choice of clockwise rotation when viewing a binocularly anti-correlated cylinder. The lack of an appropriate reward schedule also means we would not be able to control the monkey's behavior tightly. For these reasons, monkeys were not required to indicate a perceptual choice after presentation of anti-correlated cylinders during recording experiments.
Choice signals versus response to anti-correlation in single cells
We therefore adopted a different approach to investigate whether a distinct subgroup of the V5/MT cells that have characteristic responses to the anti-correlated cylinder can be identified as supporting binocular depth perception in a more general way. Initially, neurons were chosen because they were selective for the direction of rotation of the correlated cylinder (ANOVA, P < 0.05). Then the response of the same neurons to anti-correlated cylinders was recorded. After this, we went on to test the strength of the CPs in these neurons measured with bistable correlated cylinders. The strength of measured CP (Britten et al. 1996; Dodd et al. 2001) is one index of the potential contribution of these neurons in binocular depth tasks. A second index is derived from comparisons of the response of the neuron to correlated and anti-correlated binocular figures as developed earlier in the results. For 61 neurons, we collected enough data to compare the two indices.
First, we compared the phase shift measured in response to the anti-correlated cylinder against the cylinder CP. In Fig. 2A, 
depict the neurons with significant tuning to anti-correlated disparities (ANOVA P < 0.05), where phase could reliably be assigned to the fitted Gabor function. There was no association between the two measures: neither the group of cells with same sign of tuning nor the group that showed an inversion to the anti-correlated cylinder displayed generally lower CPs. But what about the neurons that show little tuning to anti-correlated disparities, i.e., those that have a low-amplitude ratio? One might expect such neurons would also show larger CPs, indicating a close link with the stereo percept of the bistable cylinder. However, there was no correlation between amplitude ratio for the anti-correlated disparities and size of the CP (Fig. 2B). Moreover, neurons that were not tuned to anti-correlated disparities (ANOVA, P > 0.05) did not show higher CPs (Fig. 2C). Thus there is no distinct subgroup of V5/MT neurons that represent the binocular depth percept regardless of the particular stimulus and task. It is also clear that there are individual neurons that carry both perceptually relevant signals in the cylinder task with correlated cylinders, related to the sign of the animal's judgment of rotation, and reliable signals about the sign of the disparity of anti-correlated cylinders, which the monkeys appear to be unable to use in making judgments about direction of rotation.
|
If we put forward a naïve model that simply pools responses from the neurons showing "in-phase" behavior and those showing inverted tuning, then the responses could potentially cancel out when testing with anti-correlated cylinder stimuli. This is only possible because the pattern of neuronal responses for the anti-correlated cylinder in V5/MT (Fig. 1D) appears different from responses previously reported in V1 for an anti-correlated single plane stimulus made up from random dots (RDS). Hardly any in-phase responses were reported there (Cumming and Parker 1997). Consequently, this simple pooling model cannot reconcile the responses actually found in V1 with psychophysical responses to anti-correlated RDS. If responses to planar RDS in V5/MT are similar to those reported for V1, this would also rule out a simple pooling model as a description of how V5/MT neurons support depth perception. We therefore investigated responses to planar anti-correlated RDS in V5/MT neurons. Because the single-plane RDS stimulus is a more simple depth stimulus than the cylinder with its two surfaces and opposite motion signs, its use allowed us to investigate not only whether anti-correlated disparities were differently represented in V5/MT than in V1, but also whether there is a select group of neurons in V5/MT that has a privileged role in binocular depth perception. Specifically we set out to compare the responses of individual neurons to anti-correlated planar RDS against their CP for the bistable cylinder stimulus.
We used an RDS pattern the dots of which moved in the preferred direction for the neuron's receptive field because of the strong direction selectivity of V5/MT neurons. We obtained quantitative tuning data to correlated and anti-correlated moving RDS patterns for 175 V5/MT neurons. Ninety-one percent (159/175) of neurons were disparity selective (ANOVA, P < 0.05) for binocularly correlated RDS. For 144/175 of these neurons, we also collected tuning for binocularly correlated cylinders; 140/144 (97%) neurons were tuned to the disparity in the cylinders. All of these 140 cells were included in the analysis of the responses to anti-correlated cylinders discussed earlier.
Figure 3A represents the response of one cell selective for the disparity of moving RDS. The neuron is disparity-selective for both correlated and anti-correlated stimuli with the two tuning curves related by an inversion. Figure 3B depicts a cell that is not selective for anti-correlated RDS. Overall, 71 (45%) of the 159 disparity-selective cells showed significantly modulated responses to anti-correlated RDS (ANOVA, P < 0.05). Again the relationship between the correlated and anti-correlated tuning curves was quantified through fitting a Gabor function (see METHODS). For the example cell in Fig. 3A, the phase shift between the two tuning curves is approximately radians and the amplitude ratio of the anti-correlated to the correlated response was 0.43. Most V5/MT neurons that are tuned to both anti-correlated and correlated binocular stimuli display this pattern of responses (Fig. 3C). However, there are also many neurons in V5/MT that show little or no selectivity for the disparity in binocularly anti-correlated patterns: Fig. 3C shows that several of those neurons that are tuned to anti-correlated RDS have low values for the amplitude ratio and 55% of the neurons in our sample are not significantly tuned to anti-correlated RDS at all (ANOVA, P > 0.05). These data are very similar to those reported for V1 (e.g., the mean of the amplitude ratios was 0.52 for V1, 0.50 here for V5/MT; phase shifts cluster around in V1) (Cumming and Parker 1997). Also, there is only a small decrease (15%) in raw firing rates for anti-correlated RDS. Across 159 disparity-selective cells, V5/MT neurons responded to the correlated RDS at zero degrees disparity with 43.5 spikes/s and to the anti-correlated RDS at zero degrees disparity with 36.6 spikes/s. Taken as a whole, if we were to apply the simple pooling model discussed earlier, the pool of V5/MT neurons has a consistent, sizeable change in response as the disparity of the anti-correlated RDS stimulus is changed.
|
|
The comparison of each individual neuron's response to the anti-correlated planar RDS, and its measured CP for the bistable cylinder again provides no support for the hypothesis that a distinct group of V5/MT neurons might carry perceptually relevant signals: there is no correlation between the amplitude ratio and CP (Fig. 5). There is also no significant correlation between the raw firing rate of a neuron to an anti-correlated RDS (at zero disparity) and the neuron's CP for a correlated cylinder (data not shown). As seen previously, the firing of a neuron can be closely related to stereo perception in one task (CP for correlated cylinders), but under other circumstances, the same neuron may carry a signal to which the monkey has apparently no access when making perceptual judgements of binocular depth (in the case of anti-correlated disparities). We conclude that neither single V5/MT neurons, nor any obvious fixed pool of them, consistently signal perceived stereo depth, regardless of the task. We hypothesize that the relationship between neuronal firing in V5/MT and judgments of binocular depth is dynamic, a point that we take up in the DISCUSSION.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
), V5/MT neurons respond systematically to the sign of anti-correlated binocular disparities. In response to these anti-correlated patterns, V5/MT neurons carry a signal that the observer fails to exploit when forming judgments of binocular depth (Cogan et al. 1993; Cumming and Parker 1997; Cumming et al. 1998). Nonetheless, the very same neurons that carry this signal often show a high CP when the monkey is judging the direction of rotation of ambiguous structure-from-motion cylinders, a task that depends critically on the sign of the stereoscopic disparities. We conclude that if these neurons contribute directly to binocular depth judgements, they do so in a task-specific way. One way that this could be implemented is through a top-down perceptual signal. First we examine whether there are ways to explain the data without such a hypothesis. The most obvious critique is to argue that one (or indeed both) of the indices that we have compared is meaningless. In relation to binocularly anti-correlated stimuli, it may be argued that these are unnatural stimuli that are rejected by the binocular depth system of the primate brain. From this perspective, the failure to find any correlation with the presence of CPs on binocular neurons is unsurprising.
However, because we demonstrate here that the pool of neurons identified by an appropriate CP for the correlated cylinder can also reliably indicate the sign of a disparity in an anti-correlated RDS, such an interpretation still requires that the pooled output of V5/MT neurons does not control the sign of depth judgments when the stimulus is anti-correlated. That is, this interpretation implies that the pool of neurons that determines the sign of depth judgments changes between correlated binocular RDS stimuli and anti-correlation. Furthermore, we already know that binocularly anti-correlated stimuli can effectively control some aspects of behavior related to binocular depth: they can drive vergence eye movements (Masson et al. 1997; Takemura et al. 2001) and they can perturb the detectability of correlated binocular disparities (Neri et al. 1999). In both these cases, the disparity of an anti-correlated stimulus has the inverse effect on behavior compared with the same disparity presented in a correlated stimulus: anti-correlated "near" disparities have an effect similar to that of correlated "far" disparities. For perceptual judgements about the sign of binocular depth, anti-correlated signals are rejected by the primate visual system. This rejection implies that coherent signals about anti-correlated disparities, which we know can inform other behaviors, must be disregarded, even though they are present in individual neurons that carry perceptually relevant signals for other binocular depth judgements. Thus the pool that determines the stereo percept has changed between these two conditions.
Similarly, it might be argued that CPs are meaningless as a neuronal correlate of behavioral decisions. The choice-related signals might be spread haphazardly across the population of V5/MT neurons in a way that bears no relation to the other properties of the neurons. This argument has been considered carefully before (Britten et al. 1996) and rejected. In the case of the cylinder data, it may be noted that the neurons with strongest CPs are also most sensitive to the stimulus-related binocular information in the cylinder (Parker et al. 2002). The value of the CP also plays a central role in the theoretical interpretation of the link between neuronal activity and sensory perception (Parker and Newsome 1998; Shadlen et al. 1996). For the present, we take both indices as meaningful and consider in more detail how our data can be reconciled with existing findings.
Last, it might be argued that the judgements of depth with correlated RDS and anti-correlated RDS represent two very different tasks that need have no relation with each other. Certainly, it is true that correlated RDS and anti-correlated RDS stimuli are readily distinguishable from one another, and in that sense, the binocular depth percepts delivered by the two stimuli differ. Therefore we might simply not understand the nature of the anti-correlated percept well enough to exclude the possibility that the same pool of V5/MT neurons contributes to their percept. However, the task set to the monkey with the cylinder is to make a perceptual judgment about the direction of rotation of the cylinder. This depends critically on judgements of the sign of the binocular disparity in the stimulus—in other words, near or far with respect to binocular fixation. The exact value of the added disparity does not contribute critically to the percept of direction of rotation for the cylinder as long as the value is above threshold. The questions to address this criticism therefore are whether the neuronal pool in V5/MT has the capacity to resolve the sign of binocular disparity and thereby also resolve the perceived direction of rotation and whether neurons that carry perceptually related signals when the monkey is judging direction of rotation have a fixed contribution to all perceptual judgments that require judgment of the sign of binocular disparity like, for example, for anti-correlated disparities. We have shown that the same pool of disparity-selective V5/MT neurons can systematically signal both the sign of binocular disparities that determines the direction of rotation of a cylinder as well as the disparity sign of planar RDS stimuli. However, in one case, these neuronal signals correlate with the resultant depth percept but for the anti-correlated stimuli they do not.
A simple static model of the link between neuronal activity and perceptual decisions is present in most of the literature on sensory decision theory (Graham 1989; Green and Swets 1966), which adopts a linear pooling of sensory signals followed by a decision stage. This model was used to analyze neuronal mechanisms of sensory decisions about moving stimuli by Newsome, Movshon, Britten, Shadlen and collaborators (Britten et al. 1992, 1996; Newsome et al. 1989; Shadlen et al. 1996). Their model pools signals from two groups of neurons with opposed direction preferences and then subtracts the summed responses of each pool to make a decision variable. The same underlying model motivates current studies of stereo depth judgments in V5/MT (DeAngelis and Uka 2001) and was used by our group (Prince et al. 2000) to study how V1 neurons contribute to stereo depth discriminations. All of these models and analyses involve the linear pooling of neuronal spike counts. The analysis of CPs has also followed in this path. In the analysis of Britten et al. (1996) and the model of Shadlen et al. (1996), CPs arise from the random increase of activity in one of the two pools. Because that pool has some form of functional label on it, this results in an increased probability that the animal reports the motion direction corresponding to the label belonging to the more activated pool.
Considering the cylinder data alone, one possible alternative to a dynamic pooling model might be to adopt a literal version of linear pooling, which adds together all the neuronal responses generated by a particular direction of the rotation of the binocularly correlated cylinder. This could be as simple as pooling all V5/MT neurons tuned to leftward motion and near disparities to signal the presence of clockwise rotation of the cylinder, but the pool for clockwise rotation might also include neurons tuned to rightward motion and far disparities. In either case, this pooling model will have a null response to binocular anti-correlation if it combines across those neurons that invert their response to cylinders under binocular anti-correlation and those that show a same sign response. The observed pattern of neuronal signals could then potentially be reconciled as consistent with the perceptual reports of direction of rotation for both correlated and anti-correlated cylinders. However, this conclusion applies exclusively to the cylinder data. A problem arises as soon as we try to apply the same model to planar anti-correlated RDS. When V5/MT neurons are tested with planar anti-correlated RDS patterns, the individual neurons either show inverted responses or they do not change their response much. The neuronal population as a whole shows an inverted response. For this reason, we are led back to the conclusion that single neurons may contribute to judgments about the sign of binocular depth in one task but not another. The pool of neurons underlying the perceptual judgment appears to be assembled dynamically.
The difference between the neuronal selectivity to anti-correlated planar RDS and cylinders that we found in this study was unexpected. Some neurons show the same pattern of tuning for disparity for both correlated and anti-correlated cylinders (with 0 phase shift between the 2 tuning curves). The inverted response (180° phase shift), which is observed with planar RDS patterns in cortical areas V1, V5/MT, and MST, can be explained by the disparity energy model or variants of it (Cumming and Parker 1997; Ohzawa et al. 1997; Read et al. 2002; Takemura et al. 2001).
It is not clear what gives rise to the pattern of neuronal responses with a phase shift of zero between the correlated and the anti-correlated cylinder. It is important to recognize that when the sign of disparity is specified for a cylinder stimulus, this describes a specific relationship between motion signals and binocular depth. Consider two correlated cylinders with opposite directions of rotation. The displays for the two stimuli contain the same motion directions, and the front and rear planes are at the same depths with respect to the binocular fixation point; the cylinders only differ in that the direction of motion that is present in the front plane of one is present in the rear plane of the other. Therefore the sign of disparity in the cylinder stimulus defines a specific conjunction of binocular depth and motion, to which many V5/MT neurons are sensitive. Because the only difference between opposite disparities is this conjunction, it is not surprising that almost all tuning curves show odd-symmetry in response to the cylinder. In some of these neurons, sensitivity to this conjunction appears to reverse when they are challenged with the binocularly anti-correlated cylinder. This is evident in the observed phase shifts, which show a bimodal distribution for the cylinder stimulus but not for the planar RDS. The preponderance of odd-symmetry in the tuning curves (for both correlated and anti-correlated stimuli) makes phase shifts cluster around half a cycle. However, whether the response to anti-correlated cylinders has the same or an inverted sign does not matter from the point of view of binocular depth judgments. In either case, the depth percept is not under the control of the binocular disparities within the stimuli, and individual neurons respond to binocular depth matches that are rejected by the perceptual system as a whole.
The substantial issue for this paper is that, although there are many V5/MT neurons that have a strong perceptual component in their neuronal firing for a perceptually bistable stimulus that appears to rotate in depth, these same neurons do not solve the binocular correspondence problem. CPs indicate that these cells in some way or other reflect stereo depth perception for the cylinder task (see also Dodd et al. 2001), but these neurons respond to binocular anti-correlation. In other experiments, electrical stimulation of small groups of V5/MT cells in a binocular depth task showed that these cortical neurons contribute to binocular depth percepts in a way that is consistent with measures of their disparity selectivity (DeAngelis et al. 1998). Therefore some experimental observations suggest that V5/MT neurons are closely associated with binocular depth judgments, whereas other observations indicate the opposite view.
CPs have been taken as an important criterion for whether a neuron contributes directly to the perceptual decision pool (Parker and Newsome 1998). If this decision pool was not fixed but dynamically assembled specifically for the task, all these observations could be reconciled. Other approaches at reconciliation bring in arbitrary assumptions to resolve the contradictions revealed by our experiments. For example, one could suggest that there are in reality two, entirely separate pools of disparity-selective neurons in V5/MT. One set of neurons has been studied with electrical microstimulation but might conceivably bear no relationship at all with the other class of neurons that showed high CPs in this study. For each set of neurons, there is a plausible case that they are on the perceptual pathway for binocular depth judgements. As a whole, this supposition of two distinct pools seems unlikely given that the neurons that we recorded here, like the cortical sites in the electrical micro-stimulation study, were selected on the basis of their direction and disparity selectivity. It would suggest that there is some other set of neurons in V5/MT that was discovered in the micro-stimulation study and should show choice-related firing but has, for whatever reason, been systematically missed in this study. Of course it is true that CP establishes a correlative link, not necessarily a causal link, between neuronal responses and behavioral decisions. However, it seems also highly unlikely that there are neurons that can be shown to have a causal effect on a perceptual judgment but show no sign of a correlative link with the same perceptual judgment.
Moreover, the present experiments here address rather directly the issue of whether neurons with choice-related firing have been systematically missed or ineffectively stimulated with electrical microstimulation. In these experiments, we know quite specifically that the anti-correlated cylinder stimulus has indeed increased the firing of neurons that have high CPs, and we know that manipulating the firing of these same neurons by altering the disparity of the anti-correlated cylinder or RDS stimulus has no reliable effect on the sign of the reported depth percept.
Also, when we compared the selectivity of neurons for anti-correlated disparities in cylinder and RDS stimuli (see Fig. 4), it became clear that disparity-selective neurons did not necessarily respond in the same way to the two different stimuli. So we might need to postulate more than two different hard-wired pools of disparity-selective neurons with specific output connectivity.
A more likely approach to reconciling the presence of CPs and the effect of micro-stimulation with a response to binocularly anti-correlated stimuli is to hypothesize that the V5/MT neurons recorded here are indeed part of the pathway for the processing of binocular depth, but their responsiveness to anti-correlation is eliminated in later processing. This would mean some form of selective pooling at a later site through specific connectivity, at least for the anti-correlated RDS. This could potentially be achieved by the same kind of dynamic selection process we discuss in the following text. It might also be achieved by a static model in which V5/MT neurons sensitive to the disparity of binocular anti-correlated stimuli project to some cortical sites not involved in depth perception, whereas the V5/MT neurons that are insensitive to anti-correlated disparities project to other cortical sites that would contribute to stereo depth perception. As discussed earlier, to make this work we would have to give up on CPs as an indicator of whether a neuron is involved in perceptual judgments.
There are indeed other neuronal sites where the responses to binocular anti-correlation appear to be wholly eliminated: in the inferotemporal (IT) cortex, neurons lose completely their characteristic sensitivity to the curvature of random-dot surfaces when challenged with anti-correlated versions of these figures (Janssen et al. 2003). It is unclear whether any parts of the parietal pathway, to which V5/MT projects, also show a consistent failure to respond to anti-correlated patterns. Certainly, in MST, which is an immediate neighbor of V5/MT, disparity-selective neurons respond to anti-correlated random-dot stereograms with an inverted pattern (Takemura et al. 2001). We know very little about disparity selectivity downstream from MST. Preliminary results from area CIP show that only 20% of neurons that were selective to the 3-D orientation of a flat random-dot surface were also selective for an anti-correlated version of the same surface (Naganuma et al. 2002). Generally, receptive fields become more complex, and extra-retinal inputs become more important for neuronal selectivity. Indeed areas like VIP or LIP already accumulate sensory evidence from cortical areas like V5/MT and transform them into a motor command or at least into an appropriate reference frame for motor commands (Sakata et al. 1997; Shadlen and Newsome 2001). It is not clear that anti-correlated responses need to be completely eliminated to achieve this, especially because there are already subpopulations in V5/MT that hardly respond to the anti-correlated stimuli. These conjectures about the nature of forward projections from V5/MT to other cortical areas are relevant and intriguing, but they do not offer an explanation for the pattern of CPs seen in V5/MT and its relationship to other properties of the neurons.
We therefore still need to consider which neurons constitute the putative "perceptual pool" in V5/MT for binocular depth judgments (as discussed by Shadlen et al. 1996 for motion). The pool should consist of only a fraction of the disparity-selective neurons in this area. The problem is to identify which neurons are included. The size of the measured choice signals for the cylinder stimulus varies between disparity-selective cells (Dodd et al. 2001 and see Figs. 2 and 5). Also, larger CPs are systematically associated with neurons that have greater sensory performance in discriminating binocular depth (Parker et al. 2002). In this paper, we have shown that there are V5/MT neurons with high CPs that modulate their response with the disparity of anti-correlated stimuli. These data suggest how the perceptual pool may be formed. The answer could be more complex than a simple question of neuronal "identity" as assumed in a static model. The comparison of choice signal and response to anti-correlation within one cell shows that there is no identifiable group of cells that always controls perceptual reports.
The paradox raised by our findings can be solved by proposing that there is a perceptual pool that forms dynamically dependent on the task. There is already strong evidence for perceptual signals of this kind in visual cortex, albeit from a different experimental paradigm—attention. To be effective, attentional processes must act dynamically on different pools of neurons according to the task. The size and nature of the neuronal modulation measured in V5/MT for spatial or featural attention suggest a focal top-down signal (Treue and Maunsell 1996; Treue and Martinez-Trujillo 1999). The magnitude of neuronal response modulation by choice in our task and the range of attentional modulation that has been reported for V5/MT are comparable (Dodd et al. 2001; Krug 2004; Seidemann and Newsome 1999; Treue and Maunsell 1999). The choice signal in V5/MT could be generated by a top-down mechanism as has been proposed to account for attentional effects. The signal generated by this mechanism should be specific to one feature of the cylinder, its direction of rotation. But the choice paradigm differs from studies on feature attention in some important respects, for example, that the animal is not specifically precued for a particular direction of rotation. From the proposed mechanism follow clear predictions. Supposing that the choice signal is related to feature attention, the choice signal should be present for a range of stimuli not just the ambiguous one. Some evidence supporting this has been reported (Dodd et al. 2001). Also, if a common top-down choice signal acts on a pool of disparity-selective V5/MT neurons, we should see changes in the interactions between neighboring neurons when we change the perceptual relevance of these neurons, for example, by changing the task or the stimulus. Work to test this hypothesis is currently being undertaken in our laboratory.
|
GRANTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: K. Krug, University Laboratory of Physiology, Oxford University, Parks Road, Oxford OX1 3PT, United Kingdom (E-mail kristine.krug{at}physiol.ox.ac.uk).
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONGRANTSREFERENCES |
|---|
Bradley DC, Chang GC, and Andersen RA. Encoding of three-dimensional structure-from-motion by primate area MT neurons. Nature 392: 714–717, 1998.[CrossRef][Medline]
Britten KH, Newsome WT, Shadlen MN, Celebrini S, and Movshon JA. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis Neurosci 13: 87–100, 1996.[ISI][Medline]
Britten KH, Shadlen MN, Newsome WT, and Movshon JA. The analysis of visual motion—a comparison of neuronal and psychophysical performance. J Neurosci 12: 4745–4765, 1992.[Abstract]
Cogan A, Lomakin A, and Rossi A. Depth in anticorrelated stereograms. Vision Res 33: 1959–1975, 1993.[CrossRef][ISI][Medline]
Cumming BG and DeAngelis GC. The physiology of stereopsis. Ann Rev Neurosci 24: 203–238, 2001.[CrossRef][ISI][Medline]
Cumming BG and Parker AJ. Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature 389: 280–283, 1997.[CrossRef][Medline]
Cumming BG and Parker AJ. Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity. J Neurosci 19: 5602–5618, 1999.
Cumming BG and Parker AJ. Local disparity, not perceived depth, is signalled by binocular neurons in cortical area V1 of the macaque. J Neurosci 20: 4758–4767, 2000.
Cumming BG, Shapiro SE, and Parker AJ. Disparity detection in anticorrelated stereograms. Perception 27: 1367–1377, 1998.[ISI][Medline]
DeAngelis GC, Cumming BG, and Newsome WT. Cortical area MT and the perception of stereoscopic depth. Nature 394: 677–680, 1998.[CrossRef][Medline]
DeAngelis GC and Newsome WT. Organization of disparity-selective neurons in macaque area MT. J Neurosci 19: 1398–1415, 1999.
DeAngelis GC and Uka T. MT neurons can account for behavioral performance in a depth discrimination task (Abstract). J Vision 1: 272a, 2001.
Ditterich J, Mazurek ME, and Shadlen MN. Microstimulation of visual cortex affects the speed of perceptual decisions. Nat Neurosci 6: 891–898, 2003.[CrossRef][ISI][Medline]
Dodd JV, Krug K, Cumming BG, and Parker AJ. Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT. J Neurosci 21: 4809–4821, 2001.
Graham N. Visual Pattern Analyzers. New York: Oxford, 1989.
Green DM and Swets JA. Signal Detection Theory and Psychophysics. New York: Wiley, 1966.
Janssen P, Vogels R, and Orban GA. Selectivity for 3D shape that reveals distinct areas within macaque inferior temporal cortex. Science 288: 2054–2056, 2000.
Janssen P, Vogels R, Liu Y, and Orban GA. At least at the level of inferior temporal cortex the stereo correspondence problem is solved. Neuron 37: 693–701, 2003.[CrossRef][ISI][Medline]
Julesz B. Foundations of Cyclopean Perception. Chicago, IL: Chicago University Press, 1971.
Krug K. A common neuronal code for perceptual processes in visual cortex? Comparing choice and attentional correlates in V5/MT. Philos Trans Royal Soc B Biol Sci 359: 929–941, 2004.
Masson GS, Busettini C, and Miles FA. Vergence eye movements in response to binocular disparity without depth perception. Nature 389: 283–286, 1997.[CrossRef][Medline]
Naganuma T, Tsutsui K, Sakata H, and Taira M. Surface-orientation-selective neurons in parietal area CIP and perception of 3D surface orientation. Soc Neurosci Abstr 28: 56.15, 2002.
Neri P, Parker AJ, and Blakemore C. Probing the human stereoscopic system with reverse correlation. Nature 401: 695–698, 1999.[CrossRef][Medline]
Newsome WT, Britten KH, and Movshon JA. Neuronal correlates of a perceptual decision. Nature 341: 52–54, 1989.[CrossRef][Medline]
Ohzawa I, DeAngelis GC, and Freeman RD. Encoding of binocular disparity by complex cells in the cat's visual cortex. J Neurophysiol 77: 2879–2909, 1997.
Parker AJ. From binocular disparity to the perception of stereoscopic depth. In: The Visual Neurosciences, edited by Chalupa LM and Werner JS. Cambridge, MA: Bradford/MIT Press, 2004, p. 779–792.
Parker AJ, Krug K, and Cumming BG. Neuronal activity and multi-stable figure perception. Philos Trans R Soc B Biol Sci 357: 1053–1062, 2002.[CrossRef][ISI][Medline]
Parker AJ and Newsome WT. Sense and the single neuron: probing the physiology of perception. Annu Rev Neurosci 21: 227–277, 1998.[CrossRef][ISI][Medline]
Poggio GF. Mechanisms of stereopsis in monkey visual cortex. Cereb Cortex 3: 193–204, 1995.
Prince SJ, Pointon AD, Cumming BG, and Parker AJ. The precision of single neuron responses in cortical area V1 during stereoscopic depth judgments. J Neurosci 20: 3387–3400, 2000.
Prince SJ, Pointon AD, Cumming BG, and Parker AJ. Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms. J Neurophysiol 87: 191–208, 2002.
Read J, Cumming BG, and Parker AJ. A simple model accounts for the response of disparity-tuned V1 neurons to anti-correlated images. Vis Neurosci 19: 735–753, 2002.[CrossRef][ISI][Medline]
Read JC and Eagle RA. Reversed stereo depth and motion direction with anti-correlated stimuli. Vision Res 40: 3345–3358, 2000.[Medline]
Sakata H, Taira M, Kusunoki M, Murata A, and Tanaka Y. The Trends Neurosci Lecture. The parietal association cortex in depth perception and visual control of hand action. Trends Neurosci 20: 350–357, 1997.[CrossRef][ISI][Medline]
Salzman CD, Britten KH, and Newsome WT. Cortical microstimulation influences perceptual judgements of motion direction. Nature 346: 174–177, 1990.[CrossRef][Medline]
Seidemann E and Newsome WT. Effect of spatial attention on the responses of area MT neurons. J Neurophysiol 81: 1783–1794, 1999.
Shadlen MN, Britten KH, Newsome WT, and Movshon JA. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J Neurosci 16: 1486–1510, 1996.
Shadlen MN and Newsome WT. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86: 1916–1936, 2001.
Takemura A, Inoue Y, Kawano K, Quaia C, and Miles FA. Single-unit activity in cortical area MST associated with disparity-vergence eye movements: evidence for population coding. J Neurophysiol 85: 2245–2266, 2001.
Treue S and Martinez-Trujillo JC. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399: 575–579, 1999.[CrossRef][Medline]
) and its correlated counterpart (
0.05). The amplitude ratio of the 2 tuning curves is 0.11. D: summary of 88 cells with significant tuning to correlated and to anti-correlated cylinders (ANOVA, P < 0.05). The phase difference is plotted against amplitude ratio. Both measures were calculated from the Gabors fitted to the tuning curves obtained with anti-correlated and correlated cylinders (see A + B). The frequency histogram below the graph shows a bimodal distribution of response properties. Of the 88 cells, about half the cells show an inversion of selectivity to anti-correlation and about half show same sign tuning.
