HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) All Versions of this Article: 91/6/2607    most recent 00858.2003v1 Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via ISI Web of Science (19) Citing Articles via Google Scholar Google Scholar Articles by Frazor, R. A. Articles by Crane, A. M. Search for Related Content PubMed PubMed Citation Articles by Frazor, R. A. Articles by Crane, A. M.

Visual Cortex Neurons of Monkeys and Cats: Temporal Dynamics of the Spatial Frequency Response Function

Department of Psychology and Center for Perceptual Systems, University of Texas, Austin, Texas 78712

Submitted 2 September 2003; accepted in final form 29 January 2004

	ABSTRACT

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
We measured the responses of striate cortex neurons as a function of spatial frequency on a fine time scale, over the course of an interval that is comparable to the duration of a single fixation (200 ms). Stationary gratings were flashed on for 200 ms and then off for 300 ms; the responses were analyzed at sequential 1-ms intervals. We found that 1) the preferred spatial frequency shifts through time from low frequencies to high frequencies, 2) the latency of the response increases as a function of spatial frequency, and 3) the poststimulus time histograms (PSTHs) are relatively shape-invariant across spatial frequency. The dynamic shifts in preferred spatial frequency appear to be a simple consequence of the latency shifts and the transient nature of the PSTH. The effects of these dynamic shifts on the coding of spatial frequency information are examined within the context of several different temporal integration strategies, and pattern-detection performance is determined as a function of the interval of integration, following response onset. The findings are considered within the context of related investigations as well as a number of functional issues: motion selectivity in depth, "coarse-to-fine" processing, direction selectivity, latency as a code for stimulus attributes, and behavioral response latency. Finally, we demonstrate that the results are qualitatively consistent with a simple feedforward model, similar to the one originally proposed in 1962 by Hubel and Wiesel, that incorporates measured differences in the response latencies and the receptive field sizes of different lateral geniculate nucleus inputs.

	INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
It has been known for more than three decades that most visual cortex neurons are selective along the dimension of spatial frequency. The average spatial frequency bandwidth is approximately 1.5 octaves (e.g., Campbell et al. 1969; Cooper and Robson 1968; De Valois et al. 1982; Movshon et al. 1978; Robson et al. 1988; for general reviews see De Valois and De Valois 1988; Geisler and Albrecht 2000; Shapley and Lennie 1985). This selectivity is very different from that of the neurons in the lateral geniculate nucleus (LGN) of the thalamus and the ganglion cells in the retina, which are not very selective for spatial frequency or orientation. In comparison, the spatial frequency bandwidth for neurons in the LGN ranges from 3 to >5 octaves (e.g., Croner and Kaplan 1995; Derrington and Lennie 1984; Hicks et al. 1983; Irvin et al. 1993; Kaplan and Shapley 1982; Rodieck and Stone 1965; So and Shapley 1981; Troy 1983a,b; Xu et al. 2002). Although it is unknown exactly how this transformation in the selectivity, from the LGN to the cortex, is produced, several different classes of models have been proposed (e.g., Anderson et al. 2000a,b; Carandini and Ringach 1997; Carandini et al. 1999; Chance et al. 1998; Douglas et al. 1995; Gilbert et al. 1990; Hubel and Wiesel 1962; Troyer et al. 1998; for recent reviews, see Albrecht et al. 2002, 2003; Ferster and Miller 2000).

Most measurements of spatial frequency selectivity have been performed using moving stimuli that were presented for relatively long durations (generally several seconds) to approximate a steady-state condition. However, natural saccadic inspection of a visual scene typically produces transient stimulation: 200- to 300-ms fixations separated by rapid eye movements. Although there are minor differences in the temporal dynamics of saccadic eye movements among humans, macaque monkeys, and cats, fixation durations of approximately 200 ms are typical across all 3 species (Evinger and Fuchs 1978; Fuchs 1967; Stryker and Blakemore 1972; for general reviews see: Carpenter 1991; Ditchburn 1973).

There is now a growing body of evidence to indicate that cortical neurons respond differently under transient, as opposed to steady-state, conditions. For example, 1) the temporal frequency tuning measured using steady-state stimuli shows relatively little low-frequency attenuation (Albrecht 1995; Hawken et al. 1996; Movshon et al. 1978), whereas the temporal response profile measured using transient stimuli decays quite rapidly, faster than would be expected based on the temporal frequency tuning (Albrecht et al. 2002; Müller et al. 1999, 2001; Tolhurst et al. 1980)¹; 2) the temporal frequency tuning measured using steady-state stimuli varies as a function of contrast (Albrecht 1995; Hawken et al. 1992; Holub and Morton-Gibson 1981), whereas the temporal response profile measured using transient stimuli is relatively invariant as a function of contrast (Albrecht et al. 2002); 3) the variability of cortical neurons is approximately proportional to the mean firing rate under steady-state conditions (e.g., Geisler and Albrecht 1997; Softky and Koch 1993; Tolhurst et al. 1983), whereas under transient conditions the relationship between the mean and variance is more complex (Albrecht et al. 2002; Mechler et al. 1998; Müller et al. 1999, 2001; for recent reviews of this literature, see Albrecht et al. 2002, 2003). Given these differences between the responses of visual cortex neurons under steady-state conditions, as opposed to transient conditions, it is important to measure the spatial frequency tuning under transient conditions that are more comparable to natural fixation.

Two recent investigations of spatial frequency tuning have presented gratings for very brief temporal intervals (20–30 ms) in rapid, temporally contiguous succession; a reverse correlation technique was then used to measure the linear component of the responses (Bredfeldt and Ringach 2002; Mazer et al. 2002).² The goal of these studies was to measure the temporal dynamics of spatial frequency tuning over the course of the first 100 ms or so following stimulus onset. Although the results of the two studies are, on the whole, quite consistent, there are differences. First, Mazer et al. reported that the spatial frequency tuning changed very little through time (as they pointed out, the tuning was largely separable), whereas Bredfeldt and Ringach reported that the spatial frequency tuning changed substantially through time (i.e., the tuning was inseparable). Specifically, Bredfeldt and Ringach reported that during the initial time periods (at the onset of the responses) the tuning was quite broad, spanning both low frequencies and high frequencies (as they pointed out, very similar to the tuning of LGN neurons); however, through time, the responses to low frequencies were diminished, and thus the peak of the spatial frequency tuning shifted from low frequencies to high frequencies. Second, Mazer et al. reported that the latency of the response increased as spatial frequency increased (for the component of the response that they reported was not separable), whereas Bredfeldt and Ringach did not report a change in the latency as a function of spatial frequency. Specifically, they found that the onset of the responses to high frequencies and low frequencies appeared concurrently.

The steady-state method of stimulation (with durations on the order of seconds) and the brief, temporally contiguous method of stimulation (with durations on the order of 20–30 ms) have both provided important information about the properties of visual cortex neurons. However, these methods of stimulation are both different from the stimulation that occurs during normal saccadic viewing. In the present study we investigated the spatial frequency tuning of striate visual cortex neurons using stationary gratings that were presented for a temporal interval that is roughly comparable to the duration of a single fixation (i.e., 200 ms). Further, the responses were measured on a fine time scale to investigate the temporal dynamics of spatial frequency tuning during this interval.

Using this stimulus protocol, we find that for most neurons the latency of the response increases as the spatial frequency increases and that the peak of the spatial frequency tuning shifts through time from low frequencies to high frequencies. Interestingly, these dynamic changes are qualitatively consistent with what would be expected from the model proposed four decades ago by Hubel and Wiesel (1962) if one assumes, in the monkey, that magnocellular and parvocellular neurons, or in the cat, that X and Y neurons, converge on the same cortical neuron.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
Recording and physiology

The procedures for the paralyzed anesthetized preparation, the electrophysiological recording, the stimulus display, and the measurement of neural responses using systems analysis were similar to those described elsewhere (Albrecht and Geisler 1991; Albrecht et al. 2002; Geisler and Albrecht 1997; Geisler et al. 2001; Hamilton et al. 1989; Metha et al. 2001). All experimental procedures were approved by the University of Texas at Austin Institutional Animal Care and Use Committee, and conform to the National Institutes of Health guidelines. In brief, young adult cats (Felis domesticus) and monkeys (Macaca fascicularis or Macaca mulatta) were prepared for recording under deep isoflurane anesthesia. Following the surgical procedures, isoflurane anesthesia was discontinued. Anesthesia and paralysis were maintained throughout the duration of the experiment using the following pharmaceuticals. For cats, anesthesia was maintained with sodium pentothal (2–6 mg·kg^-1·h^-1). For monkeys, anesthesia was maintained with sufentanil citrate (2–8 µg·kg^-1·h^-1). For both species, paralysis was maintained with gallamine triethiodide (10 mg·kg^-1·h^-1) as well as pancuronium bromide (0.1 mg·kg^-1·h^-1). The physiological state of the animal was monitored throughout the experiment by continuous measurement of the following quantitative indices: body temperature, inhaled/exhaled respiratory gases, pressure in the airway, fluid input, urine output, urinary pH, caloric input, blood glucose level, electroencephalogram, and electrocardiogram. Microelectrodes were inserted into regions of the striate visual cortex such that the receptive fields of the neurons were located within 5° of the visual axis. Three different types of microelectrodes were used: varnish-insulated tungsten, glass pipette, or glass-coated platinumiridium. The impedances of the microelectrodes ranged from 8 to 21 M.

Action potentials were collected with a temporal accuracy of 0.1 ms and then binned to produce a poststimulus time histogram (PSTH). Note that within this report, and elsewhere (Albrecht et al. 2002), we use the terms PSTH and temporal response profile interchangeably. The bin size for the PSTH was 10 ms, and this average (centered within the 10-ms time bin) was computed every millisecond: a 10-ms running average evaluated at every millisecond throughout the entire time course of the response (cf. De Valois and Pease 1973; Enroth-Cugell and Robson 1966; Gerstein 1960; Levick and Zacks 1970). Note that for some analyses (as will be noted, when appropriate), the PSTH bin size was 1 ms.

Stimulus presentation

The stimuli were presented on a monochrome Image Systems monitor at a frame rate of 100 Hz, with a mean luminance of 27.4 cd/m². To overcome the nonlinearities inherent in visual displays, both hardware and software methods were used to ensure a linear relationship between the requested luminance and the measured luminance. The precision of these methods to overcome the nonlinearities in the visual display was assessed through quantitative measurements that were performed before, during, and after each experiment.

Preliminary measurements

Before the main experimental protocol, several preliminary experiments were performed, using drifting gratings, to determine the optimal orientation, spatial frequency, temporal frequency, and direction of motion. First, the optimal parameters for these 4 dimensions were determined in a qualitative fashion by adjusting the stimulus and listening to the spike trains. Second, the optimal parameters were determined quantitatively by drifting 10 contiguous cycles of each stimulus along a given dimension. The response measure was the average rate of firing. The presentation sequence of the stimulus dimensions was: temporal frequency, orientation, spatial frequency, and contrast. The responses were plotted on-line and the optimal values were determined. Third, step two was repeated with 4 randomly interleaved repetitions of 10 contiguous cycles of each stimulus along a given dimension. Finally, the contrast response function was measured using stationary gratings, as described in Albrecht et al. (2002). For the dimension of contrast, the minimum detectable contrast, half-saturation contrast, and saturation contrast (i.e., the contrast above which further increases in contrast produce little or no further increases in response) were determined (cf. Albrecht and Hamilton 1982; Bonds 1991; McLean and Palmer 1996; Sclar et al. 1990). In all of the experiments reported here, the stimuli were confined to the conventional receptive field, which was determined by expanding the size (the length and the width separately) of an optimal drifting sine wave grating (presented at a saturating contrast) until the neuron's response stopped increasing (DeAngelis et al. 1994; De Valois et al. 1985). The contrast of the grating was held constant over the conventional receptive field; the "windowing function" was one half of one cycle of a cosine (0 to 180°). The function was shifted and scaled to vary between one and zero, thus gradually modulating the contrast at the boundaries. The degree of direction selectivity was measured (Albrecht and Geisler 1991). Cells were classified as simple cells or complex cells using the criteria described by De Valois et al. (1982) as well as Skottun et al. (1991). Specifically, cells were classified as complex cells if the magnitude of the average rate of firing exceeded the magnitude of the first harmonic response. These measurements were performed in 3 monkeys and 3 cats. (Note that the same animals, and oftentimes the same cells, were also used for other stimulus protocols.) The cell types were distributed as follows: 25 monkey cells (18 complex; 7 simple) and 39 cat cells (24 complex; 15 simple).

Stimulus protocol: stationary gratings at different spatial frequencies and phases

Following the preliminary experiments, optimal stationary grating patterns were presented at a saturating contrast level (which was determined for each cell), in 8 different spatial phases, each separated by 45°. Twelve different spatial frequencies were presented; the range of these frequencies was determined based on the responses to drifting gratings such that frequencies on either side of the preferred frequency were sampled. Each of these 12 frequencies was presented at each of the 8 spatial phases, making a total of 96 unique combinations of phase and spatial frequency. Each unique combination was turned on (flashed) for 200 ms and then turned off for 300 ms, with a minimum of 40 repeated presentations (and a maximum of 80). During the 300-ms interval, when the stationary grating was turned off, the animal viewed the mean luminance. As described in detail below, this interstimulus interval (ISI) was introduced to minimize potential interactions between the sequential stimuli. The stimuli were presented in a counterbalanced fashion such that all stimulus conditions were presented an equal number of times, in a random order. With 40 repeated presentations of each condition, this stimulus protocol required about 30 min to complete.

The 300-ms ISI was introduced for several reasons. First, this interval permits recovery from any short-term pattern adaptation (Müller et al. 1999, 2001) or rapid local light adaptation (Adelson 1982; Crawford 1947; Geisler 1981; Hayhoe et al. 1987; Saito and Fukada 1975; for reviews see Hood 1998; Shapley and Enroth-Cugell 1984). Second, as will be shown in the RESULTS, for most cells, the latency of the response increases as the spatial frequency increases; thus if the stimuli are presented in close temporal proximity, then the responses to the 2 stimuli might overlap in time. For example, if a high spatial frequency is presented just prior to a low spatial frequency, with no ISI, the responses to the 2 stimuli will overlap in time (or even reverse order). Third, when a stationary stimulus is turned off, many cells produce large OFF responses that could overlap in time with the ON responses to a subsequent stimulus presented contiguously. In some cases, these OFF responses are larger than the ON responses and they can last nearly as long as the ON responses; none of the OFF responses in this sample lasted as long as 300 ms. Separating the stimuli by 300 ms precludes any of these problems.

Presenting the grating in 8 different spatial phases ensures that 1) the space average luminance remains equivalent throughout the course of the experiment over the entire receptive field, 2) both optimal and nonoptimal spatial phases are sampled, and 3) many different regions of the receptive field are stimulated with different luminance increments and decrements. Further, presenting the stationary grating in 8 different spatial phases, and then computing the average, minimizes the potential errors that are introduced by not having the origin of the stimulus located at the exact center of the receptive field. Minor offsets, or any residual eye movements (cf. Forte et al. 2002), can potentially introduce substantial errors in the estimates of the spatial frequency tuning. For example, simulations (RA Frazor, unpublished observations) show that if the origin of the stimulus (i.e., the location where all spatial frequencies are aligned in the same phase) is offset from the center of the receptive field (which is not easy to measure with a high degree accuracy; cf. Albrecht 1995; Hamilton et al. 1989) by as little as a quarter of a cycle, then the estimate of the peak spatial frequency will be off by 0.5 octaves and the estimate of the bandwidth will be off by 10%. The simulations also show that these errors are eliminated by averaging across the 8 spatial phases.

Finally, given that we are averaging the PSTHs across all of the spatial phases (for a given spatial frequency), it is important to point out that we have performed an analysis to assess the degree of any systematic variation in the overall shape of the PSTHs (for both simple and complex cells) at the different spatial phases (see Analysis of the variation in the PSTHs at different spatial phases, in the METHODS section of Albrecht et al. 2002). In general, we found that, although the amplitude of the response often varied as a function of spatial phase, the shape of the temporal response profile (i.e., the PSTH) was relatively invariant as a function of spatial phase. Approximately 95% of the variation across all 8 spatial phases could be accounted for by simply scaling the amplitude of the average of all 8 spatial phases. This observation held true for both simple and complex cells. Further, as noted in that report, there were no obvious trends in the amount of variance accounted for across animal type or cell type. Given that there is very little systematic residual variation (the median value was 4.2%) in the overall shape of the PSTHs as a function of spatial phase (compared with the average across spatial phase), and for the reasons discussed in the preceding paragraph, it therefore seemed reasonable to average the responses across spatial phase for both complex cells and simple cells.

The stimulus protocol, saccades, and saccadic suppression

Although the 200-ms presentation interval provides a good approximation to the average duration of a single fixation in humans, macaque monkeys, and cats (Evinger and Fuchs 1978; Fuchs 1967; Stryker and Blakemore 1972; for general reviews see Carpenter 1991; Ditchburn 1973), the 300-ms ISI does not provide a good approximation to the average duration of a single saccade. In comparison to humans, macaque monkeys make faster saccades (25–55 ms; Fuchs 1967) and cats make slower saccades (50–150 ms; Evinger and Fuchs 1978). Note, however, that because visual cortex neurons are so selective along many different stimulus dimensions, it is highly unlikely that any single neuron will be stimulated during every fixation while viewing a complex natural image (see Geisler and Albrecht 1997; their Fig. 13 and related text). In fact, it is highly likely that there will be many fixations in between. We chose 300 ms to minimize interactions between stimuli and to maximize data collection.

Another way in which this experimental protocol does not approximate normal saccadic inspection of a visual scene stems from the fact that the eyes are paralyzed and thus there is a lack of coordination between the time course of the stimulus presentation protocol and any potential saccadic suppression signals, which might be generated by a corollary discharge from an oculomotor mechanism that produces saccadic eye movements. However, although saccadic suppression has been demonstrated psychophysically in the visual system as a whole (e.g., Campbell and Wurtz 1978; Riggs et al. 1974; Volkmann et al. 1968 1978; however, see Carpenter 1991; Greenhouse and Cohn 1991), physiological experiments in awake behaving monkeys suggest that saccadic suppression does not appear to be a factor at the level of the striate cortex (Bair and O'Keefe 1998; Gur and Snodderly 1997; Gur et al. 1997; Judge et al. 1980).

Quantitative index of the spatial frequency shift

As will be shown in the RESULTS, for most cells the latency of the response increases as a function of spatial frequency. This fact, coupled with the transient nature of the PSTH, produces a shift in the peak of the spatial frequency tuning through time from low frequencies to high frequencies. In general, this pattern of results is systematic and robust within the large-amplitude initial transient component of the temporal response profile for all of the cells that show a frequency shift, and thus the pattern of results can be described during this interval with a reliable, quantitative index: the peak of the spatial frequency response function during this interval. However, because there is a great deal of heterogeneity in the shapes of the temporal response profiles from cell to cell (see Albrecht et al. 2002; Figs. 4 and 5), the pattern of results following the initial transient varies from cell to cell and becomes more difficult to describe in a systematic, unified fashion. To be more specific, as the amplitude of the response decreases through time following the initial transient, the trends in the frequency shifts become more complex from cell to cell. Further, the responses are more variable within each cell because the amplitude is reduced and thus the signal-to-noise ratio is also reduced, as a consequence of the mean–variance relationship (for a review, see Geisler and Albrecht 1997). For example, for those cells that show a trough after the initial transient, the same trends that are observed during the initial transient are generally repeated in the rise to the sustained component. For those cells that do not show a prominent trough, and for those cells that show a small-amplitude sustained component, no systematic trends can be observed in the spatial frequency tuning following the initial transient, although presumably, the frequency and latency shifts are present during this interval as well. For these reasons, the quantitative index of the shift in the peak of the spatial frequency response function (i.e., the spatial frequency that produced the largest response) was restricted to the large-amplitude initial transient component of the temporal response profile, and the extent of this interval was determined in a qualitative fashion for each individual cell. Specifically, the interval began when the responses to the frequency with the shortest latency exceeded the spontaneous activity and the interval ended when the responses to the frequency with the longest latency reached a local minimum. For this sample of cells, the mean value of the duration of the initial transient component is 41.3 ms (SD = 17.7). It should be noted that the determination of the extent of the initial transient component, in addition to the specific range of spatial frequencies that was sampled, could affect the estimates of the magnitude of the frequency and latency shifts, given in Fig. 7 and Table 1, for this sample of neurons.

View larger version (19K): [in this window] [in a new window]   FIG. 4. The spatial frequency that produced the largest response is plotted as a function of time during the large-amplitude initial transient component of the PSTHs for the cells shown in Figs. 1, 2, 3. This was taken from the raw responses to the discrete set of frequencies that were presented. Peak spatial frequency is plotted every 3 ms in B and C, and every 4 ms in the other panels. Note that the preferred spatial frequency increases as a function of time.

View larger version (19K): [in this window] [in a new window]   FIG. 5. The latency from stimulus onset to the peak of the response (i.e., the peak of the PSTH) is plotted as a function of spatial frequency, for the cells shown in Figs. 1, 2, 3. Note that the latency increases as a function of spatial frequency.

View larger version (21K): [in this window] [in a new window]   FIG. 7. Histograms showing the spatial frequency shifts (A) and the latency shifts (B) for the entire sample of neurons. The black portion of each bar represents neurons recorded from cat area 17, and the white portion represents neurons recorded from monkey V1.

Given the inherent variability within the responses of visual cortex neurons, and the finite number of repeated presentations of any given stimulus, it is possible that observed patterns in the responses are a consequence of random variation as opposed to systematic variation. To determine the likelihood that the observed variation in the responses is a consequence of chance alone we assess the sampling distribution that would be expected by chance alone (see, e.g., Albrecht et al. 2002; Edgington and Bland 1993; Gallant et al. 1996; Müller et al. 2001; for a general discussion see Edgington 1995).

In this report, we use this statistical analysis to assess the degree to which PSTHs for a given cell are invariant across spatial frequency. Specifically, we compute the average PSTH and then scale and shift this average to fit the PSTH for each spatial frequency. To determine the amount of residual variation (following scaling and shifting the average PSTH) that would not be expected from chance alone, we performed the following analysis. First, the PSTH was averaged across all spatial frequencies, without any scaling. Second, the average PSTH was scaled and shifted to fit the measured responses at each spatial frequency. Third, the relationship between the mean rate of firing and the variance was measured. With these properties measured (the average PSTH, the optimized values for the scaling and shifting, and the mean–variance relationship), the following statistical analysis was performed. 1) Random variation was introduced around the scaled and shifted average PSTH for each spatial frequency (given the mean–variance relationship and the exact number of repeated presentations). 2) These "new-randomized" PSTHs were then averaged across spatial frequency. 3) The "new-randomized" average PSTH was scaled and shifted to fit the "new-randomized" PSTH for each spatial frequency. 4) The percentage of variation accounted for was computed. 5) Steps 1–4 were repeated on 10,000 occasions to obtain the sampling distribution for the estimate of the residual variation that would be expected based on the specific properties of each cell under the null hypothesis that there was no systematic variation in the shape of the PSTH across spatial frequency.

We also use this statistical analysis to assess whether the observed latency and frequency shifts are greater than what would be expected by chance alone. Specifically, for each individual neuron in the sample, the average PSTH was scaled and shifted to fit the measured responses at each spatial frequency; in so doing, the latency and frequency shifts were removed. Then, random variation was introduced around the scaled and shifted average PSTH for each spatial frequency (given the mean–variance relationship and the exact number of repeated presentations). Finally, the latency and frequency shifts were measured. This procedure was repeated on 10,000 occasions to obtain the sampling distributions for the latency and frequency shifts that would be expected based on the specific properties of each cell under the null hypothesis that there was no latency or frequency shift.

Detection performance as a function of the integration interval

Detection performance was measured using methods that are similar to those described in detail elsewhere (Albrecht et al. 2002; Geisler and Albrecht 1995, 1997; see also Müller et al. 2001; Tolhurst et al. 1983; for a general discussion within this context see Barlow 2003). Specifically, the signal-to-noise ratio was measured as a function of the interval of integration following response onset. The signal-to-noise ratio d' is equal to the absolute value of the difference in the means divided by the square root of the average variance. Detection accuracy is related to d' by the standard normal integral; when d' is equal to 1.0, this is equivalent to 75% correct detection in a 2 alternative forced choices task (see Green and Swets 1966). To measure this detection performance, the following analysis was performed, using PSTHs where the bin size was 1 ms (not 10 ms). First, the PSTH for the optimal spatial frequency was averaged across all spatial phases. Second, this average PSTH was scaled to match the response at the optimal spatial phase. (The average PSTH was used because it is the most reliable measure of the PSTH shape; see RESULTS.) Third, we measured the relationship between the response mean and the response variance for each integration interval; that is, the variance proportionality constant (see Geisler and Albrecht 1997). Fourth, using this scaled PSTH and the variance proportionality constant, we determined the mean and the SD of the responses within the scaled PSTH for temporal intervals that increased in 1-ms steps, starting at the point in time when the responses to the preferred spatial frequency began (i.e., when we determined that the response appeared to be greater than the spontaneous rate of firing). Finally, the signal-to-noise ratio was determined for each interval relative to the mean and SD of the spontaneous rate of firing (i.e., the base rate), measured over the same time interval. In so doing, this procedure assesses the ability of the cell to detect its optimal spatial frequency and phase in comparison to no stimulus.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
The goal of the present set of experiments was to measure the development of spatial frequency tuning of visual cortex neurons, on a fine time scale, over the course of a temporal interval comparable to the duration of a single fixation during normal saccadic inspection of a visual scene. With this goal in mind, we measured and analyzed the responses of 64 striate visual cortex neurons (39 neurons recorded from within area 17 of the cat visual cortex and 25 neurons recorded from within area V1 of the monkey visual cortex), as a function of spatial frequency. The stimuli were stationary grating patterns that were turned on for 200 ms and then turned off for 300 ms. Twelve different spatial frequencies were each presented in 8 different spatial phases; the PSTH at each spatial frequency was averaged across the repetitions at each spatial phase and then the PSTH was averaged across all of the 8 spatial phases.

PSTH as a function of spatial frequency

Figure 1 shows the responses of 3 neurons recorded from within the monkey visual cortex (A, C, and E), and 3 neurons recorded from within the cat visual cortex (B, D, and F), measured over the course of a 200-ms interval, for different spatial frequencies. Each curve shows the PSTH for a particular spatial frequency, evaluated at 1-ms intervals using a 10-ms running average (see METHODS). The systematic trends in the initial transient component are more easily seen in Fig. 2 responses of the same cells plotted on a restricted time scale.

View larger version (35K): [in this window] [in a new window]   FIG. 1. The responses of 6 neurons, plotted as a function of time and spatial frequency. Stimulus was a stationary grating presented for a brief interval, comparable to the duration of a single fixation (200 ms). The interstimulus interval (ISI) was 300 ms. Twelve different spatial frequencies were presented in 8 different spatial phases. Each unique spatial frequency and phase combination was presented on 40 separate occasions, in a random fashion. Responses at a given spatial frequency were averaged across the repetitions and the spatial phases. A: responses measured for a neuron recorded from within the monkey visual cortex: poststimulus time histograms (PSTHs) are plotted for 12 different spatial frequencies: 1.17, 1.24, 1.4, 1.56, 1.87, 2.33, 2.8, 3.11, 3.5, 3.73, 4.67, and 5.6 cycles/deg. Note that for each PSTH there is a large magnitude initial transient component followed by a brief trough and a subsequent return to a smaller-magnitude sustained component. Further, note that the shapes of the PSTHs are qualitatively similar across the different spatial frequencies; however, as spatial frequency changes, the PSTHs appear to scale vertically and shift horizontally. C and E: responses of 2 additional monkey neurons. The same basic pattern of results is evident. The spatial frequencies presented to the cell in C were: 0.67, 0.8, 1.0, 1.2, 1.33, 1.5, 1.6, 2.0, 2.4, 2.67, 3.0, and 4.0 cycles/deg. Spatial frequencies presented to the cell in E were: 0.6, 0.67, 0.8, 1.0, 1.07, 1.20, 1.33, 1.6, 2.0, 2.4, 2.67, and 3.0 cycles/deg. B, D, and F: responses for 3 neurons recorded from within the cat visual cortex. Once again, the shapes of the PSTHs for each cell appear to be relatively invariant and, although it is somewhat difficult to visualize, as spatial frequency changes, the PSTHs appear to scale vertically and shift horizontally. Spatial frequencies presented to the cell in B were: 0.14, 0.18, 0.2, 0.22, 0.26, 0.34, 0.36, 0.4, 0.44, 0.54, 0.66, and 0.80 cycles/deg. Spatial frequencies presented to the cell in D were: 0.34, 0.36, 0.4, 0.44, 0.54, 0.66, 0.8, 0.88, 1.0, 1.06, 1.34, and 1.6 cycles/deg. Spatial frequencies presented to the cell in F were: 0.66, 0.8, 0.88, 1.0, 1.06, 1.34, 1.6, 1.78, 2.0, 2.66, 3.2, and 4.0 cycles/deg.

View larger version (48K): [in this window] [in a new window]   FIG. 2. The large-amplitude initial transient component of the responses of the cells shown in Fig. 1. The time scale is restricted to the initial transient portion of the responses to facilitate visualization of the rapid dynamic changes that occur as a function of spatial frequency during this time frame. Each point is the average response that occurred within a running 10-ms time bin, plotted in 1-ms increments. Note that 1) the shapes of the PSTHs for each cell are relatively invariant as a function of spatial frequency, 2) the magnitude of the response changes as a function of spatial frequency, and 3) the latency of the response changes as a function of spatial frequency. As will be shown in Figs. 3, 4, and 7, this shift in the latency of the response is systematic: as the spatial frequency increases, the latency increases.

Spatial frequency tuning through time

Figure 3 shows 3-dimensional plots ("ribbon-plots") of the measured responses illustrated in Figs. 1 and 2. Spatial frequency is plotted along the x-axis, time is plotted along the y-axis, and response magnitude is plotted along the z-axis. Note that there is no smoothing of the 10-ms time bins in these plots. Thus for example, in Fig. 3A, there are 432 discrete measured average responses; that is, the average responses to 12 spatial frequencies measured over 36 time bins, with each bin being the average response to 320 repetitions of the same spatial frequency at that point in time. These plots illustrate the changes in spatial frequency tuning that occur over time. As can be seen, these frequency–time plots are tilted diagonally to the right, indicating that spatial frequency tuning shifts from low frequencies to high frequencies through time. Early in time, tuning is restricted to low frequencies and there are no responses to high frequencies. Later in time, tuning is primarily restricted to high frequencies because the responses to low frequencies are greatly diminished. This pattern of results corresponds to 1) the fact that the latency of the response to low spatial frequencies is shorter than the latency of the response to high frequencies and 2) the fact that the responses to all spatial frequencies are transient (cf. the transient responses illustrated in Figs. 1 and 2). For example, in Fig. 3A, at the early time intervals, the tuning is restricted to low frequencies because the responses to high frequencies are delayed and thus those responses are not yet contributing (cf. Fig. 2A); conversely, later in time, the tuning is primarily restricted to high frequencies because the responses to low frequencies have decayed and thus those responses are no longer contributing as strongly.

View larger version (95K): [in this window] [in a new window]   FIG. 3. Three-dimensional plots ("ribbon-plots") showing the magnitude of the response as a function of spatial frequency and time, for the cells shown in Figs. 1 and 2. Spatial frequency is plotted along the x-axis, time is plotted along the y-axis, and response magnitude (% maximum) is plotted along the z-axis. The time frame is restricted to the initial transient portion of the response to visualize the systematic, robust dynamic changes in the spatial frequency response function through time. Ribbons (double lines) connect the actual responses at each spatial frequency contained within the running 10-ms average, plotted in 1-ms increments, without any further smoothing. Note that the latency to the onset of the response increases as a function of spatial frequency. Further, because of the latency shift and the transient nature of the responses, as time proceeds, the responses to the high frequencies increase and the responses to the low frequencies decrease; thus the peak of the spatial frequency response function shifts, through time, from lower frequencies to higher frequencies.

Some cells show little or no latency shift

It is important to point out that the pattern of results illustrated in Figs. 1, 2, 3, 4, 5 is not representative of all of the neurons in the sample. Specifically, some of the neurons do not show any shift in the latency as a function of spatial frequency, whereas other neurons show only a minor shift. None of the neurons in this sample showed a statistically significant negative shift (see analysis given below). Figure 6 plots the responses of a neuron for which the latency of the response is essentially equivalent for the 12 different spatial frequencies (which in this case spanned a relatively large range: 2.6 octaves). As can be seen, for this neuron there is little, if any, latency or frequency shift. This observation holds for other cells as well.

View larger version (30K): [in this window] [in a new window]   FIG. 6. Responses as a function of time and spatial frequency for a neuron that shows little or no shift in latency as a function of spatial frequency. The time frame is restricted to the initial transient component. Each curve is for a different spatial frequency. Each point is the average response that occurred within the running 10-ms time bin, plotted in 1-ms increments.

The spatial frequency shift through time can be quantified for the entire sample of neurons by measuring the difference between the peak spatial frequency at the beginning and at the end of the large-amplitude initial transient component (see METHODS). Figure 7A plots this quantitative index of the frequency shift (expressed in terms of octaves) for the entire sample of neurons. As can be seen, the frequency shift is small for some cells and large for others, with a virtual continuum in between. Across the sample as a whole, the frequency shift is approximately one octave (mean = 0.91; median = 0.99). The breakdown of the frequency shift across animal type and cell type is given in Table 1; the asterisks indicate pairwise comparisons that are statistically significant at the 0.05 level of confidence (using a t-test). Using a randomization test (see METHODS), we determined that the frequency shifts for 20 of the 64 neurons fall within the 95% confidence interval expected from chance alone (i.e., for these 20 neurons the shifts are not statistically significant). Of these 20 neurons, 3 are monkey simple cells, 4 are monkey complex cells, 7 are cat simple cells, and 6 are cat complex cells. The median frequency shift for these cells is 0.09 octaves.

The latency shift can be quantified for the entire sample of neurons by measuring the difference between the latency for the lowest and highest spatial frequencies. Figure 7B plots this quantitative index of the latency shift for the entire sample of neurons. As can be seen, the latency shift is small for some cells and large for others, with a virtual continuum in between. Across the sample as a whole, the latency shift is approximately 30 ms (mean = 32.8 ms; median = 31.8 ms). The breakdown of the latency shift across animal type and cell type is given in Table 1. Using a randomization test (see METHODS), we determined that the latency shifts for 5 of the 64 neurons are not statistically significant. Of these 5 neurons, 1 is a monkey complex cell, 2 are cat simple cells, and 2 are cat complex cells. The median latency shift for these cells is 5.3 ms.

Average PSTH as a function of spatial frequency

In a previous investigation (Albrecht et al. 2002), we measured the responses of visual cortex neurons as a function of contrast using a stimulus protocol that was similar to the protocol used in this investigation (i.e., the stimuli were stationary gratings that were flashed on for 200 ms and off for 300 ms). In that study, we found that the shape of the temporal response profile (the PSTH) was relatively invariant as a function of contrast. Specifically, we found that, across the sample as a whole, approximately 95% of the variation in the responses as a function of contrast could be accounted for by simply shifting and scaling the average PSTH.

Visual inspection of the responses shown in Fig. 1 within this report suggests that the temporal response profiles for these cells are qualitatively similar across spatial frequency. To assess this observation quantitatively for all of the neurons, we calculated the percentage of variation that could be accounted for by simply scaling and shifting the average PSTH. Across the sample as a whole, approximately 90% of the variation can be accounted for by shifting and scaling the average temporal response profile (median value = 92%; mean value = 89%). To assess whether the residual variation is random, as opposed to systematic, we performed a statistical analysis (see METHODS). The results of this analysis reveal that for 7 of the 64 neurons the residual variation is indeed statistically significant at the 99% confidence level. In other words, for these 7 neurons (all monkey complex cells), there are systematic changes in the shapes of the temporal response profiles as a function of spatial frequency. However, it is important to note that the magnitude of this systematic residual variation is relatively small: both the mean and the median value are approximately 3%. At the 95% confidence level, 14 of the 64 neurons (9 monkey complex cells and 5 cat complex cells) show systematic deviations from the average temporal response profile; the magnitude of the systematic deviation for these 14 neurons is about 2%.

These results suggest that the spatial frequency shifts that can be seen in Fig. 3 are a consequence of 1) the latency shift as a function of spatial frequency, coupled with 2) the transient nature of the invariant temporal response profile. Further, these results have an important implication: If the latency shift is taken into consideration and the PSTHs are aligned, then the shape of the spatial frequency tuning function is relatively invariant (cf. Figs. 2, 3, 6, 9, and 10 in Albrecht et al. 2002).

View larger version (17K): [in this window] [in a new window]   FIG. 9. Detection performance as a function of the interval of integration for the cells shown in Figs. 1, 2, 3, 4, 5. Application of this approach to the present set of measurements is described in the METHODS (see also Albrecht et al. 2002; Geisler and Albrecht 1995, 1997; Müller et al. 2001; Tolhurst et al. 1983; for a general discussion within this context, see Barlow 2003). The interval of integration is plotted along the x-axis and the signal-to-noise ratio (d') is plotted along the y-axis. The integration intervals all start at a fixed point in time (29, 27, 39, 23, 29, and 27 ms after stimulus onset, A–F, respectively) and then progressively increase in 1-ms increments. Note that performance improves rapidly, reaching a d' = 1.0 within a very brief period of integration time, and then approaches a maximum value shortly thereafter.

View larger version (30K): [in this window] [in a new window]   FIG. 10. Demonstration that the temporal dynamics of the spatial frequency response function can potentially be produced by a simple feedforward model. In the top portion of the figure, the simple model is pictorially represented for 2 arbitrary striate cortex receptive field symmetries: even-symmetric on the left and odd-symmetric on the right. Note that the model does not require even-symmetry vs. odd-symmetry and could be extended to asymmetrical receptive fields (Field and Tolhurst 1986; Hamilton et al. 1989). Each circle represents the spatial location of the center mechanism of a lateral geniculate nucleus (LGN) input neuron. The small circles represent P cells and the large circles represent M cells; the plus signs represent ON-center cells and the minus signs represent OFF-center cells (cf. Reid and Alonso 1995). For ease of viewing, 1) the surrounds are not shown, and 2) the M and P cells are displaced vertically. However, in the model, the surrounds are present and the M and P inputs are overlapping. Thus for example, on a given row for the even-symmetric receptive field, there would be a total of 6 different LGN inputs: 3 M cells (1 ON-center and 2 OFF-center) and 3 P cells (1 ON-center and 2 OFF-center). Even though several rows are shown, there was no attempt to model the elongated properties that produce orientation selectivity, and thus there is nothing special about the vertical arrangement of the cells in the illustration. Details of the model are given in the APPENDIX. It is important to note that the fundamental parameters of this model that are responsible for producing the frequency and latency shifts are not allowed to vary: they are taken directly from physiological measurements in the published literature. In the bottom portion of the figure, A–F show the predictions of the model for the cell that is illustrated in Fig. 2C. Dots show the measured responses to 6 different spatial frequencies (0.8, 1.0, 1.2, 1.33, 1.5, and 1.6 cycles/deg, A–F, respectively); the solid curves show the predictions of the model. The arrow in each panel indicates the same point in time, which is the latency to the peak of the response for the lowest spatial frequency. As can be seen, the predictions are qualitatively consistent with the responses of this representative cell: 1) the temporal response profile is relatively shape-invariant and 2) as the spatial frequency increases, the latency to the peak of the response increases.

The complex changes in the spatial frequency response function that occur through time could potentially provide information to subsequent neurons beyond what is contained in the total spike count, summed over the duration of a single fixation. Whether this additional information is extracted by subsequent neurons depends on how the responses are integrated. Therefore it is worth considering the consequences of several different temporal integration strategies that could, in principle, be used by subsequent neurons.

One strategy would be to integrate the responses of visual cortex neurons over relatively brief temporal intervals. Consider, for example, the effect of integrating over specific 10-ms time intervals for the cell illustrated in Fig. 3A. When the running 10-ms integration interval is centered at 40 ms, as opposed to 60 ms, the spatial frequency tuning is considerably different; the ranges of spatial frequencies covered are almost nonoverlapping. Further, the bandwidths of these tuning functions for these brief integration intervals would be only a fraction of the bandwidth for a long integration interval. In general, as the integration intervals become longer, these changes in spatial frequency tuning become smaller (although the responses become more reliable; see detection analysis, below).

Next, consider the strategy of integrating (summing action potentials) over intervals that begin at a fixed point in time: at stimulus onset, or at some point after stimulus onset. Here we consider the effects of increasing the duration of the summation interval on spatial frequency tuning. We take the beginning of the intervals to be the time at which the shortest latency responses begin. Figure 8 plots the spatial frequency tuning for progressively longer integration intervals, increasing in 1-ms steps, for the 6 neurons illustrated in Fig. 1. (Note that in this analysis the bin size for the PSTHs is 1 ms, not 10 ms.) The smooth curve through the data points, at each sequential integration interval, shows the best fit of a function that provides a good description of the spatial frequency tuning of visual cortex neurons (an asymmetrical Gabor function; see Geisler and Albrecht 1995, 1997). For example, to be even more specific, consider the cell shown in Fig. 8A. The first set of data points and the fitted curve at the bottom of the panel, show the spatial frequency tuning when the responses are integrated over only 1 ms, starting at 40 ms (see figure caption); the second set of data points and fitted curve, just above the lowermost one, show the spatial tuning when the responses are integrated over a total of 2 ms (i.e., 40 and 41 ms); the integration interval for the next set includes 40, 41, and 42 ms, for a total of 3 ms, and so forth. As the integration interval increases, the spatial frequency tuning is initially dynamic for very short integration intervals and then becomes relatively stable for longer, but still brief, integration intervals. For the first integration interval, with a duration of 1 ms, the peak spatial frequency is 0.89 cycles/deg, whereas for the integration interval with a duration of 20 ms, the peak frequency is 2.11 cycles/deg. After this point in time, the peak frequency changes very little; for example, when the integration process is extended to 200 ms (not shown in the figure), the peak frequency is 2.26 cycles/deg.

View larger version (58K): [in this window] [in a new window]   FIG. 8. Spatial frequency response functions for different temporal integration intervals for the cells shown in Figs. 1, 2, 3, 4, 5. In this analysis, the bin size for the PSTHs is 1 ms (not 10 ms). For all of the curves, the integration intervals begin at a fixed point in time (following stimulus onset) that is approximately equal to the onset of the response for the lowest frequencies. For each sequential curve the integration interval is increased by 1 ms. For the cell shown in A there are 20 different integration intervals, each one starting at 40 ms; thus the longest integration is from 40 to 60 ms. Starting times for the cells in B–F are 36, 52, 40, 42, and 40 ms, respectively; the longest integration intervals in B–F are 14, 16, 26, 16, and 18 ms, respectively. Note that, although the spatial frequency response function is initially quite dynamic, it becomes relatively stable as the integration interval increases.

Effect of the integration interval on detection performance

For most visual cortex neurons, a large fraction of the action potentials to stationary stimuli occurs during the initial transient component of the PSTH (cf. Fig. 1; see also Albrecht et al. 2002; Müller et al. 1999, 2001; Tolhurst et al. 1980). Given this observation, it is possible that reliable information could be transmitted to subsequent neurons early within a single 200-ms fixation interval. However, the reliability of the transmitted information cannot be determined from the PSTH alone because both the mean and the variance of the response change throughout the time course of the 200-ms interval (Albrecht et al. 2002; Frazor 2002; Müller et al. 1999, 2001). To assess the reliability of the transmitted information, we measured detection performance as a function of the duration of the integration interval using a signal detection analysis (see METHODS, as well as Albrecht et al. 2002; Geisler and Albrecht 1995, 1997; see also Müller et al. 2001; Tolhurst et al. 1983; for a general discussion within this context see Barlow 2003).

Figure 9 plots the signal-to-noise ratio (d') as a function of the duration of the integration interval, for the cells shown in Figs. 1, 2, 3, 4, 5. As can be seen, the detection performance improves quite rapidly as the integration interval increases and becomes quite reliable in a brief period of time. The results of this analysis for the sample as a whole are summarized in Table 2. As can be seen, the 75% correct detection threshold (d'= 1.0) is achieved within a relatively brief period of time after response onset, approximately 25 ms (mean = 24.7, median = 18.7, SE = 2.9). Further, 90% of the maximum performance is achieved approximately 54 ms after response onset for the sample of monkey neurons and 109 ms after response onset for the sample of cat neurons. This difference between cats and monkeys could potentially be related to the fact that the duration of saccades is shorter in monkeys (25–55 ms; Fuchs 1967) than that in cats (50–150 ms; Evinger and Fuchs 1978). Because of the longer duration of the saccades in cats, they are afforded a longer interval in which to assess the information contained within the previous fixation.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION APPENDIX ACKNOWLEDGMENTS REFERENCES

 
The primary goal of the present set of experiments was to measure the development of spatial frequency tuning in visual cortex neurons through a temporal interval that is comparable to the duration of a single fixation during natural viewing. To this end, we presented stationary gratings for brief intervals (200 ms) and recorded the action potentials on a fine time scale. We found that, for the majority of neurons, the spatial frequency tuning is dynamic through this temporal interval. Specifically, for most cells, the latency of the response increases as the spatial frequency increases (e.g., Figs. 1, 2, 3, 5, 7B). Thus as time proceeds, the spatial frequency tuning shifts from lower frequencies to higher frequencies (e.g., Figs. 3, 4, and 7A). In the following, we evaluate several potential functional and structural implications of these dynamic changes. We also discuss the results in relationship to other investigations of the dynamics of spatial frequency tuning. Finally, we propose a simple feedforward model, similar to the one proposed by Hubel and Wiesel (1962), that incorporates known differences in the properties of LGN cells.

Motion selectivity in depth

When an object approaches, the spatial frequency content of its retinal image will shift over time from higher frequencies toward lower frequencies. Thus if the latency of the response of a given neuron increases as a function of spatial frequency, then the neuron would produce a larger response to approaching stimuli than to either stationary or receding stimuli because the excitation from the higher frequencies and lower frequencies would overlap in time, for the approaching object. Given that the latency of the response for most visual cortex neurons does, in fact, increase as a function of spatial frequency (see Fig. 7B), one hypothesis to consider is that cortical neurons are selective for motion in depth (i.e., behave like looming detectors or approach detectors; Akase et al. 1998; Cynader and Regan 1978; Poggio and Fischer 1977; Toyama and Kozasa 1982; Wang and Frost 1992; for a review of the related literature see Sherk and Fowler 2001). For this hypothesis to be viable, the rates of frequency shift through time would have to match the rates of motion in depth that would be useful within a behavioral context. However, the mean rate of change for this sample of neurons is approximately 0.05 octaves/ms. This rate of change is too fast to afford an adaptive response because it is not reached until an object is so close to the eye that there would be <50 ms until impact. Nonetheless, several cells in the sample have rates of frequency shift that are substantially less than the mean. The cell with the slowest rate of change could respond to an approaching object when the time to impact is about 140 ms. This cell could possibly contribute to some behaviors such as the menace reflex, which occurs on the order of 80–100 ms (LeGrand 1957; Westheimer 1968