| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Department of Neuroscience, Brown University, Providence, Rhode Island
Submitted 30 May 2007; accepted in final form 5 December 2007
 |
ABSTRACT |
|---|
|
|
|
INTRODUCTION |
|---|
|
However, the reductionist approach may provide a description of V1 that is incomplete for more natural situations. Natural scenes differ from typical stimuli not only in their complexity and scale but also statistical properties (Ruderman 1994
; Ruderman and Bialek 1994; Simoncelli and Olshausen 2001). Moreover, in natural vision, visual stimuli usually enter RFs via saccades, which have been shown to distort perception (Carpenter 1988; Latour 1962; Ross et al. 2001; Thiele et al. 2002; Volkmann et al. 1968). There is a natural cadence to fixations and saccades that establishes a temporal relationship between stimuli that is abolished in traditional V1 experiments.
Studies examining natural vision and the significance of experimental reductions go back decades, but there has been increasing interest in recent years (Kayser et al. 2004). Various studies have explored different issues related to natural viewing such as receptive field structure, preferred stimuli, and contextual modulation. To a first approximation, neuronal responses in more natural situations are similar to those obtained in reduced paradigms. For example, it has been shown that reverse correlation with natural images or movies yields estimates of receptive field structure and preferred orientation similar to estimates obtained with simple flashed stimuli (Ringach et al. 2002; Smyth et al. 2003). With an analysis of spike bursts, it is possible to estimate RF location and orientation preference even during free viewing (Livingstone et al. 1996). Regarding saccadic eye movements, it has been reported that stimuli brought into a RF via saccade evoke a response similar to the same stimulus swept into the RF or flashed to a fixating animal (DiCarlo and Maunsell 2000; Gawne and Martin 2002; Richmond et al. 1999; Wurtz 1969).
However, there are compelling indications that visual responses in more natural situations are different from those in reduced paradigms. David et al. (2004) found that a RF model built on responses to natural stimuli is better able to predict responses to novel natural stimuli than a RF model built on responses to gratings, suggesting that the inherent statistics of natural scenes are important. Kayser et al. (2003) found that natural images evoke local field potentials and multiunit activity that are qualitatively different from responses to gratings. Extra-RF interactions are also different with natural stimuli, again reflecting the importance of higher-order natural statistics (Guo et al. 2005). Vinje and Gallant (2000) found that extra-RF input with natural images alters the selectivity and sparseness of V1 responses. Concerning eye movements, saccades have been found to alter basic receptive field properties in the LGN (Reppas et al. 2002), V3 (Nakamura and Colby 2002), and V4 (Tolias et al. 2001), as well as in parietal areas (Duhamel et al. 1992; Kusonoki et al. 2000).
The present study used a progression of four paradigms that intermix and contrast the effects of simple versus natural stimuli and stimuli brought to receptive fields by saccade versus flash. Rather than adopt a free-viewing approach, the principal experiments used a technique that preserved key elements of natural vision while allowing tighter control over stimulus and behavior. This permitted us to average responses over multiple stimulus presentations, which is impossible with free viewing and allowed us to directly compare responses to identical stimuli in reduced and more natural situations. In light of previous research, we did not construct RF kernels with natural stimuli, characterize optimal stimuli or stimulus tuning, or study responses associated with saccadic suppression. Our focus was entirely on comparing response patterns in a standard laboratory paradigm (fixating animal with flashed stimuli) to responses with stimuli as identical as possible in more natural situations including natural context and saccades. We find that both natural scenes and saccades have significant effects on neural responses. Consistent with previous studies of surround suppression, responses to stimuli embedded in a natural scene were typically less than when the stimulus was presented in isolation. Much more surprising, responses were significantly enhanced when stimuli entered RFs with saccades compared with when they appeared in RFs during steady fixation. Together, natural scenes and saccades sculpted responses in a way that that would be difficult or impossible to predict from traditional V1 experiments. The different response magnitudes and temporal response patterns suggest that the cortical network activated in natural vision is different from that in more reduced paradigms. A brief report of these results appeared previously in abstract form (MacEvoy et al. 2002).
|
|
METHODS |
|---|
|
Two adult female rhesus monkeys (Macaca mulatta) weighing 5.0 kg (monkey HZ) and 6.0 kg (monkey PN) served as subjects. All procedures conformed to National Institutes of Health guidelines and were approved by the Brown University Institutional Animal Care and Use Committee. In aseptic surgery, each was implanted with a head post and a recording chamber centered over area V1. After recovery, animals were trained to sit comfortably in a primate chair (Crist Instruments, Hagerstown, MD) with their heads fixed. Once the animals became accustomed to the chair and headpost, they were trained to saccade to and hold fixation on visual cues. Prior to participating in this study, monkey PN had served as a subject in unrelated experiments requiring a simple fixation task.
Extracellular physiology
Single-unit recordings in V1 were made with epoxy-coated tungsten electrodes (FHC, Bowdoinham, ME). Electrode potentials were amplified, band-pass filtered between 300 and 3,000 Hz, and observed with an oscilloscope and audio monitor. Single unit activity was isolated by template matching (Alpha Omega Engineering, Nazareth Illit, Israel). During experiments, spike times were recorded by a computer running Tempo data collection and experiment management software (Reflective Computing, St. Louis, MO), which was also used to monitor behavior and control stimulus presentation. Eye position was monitored by an infrared pupil tracking system (ISCAN Corp., Burlington, MA) with 60Hz temporal resolution. After calibration, the precision of the eye location measurements was 0.3°. Once single-unit activity was isolated, RF boundaries were mapped using manually-controlled light bars presented on a monitor. RFs were manually defined as regions from which an optimally oriented bar evoked action potentials.
Visual stimuli
All experimental stimuli were presented on a computer monitor at 1024 by 768 pixel resolution, positioned 40 cm from the seated animal and subtending 52.5° by 38.4°. The principal stimulus was a white bar, optimized for orientation and confined to the RF of the cell under study. Bar width was set for each neuron to evoke a maximal response. We did not employ quantitative tests to differentiate simple and complex cells, but found that a positive-contrast (white) bar was sufficient to elicit strong responses from all neurons. In all stimulus conditions, the bar appeared on a small gray square at least twice the diameter of the RF, referred to as the "buffer". The buffer had the same mean luminance as the background, and ensured that RF stimulation was identical across all conditions.
The bar/buffer combination was presented in two different visual contexts. In the "natural background" context, the bar and buffer were superimposed on a grayscale natural scene image drawn from a published database (van Hateren and van der Schaaf 1998). All neurons were studied with the same natural image background, though additional natural images were used on some cells. The position of RFs on the natural image varied from neuron to neuron. In the "gray background" context, the bar and buffer were presented on a gray screen set to the same mean luminance as the natural image. Since this was the value also selected for the luminance of the buffer, in these conditions the bar was the sole discernible stimulus present. Schematic diagrams of stimuli are shown in Fig. 1 A.
|
For each visual context, trials proceeded in one of two behavioral modes. (Fig. 1B). In "flash" mode, a trial began with the appearance of the natural or gray background and the superimposed buffer, but without the bar stimulus itself. After a short delay, an initial fixation spot appeared at one of two possible locations on the screen, one 5° to the left of center and the other 5° to the right. The animal was required to visually acquire the fixation spot within 3 s (though both animals usually did so within a few hundred milliseconds), and maintain fixation within 0.5° of the spot for 300 ms. The first fixation spot was then extinguished and a new fixation spot appeared at the center of the screen. If the monkey failed to move its gaze to within 0.5° of the new fixation location within 300 ms, the trial was aborted. With acquisition of the central fixation spot, the RF of the neuron under study fell within the buffer patch. After the animal maintained fixation at the central location for 500 ms, the bar stimulus was flashed on in the RF, and the monkey was required to maintain fixation for an additional 1.5 s, after which the stimulus was extinguished and a liquid reward was given. Spike times were recorded for a period beginning 0.5 –3 s prior to bar stimulus presentation and extending to the end of the trial.
"Saccade" mode conditions also began with the natural or gray background and superimposed buffer, but with the bar stimulus already inside the buffer. A fixation spot appeared at one of the same two off-center locations used in flash conditions and the monkey was required to acquire this first fixation spot and hold it for 300 ms. A second fixation point appeared at the central location and the monkey had to acquire it within 300 ms. In saccade mode, the resulting saccade carried the RF onto the bar stimulus. The monkey subsequently had to keep its gaze on the fixation spot for 1.5 s to receive a liquid reward. Stimulus onset in saccade conditions refers to the time at which the animal's gaze entered (and stayed within) the central fixation window. Since the fixation window, at 1° square, was smaller than some RFs, it was possible that the bar stimulus could have entered RFs slightly before the eyes entered the fixation window. However, based on our measurements of eye velocity and RF size, this error is <3 ms.
Our stimulus set was thus composed of four principal conditions: natural background flash, natural background saccade, gray background flash, and gray background saccade. All stimuli were randomly interleaved.
Data analysis
Spike times were collected with 1 ms temporal resolution. Peristimulus time histograms (PSTHs) were computed off-line and data rebinned at 10 ms. To compute population response histograms, we normalized single-cell responses by scanning each cell's PSTHs for the bin with the single highest spike count across all conditions. We scaled every other bin to the cell maximum and generated population histograms by adding corresponding bins from all cells. The resulting set of population response functions were then renormalized to the maximum response across all conditions. For presentation only, we converted PSTHs to spike density functions smoothed by a Savitzky-Golay least squares filter using a second order polynomial fit to spike rates within a 50 ms window. Standard errors for spike density functions were computed across smoothed single-trial responses.
All statistical tests are based on raw trial-by-trial spike counts during the first 500ms following stimulus onset.
Calibrating response latency in saccade mode
Since data collection in saccade mode trials was triggered by acquisition of the central fixation point, any delay in our eye tracking system could result in artificially short response latencies. To calibrate the system, we passed a square-wave voltage signal from a function generator to an oscilloscope in X-Y mode to produce a bright dot that jumped between two positions. This artificial eye position signal was detected by the eye tracking equipment. To quantify the delay in the eye tracking system, we compared the timing of the simulated saccades to the output of the function generator. The average delay was 27.6 ms. To compensate for this delay, all saccade mode PSTHs were shifted by 28 ms prior to performing any analysis. To be sure that our findings were not artifacts of the shift procedure, we always recomputed our statistical analysis with unshifted data. Doing so actually increased the frequency of the effects described in Results. Thus the effect of any unnecessary or excessive time-shifting would have been to decrease our estimate of the frequency of significant response differences, not inflate it.
|
|
RESULTS |
|---|
|
Response statistics during free-viewing
As a preliminary step, we wished to compare visual responses recorded during free-viewing of a natural scene image with responses to a more conventional lab stimulus (i.e., an isolated bar stimulus set to a cell's preferred orientation and flashed in the RF). To quantify free-viewing responses we recorded spike times and eye position during the 1 s intertrial interval between each fixation trial. During these intervals, the stimulus monitor was preloaded with the background, either natural or gray, for the succeeding fixation trial. We computed firing rates within a sliding 50 ms window stepped in 10 ms increments through the first second of every free-viewing epoch in which a natural image was preloaded, and through all bar presentation periods. Although the monkeys were not explicitly required to restrict their gaze to the monitor during this period, the large subtense of the screen combined with the monkeys' desire to locate the forthcoming fixation spot likely produced stimulation that was dominated by the natural image. We excluded from this analysis any periods when the eye position signal was lost due to closed eyelids, although in practice we rarely encountered closures longer than blinks.
Figure 2 A plots the distributions of firing rates computed from all possible window locations for one cell. For well over half the window positions in free-viewing epochs, this cell fired at rates <20 spikes per second, which corresponds to at most 1 spike every 50 ms. In contrast, the cell responded at over 40 spikes per second in most response epochs during presentation of the bar stimulus. Mean firing rates across all window positions during free-viewing and bar presentation were 15.7 and 43.6 spikes per second, respectively. Although dramatic, this difference is not surprising, since stimuli entering the cell's RF during free-viewing were unlikely to match the cell's preferences. Despite the drastically different means, the range of responses was similar: the peak response in any 50 ms window during free-viewing was 140 spikes per second, and the response to the optimal bar peaked at 160. For this cell, responses to a natural stimulus fell within the range of responses elicited by an optimized stimulus. The point we take to be noteworthy from these data are that natural stimuli could drive this V1 neuron, and the others we analyzed, nearly as much as an optimized laboratory stimulus, though this was a rare event.
|
While neural responses are not usually formally characterized by peak firing rates, it is not uncommon to quickly read a PSTH by picking off the maximum rate to assess the robustness of response (despite the common knowledge that peak rates are highly sensitive to PSTH binning). Our results show, not surprisingly, that the high peak firing rates commonly recorded with optimal stimuli do not accurately describe the normal response properties of a neuron in natural situations. The data in Fig. 2D underscore this point by comparing the peak response to an optimal stimulus with the average free-viewing response. The difference between the distributions for these data are striking. Clearly, firing rates evoked by an optimal stimulus provide an incomplete and/or misleading description of cells' activity during conditions similar to natural vision.
Effect of background on bar responses
To allow averaging across trials, the remainder of the results looks at the effects of background type and saccades in guided saccade paradigms rather than free-viewing. Fig. 3 A (top panel) shows the responses of a single neuron to a bar flashed in its RF on either a natural image or gray background. For this cell, the bar elicited a markedly more robust response on the gray background than on the natural background, even with identical RF stimulation and overall stimulus luminance. The bottom panel of Fig. 3A illustrates the effect of background for a bar brought into the RF with a saccade, for the same cell. Once again, the cell had a higher firing rate with the gray background, indicating that the effect of background persisted when the bar was carried into the RF with an eye movement as it would be during natural vision. For both pairs of responses in Fig. 3A, the effect of background only became apparent following the response peak, roughly 100 ms after stimulus onset. (These responses and all hereafter, unless otherwise noted, represent combined data from trials beginning with saccades from the left and right initial fixation points.)
|
Overall, the natural image background appeared to exert a suppressive effect on responses to a bar in the RF. This is consistent with previous studies of surround suppression reported in V1, in which high-contrast extra-RF stimuli have been shown to exert greater suppression than equiluminant homogeneous fields (Cavanaugh et al. 2002; Jones et al. 2001; Levitt and Lund 1997; MacEvoy et al. 1998; Sceniak et al. 1999, 2001; Sillito et al. 1995). Most previous studies were conducted with simple stimuli such as lines or gratings in the surround and it is important to note that we cannot say our results depend on the "naturalness" of the stimuli we used, though data from Guo et al. (2005) suggest that the statistics of natural images play a role. In any event, it appears that the luminance contrast in the natural image background produced greater surround suppression despite the same mean luminance in natural and gray backgrounds. With this in mind, for a subset of neurons we recorded responses to a modified natural background condition that used a buffer patch set at four times the RF diameter. Figure 4 shows average responses to this stimulus, along with responses evoked with the standard small buffer and the full gray screen. Not surprisingly, we found the average response in the large buffer condition was intermediate between the responses in the small buffer and full screen gray conditions.
|
Next, we analyzed differences in responses to the bar stimulus resulting from the mode with which it entered the RF: either as a flash during fixation or via saccade. Figure 5 A compares the responses of one neuron to identical bars differing only in their mode of RF entry. For a bar and buffer embedded in the natural background (Fig. 5A, top), the cell responded with a markedly higher response after a saccade than during fixation. When the bar was on the gray background (Fig. 5A, bottom), the saccade stimulus likewise elicited a higher response, though to a lesser extent.
|
Interaction between the effects of context and mode
On both natural and gray backgrounds, we find that responses in saccade trials are greater than on flash trials. In the normalized population average, the response enhancement associated with saccades was greater with the natural background than with the gray background. Understanding the basis for this difference is critical. Because we measured it as a percentage of total response, it is possible that the apparent difference in enhancement could be the misleading result of a constant saccade-related effect combined linearly with different response rates on the two different backgrounds. Alternatively, this difference might indicate a nonlinear interaction between natural scenes and saccades that causes the saccade enhancement to be larger in absolute terms (i.e., not just percentage of response) with the natural background. If this were the case, it would indicate that responses with natural scenes and saccades could not be easily predicted from the independent contributions of these factors.
To make a more direct comparison between the effects of saccades on the natural and gray backgrounds, we performed a two-way ANOVA on firing rates evoked by all four principal stimulus conditions: natural background flash, natural background saccade, gray background flash, and gray background saccade. Consistent with our previous analysis, we found that both background type and behavioral mode had highly significant effects on firing rates (P < 0.0001) at the population level (n = 59). Moreover, there was a highly significant interaction between the effects of background type and behavioral mode (P = 0.0005). This nonlinearity corresponded to a significantly greater difference, on average, between flash and saccade conditions when stimuli appeared on the natural background versus the gray background (as in Fig. 5). In other words, the interaction between background context and mode is a significant and real effect, rather than an artifact of the analysis. Further, the existence of an interaction means that independent analyses of the effects of background type and behavioral mode could not have predicted the population response under conditions mimicking natural vision.
Figure 6 illustrates in another way the interaction between background (surround suppression) and presentation mode. The lower and upper curves in this figure show the average response of 19 neurons in the flash condition with the standard small buffer patch on a natural scene background (dashed line) and a full-screen uniform gray background (heavy solid line). The light solid trace shows the response with the standard small buffer patch on a natural background in the saccade condition. The average response in the saccade condition was, like the response in the flashed condition with the large buffer (Fig. 4), intermediate between the other curves. Thus the effect of introducing the stimulus into the RF via saccade was a reduction in surround suppression similar to the reduction observed when the suppressive stimulus is removed from the immediate vicinity of the RF (in the flash conditions). This suggests that the impact of surround suppression on V1 activity during natural vision is not as great as would be predicted by experiments using conventional fixation paradigms.
|
Possible causes of mode-dependent response differences
The most surprising finding in our data are that responses to identical stimuli are often significantly different when a stimulus is flashed or enters the RF via saccade. Because this finding has broad and important implications for the interpretation of V1 recordings, we investigated possible causes of the mode difference.
FIXATIONAL EYE MOVEMENTS.
Previous studies of the effects of fixational eye movements have yielded conflicting reports of response enhancement and suppression (Leopold and Logothetis 1998; Martinez-Conde et al. 2000; Snodderly et al. 2001). Conceivably, differences in fixational eye movements (too small to abort a trial) in flash and saccade conditions might underlie the higher saccade condition responses. A reason to speculate that fixation differences could occur has to do with timing—in saccade conditions, stimulus onset coincided with the end of a large saccade (or sometimes a corrective saccade), while in flash conditions stimulus onset followed the end of a large saccade by several hundred milliseconds of steady fixation. Indeed, we found differences in eye speeds averaged across trials from all recording sessions (Fig. 7 A). The speed trace for saccade conditions shows both the large saccade that brings the RF to the bar stimulus at time 0, as well as a second peak some 200 ms later, corresponding to corrective and fixation saccades that keep the eye within the fixation window.
|
STIMULUS MOTION. In saccade conditions, the stimulus bar quickly moved into the RF as the final fixation point was acquired. There was no similar bar movement in flash conditions because steady fixation was required before the bar appeared. Although the duration of the bar motion in saccade conditions was only a few milliseconds, corresponding to the time it took the eyes to traverse a distance half the RF size, it is conceivable that this movement causes the higher responses in saccade conditions. We considered this unlikely since differences in saccade and flash responses were not typically evident in the earliest portion of the response, at the time closest to the short period of stimulus motion. Nonetheless, we analyzed our data for evidence of a contribution of bar movement to saccade condition responses.
Since V1 neurons are most sensitive to motion orthogonal to their preferred orientation, we predicted that if significantly higher responses in saccade conditions were a result of bar motion, they should occur more frequently among neurons with preferred orientations orthogonal to the axis of eye movement. Because all saccades we directed were horizontal, this means that cells with significant mode differences should have had preferred orientations biased toward vertical, while cells without significant differences should have tended to have preferred orientations closer to horizontal. This is not what we found. After expressing each cell's preferred orientation relative to horizontal, we found that the mean preferred orientations for cells with and without significant mode differences with the natural background were nearly identical (49° and 52° from horizontal, respectively). Furthermore, both groups contained neurons with preferred orientations spanning the entire range between 0° and 90°, as illustrated in Fig. 8. Therefore it does not appear that bar motion during saccades can account for the response difference in flash and saccade conditions.
|
reported that the response to an optimal bar stimulus present during a saccade combined nonlinearly with the response to a postsaccadic bar. In the fixation condition, there was a delay between acquisition of the final fixation point and presentation of the bar stimulus, which might have minimized any effect of prior RF stimulation. A second possibility is that activity is not "carried over" from one fixation to the next, but instead some type of adaptation or gain adjustment from the prior fixation affects responsivity on the final fixation. This latter effect we refer to as "response history". Residual activity is examined in this section and response history is discussed generally and in the context of mode differences in the following section. To measure residual activity preceding the bar stimulus, we analyzed cells' responses in flash conditions during the period immediately following the saccade to the central fixation point, but before the bar appeared (dark shaded region in top of Fig. 9 A). RF stimulation was essentially identical for saccade and flash conditions prior to that point since the previous fixation points and saccade trajectories were the same for both. Therefore activity recorded in flash conditions immediately following the last saccade reflected residual activity that might affect saccade condition responses to the bar stimulus. Figure 9B plots each cell's response to the bar on the natural background in the saccade condition against residual activity from the corresponding fixation condition, both normalized to the cell's response to the bar flashed on the gray background. Linear regression through these data yields a line with a small but significant positive slope (P < 0.0001) and an R2 value of 0.12. Residual activity therefore appears to make some contribution to higher saccade responses, but is clearly not the determining factor. Moreover, residual activity cannot explain higher saccade responses observed with the gray background, when no comparable pre- or trans-saccadic stimuli were present.
|
In saccade conditions, the RF covered a portion of the natural image background until the final saccade carried the bar into the RF. On the other hand, in flash conditions the RF rested on the blank gray buffer patch for several hundred milliseconds prior to bar onset. Conceivably, differences in saccade and flash conditions could have been related to differing response histories produced by these prebar stimuli.
Before examining what contribution response history made to flash versus saccade mode differences, we first explored the role that response history played in determining responses in general. Gawne and Woods (2003) previous study with Walsh patterns suggests that responses interact across saccades. Recall that in our experiments each saccade condition was composed of two "sub-conditions": one with a saccade from the left, and another with a mirror-image saccade from the right. Since the prebar contents of RFs differed dramatically for each saccade direction (for examples see insets, Fig. 10 A), and often elicited different responses, by comparing the left and right saccade "sub-conditions" we could evaluate the contribution of history to responses elicited by identical postsaccade stimuli. For each cell, we computed a response margin (RM) between left and right saccade sub-conditions by dividing the difference in average bar responses by their sum. (Responses were taken from the first 500 ms that the bar was in the RF.) For each cell, the RM is a value between –1 and 1, corresponding to no bar response in the left saccade sub-condition and no bar response in the right saccade sub-condition, respectively. A value of 0 denotes identical average responses. Response margins so derived are referred to as "bar" RMs. Next we computed RMs from cells' responses during the 500 ms period immediately preceding entry of the bar into RFs. These are referred to as "prebar" RMs. We also computed RM values for the flash conditions, which were similarly composed of two left and right sub-conditions—see METHODS for full condition descriptions. For these, time 0 is the onset of the bar, and the prebar epoch corresponds to the 500 ms before stimulus onset during which the RF was covered by the gray buffer.
|
To understand whether this adaptation effect could explain differences between behavioral modes, we next computed RMs for each cell from differences between saccade and flash conditions. Resulting prebar and bar RMs are plotted in Fig. 10B. Two points are plotted for each neuron, one for each saccade direction. If adaptation were to account for the tendency of neurons to have higher saccade responses, the distribution of data in 10B should show two features. First, there should be a negative dependency of bar RMs on prebar RMs, just as in 10A; this is clearly the case, evinced by the significantly negative slope of the regression line (P = 0.0002). More importantly, however, prebar RMs should fall predominantly to the left of zero on the x-axis, denoting lower prebar responses in saccade conditions relative to flash conditions. In fact, we find that the opposite is true: the median prebar RM was 0.2, indicating that a majority of neurons had higher responses during prebar epochs in saccade conditions. This is inconsistent with an adaptation-based explanation for our populations tendency toward higher responses in saccade conditions. To be sure, adaptation may have contributed to higher bar responses in saccade conditions for some neurons (and lower saccade responses among others). However, it appears that saccades produce an additional form of response enhancement that is independent of response history. This independence is captured by the significantly elevated (P < 0.0001) y-intercept of the regression line in 10B, which illustrates that even among neurons with no difference in prebar activity, saccade conditions tended to produce higher responses during the bar epoch.
|
|
DISCUSSION |
|---|
|
Influence of natural context and eye movements on V1 responses
We find that during free-viewing, the mean and peak responses are lower than during stimulation with an optimized bar stimulus. However, this difference is much smaller than one might guess given the uncontrolled contents of RFs during free-viewing; free-viewing can elicit quite strong responses. That said, an important comparison is between the peak response to an optimal bar and the mean free-viewing response. In the large literature on V1, data are typically presented in post stimulus time histograms and there is a tendency to focus on the peak of the response as indicative of a cell's properties. The fact that the mean free-viewing response is drastically less than this peak serves as a reality check for what neurons are doing most of the time outside the laboratory.
In more controlled conditions, we found that both background type and viewing mode significantly influenced V1 responses to a standard stimulus. Lower responses to the bar embedded in the natural scene background are consistent with previous reports of response suppression from stimuli beyond classical RF boundaries both with reduced stimuli (Blakemore and Tobin 1972; Cavanaugh et al. 2002; DeAngelis et al. 1994; Fries et al. 1977; Jones et al. 2001; Kastner et al. 1997; Knierim and van Essen 1992; Levitt and Lund 1997; MacEvoy et al. 1998; Muller et al. 2003; Nelson and Frost 1978; Rossi et al. 1996; Sceniak et al. 1999; Sengpiel et al. 1997; Series et al. 2003; Sillito et al. 1995; Toth et al. 1996; Walker et al. 1999) or with natural images (Gallant et al. 1998; Guo et al. 2005; Vinje and Gallant 2000, 2002). Neglecting the role of eye movements, natural backgrounds suppress V1 activity, even when the luminance of a comparative gray background is equated to the mean of the natural background.
More surprising were the higher responses we observed in saccade conditions, an effect which has not been previously reported. Motivated by perceptual saccadic suppression (Ross et al. 2001), several studies have examined the effects of saccades on V1 responses to stimuli in RFs during eye movements (Battaglini et al. 1986; Judge et al. 1980; Thiele et al. 2002; Yu and Lee 2000). Judge et al. (1980) employed an approach similar to ours to examine the capacity of pre- and postsaccadic stimuli to modify responses to trans-saccadic stimuli. Consistent with their focus on trans-saccadic stimuli, however, Judge et al. made no comparison between responses to postsaccadic stimuli flashed in RFs versus brought into RFs via saccade, nor any evaluation of the effects of saccades by themselves (i.e., absent trans-saccadic stimuli) on responses to subsequent stationary stimuli. In contrast, our study is among the few (Gallant et al. 1998; Gawne and Martin 2002; Livingstone et al. 1996; Wurtz 1969) to directly compare the effects of saccades on responses to subsequent stimuli viewed during fixation (i.e., the period of normal visual perception). At first glance, there appears to be striking disagreement between various studies as some report different responses with free viewing and fixation (Gallant et al. 1998; Livingstone et al. 1996) and others report generally no difference in response between stimuli coming into RFs via saccade or flash (Gawne and Martin 2002; Wurtz 1969). The reason the latter studies did not report consistent saccade effects is probably to a degree a consequence of data analysis. Wurtz' early study was largely qualitative. Gawne and Martin did quantify their results, but the temporal analysis focused on response latency. Richmond et al. (1999) show the response of a neuron which appears qualitatively different with flash and saccade stimulus presentations, but the focus of the analysis was orientation tuning and response variability. Livingstone et al. (1996) reported responses of V1 neurons to bars and circles during free viewing, but did not provide any comparison with responses to similar stimuli when presented traditionally. Gallant et al. (1998) compared responses to natural images during free-viewing with image patches presented during fixation. However, in that study, responses were averaged over multiple different fixations, and side-by-side comparisons of responses to identical stimuli with and without eye movements were not made. Our distinct experimental and analysis approach complements these previous studies.
Several factors may have contributed to the higher responses we found in saccade conditions. We analyzed possible roles of fixational saccades, residual activity carried into the final fixation, and movement of the bar stimulus across the RF in saccade trials, all of which are present during normal visual behavior outside the laboratory. While none of these factors appears sufficient to explain higher responses in saccade conditions, it is possible that their effects may have combined to produce the significant response differences we observed. Testing this conjecture merits further study. Regardless of the source of the response differences we observed, our data demonstrate that responses under natural vision conditions are not fully predicted by results of traditional experiments, especially when eye movements come into play. The effects of background type and behavioral mode combined in a complex fashion to produce responses that could not have been predicted by measuring each in isolation. The difference in response between flash and saccade conditions was significantly greater with a natural background than with a uniform gray background; looked at differently, the same data show that the impact of a suppressive stimulus is significantly smaller when stimuli enter RFs via saccades. Whether eye movements per se are necessary to produce the enhancement we saw, or whether the same effects could have resulted from artificial stimulus motion mimicking saccades is a topic we are presently investigating. Recent work in cat visual cortex has shown that saccades induce long-lasting changes in the frequency spectrum of neuronal firing that differ from those produced by comparable stimulus changes without eye movements (Moeller et al. 2007).
Our experimental conditions were similar to those employed by Kusunoki et al. (2000) in their study of the macaque lateral intraparietal area (LIP). That study, like ours, compared responses of neurons to stimuli which appeared suddenly during fixation to those that entered RFs via saccades. Kusunoki et al. found that responses in LIP neurons were greater when stimuli were flashed in RFs than when they entered with a saccade, which the authors concluded was a result of the greater salience of flashed stimuli. Although we found the opposite, it is possible that postsaccadic enhancement is the signature of the equally important efforts of the visual system to maintain a stable representation of the visual world across eye movements. Just as saccadic suppression likely assists this process by filtering extraneous transient stimuli, postsaccadic enhancement may assist it by emphasizing sources of stable stimulation.
Response history effects on V1 activity
We found that there is a significant effect of response history on V1 responses. In our experiment, high prebar activity was generally followed by lower subsequent bar responses, consistent with adaptation. Presumably this is a general effect always at work in natural vision—there is an interaction between responses across fixations. Temporal context is rich during natural vision, but is usually ignored in laboratory experiments with long interstimulus intervals. As to the larger responses we observed in saccade conditions, we are able to reject response history as a cause of this effect. Contrary to the general influence of response history, larger saccade condition responses to the bar stimulus were usually preceded by periods of relatively high, not low, activity compared with flash conditions. This result is consistent with that of Gawne and Woods (2003), wherein V1 responses to checkerboard patterns were less when preceded across a saccade by a similar pattern than when preceded by a uniform gray field. We extend this finding by showing that it is not merely the presence of a prior stimulus in the receptive field that determines subsequent suppression, but the magnitude of the response it evokes.
Interactions between natural scenes and saccades
When we combined the natural background with saccades, resulting responses were generally not predicted by linear combination of the effect of each alone. Saccades counteracted the inhibitory effect of the natural background to a greater extent than predicted by their impact on responses to stimuli presented on the gray background. This nonlinearity suggests that there is less surround suppression during natural vision than in conventional laboratory paradigms. Our data do not allow us to determine precisely why this occurred. As reviewed earlier, several factors come into play during natural vision that are ignored in most explorations of surround suppression, including fixational eye movements and residual responses from prior RF stimuli. Each of these could partially counteract surround suppression during natural vision.
The reason why surround suppression should be sensitive to saccades is not clear. One possibility is that at the end of a saccade, much of visual cortex is activated by the new stimuli introduced into essentially all receptive fields. Perhaps this leads to a different level of inhibition in the network compared with focal stimulation of a small area of cortex with an isolated flashed stimulus. Less surround suppression following saccades might facilitate maintenance of a stable visual representation across eye movements. Alternatively, higher responses after saccades may serve no purpose on their own, but rather represent a rebound from saccadic suppression. Whatever the reason for interactions between natural scenes and saccades, the point remains that it is difficult to predict the responses of neurons under natural vision conditions based on results obtained under reduced conditions. Surround and saccade influences exert opposing effects on the magnitude of V1 responses, and can combine in a complex fashion. Thus any consideration of the behavior of neurons during natural vision should take into account not only the independent effects of the visual context and behavioral mode, but also the potential for interactions between the two. The differences in the time course and magnitude of V1 responses that we observe in reduced and natural paradigms may have important implications for the cortical circuitry activated in the different situations.
|
|
GRANTS |
|---|
|
|
|
ACKNOWLEDGMENTS |
|---|
|
|
|
FOOTNOTES |
|---|
Address for reprint requests and other correspondence: M. A. Paradiso, Dept. of Neuroscience, Brown University, Providence, RI 02912
|
|
REFERENCES |
|---|
|
Blakemore C, Tobin EA. Lateral inhibition between orientation detectors in the cat's visual cortex. Exp Brain Res 15: 439–440, 1972.[Web of Science][Medline]
Carpenter RH. Movements of the Eyes. London: Pion, 1988.
Cavanaugh JR, Bair W, Movshon JA. Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. J Neurophysiol 88: 2530–2546, 2002.
David SV, Vinje WE, Gallant JL. Natural stimulus statistics alter the receptive field structure of v1 neurons. J Neurosci 24: 6991–7006, 2004.
DeAngelis GC, Freeman RD, Ohzawa I. Length and width tuning of neurons in the cat's primary visual cortex. J Neurophysiol 71: 347–374, 1994.
DiCarlo JJ, Maunsell JH. Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing. Nat Neurosci 3: 814–821, 2000.[CrossRef][Web of Science][Medline]
Duhamel JR, Colby CL, Goldberg ME. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255: 90–92, 1992.
Fries W, Albus K, Creutzfeldt OD. Effects of interacting visual patterns on single cell responses in cats striate cortex. Vision Res 17: 1001–1008, 1977.[CrossRef][Web of Science][Medline]
Gallant JL, Connor CE, Van Essen DC. Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. Neuroreport 9: 2153–2158, 1998.[Web of Science][Medline]
Gawne TJ, Martin JM. Responses of primate visual cortical neurons to stimuli presented by flash, saccade, blink, and external darkening. J Neurophysiol 88: 2178–2186, 2002.
Gawne TJ, Woods JM. The responses of visual cortical neurons encode differences across saccades. Neuro Report 14: 105–109, 2003.[CrossRef][Web of Science][Medline]
Guo K, Robertson RG, Mahmoodi S, Young MP. Center-surround interactions in response to natural scene stimulation in the primary visual cortex. Eur J Neurosci 21: 536–548, 2005.[CrossRef][Web of Science][Medline]
Jones HE, Grieve KL, Wang W, Sillito AM. Surround suppression in primate V1. J Neurophysiol 86: 2011–2028, 2001.
Judge SJ, Wurtz RH, Richmond BJ. Vision during saccadic eye movements. I. Visual interactions in striate cortex. J Neurophysiol 43: 1133–1155, 1980.
Kastner S, Nothdurft HC, Pigarev IN. Neuronal correlates of pop-out in cat striate cortex. Vision Res 37: 371–376, 1997.[CrossRef][Web of Science][Medline]
Kayser C, Kording KP, Konig P. Processing of complex stimuli and natural scenes in the visual cortex. Curr Opin Neurobiol 14: 468–473, 2004.[CrossRef][Web of Science][Medline]
Kayser C, Salazar RF, Konig P. Responses to natural scenes in cat V1. J Neurophysiol 90: 1910–1920, 2003.
Knierim JJ, van Essen DC. Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. J Neurophysiol 67: 961–980, 1992.
Kusonoki M, Gottlieb J, Goldberg ME. The lateral intraparietal area as a salience map: the representation of abrupt onset, stimulus motion, and task relevance. Vision Res 40: 1459–1468, 2000.[CrossRef][Web of Science][Medline]
Latour P. Visual threshold during eye movements. Vision Res 2: 261–262, 1962.[CrossRef][Web of Science]
Leopold DA, Logothetis NK. Microsaccades differentially modulate neural activity in the striate and extrastriate visual cortex. Exp Brain Res 123: 341–345, 1998.[CrossRef][Web of Science][Medline]
Levitt JB, Lund JS. Contrast dependence of contextual effects in primate visual cortex. Nature 387: 73–76, 1997.[CrossRef][Medline]
Livingstone MS, Freeman DC, Hubel DH. Visual responses in V1 of freely viewing monkeys. Cold Spring Harb Symp Quant Biol 61: 27–37, 1996.
MacEvoy SP, Kim W, Paradiso MA. Integration of surface information in primary visual cortex. Nat Neurosci 1: 616–620, 1998.[CrossRef][Web of Science][Medline]
MacEvoy SP, Hanks TD, Paradiso MA. Responses of macaque V1 neurons with natural scenes and saccades. Soc Neurosci 662.4, 2002.
Martinez-Conde S, Macknik SL, Hubel DH. Microsaccadic eye movements and firing of single cells in the striate cortex of macaque monkeys. Nat Neurosci 3: 251–258, 2000.[CrossRef][Web of Science][Medline]
Moeller GU, Kayser C, Konig P. Saccade-related activity in areas 18 and 21a of cats freely viewing complex scenes. Neuroreport 18: 401–404, 2007.[CrossRef][Web of Science][Medline]
Muller JR, Metha AB, Krauskopf J, Lennie P. Local signals from beyond the receptive fields of striate cortical neurons. J Neurophysiol 90: 822–831, 2003.
Nakamura K, Colby CL. Updating of the visual representation in monkey striate and extrastriate cortex during saccades. Proc Natl Acad Sci 99: 4026–4031, 2002.
Nelson JI, Frost BJ. Orientation-selective inhibition from beyond the classic visual receptive field. Brain Res 139: 359–365, 1978.[CrossRef][Web of Science][Medline]
Reppas JB, Usrey WM, Reid RC. Saccadic eye movements modulate visual responses in the lateral geniculate nucleus. Neuron 35: 961–974, 2002.[CrossRef][Web of Science][Medline]
Richmond BJ, Hertz JA, Gawne TJ. The relation between V1 neuronal responses and eye movement-like stimulus presentations. Neurocomputing 26: 247–254, 1999.[CrossRef]
Ringach DL, Hawken MJ, Shapley R. Receptive field structure of neurons in monkey primary visual cortex revealed by stimulation with natural image sequences. J Vis 2: 12–24, 2002.[CrossRef][Medline]
Ross J, Morrone MC, Goldberg ME, Burr DC. Changes in visual perception at the time of saccades. Trends Neurosci 24: 113–121, 2001.[CrossRef][Web of Science][Medline]
Rossi AF, Rittenhouse CD, Paradiso MA. The representation of brightness in primary visual cortex. Science 273: 1104–1107, 1996.[Abstract]
Ruderman DL. The statistics of natural images. Network Comp Neural Sys 5: 517–548, 1994.[CrossRef]
Ruderman DL, Bialek W. Statistics of natural images: Scaling in the woods. Physical Rev Lett 73: 814–817, 1994.[CrossRef][Web of Science][Medline]
Sceniak MP, Hawken MJ, Shapley R. Visual spatial characterization of macaque V1 neurons. J Neurophysiol 85: 1873–1887, 2001.
Sceniak MP, Ringach DL, Hawken MJ, Shapley R. Contrast's effect on spatial summation by macaque V1 neurons. Nat Neurosci 2: 733–739, 1999.[CrossRef][Web of Science][Medline]
Sengpiel F, Sen A, Blakemore C. Characteristics of surround inhibition in cat area 17. Exp Brain Res 116: 216–228, 1997.[CrossRef][Web of Science][Medline]
Series P, Lorenceau J, Fregnac Y. The "silent" surround of V1 receptive fields: theory and experiments. J Physiol 97: 453–474, 2003.
Sillito AM, Grieve KL, Jones HE, Cudeiro J, Davis J. Visual cortical mechanisms detecting focal orientation discontinuities. Nature 378: 492–496, 1995.[CrossRef][Medline]
Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annu Rev Neurosci 24: 1193–1216, 2001.[CrossRef][Web of Science][Medline]
Smyth D, Willmore B, Baker GE, Thompson ID, Tolhurst DJ. The receptive-field organization of simple cells in primary visual cortex of ferrets under natural scene stimulation. J Neurosci 23: 4746–4759, 2003.
Snodderly DM, Kagan I, Gur M. Selective activation of visual cortex neurons by fixational eye movements: implications for neural coding. Vis Neurosci 18: 259–277, 2001.[CrossRef][Web of Science][Medline]
Thiele A, Henning P, Kubischik M, Hoffmann KP. Neural mechanisms of saccadic suppression. Science 295: 2460–2462, 2002.
Tolias AS, Moore T, Smirnakis SM, Tehovnik EJ, Siapas AG, Schiller PH. Eye movements modulate visual receptive fields of V4 neurons. Neuron 29: 757–767, 2001.[CrossRef][Web of Science][Medline]
Toth LJ, Rao SC, Kim DS, Somers D, Sur M. Subthreshold facilitation and suppression in primary visual cortex revealed by intrinsic signal imaging. Proc Natl Acad Sci USA 93: 9869–9874, 1996.
van Hateren JH, van der Schaaf A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proc R Soc Lond B Biol Sci 265: 359–366, 1998.[Medline]
Vinje WE, Gallant JL. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287: 1273–1276, 2000.
Vinje WE, Gallant JL. Natural stimulation of the nonclassical receptive field increases information transmission efficiency in V1. J Neurosci 22: 2904–2915, 2002.
Volkmann FC, Schick AM, Riggs LA. Time course of visual inhibition during voluntary saccades. J Opt Soc Am 58: 562–569, 1968.[Medline]
Walker GA, Ohzawa I, Freeman RD. Asymmetric suppression outside the classical receptive field of the visual cortex. J Neurosci 19: 10536–10553, 1999.
Wurtz RH. Comparison of effects of eye movements and stimulus movements on striate cortex neurons of the monkey. J Neurophysiol 32: 987–994, 1969.
Yu SX, Lee TS. What do V1 neurons tell us about saccadic suppression? Neurocomputing 32–33: 271–277, 2000.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| Visit Other APS Journals Online |
TOP
ABSTRACT
INTRODUCTION
, response distribution for free-viewing; 
, responses to flashed optimal bars. B: firing rates in flashed-bar and free-viewing conditions for all cells, averaged across all window positions. Error bars are SE. C: comparison of peak firing rates across the cell population in flashed-bar and free-viewing conditions. Peak rates are the highest firing rates observed in any 50-ms window across all trials for each cell. Symbol sizes are proportional to number of cells plotted at each location. D: comparison of average free-viewing activity to the peak response to a flashed bar. In more natural visual situations, V1 activity is well below the response rates commonly reported in experiments with optimal stimuli. In figures B–D, — are diagonals.
, distribution of preferred orientations, relative to horizontal (0°), for cells showing significant differences between saccade and flash conditions. The distribution for cells without a significant mode difference is shown in gray. Although these distributions peak at different orientations, their means (
) are nearly identical, suggesting that the difference between flash and saccade responses does not result from movement of the bar across the RF during saccades.
