JN AJP: Renal Physiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

J Neurophysiol 98: 3163-3170, 2007. First published October 10, 2007; doi:10.1152/jn.00975.2007
0022-3077/07 $8.00
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
98/6/3163    most recent
00975.2007v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (1)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ikeda, T.
Right arrow Articles by Hikosaka, O.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Ikeda, T.
Right arrow Articles by Hikosaka, O.

Positive and Negative Modulation of Motor Response in Primate Superior Colliculus by Reward Expectation

Takuro Ikeda1,2,3,5 and Okihide Hikosaka4,5

1Department of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi; 2Core Research for Evolutional Science and Technology, Japan Science and Technology Agency, Kawaguchi; 3Brain Science Research Center, Tamagawa University Research Institute, Tokyo, Japan; 4Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, Maryland; and 5Department of Physiology, Juntendo University, School of Medicine, Tokyo, Japan

Submitted 29 August 2007; accepted in final form 4 October 2007


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Expectation of reward is crucial for goal-directed behavior of animals. However, little is known about how reward information is used in the brain at the time of action. We investigated this question by recording from single neurons in the macaque superior colliculus (SC) while the animal was performing a memory-guided saccade task with an asymmetrical reward schedule. The SC is an ideal structure to ask this question because it receives inputs from many brain areas including the prefrontal cortex and the basal ganglia where reward information is thought to be encoded and sends motor commands to the brain stem saccade generators. We found two groups of SC neurons that encoded reward information in the presaccadic period: positive reward-coding neurons that showed higher activity when reward was expected and negative reward-coding neurons that showed higher activity when reward was not expected. The positive reward-coding usually started even before a cue for target position was presented, whereas the negative reward-coding was largely restricted to the presaccadic period. The two kinds of reward-coding may be useful for the animal to select an appropriate behavior in a complex environment.


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
The superior colliculus (SC) is a key structure that dictates the occurrence of saccadic eye movements (Sparks 1986Go). Neurons in its intermediate layer exhibit a stereotyped burst of spikes just before a saccade occurs in a particular direction and amplitude (Sparks et al. 1976Go). This is because the phasic signal from the SC neurons is sent to the brain stem saccade generators, which then excite or inhibit extraocular motoneurons in a pulsatile fashion (Raybourn and Keller 1977Go; Takahashi et al. 2005Go). It is thus not surprising that the presaccadic activity of SC neurons is related to saccade parameters such as saccade velocity (Aizawa and Wurtz 1998Go; Hikosaka and Wurtz 1985Go; Lee et al. 1988Go; Walton and May 2003Go), saccade endpoint (Sparks et al. 1976Go; Wurtz and Goldberg 1972Go), and saccadic reaction time (Basso and Wurtz 1998Go; Dorris and Munoz 1998Go).

On the other hand, saccadic eye movements are known to be modulated by reward expectation (Takikawa et al. 2002bGo): their velocities are higher and reaction times are shorter when a larger-than-average reward is expected. A prediction from these observations would be that the presaccadic activity of SC neurons is enhanced when a larger reward is expected, which then leads to a quicker and faster saccade. This hypothesis seems plausible given the fact that brain areas that are likely to send signals to the SC contain many neurons the activity of which is modulated by expected rewards. They include the frontal eye field (FEF) (Ding and Hikosaka 2006Go; Roesch et al. 2006Go), supplementary eye field (SEF) (Amador et al. 2000Go; Coe et al. 2002Go), dorsolateral prefrontal cortex (Kobayashi et al. 2002Go, 2007Go; Leon and Shadlen 1999Go; Tsujimoto and Sawaguchi 2005Go; Watanabe 1996Go), parietal cortex (Platt and Glimcher 1999Go; Sugrue et al. 2004Go), caudate nucleus (Cromwell and Schultz 2003Go; Hollerman et al. 1998Go; Itoh et al. 2003Go; Kawagoe et al. 1998Go; Lauwereyns et al. 2002Go; Takikawa et al. 2002aGo), and substantia nigra pars reticulata (SNr) (Sato and Hikosaka 2002Go). Furthermore, we previously showed that the visual responses of SC saccadic neurons to the cue for a future saccade were enhanced when the cue indicated the presence, as opposed to the absence, of the upcoming reward (Ikeda and Hikosaka 2003Go). It was unknown, however, whether the same type of reward modulation occurs for the presaccadic activity of SC neurons. This is true for other brain areas: reward modulations were closely examined for visual responses and delay activity but not for presaccadic activity, except for one study on caudate neurons (Watanabe et al. 2003Go).

We therefore decided to test the hypothesis that the presaccadic activity of SC neurons is modulated by expected rewards. The results obtained in the present study support the hypothesis. However, contrary to our expectation, we found two groups of SC neurons, one enhanced and the other depressed by the presence of an upcoming reward.


   METHODS
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
General

We used two male Japanese monkeys (Macaca fuscata). Detailed methods were described elsewhere (Takikawa et al. 2002aGo). Under anesthesia using pentobarbital sodium, we implanted a head holder, a chamber for unit recording, and an eye coil under surgical procedures. Surgical procedures were conducted in aseptic conditions. A scleral eye coil was implanted in one eye for monitoring eye position. A recording chamber was placed over the occipital cortex. All surgical and experimental protocols were approved by the Juntendo University Animal Care and Use Committee, and were in accordance with the National Institutes of Health Guide for Care and Use of Animals.

Behavioral tasks

The monkey sat in a primate chair in a dimly lit and sound attenuated room with his head fixed. In front of him was a tangent screen onto which small red spots of light were backprojected using two light-emitting diodes (LED) projectors. We trained monkeys to perform a memory-guided saccade task with an asymmetrical reward schedule, a one-direction-rewarded version of a memory-guided saccade task (1DR; Fig. 1). A trial started with the onset of a central fixation point which the monkey had to fixate. A cue stimulus, a small red spot of light, came on 1 s after fixation onset for 100 ms. The monkey had to keep fixation and remember the cued location. The fixation point turned off after 1–1.5 s, and the monkey had to make a saccade to the cued location. The correct saccade was indicated by a tone stimulus. The cue stimulus was presented at one out of two possible locations: one was located in the response field (RF) of the recorded neuron; the other was located outside the RF in the diametrically symmetric position relative to the fixation point (Fig. 1, bottom). The cue location was chosen pseudo-randomly across trials such that every sub-block of four trials contained two trials for each of the two locations randomly.


Figure 1
View larger version (13K):
[in this window]
[in a new window]

  		FIG. 1. Memory-guided saccade task in 1-direction-rewarded condition (1DR). Top: timing of stimulus presentation and eye movements. Bottom: The cue stimulus was presented either inside the neuron's response field (RF) or opposite to the RF (non-RF). In the RF-rewarded block (left), only the RF cue indicated that reward would be given after the correct saccade. In the non-RF-rewarded block (right), only the non-RF cue indicated reward. We compared the neuronal activity preceding saccades to the RF cue when the saccade was followed by a reward (top-left condition) and when it was followed by no reward (top-right condition). See METHODS for detail.

 
A unique feature of the 1DR task was that only one of the two locations was rewarded. The rewarded target position was fixed in a block of 40–60 successful trials. Thus there were two kinds of blocks: RF-rewarded block (the rewarded position was in the RF) and non-RF-rewarded block (the rewarded position was outside the RF). Even for nonrewarded trials, the monkeys had to make a correct saccade. If the saccade was incorrect, the same trial was repeated. For each recorded neuron, we obtained data on at least one set of the RF-rewarded block and the non-RF-rewarded block. The order of two blocks was randomized across neurons. We usually repeated the two blocks to confirm the reliability of the data.

Recording procedure

Single-unit recordings were performed using tungsten electrodes (Frederick Haer, Bowdoinham, ME). The electrode was inserted into the brain through a stainless steel guide tube (diameter: 0.8 mm), which was used to penetrate the dura. A hydraulic micro drive (Narishige, MO 95-S) was used to advance the electrode into the brain. To determine the depth of recording, we used electrophysiological measures. The entry of the electrode into the SC was indicated by a sudden appearance of multi-unit neuronal activity or cell injury. We then backed up the electrode slowly until no neuronal activity was detected. We then advanced the electrode until any neuronal activity appeared. We determined this depth as the surface of the SC. Eye movements were recorded using the search coil method (Enzanshi Kogyo, MEL-20U). Eye positions were sampled at 500 Hz. Extracellular spikes were sampled at 1 kHz.

Data analysis

We used only correct trials for analysis. The correct rates for rewarded trials were 96.7% (monkey G) and 90.3% (monkey A). The correct rates for nonrewarded trials were 91.9% (monkey G) and 86.9% (monkey A). The correct rates were significantly higher for rewarded trials than nonrewarded trials for both monkeys (P < 0.0001 for both monkeys, {chi}2 test).

Analysis of saccadic eye movements

The eye velocity at time t was calculated as follows

Formula
x(t) and y(t) are the horizontal and vertical eye position at time t. Then we convolved the velocity using the vector [1/4, 2/4, 1/4] for smoothing. The saccade onset was determined as the time when the smoothed eye velocity first exceeded a threshold value (40°/s) and the end of saccade was determined as the time when the velocity returned to less than another threshold value (28°/s) after saccade onset. If the time interval between the onset and end of the saccade was <25 ms, that movement was not considered as a saccade.

Analysis of neural activity

Neuronal responses were analyzed based on spike density functions created with a Gaussian function with SD of 5 ms. The following analyses and figures were based on these spike density functions. For population histograms, we estimated the 95% confidence intervals using a bootstrap method with 1,000 iterations.

Determination of saccadic activity in SC neurons

The presence of saccadic activity was determined for each neuron by the following criteria. First, the firing rate in a presaccadic period (duration: 100 ms, starting 80 ms before saccade onset) must be higher in RF-cued trials than in non-RF-cued trials in the RF-rewarded block (Mann-Whitney U test, P < 0.01). Second, the peak of mean firing rate around saccade onset (±30 ms) in RF-cued trials must be >40 spike/s.

SC neurons are known to have strong spatial selectivity, and we selected neurons that showed strong spatial selectivity in the presaccadic period. Thus we only analyzed neural activity in RF-cued trials when the monkeys made a saccade to the RF of the recorded neuron.

Partial correlation analysis

To examine the relationship between saccadic neuronal activity and reward expectation, we calculated partial correlation coefficient (Spearman's rank correlation coefficient) between saccadic activity and the reward ({rho}REW) controlling for saccade parameters (peak saccade velocity, saccade accuracy, and saccadic reaction time). Saccadic activity was the firing rates in a presaccadic period (duration: 100 ms, starting 80 ms before saccade onset) and reward value was set to 1 if the trial was rewarded and 0 if the trial was not rewarded. Significance of the {rho}REW was determined by a Student's t distribution for a transformation of the correlation. If {rho}REW of a neuron was significantly different from 0 (P < 0.05), we classified the neuron as a reward-coding neuron.

To see how reward information is coded throughout the trials, we calculated partial correlation between neural activity and the reward in different time windows: delay period (100-ms window starting 100 ms before fixation offset) and post cue period (100-ms window starting 50 ms after cue onset).

We also calculated partial correlation coefficient between saccadic activity and peak saccade velocity controlling for reward, saccade accuracy, and saccadic reaction time ({rho}VEL), partial correlation coefficient between saccadic activity and saccade accuracy (distance between cued location and saccade endpoint) controlling for reward, peak saccade velocity and saccadic reaction time ({rho}ACC), and partial correlation coefficient between saccadic activity and saccadic reaction time controlling for reward, peak saccade velocity and saccade accuracy ({rho}SRT). For calculating {rho}VEL and {rho}ACC, we used presaccadic activity (100 ms starting 80 ms before saccade onset). For calculating {rho}SRT, we used different time window around fixation offset (100 ms starting 50 ms before fixation offset).

Build-up index

Two types of neurons have been identified among saccadic neurons in the SC (Munoz and Wurtz 1995Go): burst neurons that exhibit a burst of spikes just before saccades and build-up neurons that showed more gradual change of activity before saccade without or in addition to the burst activity. To determine whether the neuron had build-up activity, we calculated a build-up index. [build-up index] = [preparatory activity]/[peak mean firing rate].

[Preparatory activity] was defined as the mean firing rate in 100 ms starting 200 ms before saccade onset. [Peak mean firing rate] was peak firing rate around saccade onset (–30 to +30 ms) of mean spike density function. We used trials in which monkeys made correct saccade to the RF of the recorded neuron. Build-up index was computed using both rewarded and nonrewarded trials.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
We recorded 179 neurons in the superior colliculus in two male Japanese monkeys (M. fuscata). Among them, 101 neurons (56%) showed significant saccadic activity in 1DR task (see METHODS). We tested whether the saccadic activity of SC neurons was modulated by reward expectation. Figure 2A shows the comparison of the saccadic activity between rewarded trials and nonrewarded trials for each neuron. Nearly 40% of the neurons (39/101, 39%) showed modulation by reward expectation (P < 0.05, t-test). Some of them showed higher saccadic activity when reward was expected (26/39, 67%), and the other neurons showed higher saccadic activity when reward was not expected (13/39, 33%). This result suggests that the saccadic activity of SC neurons is not a simple motor command but is significantly modulated by reward expectation.


Figure 2
View larger version (13K):
[in this window]
[in a new window]

  		FIG. 2. The modulation of saccadic activity by reward expectation. A: scatter plot of saccadic activity of SC neurons (n = 101). Each dot represents data from a single neuron, indicating the mean firing rate in rewarded trials (ordinate) and the mean firing rate in nonrewarded trials (abscissa). bullet and Figure 2, neurons that showed a significant difference in mean firing rate between rewarded and nonrewarded trials (t-test, P < 0.05). The mean firing rates were calculated using a 100-ms time window starting 80 ms before saccade onset. B: histogram of partial correlation coefficient between saccadic activity and reward controlling for saccade parameters ({rho}REW). {blacksquare} and Figure 2, neurons the {rho}REW of which were significantly different from 0. dashed line, estimated distribution of the {rho}REW by random permutation.

 
However, given the fact that the saccade parameters such as peak saccade velocity and saccadic reaction time are modulated by the reward expectation (Takikawa et al. 2002bGo), this raises a question: does the reward-dependent modulation of saccadic activity shown in Fig. 2 actually code reward expectation or saccade parameters? To answer this question, we calculated partial correlation coefficient between saccadic activity and reward condition controlling for saccade parameters ({rho}REW). This procedure enabled us to see the relation between reward expectation and neural activity without any influence from saccade parameters. Figure 2B shows the histogram of {rho}REW of 101 saccade related neurons. Thirty-two neurons (32/101, 32%) had {rho}REW which was significantly different from 0 (P < 0.05). Among these neurons, 18 neurons showed positive {rho}REW indicating higher activity in rewarded trials, and 14 neurons showed negative {rho}REW indicating higher activity in nonrewarded trials.

One might still wonder if these neurons were determined to be reward-modulated simply by chance. To examine this possibility, we estimated the distribution of {rho}REW assuming that there are no correlation between neural activity and reward expectation using a permutation test (1,000 times for each of 101 neurons), and the result is shown as a dashed line in Fig. 2B. Our experimental data were distributed much wider than the estimation based on no correlation (variance = 0.108 and 0.0282, respectively, P < 0.00001 by F test). This suggests that those neurons the {rho}REW of which was significantly different from 0 actually coded reward information. We classified these neurons as positive reward-coding neurons (Fig. 2B, {blacksquare}) and negative reward-coding neurons (Fig. 2B, {blacksquare}). In the following we illustrate the characteristics of these two types individually.

Figure 3A shows an example of positive reward-coding neuron. This neuron showed higher activity in rewarded trials in the presaccadic period as expected by the classification criterion. This positive reward modulation was not limited in the presaccadic period: the neuron showed higher activity in rewarded trials throughout the trial from the precue period until saccade execution. The population analysis of 18 positive reward-coding neurons showed a similar tendency (Fig. 3B). Figure 4A shows an example of negative reward-coding neuron. Unlike positive reward-coding neurons, the negative reward modulation was limited to saccade related activity. This neuron showed positive reward modulation around cue onset and even in the delay period. The averaged histogram of 14 negative reward-coding neurons showed negative reward modulation only around the saccade onset (Fig. 4B).


Figure 3
View larger version (20K):
[in this window]
[in a new window]

  		FIG. 3. A: example of positive reward-coding neuron. The spike density function and the rasters are aligned on either cue onset (left) or saccade onset (right). Black indicates rewarded trials and gray indicates nonrewarded trials. Only the trials in which cue was presented in the RF of the neuron are shown. B: population histogram of positive reward-coding neurons (n = 18). The average spike density histogram around cue onset (left) and saccade onset (right). Black line, data from rewarded trials; gray line, data from nonrewarded trials. Only the trials in which cue was presented in the RF of the neurons are shown. Shaded areas are estimated 95% confidence intervals of average spike density for rewarded trials and nonrewarded trials.

 

Figure 4
View larger version (24K):
[in this window]
[in a new window]

  		FIG. 4. A: example of negative reward-coding neuron; B: population histogram of negative reward-coding neurons (n = 14). The same format as in Fig. 3.

 
To see the difference in the time course of reward-coding between the positive type and the negative type, we calculated {rho}REW in the postcue period (100-ms time window starting from 50 ms after cue presentation) and the delay period (100 ms before fixation offset). Figure 5 shows the results. Most positive reward-coding neurons showed positive reward modulation also in the postcue period (P < 0.05, 16/18, 89%) and the delay period (P < 0.05, 10/18, 56%). In contrast, the negative reward-coding neurons showed no or inconsistent reward modulations in the postcue period (positive: 3/14, negative: 5/14, no modulation: 6/14) and the delay period (positive: 1/14, negative: 1/14, no modulation: 12/14). Most of the positive reward-coding neurons showed positive reward modulation also in the peri-cue period (100-ms time window starting from 50 ms before cue onset; 14/18, 78%, P < 0.01 by Mann-Whitney U test). This ratio was significantly higher than the negative reward-coding neurons (5/14, 36%) and the other neurons (21/69, 30%; Fisher's exact test, P = 0.03 and P = 0.0004, respectively).


Figure 5
View larger version (19K):
[in this window]
[in a new window]

  		FIG. 5. Comparisons of {rho}REW between presaccadic period and postcue period (A) and between presaccadic period and delay period (B). Postcue period is 100-ms time window starting from 50 ms after cue presentation, and delay period is 100-ms time window starting from 100 ms before fixation offset. bullet and Figure 5, positive reward-coding neurons and negative reward-coding neurons, respectively. Three histograms show distributions of {rho}REW in presaccadic period (top), postcue period (top right) and delay period (bottom right). Among 101 saccade related neurons, 3 neurons were excluded from the analysis in delay period because those neurons did not fire in that period.

 
We then asked whether the reward-related SC neurons also code saccade parameters (velocity, accuracy, and reaction time). We computed partial correlation coefficient between neural activity and one of the saccade parameters ({rho}VEL: velocity, {rho}ACC: accuracy, {rho}SRT: reaction time) while controlling for reward and other saccade parameters (Fig. 6). The distributions of partial correlation coefficients for all saccadic neurons were slightly but significantly shifted from zero for all saccade parameters. {rho}VEL was positively shifted (average ± SD = 0.11 ± 0.20, P < 0.000001 by t-test), {rho}ACC and {rho}SRT were negatively shifted ({rho}ACC: average ± SD = –0.10 ± 0.22, P < 0.0001 by t-test, {rho}SRT: average ± SD = –0.10 ± 0.19, P < 0.000001 by t-test), indicating that if the population activity of SC is higher, the saccade will be faster and more accurate with a shorter reaction time. However, these tendencies were less clear for reward-coding neurons (positive: black; negative: gray) than for nonreward-coding neurons (white); a significant shift from zero was observed only for {rho}SRT (Fig. 6C, positive reward-coding neurons: average ± SD = –0.13 ± 0.20, P = 0.014, negative reward-coding neurons: average ± SD = –0.11 ± 0.14, P = 0.020 by t-test). Only small portions of reward-coding neurons showed significant partial correlations to one or two of the saccade parameters (P < 0.05, positive reward-coding neurons: 1/18 for {rho}VEL, 3/18 for {rho}ACC, 2/18 for {rho}SRT; negative reward-coding neurons: 1/14 for {rho}VEL, 1/14 for {rho}ACC, 1/14 for {rho}SRT). In contrast, larger portions of nonreward-coding neurons showed significant partial correlations to the saccade parameter (P < 0.05, 12/69 positively for {rho}VEL, 2/69 positively and 12/69 negatively for {rho}ACC, 2/69 positively and 11/69 negatively for {rho}SRT). These results suggest that the saccadic reaction time is influenced by the population activity of SC neurons including reward-coding neurons, but the peak saccade velocity and the saccade endpoints are determined largely by nonreward-coding neurons.


Figure 6
View larger version (8K):
[in this window]
[in a new window]

  		FIG. 6. Histograms of partial correlation coefficient between SC neural activity and saccade parameters: A: peak saccade velocity ({rho}VEL), B: saccade accuracy ({rho}ACC), and C: saccadic reaction time ({rho}SRT). All saccade related neurons (n = 101) were included in A and B. Three neurons that showed no activity in 100- ms window around fixation offset were excluded in C. Black, gray and white areas indicate positive reward-coding neurons, negative reward-coding neurons, and nonreward-coding neurons, respectively. Dashed lines dividing each area of the histograms indicate the average of partial correlation coefficients for each group (white dashed lines for positive and negative reward-coding neurons, black dashed line for nonreward-coding neurons).

 
We found that the recording sites were significantly deeper for the positive reward-coding neurons [1.7± 0.56 (SD) mm from the surface of the SC] than for the negative reward-coding neurons [1.2 ± 0.73 (SD) mm; t-test, P = 0.020]. It has been reported that there are two classes of saccade related neurons in SC, build-up neurons and burst neurons, and that build-up neurons are located more ventrally than burst neurons in the intermediate layer of SC (Munoz and Wurtz 1995Go). Indeed, the population activity of positive reward-coding neurons (Fig. 3B) suggests that these neurons exhibit tonic firing in the delay period between cue onset and saccade onset, a feature that characterizes "build-up neurons". The build-up index (see METHODS) was significantly higher for positive reward-coding neurons than for negative reward-coding neurons (Mann-Whitney U test, P = 0.020: positive reward-coding neurons, 0.25 ± 0.14; negative reward-coding neurons, 0.13 ± 0.14, average ± SD). These results suggest that positive reward-coding neurons are more likely to be build-up neurons and negative reward-coding neurons are more likely to be burst neurons.


   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Using an asymmetrically rewarded memory-guided saccade task, we found that many neurons in the intermediate layer of SC showed reward-dependent modulations in saccadic activity. We previously reported that the visual responses of SC neurons are often facilitated by reward expectation (Ikeda and Hikosaka 2003Go). In the present study, we found two types of reward modulation in saccadic activity: some neurons showed higher activity in rewarded trials (positive reward-coding neurons), whereas other neurons showed higher activity in nonrewarded trials (negative reward-coding neurons).

Most of the positive reward-coding neurons also showed positive reward modulation in the peri-cue and delay periods. The modulation in the peri-cue period is basically the same as a type of reward modulation we reported previously (bias type) (Ikeda and Hikosaka 2003Go) and similar to the reward modulation found in caudate nucleus (Kawagoe et al. 1998Go; Takikawa et al. 2002aGo) and SNr (Sato and Hikosaka 2002Go). Because this modulation started before cue presentation before the monkeys knew whether they will be rewarded or not, it is likely to be related to the monkey's desire to make a saccade to the rewarded position regardless of the position of the upcoming target. In other words, the positive reward-coding neurons appear to set an a priori goal as a position associated with a reward and prepare for a saccade to the goal. In this sense, it seems reasonable that the positive reward-coding neurons behaved like build-up neurons (Munoz and Wurtz 1995Go). The build-up activity is higher when the saccade is more likely to be made to the neuron's response field (Basso and Wurtz 1998Go; Dorris and Munoz 1998Go) and therefore is considered to be related to the motor preparation. In our task, however, only one of the two targets was presented in both rewarded and nonrewarded trials, and therefore the reward modulation in the delay period cannot be explained by the likelihood of the saccade to the neuron's response field. It is also suggested that the SC is related to target selection (Carrelo and Krauzlis 2004Go; Horwitz and Newsome 2001aGo,bGo; Krauzlis and Dill 2002Go; McPeek and Keller 2002Go, 2004Go) and spatial attention (Goldberg and Wurtz 1972Go; Kustov and Robinson 1996Go; Robinson and Kertzman 1995Go). The object or location that has been associated with rewards would attract more attention and would be chosen to be the target of orienting responses. Thus the positive reward modulation may share the same neural mechanisms in the SC with target selection, spatial attention, and motor preparation, possibly receiving reward information from the basal ganglia.

On the other hand, the function of the negative reward-coding neurons is less clear. Their activity pattern was different from that of the positive reward-coding type in two respects. First, their activity was largely confined to the peri-saccadic and postcue periods. Second, the reward modulation in the postcue period was weak and mixed. In other words, the negative reward-coding neurons, as a population, were not tuned to the nonrewarded position until saccade initiation was imminent. Thus they are less likely to be related to target selection, spatial attention, or motor preparation based on the immediately expected reward. Then what could their function be? In the following we will discuss two possible functions: they may serve to obtain delayed rewards and they may serve to orient to objects, which is irrelevant to the reward.

The first possibility is that the negative reward-coding neurons are necessary to obtain delayed rewards. In the nonrewarded trials, monkeys could not get reward immediately but had to make a correct saccade to get rewards in future trials. It has been known in both human and animal subjects that if the delivery of a reward is delayed, its value is decreased with time, a phenomenon called "temporal discounting" (Green and Myerson 2004Go). Recent studies have suggested that different brain areas are involved in aiming at immediate and delayed rewards, respectively (Hariri et al. 2006Go; Kobayashi et al. 2007Go; McClure et al. 2004Go; Roesch et al. 2006Go; Tanaka et al. 2004Go; Winstanley et al. 2004Go). For example, expected immediate rewards are associated with activation in the striatum and the orbitofrontal cortex, whereas expected delayed rewards are associated with activation in the dorsolateral prefrontal cortex and the inferior parietal cortex. It is thus possible that the two types of reward-coding SC neurons receive inputs selectively from these brain areas. Most likely among them in terms of the known anatomical connections are the dorsolateral prefrontal cortex and the striatum. The dorsolateral prefrontal cortex is thought to send task-related signals to SC (Johnston and Everling 2006Go), and the caudate nucleus (part of the striatum) is known to send outputs to the SC through the substantia nigra pars reticulate (Hikosaka et al. 2000Go). Using the same 1DR task, Kobayashi et al. (2007)Go found that neurons in the dorsolateral prefrontal cortex tend to maintain spatial selectivity even when reward was not immediately available, whereas neurons in the caudate nucleus tend to show spatial selectivity preferentially when an immediate reward was expected. Alternatively, the reward-coding SC neurons may receive inputs from different groups of caudate neurons. Again using the 1DR task, Watanabe et al. (2003)Go found in the caudate positive reward-coding saccadic neurons and negative reward-coding saccadic neurons that may send signals separately to positive and negative reward coding neurons in SC.

The second possibility is that the negative reward-coding neurons are necessary to orient to an object that is not rewarding or even aversive or dangerous. This behavior may be ecologically advantageous because it is crucial to identify and examine any object, animal, or human, perhaps before starting approaching or avoidance behavior. For example, macaque monkeys would frequently look at higher-status monkeys but quickly avert gaze from them (Deaner et al. 2005Go). Support for this argument is the evidence suggesting that the SC is involved in avoidance behavior as well as orienting behavior (Sahibzada et al. 1986Go).

We so far have discussed the reward-coding SC neurons in relation to saccadic eye movements. Although the saccadic activity of SC neurons, as a whole population, showed some correlations with saccade parameters (velocity, accuracy, and reaction time), we could not find clear correlations between the activity of reward-coding neurons and the saccade parameters except a weak correlation with saccadic reaction time. However, it is possible that the reward-coding SC neurons are not involved in the control of saccades directly while contributing to the orienting responses. Recent studies suggest that descending commands from SC is selectively gated to control eye-head gaze shift (Corneil et al. 2004Go). And the fact that low levels of stimulation current in the intermediate layer of SC can induce neck EMG or head movement without gaze shifts (Corneil et al. 2002Go; Pélisson et al. 2001Go) suggests that low-frequency build-up activity of the positive reward-coding neurons preceding eye movement may be related to the bias in head position independent of gaze shift. Although highly speculative, this hypothesis further raises interesting implications. It has been suggested that SC output neurons make synaptic contacts with oculomotor interneurons (e.g., burst neurons and omnipause neurons) (Chimoto et al. 1996Go; Gandhi and Keller 1997Go; Raybourn and Keller 1977Go; Sugiuchi et al. 2005Go) as well as reticulo-spinal neurons that control both eye and body (particularly neck) movements (Grantyn and Berthoz 1987Go). Following the preceding discussion on the functional differentiation of SC neurons, we speculate that the SC reward-coding neurons have preferential connections to the reticulo-spinal neurons because these neurons are not specifically controlling the saccade parameters. Among reward-coding neurons, the positive type might drive the contralateral reticulo-spinal circuits for approaching behavior, while the negative type might drive the ipsilateral reticulo-spinal circuits for avoidance behavior (Buckenham and Yeomans 1993Go; Westby et al. 1990Go).

In conclusion, the saccade related activity of SC neuron is often modulated by reward expectation in either a positive or negative manner. These reward modulations, possibly affected by the input from the prefrontal cortex and the basal ganglia, might be important in selecting the appropriate behavior in the complex environment where there is a conflict between internal motivation and external signals.


   GRANTS
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
ACKNOWLEDGMENTS
 REFERENCES
 
This work was supported by Young Scientists (B) Grant-in-Aid 18700384 to T. Ikeda.


   ACKNOWLEDGMENTS
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
REFERENCES
 
We thank M. Sakagami, S. Kobayashi, and H. Itoh for helpful comments and M. Kato for designing the computer programs.


   FOOTNOTES
 
The costs of publication of this article were defrayed in part by the payment of page charges. The article must therefore be hereby marked "advertisement" in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Address for reprint requests and other correspondence: T. Ikeda, Dept. of Developmental Physiology, National Institute for Physiological Sciences, Okazaki, Aichi 444-8585, Japan (taikeda-ns{at}umin.ac.jp)


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 METHODS
 RESULTS
 DISCUSSION
 GRANTS
 ACKNOWLEDGMENTS
 REFERENCES
 
Aizawa H, Wurtz RH. Reversible inactivation of monkey superior colliculus. I. Curvature of saccadic trajectory. J Neurophysiol 79: 2082–2096, 1998.[Abstract/Free Full Text]

Amador N, Schlag-Rey M, Schlag J. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84: 2166–2170, 2000.[Abstract/Free Full Text]

Basso MA, Wurtz RH. Modulation of neuronal activity in superior colliculus by changes in target probability. J Neurosci 18: 7519–7534, 1998.[Abstract/Free Full Text]

Buckenham KE, Yeomans JS. An uncrossed tectopontine pathway mediates ipsiversive circling. Behav Brain Res 54: 11–22, 1993.[CrossRef][ISI][Medline]

Carello CD, Krauzlis RJ. Manipulating intent: evidence for a causal role of the superior colliculus in target selection. Neuron 43: 575–583, 2004.[CrossRef][ISI][Medline]

Chimoto S, Iwamoto Y, Shimazu H, Yoshida K. Monosynaptic activation of medium-lead burst neurons from the superior colliculus in the alert cat. J Neurophysiol 75: 2658–2661, 1996.[Abstract/Free Full Text]

Coe B, Tomihara K, Matsuzawa M, Hikosaka O. Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22: 5081–5090, 2002.[Abstract/Free Full Text]

Corneil BD, Olivier E, Munoz DP. Neck muscle responses to stimulation of monkey superior colliculus. I. Topography and manipulation of stimulation parameters. J Neurophysiol 88: 1729–1749, 2002.

Corneil BD, Olivier E, Munoz DP. Visual responses on neck muscles reveal selective gating that prevents express saccades. Neuron 42: 831–841, 2004.[CrossRef][ISI][Medline]

Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89: 2823–2838, 2003.[Abstract/Free Full Text]

Deaner RO, Khera AV, Platt ML. Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr Biol 15: 543–548, 2005.[CrossRef][ISI][Medline]

Ding L, Hikosaka O. Comparison of reward modulation in frontal eye field and caudate of the macaque. J Neurosci 26: 6695–6703, 2006.[Abstract/Free Full Text]

Dorris MC, Munoz DP. Saccadic probability influences motor preparation signals and time to saccadic initiation. J Neurosci 18: 7015–7026, 1998.[Abstract/Free Full Text]

Gandhi NJ, Keller EL. Spatial distribution and discharge characteristics of superior colliculus neurons antidromically activated from the omnipause region in monkey. J Neurophysiol 78: 2221–2225, 1997.[Abstract/Free Full Text]

Goldberg ME, Wurtz RH. Activity of superior colliculus in behaving monkey. II. Effect of attention on neuronal responses. J Neurophysiol 35: 560–574, 1972.[Free Full Text]

Grantyn A, Berthoz A. Reticulo-spinal neurons participating in the control of synergic eye and head movements during orienting in the cat. I. Behavioral properties. Exp Brain Res 66: 339–354, 1987.[ISI][Medline]

Green L, Myerson J. A discounting framework for choice with delayed and probabilistic rewards. Psychol Bull 130: 769–792, 2004.[CrossRef][ISI][Medline]

Hariri AR, Brown SM, Williamson DE, Flory JD, de Wit H, Manuck SB. Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. J Neurosci 26: 13213–13217, 2006.[Abstract/Free Full Text]

Hikosaka O, Takikawa Y, Kawagoe R. Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol Rev 80: 953–978, 2000.[Abstract/Free Full Text]

Hikosaka O, Wurtz RH. Modification of saccadic eye movements by GABA-related substances. I. Effect of muscimol and bicuculline in monkey superior colliculus. J Neurophysiol 53: 266–291, 1985.[Abstract/Free Full Text]

Hollerman JR, Tremblay L, Schultz W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80: 947–963, 1998.[Abstract/Free Full Text]

Horwitz GD, Newsome WT. Target selection for saccadic eye movements: direction-selective visual responses in the superior colliculus. J Neurophysiol 86: 2527–2542, 2001a.[Abstract/Free Full Text]

Horwitz GD, Newsome WT. Target selection for saccadic eye movements: prelude activity in the superior colliculus during a direction-discrimination task. J Neurophysiol 86: 2543–2558, 2001b.[Abstract/Free Full Text]

Ikeda T, Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39: 693–700, 2003.[CrossRef][ISI][Medline]

Itoh H, Nakahara H, Hikosaka O, Kawagoe R, Takikawa Y, Aihara K. Correlation of primate caudate neural activity and saccade parameters in reward-oriented behavior. J Neurophysiol 89: 1774–1783, 2003.[Abstract/Free Full Text]

Johnston K, Everling S. Monkey dorsolateral prefrontal cortex sends task-selective signals directly to the superior colliculus. J Neurosci 26: 12471–12478, 2006.[Abstract/Free Full Text]

Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411–416, 1998.[CrossRef][ISI][Medline]

Kobayashi S, Kawagoe R, Takikawa Y, Koizumi M, Sakagami M, Hikosaka O. Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward. Exp Brain Res 176: 341–355, 2007.[ISI][Medline]

Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M, Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87: 1488–1498, 2002.[Abstract/Free Full Text]

Krauzlis R, Dill N. Neural correlates of target choice for pursuit and saccades in the primate superior colliculus. Neuron 35: 355–363, 2002.[CrossRef][ISI][Medline]

Kustov AA, Robinson DL. Shared neural control of attentional shifts and eye movements. Nature 384: 74–77, 1996.[CrossRef][Medline]

Lauwereyns J, Watanabe K, Coe B, Hikosaka O. A neural correlate of response bias in monkey caudate nucleus. Nature 418: 413–417, 2002.[CrossRef][Medline]

Lee C, Rohrer WH, Sparks DL. Population coding of saccadic eye movements by neurons in the superior colliculus. Nature 24: 357–60, 1988.

Leon MI, Shadlen MN. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24: 415–425, 1999.[CrossRef][ISI][Medline]

McClure SM, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science 306: 503–507, 2004.[Abstract/Free Full Text]

McPeek RM, Keller EL. Saccade target selection in the superior colliculus during a visual search task. J Neurophysiol 88: 2019–2034, 2002.[Abstract/Free Full Text]

McPeek RM, Keller EL. Deficits in saccade target selection after inactivation of superior colliculus. Nat Neurosci 7: 757–763, 2004.[CrossRef][ISI][Medline]

Munoz DP, Wurtz RH. Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. J Neurophysiol 73: 2313–2333, 1995.[Abstract/Free Full Text]

Pélisson D, Goffart L, Guillaume A, Catz N, Raboyeau G. Early head movements elicited by visual stimuli or collicular electrical stimulation in the cat. Vision Res 41: 3283–3294, 2001.[CrossRef][ISI][Medline]

Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 400: 233–238, 1999.[CrossRef][Medline]

Raybourn MS, Keller EL. Colliculoreticular organization in primate oculomotor system. J Neurophysiol 40: 861–78, 1977.[Free Full Text]

Robinson DL, Kertzman C. Covert orienting of attention in macaques. III. Contributions of the superior colliculus. J Neurophysiol 74: 713–721, 1995.[Abstract/Free Full Text]

Roesch MR, Taylor AR, Schoenbaum G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51: 509–520, 2006.[CrossRef][ISI][Medline]

Sahibzada N, Dean P, Redgrave P. Movements resembling orientation or avoidance elicited by electrical stimulation of the superior colliculus in rats. J Neurosci 6: 723–733, 1986.[Abstract]

Sato M, Hikosaka O. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22: 2363–2373, 2002.[Abstract/Free Full Text]

Sparks DL. Translation of sensory signals into commands for control of saccadic eye movements: role of primate superior colliculus. Physiol Rev 66: 118–171, 1986.[Abstract/Free Full Text]

Sparks DL, Holland R, Guthrie BL. Size and distribution of movement fields in the monkey superior colliculus. Brain Res 113: 21–34, 1976.[CrossRef][ISI][Medline]

Sugiuchi Y, Izawa Y, Takahashi M, Na J, Shinoda Y. Physiological characterization of synaptic inputs to inhibitory burst neurons from the rostral and caudal superior colliculus. J Neurophysiol 93: 697–712, 2005.[Abstract/Free Full Text]

Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science 304: 1782–1787, 2004.[Abstract/Free Full Text]

Takahashi M, Sugiuchi Y, Izawa Y, Shinoda Y. Synaptic inputs and their pathways from fixation and saccade zones of the superior colliculus to inhibitory burst neurons and pause neurons. Ann NY Acad Sci 1039: 209–219, 2005.[CrossRef][ISI][Medline]

Takikawa Y, Kawagoe R, Hikosaka O. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J Neurophysiol 87: 508–515, 2002a.[Abstract/Free Full Text]

Takikawa Y, Kawagoe R, Itoh H, Nakahara H, Hikosaka O. Modulation of saccadic eye movements by predicted reward outcome. Exp Brain Res 142: 284–291, 2002b.[CrossRef][ISI][Medline]

Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7: 887–893, 2004.[CrossRef][ISI][Medline]

Tsujimoto S, Sawaguchi T. Neuronal activity representing temporal prediction of reward in the primate prefrontal cortex. J Neurophysiol 93: 3687–3692, 2005.[Abstract/Free Full Text]

Walton MM, Mays LE. Discharge of saccade-related superior colliculus neurons during saccades accompanied by vergence. J Neurophysiol 90: 1124–1139, 2003.[Abstract/Free Full Text]

Watanabe K, Lauwereyns J, Hikosaka O. Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J Neurosci 23: 10052–10057, 2003.[Abstract/Free Full Text]

Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 382: 629–632, 1996.[CrossRef][Medline]

Westby GW, Keay KA, Redgrave P, Dean P, Bannister M. Output pathways from the rat superior colliculus mediating approach and avoidance have different sensory properties. Exp Brain Res 81: 626–638, 1990.[CrossRef][ISI][Medline]

Winstanley CA, Theobald DE, Cardinal RN, Robbins TW. Contrasting roles of basolateral amygdale and orbitofrontal cortex in impulsive choice. J Neurosci 24: 4718–4722, 2004.[Abstract/Free Full Text]

Wurtz RH, Goldberg ME. Activity of superior colliculus in behaving monkey. III. Cells discharging before eye movements. J Neurophysiol 35: 575–586, 1972.[Free Full Text]




This article has been cited by other articles:


Home page
 J. Neurophysiol.Home page
M. Isoda and O. Hikosaka
A Neural Correlate of Motivational Conflict in the Superior Colliculus of the Macaque
J Neurophysiol, September 1, 2008; 100(3): 1332 - 1342.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
S. Rezvani and B. D. Corneil
Recruitment of a Head-Turning Synergy by Low-Frequency Activity in the Primate Superior Colliculus
J Neurophysiol, July 1, 2008; 100(1): 397 - 411.
[Abstract] [Full Text] [PDF]

<
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
98/6/3163    most recent
00975.2007v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (1)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Ikeda, T.
Right arrow Articles by Hikosaka, O.
Right arrow Search for Related Content
PubMed