Chapter 5: The Retinal Representation

• An Overview of the Retina
• Retinal Structure
• Retinal function: Specialization
• Retinal function: image contrast and adaptation
• Visual Streams
• Methods of classifying neurons
• Morphology of Parasol and Midget Ganglion Cells
• Variation with retinal eccentricity
• Central Projections
• Conduction Time and Contrast Gain
• Visual Information Encoded by the Parvocellular and Magnocellular Pathways
• Retinal Ganglion Cell Response To Light
• Center-Surround Organization
• Measurements of Receptive Fields
• Steady-state Measurements
• The Two-Dimensional Receptive Field
• Contrast Sensitivity Functions
• Why contrast patterns are important
• Connections to Different Cone Types
• Spatio-Temporal Analysis: Lines and Spots
• Spatio-Temporal Measurements: Harmonic Functions
• Space-time separability
• The Difference of Gaussian Model
• Retinal Light Adaptation
• Contrast Sensitivity: Dependence on Mean Intensity
• Comparison with Behavioral Contrast Sensitivity

An Overview of the Retina

In this chapter we will review the structure of the retina and its role in organizing visual information. The retina is a thin layer of neural tissue that lines the eye. After the retinal image is encoded by the photoreceptors, neurons within the retina transform the photoreceptor signals into a new representation that is carried by the optic nerve to a variety of locations in the brain. The retina is important for several reasons. First, the retina is important to neuroscientists because it is a very accessible part of the central nervous system making it an important site for scientific study Second, the retina is important to clinicians since it is the only part of the central nervous system that can be examined directly, by using an opthalmoscope. Third, the retina is important to vision scientists because it has several important visual functions, including encoding the image and transforming it into a collection of separate pathways that send information about the entire retinal image to the brain. Since retinal neurons develop from the same progenitor cells that give rise to the brain, the organization of information within these retinal pathways is also an important clue about the organization within the brain as well.

Retinal structure

Over most of its extent, the primate retina is approximately 0.5 mm thick and consists of three layers of cell bodies and two layers containing the synaptic interconnections between the neurons. Near the optical axis of the eye, however, the primate retina contains a specialized region, the fovea, consisting of only a single layer of neurons, the cone photoreceptors. Both of these structural properties of the retina can be seen in the anatomical cross-section section of a human retina shown in Figure 4.1.

Figure 5.2 shows two types of retinal neurons and identifies some of their parts, including the dendritic fields, cell bodies, and axon. The dendritic fields receive input from other neurons; the axon, which may branch, carries the neuron’s output to its destination\footnote Some neurons have no axon, exerting their influence only at local interconnections at synapses within the dendritic field. The shape of a neuron’s dendritic field and its axonal branches are generally important features for distinguishing broad classes of neurons. We will use many features, including the locations of cell bodies, dendrites and axons; the size and shape of their cell bodies and dendritic fields; and, their interconnections with other neurons, to classify and understand retinal neurons.

There are five basic categories of retinal neurons, although each category has several subclassifications. The major categories of retinal neurons are distinguished by the location of their cell bodies, dendritic fields, and axon terminals. The photoreceptors’ cell bodies are located in the outer nuclear layer of the layer. The synaptic terminals of the photoreceptors make contact with the dendritic fields of the bipolar and horizontal cells in the outer plexiform layer. The cell bodies of the bipolar and horizontal cells are located in the inner nuclear layer. Both the dendrites and the branching axon terminals of the horizontal cells make connections with cells in the outer nuclear layer. The bipolar cells, however, make connections onto the the dendrites of the ganglion cells within the inner plexiform layer. Since only the bipolar cells link the signals in the outer and inner plexiform layers, all visual signals must pass through the bipolar cells.

The amacrine cell bodies are also located in the inner nuclear layer. Santiago Ramon y Cajal gave these cells their name to indicate that they have no identifiable axons, but only dendrites. The dendritic fields of the amacrine cells make connections with the dendritic fields of the ganglion cells in the inner plexiform layer. The retinal ganglion cell bodies are located in the ganglion cell layer, and their dendritic fields connect with the axon terminals of the bipolars and the dendritic fields of the amacrine cells.

The axons of the retinal ganglion cells provide the only retinal output signal. The ganglion cell axons comprise the optic nerve and they exit from the retina at a single location in the retina called the optic disk. There are no photoreceptors at the optic disk, so we do not encode or perceive the light that falls there. Consequently, the optic disk is also called the blindspot. We are not aware of the blindspot in our eyes ordinarily since the portion of the visual field falling in the blindspot of one eye falls on functional a retina in the other eye. You can perceive your blindspot by closing one eye and then carefully fixating with the other eye on the “x” in Figure 5.3. Move the book to a viewing distance where the white square disappears, roughly 25 cm from your nose. Notice that the white square and dot within the texture pattern fill in, while the dots at a comparable visual eccentricity are still visible. Thus, the dot disappears because of the blindspot and not because of a loss of visual acuity in the periphery.

Retinal function: specialization

The retina segregates visual information into parallel neural pathways specialized for different visual tasks. In earlier chapters, we reviewed one example of neural specialization in the retina: there are two different types of photoreceptors, the rods and the cones, that both sample the image. These two photoreceptors types are responsible for encoding the visual image in different intensity ranges.

The segregation of rod and cone signals continues through several synaptic connections within the retina. Kolb and her collaborators have shown that the signals initiated within the rods follow a separate rod pathway within the retina until the signals arrive at the retinal ganglion cells (see Figure 5.4). The rods make connections with a class of bipolars, the rod bipolars, that integrate the responses of many different rod photoreceptors. Presumably, pooling the signals from many rods enhances the sensitivity of this rod pathway. The rod bipolars synapse directly onto type AII amacrine cells, but unlike other bipolars the rod bipolars do not synapse directly onto ganglion cells. The rod bipolars synapse on the AII amacrine cells within a narrow level of the inner plexiform layer. Finally, the AII amacrines synapse onto retinal ganglion cells both within the same level of the inner plexiform layer and also at a a second level of the inner plexiform layer. The ganglion cells that connect with the AII amacrine cells also receive signals via a cone-initiated pathway. Hence, the rod pathway merges with a cone-initiated pathway and disappears as a unique entity at this point.

By examining the properties of the rod pathway, we can see certain general features that might also be true of the organization of other visual pathways. First, the rod pathway only exists over a few synapses, serving its function and then merging with the main visual signal. By converging onto the main processing stream, central processing elements that define shape, form, and so forth can analyze both the rod and cone signals and need not be duplicated. Thus, a visual pathway may serve its purpose within a few synapses and then disappear.

Second, we see that visual streams may be created to serve fairly rudimentary functions, such as enhancing some aspect of the information in the image. The strong convergence of signals within the rod pathway — a single rod bipolar may integrate the signal from 1500 rods — makes the rod pathway well-suited to capturing information at low light levels while paying a penalty in terms of visual acuity. Hence, visual pathways may be created to achieve a special computational goal.

There are probably many types of information, in addition to the illumination level, that are formed for special computational purposes. For example, some types of behaviors may require precise visual information of one sort; say, to track a moving object. To improve this type of performance, one may require a pathway with excellent temporal resolution. Other behaviors may require precise visual information of another sort; say, to identify a texture pattern may require high spatial acuity. This might require a pathway with very high spatial sampling resolution. Rodieck et al. (1993) estimate that there may be as many as twenty pathways originating within the retina, and that each of these pathways communicates its signal to a different location in the central nervous system. Like the rods and cones, each subcategory of retinal ganglion cell obtains a fairly complete copy of the retinal image. We may presume that each subcategory of ganglion cell type specializes in communicating about certain types of visual information.

We will refer to the connected series of neurons carrying information in parallel as visual streams or visual pathways. The precise site where information is segregated into different visual streams within the retina is not certain, but it seems likely that the segregation begins immediately at the output of the rods and cones. In fact, there appear to be about 15 to 20 different bipolars that make contact with each cone. The information encoded by each of the bipolars may serve as the starting point for the visual streams that have been identified at points further along the visual pathway (Rodieck, et al. 1993; Boycott and W\”assle, 1991).

%Morphological classif of bipolar cells of the primate retina

Evidently, one of the important functions of the retina is to organize the information encoded by the photoreceptors into a collection of visual streams. Presumably, the purpose of these separate visual streams is to communicate relevant image information efficiently to brain areas engaged in specialized types of visual processing. This observation reinforces the view that the existence of these visual streams can be an important clue about the functional organization of the visual pathways. We presume that each visual stream carries an efficient representation of the spatio-temporal component of the image that is most relevant for task carried out in visual area where the ganglion cell output is sent. Hence, by studying the response sensitivity to different kinds of stimuli of the neurons within a visual stream, we learn something about the functional role of that stream.

Retinal function: image contrast and adaptation

There are some visual challenges that are common to the information carried on all of the specialized visual streams. It makes sense to try to solve these types of problems when the signals are close together, as in the retina, rather than after the visual signals have reached widely separated destinations within the brain.

A fundamental challenge that is common to the signals carried by all retinal neurons is this: they must remain sensitive as the ambient light intensity varies over many orders of magnitude. This is a challenge for the nervous system because neurons have a very limited response range.

Neurons in the peripheral visual system solve this problem, in part, by signaling the local contrast in the image rather than the absolute stimulus level. The local contrast is the percent change in the image intensity relative to the local average. The range of contrasts in a typical image is constant as ambient illumination level changes and typically spans no more than two orders of magnitude. By coding contrast, rather than absolute level, neurons with small dynamic range can convey essential information about the retinal image despite enormous variations in the absolute level. In later chapters we will review computational issues and we will find that in contrast is an important signal in its own right. The contrast signal is closely coupled to the properties of surfaces, and surfaces are often the visual entity we want to identify or recognize\footnote In some books the dynamic range problem is treated by explaining that the photoreceptors respond to light intensity using a compressive function of intensity, such as a logarithmic or power function. A compressive function maps a light stimulus ranging over six orders of magnitude into neural responses of one to two orders of magnitude above their intrinsic variability. In the modern literature, this view has been substantially replaced by a formulation based on stimulus and response contrast.}.

Visual Streams

We will begin by reviewing the kinds of methods we can use to classify retinal neurons. Then, we will review the principal features of the information carried within two specific visual streams, the parvocellular pathway and the magnocellular pathway. We focus on these two streams because we know most about them and because their output represents a very large fraction of the total output of the retina.

Methods of classifying neurons

The form and structure of a neuron, including its dendritic field, cell body, and axonal projections, are called the neuron’s morphology, The most fundamental method of distinguishing categories of neurons is simply to study their morphology. A second type of data we can use is the neuron’s electrical responsiveness to different signals, that is its electrophysiology. A third type of data we can use is to study the chemical substances used to build the neuron, that is the neuron’s biochemistryv. A fourth type of data is the anatomical pattern of interconnections a neuron makes with other neurons. The most satisfying classification of neurons occurs when the evidence from these different sources converge.

We used all of these methods to distinguish the photoreceptors into rods and cones. The rods and cones can be classified based on their morphology of the cell (rod-like shape versus cone-like shape), the type of photopigment they contain, their electrical response to light, and their interconnections (rods make no connections in the fovea). Taken together, the classification of rods and cones also suggests a difference in function, namely that cones carry visual information used for high acuity tasks and rods are specialized for low illumination conditions.

It is natural to use our successes at peripheral levels to guide our next analysis of cellular function. So, we begin the analysis of the retinal ganglion cells by considering how we can use the measurements to categorize the retinal ganglion cells into groups serving various visual functions.

Morphology of Parasol and Midget Ganglion Cells

When examining the retinal ganglion cell layer using a light microscope, one sees ganglion cells of many different sizes, shapes and patterns of dendritic fields. In an extraordinary set of studies, Santiago Ramon y Cajal examined the retinal cell types in many mammalian eyes, but no primates, and identified the basic anatomical structure of the retina. To classify neurons, Cajal used several morphological properties, relying mainly on the location of the dendritic arbor terminations. Figure 5.5 shows Cajal at work, along with one of his sketches of the mammalian retina.

The modern era of anatomical studies in the primate retina began with the work of S. Polyak (1941; 1957) who wrote a remarkable pair of books describing his investigations into the primate visual system. Polyak described many aspects of the anatomical structure of the retina specifically and the primate vertebrate visual system generally. In his work on the retinal ganglion cells, Polyak identified five different categories of cells using the size of their cell bodies and the properties of their dendritic fields. One of the principal classifications he made, and the one that will concern us here, was based on the size and spread of the retinal ganglion cell dendritic arborizations. At most locations within the retina one can identify some neurons whose dendritic fields are relatively dense and compact compared to other retinal ganglion cells. Near the fovea these neurons are particularly conspicuous since they make contact with only a single bipolar which in turn makes contact with only a single cone. Polyak was the first to identify these retinal ganglion cells which are abundant in the primate but absent in other mammals. Polyak named these cells midget ganglion cells. Several midget ganglion cells at different positions within the retina are illustrated in Figure 5.6a.

The morphology of the midget ganglion cells contrasts with a second class of ganglion cells shown in Figure 5.6b and called parasol cells by Polyak. The parasol cells have a sparse dendritic tree and medium to large cell bodies. The drawings in Figure 5.6 compare a sampling of the midget and parasol cells at several distances from the fovea.

Variation with retinal eccentricity

As Polyak noted, the size of many types of retinal neurons increases with distance from the fovea. For example, a cell body or dendritic field that is relatively large in the fovea will be relatively small in the periphery. While Polyak says that the midget ganglion cells are present throughout the retina, due to retinal inhomogeneity the midget ganglion cells in the periphery are larger than the parasol cells near the fovea. If absolute size is not a reliable indicator, what measurement can we use to decide whether neurons at different eccentricities are of the same type? In a seminal paper, Boycott and W\”assle (1974) showed how to make such a measurement in the retina of the domestic cat. The idea is simple and elegant: make measurements that span a wide range of retinal eccentricities and compare the trends within the population. Boycott and W\”assle’s methods and observations have been extended from cat to the primate and human (Perry and Cowey 1981,1984; Levinthal et al., 1981; Rodieck et al. ,1985; Watanabe and Rodieck ,1989; Dacey, 1992).

Figure 5.7 shows that the size of dendritic fields of the midget and parasol ganglion cells increase with eccentricity in the human retina. Although both cell types increase in size, within each cell type the dendritic field size varies smoothly and at each retinal eccentricity the sizes of the two populations remain distinct. The graph in Figure 5.7 suggests that the signals from the midget cells form one unified visual stream and the parasol cells form a second visual stream. \nocite{Dacey1992} %and petersen

Further evidence that these two classes of neurons form independent visual streams comes from their coverage of the retinal image. Both midget and parasol cells are present at every location within the retina. Thus, each class of neuron encodes a complete copy of the retinal image.

Although the two populations both receive a complete copy of the image, they do not encode the image at the same spatial resolution. In the fovea, midget ganglion cells receive input from a single cone (N.B. This does not mean that a cone sends its output to a single bipolar or ganglion!). From the spread of their dendritic fields, we see that the parasol cells receive convergent input from a much wider area of the retina. The fine resolution achieved by the midget ganglion cells means that more of them than parasol cells are needed to encode the entire retinal image. There appear to be 7 or 9 times as many midget cells as parasol cells (Sterling, et al., 1994; Perry et al., 1984). The midget ganglion cells encode the spatial image up to the full sampling resolution of the photoreceptors, roughly 60 cycles per degree. The smaller number of parasol cells are capable of encoding the signal up to a spatial resolution of 20 cycles per degree.

Central Projections

A second method of identifying visual streams originating in the ganglion cell layers is to consider how the retinal ganglion By injecting tracer substances that are carried from the brain back to the retina, we can identify where each type of retinal ganglion cell sends its outputs.

Perry, Oehler and Cowey (1984) studied how different cell types send their axons to brain structures. They injected a tracer substance called horseradish peroxidase into the optic nerve. When horseradish peroxidase is absorbed by a neuron, it is transported throughout the neuron. Thus, if the horseradish peroxidase is absorbed in an axon, it is transported back to the cell body. Conversely if the horseradish peroxidase is absorbed in the cell body, it will be transported down to the axon terminals. The presence of horseradish peroxidase within a neuron can be established by appropriate histochemistry. By injecting the tracer into the optic nerve, they could identify the appearance of all cell types when stained by horseradish peroxidase.

Perry, Oehler and Cowey (1984) also introduced horseradish peroxidase into the lateral geniculate, a nucleus located in the thalamus, that is a major recipient of axons from the retina. The majority of the retinal ganglion cells make a connection with this nucleus, so that the horseradish peroxidase was transported to many retinal ganglion cells. They estimate that 90 percent of monkey retinal ganglion cells send their axons to the lateral geniculate layers. While the preponderance of retinal ganglion cells containing horseradish peroxidase could be classified as either parasol or midget, there is some evidence that at least two other types of retinal ganglion cells also send their outputs to the lateral geniculate nucleus (Rodieck, et al., 1993).

There is considerable regularity in the distribution of axons from the parasol and midget neurons within the primate lateral geniculate nucleus. The primate lateral geniculate nucleus contains six different layers (see Figure 5.9). The four superficial layers contain neurons with small cell bodies and are called the parvocellular layers. The two deeper layers contain neurons with large cell bodies and are called the magnocellular layers. The axons of parasol and midget retinal ganglion cells make connections in different layers of the lateral geniculate nucleus. The axons of the midget retinal ganglion cells terminate in the parvocellular layers, while the axons of the parasol cells terminate in the magnocellular layers. In addition to the cell bodies within parvocellular and magnocellular layers, there are also cell bodies that fall in between in regions called the intercalated zones. These zones may receive signals from yet another class of retinal ganglion cells.

The consistency in the shape of the cells and their central projections suggests that the midget and parasol cells form separate visual streams. The pathway that begins with the midget ganglion cells and terminate within the parvocellular layers of the lateral geniculate nucleus is called the parvocellular pathway, while the pathway that begins within the parasol cells and terminate within the magnocellular layers of the lateral geniculate is called the magnocellular pathway. The significance of these pathways for visual perception is the source of much current experiment and speculation. How far within the visual pathways are these signals segregated? Do the signals on these pathways carry information with different and specialized perceptual significance? In the next sections I will review some of the differences in how these neurons respond to light. I will discuss experiments that address the broader topic of the perceptual significance of these pathways at several points throughout the book.

Although the majority of retinal ganglion cells send their outputs to the lateral geniculate, there are many other destinations for the optic tract fibers. For example, Perry and Cowey (1980) introduced horseradish peroxidase into the monkey superior colliculus a nucleus in the mid-brain that is known to receive input from retinal ganglion cells. They found that about ten percent of the retinal ganglion cells send axons that terminate in the superior colliculus. None of the labeled cells were midget or parasol ganglion cells. Rodieck et al. (1993) review a broad range of measurements concerning visual streams of retinal origin. They conclude that each subcategory of ganglion cell sends its output to a single destination in the brain, making the morphology of retinal ganglion cells a very important clue in determining the organization of the visual streams that originate in the retina.

The majority of the retinal output is sent to the lateral geniculate nucleus. But, the retinal connections in the lateral geniculate nucleus account for only about 10 percent of the synapses. Nearly 60 percent of the synapses in the lateral geniculate are signals from the cortex and the remaining synapses are connections with other parts of the brain (Sherman and Koch, 1990).

Conduction Time and Contrast Gain

There are several differences in the way neurons in the parvo- and magnocellular pathways code information. These differences are clues about the kind of visual information represented by these visual streams and the function these streams serve in vision.

First, the conduction time for electrical signals traveling from the optic chiasm to the parvocellular layers of the lateral geniculate nucleus is longer than the conduction time to the magnocellular neurons. Schiller and Malpeli (1979) measured the conduction time for an electrical stimulus originating in the optic chiasm to travel to different layers in the lateral geniculate nucleus. The signal arrives later in the parvocellular layers than it does in the magnocellular layers, as illustrated by the histograms in Figure 5.10

Second, the response of neurons in these two pathways to contrast patterns differs reliably. Kaplan and Shapley (1982, 1986) observed that as the stimulus contrast of a sinusoidal grating pattern increases, the response of neurons in the magnocellular pathway changes more rapidly than neurons in the parvocellular pathway. Figure 5.11 compares the contrast-response curves of a neuron in the parvocellular pathway and a neuron in the magnocellular pathway. The horizontal axis measures the stimulus contrast and the vertical axis measures the neuron’s response as a percent change from the spontaneous response level. The contrast-response curve of the neuron in the magnocellular pathway increases more rapidly with stimulus contrast and also saturates at a lower contrast. The slope of the contrast-response curve is called the neuron’s contrast-gain. We can summarize the results in Figure 5.11 by saying that magnocellular neurons have higher contrast-gain than parvocellular neurons; the contrast-gain ratio is approximately eight.

While they original observed the difference in contrast-gain in the lateral geniculate nucleus, Kaplan and Shapley went on to show that this difference can be traced to differences in the response gains of the midget and parasol neurons within the primate retina. Quite possibly, the differences in these signals may begin at the bipolar connection to the cones themselves.

Visual Information Encoded by the Parvocellular and Magnocellular Pathways

Anatomical and physiological measurements suggest that the parvocellular and magnocellular pathways carry different types of information to the brain. We can try to evaluate this hypothesis by removing one of the pathways by introducing a lesion in to the pathway and studying the changes in an animal’s performance due to the lesion.

When we perform a lesion, we must be careful not to damage fibers that are merely passing by, or else we will have lesioned remote sites that are the source or destination of the fibers. The substance ibotenic acid is particularly useful for lesion studies because it destroys cell bodies of neurons but spares axons. Ibotenic acid has been applied to lesion neurons in the magnocellular or parvocellular layers. By studying changes in performance after ibotenic lesions of the parvocellular and magnocellular pathways, we learn something about the information present on these two visual streams (Schiller and Logothetis, 1990; Merigan et al. 1991ab; Lynch et al.).

When cells in the parvocellular layers of a monkey’s lateral geniculate nucleus are destroyed, performance deteriorates on a variety of tasks, such as color discrimination and pattern detection. Since the parvocellular pathway includes more than seventy percent of the retinal ganglion cells, perhaps this result is not terribly surprising. When cell bodies in the magnocellular layers are destroyed many visual performances are unaffected. The results of several behavioral measurements before and after these lesions are summarized in Figure 5.12 (Merigan et al., 1991a).

The most informative result is this: when neurons in the magnocellular layers are destroyed, the animal is less sensitive to rapidly flickering low spatial frequency targets. This loss of sensitivity shows that the magnocellular pathway contains the best information about this aspect of the image. This suggests that the magnocellular pathway is a specialization that improves our ability to perform tasks requiring high temporal frequency information.

What type behaviors depend on the low spatial frequency and high temporal frequency information represented by the magnocellular pathway? Two examples of visual tasks that require precise and rapid information about rapidly varying image signals are motion detection and motion-tracking. The central projections of the magnocellular pathway we will review later, coupled with the significance of the perceptual signals, suggest that the magnocellular pathway plays an important role in providing high quality information used in motion perception.

What conclusion can we draw from these lesion studies? The information carried by the neurons in the magnocellular pathway provide the best information in the low temporal and high spatial frequency components of the image. Performance on motion tasks and other tasks that require this information is better when the magnocellular pathway signal is available. The signals are not absolutely necessary to perform the task. This was shown by Merigan, Bryne and Maunsell (1991a), who studied motion perception in monkeys with magnocellular pathway lesions. They found that performance deficits on motion tasks could be compensated for simply by increasing the stimulus contrast; that is, one can compensate for the loss of information on the magnocellular pathway by improving the quality of the information on the parvocellular pathway. Hence, the magnocellular pathway contains information that is particularly useful for certain kinds visual tasks, such as motion perception. Discovering where these signals are sent in the central brain should provide us with some useful ideas about where we compute and perceive motion, as well as other visual tasks requiring this type of information. \comment{ TITLE: Does primate motion perception depend on the magnocellular pathway? AUTHOR: Merigan, W H (Box 314, Univ. Rochester Med. Cent., Rochester, N.Y. 14642.); Bryne, C E; Maunsell, J H R PUBLICATION: Journal Of Neuroscience. 1991 11(11):3422-3429. J Neurosci. }

Retinal Ganglion Cell Response To Light

The output of the retinal ganglion cells consists of a series of discrete electrical impulses called action potentials or spikes. We measure ganglion cell responses by recording the temporal pattern of action potentials caused by light stimulation. Retinal ganglion cells form part of a pathway that transforms the light into a temporal series of electrical pulses. The properties of the neural transformation from a light signal to a pattern of action potentials is one of the main types of evidence we have about a neuron’s functional significance, and hence the pathway’s role in vision.

Within the field of electrophysiology, the field that studies the electrical response of neurons, the transformation associated with a neuron is called the neuron’s receptive field. The receptive field concept, first used in vision by H.K. Hartline (1939), is a cornerstone of the electrophysiolgist’s description of the action of visual neurons. The receptive field concept, like the notion of a transformation is quite general. Hence, the receptive field notion is used also to describe neural properties in other sensory and motor areas as well (Mountcastle, 1957).

Classically, the visual receptive field of a neuron was defined as the retinal area in which light influences the neuron’s response. This region can be defined by positioning small flashes of light and simple moving bars and evaluating when the neuron responds and fails to respond. The responses of many neurons in the visual pathway are influenced only by light falling within narrow regions of the retina, and hence small regions of the visual field. The region of the visual field in which these flashes of light and bars influence the neuron’s response is called the neuron’s classically-defined receptive field. This description is relatively easy to obtain and provides a useful preliminary description of the neuron’s transformation.

Although we refer to a neuron’s receptive field, in fact the receptive field depends on the properties of the entire visual pathway, beginning with the optics and including the transformation by the neuron itself. In some cases, when there is feedback descending upon the neuron from central brain regions, the receptive field we measure at a neuron includes contributions from many places within the visual pathways (though there is no feedback to the retinal ganglion cells).

Center-Surround Organization

Several important properties of ganglion cell receptive fields were discovered by measuring the classically defined receptive field. In this section we will consider how one important property, center-surround organization, was measured.

There are two locations in the visual pathways where one can conveniently record the spiking activity of retinal ganglion cells (see Figure 5.13). One can measure the electrical activity from the cell bodies of the retinal ganglion cells, which are on the surface of the retina closest to the cornea, or one can insert the microelectrode in the optic nerve which contains the axonal fibers that emerge from the cell bodies and carry the signal to the cortex. When the electrode is positioned properly with respect to the cell body of a neuron, or an axon in the optic nerve, we can record action potentials.

When the stimulus is a large uniform field, most retinal ganglion cells respond with a random stream of action potentials. A typical retinal ganglion cell response to uniform illumination might consist of 50 spikes per second. For most retinal ganglion cells, the temporal sequence of the spiking activity has no systematic temporal structure, so that the chance of a spike occurring in the next brief interval of time is approximately constant. We call the average number of action potentials per unit time, in the presence of a constant field, the spontaneous firing rate of the retinal ganglion cell.

S. Kuffler was the first to define the receptive field of of mammalian retinal ganglion cells. They used small points of light flashed at different retinal positions, and they recorded the difference between the spontaneous firing rate and the response when the point of light was presented at different points on the retinal surface. An example from Kuffler’s measurements is shown in Figure ??. A small spot of light flashed within a central region of the receptive field causes an increase in firing relative to the spontaneous activity. When the spot is placed on a surrounding area, there is a measurable decrease in the cell’s activity. The intermediate region showed some excitation at the beginning of the stimulus and some inhibition at stimulus extinction (Kuffler, 1953; Barlow, et al., 1957).

The responses illustrated in Figure 5.14 define an on-center, off-surround receptive field. About half of the retinal ganglion cells respond this way. The remaining ganglion cells are inhibited by light falling on the center and excited by light falling on the surround. These are called off-center, on-surround cells. Most mammalian retinal ganglion cells exhibit the basic center-surround organization.

The dendritic fields of retinal ganglion cells with on-center are segregated in the inner plexiform layer of the retina from retinal ganglion cells with off-center receptive fields. Ganglion cells that make connections in the upper portion of the inner plexiform layer have an off-center, while neurons that synapse in the lower half of the inner plexiform layer have on-center receptive fields. Hence, the anatomy and electrophysiology suggest that there are at least two types of visual pathways, an on-center and an off-center pathway, emerging from the retina.

%In the human retina, the receptive field sizes of

%on-center neurons appears to be somewhat larger than the receptive

%fields of off-center neurons (Nelson et al., 19XX).

\nocite{Nelson et al, Kolb …} \nocite{BoycottandWassleReviewArticle} \nocite{Dacey1992} % and Petersen, PNAS

Measurements of Receptive Fields

The classical receptive field is only a partial description of the neuron’s response properties. To understand a neuron’s transformation of the light signal completely, we would like to predict the pattern of action potentials in response to any visual stimulus. To describe the transformation from light to neural activation completely, we need to develop a more systematic method of measuring the neuron’s response. Since there are many possible visual stimuli, we need to develop a method so that we can make a small set of measurements and then use these measurements to predict the responses to all other stimuli.

Linear systems theory provides some guidance on the question of how to measure the receptive field completely. If a neuron’s responses to light satisfies the principle of superposition, then we can use a few measurements to predict how the neuron will respond to many other visual stimuli. As we shall see, in the primate retina and lateral geniculate nucleus, linearity provides a satisfactory account of a large part of the neural response.

How do we test whether the input-output relationship of a retinal ganglion cell satisfies the principle of superposition? A simple and direct test, for one pair of stimuli, is shown in Figure 5.15. In this study Enroth-Cugell and Pinto (1970) examined the center-surround antagonism of cat retinal ganglion cells. They studied the superposition of two stimuli: a spot placed in the center of the ganglion cell receptive field and an annulus placed in the antagonistic surround. The response to the spot is illustrated in part (a) of the figure and the response to the annulus is illustrated in part (b).

If the ganglion cell response obeys the principle of superposition, we should be able to predict the temporal response when we present the spot and the annulus together. The observed response and the predicted response are shown in panels (c) and (d) of the figure. They are compared in panel (e) of the figure. For this pair of stimuli, and this retinal ganglion cell, the principle of superposition predicts the neuron’s complete response very well.

Steady-state Measurements

To build a complete description of the neural response properties, we must include time in our characterization of the neuron’s receptive field. We will consider the full space-time receptive field later in this chapter. But, it is simpler to begin with an example in which we eliminate time as a variable; we will consider only the response after the stimulus has been presented for several seconds and the neuron’s response has stabilized. This asymptotic response is called the steady-state response of the neuron. By measuring the steady-state response of the neuron, we remove time as a factor in our analysis.

As in all cases of linear systems studies, we must specify both the input and output signals carefully. One of the important advances in recent years has been discovering insightful definitions for the input and output stimuli we use to measure the neurons receptive fields. Both of these definitions are based on the notion of contrast.

\paragraph{The stimulus.}

We use a spatial image as the input stimulus. We will represent the spatial image as the sum of two components. One part is the average light level of the stimulus, and the second part is the variation of the intensity about the mean. The mean level of the stimulus is always a positive number. The variation around the average light level is the stimulus contrast pattern. The stimulus contrast contains both positive and negative values.

Consider the simple case of a one-dimensional stimulus that varies along one spatial dimension and is constant in the second dimension. We can specify the stimulus contrast with respect to a single spatial variable, say . The formula that relates the stimulus intensity to the mean background intensity and the stimulus contrast is

(1)  

where is the stimulus intensity, is the mean background intensity, and is the stimulus contrast.

\paragraph{The response.}

Suppose that a neuron’s spontaneous firing rate is . Now, suppose we introduce a contrast pattern, , and the neuron’s steady-state response becomes spikes per second. We define the change in the retinal ganglion cell rate of response, , as the neuron’s response. Thus, just as we consider the input to be the stimulus change from the mean background level, so too we consider the neuron’s output to be the change from the average spontaneous response of the neuron.

\paragraph{Testing contrast linearity.}

To test superposition, we must measure the response to two different contrast patterns and their sum. Suppose we use two contrast patterns, and and corresponding changes in the steady-state response of and . To test superposition we combine the two contrast patterns to form a new stimulus,

(2)  

We expect that the steady-state response to the new test pattern will be . Be sure to compare Equation 1 and 2; the new stimulus pattern is formed by adding together the {\bf contrast terms}, and , not the two intensity patterns.

If the change in the neuron’s response satisfies the principle of superposition, we can model the response to any contrast pattern by defining a system matrix that describes the neuron’s transformation, i.e., receptive field. The relationship between the contrast stimulus, the system matrix, and the response is shown in Figure 5.16a. We represent the one-dimensional stimulus contrast pattern as a column vector. The output is the retinal ganglion cell response. If the neuron is linear, there is a matrix that maps the input contrast vector to the neural response. The system matrix is a matrix whose entries define the one-dimensional receptive field of the neuron. We can use the system matrix to predict how the neuron will respond to any one-dimensional contrast pattern. We say that the entries of the matrix define the linear, steady-state, receptive field of the neuron.

By examining the matrix tableau, you can see that to estimate the entries of the system matrix we need to measure the response to a series of lines at different positions on the retina. The response to each line defines the corresponding entry in the system matrix. The curve in Figure 5.16b is a graphical representation of the entries of the system matrix, that is, of the neuron’s one-dimensional receptive field.

The Two-Dimensional Receptive Field

By measuring with points of light rather than lines, we can measure a two-dimensional steady-state receptive field. Figure 5.17 shows two ways to represent the two-dimensional receptive field of a retinal ganglion cell. The height of the curve in Figure 5.17a shows change neuron’s light response as we stimulate with a point of light at different locations in the visual field. The large positive values in the center of the diagram indicate that this neuron is excited by stimuli in a central region. The negative values in the surrounding region show the inhibition by point stimuli surrounding the central region. Notice that the effect at each point in the inhibitory surround is very small, but the inhibitory surround covers large area compared to the excitatory center. The picture captures quantitatively the center-surround antagonism that Kuffler discovered in cat retinal ganglion cells. Figure 5.17b represents the same measurements as an image. Light regions show where the neuron is excited by a spot and a dark regions show where the neuron is inhibited.

Contrast Sensitivity Functions

Often, it is useful to characterize the response of a linear system in terms of the system’s response to harmonic functions. It is helpful to understand the information encoded by neurons by plotting their response to harmonic functions as well.

To measure a neuron’s contrast sensitivity function, we determine the amount of contrast necessary in the stimulus that is required to elicit a criterion level of response from the neuron. When a contrast pattern is ineffective at influencing the neuron, we need to present the pattern at high contrast to elicit the response. When a pattern is well-suited to the neuron’s receptive field, a small amount of contrast will elicit the criterion response level. We call the amount of necessary to elicit the criterion response the contrast threshold. The inverse of contrast threshold is called contrast sensitivity.

From Figure 5.18, you can see how a center-surround ganglion cell will respond to cosinusoidal patterns whose peak is centered over the receptive field of the neuron. When the spatial frequency is very low, a bright bar in the stimulus covers both the excitatory center and the inhibitory surround and the steady-state response is small. The most effective spatial frequency has bright bars imaged on the excitatory part of the linear receptive field, and dark bars on the opposing surround. This spatial frequency is well-matched to the receptive field and we will observe a strong neural response. If the frequency is higher still, parts of the cosinusoid greater and less than the mean both fall within the excitatory and inhibitory regions. The net effect of the stimulus averages out to a small response, so that high spatial frequencies are ineffective stimuli.

We summarize the neuron’s response to harmonic functions at a range of spatial frequencies using the contrast sensitivity function. Different aspects of the function provide us with information about the neuron’s spatial receptive field. The most effective spatial frequency provides information about the overall size of the receptive field. The extent of the fall off in sensitivity at low spatial frequencies provides information about the strength of the opposing surround. Finally, the fall-off in sensitivity at high spatial frequencies describes the size of the receptive field center since the highest spatial frequency the cell responds to is limited by the size of the receptive field center. Neurons with small receptive field centers respond well to high spatial frequency targets, while neurons with large centers do not.

Figure 5.19 shows a contrast sensitivity function from a linear cells in the parvocellular layers of a monkey lateral geniculate nucleus. The receptive fields of these neurons are indistinguishable from the receptive fields of midget ganglion cells\footnote{To describe a general linear receptive field, we must measure the neuron’s response using both sinusoidal and cosinusoidal contrast patterns. The receptive fields of retinal ganglion cells can be measured using only cosinusoids centered on the peak because the receptive fields are even-symmetric. A function is said to have even symmetry if . A function has odd symmetry if . When a receptive field is even-symmetric, it will have zero response to any odd-symmetric inputs, so we need to measure only the response to even-symmetric inputs. For retinal ganglion cells, then, the contrast sensitivity function is a complete description of the receptive field in this case.}.

The contrast sensitivity function of a retinal ganglion cell is an alternative way to represent the cell’s receptive field. There are several characteristic features of retinal ganglion cell contrast sensitivity functions. First, retinal ganglion cell contrast sensitivity functions are single-peaked. Single-peaked contrast sensitivity functions are called bandpass and the peak-frequency is called the center frequency. With experience, the contrast sensitivity function becomes an intuitive way to understand some aspects of the neuron’s receptive field.

Why contrast patterns are important

In early studies of the neural response to light, the stimulus intensity, , was treated as the input variable. In this case, the linear systems methods fail severely. If we wish to apply linear methods to characterizing the responses of retinal ganglion cells, it is important to fix the mean level, and to treat the stimulus contrast, , as the input.

Formulating our experiments in terms of contrast does not make a nonlinear system linear. The nonlinear behavior of neurons becomes quite clear when we compare measurements at different mean levels. But, by organizing our measurements around the contrast responses at a fixed mean level, we can use linear methods to characterize the receptive field for perturbations around the mean level. To describe the neuron fully, we must combine the neuron’s response across many mean intensity levels. This forces us to acknowledge the nonlinear aspects of the neuron’s response.

The change in the system performance as we vary the mean stimulus level is called visual adaptation. We can measure the effects of visual adaptation in individual neurons, beginning in the retina, as well as in the behavior of animals and people. The way in which performance of neurons and people change as the mean illumination varies, that is visual adaptation, is one of the most important topics in vision. By formulating our stimulus in terms of a mean intensity level and a contrast pattern, we segregate out the effects of visual adaptation from the local contrast effects. Thus, formulating our experimental input this way is an important decision that has ramifications for how we study many important aspects of vision.

Most of modern vision science relies uses contrast as the key experimental parameter. The linearities that I will describe in this chapter and the following chapters are usually measured in the contrast domain, at a fixed mean level. As we change the mean level, the properties of the linear system we estimate change as well. This is a fundamental non-linearity in the system’s behavior. A complete theory must weave together how the locally linear responses, on a single mean background, vary with the mean. We will review this topic, called adaptation, at the end of this chapter and then again in later chapters.

Connections to Different Cone Types

In the primate fovea, the center response of the midget ganglion cells depends on the light captured by a single cone. Hence, the centers will inherit the wavelength sensitivity of the photopigment in that cone’s outer segment. Parvocellular Pathway neurons with receptive fields near the primate fovea often have a center response that is either an or cone. roughly equal to the optical blur imposed by diffraction (2.4 – 4.2 minutes of arc), which is consistent with the center response being due to a single cone.

There are two views concerning the connections of cones to the opponent-surround responses in ganglion cell receptive fields. Lennie (1980) suggested that the surround is driven by a random collection of cones, including both the and types. Reid and Shapley (1992) attempted to measure the surround and concluded that only a single class of cones contributes to the surround response. When the center response of a cell in the parvocellular pathway is from a cone, the surround response is due to cones. Conversely, when the center is from the the surround is from the . At the moment, the question of the segregation of cone signals in the surround of parvocellular pathway neurons is unresolved (see also, Gouras and Evers, 1989).

There is some agreement that the centers and surround of neurons in the magnocellular pathway receive a signal from both cone types. An on-center parasol cell receives an excitatory signal from the and cones, while the the surround signal is inhibitory and originates in both of these cone classes. The relative strength of the signals from these cone classes to the surround may vary (Derrington et al., 1984).

Mariani (1983) and Dacey and Lee (1994) have shown that the signals from the cones are coded within a specialized visual stream. Neurons that receive a signal from the cones have large receptive fields (18 min). As I noted in Chapter 2, the chromatic aberration of the optics is very strong in the short-wavelength region. Since the image from a short-wavelength light source will be blurred, the large size of these receptive fields is not necessarily a disadvantage. Mariani’s anatomical studies described the existence of bipolar neurons that make contact with a few widely spaced cones that appeared to be consistent with the cone mosaic. His observations suggest that the spatial connectivity of the cone signals also plays a role in creating large center receptive fields (see also, Koutama and Marschak 1992).

D. Dacey and B. Lee have confirmed the existence of a visual stream specialized for carrying information about the cone signal and made several important and novel observations. First, Dacey showed that there is a morphologically distinct type of retinal ganglion responsible for carrying the cone signal. These bistratified ganglion cells have dendritic trees that are stratified into two tiers near the inner and outer borders of the inner plexiform layer where the on- and off-center receptive field neurons stratify. Based on the anatomical connectivity of these neurons, Dacey suggested that these neurons carry a cone excitatory signal. Using electrophysiological measurements, Dacey and Lee made two additional and surprising observations. First, their sample of midget and parasol ganglion cells contained no input from the cones. Second, they showed that all of the bistratified ganglion cells had a excitatory input (Dacey, 1993; Dacey and Lee, 1994).

The bistratified neurons send their outputs to the parvocellular layers of the lateral geniculate nucleus (Rodieck et al., 1993). Using electrophysiological methods, one can measure receptive fields in that nucleus whose excitatory centers are driven by signals from the cones. It is also possible that these neurons project to the intercalated zones of the lateral geniculate. Because the sampling resolution of the cone mosaic is poor, we do not expect these neurons to make up a large fraction of the total population. Nonetheless, they are important since at present they are the only cell class known to carry the cone signal (Wiesel and Hubel, 1964; Gouras, 1969; Derrington et al., 1984).

Spatio-Temporal Analysis: Lines and Spots

Up to now, we have considered only the steady-state response, thus excluding time in order to simplify our analysis. Now we consider how to include the temporal response of the neuron in our measurement of the receptive field.

Figure 5.15 illustrates an average temporal response of a cat retinal ganglion cell to a flashed target. To obtain the curves shown in the figure, the experimenters presented the test light and recorded the resulting neural activity. If we present the test flash repeatedly, we will find that the resulting pattern of spikes differs slightly each time. The differences will be small, however, so that we can sum together all of the responses obtained from, say, 50 repetitions of the test flash. We can compute the average number of action potentials at each moment in time following the flash, and plot this as a curve. This curve is called the peri-stimulus time histogram, (PSTH).

The data in Figure 5.15 and other experimental measurements have shown that for certain cells in the cat retina, linearity holds rather well (Enroth-Cugell and Robson, 1984). Linearity has not been extensively tested in primate retina. To a fair approximation, linearity has been confirmed in measurements in the parvocellular pathway within the primate lateral geniculate nucleus (Derrington and Lennie, 1984). Hence, based on linear methods we can measure the responses to a collection of basic stimuli and use these responses to predict the responses to many other stimuli.

Figure 5.20a shows two simulated PSTHs for an on-center cell response to a briefly flashed line. One PSTH shows the simulated response to a line flashed over the center of the receptive field, and the second PSTH is for a line flashed over the opposing surround. By measuring the responses to briefly flashed lines at many receptive field positions, we can specify the space-time receptive field of the neuron (Stevens and Gerstein, 1976). I have collected a series of these simulated responses in a surface plot shown in Figure 5.20b. One axis of the figure measures time and the second axis describes spatial position of the test line. Each curve along the time axis shows a simulated PSTH for the for a single spatial position of the line. When the position of the line is in the receptive field center, the simulated response is large and begins soon after the stimulus. When the line is positioned over the receptive field surround, the simulated response is weaker, delayed, and of opposite sign.

Since any one-dimensional time-varying stimulus is the sum of a set of briefly flashed lines, a collection of measurements like the simulations in Figure 5.20b permit us to predict the response to any such stimulus. Such measurements define the space-time receptive field of this simulated neuron for one spatial dimension. This method of defining space-time receptive fields has been applied mainly to study the cat visual pathway (Stevens and Gerstein, 1976; Palmer and Davis, 1981; Emerson et al, 1992).

Spatio-Temporal Measurements: Harmonic Functions

It is also possible to measure receptive field properties using harmonic functions, though in this case, we need to use harmonic functions in space and time. We can create space-time harmonic functions by multiplying cosinusoidal contrast patterns in space and time, creating a stimulus called a contrast-reversing grating. Such a pattern is the product of a spatial harmonic with frequency cycles per degree and a temporal harmonic with frequency Hz. The formula for a contrast-reversing grating made from cosinusoids is

(3)  

where is the mean background intensity.

Figure 5.21 shows the contrast-reversing in two different forms. At each moment in time the grating is a one-dimensional spatial frequency grating, with spatial frequency . At each point in space the time-varying contrast is a cosinusoidal function of time, with temporal frequency . In part (a) of the figure the function is shown as a surface plot. One axis of the plot represents time and the other space. Individual lines in both of these dimensions are sinusoidal. In part (b) of the figure the function is shown as an intensity image. Again, one dimension of the image represents time and the other space. The image intensity is bright at those points where the function has positive contrast and dark where the function has negative contrast.

Neurons in the lateral geniculate nucleus respond linearly for stimuli as high as 30 percent contrast. Hence, it is appropriate to use linear methods to measure the spatial-receptive fields of these neurons. Moreover, since the receptive field of neurons in the retinal and lateral geniculate nucleus in the parvocellular pathway are even-symmetric, we can measure the space-time contrast sensitivity function using only cosinusoidal functions centered on the receptive field. Because the neurons are linear and temporally shift-invariant, the PSTH is a harmonic function at the same temporal frequency of the input stimulus, . The amplitude and phase of the PSTH modulation depends on the temporal and spatial frequency of the contrast-reversing pattern used as an input.

The response to contrast-reversing patterns of a parvocellular neuron in the monkey lateral geniculate nucleus is shown in Figure 5.22. This surface plot was derived by interpolating measurements reported by Derrington and Lennie (1984). This cell responded to temporal and spatial modulations up to 15 Hz and 15 cycles per degree. From the shape of the neuron we can make several observations about the neuron’s responsivity. First, the reduced sensitivity at low spatial frequencies shows that the neuron had a significant opponent surround. Second, the neuron responds well to all tested temporal frequencies when we measured with a low spatial frequency stimulus, but the neuron responds poorly at high temporal frequencies when measured using a high spatial frequency stimulus. Since the response to the high spatial frequency stimulus is mediated through the on-center, the data show that the temporal sensitivity from the on-center and opposing surround may be different. It is time to consider the general question of the interdependence of spatial and temporal sensitivity.

Space-time separability.

We have defined the neuron’s receptive using two complementary representations. In one case, the we have defined the receptive field using briefly flashed lines, and in the second case we have defined the receptive field using harmonic functions of space-time. We can use either method to measure and describe a linear neuron’s receptive field.

No matter which description we use, the neuron’s sensitivity depends jointly on space and time. When our description of the receptive field includes both space and time, how can we define a neuron’s spatial receptive field? Or, how can we define its temporal receptive field?

Suppose that we refer to the neuron’s space-time receptive field, illustrated in Figure 5.20b, using the function . This function defines the response of the neuron to a line briefly flashed at position at time following the flash. We can ask whether a neuron has a unique spatial receptive field by examining this function at several different moments in time. That is, we might fix time at a value of and consider the function of position, , that defines the spatial receptive field at time . We can compare this spatial receptive field with the measurements at a second time, say, . We would say that the neuron has a unique spatial receptive field when the two functions are essentially the same, say,

   

where is a scalar constant. If the spatial receptive field at each moment in time is the same except for a constant scale factor, then we say the neuron has a well-defined spatial receptive field.

A neuron will only have a well-defined spatial receptive field when the space-time receptive field is a separable function of space and time. Separability means that the receptive field function can be written as the product of two functions, one that depends only on space and the other that depends only on time, namely

(4)  

If the space-time receptive field is separable, then, it follows from the definition that the spatial receptive fields at two times will always be related by the scale factor . In this case the function is a meaningful definition of the neuron’s spatial receptive field. The function is a meaningful definition of the neuron’s temporal impulse response function. If the space-time receptive field is separable, then it is possible to show that the space-time contrast sensitivity function will also be separable.

As the plots in Figure 5.20b and Figure 5.22 show, the space-time receptive fields of neurons in the parvocellular pathway are not space-time separable. Hence, these neurons do not have unique spatial or temporal response properties\footnote{ Although I will not go into the details here, by considering the connections to specific cone types you can convince yourself that the receptive fields will not be separable with respect to space and wavelength.} In part because of this complexity, it is important to go beyond the descriptions of the neuron’s response we obtain from linear measurements, and to build a model to predict the neuron’s response to space-time patterns.

The Difference of Gaussian Model

In the mid 1960s, R. W. Rodieck and Enroth-Cugell and Robson introduced a linear receptive field model that has served as the basis for most subsequent models of linear retinal ganglion cells. The basic model is important in vision science broadly since the ideas in the model have been used in many different areas including work in the visual psychophysics of spatial perception and in computer vision vision work addressed to edge-detection and image segmentation. The receptive field model is now called the Difference of Gaussian model (Rodieck, 1965; Enroth-Cugell and Robson, 1966, Enroth-Cugell, Robson, Schweitzer-Tong and Watson, 1983).

The difference of Gaussian model supposes that the neural response results from the combined signal of two separate mechanisms called the center and the surround. The center mechanism receives all of its input from a small central region and the surround mechanism receives its input from a region that includes the center and the surrounding region.

Both the center and surround are assumed to respond to contrast stimuli as space-time separable linear systems. Because each mechanism is separable, we can describe them as having meaningful spatial and temporal sensitivities. The curves describing the spatial sensitivity of the center and surround mechanisms are shown in Figure 5.23a. Both curves follow the shape of a Gaussian distribution, also known as the normal curve.

According to the difference of Gaussian model, the output of the neuron can be predicted by summing together the temporal response from the center mechanism with a temporal signal from the surround mechanism. The model further assumes that the temporal response from the surround is different from the temporal response of the center, generally being slower to develop over time. Because of this delay between the center and surround responses, the behavior of the cell as a whole is not space-time separable even though the responses of the component mechanisms are space-time separable. Hence, the model neuron does not have a unique spatial receptive field, nor does it have a unique temporal response function.

The difference of Gaussian model has been useful to vision scientists in several different ways. First, the model simplifies our calculations and thinking about receptive fields. The difference of Gaussian model predicts the response of neurons from a calculation that requires us to specify only a few unknown parameters, such as the width of the center and surround mechanisms receptive fields. The model provides a very efficient method of describing neural space-time receptive fields, and thus a convenient way of comparing the receptive fields of neurons.

Just as important, the model provides a framework for posing new and interesting questions. You should notice that the idea of a center and surround mechanism are entirely theoretical. I have not offered any direct proof of their existence. By building a model to efficiently describe the results, we have generated a new hypothesis about how the responses of neurons are created. In probing to identify whether these are merely useful theoretical tools, or whether there are true anatomical center and surround pathways, we will be led to explore new and interesting aspects of the visual encoding of light in the retina.

Finally, the model is specified in enough detail that it can be used in a variety of branches of vision science. As we shall see in later chapters, the difference of Gaussian model has been used as a key element to describe some of the limitations in human visual sensitivity to patterns. The model has also been used to describe certain aspects of how computational models of how images are represented in the visual pathways.

Retinal Light Adaptation

We have divided the study of receptive fields into two parts. We began by studying the neuron’s response to contrast patterns presented on a fixed mean background. Contrast patterns are perturbations about the mean background field; by studying the responses to these stimuli we have been able to apply linear methods successfully to specify the transformation from light signal to neural impulses.

When we measure with respect to contrast instead of absolute intensity tests of linearity have a better chance of succeeding. To a significant degree, this is because contrast stimuli limit the range of intensities in the the image. For example, the peak intensity in a 100 percent contrast sinusoidal pattern is only a factor of 2 greater than the mean background intensity. The range of contrasts that we encode in a typical image, from the least contrast we can detect to 100 percent contrast, is no more than 2 orders of magnitude.

Through the course of a day, however, the range of image absolute intensities we experience typically exceeds 6 orders of magnitude. The visual pathways do not remain linear over this enormous range. When we study the response to contrast stimuli, we are attending to the local response of a globally nonlinear system. To fully understand the properties of neurons, we must also analyze how their responses change as we vary the mean background level. The changing neural and behavioral responses as a function of mean background intensity varies is called visual adaptation. In this section we will consider how to the locally linear measures we have developed using contrast must be extended when we consider the visual pathways over a wider range of mean signal levels.

Contrast Sensitivity: Dependence on Mean Intensity

Figure 5.24a shows several contrast sensitivity functions from a cat retinal ganglion cell. These functions were measured on a wide range of mean intensities. When the mean intensity is relatively low, the neuron responds poorly to relatively high spatial frequencies\footnote{ In general, cats have very poor spatial resolution compared to primates. The spatial frequency range of this neuron, and cats performance as measured behaviorally, is far below the normal range for primate visual acuity.} At low background intensities the contrast sensitivity function shows little bandpass behavior, suggesting that there is little effect of the inhibitory surround. As the background intensity increases, the contrast sensitivity function changes shape. The reduced sensitivity to low frequency sensitivity stimuli becomes apparent, indicating that the opponent surround is more significant.

The change in the contrast sensitivity function with mean background intensity is a clue about how visual adaptation compensates for the change in mean background intensity. At low mean levels the neuron simply sums all of the quanta incident within the receptive field. At these low levels there is little surround inhibition, and thus there is little fall-off in the low frequency portion of the contrast sensitivity function. At higher mean levels, when quanta are more abundant, and the spatial receptive field of the neuron becomes relatively more sensitive to stimuli whose intensity varies within the receptive field. The opposing surround means that the neuron no longer sums the response to every quantum. Instead, the neuron responds better when there is variation in the intensity level within the spatial receptive field.

At the high mean background intensities, the contrast sensitivity functions are rather similar. For example, the three highest mean intensities used in the measurements in Figure 5.24 differ by a factor of 100. Yet, in the low spatial frequency range the contrast sensitivity curves measured at these levels differ only by about a factor of three. Panel (b) in the Figure shows the relatively small change in contrast sensitivity with mean background by comparing the cell’s contrast sensitivity to a 0.2 cpd pattern measured on different background intensities. As mean intensity changes over five orders of magnitude, the contrast sensitivity to this pattern varies one order of magnitude.

The relative constancy in contrast sensitivity reinforces the view that the response of the neuron is more closely coupled to the stimulus contrast than the absolute intensity level. This suggests that the contrast variable may be the key stimulus variable represented by the neuron’s activity.

Image contrast and image intensity are inter-related quantities. The relatively constant sensitivity to contrast implies that the neuron’s sensitivity to absolute light level varies with changes in mean intensity level; Figure 5.25 shows this variation in sensitivity to light. The vertical axis of the graph measures the logarithm of the light intensity of an incremental flash needed to cause a fixed increase in firing rate from the spontaneous rate. This is a measurement of the threshold sensitivity to the incremental test flash. The horizontal axis of the graph measures logarithm of the mean background intensity. Since the slope of the increasing portion of the graph is close to one, we can conclude that the threshold increases roughly in proportion to mean background intensity. This relationship between threshold and background level was first discovered from measurements of human behavior. The relationship is called Weber’s law, to honor the scientist who first discovered the principle.

As the mean intensity of the background varies, we can find other aspects to the change in the neural response as well. Plotted as insets to the figure are the PSTHs of the measurements to these incremental threshold flashes. The timecourse of the response to the incremental flash varies with mean intensity level, becoming somewhat brisker with sharper signals at the onset and offset of the test flash. Thus, we see that the temporal response of the neuron, like the spatial contrast sensitivity, varies with mean intensity level. These adaptations probably occur to take advantage of the improved quality of the signal available at higher light levels.

Comparison with Behavioral Contrast Sensitivity

Sensitivity to contrast and sensitivity to absolute light level are complementary measures. When the threshold to the absolute light level is proportional to the background, as Weber’s law predicts, sensitivity to contrast will be precisely constant. In most cases, however, Weber’s Law is only a rough approximation. For example, consider the contrast sensitivity functions plotted in Figure 5.24. Were Weber’s law exact, all of the contrast sensitivity functions would overlap. Plainly, this is not so. Moreover, the quality of the Weber’s law approximation depends on the spatial pattern of the stimulus. The approximation is better for low spatial frequency patterns than for high spatial frequency patterns.

Reasoning based on these types of approximations is part of the challenge confronting the student of biological systems. The laws we discover don’t have the same precision as physical laws. It is often a question of judgment as to whether the approximation to a law we see in a biological measurement is adequate to support a principled view about the function of a neuron or a visual pathway. In this case, there is some consensus that contrast is a key variable reported out by the retinal ganglion cells. In part, the consensus has developed from the neural response data we have reviewed in this chapter. And, in part, the consensus depends on the analysis that we will undertake later in this volume, in which we consider the important signals for visual function. \nocite{ShapleyReviewArticle,WeberCitation}

There is a third type of evidence that we should consider as well: this is the question of whether the properties of the contrast sensitivity function we measure at the level of individual neurons can be detected in the properties of the animal’s behavior. In Chapters 2, 3, and

The retinal ganglion cell contrast sensitivity functions have a counterpart in the behavioral contrast sensitivity functions. Pasternak and Merigan (1981) measured behavioral contrast sensitivity functions in the cat on a variety of mean background intensities (see Figure 5.26). The behavioral contrast sensitivity functions parallel the neural contrast sensitivity functions. For example, as the mean background intensity increases the behaviorally measured contrast sensitivity functions become more bandpass. Also, as the mean background intensity varies over six orders of magnitude, the contrast sensitivity changes by only one order of magnitude. Hence, there is good qualitative agreement between the response sensitivities of individual retinal ganglion cells and the animal as a whole.

The agreement between neural measurements and behavioral measurements demonstrates that the information present in the responses of individual neurons is similar to the information available to the cat making its behavioral judgments concerning the presence of the contrast pattern. This does not mean, however, that the responses of individual retinal ganglion cells govern the animal’s behavior. We have already seen, for example, that the information encoded about wavelength by the cone photoreceptors is equivalent to the information available to the human when making a color-match. Yet, it is certain that when we formulate our conscious decisions about a color-match we do not have access to the information at the cones themselves. The agreement between neural and behavioral responses is an abstract one, an agreement at a theoretical level. We shall see this type of comparison again as we move on to study visual cortex and human behavior. As we do, remember that a good method of comparing neural and behavioral responses is to analyze the information available at a point in the visual pathway with the information available to the human making a behavioral decision.