Functional neuroimaging in human subjects and single cell recordings in monkeys show that several extra-striate visual areas are activated by visual motion. However, the extent to which different types of motion are processed in different regions remains unclear, although neuropsychological studies of patients with circumscribed lesions hint at regional specialization. We, therefore, studied four patients with unilateral damage to different regions of extrastriate visual cortex on a series of visual discrimination tasks that required them, to a different extent, to integrate local motion signals in order to correctly perceive the direction of global motion. Performance was assessed psychophysically and compared with that of control subjects and with the patients' performance with stimuli presented in the visual field ipsilateral to the lesion. The results indicate considerable regional specialization in extra-striate regions for different aspects of motion processing, namely the largest displacement from frame to frame (D-max) that can sustain perception of coherent motion; perception of relative speed; the amount of coherent motion needed to sustain a percept of global motion in a particular direction; the detection of discontinuities within a moving display; the extraction of form from motion. It was also clear that a defect in local motion, i.e. D-max, can be overcome by integrating local motion signals over a longer period of time. Although no patient suffered from only one defect, the overall pattern of results strongly supports the notion of regional specialization for different aspects of motion processing.
2AMCT = two-apertures motion coherence test
CSD = constant direction
MCT = motion coherence test
MDT = motion discontinuity test
RSD = random direction
The publication over 20 years ago of an astonishing neurological patient who was so impaired on visual motion perception as to be categorized ‘motion blind’ (Zihl et al., 1983), opened the door to psychophysical studies of visual motion abilities in patients with lesions involving the many, subsequently discovered, motion responsive cortical areas. There are very few patients indeed whose perception of motion is totally or almost totally destroyed by brain damage (Zihl et al., 1991; Vaina et al., 1990; Vaina, 1998). In such instances the lesion is invariably large, bilateral, and includes a significant portion of the extra-striate visual cortex and/or the underlying white matter. In contrast, the abolition of colour vision in cortical achromatopsia is more common and can be caused by a much smaller lesion, typically centred on the lingual gyrus and caudal part of the fusiform gyrus. The likely explanation for the striking difference is that different aspects of visual motion are processed in a variety of relatively widely dispersed extra-striate visual areas and beyond in the parietal and temporal lobes, which are supplied by different blood vessels (e.g. PCA or inferior branches of MCA), and it is unlikely that in the case of a single stroke all of these areas would be involved. It is also unlikely that brain damage of a different aetiology would permanently affect all these areas at the same time. Thus, in most cases there are residual neuroanatomical substrates for the perception of particular aspects of visual motion. This selective sparing makes it possible to study the human visual motion system with psychophysical techniques in order to look for dissociations of motion perceptual abilities in patients with different lesions and hence to make functional and anatomical inferences about the independence of motion mechanisms and about their likely neuronal substrate in humans (e.g. Vaina et al., 1990, 1999; Schenk and Zihl, 1997a, b; Clifford and Vaina, 1999). Many neuropsychological studies of acquired deficits of motion processing rest on the assumption that visual motion processing is strictly hierarchical; thus, a deficit in a low level (e.g. local-motion) task, such as the direction or speed of a single moving object, would necessarily entail a deficit on higher level motion tasks, such as extracting form from motion. An exception to this was first demonstrated in the study of patient AF (Vaina et al., 1990), whose local motion mechanisms were severely impaired but whose perception of biological motion and three-dimensional form from motion were unimpaired. In the present paper we take this kind of investigation a stage further and specifically ask whether the motion system at the cortical level is not strictly hierarchically organized, and whether the psychophysically well-studied mechanisms of motion integration and segregation can be selectively and differentially disrupted by brain lesions. Integrative processes are those leading to the spatial and temporal amalgamation of moving elements into a perceptual whole, while segregation processes lead to segmentation of the visual scene and to motion contrast (Braddick, 1993). Such processes are essential for object perception and scene segmentation. Furthermore, we investigate whether these motion integration mechanisms might help patients to compensate for selective deficits of local motion processing.
Subjects and methods
Normal right-handed naive observers with good acuity and contrast sensitivity and no known neurological or psychiatric disorders and four patients (three male and one female) with focal unilateral brain damage resulting from a single stroke participated in a psychophysical study of their motion perception. The normal subjects, drawn from family members or friends of patients, were matched by age and background to the patients, i.e. the mean age was roughly the same, as was the spread of ages. Two thirds of the control subjects were female, but this should not matter as the performance of the patients was also compared in their impaired and normal hemifields. Every data point reported here resulted from at least two independent testing sessions. All the stimuli were displayed for 1 s on each trial. In the first five experiments, we used an adaptive staircase procedure in order to determine discrimination thresholds (Saiviroporoon, 1992). Threshold was calculated as the mean of the last six reversals in the staircase. In Experiment 6, we used constant stimuli and details are described and discussed in the text. All stimuli were generated by a Macintosh computer and displayed under computer control on the Macintosh RGB monitor. Subjects were seated comfortably in a dark room and viewed the computer screen from a distance of 60 cm. They were asked to maintain fixation on a small bright 0.5 × 0.5 square degree white mark, placed at midline level 2° to the left or right of the outer margin of the stimulus area. For every trial of every experiment one experimenter sat just to one side of the display, facing the subject, and watched the subject's eyes and the position of the specular reflection from the display with respect to the pupil of one eye. Although tiny eye movements would be invisible, it was straightforward to detect a movement of several degrees that might bring the display within the good hemifield during the stimulus presentation. This occurred rarely and such trials were immediately rejected and repeated.
Six psychophysical tasks were used with all four patients (only three patients participated in Experiment 6) and with subsets of age matched normal control subjects. The experimental conditions in the first two psychophysical tasks, D-max (2 frames and 6 frames) and Speed Discrimination [random walk and constant direction (CSD)], were designed to assess, first, mechanisms of local motion measurement and, second, the mechanism of integration of motion information time (temporal integration or recruiting). Specifically, Experiment 1 (D-max) addressed the spatial limit of direction of motion perception over 2 frames (D-max) and integration over time of direction information (6 frames D-max). In Experiment 2, we used arrays of sparse random dot kinematograms (RDKs) to compare the ability to perform local speed discrimination (Experiment 2A) where the dots were moving in random directions (CSD), to speed discrimination (Experiment 2B), and the dots moved coherently in the same direction over several frames making temporal integration possible. Experiment 3 [motion coherence test (MCT)] was a direction discrimination task with sparse RDKs, where the net direction of motion could only be perceived if subjects were able to integrate spatially over the display and extract the direction of motion signal in the presence of a variable proportion of masking motion noise. Experiment 4 [two-apertures motion coherence test (2AMCT)], was used as a control to assess, first, whether a deficit in the motion coherence task involved a deficit in coherence perception or in direction discrimination in a noisy display, and second, whether a smaller area of spatial integration [comparable to the area in Experiment 5; motion discontinuity test (MDT)] is sufficient to perceive global motion, especially in the patients whose performance on the motion coherence task (Experiment 3) was normal. Experiments 5 and 6 addressed the relationship between the local and global (integrative) mechanisms by assessing patients' ability to perceive a single discontinuity in a global motion display with a variable level of coherence (Experiment 5: MDT) and to perceive the direction of pattern motion of rigid-plaids (Experiment 6: plaids). Details of all these tests are described below in the corresponding sections.
In order to compare and evaluate the performance of patients and that of normal subjects, we computed Z-scores which provide a common statistical way of standardizing data on the same scale so that a satisfactory comparison can be made. A value of Z > 2 is considered to indicate that performance is impaired.
Four patients (RA, FD, GF and AMG) who sustained a single infarct involving the extra-striate visual areas, participated in a series of visual motion perception studies and demonstrated unexpected and contrasting configurations of visual motion deficits. All the patients and the normal control subjects gave informed consent to participate in research, according to the requirements of the Boston University Human Subjects Committee. For >15 months, the patients were studied with an extensive battery of psychophysical motion tests and with control neuropsychological tests. For each patient, in Fig. 1 we show a lateral view of the three-dimensional reconstruction of the brain surface (FD and AMG) or of the hemisphere sliced through the lesion in parasagittal and coronal planes through the centre of the infarct (RA and GF). Much more extensive structural images have been published elsewhere in connexion with previous research on their visual perception (FD, Vaina and Cowey, 1996; RA, Vaina et al., 1998; TF, Vaina et al., 2000; AMG, Vaina et al., 2003).
(1 and 2) Three-dimensional reconstruction of RA's brain showing the lesion (black) on a parasagittal section through the lesion, left, and then on a coronal section, right. For all patients, images were acquired using the same GE 1.5 T Advantage System 4.8. Imaging parameters were FOV 24 cm, 3 mm slice thickness, interleaved acquisition, TR 3000 ms, TE 80 ms. Data used for the surface reconstruction were stored/analysed in 1.5 mm thick coronal slices. (3 and 4) Similar reconstruction of the brain of patient GF. (5 and 6) The cortical lesion on the lateral surface in FD and AMG. These are surface views of the hemisphere and not slices. The references to the previously published detailed structural images are cited in the text for each patient.
Patient RA was a right handed man, aged 66 at the time of his right hemisphere stroke, which initially produced a mild left homonymous inferior quadrantanopia with slight spread into the left upper field. Acuity, with corrective glasses, was 20/20. Figure 1 (1 and 2) illustrates the infarct, which is predominantly cortical and involves the right occipital lobe, extending dorsally and rostrally from the occipital pole. The lesion also slightly involves the region of the calcarine fissure. At the time of the psychophysical testing reported here RA had good performance on static form assessed with a computerized version of the Efron Shapes Test (Efron, 1972) and colour discrimination tasks by the Farnsworth-Munsell 100 Hue test (Farnsworth, 1943). Contrast sensitivity for detection and discrimination of both moving (direction) and static gratings was normal. On the Randot Stereo test (Chicago, IL, 1960), he was able to distinguish that something was there but was unable to make out the form—the star was described as a ‘teddy-bear’, and the triangle as a ‘pitcher’. Binocular stereopsis was also tested by rapidly alternating the view of each eye with electronic shutters in the viewing spectacles and in temporal correspondence with alternate frames of the VDU (Amiga, Commodore Inc.). His depth perception was normal for stimuli presented in the right visual field, but it was absent for stimuli presented in the left visual field, contralateral to his right occipital lesion. Form perception, either static or based on motion contrast (speed or direction differences), was impaired for stimuli presented in the visual field contralateral to his lesion, although we made sure that the stimuli were never presented in the area of initial visual field loss. Object recognition was normal, but he was impaired on recognition of common objects presented as silhouettes using the VOSP test (Visual Object and space Perception Battery, Warrington and James, 1991). Previously we contrasted RA's selective deficits of first-order motion perception with his normal performance on second-order motion tasks (Vaina et al., 1998, 1999; Vaina and Soloviev, 2004), as well as his normal heading perception with his very impaired perception of three-dimensional structure from motion (Royden and Vaina, 2004).
Patient FD was a 41-year-old right-handed man, who was healthy until his sudden left hemisphere infarct. His lesion in the left hemisphere is shown in Fig. 1 (5). The infarct (black) was restricted to the superficial aspect of the posterior part of the superior temporal sulcus, involving the temporal and the temporo-parietal region slightly. Psychophysical testing took place only after the patient's recovery had stabilized, at which time discrimination of form defined by total flux, or by motion contrast (direction or speed) was normal. Contrast sensitivity, form, stereopsis, and colour discrimination assessed by the same tests as those for RA, were normal. Acuity was 20/20 in both eyes and he had full visual fields. FD participated in an extensive study of his visual motion perception, and demonstrated several unusual dissociations. For example, he was permanently impaired on second-order motion perception, but on almost all first-order motion tasks his deficit resolved a few weeks after the infarct (Vaina and Cowey, 1996; Vaina et al., 1999; Vaina and Soloviev, 2004). Data presented here were obtained after FD recovered from the initial deficits (described in Vaina and Cowey, 1996).
Patient GF was a 68-year-old right-handed man who underwent a left hemisphere infarct that produced severe anomia and alexia without agraphia. In Fig. 1, 1–3 and 1–4, respectively, show the view of a parasagittal and coronal slice of patient GF's brain. The lesion involves the left occipital temporal area and extends anterior to the posterior temporal region. Although the lesion was somewhat extensive, there was very little white matter involvement. He had a transient right hemianopia which resolved in 6 weeks. At the time of obtaining the data presented here, his acuity was 20/20 in both eyes and, using the same tests described above (Patient RA), contrast sensitivity, colour, static and moving two dimensional shapes discrimination were all normal. However, binocular stereopsis was absent in both visual fields.
Patient AMG was a 52-year-old right handed woman who underwent a left hemisphere infarct. Figure 1 shows a left lateral view of patient AMG's brain, indicating the cortical involvement of her lesion. The infarction is predominantly located in the left lateral occipital lobe extending into the posterior parietal lobe (the lesion crosses the parietal–occipital sulcus). The deeper components of the lesion (not shown) reach the posterior part of the sylvian fissure within the white matter. Initially she had difficulties with arithmetic, spelling, and short term memory. She still showed a minor right inferior quadrantanopia for small stimuli and this region of her right hemifield was, therefore, avoided in the tests described here. Her performance on discrimination of form by luminance contrast or by motion contrast was normal, as were colour discrimination, contrast sensitivity and temporal frequency (Vaina et al., 2003). However, she remained impaired on a large number of visual motion tasks (Vaina, 1998; Vaina et al., 2003).
Experimental paradigm, results and interpretation
The first five psychophysical experiments consisted of dynamic random dot displays, and the last experiment consisted of superimposed moving gratings differing in spatial frequency. Details of the stimuli are described below for each experiment.
Experiment 1: D-max
D-max is defined as the maximum displacement that can be perceived as moving coherently in random dot displays (Braddick, 1974). As such it tests the spatial limits of motion. D-max was initially conceived to address the limits of local motion measurements in a two-frames presentation (Braddick, 1974). Later studies have investigated how the visual system may combine motion information over time (Nakayama and Silverman, 1984; Snowden and Braddick, 1989). The D-max test was adapted from Nakayama and Silverman, 1984. There are two conditions: (i) two frames and (ii) six frames. In both, the display consisted of a dense dynamic random ‘dot’ pattern (50% black pixels and 50% white pixels) viewed through a square aperture subtending 10° × 10° (Fig. 2A). Between successive frames, the dots from the previous frame were displaced coherently by a distance (delta) in the direction of motion of that display. In a four-alternative forced-choice task, the observer was asked to choose whether the display moved right, left, up or down. Using a staircase procedure, the magnitude of the displacement increased until the observer could no longer perceive coherent motion in a specific direction.
(A) Schematic representation of the D-max test display, which could be shifted up (as illustrated by the arrows), down, left or right. (B) The threshold displacement, i.e. D-max, for detecting the direction of motion in the 2 frame, 500 ms condition. C–F show the performance of the 4 patients and 6 control subjects on the task in which D-max was measured with displays of 2 frames or 6 frames and at different durations of the intermediate frames in the 6 frame condition. C–F illustrate the difference in performance between the D-max 6 frames and 2 frames as the ratio of the values of D-max for the two conditions.
In condition (i), each of the 2 frames was displayed for 500 ms with 0 inter-frame interval. In condition (ii), designed to investigate the effect on the D-max magnitude of integration or recruitment over time, the stimulus consisted of six successive frames with five pauses between them. The intermediate frames were displayed for the following time-duration: 15, 30, 45, 60, 75, 90, 105 or 120 s. The first and last frames were presented for 500 ms.
The results are shown in Fig. 2. When only two frames were shown (Fig. 2B), D-max was significantly lower in the hemifield contralateral to the lesion in patients RA, GF and AMG, indicating an impairment of local motion processing. The most conspicuous feature of the results of the 6-frames D-max is shown in Fig. 2C–F, where performance is expressed as the difference in the ratio of D-max for 2 and 6 frames at different frame durations. In patients RA and AMG, there was a prominent increase in the ratio across the entire range of frame durations. AMG's results on the 6 frames test were completely within the normal range (Vaina et al., 2003). RA's results were slightly, but not statistically significantly below the range of the controls (Z < 1.5 for each condition). Expressing the results as the ratio between 2 and 6 frames D-max, Patients FD and GF appeared to be within the normal range. This is certainly true for FD, whose 2 frames D-max was normal. However, since GF was impaired on the 2 frames D-max in the right visual field, a good performance would have entailed a larger value of the ratio between 6G/2F D-max. The results suggest that GF remained impaired even on the 6 frames D-max.
Nakayama and Silverman (1984) were the first to suggest two stages in motion processing with different temporal properties: a fast early stage, which performs the local motion measurement (2-frames D-max) followed by a slower, integration stage. AMG's and RA's significant increase of the 6-frame D-max (Fig. 2C and F) compared to their abnormally low 2-frames D-max, suggests that these two patients might exploit temporal integration to overcome their deficit (Fig. 2B) in local motion processing. Importantly, the results suggest that the local motion stage is not necessarily the limiting stage in determining psychophysical performance.
Experiment 2: speed discrimination
This test measures the perceived relative speed of two simultaneously presented RDK's with a dot lifetime of 11 frames (frame duration was 45 ms). The stimulus (Fig. 3) consisted of dynamic RDKs displayed in two elongated apertures each subtending 5° × 10°. The apertures were arranged one above the other in the centre of the monitor, and the distance between their centres was 6°. Each aperture contained 50 computer-generated dots. A total of 22 frames were displayed in 1 s. The speeds of the dots, defined as a function of the distance a dot was displaced between successive frames, was uniform within an aperture and was assigned independently for each aperture. The speed of the dots in the standard was fixed at 3°/s and the initial speed of the dots in the test stimulus was 6°/s (twice that of the standard). The speed of the test stimulus was varied using the adaptive staircase procedure. On half of the trials, randomly chosen, the test stimulus occupied the top half of the display. The subject had to indicate in which of the two apertures (the top or bottom) the dots appeared to move faster. There were two different conditions: (i) RSD, in which in any single trial each dot took a two-dimensional random-walk of constant step size defined by the speed (Fig. 3A). The direction in which any dot moved, extracted from a 360° range, was independent of its previous direction and also of the displacement of the other dots. The resolution of the monitor constrained the direction sampling to every 45°. (ii) CSD, in which direction of the dots within each aperture was the same throughout the duration of a trial. However, the direction could differ between the two apertures and from one trial to another (Fig. 3B). In order to minimize distance/duration cues for judging the speed of the dots in a trial, the duration of the stimuli in the tests was randomly varied by ±20% (McKee et al., 1986).
Schematic representation of the two speed tests: (A) Local speed discrimination (random walk-RSD) and (B) CSD (temporal recruiting-CSD) In each condition the subject had to select in a two-spatial alternatives forced choice paradigm the aperture with the higher speed. Results for the 4 patients and 10 control subjects in the tests of speed discrimination are represented in (C and D) respectively. The y-axis portrays the percent of speed difference needed to make reliable speed discrimination, and the x-axis indicates the subjects.
The results are shown in Fig. 3A and B. (i) Only FD's performance was normal for stimuli presented in either visual field. GF and, especially, AMG were impaired when stimuli were presented in the visual field contralateral to the lesion, while RA's performance was impaired in both visual hemifields. (ii) RA remained significantly impaired on this task. The performance of the other three patients was not statistically significantly different from that of the normal subjects.
In the test of local speed discrimination (Fig. 3A and C) all dots moved in different directions at random, therefore speed discrimination was computed by a local computation. The deficits on this task of patients AMG, RA, and GF indicate that their local computation mechanism was impaired in the visual field contralateral to the lesion (it should be noted that RA was impaired in both fields). Naturally, the results are similar to these patients' results on the 2 frames D-max, since D-max values reflect the subjects' ability to perceive speed of motion. FD's score was normal. However, when all the dots moved in the same direction (Fig. 3D) both AMG and GF were no longer significantly different with respect to hemifield. RA's performance remained significantly different from that of the normal controls. As with the tests of D-max, the result indicates that when local motion processing provides the only possible means of performing the task—as with random motion directions, where every dot's direction changes from frame to frame—patients RA, GF and AMG are impaired. But when integration is possible over a longer time period (22 frames), all three patients improved their performance and became indistinguishable from normal in GF and AMG (Z < 2). The difference in performance of the two versions of the speed task cannot be attributed to the subject's use of distance and duration cues, since the duration that a dot spent in the aperture was varied. We suggest that this difference is due to the patient's ability to take some advantage of temporal recruiting to judge speed in global motion displays.
Experiment 3: MCT
This test was adapted from Newsome and Paré (1988). The stimuli were RDKs with a correlated motion signal of variable strength embedded in motion noise (Fig. 4A). The strength of the motion signal, i.e. the percentage of dots moving in the same direction, varied from 100 to 0%. The stimulus was presented in a circular aperture 10° in diameter, situated 2° left or right of the fixation mark. Dot density was 2 dots/degree and speed was 3°/s. In a forced-choice task, the subject had to report the direction of motion: up, down, left or right. During a trial, all the dots in the display had the same probability of being correlated with a dot in the next frames, such that at lower correlation probabilities it was unlikely that a single dot could be followed as a linear succession of dots over several frames; therefore, direction of movement had to be derived from a global computation, which requires spatial integration of local motion measurements.
(A) Schematic representation of the dynamic stimuli in the MCT. Each stimulus presented dots in rapid succession and each dot survived for a brief period of time before being replaced. In the three examples shown in A, all the dots move randomly at the left, half of them move coherently at the centre, and all dots move coherently at the right. (B) Results on the MCT. The y-axis shows the proportion of coherence necessary to perform the task at ∼75% correct. The x-axis shows the subjects.
The results are shown in Fig. 4B. Both RA and AMG performed normally in both visual hemifields. FD and GF were impaired in the hemifield contralateral to the lesion, suggesting that their ability to spatially integrate the motion signal and extract the net direction of motion was impaired.
RA and AMG's normal performance on this task was intriguing. The likely explanation is that, especially at low coherence, performing this task involves spatial rather than temporal integration, i.e. for discriminating the net direction of motion the subject must pool information over a spatially extended area to overcome the non-directional motion noise. The subjects are presumably not using temporal integration in this task, as they can perform it at such low levels of coherence that it is most unlikely that any given dot will move in the direction of the signal for more than two frames in a row. In fact, AMG and RA scored no differently from the normal subjects on this task even when only two frames were presented (data not shown here). Despite AMG and RA's deficits on local motion measurements, their ability to perceive motion stimuli is greatly enhanced by spatial and temporal integration of motion signals, even becoming normal in some conditions. The latter finding could help to explain why monkeys in which area MT has been permanently damaged by neurotoxic lesions recover their ability to discriminate the direction of motion in random dot displays of variable coherence (Newsome et al., 1985, 1986; Newsome and Paré, 1988; Yamasaki and Wurtz, 1991).
Experiment 4: 2AMCT
This test was used as a means of distinguishing between the possibility of impairment in motion coherence resulting from faulty perception of coherence per se or from a deficit in perceiving direction in noisy stimuli, and for determining whether deficits in the MDT could be due to the smaller spatial integration area (half of the area used in the MCT). The stimulus (adapted from Downing and Movshon, 1989) consisted of two dynamic RDKs displayed one above the other, each aperture subtending 6° in diameter. One of the apertures contained only motion noise, with the same qualities as described in the MCT. The other, identical to the motion coherence display, contained a variable proportion of signal dots masked by motion noise. In a 2AFC task subjects were first asked to choose the kinematogram (top or bottom) which contained the coherent motion signal and, second, to discriminate the direction of motion (up, down, left or right) in the chosen aperture. An error in identifying either the coherent display or in correctly identifying the direction of motion in the coherent display resulted in an increase in the strength of the motion signal in the next trial.
The results are shown in Fig. 5B. Patients RA, FD and GF were substantially impaired in the hemifield contralateral to the lesion, whereas AMG performed better than the mean score for the 7 controls.
(A) Schematic representation of the stimulus in the 2AMCT. The stimulus consists of two RDKs, displayed one above the other. Each was displayed in a circular aperture 6° in diameter and the two apertures were situated 2° left or right of a small black fixation mark. The display characteristics (density, display time and speed) were identical to the MCT and MDT. One of the apertures contained only noise, the other had a variable amount of coherence. (B) Results on the 2AMCT. Patients RA and FD were impaired for stimuli shown in the visual field contralateral to the lesion. GF was impaired in both visual fields. Unlike the other three patients, AMG had normal performance on this task.
This control task addressed both the area of integration of motion signal and the potential dissociation between perceiving coherence and direction in noisy motion displays. AMG was the only patient who performed normally on this task. She detected effortlessly which of the two displays contained coherently moving dots, even at a very low level of coherence (7%) and correctly reported the net direction of motion within the stimulus. The other three patients were impaired on this task, and GF was impaired in both hemifields. FD and GF's impairment lay in detecting coherence from non-coherence, but at levels where they could perform this task they also correctly discriminated direction. It is interesting that these patients' performance was much more impaired on this two-step task than on the motion coherence task. A likely explanation could be that the area of integration was smaller here than in the previous test. However, it is intriguing that at higher coherence levels they, especially GF, made errors in determining which of the two displays contained coherently moving motion. However, once they could correctly identify the aperture that contained coherently moving dots, they made no errors on direction discrimination. RA, on the other hand, was impaired on direction discrimination even at levels where he could correctly determine which aperture contained the coherent motion. He needed <20% coherence to recognize it as such, but in the visual field contralateral to his lesion he could only reliably discriminate direction at coherence >40%.
It seems clear that provided patients AMG, FD and GF can detect global motion they can, as expected, discriminate its direction. But patient RA, paradoxically, could not discriminate direction even when he consistently correctly identified which aperture contained the coherent motion. Discriminating coherence and direction are, therefore, not inseparable. The importance of the area over which spatial integration is possible is also indicated by the greater defect in this task than with the larger displays used in Experiment 3. However, the precise relationship between performance, the areas over which spatial integration is possible, and the effect of a particular lesion remain to be investigated.
Experiment 5: MDT
As in the previous test, the stimuli were RDKs with a correlated motion signal of variable strength embedded in noise and presented in a circular aperture 10° in diameter. Here, half of the trials contained an imaginary line bisecting the display and entirely defined by the difference in direction of motion on the two sides of the imaginary boundary. On the other half of the trials there was a homogeneous display in which all the signal dots moved in a single direction. Dot density was 2 dots/degree and speed was 3°/s. The subject's task was to determine whether the display was homogeneous or had a discontinuity.
The results are shown in Fig. 6B. Patient RA required almost 100% coherence in order to reliably perceive discontinuity in the stimulus presented in the visual field contralateral to the lesion, while AMG required >50% coherence, which is ∼5 times more than the coherence needed by the normal controls and the other two patients whose performance was not statistically different from that of the normals (Z < 1.5).
The MDT test. (A) The figure shows a schematic illustration of the stimulus, on the left a homogeneous RDK and and on the right an RDK in which the opposite directions of motion of the signal dots define an imaginary line of discontinuity. This discontinuity line could have one of the four orientations as defined by the diameter of the circular aperture at cardinal directions and 45° inclinations. The direction of signal dots (filled dots) was always vertical, and in the discontinuous case signal dots moved in opposite directions on the two sides of the imaginary line bisecting the display. (B) The figure shows the results for the four patients and 16 control subjects. The y-axis portrays the proportion of coherent dots necessary for reliably performing the task (i.e. to determine whether the stimulus was homogeneous or had a discontinuity). The x-axis shows the subjects.
FD and GF had normal performance on this task. In stark contrast, RA was unable to detect discontinuity in the visual field contralateral to his lesion. If we assume that RA had difficulty in detecting the motion direction in noise in the smaller displays (∼30 square degrees, as opposed to 79 square degrees in the motion coherence task), it is understandable that he would have difficulty with the MDT task at levels of coherence where his performance on motion coherence was normal. Patient AMG also needed a significantly higher percentage of signal dots than that required by normal subjects and by herself in the other visual hemifield (Fig. 6B). We suggest that this can be interpreted as a deficit of integration of local and global motion mechanisms or as a deficit in one of these two mechanisms (Vaina et al., 1994). The global motion mechanism operates at a larger spatial scale, and it is necessary to integrate spatially the noisy motion signal and reduce noise. The local mechanism is necessary to detect the existence of a boundary. As the boundary changed orientation from trial to trial, subjects could not focus on one ‘spot’ in the display and determine whether it was homogeneous or it contained a discontinuity. Both RA and AMG had normal performance on the motion coherence task, suggesting that their global motion mechanisms were normal, yet their deficit in local speed discrimination suggests that the local motion mechanisms were impaired. This would account for the deficits in the discontinuity task.
It is harder to account for FD's and GF's normal performance on this task, especially GF's, because he was impaired on both the local speed discrimination (local mechanisms) and on motion coherence (global mechanisms). Moreover, their performance worsened with the smaller area being available for motion integration.
Experiment 6: plaids
Moving plaid patterns (Adelson and Movshon, 1982) generated by superimposition of two sine-wave gratings moving in different directions were viewed through a circular aperture subtending 8° in diameter. The stimulus was surrounded by a uniform grey of 22 cd/m2 of the same hue and luminance as the mean luminance of the pattern. Two-dimensional spatial patterns were generated by drawing two one-dimensional ramps oriented obliquely at 60° on either side of vertical (the resultant direction). The two components always had the same contrast of 30%. The speed of the two gratings was constant at 3°/s and the spatial frequency of the referent grating was fixed at 1 cycle/degree, and the other varied between 0.25 and 2.5 cycle/degree. The temporal frequency was accordingly varied to always provide a speed of 3°/s. Every pattern was repeated 22 times in random order. The observers indicated whether the moving pattern appeared ‘coherent’ or ‘transparent’ (incoherent).
The results are shown in Fig. 7B. Only patients RA, AMG and FD were available for testing and their results are compared with those of five control subjects. In the visual hemifield contralateral to the lesion, RA and AMG could only perceive rigidly moving plaids when the two component gratings had identical spatial frequency. In contrast, in the ipsilateral hemifield their performance was no different from that of the control subjects and they perceived rigidly moving plaids for spatial frequencies varying up to a factor of 2. Patient FD's performance did not differ from that of the normal controls for stimuli presented in either hemifield. To be certain that the deficit in the contralateral field was not due to a small scotoma that prevented the patients from seeing the entire stimulus, we asked each patient to outline the stimuli on the computer screen. None of the patients showed any difficulty in delineating the stimuli in either visual field, while maintaining fixation.
Plaids test. (A) A schematic illustration of the generation of the stimulus. The individual gratings appear to move in directions orthogonal to the stripes. However, the two overlapping gratings produce either a percept of a single plaid moving in the direction of the arrow (coherent motion) or two separate gratings that move over one another (non-coherent or transparent motion). (B) Results from 3 patients and 5 normal subjects. The y-axis shows different spatial frequency values. The x-axis shows the subjects. For stimuli presented in the contralesional field, patients RA and AMG could perceive coherent motion only when the two gratings had exactly the same spatial frequency.
To perceive rigidly moving plaids, even when the direction of each grating is detected independently at different spatial frequencies, subjects must combine motion information across a range of spatial frequencies (spatial scales). It is possible that AMG's and RA's results may be accounted for by the fact that they were unable to integrate across spatial scales in their impaired field, contralateral to the lesion.
General discussion and conclusion
A particularly informative outcome of this study is revealed by the comparison of the performance of the patients on MCTs, 2AMCT, random speed and constant speed, as summarized in Table 1. In 2AMCT, patients FD and GF were impaired on coherence detection whereas they were not at all impaired on direction discrimination in noisy motion stimuli (when they could correctly determine which of the apertures contained coherently moving dots). This is consistent with their deficits in the MCT, which points to an impairment on spatial integration of noisy motion signals (up to a certain proportion of noise). However, in the former task patient RA was impaired on direction discrimination but not on coherence. This is consistent both with his roughly normal performance on motion coherence, and with his impaired performance on the constant speed task, where performing the task effectively requires the subject to compare the speeds of the often very different directional fields of dots with a limited point lifetime of 11 frames. This result is not in conflict with RA's normal performance on the 6-frames D-max, because there, in the single field of RDKs, all the dots moved in the same direction and no visual noise was present.
An additional important result is the dissociation of deficits on the 2-frame D-max and 6-frame D-max, Speed in RSD and in CSD and motion coherence. The observed dissociations between preserved and impaired abilities to perform these tasks suggest that local motion perception is not the limiting stage in one's ability to process motion information. Our results indicate that in patients with selective motion deficits, temporal and/or spatial integration mechanisms remain unimpaired. Do these latter mechanisms help the patients to perform well on higher-level motion tasks, like biological motion? This is an important question that we are now pursuing.
Although the patients' lesions were carefully analysed, in two of the four patients they were too large to allow a fine grain description of their anatomical relationship to specific cortical visual areas and, therefore, their relationship to functional maps of the human visual cortex as revealed by fMRI. Nevertheless, it is clear that FD has a lesion of the lateral cortex just dorsal to area hMT+, whereas RA's lesion is medial and in the territory of V2 and V3. RA is impaired on speed in noisy displays, MDT, 2AMCT, and plaids but not on speed in coherent displays or on motion coherence. FD is very different in being impaired on motion coherence, whether for 1 or 2 apertures, and little or not at all impaired on the other tasks. Functional neuroimaging, which we are now performing on both normal subjects and patients could reveal why RA and FD are so different. GF's impairment is like FD's with the notable exception of speed discrimination in noisy stimuli, where GF resembled RA. It is interesting that GF's lesion (Fig. 1) approaches both the lateral and medial surfaces but without actually overlapping the area of cortical damage in FD and RA. AMG, whose lesion is more dorsal than that of the other three patients, most resembles RA with respect to performance on the two speed tasks, motion coherence (normal), discontinuity and plaids. But AMG was as good as control subjects with motion coherence and 2AMCT. We still have too little information about functional specialization as revealed by fMRI and with respect to the current motion displays to interpret her defect satisfactorily with respect to regional cortical anatomy, despite demonstrations of multiple areas that respond to visual motion (Sunaert et al., 1999).
The research was supported by the National Institutes of Health grant EY-ROI-EY-07861 to L.M.V. and by a UK MRC grant and Oxford McDonnell Network grant to A.C.
Vaina LM, Le May M, Bienfang DC, Choi AY, Nakayama K. Intact ‘biological motion’ and ‘structure from motion’ perception in a patient with impaired motion mechanisms: a case study. Vis Neurosci1990; 5: 353–69.
Vaina LM, Gryzwacz NM, Saiviroonporn P, LeMay MM, Bienfang DC, Cowey A. Can spatial and temporal motion integration compensate for deficits in local motion mechanisms? Neuropsychologia2003; 41: 1817–36.