OUP user menu

Impaired spatial working memory across saccades contributes to abnormal search in parietal neglect

Masud Husain, Sabira Mannan, Tim Hodgson, Ewa Wojciulik, Jon Driver, Christopher Kennard
DOI: http://dx.doi.org/10.1093/brain/124.5.941 941-952 First published online: 1 May 2001

Summary

Visual neglect of left space following right parietal damage in humans involves a lateral bias in attention, apparent in many search tasks. We hypothesized that parietal neglect may also involve a failure to remember which locations have already been examined during visual search: an impairment in retaining searched locations across saccades. Using a new paradigm, we monitored gaze during search, while simultaneously probing whether observers judged they had found a new target, or judged instead that they were re-fixating a previously examined target. A patient with left neglect following focal right parietal infarction repeatedly re-fixated right locations. Critically, he often failed to remember that these locations had already been searched, treating old targets as new discoveries at an abnormal rate. In comparison, healthy age-matched control subjects rarely re-fixated targets, and mistook old targets as new targets even more rarely. The frequency of such mistakes in the parietal patient, for different conditions, correlated with the severity of his neglect. Control experiments indicated no perceptual localization deficit in non-search tasks. These results suggest a deficit in retaining searched locations across saccades in parietal neglect, in addition to the lateral spatial bias. Moreover, the former deficit exacerbates the latter, such that patients do not realize that the rightward locations favoured by their bias have already been examined during previous fixations and, for this reason, they saccade back to them repeatedly. The combination of the two deficits (a lateral bias plus a deficit in retaining locations already searched) may thus explain the pathological pattern of search that characterizes parietal neglect: why stimuli on the right are re-examined recursively, as if being searched for the first time, and hence why stimuli on the left continue to be ignored even with unlimited viewing time. These proposals accord with recent electrophysiological and functional imaging data, demonstrating posterior parietal involvement in the retention of target locations across saccades.

  • neglect
  • parietal
  • visual search
  • saccades
  • spatial working memory

Introduction

The syndrome of unilateral neglect can occur following unilateral damage to various brain regions, but is most frequent and enduring in patients with lesions involving the right inferior parietal lobe (Vallar and Perani, 1986). Clinically, these individuals have a bias to orientate towards objects or events located to their right while ignoring those to the left, often being unaware of the latter. For example, when asked to search for visual targets among distractors in bedside cancellation tasks, right-hemisphere neglect patients typically mark targets only on the right, missing targets towards the left even when given unlimited time to view a scene, although they can usually detect the same left targets if presented in isolation (Eglin et al., 1989). The leftward neglect in search and cancellation tasks is attributed commonly to a lateral bias in attention towards the right and/or a difficulty in disengaging attention from items on the right side (Posner et al., 1984; Kinsbourne, 1987; Bisiach and Vallar, 1988; De Renzi et al., 1989; Eglin et al., 1989).

It is increasingly recognized that clinical neglect may involve several component deficits (Vallar, 1998), each of which may exacerbate the others. Here we consider one possible component that has received relatively little attention to date: a possible deficit in retaining visual locations across saccades during search. We propose that when combined with a spatial bias towards the right side, such a deficit could exacerbate the tendency to search right items repeatedly, as the patients would not realize that the rightward locations favoured by their bias had already been examined during previous fixations, and so may saccade back to them recursively for this reason.

We had several grounds for suspecting that a deficit in retaining searched locations across saccades might arise following the parietal lesions typically associated with neglect. First, recent evidence from electrophysiological recording in monkey posterior parietal cortex indicates that neurones there may be involved in representing visual locations across saccades, and in maintaining a memory trace for the location of saccadic targets across delays (Mazzoni et al., 1996; Colby and Goldberg, 1999). Secondly, functional imaging in humans has revealed a network of brain areas involved in maintaining information about visual location in a variety of `spatial working memory' tasks (Jonides et al., 1993; Owen et al., 1996a; Courtney et al., 1997; D'Esposito et al., 1998; LaBar et al., 1999), including saccades to remembered locations (Anderson et al., 1994; O'Sullivan et al., 1995). This network is strikingly similar to the brain areas implicated in neglect patients' lesions. Finally, two studies have reported that patients with parietal lesions can encounter difficulty in retaining target location across saccades, in speeded double-step saccade tasks (Duhamel et al., 1992a; Heide et al., 1995). However, the possible implications of these findings for search tasks in neglect patients have not been examined previously.

We hypothesized that, when combined with a lateralized spatial bias to the right, a failure to retain searched target locations across saccades should result in repeated re-examination of previously fixated items on the right, with the patient considering these to be new discoveries, due to a failure to retain previously searched locations across saccades. While re-fixation of rightward items has been observed previously in neglect patients (e.g. see Fig. 7 in Behrmann et al., 1997), it has rarely been discussed, and no existing study has tested whether such re-fixations are associated specifically with a failure to remember which locations have already been searched.

Here we examined this hypothesis by combining measures of eye position during visual search with a measure of whether observers judged they currently were fixating a particular target for the first time, or instead judged that they had examined it previously. We found that healthy individuals very rarely mistook previously searched target locations for new ones. We also tested a patient with left neglect after focal right parietal damage (sparing the frontal lobe). This case was particularly suitable for determining whether a deficit in retaining searched locations across saccades might be associated with the parietal structures suggested by recent electrophysiological and imaging evidence (Anderson et al., 1994; O'Sullivan et al., 1995; Mazzoni et al., 1996; Colby and Goldberg, 1999), rather than with control structures in the frontal lobe which often have been linked to more general `working memory' function (Owen, 1997).

Methods

Patient details

G.K., a 68-year-old patient, has shown persistent left neglect since he suffered infarction of the right inferior parietal lobe. Figure 1 demonstrates his infarct. The lesion was plotted using MRIcro (http://www.psychology.nottingham.ac.uk/staff/cr1/mricro.html) and then rendered (Fig. 1B). Note the sparing of the frontal lobe, which allows a test of our hypothesis that parietal damage may be sufficient to induce a deficit in retaining searched locations across saccades, which could contribute to pathological search in the neglect syndrome.

Fig. 1

(A) Axial T1-weighted MRI demonstrating infarction of the right parietal lobe with sparing of frontal structures in G.K. (B) Extent of the lesion on rendered reconstruction.

On examination, throughout the year of study, G.K. had no visual field defect; he demonstrated left visual and tactile extinction; there was no weakness or primary disturbance of sensation in his limbs. He showed left visual neglect on line bisection (erring a mean of 5.1 cm to the right of true midline on 18 cm horizontal lines). On standard pen-and-paper cancellation tasks, which require crossing out of targets embedded in an array of distractors, he neglected targets on the left. On the Mesulam shape-cancellation task (Mesulam, 1985), the day after his infarct he cancelled 17 out of 30 targets on the right and 0 out of 30 on the left. One year following his infarct, he scored 30 out of 30 on the right but only 2 out of 30 on the left. Whenever he was tested, he appeared to perform cancellation tasks in a systematic way, starting at the right edge of the array and working leftwards. Importantly, throughout all these pen-and-paper cancellation tasks (n = 8), he never re-cancelled targets he had already marked with any additional mark (though he may well have re-fixated them), as discussed later.

His digit span was 10 forwards and five backwards. In contrast, he had a spatial span of only three on the Corsi blocks test (De Renzi et al., 1977), regardless of whether the blocks were placed directly in front of him or to the right of the mid-sagittal plane. On the CANTAB computerized neuropsychological test battery (Owen et al., 1990), his spatial span was also only three. This dissociation between performance on tests of visuospatial and verbal working memory resembles that reported by Hanley and colleagues in a right hemisphere haematoma patient without neglect (Hanley et al., 1991; see also Ellis et al., 1996 for further work on spatial working memory).

As a further test of any spatial working memory deficit in this parietal neglect patient, we assessed him using the standardized spatial working memory task from the CANTAB battery (Owen et al., 1990), a version of the self-ordered pointing task (Petrides and Milner, 1982) with well-established age-matched norms (Fray, 1996). G.K. had to search through a variable number of boxes, scattered pseudorandomly within a virtual 7 × 5 grid, at different locations on a touch screen. Touching a box revealed its contents. G.K.'s task was to look for a single token, hidden in only one box from the array on each trial. When this token was found, the next token was concealed by the computer program in another box. G.K. was told that once found, a token would never be re-hidden in the same box.

Efficient search in this task (as in many other search tasks) requires spatial working memory for locations that have already been examined (and, in this case, found to contain tokens). G.K. made many search errors, with 130 `re-visits' where he touched again on the screen locations which he previously had found to contain tokens (5th percentile score for age-matched control subjects = 71). We asked the question `could this error rate be due merely to neglect of boxes on the left?' As an initial assessment of this, we excluded trials with tokens hidden on the left third of the array. This still gave a relatively high re-visiting score for G.K. of 51 (expected 5th percentile score for matched normals would be only 36, assuming an equal distribution of errors across space). Even for tokens within just the right third of the array, G.K. still showed an abnormally high rate of re-visits (34, exceeding the normal 5th percentile of 26). It may be useful to adapt the CANTAB task in future studies of neglect, so that only one vertical column of boxes is presented.

The results of this standardized CANTAB task are consistent with a spatial working memory deficit for search tasks in G.K. However, this standard task may be suboptimal for ruling out any contribution from neglect per se. It differs from standard search or cancellation tasks, because a trial terminates only when a token is found. Hence the patient may feel compelled to re-check locations known to have been visited previously, if other locations are neglected. In contrast, standard search and cancellation tests are self-terminating, with the patients instructed to find as many targets as they can, stopping when they cannot find more. Here we developed several new tests for G.K., specifically to assess any deficit in retaining searched locations across saccades, within standard search tasks. The aim was to determine the possible role that such a deficit might play in the pathological search that characterizes parietal neglect. We recorded eye movements during natural visual search. In addition, we explicitly required observers to indicate, as they examined each location, whether they were doing so for the first time.

Apparatus and stimuli

Our tests all involved variations on a search task in which G.K. (and control subjects) had to look for specified targets, sometimes in the presence of distractors. The stimuli resembled standard cancellation tests for neglect, but the search task differed as it never required G.K. to leave a visible mark on the targets visited (just as for visual search in real life). Instead, we monitored his eye position, measuring whether particular items were fixated and re-fixated. Note that, in themselves, any re-fixations would be insufficient to prove a deficit in retaining searched locations across saccades. They might arise due to checking, rather than to forgetting which locations had already been searched. Crucially, therefore, we also required G.K. (and normal controls) to click a response button only when they looked at a `new' target. This allowed us to test directly whether re-fixated items were remembered as having been examined previously (`old'), or whether instead they were mistakenly treated as new, producing re-clicks due to a failure to retain their location as already searched, across intervening saccades. As in standard search and cancellation tests, each trial was self-terminated.

G.K. and four age-matched neurologically healthy individuals (mean age 68.5 years) participated. Consent was obtained according to the declaration of Helsinki and approved by the hospital ethical committee. Subjects sat with their head in a chin rest at 57 cm from a 16 inch monitor, operating at 60 Hz with a resolution of 640 × 480 pixels (subtending 29 × 22°). Six different types of search displays were presented. Across all the search tasks, the left and right halves of the screen contained on average an equal number of stimuli. Eye position was recorded with an infrared-based eye recording system (SensoMotoric Instruments TM, GmbH, Berlin, Germany) with a temporal resolution of 250 Hz and spatial accuracy of <0.5°. This system included a head-mounted camera which compensated for any head movements. Eye monitoring was conducted on-line throughout.

Before experimental trials began, sample search displays were shown and the targets specified. Details of each search display are given below in Experiments 1–4. In all the search experiments, each trial began with fixation of a central cross for 1 s, followed by a search display initiated by the experimenter, in which all targets had to be found. The instruction was to look directly at each target found and, while fixating it, to click the button only if it was a newly discovered target. It was emphasized that any target previously found should not be re-clicked if it was looked at again. Self-termination of search was indicated by subjects removing their hand from the button.

Eye position data were analysed off-line in two stages. Saccades were identified using a velocity algorithm (saccade start defined as a change of position over two consecutive samples exceeding a velocity of 25°/s). The mean positions of fixations and intervening saccades were then superimposed on the search display for each trial. A fixation was attributed to a search element (target or distractor) if its mean position occurred within 1° of the edge of that element. Two or more consecutive fixations falling on the same element were considered part of the same compound fixation. Re-fixations were scored when at least one fixation intervened away from that element. Most of G.K.'s click responses (e.g. 97% for the search displays used in Experiment 1) were made during a fixation, and the few response presses occurring mid-saccade were attributed to the previous fixation (if the click occurred with eye position still within 2° of an item). Scoring of a button press as a `re-click' required one or more intervening fixations elsewhere between successive clicks on an item. In practice, numerous saccades intervened between the re-clicks observed for G.K. (see below).

Experiment 1 was performed in one experimental session. In the second session reported here, the five different types of search screens used in Experiments 2 and 3 were presented in random order. Experiment 4 was performed in a different session.

Statistics

Analyses were performed using StatView (SAS Institute Inc., Cary, USA) and SigmaPlot (SPSS Science, Chicago, USA); Jonckheere statistic was derived by hand.

Results

Experiment 1: Ts among Ls

We first used displays of 19 targets (letter Ts) embedded among 44 distractors (letter Ls), with each element randomly located within a virtual 9 × 7 grid of 3.2 × 3.1° cells (Fig. 2A). Each element subtended 1°, and their mean edge-to-edge separation was 2°. Search performance on this paradigm (assessed by re-fixation rate and re-click rate for targets already visited) allowed us to determine whether there was a deficit in retaining previously fixated locations as already searched, on a task that closely resembles standard pen-and-paper bedside search tasks (except that no visible mark is made at searched locations, just as for natural search tasks in daily life).

Fig. 2

(A) An example of a 19 Ts + 44 Ls search array on which are superimposed fixations (red dots) and interpolated scanpaths (yellow lines) of patient G.K. Note the profound neglect of the left side of the array and the many re-fixations of stimuli on the right. (B) An example of G.K.'s scanpaths on a `strip' display, showing less neglect.

G.K. consistently neglected left targets (n = 7 displays). Figure 2A shows an example of his scanpath, demonstrating two important and characteristic features of his search. First, he was clearly able to make many leftward (contralesional) saccades. Indeed, over the seven trials, the total number of leftward saccades (109) did not differ from the number of rightward saccades (97; t-test, P > 0.5); nor did the percentage of leftward versus rightward saccades differ for search within different quartiles of the display (t-test, P > 0.5 in all cases). Thus, although G.K. neglected the left side of the display, he nevertheless made leftward saccades as frequently as rightward ones, confirming previous observations on search in neglect by Niemeier and Karnath (Niemeier and Karnath, 2000). G.K.'s bias to search the right side of the display therefore cannot be attributed simply to a bias to make rightward saccades.

Secondly, G.K. frequently re-fixated targets, whereas age-matched healthy individuals hardly ever did so. Over the seven trials, G.K.'s re-fixation rate was 13 times higher than that of the four age-matched normal controls. His median re-fixation rate (i.e. number of re-fixations on targets, divided by the number of individual targets fixated) was 4.10, compared with a rate of 0.31 in controls (upper limit of 99% confidence interval for normals = 0.72; P < 0.01). Crucially, G.K. also had an abnormally high re-click rate with the response button for targets (where re-click rate is defined as the number of re-clicks on target, divided by the number of individual targets clicked upon). In contrast, healthy age-matched control subjects re-clicked even more rarely than they re-fixated. G.K.'s median re-click rate was 1.73, compared with 0.05 in controls (upper limit of 99% confidence interval = 0.18; P < 0.01). Thus, G.K. demonstrated abnormally high rates of re-fixation on previously visited targets, and critically also showed abnormally high rates of re-clicking for these (his re-click rate was 34 times that of normal control subjects), thus indicating a deficit in retaining these locations as previously searched, across intervening saccades.

If re-clicking was due to some form of compulsive perseveration, one might expect that it would occur immediately after a click. However, the mean interval between click and re-click in G.K. was 15 s (with intervening saccades occurring at a mean rate of 3/s). Only 14% of his re-clicks occurred within 1 s of a click. We therefore attribute the abnormal re-clicks in G.K. to impaired spatial working memory for the positions already examined, across saccades. On this interpretation, his re-click rate should be reduced if the spatial memory load of the search task is reduced, as tested in the following studies.

Experiment 2: `strip' displays

We sought to measure how changes in spatial working memory load might affect performances in the search task, especially the degree of re-clicking and of neglect in G.K. Our second type of search display had a reduced set size (and thus fewer positions to be retained in memory as already fixated, across saccades during search). In order to hold constant the stimulus density, and the ratio of targets to distractors, we reduced the total set size by presenting only the middle horizontal third (or `strip') from the previous displays, leading to new displays like that shown in Fig. 2B (compare with the original display in Fig. 2A). In these `strip' displays, the number of targets (Ts) was six or seven, and there were 11 or 12 distractors (Ls).

An example of G.K.'s scanpath on performing this search is shown in Fig. 2B. Re-fixation rate was not significantly affected (median re-fixation for `strip' displays = 2.0 versus 2.92 for 19 Ts + 44 distractors; n = 7 of each search). Critically, however, G.K.'s re-click rate was significantly reduced for the horizontal `strip' condition, when compared with the full screen displays (median re-click rate for strip = 0.25 versus 0.63 for 19 Ts + 44 distractors; Mann–Whitney U = 6, P < 0.02). This was consistent with our interpretation, indicating that when spatial memory load was reduced (i.e. with a smaller display set size, as in the strip displays) there was a reduction in G.K.'s pathological tendency to treat previously visited targets as new discoveries. Note also the tendency for this to be accompanied by more fixations towards the left (compare Fig. 2B with Fig. 2A), which we discuss later.

However, one might argue that the reduced re-click rate for strip displays is due to some other factor, e.g. narrowing possible stimulus locations may simplify search. Accordingly, the next study (actually performed within the same experimental session) manipulated spatial working memory load in a more straightforward manner, by varying only the number of targets in displays which had no distractors, without narrowing the possible stimulus locations.

Experiment 3: varied number of targets

We used three types of `target-only' displays, with either 19, six or two Ts, and with target position equally unconstrained for the larger and smaller set sizes. There were now no distractors, with targets only located in the same 9 × 7 virtual grid used to generate displays such as Fig. 2A. Set sizes of 19, six or two targets were presented in random sequence (n = 7 of each display).

As predicted, G.K.'s pathological re-click rate increased systematically (Fig. 3) with the number of targets in each display (Jonckheere statistic = 118, P < 0.005; Jonckheere non-parametric ordered alternatives test). Note that this was found even though our scoring of re-click rate is highly conservative, being weighted against our prediction. For two targets clicked, just one re-click would give a re-click rate of 0.5; for eight targets clicked, one re-click would yield a value of only 0.125. Thus, even small absolute numbers of re-clicks in displays with fewer targets would give higher re-click rates, yet we actually found higher rates when there were more targets. This is consistent with our prediction that when spatial memory load is increased (by more targets), the patient should increasingly indicate that he considers previously fixated items to be a new discovery. Figure 3 also displays, for comparison, the rate of re-clicking from Experiment 1 for displays with 19 Ts and 44 Ls. We attribute the higher re-click rate in that condition, compared with displays with 19 Ts alone, to the increased burden placed on spatial memory by adding distractors (Ls, which are frequently fixated during search for Ts). Re-fixation rate did not differ significantly across search displays with increasing numbers of targets (median rates were 2.5, 1.5 and 1.9 for two, six and 19 Ts, respectively).

Fig. 3

The re-click rate increased systematically with increasing number of targets in patient G.K.

Neglect in relation to re-click rate across tasks

How does the proposed deficit in retaining locations already searched, across saccades (as evidenced by the re-click rate), affect patient G.K.'s neglect? Our account predicted more neglect when retention of previously searched locations was most impaired (i.e. with high re-click rates, under conditions of high load on spatial memory). To address this, we plotted, for all the search tasks used so far, median re-click rate (our measure of deficient retention of searched locations) against the percentage of targets missed (all misses were on the left, and thus more misses meant more neglect). Note that these measures are again conservative as regards our predictions, since misses for small absolute numbers of targets are scored as more neglect (in percentage terms) for the smaller set sizes that produced least re-clicking. Figure 4 shows that the degree of neglect, even when so defined, increased with the rate of re-clicking (r = 0.87, P < 0.01) as we predicted. Thus, the more G.K. re-clicked on targets towards the right, indicating that he considered them to be new discoveries, the more likely he was to neglect targets towards the left. This suggests a close functional relationship between a deficit in retaining previously searched locations across saccades, and the severity of neglect. This is in agreement with our proposal that a failure to remember that particular locations have already been visited on the right contributes to the continued neglect of locations on the left, by inducing recursive search of right locations.

Fig. 4

The degree of neglect in G.K. was significantly correlated with the rate of re-clicking upon targets across search tasks. Key: a = two Ts, b = strip, c = six Ts, d = 19 Ts and e = 19 Ts + 44 Ls.

We acknowledge that the degree of attention directed to the right may vary across the displays we used. For example, vertically extended arrays (Fig. 2A) may attract attention more strongly to the right than `strip' displays (Fig. 2A) simply because in these arrays there are more stimuli on the right. However, this alone would not explain why G.K. re-clicked at a pathological rate, nor why his re-click rate correlated with the degree of neglect across several different display types. Such behaviour is naturally explained by a deficit in retaining searched locations across saccades.

Experiment 4: spatial versus non-spatial retention across saccades—circles versus objects

The targets in the displays described so far differed from each other only in location; they were identical in shape. However, if they were also to differ in identity, previously visited items could then be remembered non-spatially. Thus, a patient with a specific deficit in retaining locations across saccades might be able to remember previously fixated items by their identity, even if unable to recall their location. To address this, we used a different type of search display in which there were now 24 objects (Fig. 5), each with a unique memorable identity, intermingled with 24 identical circles (Wojciulik et al., 2001). On alternate trials, either pictures or circles were the targets for search.

Fig. 5

An example of a search array used to compare spatial and non-spatial working memory. On alternate trials, the subject was asked to search for objects (which were all different in identity) or circles (which were all identical in shape, differing only in spatial location).

We predicted that G.K.'s tendency to treat previously fixated targets as new discoveries (indexed by the re-click rate) would be reduced on object search trials compared with circle searches, because even if he did not recall previously searched object locations, he should be able to remember the identities of objects he had fixated previously. Circle and object displays used a 6 × 8 grid of 3.6 × 3.6° cells, with pictures of 24 different everyday objects from a standard set (Snodgrass and Vanderwart, 1980) embedded amongst 24 identical circles. Each element subtended 1.8°.

Re-click rate was 2.76 times greater during searches for circles than for objects (median re-click rates were 0.235 and 0.095, respectively; Mann–Whitney U = 0, P < 0.02; n = 4 of each type of search). This shows that G.K. was able to use non-spatial memory of which objects had been visited previously in order to guide his performance. Thus his impairment of retaining searched items is not a global deficit: short-term retention of visited object identities appears intact, whereas retention of searched locations is impaired.

Although re-click rate was reduced when the spatial memory burden was reduced by using memorable target identities (object search), the degree of neglect did not differ significantly between the two conditions [the percentage of targets missed (all on the left) was 20.8% for circles versus 16.7% for objects; Mann–Whitney U = 6.5, P > 0.5]. Nor did re-fixation rate differ significantly between object and circle searches (0.92 versus 1.03; U = 5.5, P = 0.5). The implications of these two findings are discussed later.

Control experiments testing visual localization

It might be argued that an apparent impairment of retaining previously fixated locations across saccades in our search tasks could be secondary to perceptual mislocalization of visual stimuli. As a consequence, patients experiencing perceptual difficulties in localizing targets might be unaware that they previously had looked at the targets when re-fixating them. We therefore devised two visual localization tasks specifically to test whether G.K.'s performance on our search tasks could be attributed simply to perceptual mislocalization of visual stimuli.

The general principle underlying both these tasks was similar. G.K. was shown a target (a T, like the targets used in our search displays) which was then masked and followed by a second target (another T). This appeared either at the same location as the first (50% of trials), or to the left (25% of trials) or right of it (25% of trials). G.K.'s task was to say whether the second target occupied the same location or a different location from the first, i.e. two alternative forced-choice discrimination. In trials where the second target occupied a different location from the first, the angular distance between them was always 2.0°. This distance was chosen because it corresponds to the mean minimum inter-target separations for the dense 19 Ts + 44 distractor search displays used in Experiment 1. We reasoned that if G.K. could discriminate readily between two targets which are separated by this angular distance, perceptual mislocalization was unlikely to explain his deficit in retaining the locations of previously fixated targets across saccades, during search. The interval between presentation of the targets was 2 s. We did not use longer durations because this would have encouraged substantial saccades between the two stimulus presentations, undermining our aim of comparing perceptual localization with retention of locations across saccades.

In the first such control test, in order to make our localization task resemble as closely as possible the localization required in our search tasks, we used a surround which consisted of a random array of 18 light grey Ts interspersed randomly among 44 Ls on a uniform dark grey background (Fig. 6A). This array remained on throughout the trial. It thus resembled our 19 Ts + 44 Ls search displays, in the size of the elements, their spacing and their randomized distribution across trials. However, it differed in one important respect: a central zone (measuring 1.6° above and below the centre of the screen and 2.0° to either side of the centre) did not contain any elements. Thus one element was initially missing in the centre of the display.

Fig. 6

Control visual localization tasks. Each panel was displayed for 2 s. The observer's task was to determine whether the second white T occupied the same or a different location from the first. In the first localization task (A) there was a surrounding array of Ts and Ls; in the second localization task (B) there was no surrounding array.

Figure 6A illustrates the sequence of events in the first localization test. Two seconds after the appearance of the random array of 18 Ts and 44 Ls, a light grey rectangle (3.1° vertically × 4.1° horizontally) appeared in the centre of the screen for 2 s and G.K. was asked to look at it. Following this, a white T (of the same dimensions as the other grey Ts in the display), which we termed target 1, appeared for 2 s. The location of this T was centred vertically but was jittered horizontally from trial to trial, always within ±0.22° of the horizontal midpoint. Target 1 was followed by a mask for 2 s. This consisted of 160 small (0.27 × 0.27°) white squares randomly superimposed on a uniform grey background, with the squares allowed to overlap each other. The mask region measured 3.1° vertically and 29.0° horizontally, centred at about the midpoint of the screen. It was followed by another white T (with the same attributes as the first white T, and surrounded by the same 44 Ls and 18 Ts) for 2 s. This target 2 appeared either in the same location as target 1, or 2.0° to the left or right. Forty trials (20 with targets 1 and 2 appearing in the same location; 10 with target 2 to the left of target 1; and 10 with target 2 to the right of target 1) were performed in a random sequence. G.K.'s responses were correct on 36 of 40 trials. Thus on 90% of trials, he could discriminate between two locations which were separated only by the mean minimum distance that had been used between items within our dense (19 Ts + 44 Ls) search task (Fig. 2A).

It might be argued that an observer could perform well on this test simply by judging the local density of elements within part of the central zone, rather than by localizing the first and second targets. Thus, if the central zone appeared empty at the end of the trial, target 2 was most likely to have occupied a different location from that occupied by target 1 (see Fig. 4A). We therefore developed a second localization task. This was exactly the same as the first except that there was no surrounding display of 18 Ts and 44 Ls. Thus, in this task, an observer could not use the local density of elements to make judgements of the relative positions of targets 1 and 2. Figure 6B illustrates the sequence of events in this task. A light grey rectangle first appeared at the centre and was followed by target 1, then the mask, and then target 2, as in the first experiment. Again, there were 40 trials (20 with targets 1 and 2 appearing in the same location; 10 with target 2 to the left of target 1; and 10 with target 2 to the right of target 1). On this task, he scored 35/40 (88%). G.K.'s relatively good performance on both control tasks indicates that frank perceptual mislocalization is unlikely to explain his deficit in retaining previously fixated locations across saccades during our search tasks (i.e. the deficit implied by his abnormally high re-click rate).

Discussion

Our experimental results show impaired retention of previously fixated visual locations, across saccades, during search in a patient with left neglect following focal infarction of the right inferior parietal lobe. Our novel tasks not only measured gaze direction during search, but also probed memory for searched locations explicitly, by requiring judgements (via the `click' responses) of whether the currently fixated target was a new discovery, or instead had already been examined during previous fixations in the search. We found that target locations frequently were re-fixated and, most importantly, were also treated as new discoveries at an abnormally high rate by the parietal patient. In contrast, healthy individuals rarely re-fixated targets, and hardly ever mistook previously fixated targets for new discoveries when they did re-fixate them. We propose that, when combined with a spatial bias to the right (Posner et al., 1984; Kinsbourne, 1987; Bisiach and Vallar, 1988; De Renzi et al., 1989; Eglin et al., 1989), an impairment in retaining previously searched locations across saccades leads to frequent returns to previously fixated items on the favoured side, thus contributing to prolonged neglect of items on the left.

The patient's search pattern showed characteristic abnormalities that are consistent with several previous studies of neglect. His search typically was restricted to the right, consistent with his clinical left neglect. In addition, he re-fixated items on the right at a highly abnormal rate (e.g. 13 times more than normal control subjects for displays like that in Fig. 2A). Such re-fixation of rightward items is apparent in some previous studies of neglect (e.g. Fig. 7 in Berhmann et al., 1997), but has rarely been commented on, although one recent eye movement study did report an abnormal rate of re-fixations by parietal patients in a dot-counting task (Zihl and Hebel, 1997).

Several influential accounts of neglect (e.g. Kinsbourne, 1987; De Renzi et al., 1989) emphasize that attention can be excessively attracted to the right end of a search array in patients with left neglect, and that parietal neglect patients may also have specific difficulties in disengaging attention from this favoured ipsilesional side (e.g. Posner et al., 1984; Eglin et al., 1989). Such deficits are undoubtedly important components of neglect, and also contribute to performance in the present patient. For instance, the `magnetic attraction' of ipsilesional items can explain why search is biased to the right. Difficulties in disengaging from right items may also arise, although it should be noted that G.K. made as many leftward as rightward saccades, characteristically returning to re-fixate right items after several intervening saccades (see below), rather than failing to break fixation. However, the rightward bias and re-fixations are not, on their own, the critical new observation of our study, which concerned the re-click rate. This revealed that the patient often considered a previously fixated target to be a novel discovery. This suggests a deficit in retaining previously searched locations across saccades, consistent with the deficit we also found on standardized tests of spatial working memory.

Furthermore, the severity of G.K.'s neglect in different conditions correlated with the frequency of his re-click errors (Fig. 4). Thus, an increased tendency to treat previously searched items on the right as new discoveries was associated with stronger left neglect. Reducing the load on spatial memory in various ways reduced his neglect, thus supporting our hypothesis that a difficulty in retaining previously searched locations, when combined with the bias towards the right, exacerbates continued neglect of the left.

It should be acknowledged that G.K. did not re-click every single target that he re-fixated, which might be taken to indicate some degree of residual spatial memory. However, the important point is that his performance was highly abnormal when compared with that of healthy observers. For instance, in our dense search task (Fig 2A), which poses the highest load on spatial memory, he re-fixated targets 13 times more than age-matched control subjects. Critically, he mistook the targets for new discoveries (i.e. re-clicked) 34 times more often than normal subjects. On this particular search task (our hardest), healthy individuals had a re-click rate of only 0.05. G.K. never managed so low a rate, even in our much simpler conditions, with the one exception of displays containing only two items, which pose a negligible spatial memory load. This is exactly the pattern predicted if a deficit in retaining searched locations across saccades determined the patient's re-click rate, as then his re-clicks should be reduced as the load on spatial memory is eliminated.

Impaired retention of searched locations versus compulsive perseveration

It is a common clinical observation than some neglect patients may make multiple visible marks on individual items during standard paper-and-pen cancellation tasks. Clearly, this behaviour cannot be due to deficient memory, since the initial mark is visible. A recent study (Na et al., 1999) documented such behaviour formally, and observed that it is most common in patients with frontal lesions. Any tendency visibly to re-cancel targets that have already been crossed out, during standard cancellation tasks, may thus reflect the compulsive perseverative tendencies which classically are associated with frontal damage (Shallice, 1988).

This behaviour (and its anatomy) is quite unlike the deficit in the present patient, G.K., whose focal parietal infarct spared the frontal lobe structurally. He never re-cancelled targets (i.e. never made multiple visible marks on individual targets) during repeated testing with paper-and-pen cancellation. Moreover, his pathological re-fixations, and his critically high rate of re-clicks, cannot be attributed merely to compulsive perseverations, for several reasons. First, there is no a priori reason for re-clicks to increase with target set size as we observed (Figs 3 and 4). It is possible to re-click compulsively just as often when confronted with two targets as with 19 targets. Secondly, our scoring of re-click rate was conservative, being weighted against our prediction: small absolute numbers of re-clicks would yield very high re-click rates for displays with fewer targets, yet we actually found higher rates when there were more targets. Thirdly, this pattern was not caused merely by displays with fewer targets taking less saccades to search than those with more targets (with more saccades perhaps then leading to more clicks). Clicks per saccade on target actually decreased with displays of 19 targets (0.41) compared with those displays with two targets (0.69). Fourthly, the reduction in re-click rate during searches for memorable objects versus indistinguishable circles (Fig. 5) supports our proposal that impaired spatial memory for locations previously searched, and not compulsive clicking, was the critical factor in determining re-clicking for previously visited targets. G.K. was able to prevent re-clicks when he could remember that a particular target had been discovered previously, by means of its unique identity. Finally, the critical re-clicks in G.K. did not occur immediately after the initial click (as one would expect in compulsive perseverative behaviour). Instead, numerous saccades typically intervened between the initial fixation/click, and the later return and re-click. On average, successive click and re-click for a given target were separated by 15 s in G.K. (and he averaged 3 saccades/s). Thus, the pathological re-clicks in G.K. were not caused by compulsive button-pressing, nor by an inability to move his gaze away from a particular target. Instead, they reflect an inability to retain a particular location as previously fixated and clicked, when the patient returns his gaze there after numerous intervening saccades.

Although there was a significant reduction in re-click rate when G.K. searched for distinctive objects rather than a circle (Fig. 5), there was no significant change in his re-fixations (nor his neglect) between these conditions (see also Wojciulik et al., 2001). We suggest that G.K.'s deficit in retaining previously searched locations led to re-fixation of right targets, regardless of whether these were objects or circles, because he could not distinguish the different small objects using peripheral vision alone. Thus, to discriminate objects, he had to fixate each one. Once a target was foveated, he was less likely to re-click it if that target was an object whose unique identity he recognized from a previous visit. Thus, re-click rate fell with this non-spatial aid to memory, but the degree of neglect did not alter because distinctive objects were helpful to memory only after they had been fixated and identified. Recursive search of targets on the right (including re-fixation rate) was therefore unaltered, and so the left neglect remained.

Spatial working memory during search requires localization across saccades

The results of our control tasks on perceptual localization (Fig. 6) demonstrate that G.K. could discriminate well between two locations which were separated by the mean minimum distance that separated targets in our densest search task (Fig. 2A). His inability to remember previously fixated target locations is therefore unlikely to be due simply to frank perceptual mislocalization. However, it may be that he has difficulty in retaining visual locations across saccades. For search tasks, this would, in effect, be equivalent to an impairment of spatial working memory for searched locations. When combined with a spatial bias to the right, an impairment in updating visual locations across saccades could thus explain the pathological pattern that we have observed, in which stimuli on the right were re-examined recursively, with the patient typically judging that he was searching them for the first time.

Electrophysiological studies in the macaque monkey are consistent with this proposal. Several studies have demonstrated a role for posterior parietal areas in working memory for stimulus location, when the monkey saccades to remembered target positions (Mazzoni et al., 1996; Colby and Goldberg, 1999). Some studies provide evidence for a specific underlying mechanism: the updated representation of visual location across saccades in posterior parietal neurones (Duhamel et al., 1992b). The results of functional imaging studies in humans are also consistent with our proposal that parietal cortex may be associated with spatial working memory for visual locations. While much research on working memory traditionally has considered the frontal cortex, right parietal activation is also observed when neurologically normal subjects engage in spatial working memory tasks (Jonides et al., 1993; Owen et al., 1996a; Petrides, 1996; Courtney et al., 1997; D'Esposito et al., 1998; LaBar et al., 1999), including saccading to remembered positions (Anderson et al., 1994; O'Sullivan et al., 1995).

Lesion studies concerning spatial working memory in humans typically have focused primarily on the role of the frontal lobes (Owen et al., 1990; Owen et al., 1996b), but classic studies on human deficits in spatial working memory (Warrington and James, 1967; De Renzi et al., 1977) suggest the involvement of more posterior sites (though most were performed before the advent of structural brain imaging). More recently, Duhamel and colleagues (Duhamel et al., 1992a) and Heide and colleagues (Heide et al., 1995) both observed a specific deficit in retaining target locations across saccades for unilateral parietal patients (although they did not relate this to pathological search behaviour, as we have done here). In their double-step saccade paradigm, subjects are shown two targets in succession, so briefly that the second target has disappeared before the saccade to the first commences. Unilateral lesions of the human parietal lobe (especially in the right hemisphere) lead to severe impairments in correctly directing gaze to the remembered locations of targets across saccades in this task (Heide et al., 1995). In right parietal patients, the deficit arises even for targets on the right (ipsilesional) side, when the previous saccade took the eye away from them in a leftward direction. This is consistent with the deficit described here in retaining rightward target locations as previously fixated, after the search has proceeded in the leftward direction, and then later returned.

Lateral spatial bias plus impaired retention of previously searched locations

We are not suggesting that the proposed deficit in retaining previously searched locations across saccades can, on its own, explain all aspects of the neglect syndrome. It is recognized increasingly that neglect may involve several component deficits (Vallar, 1998), each of which may exacerbate the others. Our aim in this paper is not to challenge previously proposed deficits, but to suggest an additional component: a failure to retain visual locations across saccades. On its own, such a deficit might not produce neglect but, when combined with the lateral bias to the right, which is the most striking clinical feature of neglect, the proposed deficit would exacerbate neglect of the left, by inducing repeated returns to rightward locations which the patient would consider to be fixated for the first time.

One might ask how a deficit in retaining previously searched locations would contribute to neglect in standard cancellation tasks, given that visible marks are made at previously visited locations in such tasks, which might appear to circumvent any spatial memory requirement. However, it should be noted that in many paper-and-pencil versions of cancellation, the marks are not clearly visible in peripheral vision, and so may have to be re-fixated before the patient can determine whether a particular location has already been visited. If so, existing marks would not break the cycle of recursive search through rightward items (as argued above, for the search through the distinctive objects in the present study).

A failure to retain searched locations across saccades may also explain a number of previous findings using variations of standard cancellation tasks. First, it has been proposed that visible marks on the right may themselves encourage continued neglect of the left in cancellation tasks, by making the right side even more salient for the patient. In one well-known study (Mark et al., 1988), it was found that neglect patients proceeded further to the left when required to `rub out' targets rather than to mark them. However, there are at least two possible interpretations for this amelioration of neglect (which are not mutually exclusive). The first is that rubbing out targets reduces the tendency to orient to salient stimuli on the right, while the second is that rubbing out targets removes any requirement to remember whether they have been fixated previously. Secondly, a recent study (Wojciulik et al., 2001) reported that neglect in cancellation is more severe when the patient makes an invisible mark on each target found, rather than a thick red mark, consistent with the deficit we propose in retaining searched locations when there is no external cue as to whether a particular location has already been searched. As in the present study, the severity of left neglect in different conditions correlated with the frequency of re-visits to the right, consistent with the mechanism proposed here. Finally, Halligan and Marshall reported that severe left neglect in cancellation can be found even when a patient is required first to mark the left corners of the cancellation sheet (Halligan and Marshall, 1993). One possible interpretation of this would also invoke a deficit in retaining previously fixated locations across saccades. Thus, even though the patient may fixate the left edge of the array when marking the corners, this location would not be retained over subsequent saccades, such as those made when the patient begins cancelling on the right.

However, more importantly than being able to explain performance in cancellation tasks, the deficit we propose here is of direct relevance to neglect in daily life. Consider what happens when we view a scene in daily life, and search around it for relevant information. Unlike standard cancellation tests, we do not leave a mark on each location we fixate and inspect. Like the search task described here, we have to keep track of locations we have examined previously if we are to avoid re-visiting them. If that process fails and, in addition, rightward locations attract attention, the combined effects of these impairments will exacerbate each other, leading to recursive searching of the right side, and thus the continued failure to explore left space which characterizes many aspects of neglect in daily life.

Acknowledgments

We thank G.K. for his participation, and Karen Clarke for her assistance. This work was supported by grants from the Wellcome Trust to M.H. and C.K., HFSP to E.W., and by MRC and McDonnell-Pew grants to J.D.

References

View Abstract