OUP user menu

A functional imaging study of translation and language switching

Cathy J. Price, David W. Green, Roswitha von Studnitz
DOI: http://dx.doi.org/10.1093/brain/122.12.2221 2221-2235 First published online: 1 December 1999

Summary

The neural systems underlying translation and language switching were investigated using PET. Proficient German–English adult bilinguals were scanned whilst either translating or reading visually presented words in German (L1), English (L2) or alternating L1/L2. We refer to alternating L1/L2 as `switching'. The results revealed contrasting patterns of activation for translation and switching, suggesting at least partially independent mechanisms. Translation, but not switching, increased activity in the anterior cingulate and subcortical structures whilst decreasing activation in several other temporal and parietal language areas associated with the meaning of words. Translation also increased activation in regions associated with articulation (the anterior insula, cerebellum and supplementary motor area) arguably because the reading response to the stimulus must be inhibited whilst a response in a different language is activated. In contrast, switching the input language resulted in activation of Broca's area and the supramarginal gyri, areas associated with phonological recoding. The results are discussed in terms of the cognitive control of language processes.

  • PET
  • bilingualism
  • translation
  • switching
  • BA = Brodmann area
  • IC model = Inhibitory Control model
  • L1 = first language
  • L2 = second language
  • SMA = supplementary motor area

Introduction

Bilingual speakers are able to translate from one language to another and to switch between their two languages in order to communicate. Yet little is known about how these tasks are achieved. One way to proceed is to seek converging evidence from different techniques such as psycholinguistic and functional imaging studies. We first outline a cognitive model of single word translation and language switching formulated from behavioural data. We describe from this how we made predictions regarding the systems we expected to see activated in functional neuroimaging studies. Next we detail the likely anatomical correlates of these systems and then describe the PET study of translation and language switching which we used to test our predictions.

A cognitive model of translation and language switching

Analytically, the system representing words and word meanings in bilinguals (the bilingual lexico-semantic system; Votaw, 1992) comprises a number of distinct components specifying what words look like (orthography), sound like (phonology) and mean (semantics), their syntactic properties (or lemmas) and an output system that specifies the pronunciation of word forms (Patterson and Shewell, 1987; Levelt, 1989). The orthography and phonology associated with a lexical concept varies from language to language and therefore, in bilingual subjects and polyglots there will be distinct representations at the orthographic and phonological levels.

The bilingual lexico-semantic system supports many different activities such as reading material in one language rather than another, speaking in one language rather than another and switching between languages in speech production and in translation. In order to achieve these tasks, the system must be controlled. For instance, in word translation, how is it that individuals are able to produce the translation equivalent rather than merely naming the presented word (Green, 1986)? One approach to this question is to suppose that control is achieved by regulating the activation of the language systems (e.g. Grosjean, 1997; Paradis, 1997) by, for example, increasing the activation of items in the output language (De Bot and Schreuder, 1993; Poulisse and Bongaerts, 1994) or by inhibiting items in the non-target language as in the visual word recognition model of Dijkstra and van Heuven (Dijkstra and van Heuven, 1998). However, such approaches leave open the precise control mechanisms and often focus on just one kind of task (e.g. speech production or visual word recognition).

The Inhibitory Control (IC) model of Green (1998a, b), which builds on the supervisory attentional system model of Norman and Shallice (Norman and Shallice, 1980, 1986; also Shallice, 1994), proposes the notion of a functional control circuit in which there are three basic loci of control: (i) an executive locus—the supervisory attentional system used for establishing and maintaining goals; (ii) a locus at the level of language task schemas and (iii) a locus within the bilingual lexico-semantic system itself (at the lemma level). In order to speak in one language rather than another or to translate between languages, individuals establish `language task schemas'. These are effectively action schemas in the domain of language and link input to, and output from, the bilingual lexico-semantic system to responses. Language schemas at a given level are in competition and responses are produced in accordance with the currently dominating schema. Selection of a word in the correct language occurs at the lemma level by virtue of a language tag. At this locus, competitors for selection in the non-target language are inhibited.

According to the IC model, language task schemas are external to the bilingual lexico-semantic system and compete to control outputs from it. A schema for word production in the dominant language (L1) is in competition with a production schema for a second language (L2). It follows that in order to speak in L2, individuals must inhibit the schema for word production in L1. If individuals are required to switch between languages, then a currently active schema (e.g. for naming in L1 or for reaching a lexical decision in L1, i.e. deciding if a letter string is an L1 word or not) must be inhibited and the previously inhibited schema for that task activated (i.e. for naming in L2 or for reaching a lexical decision in L2). This process takes time and should yield a switching cost. Behavioural data confirm such a cost both in terms of switching between languages at the production level only (e.g. when individuals are naming numerals aloud; Meuter and Allport, 1999) and at the input level only (e.g. in lexical decision; von Studnitz and Green, 1997). Moreover, switching costs are not symmetrical: individuals take longer to switch into their more dominant language (e.g. Meuter and Allport, 1999) as would be expected if the production schema for the dominant language is more strongly inhibited and requires more time to be reactivated.

Schema competition also arises in the case of translation—a focus of the present paper. A schema for forward translation (i.e. for translating from a person's dominant language, L1, to their second language, L2: L1 → L2) specifies L1 as the target input and calls a schema for the production of L2 as output. It must also inhibit the schema for naming a stimulus word in L1. Since each stimulus presentation also activates this naming schema, there is a dynamic competition between the two schemas, analogous to the process of contention scheduling (Shallice, 1994) in which the non-relevant schema is inhibited.

An expectation of the IC model and our first prediction is that translation, relative to reading, should elicit increased activity in areas mediating the competition between schemas (prediction 1). In cases where individuals are required to switch between forward translation and backward translation (i.e. where the input language is alternating) the competition between schemas may be exaggerated since each translation schema must also be repeatedly activated and then inhibited. Executive control may be needed in such circumstances to modulate the activation of the non-target translation schema (prediction 1a) [We note by way of contrast that executive control may invariably be required when individuals have to name pictures or numerals in alternate languages. In such circumstances, resolving competition between the target and non-target candidates may be more difficult because the lemmas for candidate word forms only become available after semantic access and rely exclusively on such access.]

We turn now to the effects of translation within the bilingual lexico-semantic system. The links between word forms in different languages are complex (see Green, 1998a, Note 2). We consider possible modulations by the language task schema at the semantic level and at the level of word recognition and word production.

Psycholinguistic data emphasize two different routes for translation (Kroll and Stewart, 1994; see also Kroll and De Groot, 1997): a non-semantic, direct route in which the word forms of translation equivalents are linked at the lemma level (Jescheniak and Levelt, 1994) and an indirect semantic route in which they are connected via their meaning (i.e. their lexical concepts). According to the IC model, word selection in either route involves lemma activation and the inhibition of lemmas with the non-target language tag. These two routes appear to be differentially involved in forward and backward translation of single words. In forward translation (i.e. L1 → L2) the semantic route dominates, whereas in backward translation (i.e. L2 → L1) the lexical route dominates, reflecting the acquisition of the L2 word in the context of a pre-existing lexical concept-word form link in L1.

Kroll and Stewart asked bilinguals to translate individual words that were either blocked by category (e.g. a series of words from the category of furniture followed by a series of words from the category of vehicle and so on) or that were randomly selected from different categories. They recorded the time required to translate each one. Kroll and Stewart predicted that if translation was mediated at the semantic level then blocking items by category would tend to activate exemplars of that category and produce increased competition for selection and so delay response time (precisely this pattern is found when unilinguals name pictures blocked by category). For forward translation, participants did indeed take longer to translate words when they were blocked by category compared with when they were presented in a random order. There was no effect of category blocking for backward translation (Kroll and Stewart, 1994).

This outcome suggests that forward translation may involve more semantic processing than backward translation. On this supposition we predicted increased activation in semantic regions in forward translation (prediction 2). Differences between forward and backward translation, may also depend on word frequency which affects not only whether or not a translation equivalent is known at all, but also what is known of the meaning of any word in either L1 or in L2. Conceivably then, any differences between forward and backward translation will be greater for high frequency words since low frequency words will have sparse semantic representations in any case (predication 2a).

In addition to these possible semantic differences as a function of translation direction, other effects of language task are to be expected at the level of word recognition and production. Translation relative to reading may modulate late stages in the word production process because the name of the presented word (activated by direct connections between the input and output word forms) must be blocked during translation. Current behavioural data do not permit us to be more definite but we note that models of unilingual speech production presume an internal monitor that only edits output once phonological encoding is complete (e.g. Levelt, 1989).

Switching languages may also exert a number of effects within the lexico-semantic system, including the joint activation of different mappings of orthography to phonology (one for L1 and one for L2) that may compete to produce a phonological representation of the string. Experimental evidence suggests that bilingual individuals do generate such mappings for a current non-target language. Hebrew–English bilinguals, responding in Hebrew, showed Stroop interference even when the verbal stimulus was non-sensical in Hebrew as long as it sounded like a Hebrew colour word according to grapheme–phoneme mappings in English (Tzelgov et al., 1996). For instance, the letter string `adom' printed in green ink and pronounced according to English grapheme–phoneme mappings sounds like the Hebrew word for red. Brysbaert and colleagues, using a masked priming technique, showed that bilinguals presented with L2 targets process primes in terms of both L2 grapheme–phoneme mappings and L1 mappings (Brysbaert et al., 1999). Switching might encourage the joint activation of such mappings and yield a conflict. Our third prediction, therefore, is that switching and translation might show different patterns of activation: regions associated with mapping orthography to phonology might be sensitive to switching and regions associated with the production of responses sensitive to translation (prediction 3).

In summary, our predictions regarding the cognitive systems we expect to see activated in a functional neuroimaging study of translation and language switching are as follows. (i) Translation, relative to reading, will be associated with increased activation in the regions involved in the control of action. Specifically, translation should elicit increased activity in areas mediating the competition between schemas (prediction 1). Executive control may be needed to modulate the activity of competing schema when the input language is switching (prediction 1a). (ii) There will be differential semantic involvement as a function of translation direction. Forward translation will activate semantics more than backward translation (prediction 2) and this effect will be exaggerated for high frequency words (prediction 2a). (iii) Switching and translation might show different patterns in regions associated with mapping orthography to phonology and in regions associated with the production of responses (prediction 3).

Predicted patterns of activation for translation and language switching—neuroanatomical regions

Our basic assumption, embodied in the IC model, is that the mechanisms of language control share much in common with the control of action in general. The notion of a functional control circuit implies that a number of regions are implicated in such control. Thus, contrary to Penfield and Roberts (Penfield and Roberts, 1959), for instance, we suppose no unitary switch mechanism specific to changing language. Indeed, neuropsychological case reports provide no warrant for it. Evidence that the supramarginal gyri are critical (Herschmann and Potzl, 1983; Kauders, 1983; Potzl, 1983) is countered by the patients with lesions in such regions without switching problems (Gloning and Gloning, 1983; Minkowski, 1983), yet the supramarginal gyri, we will suggest, are nevertheless involved in switching. On the other hand, neuropsychological data do suggest the relevance of systems (e.g. the frontal lobes) involved in the general control of action. In the following paragraphs the neuroanatomical background to our expectations is discussed, particularly in relation to the results of a previous functional imaging study of translation by Klein and colleagues (Klein et al., 1995).

We predict that translation will involve regions that have been implicated in the control of action, in particular the anterior cingulate and subcortical structures (prediction 1) and conceivably the dorsolateral prefrontal cortex in the case of switching input language during translation (prediction 1a).

The prediction that the anterior cingulate will be involved in translating comes from four functional imaging studies involving either Stroop tasks or Stroop-like tasks which all show increased activation of the anterior cingulate region (Pardo et al., 1990; Bench et al., 1993; George et al., 1994; Taylor et al., 1994). Translation shares some computational similarities with the standard Stroop task in which a dominant response schema (naming the printed word) has to be inhibited in order to allow the hue in which the word is printed to be named. Most relevant is the study by Taylor and colleagues. These researchers required participants to respond to the presentation of a single letter (e.g. D) either by naming that letter or by naming a different letter that they had learned as an associate of the target letter (e.g. J). In the latter condition individuals must keep from naming the presented letter and instead name the associated letter. Taylor and colleagues found increased activation in this condition in the left cingulate sulcus (Taylor et al., 1994). If the computational overlap between Stroop tasks and translation is accepted then translation relative to reading should induce increased activation in the anterior cingulate regions.

Our expectations about subcortical involvement in translation and language switching derive from evidence that damage to, or disruption of the basal ganglia, affects the selection for action (Lynn and Robbins, 1975, cited in Norman and Shallice, 1980; Canavan et al., 1989) and from the idea of common mechanisms for the selection of non-verbal and linguistic actions [see the synoptic view of Crosson and colleagues (Crosson et al., 1988) concerning the relationship between cortical and subcortical areas in speech production]. We might also expect differential effects in subcortical regions as a function of translation direction. For example, Fabbro and Paradis reported the case of patient C.B. with an ischaemic lesion to a small portion of putamen, head of the caudate nucleus and the anterior portion of the internal capsule of the left hemisphere, whose spontaneous speech in L1 and L3 was good, but who could not translate from L3 to L1 yet could translate from L1 to L3 (Fabbro and Paradis, 1995).

The expectation that the dorsolateral prefrontal cortex may be involved in translating, especially in the context of switching between languages, comes partly from neuropsychological case reports. The frontal lobes can play an important role in language processing with respect to inhibiting a pre-potent response (e.g. Burgess and Shallice, 1996). Indeed, damage to this region in bilinguals can lead to inadvertent language mixing in both conversation and picture naming (Stengel and Zelmanowicz, 1933) and does produce impaired performance in numeral naming during unpredictable language switching in which the language of response is cued by a change in the colour of the background (Meuter and Humphreys, 1997). Numeral naming, as pointed out earlier, contrasts with translation and so it remains unclear whether or not dorsolateral frontal activation will necessarily increase in translation or in language switching during translation.

Functional imaging studies of cued word generation (Petersen et al., 1988; Wise et al., 1991; Klein et al., 1995) also indicate the involvement of the anterior cingulate, left prefrontal and subcortical structures when subjects must generate a semantically related response rather than name the word presented. Of direct relevance to our study is the functional imaging study of translation by Klein and colleagues. These authors found increased activation in the anterior cingulate [Brodmann area (BA) 24/32] and in the dorsolateral prefrontal cortex (BA 45, 46, 47, 8, 9, 10, 11) during translation relative to repetition of auditorily-presented words. In addition, they reported activity associated with translation in the left inferior temporal lobe (BA 37/20), the superior parietal cortex (BA 7) and the right cerebellum. However, the same system of regions was also activated during word generation tasks that required participants to retrieve rhyming words or synonyms within one language, and there was no differential activation related to translation direction when the language of speech output was controlled (Klein et al., 1995). The absence of differences in the activation profiles for translation, rhyme and synonym generation in the study by Klein and colleagues is notable given that subsequent functional imaging studies [both PET and functional MRI (fMRI)] have reported activity changes when the type of word generation task is manipulated within one language (Shaywitz et al., 1995; Mummery et al., 1996; Warburton et al., 1996) and when the same task is performed in different languages (Perani et al. 1996; Dehaene et al., 1997; Kim et al., 1997). These latter studies suggest a lack of sensitivity in the study by Klein and colleagues and indicate the need for further investigation.

Areas associated with semantic processing

Recent data from functional imaging studies and from earlier neuropsychological case studies converge on the view that the left extrasylvian temporal cortex (in particular, BA 20, 38 and 39) and the left inferior frontal cortex (BA 47) mediate semantic processing (Vandenberghe et al., 1996; Price et al., 1997). If forward translation makes more use of a semantic route compared with backward translation [consistent with the data of Kroll and Stewart (Kroll and Stewart, 1994)] it should lead to enhanced activation in at least some parts of this system (prediction 2) and there should be differential effects of frequency (prediction 2a).

Areas associated with word recognition and production

In addition to areas involved in the control of language tasks and semantics, we also expect to see modulation of activity in the regions associated with word recognition and production (prediction 3). Regions sustaining word production can be divided into those involved in articulation, i.e. the premotor cortex, supplementary motor area (SMA) and cerebellum, and those involved in retrieving phonology, i.e. left lateral posterior basal temporal cortex (BA 37), left anterior inferior parietal cortex (BA 40) and left inferior frontal cortex (BA 44) (see Price, 1997). We note in particular that the supramarginal gyri have been implicated in mapping orthography to phonology (Price, 1997). The supramarginal gyri are activated when (i) subjects make phonological judgements on auditorily or visually presented words (Demonet et al., 1994; Price et al., 1997); (ii) words are contrasted to pictures (Vandenberghe et al., 1996; Moore and Price, 1999); and (iii) pseudowords are contrasted to real words. Given the behavioural evidence that L1 and L2 mappings can be jointly activated, we predicted increased activation in this region during language switching. Our study design (see below) allows us to determine which of these language systems are modulated by translation and language switching.

PET study of translation and language switching

The study we report in this paper focuses on the neural correlates of translation and the mechanisms required to switch between languages in German subjects who were highly proficient in speaking English. There were two tasks (translation and reading); three types of stimuli [L1 (German), L2 (English) or alternating L1 and L2 (i.e. the switching conditions)]; and two levels of word frequency (high and low). The effect of translation can be examined independently of the effect of language because either input or output can be kept constant. For instance, the input is constant for reading in L1 or translating from L1 to L2, and the output is constant for reading in L1 and translating from L2 to L1. This design is similar to that of Klein and colleagues (Klein et al., 1995) except (i) the modality of presentation was visual, rather than auditory; (ii) we included a condition to examine switching between languages; (iii) we included word frequency as a factor; and (iv) there were six translation conditions per subject, rather than two.

Although we focus on the neural correlates of translation and the mechanisms required to switch between languages, our study design allows us to consider regions involved in the perception or production of L1 and L2 (subject to certain considerations we note below). Previous neuroimaging studies have shown variable results concerning the neural organization of two languages. These studies can be divided into those investigating differences in language comprehension and those investigating differences in language of production.

For language comprehension, Perani and colleagues showed, using PET, that a wider set of regions, particularly the bilateral temporal poles, were more active when individuals listened to stories in their first language (L1) compared with their second language (L2), but there were no areas that were more active for L2 (Perani et al., 1996). A similar paradigm was adapted for an fMRI study by the same group of authors (Dehaene et al., 1997). Individual subject analyses revealed that while there is consistent activation for L1 across subjects, there is inconsistent activation associated with L2 that is not detected when subjects are grouped together. Subsequently, the same authors (Perani et al., 1998) have demonstrated that differences in the comprehension of L1 and L2 depend on the proficiency of the subjects.

For language production, similar brain regions have been observed to be active for L1 or L2 during repetition (Klein et al., 1994) and translation (Klein et al., 1995) of single words. However, an fMRI study by Kim and colleagues using a sentence generation task suggests that, although a common representation for language production may exist for early bilinguals (those exposed to two languages from infancy), distinct regions in Broca's area may mediate sentence production of L1 and L2 in late bilinguals (Kim et al., 1997). However, this finding has not been replicated in a recent fMRI study reported by Chee and colleagues (Chee et al., 1999). We note that differences between studies could reflect the type of language processes engaged by the tasks, the language itself (e.g. French or Chinese), the age of language acquisition, the proficiency of the participants and/or the resolving power of the technique (fMRI or PET).

In the present study, the participants were highly proficient bilinguals. Nevertheless, our experimental design included a manipulation of word frequency. By looking for consistent and differential effects for low and high frequency words, we were able to look for effects of task that were dependent or independent of language proficiency. Most commonly, individuals either know the translation equivalent or respond `don't know'. On occasion, individuals may experience a tip of the tongue state, i.e. they can access the word's lemma but cannot retrieve the word form. In such circumstances they might engage in extensive search. However, the present study reduces any tendency for subjects to search extensively for the translation equivalents by instructing them to say `No' or `Nein' (depending on the language of output) within the 3 s inter-stimulus interval. This procedure will, however, increase the trials recorded as errors, particularly in the low frequency condition where individuals are less likely to have ready access to the translation equivalents.

Method

Participants

The participants were six volunteer German/English right-handed male bilinguals with normal or corrected vision. Their mean age was 30.5 years (SD = 5.3) ranging between 25 and 39 years. All grew up with German and learned English as their first foreign language at a mean age of 8.8 years (SD = 2.3). At the time of the experiment they had been fluent in English for 9.8 years (SD = 4.5), range 5–17 years. In order to gain a measure of their language proficiency, we asked the participants, as part of a language background questionnaire, to rate their own proficiency in the two languages by assessing their relative proficiency in German compared with English out of a total 100%. The results of this indicated higher estimates for German than for English (62% versus 38%, SD = 15.1). Participants reported their current use of the two languages (out of a total 100%) to be predominantly English (written, 70% versus 30%, SD = 31.5; spoken, 61% versus 39%, SD = 27.0). Participants were given a choice of language to fill in the questionnaire; four chose the English version.

Handedness

All participants were strongly right handed on the Edinburgh Handedness Inventory (Medical Research Council Speech and Communication Unit, March 1970) consistent with left cerebral dominance for language. All participants gave informed consent to undergo PET scanning and MRI scanning. The study was approved by the combined Ethics Committee of the National Hospital of Neurology and Neurosurgery and the Institute of Neurology, London, and the Administration of Radioactive Substances Advisory Committee (UK).

Experimental design

The study was a fully-repeated measures 3 × 2 × 2 factorial design with the factors of language (L2, L1 or alternating L2/L1); task (reading or translating) and frequency (high or low). The materials, comprising separate matched sets of high frequency words and matched sets of low frequency words, were rotated across the task conditions (reading and translating) on a screen over the six participants. In consequence, a given lexical entry (e.g. the one for horse) appeared over participants both as an L2 word (`horse') and as an L1 word (`Pferd') and (over participants) was read and translated. Any differential effects of translation or language switching could therefore be examined relative to the reading baseline controlling for both stimulus input and response output. For each participant a set of high frequency words alternated with a set of low frequency words yielding 12 sets of trials overall presented in a counterbalanced order. There was no repetition of words within a participant.

Material

Six matched sets of 20 high frequency words in L2 plus their translation equivalents in L1 and six matched sets of 20 low frequency words in L2 plus their translation equivalents in L1 were used. These sets were matched as far as possible for letter length and syllable length (Table 1). Since the items were rotated across conditions, the mean frequency counts refer to these items when they were presented as L2 words and when they were presented as L1 words. The frequency counts for L2 words were derived from the Kucera and Francis norms for English (a corpus of one million words) (Kucera and Francis, 1967), whereas those for L1 words were derived from the Ruoff norms for German (a corpus of half a million words) (Ruoff, 1990). Retrospectively, we also computed mean word frequency (per million) for L2 and L1 words using the CELEX lexical database (Baayen et al., 1993); the Pearson correlations of the CELEX values with the Kucera and Francis values (English words) and with the Ruoff values (German words) were r = 0.958, P < 0.00001 and r = 0.544, P < 0.001, respectively (the corresponding values for Kendall's tau-b were also highly significant, 0.775 and 0.571, P < 0.0001 in both cases].

View this table:
Table 1

Mean frequencies for English words (Kucera and Francis, 1967) and for German words (Ruoff, 1990) and for both English words and German words (CELEX), together with their mean letter length and their mean syllable length (SDs in brackets)

CELEXKucera and FrancisRuoffLetter lengthSyllable length
English
High frequency169.6 (217.3)87.1 (211.4)4.7 (1.1)1.3 (0.5)
Low frequency 6.7 (7.2) 6.3 (6.4)4.7 (1.3)1.3 (0.6)
German
High frequency118.7 (217.7)104.2 (210.8)4.8 (1.1)1.5 (0.5)
Low frequency 4.2 (5.2) 2.1 (4.2)5.4 (1.5)1.8 (0.5)

Procedure and instructions

Participants were presented with both written and oral instructions before undergoing the PET study. They were informed that in the experiment proper they would be presented with 12 blocks of trials, consisting of 20 trials each. On each trial a word would appear on the screen overhead. Depending on the type of block, they would be required either to read the word and mouth its pronunciation or to mouth its translation, without generating any sound in either case (see Price et al., 1996bor prior use of such a technique). These instructions applied to both the single language and to the switching conditions. Participants were informed that their mouthing of each word would be monitored on the visual display in the adjacent room. For the translation conditions, they were additionally instructed that if they did not know the correct translation then they should mouth `Nein' if they were translating from L2 into L1 and mouth the word `No' when translating from L1 into L2. For the switching conditions, they were informed that successive words would be in different languages.

Before the experiment began each participant practised on each of the conditions using a separate set of words. In the 8-min interval between the blocks, participants were asked, where appropriate, which words they could not translate and were informed of their task for the next block of trials. Thirty seconds before a block began they were briefly reminded of their task. Presentation was paced, with words presented at the rate of one every 3 s, allowing time for individuals to mouth their response before the arrival of the next stimulus word, precluding extended word search.

PET scanning methods and data analysis

The brain was scanned with an ECAT EXACT HR + PET scanner (CTI Siemens, Knoxville, Tenn., USA). Participants received a 20-s intravenous bolus of H215O at a concentration of 55 Mbq/ml and a flow rate of 10 ml/min through a forearm cannula. The 12 scans from each participant were realigned using the first image as a reference. A T1-weighted MRI, (Siemens 2 Tesla Magnetom Vision MRI camera, Erlangen, Germany) was coregistered to the mean PET image for each subject and then stereotactically transformed to a standard MRI template in Talairach and Tournoux (Talairach and Tournoux, 1988) space. The same transformation matrix was subsequently applied to the PET images. The data were analysed with statistical parametric mapping (SPM 97, Wellcome Department of Cognitive Neurology, London, UK), implemented in Matlab (Mathworks Inc. Sherborn, Mass., USA) using standardized procedures (Friston et al., 1995a, b) and were smoothed with a Gaussian filter of 16 mm. Condition and subject effects were estimated according to the general linear model at each voxel. To test hypotheses about regionally specific condition effects, these estimates were compared using linear compounds or contrasts. The resulting set of voxel values for each contrast is an SPM of the t-statistic.

We report effects as significant if there were more than three voxels activated above a threshold of P < 0.001, uncorrected. The linear contrasts can be summarized as follows.

  • (i) The effect of translation: translation conditions were contrasted to reading conditions for high and low frequency words in L1, L2 and alternating L1 and L2 (six contrasts). Regions that were common to all contrasts were identified with conjunction analysis (Price and Friston, 1997), which sums over the effects and excludes regions where there are significant differences between the contrasts (i.e. interactions). We also excluded voxels from the common effect that were not significant (at a threshold of P < 0.05) for both high and low frequency conditions. Regions that were activated by translation specifically for either high or low frequency words were then identified, and qualified by checking the significance of the direct contrast between high and low frequency words on translation scans only. Similarly, regions that were activated by translation specifically for either switching or no switching were identified, and qualified by checking the significance of the direct contrast between switching and no switching on translation scans only.

  • (ii) The effect of switching: switching conditions (alternating L1 and L2) were contrasted with the no switching conditions (only L1 and only L2) for high and low frequency words during translation and reading (four contrasts). Conjunction analysis (see above) identified effects of switching that were common to task and word frequency and effects of switching that were specific to task or word frequency.

  • (iii) The effect of frequency: high versus low frequency conditions for translation and reading in L1, L2 and alternating L1 and L2 (six contrasts). We did not investigate the main effect of frequency but used these contrasts to qualify effects of frequency during translation and switching.

  • (iv) L1 versus L2 input: in order to control for task, we used conjunction analysis to find effects that were common to: reading L1 versus reading L2, and translating L1 versus translating L2.

  • (v) L1 versus L2 output: in order to control for task, we used conjunction analysis to find effects that were common to reading L1 versus reading L2, and translating L2 versus translating L1.

Results

Behavioural

As expected, word frequency affected the number of words participants reported being unable to translate. On such occasions individuals responded either by mouthing the word `No' or by mouthing the word `Nein' as appropriate (see above). Translation misses averaged 2.5% (9 out of 360) for high frequency words compared with an average of 30.8% (113 out of 360) for low frequency words. There was no differential effect of either translation direction or switching on reported misses for either high or low frequency words.

Neuroimaging

Translation versus reading: increases

Translation relative to reading increased activation in the anterior cingulate and bilateral subcortical structures (putamen and head of caudate) irrespective of word frequency, language and task (switching or no switching) (Table 2A). There was no significant activation in the dorsolateral prefrontal cortex for any of the conditions. Other areas found to be involved in translation, irrespective of word frequency, language and switching were the left anterior insula, the left and medial cerebellum and the SMA. For low frequency words only, there was also significant activation in the right cerebellum, and this difference between low and high frequency translation was significant. For switching only, there was activation in the left medial fusiform and more extensive activation of both the left and right ventral cerebella. There were no regions that were specifically (or differentially) activated for forward translation or backward translation, high frequency words or the no switching conditions.

View this table:
Table 2

The effect of translation relative to reading

Anatomical locationCoordinates of significant activationZ-scores
xyzAllHFLFLF – HFSw – No
Coordinates are given in the order x, y, z according to the atlas of Talairach and Tournoux (Talairach and Tournoux, 1988). Z-scores: All = high frequency and low frequency words; HF = high frequency words only; LF = low frequency words only; LF – HF = low frequency minus high frequency words, for translation only; Sw – No = switching minus no switching, for translation only.
(A) Translation—increases in activation (relative to reading)
Switching and no switching (except *switching only)
Anterior cingulate 2 20 2042.44.1
Left putamen/head of caudate–16 18 03.42.73.2
–18 22 163.42.72.5
Right putamen/head of caudate 16 26 24.13.33.3
18 14 44.13.33
18 8 143.52.22.9
SMA 6 12 6242.93.4
Left anterior inferior insula–30 20–1042.93.7
–24 16 –44.73.63.2
Medial cerebellum –6–66–244.73.43.8
8–64–164.43.63.4
Left medial fusiform/left cerebellum–26–38–164.0*3.7*2.6*3.7
–28–42–244.23.43.2
–14–60–284.73.93.6
–12–60–104.24.33.3
–34–66–243.63.82.5
–28–42–443.2*3.3*1.9*3.9*
Right cerebellum 40–88–384.12.14.33.8
38–66–383.51.73.72.7
30–50–383.21.73.22.23.3
(B) Translation—decreases in activation (relative to reading)
Switching and no switching
Medial superior frontal (BA 10) –8 56 204.23.53.5
Posterior cingulate/precuneus (BA 31/7) –2–46 324.13.54.2
Left middle temporal (BA 21)–66–46 64.12.83.4
Left posterior temporoparietal (BA 39)–46–70 184.32.93.7
–52–58 263.83.22.5
No switching only
Right middle temporal (BA 21)–60–54 –63.923.8
Right inferior temporal (BA 20) 54–20–284.13.32.8

Translation versus reading: decreases

Areas that were deactivated during translation relative to reading (Table 2B), irrespective of word frequency, language and switching were regions that have previously been associated with semantic decision tasks—the medial superior frontal gyrus (BA 10), left middle temporal cortices (BA 21), left posterior temporoparietal region (BA 39) and the posterior cingulate/precuneus (BA 31/7). In the no switching conditions (i.e. when the language was all L1 or all L2), but not for alternate L1 and L2, there was also deactivation for translation relative to reading in the right middle (BA 21) and inferior (BA 20) temporal cortices. There were no differential effects of frequency on translation decreases and no significant effect of translation direction.

The minimal effect that word frequency had on differences between translation and reading might appear surprising given the high proportion of errors (30.8%) that subjects made translating low frequency words. As described above, the instructions to our subjects were to say `No' or `Nein' if a translation was not readily available in order to prevent subjects initiating a search for the right word. Indeed, there was no activity, particularly in the prefrontal regions, that could be associated with an executive search specifically during the low frequency condition. The only additional activation for low relative to high frequency words was in the right cerebellum with a corresponding trend in the SMA. This effect may have arisen from the increased `No' or `Nein' responses in the low frequency conditions. More plausibly, frequency affects the ease with which a word can be pronounced. With practice (i.e. for high frequency words), there is a shift from more controlled to more automated and encapsulated processes yielding faster and less variable production times (Segalowitz and Segalowitz, 1993; see also Kirsner et al., 1993). This has been explained in terms of a shift from compiling syllables to retrieving `precompiled articulatory units' (see Hagoort et al., 1999).

Switching between L1 and L2 versus no switching (Table 3)

View this table:
Table 3

Significant activation for switching relative to no switching, for translation and reading

Anatomical locationCoordinates of significant activationZ-scores
xyzAllHFLF
See Table 2 legend for details.
Left inferior frontal (BA 44)–50 6323.433.8
Left supramarginal (BA 40)–40–46403.52.63.4
Right supramarginal (BA 40) 44–42443.22.23.8

Irrespective of word frequency, switching relative to the all L1 and all L2 conditions increased activation in the left inferior frontal region (BA 44, Broca's area) and bilateral supramarginal gyri (BA 40). As recorded in the section on translation increases above, switching during translation also enhanced translation specific activity in bilateral ventral cerebellum and the left medial fusiform. There were no areas with decreased activation for switching irrespective of task and no differential effects of frequency on the switching or no switching conditions. Figure 1 illustrates significant regions of activation for translation and switching.

Fig. 1

An illustration of the activation patterns during translation (relative to reading) and switching relative to no switching. Activations are superimposed on to different slices from the standard brain template from The Montreal Neurological Institute. The numbering on the slices refers to the position relative to the anterior–posterior intercommissural line. The precise position of the activations in co-ordinates and significance levels are given in Tables 2A and 3.

Language input and language output differences

In addition to examining the effects of translation and switching we also contrasted the activation patterns associated with language of input or language of output. By combining data for reading and translation, input and output could be manipulated independently. However, we give less weight to these findings since L2 and L1 words, in general, cannot be matched for phonological or orthographical characteristics. Differences between orthography and phonology may therefore account for differences seen when any two languages are directly compared (e.g. Perani et al., 1996e.g. Perani et al., 1997; et al., 1997; Kim et al., 1997). Differences between frequency and word length were minimized, but the L1 words were slightly longer than the L2 words (L1 words had a mean of 5.2 letters with 1.7 syllables, L2 words had a mean of 4.7 letters and 1.3 syllables). A previous study that manipulated word length and number of syllables (see Price, 1997) showed that increasing the word length from three to five letters, increased activity in the left inferior frontal, temporal and parietal cortices (BA 44, 40 and 20) when the words were low frequency, but no differences were detected when the words were high frequency. If the small difference in the length of L2 and L1 words is relevant then increased activation might be expected in those regions for the slightly longer L1 words. Such an effect was not observed. Therefore, the activation changes as a function of language that are reported below do not appear to relate to effects of word length.

L1 input versus L2 input

Irrespective of frequency, presentation of L1 words, relative to L2 words, increased activation in the left temporal pole, and the left and medial superior frontal cortex (Table 4A).

View this table:
Table 4

Language input and language output differences

Anatomical locationCoordinates of significant activationZ-scores
xyzAllHFLF
See Table 2 legend for details.
(A) German input relative to English input
Left superior frontal (BA 9)–2854 303.42.72.5
Medial superior frontal (BA 9/10) 056 143.42.23.1
Left temporal pole (BA 20)–48–6–283.233.1
(B) English output relative to German output
Right lingual (BA 18) 10–56 83.42.83.7
Left supramarginal gyrus (BA 40)–48–38 403.22.53.6

L1 output versus L2 output

Irrespective of frequency, L2 output relative to L1 output resulted in activation of the right medial extrastriate cortex and the left supramarginal gyrus (Table 4B). As reported above, the left supramarginal gyrus was also more active for switching than for no switching. Activity in this region is maximum for the switching conditions and least for the L1 production conditions.

There were no areas more active for L1 output relative to L2 output, nor were there areas that were more active for L2 input relative to L1 input.

Discussion

Mechanisms of translation

We anticipated on the basis of the IC model that translation (relative to reading) would increase activity in areas associated with the control of action (prediction 1). In particular, we predicted activation of the anterior cingulate, subcortical structures and possibly the dorsolateral prefrontal cortex, especially in the context of switching between languages (prediction 1a). Our study confirmed increases in the first two regions. Specifically, it was the bilateral putamen and head of caudate that were maximally activated during translation. In our experiment, we did not include conditions other than translation and therefore we cannot say whether or not activation in the anterior cingulate and subcortical structures was specific to translation. However, we are able to refer to the study by Klein and colleagues (Klein et al., 1995) which, by virtue of no differences between the anterior cingulate and subcortical activity for translation or word generation, suggests a general control system that is shared by different tasks.

Unlike the strong activation of the dorsolateral prefrontal cortex in the auditory translation task reported by Klein and colleagues (Klein et al., 1995), we detected no increased activation in this region during translation. However, Raichle et al. (Raichle et al., 1994) have found that prefrontal activation decreases with over-learning, whereas opercular/insula activation increases. It may be that our bilingual participants were more proficient than those scanned in the study by Klein and colleagues. Alternatively, the prefrontal activity observed in that study may relate to the implementation of a search strategy that was prevented in our study by instructing the subjects to say `No' or `Nein' if a translation equivalent was not immediately apparent (see Introduction and Results sections).

We also anticipated that translation (relative to reading) would modulate areas associated with semantic processing, particularly for forward translation (predictions 2 and 2a), and areas associated with speech production (prediction 3). Activation in regions associated with semantic processing (left extrasylvian temporal and temporoparietal cortices, BA 20, 21, 39) decreased for translation (relative to reading) irrespective of translation direction or frequency, and there were no significant interactions between these variables. These results are not consistent with the predictions (2 and 2a) from psycholinguistic data which indicate two routes for translating single words: a direct lexical route and an indirect semantic route with the latter more functionally relevant during forward translation (L1 → L2) than during backward translation (L2 → L1) (see Introduction).

One possibility is that our highly proficient bilingual participants were able to translate using the direct route (i.e. without semantic involvement). However, it is pertinent to note that the IC model presumes that selection of a translation equivalent involves inhibition of activated competitors. This is achieved for both routes by inhibiting lemmas with non-target language tags. We consider it more likely therefore that regions associated with semantic processing are initially activated during both forward and backward translation and that this phase is followed by greater or more prolonged deactivation as the system eliminates semantic competitors. If this is the case, the effects of translation direction might only be obtained during the initial activation phase. We require an imaging technique with a higher temporal resolution locked to specific stimulus events to test this prediction.

With respect to the areas associated with the later stages of speech production (prediction 3), we found increased activation for translation (relative to reading) in the SMA, a ventral region in the left anterior insula and the cerebella, with more activation in the right cerebellum for low frequency words. These areas are associated with articulation (see Dronkers, 1996; Price et al., 1996b; Wise et al., 1999). However, unlike Klein and colleagues (Klein et al., 1995) we found no effect of translation in the left posterior inferior temporal lobe. This region is involved in modality independent phonological retrieval (Price et al., 1996a; Price and Friston, 1997; Price, 1998), and phonological retrieval is required for translating in both modalities. In this case we believe the study differences may relate to the respective baseline tasks used. The left posterior inferior temporal cortex may be more involved during reading (our baseline) than during repetition (the baseline used by Klein and colleagues) because, during the latter, the phonological output is already specified by the phonological input. Hence, the differential demands placed on phonological retrieval will be greater relative to auditory repetition (Klein et al., 1995) than when the baseline is reading (our study). Further experiments are required to examine the validity of this hypothesis and to determine whether the inconsistency between our study and that by Klein and colleagues (Klein et al., 1995) relates to proficiency or to other factors such as the modality of input (visual versus auditory).

Overall, our data confirm that translation modulates regions specifically associated with semantics and articulation. They are consistent with the following proposal: during translation the demands placed on articulatory output increase because the response associated with the input orthography must be inhibited while the response associated with the translation equivalent is activated. Such control of articulation during translation appears to be governed by activity in the anterior cingulate and subcortical structures, which are associated with a general control system as proposed by the IC model.

Mechanisms of switching

In our tasks, language switching was both predictable and cued by the input (L1 or L2 words) and we failed to detect any increased activation or deactivation in a region associated with executive control during translation (i.e. the dorsolateral prefrontal cortex), which might have been expected given that translation schemas must be repeatedly called and suppressed (prediction 1a). However, the finding that switching activates a dorsal region of the left posterior inferior frontal cortex and bilateral supramarginal gyri does indicate that switching modulates word processing at a phonological stage (prediction 3). For example, this region of the dorsal left posterior inferior frontal cortex has been associated with phonemic segmentation because it was more active for phonemic detection than for pitch detection—the latter task controls for acoustic input, demands on memory, attention, decision making and response execution (Zatorre et al., 1996). The supramarginal gyri (as noted in the Introduction) have previously been implicated in mapping orthography into phonology (see Price, 1998). Therefore, differential activation during language switching supports the expectation that the demands placed on orthographic to phonological mapping increase as participants alternate between L1 and L2, which demand different mappings.

Taken together, these results indicate that switching and translation modulate different components of the language system (phonological recoding in the case of the former and semantics and articulation in the case of the latter), consistent with prediction 3. However, it is also possible that the apparent independence of the mechanisms involved in language switching and in translation partly reflects the nature of our tasks. In other circumstances we might expect to see anterior cingulate and subcortical activation during switching, for example, when orthography cannot be relied on to trigger the relevant production schema, as when individuals are required to read aloud inter-lingual homographs (such as `taste' meaning musical key in German). We might also see activation in the dorsolateral prefrontal region when switching is unpredictable. In general, the involvement of these systems will depend on the language task: naming pictures in alternate languages, for instance, even with regular switching, might be expected to show dorsolateral frontal activation.

Differences between languages

Irrespective of word frequency, the perception of L1 (relative to L2) was associated with increased activation in the left temporal pole and the left and medial superior frontal cortex (BA 9/10). Increased activation in the left temporal pole for L1 is consistent with data from Perani and colleagues (Perani et al., 1996) who contrasted activity evoked on hearing stories in L1 with hearing stories in L2. This region has been associated with the semantic priming of content words (Nobre and McCarthy, 1995), the comprehension of words and stories (Mazoyer et al., 1993) and the recognition of objects (Price et al., 1996a) and familiar faces (Gorno-Tempini et al., 1998). Together, these previous studies suggest that the activity we detect relates to increased comprehension during the perception of L1 relative to the perception of L2. Plausibly, activity detected in the left/medial superior frontal cortex may also relate to semantic processing because these areas are part of a distributed language system differentially involved in semantic decisions relative to visual decisions (Vandenberghe et al., 1996). However, semantic decisions involve a number of different processes in addition to the activation of semantic associations (Price et al., 1997). The distributed semantic system identified in the study by Vandenberghe and colleagues (Vandenberghe et al., 1996) involved various temporal, parietal and frontal regions that could be executing several different, as yet undefined, functions. Other results suggest that the degree to which semantic processing regions are engaged by L1 more than by L2 depends on the proficiency of the bilingual speaker (Perani et al., 1998). In the study by Perani and colleagues (Perani et al., 1998), the participants were proficient in their second language and only a part of the distributed semantic system was more engaged during the perception of L1. Less proficient bilinguals are likely to show wider differences between L1 and L2 (Dehaene et al., 1997).

Whilst there was increased activity for the perception of words in L1 relative to L2, there were no areas where there was enhanced activation for the perception of L2 relative to L1, or for the production of L1 relative to L2. Nor did we replicate the findings of Klein and colleagues (Klein et al., 1995) who found increased left putamen activation for speaking in L2. The inter-study difference in this latter result may reflect the modality of the task (visual versus auditory input) or the proficiency of the participants. However, it was not the case that our study was insensitive to differences in the language of response because speaking L2 resulted in increased activation in right medial extrastriate cortex and the left supramarginal gyrus. Activity in the left supramarginal gyrus was also increased for switching relative to no switching irrespective of task (see above), resulting in maximum activity for the switching conditions and least activity for the L1 output conditions. This pattern may relate to the relative demands placed on phonological recoding which is more difficult when participants are required to speak in their second, less familiar language and even more demanding when they have to switch repeatedly from one language to another (see above). We cannot, however, dismiss the possibility that differences in the left supramarginal gyrus arose because of differences in phonological structure that could not be controlled across languages in this experiment. Another possibility is that, in this study, L2 was English which has a less consistent relationship between orthography to phonology than German, leading to increases in left supramarginal activation.

Conceivably, individuals who acquire both their languages early are better able to programme phonological responses and are less likely to have a foreign accent. Such differential proficiency may explain the results of Kim and colleagues (Kim et al., 1997) who found that when participants acquired L2 later in life, different areas of left Broca's area were activated when generating sentences in L2 and in L1. The present study demonstrates that the left supramarginal gyrus, which was not investigated in the study by Kim and colleagues, also shows differential activation for L2 relative to L1 production. The other factor that needs to be considered is the methodology. Dehaene and colleagues (Dehaene et al., 1997), using fMRI, have shown that there are wide individual differences in the effect of language that would be missed when activation is averaged over subjects as in this study. Further studies are clearly required to resolve these issues.

Summary and conclusions

On the basis of the IC model (Green, 1998a), we predicted (prediction 1) that translation would engage specific neuroanatomical sites mediating the control of action. Consistent with this prediction, translation increased activation of the anterior cingulate and subcortical structures. However, there was no increase in dorsolateral frontal activity for translation under conditions of language switching (prediction 1a). Further studies are needed to investigate whether dorsolateral frontal activity arises during translation when orthography cannot be relied on to trigger the relevant translation schema (as in present study) or when language switching is unpredictable (as opposed to predictable as in present study).

We also predicted on the basis of prior behavioural research (prediction 2) that forward translation would entail more semantic processing than backward translation (particularly for high frequency words, prediction 2a). Contrary to this prediction, activation decreased in the left extrasylvian and temporoparietal cortices regardless of translation direction, and there were no effects of frequency. Such decreases are consistent with the role of inhibitory processes in selecting among competitors for production. The IC model predicts that an activation phase precedes an inhibitory phase, but a more refined functional imaging technique is needed to establish this possibility. Whether or not the activation phase will show differences as a function of translation direction is perhaps moot given the minimal differences in the language processing of highly proficient bilinguals observed in the study by Perani and colleagues (Perani et al., 1998).

We also predicted (prediction 3) that translation and switching would differentially modulate activity in the word recognition and word production systems. As expected, switching modulated activity in regions associated with mapping orthography to phonology (e.g. the supramarginal gyri). In contrast, translation modulated activity in regions associated with articulation (i.e. the SMA, the cerebellum and a ventral region in the left anterior insula). We did not predict the precise nature of these effects. However, the novel finding of the partial independence of the mechanisms mediating translation and switching merits further exploration. Behaviourally, we need to understand how lexical and sub-lexical processes affect translation time and precisely how the stimulus word is represented before being blocked from production.

Our findings are relevant to the interpretation of particular disorders that can arise in bilingualism. For example, the role of both cortical and subcortical structures in the control and performance of language tasks in bilinguals suggests that at least certain types of bilingual aphasia may reflect deficits in controlling relatively intact lexico-semantic systems. Consider, for instance, the recovery pattern of the French–Arabic speaker A.D. [one of two similar cases reported in a study by Paradis and colleagues (Paradis et al., 1982)]. On day 18 after a moped accident, A.D. could speak Arabic spontaneously but could not translate into it. In contrast, she could translate into French even though her spontaneous use of French was poor. The following day she showed the converse pattern: she could speak French but could not translate into it, whereas she could translate into Arabic but could not speak it spontaneously. Comprehension in French and Arabic was good throughout this period. Such a pattern of recovery is not readily explained in terms of damage to the lexico-semantic system itself but is consistent with a difficulty in controlling the outputs of such a system. Indeed, the selective recovery of just one language may reflect a control problem too since such an outcome may reflect a difficulty in overcoming the inhibition of a functional subsystem (see also Paradis, 1997). Functional imaging studies will have an important role to play in reaching an appropriate assessment of bilingual aphasics and in guiding their effective treatment.

Further studies are required to address questions raised by this study. For example (i) why is translation accompanied by increased dorsolateral frontal activity in the auditory translation task of Klein and colleagues (Klein et al., 1995) but not in our visual translation task? (ii) How important are deficits in control processes in bilingual aphasics? (iii) What is the nature of the deactivation in the semantic system during translation: does presentation of a word for translation elicit momentary activation in the semantic system followed by deactivation?

Acknowledgments

We wish to thank the Wellcome Trust for supporting the study; Professor Robert Turner for facilitating it; and the radiographers of the Functional Imaging Laboratory for their help in the process of data acquisition. We also with to thank Billi Randall for providing us with the CELEX frequency data for English and Stevie Sackin for providing us with the CELEX frequency data for German. We are also grateful to Professor Richard Frackowiak, Professor Tim Shallice and the reviewers for helpful comments on previous drafts.

References

View Abstract