OUP user menu

Visual activity predicts auditory recovery from deafness after adult cochlear implantation

Kuzma Strelnikov , Julien Rouger , Jean-François Demonet , Sebastien Lagleyre , Bernard Fraysse , Olivier Deguine , Pascal Barone
DOI: http://dx.doi.org/10.1093/brain/awt274 3682-3695 First published online: 17 October 2013


Modern cochlear implantation technologies allow deaf patients to understand auditory speech; however, the implants deliver only a coarse auditory input and patients must use long-term adaptive processes to achieve coherent percepts. In adults with post-lingual deafness, the high progress of speech recovery is observed during the first year after cochlear implantation, but there is a large range of variability in the level of cochlear implant outcomes and the temporal evolution of recovery. It has been proposed that when profoundly deaf subjects receive a cochlear implant, the visual cross-modal reorganization of the brain is deleterious for auditory speech recovery. We tested this hypothesis in post-lingually deaf adults by analysing whether brain activity shortly after implantation correlated with the level of auditory recovery 6 months later. Based on brain activity induced by a speech-processing task, we found strong positive correlations in areas outside the auditory cortex. The highest positive correlations were found in the occipital cortex involved in visual processing, as well as in the posterior-temporal cortex known for audio-visual integration. The other area, which positively correlated with auditory speech recovery, was localized in the left inferior frontal area known for speech processing. Our results demonstrate that the visual modality’s functional level is related to the proficiency level of auditory recovery. Based on the positive correlation of visual activity with auditory speech recovery, we suggest that visual modality may facilitate the perception of the word’s auditory counterpart in communicative situations. The link demonstrated between visual activity and auditory speech perception indicates that visuoauditory synergy is crucial for cross-modal plasticity and fostering speech-comprehension recovery in adult cochlear-implanted deaf patients.

  • plasticity
  • cochlear implant
  • audio-visual
  • sensory loss
  • prediction


In the 17th century, Molyneux questioned whether a man born blind would, after having his sight restored, be able to visually recognize shapes that he would have been able to discriminate previously by touch. Since then, there has been a long debate regarding the capacity of the human brain to process sensory information after a long period of deprivation (Rauschecker, 1995). The response to Molyneux’s question is essential, as it concerns our understanding of the brain mechanisms for cross-modal plasticity and intermodal interactions that can be explored in blind or deaf patients. Only recent data regarding curable congenital cataracts have brought about some insights into Molyneux’s problem as it relates to blindness (Held et al., 2011). Indeed, these data revealed that sensory deprivation since birth altered crossmodal processing but this ability can be restored after the short sensory experience. Further hints to the answer of Molyneux’s problem can now be obtained by the evaluation of profoundly deaf patients recovering hearing functions through a cochlear implant.

In both humans and animals, the loss of one sensory modality induces compensatory mechanisms expressed as an increase in performances of the spared modalities (Rauschecker, 1995; Roder et al., 2004; Bavelier et al., 2006). As with that observed in congenital deaf cats (Lomber et al., 2010), early deafened humans show some visual abilities that exceed those reported in normal hearing subjects, mainly expressed as enhanced ‘reactivity’ to visual events (Pavani and Bottari, 2012) and enhanced spatial attention (Neville and Lawson, 1987; Dye and Bavelier, 2010). Cross-modal perceptual compensation is accompanied by some functional reorganization, including the colonization of the deprived areas by other sensory modalities (Bavelier and Neville, 2002; Collignon et al., 2011). The extent of cross-modal reorganization is highly dependent on the age at which the sensory deprivation occurs. Between blind subjects with early visual loss and late visual loss, there are striking differences in the hierarchical visual stages recruited to process auditory or somatosensory information as well as differences in the function of the reorganized visual system (Sadato et al., 1996; Buchel et al., 1998; Burton et al., 2002; Bedny et al., 2012). Likewise, after complete deafness, animal studies have demonstrated the colonization of the auditory system by visual functions in both congenital and adult deafness (Allman et al., 2009; Lomber et al., 2010; Barone et al., 2013). Though in deaf human subjects the auditory cortex can be activated by speech-related visual inputs (Nishimura et al., 1999; Petitto et al., 2000; Capek et al., 2008), there is still a debate on what extent the auditory system is affected by cross-modal plasticity with respect to the age of the onset of deafness (Lee et al., 2003; Lambertz et al., 2005; Karns et al., 2012). The aim of the present study was to evaluate how cortical plasticity and cross-modal reorganization impact the recovery of speech comprehension in late deaf cochlear-implanted adult patients.

It is now well established that the success of cochlear implants for speech perception depends on the age at which cochlear implant is performed as well as the duration of deafness (Kral and O'Donoghue, 2010). In the case of early deafness, there is now compelling evidence that the critical period, occurring around the age of 2 years (Kral and Sharma, 2012), reflects the potential of brain plasticity, which is critical for the recovery of auditory functions through the neuro-prosthesis. Further, the acquisition of language depends on the period during which language experience occurs independently of the modality of such experience (including the visual modality; Mayberry et al., 2002) and thus visual input could be helpful to form the semantic modality-independent network for language processing in the vicinity of the auditory regions (Leonard et al., 2012). However, after late cochlear implantation, cochlear implant outcomes rely on cross-modal cortical reorganization (Lee et al., 2001) and after prelingual deafness the colonization of the auditory cortex by visual processing (Petitto et al., 2000) is associated with low auditory speech recovery (Lee et al., 2007).

In post-lingually deaf adult cochlear implant users, the high progress of speech recovery is observed during the first year after cochlear implantation, but there is a large range of variability in the level of cochlear implant outcomes and the temporal evolution of recovery (Rouger et al., 2007). Once the obvious pathological grounds are discarded (cochlear malformation, ossification, auditory neuropathy, etc.), the causes of such variability are multiple, including severity, duration and the age of onset of the hearing loss (Lazard et al., 2012b; Blamey et al., 2013). In addition to these multiple origins, we assume that the success of rehabilitation with the implant relies also on the functional plasticity of the brain as proposed for prelingual deafness. Post-lingually deaf adult patients develop distinctive adaptive strategies to compensate both for deafness and for the impoverished information delivered by the implant (Lazard et al., 2012a). During the period of deafness, subjects maintain oral comprehension by developing lip reading. After implantation, the use of lip reading does not decrease, because the crude information transmitted by the cochlear implant requires the persistent use of visual cues, especially in noisy situations (Rouger et al., 2007). Because of the complementary nature of the visual and the auditory information, during progressive hearing recovery, the capacities of the patients for speech intelligibility rely principally on the visual and visuo-auditory processing of speech (Rouger et al., 2007; Barone and Deguine, 2011). Such visuo-auditory synergy for speech perception is expressed at the brain level by specific cortical cross-modal reorganizations that involve both visual (Giraud et al., 2001) and auditory cortical areas (Strelnikov et al., 2010; Rouger et al., 2012). In the latter case, there is a gradual regression of the cross-modal reorganization of the auditory areas in parallel to the progressive auditory recovery made possible by the cochlear implant (Rouger et al., 2012). Based on evidence that the auditory cortex can process non-auditory information in adult deaf patients, the main question is to what extent such cortical plasticity can interfere with auditory recovery after cochlear implantation.

Until now, predicting factors were assessed only in prelingually deaf cochlear-implanted children and by using brain metabolic levels at rest before implantation (Lee et al., 2007). Such strategies for evaluating brain activity before implantation are limited, because they do not provide information about the ability of the auditory cortex to respond to the reactivation of sensory inputs through the cochlear implant. Previous brain imaging studies have observed, in patients with cochlear implants with a weak speech comprehension recovery, a low level of activation of the auditory areas in response to speech sounds or voice stimuli (Green et al., 2005; Coez et al., 2008). These results suggested that the sensitivity of the auditory areas to speech sounds could help evaluate recovery after cochlear implantation.

Here, we have developed a predictive study in adults with post-lingual deafness, based on brain activity measured during an active speech-perception task shortly after the time of activation of the implant. The promising benefit of this procedure in comparison with a pre-implantation evaluation is that it allows us to develop an analysis based on the initial potential of the brain to respond to the electrical stimulation by the implant. We hypothesized that speech comprehension in post-lingually deaf cochlear implant users relies strongly on visual and visuo-auditory synergy (Barone and Deguine, 2011) and consequently we explored both modalities (visual and visuo-auditory) during brain imaging sessions. To investigate whether activity in specific cortical regions would account for the recovery of auditory speech comprehension, we analysed the relationship between individual brain activity at the time of implantation and performance in auditory word perception several months after cochlear implantation.

Materials and methods


We included 10 cochlear-implanted adult patients with post-lingual deafness in this Embedded Image neuroimaging study. Participants were native French speakers with self-reported normal or corrected-to-normal vision and without any neurological, language or cognitive disorders. All cochlear implanted adult patients had post-lingually acquired profound bilateral deafness (defined as a bilateral hearing loss >90 dB). In the majority of the patients, the aetiologies and durations of deafness are unknown (Table 1). The clinical implantation criteria included word and open-set sentence auditory-recognition scores <30% under best-aided conditions (i.e. with conventional acoustic hearing aids). Cochlear-implanted patients were recipients of a unilateral cochlear implant, five on the left side and five on the right side. Cochlear implants were completely inserted in the scala tympani, allowing an optimal activation of stimulating electrodes. Conventional acoustic hearing aids were not used either during the PET scan or during the speech-perception scores acquisition. In 8 out of 10 patients, contralateral residual hearing was either absent or only weak, with thresholds >70 dB SPL (sound pressure level) for the low-frequency ranges. Only two patients presented a threshold of 40–55 dB SPL for the frequency range of 125–500 Hz. The group of patients included seven females and three males aged between 35 and 81 years (mean 53.9 years).

View this table:
Table 1

Characteristics of patients in the study

Patient IDAge (years)SexDeafness (years)AetiologyImplantProcessorSideT0 (days)Visual PETAudio- visual PETPre-opPost-op
CI0281Female>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GR251592065
CI0339Male>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GR2269843065
CI0439Female>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GR888905085
CI0657Female>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GL261752015
CI0769Male>5Chronic otitisMedel SonataTempo plusL564842585
CI0839Male>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GL97578050
CI0962Female>5UnknownAdvanced Bionics Hi Res 90KAuriaL368945590
CI1064Female>10Idiopathic sudden SNHLAdvanced Bionics Hi Res 90KAuriaL95374060
CI1154Female>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GR1558804545
CI1235Female>20UnknownNucleus CI24 Contour AdvancedEsprit 3 GR170881060
  • Deafness = deafness duration; T0 = the time of the first PET examination, from implant activation, in days; Pre-op = the auditory recognition score before the operation %; Post-op = the highest score of word recognition 4–7 months post-implantation %.

Six normal-hearing subjects with no neurological disorder and with normal or corrected-to-normal vision were included as control subjects to verify the specificity of the results to the mechanisms of recovery from profound deafness (Rouger et al., 2012).

All participants gave fully informed consent before their inclusion in this study, in accordance with the standards of local ethics committees (CPPRB Toulouse I, n° 1-04-47, Toulouse, France).


During the PET session, we used French disyllabic words (e.g. /sitrõ/, English ‘lemon’) and meaningless temporally-reversed dissyllabic words (non-words). Words and non-words were pooled into lists of 40 stimuli each, including 20 words and 20 non-words in random order. These lists were equalized for syllabic structure (CV/CVC/CCV), language use frequency (Brulex), and anterior–posterior phonemic constitution. All stimuli were uttered by a female French speech therapist who was given the instruction to pronounce each word using a normal pronunciation with an even intonation, tempo and vocal intensity. Utterances were recorded in a soundproof booth with a professional digital video camera. The video recording showed the speaker facing the camera, with her entire head against a uniform yellow background. The video was digitized at 25 frames per second with a 720 × 576 graphic resolution. Visual stimuli were extracted using Adobe Premiere Pro 7.0 (Adobe Systems), including a short rest-time (∼200 ms) before and after each word. All stimuli were finally exported in MPEG-2 video format with maximum encoding quality.

Positron emission tomography

PET with H215O used in this study reflects the local cerebral blood flow. H215O has a 2-min half-life; thus, pauses of 15 min were used between the scans. Cochlear-implanted deaf patients were scanned in a shielded darkened room with their head immobilized and transaxially aligned along the orbito-meatal line with a laser beam, with the position controlled before each acquisition. Measurements of the regional distribution of radioactivity were performed with an ECAT HR+ (Siemens®) PET camera with full volume acquisition (63 planes, thickness 2.4 mm, axial field-of-view 158 mm, in-plane resolution ≈4.2 mm). The duration of each scan was 80 s; ∼6 mCi of H215O was administered to each subject for each individual scan. There were two runs per each condition (see below). Three images were acquired during each run and eight runs resulted in 24 images per subject. The rest images from one patient were excluded due to a technical issue.

Stimulation according to the experimental conditions began ≈20 s before data acquisition and continued until scan completion. Instructions for the experiment were given to subjects before each tomography and repeated before each run.

Conditions of positron emission tomography sessions

The PET examination was performed as early as possible after the implant activation, which occurred ∼1 month after the surgical cochlear implantation. The delay between the time of activation of the implant and the brain imaging session ranges between 1 and 22 days (mean 8 days, Table 1).

During each individual session, we used three different stimulation conditions that corresponded to a resting state level and a visual or audio-visual speech discrimination task. During the ‘rest’ condition, subjects lay in the chamber, eyes closed, without any auditory stimulation. In the ‘visual speech’ condition the task consisted of a word discrimination task performed through speech reading. Words and non-words uttered by a female speaker were presented in the visual modality without any auditory stimulation. The ‘audio-visual speech’ condition consisted of presenting videos with sound of the same female speaker pronouncing words. In the visual and audio-visual speech conditions, subjects had to distinguish words from non-words through a yes/no two-alternative forced choice task using a two-button computer mouse with their right hand. Presentation rate was 5 s per word, including visual presentation of the word (∼1000 ms per word including pre- and post-rest times) before a black screen with a white fixation cross in its centre.

Speech comprehension

A speech-therapist evaluated all the patients by means of free-field vocal audiometry, using French disyllabic words, in order to test their auditory performance. The word recognition scores were collected during the visits of the patients to the speech therapist before the implantation and monthly after the activation of the cochlear implant. All subjects were tested on open-set recognition for French disyllabic words, obtained from the commonly-used French speech therapist list developed by Fournier (Rouger et al., 2007). Only words that the subjects correctly repeated verbally were considered correct responses (% correct score). At each period of the evaluation of speech comprehension, we used a new set of items to avoid memory-biased responses of the patients. Our strategy was to evaluate the performance of the patient after a similar and relatively restricted period of experience of the implant at the end of the first semester (6 months). In a longitudinal study (Rouger et al., 2007), we observed that at 6 months post-implantation, ∼20% of the patients presented low performance in speech comprehension (word recognition <50%), whereas this performance level was present in only 8% of the patients after 1 year of cochlear implant experience. In contrast, at 6 months, >40% of them reached an optimal performance level (word recognition >75%). Indeed, by collecting data at ∼6 months, we were able to assess not only the potential of recovery of the patients but also the rate of temporal evolution, as both parameters vary across patients. Further, the variability in performance is needed to maximize the power of the correlation analysis and avoid some ceiling effects that would occur from collecting the data later in the period of auditory recovery. Indeed, we used the word comprehension score, which was stable across repeated visits at the end of the first semester post-implantation.

Data analysis

Neuroimaging data were analysed with SPM5, including the standard procedures of image preprocessing (realignment, spatial normalization, smoothing with 8 mm Gaussian kernel), defining the models and their statistical assessment. To search for a causal relationship between brain activity and auditory recovery we conducted two complementary analyses on non-contrasted activity maps. Firstly, we performed a regression analysis to detect brain areas whose activity level correlated with the word recognition score that had been collected by the speech therapist and obtained 6 months after the PET scan session. The ensuing images were estimated in the whole brain analysis using voxel-level t- and z-values, which corresponded to P < 0.05 with a family-wise error correction for multiple non-independent comparisons. We systematically applied a cluster extent threshold of 10 voxels. Next, an individual analysis was performed for each subject. The global activity covariate was used as a nuisance effect in the general linear model (analysis of covariance) corresponding to the one-sample t-test, without the scores. In each patient, a sphere with 4 mm radius (corresponding to the used smoothing of 8 mm) was placed at the peak issued from the whole brain group analysis (see above). The mean activity was calculated in this sphere in individual patients. Lastly, the mean relative difference between the value from the sphere and the global activity value was calculated per subject and correlated with the scores obtained at 6 months.

It should be noted that at the group level, we took one voxel at the peak of activity, but in individual measures, we took a sphere, which included 32 voxels, and calculated a mean within this sphere. The inclusion of the large number of supplementary data from other voxels in individual analysis solves the problem of the possible dependencies.


Behavioural performances

At the time of the cochlear implant activation, the patients had a profound hearing loss with a mean auditory speech comprehension score of 28 ± 14% [standard deviation (SD)] as assessed by a speech therapist. Further, after a progressive recovery, the patients reached a stable performance in auditory word comprehension of 62 ± 16% at the end of the first semester (Table 1). This limited set of patients presented improved auditory scores at ∼6 months (15–90%) with respect to the pre-implantation scores (0–50%), which is representative of that obtained in a larger population of patients with cochlear implants (Rouger et al., 2007).

The overall performance on the lexical word/non-word discrimination task during PET scanning sessions was 68.9 ± 9.7% for the visual conditions and 80.4 ± 12% during the audio-visual task. We applied Signal Detection Theory to separate perceptual (d’) and decision-level effects on the responses (Green and Swets, 1966). Using word stimuli as the targets, we computed the index of discriminability d’ (higher d' means that the signal can be more readily detected in individual patients). The values in patients with cochlear implants were significantly higher for audio-visual conditions than for visual conditions (1.73 ± 0.6 versus 0.95 ± 0.7, Mann and Whitney U-test, P < 0.01) suggesting that the patients were actively engaged in the word discrimination task.

Brain activity level and auditory speech comprehension recovery

First, we searched for the dependence between the recovery level of those patients evaluated after 6 months of experience with the implant and the brain activity recorded immediately post-implantation. We observed only four cortical regions that present such characteristics: the right occipital, the left inferior frontal, the right posterior temporal cortices, and the right middle superior temporal gyrus and sulcus (STG/STS) (Table 2).

View this table:
Table 2

Brain areas with significant activity in the regression analysis with auditory scores at 6 months (group analysis)

Brain regionP-value (corrected)n voxelsz-valuexyz
Rest condition
Right occipital<0.001666.7430−1002
Left inferior frontal<0.001395.83−30424
Left post temporal<0.001295.72−56−38−10
Right middle STG<0.001**2626.762−16−6
Visual condition
Left inferior frontal<0.001787.18−32422
Right occipital0.001255.8330−1004
Right middle STG<0.001**2656.7862−14−6
Audio-visual condition
Left inferior frontal<0.0011046.99−32420
Right occipital<0.001305.5130−1004
Left post temporal0.002146.12−56−36−8
Right middle STG<0.001**1216.6560−16−6
  • **Indicates negative correlations.

First of all, the whole brain regression analysis revealed that the activity in the right occipital visual cortex immediately post-implantation was significantly related to the auditory scores at ∼6 months (Fig. 1A and Table 2). This positive significant correlation was obtained in the three brain imaging conditions: resting state, visual speech reading and audio-visual word discrimination. As we had an a priori hypothesis about occipital activity, we applied the correlation analysis on an individual basis with spheres at the peaks of the most significant clusters for each condition. This analysis resulted in significant and high correlation values. Indeed, the correlation level (r-values) between the activity level in the right occipital cortex and the auditory scores were 0.9 for the resting condition, 0.8 for the visual speech reading, and 0.5 for the audiovisual condition (Fig. 1B).

Figure 1

The visual cortex activity in the regression analysis with auditory scores at the end of the first semester post-implantation. (A) The activity is presented at the threshold level of P < 0.05 with family-wise error correction for the rest condition. (B) Spatial overlap is presented with respect to resting activity. The activity in the visual condition is shown in green, and the activity in the audio-visual condition is shown in blue. The activities are presented at the threshold level of P < 0.05 with a family-wise error correction.

In all cases, the correlated clusters in the occipital region corresponded mainly to the extrastriate visual cortex [Brodmann area (BA) 18] but they partly involved the primary visual cortex. According to the probabilistic cytoarchitectonic maps (Eickhoff et al., 2006), in the resting condition, the occipital cluster has a 40% probability of being located in BA17 and an 80% probability of being located in BA18. Similarly, in the visual condition the cluster has a 50% probability of being located in BA17 and a 90% probability of being located in BA18. Lastly, in the audio-visual condition, the probability that the occipital cluster will be located in BA17 is 50% and there is a 90% probability it will be located in BA18. Considering the Euclidean distances between the peaks, we did not find a difference between the visual and audio-visual conditions. Both peaks in the visual and audio-visual conditions were located 2 mm apart from the peak at rest, which fell within the smoothing precision of 8 mm.

By looking at the overlapping surface and the Euclidian distances between peaks of activity we showed that the spatial extent of the right occipital cluster in the visual condition had a 76% overlap with that obtained in the at rest condition, and at 53% with that obtained in the audio-visual condition (Fig. 1B and Table 3). The overlap between the right occipital clusters during the visual and audio-visual stimulations was 47%. All together, the anatomical localization studies suggest that the three occipital clusters, which are correlated with the values of auditory recovery, represent the same visual cortical region that includes mainly BA18.

View this table:
Table 3

Overlap between clusters and the euclidian distances between the peaks in different conditions for positive and negative regressions in group analysis

Visual versus RestAudio-visual versus RestVisual versus Audio-visual
ClustersOverlap(%)Distance (mm)Overlap (%)Distance (mm)Overlap (%)Distance (mm)
Right occipitalc422432533
  • aOverlap between the peaks for positive regressions.

  • bOverlap between the peaks for the negative regression in the right STG/STS.

  • cOverlap between clusters in the right occipital cortex detected in the negative regression with activity in the right STG/STS.

The second brain region also associated with the auditory recovery of the cochlear implanted deaf patients was located in the frontal pole (Fig. 2 and Table 2). The activity level after deafness observed in the left inferior frontal pole presented a significant positive correlation with the auditory score of speech comprehension at 6 months post-implantation. This correlation was observed at the whole brain analysis level in all the three conditions (rest, visual and audio-visual), and at the single subject level, high correlation values of 0.8, 0.9 and 0.9 were observed for the rest, visual and audio-visual conditions, respectively. The cluster was located mainly in the deep part of the inferior frontal sulcus, corresponding to BA45–46, including a portion of Broca’s area. Furthermore, the clusters revealed by the correlation analysis in the three conditions overlapped to a large extent (Table 3) suggesting that the same local frontal territory was revealed by the correlation analyses. The peak to peak distances between the rest and visual condition were 3 mm, 5 mm when comparing rest and audio-visual clusters and 2 mm only when calculated between the visual and audio-visual clusters.

Figure 2

The frontal cortex activity as shown in the regression analysis with auditory scores at the end of the first semester post-implantation. The activity is presented at the threshold level of P < 0.05 with a family-wise error correction for the audio-visual condition. The correlations of the left frontal activity early post-implantation with auditory scores at the end of the first semester (T1) for the audio-visual, visual and rest conditions are illustrated.

The correlation analysis revealed a third cluster located in the left posterior temporal region (Table 2). This positive correlation was observed only for the rest and the visual speech reading conditions. These two clusters were overlapping at 50% and presented a peak-to-peak distance of 3 mm (Table 3).

In a second analysis, we searched for cortical areas that would present an activity level negatively correlated with the auditory speech comprehension performances several months after cochlear implantation. Following the whole brain analysis, a single cluster emerged from the correlation analysis in all conditions (rest, visual and audio-visual), and it was located in the right middle STG/STS (Fig. 3 and Table 2). The correlation value during the rest condition was r = −0.9, −0.8 during the visual condition, and −0.7 for the audio-visual condition. A comparison of the locations of the three correlated clusters suggested that they represent the same cortical region, as shown by the degree of overlap and by the small peak to peak distances between them (Table 3).

Figure 3

Activity in the right middle STG/STS negatively related to the auditory recovery scores.

Cortical network analysis

The correlation analysis revealed three regions showing a positive correlation with the auditory recovery and only one presented a negative correlation. These results might suggest the existence of an interconnected cortical network, in which these areas act in opposite directions and impact the cortical plasticity involved in the auditory recovery differently. To test this hypothesis, for each subject we calculated the mean of the activity in the 4 mm radius sphere centred on the coordinates of the middle STG/STS activity. Then we performed a negative regression analysis to reveal the cortical areas that could be inversely related to the activity in the middle STG/STS. Indeed, for each condition (rest, visual, audio-visual), this analysis revealed significant activity in a single region located in the right visual occipital cortex (Fig. 4 and Table 4). The spatial overlap of the correlated clusters between the conditions ranged between 42 and 53% with no significant differences in the peak-to-peak distances (Table 3). The cortical location of this occipital region, inversely correlated to the right middle STG/STS activity, was close to the one observed following the original correlation analysis (cf. Figs 1 and 4) whereas the occipital peak anti-correlated with middle STG/STS was ∼10 mm more medial. However, we observed a large overlap between these occipital activities, with values ranging from 62% (rest) to 84% (visual) and 83% (audio-visual), which suggests that the same visual region is involved.

Figure 4

Activity in the right occipital cortex is negatively correlated with activity in the right middle STG/STS.

View this table:
Table 4

Activity in the right occipital cortex negatively correlated with activity in the right STG/STS in group analysis

Right occipital cortex
ConditionP-value (corrected)n voxelsz-valuexyz

Supplementary correlation analysis

We have collected the levels of the patients’ auditory recovery at ∼6 months after implantation in order to obtain a large range of performance values and to increase the statistical power of the correlation analysis. However, as shown in Table 1, one of the patients had a poor recovery in the word recognition score (15%). We checked whether this patient influenced the observed correlations by excluding him from the analysis. For the occipital activity, a significant correlation remained in the rest condition (r = 0.7, P < 0.05). For the frontal activity, the significant correlation persisted in all three conditions (rest: r = 0.8; visual: r = 0.7; audio-visual: r = 0.8, P < 0.05). Lastly, concerning the negative relationships, the rest condition activity in the right middle STG/STS survived the exclusion of this patient (r = −0.8, P < 0.05).

Further, we investigated whether the auditory scores of the patients collected at the time of implant activation are related to the brain activity obtained during the PET session. This analysis provided negative results, no correlation with brain activity was detected for the three conditions (rest, visual, and audio-visual, P > 0.1) in the reported regions. First, this brings further support to our strategy of considering the performances at 6 months after implantation. Second, this could also suggest that the observed correlation analysis is sensitive to the adaptive strategy developed by the patients with cochlear implants during the first months after cochlear implantation.

In a final series of analyses, we searched for any influence of several parameters on the brain activity-based correlation analysis. First, the levels of brain activity in the reported regions did not correlate with the age of the patients (P > 0.13). Second, concerning the duration of hearing loss, the deafness was progressive in all but one patient, and the duration of hearing loss was > 20 years in 7 of 10 patients. Because of this progressive hearing impairment, the duration of deafness could not be reliably defined and consequently we did not attempt to correlate this measure with any of the brain activity patterns.

The influence of deafness aetiology could not be estimated, because most of the patients have unknown aetiologies. We compared the sides of implantation, and no effect on brain activity was found (P > 0.3, Mann-Whitney test). The effect is also absent for the speech perception scores at 6 months (P > 0.9, Mann-Whitney test).

As few patients presented some residual hearing, we asked whether the pre-implantation performances in speech comprehension could be related to the activity level in the described regions using the best hearing threshold (dB) for each patient. We did not observe a significant correlation following this analysis (P > 0.5), suggesting that the residual hearing does not influence the activity level in the occipital, frontal, or middle STG/STS regions.

Further, we have been able to collect the performance of the same patients in speech comprehension several years later (range 24–30 months post-cochlear implant). We observed that the performances at 6 months are correlated to the later ones, meaning that they are representative of the long-term auditory recovery of the patients (r = 0.7, P < 0.05). However, due to the ceiling effects in some patients in our cohort several years post-implantation, there was less variability in these scores and predictive correlations with brain activity in the reported areas were present only as non-significant tendencies (P > 0.07).

Lastly, in order to verify the specificity of brain activity during recovery from deafness, we applied the same correlation analysis to our previous data obtained in normal hearing subjects (Rouger et al., 2012) who performed the same visual word discrimination task in the same PET paradigm. From the correlation analysis performed in the control group, none of the areas obtained in the patients with cochlear implants emerged (P > 0.1). Similarly, these regions did not emerge when a correlation analysis was performed with a visual non-speech related control task (data not shown); this suggests that our results obtained in deaf patients are specific to speech comprehension recovery through the cochlear implant.


Speech comprehension recovery in adult cochlear implant: global network

In this study, we investigated whether the initial brain activity shortly after cochlear implantation could correlate with the auditory recovery after 6 months post-implantation and, consequently, could be predictive of the cochlear implant outcomes.

The whole-brain regression analysis uncovered a limited set of brain regions in which the activity level was significantly related to the auditory scores obtained after 6 months of experience with the implant. The use of a more liberal statistical threshold revealed that these cortical regions are in fact embedded in a complex cortical network encompassing the right occipital, the left inferior frontal, and the left posterior temporal regions of the brain, as well as the right middle STG/STS (Fig. 5A). In addition to the frontal cortex, there is clearly an occipito-temporal stream that is associated with a high degree of recovery in auditory speech comprehension. In contrast, a high activity level in the regions running along the right middle STG/STS is predictive of weak speech recovery in patients with cochlear implants.

Figure 5

Model of crossmodal reorganization and auditory recovery. (A) Representation at a liberal statistical threshold (P < 0.01) of the cortical areas, in which the activity level at time of implantation are positively (red) or negatively (blue) correlated to the auditory speech comprehension observed 6 months after cochlear implantation. This representation reveals the opposite implication of a ventral occipito-temporal network and the right middle STG/STS region. (B) A schematic representation of the impact of intra- and cross-modal plasticity in cochlear implant (CI) outcomes in both pre- and post-lingually deaf patients. In prelingually deaf cochlear implant users, a high level of activity in both the visual (intra-modal compensation) and the middle STG/STS (crossmodal reorganization) regions are predictive of a weak auditory recovery. In patients with cochlear implants with post-lingual deafness, a high level of intra-modal compensation in the visual area and a weak level of crossmodal reorganization are associated with a high cochlear implant outcome. In consequence, the activity levels in the visual and STG/STS areas are inversely correlated. Intramodal and crossmodal thresholds constrain the reversibility of the functional reorganization, making a clear distinction between the potential for auditory recovery in pre- and post-lingually deaf patients.

Though our unique approach has no direct analogues in the studies of patients with cochlear implants, we will discuss some similar studies, which help to understand our results. In comparison to a similar approach performed during the acquisition of speech comprehension of congenitally deaf patients with cochlear implants (Lee et al., 2005, 2007), our analysis revealed both common and different regions correlated with speech comprehension recovery. In particular, the congenitally deaf patients’ high activity level in both the visual and middle STG/STS is associated with a low level of auditory recovery, leading to a positive correlation between the activity levels in these two regions. These dissimilarities between developmental and adult studies reflect fundamental differences in the impact of cortical plasticity in the recuperation of speech comprehension in cochlear implant users, as we consider later in the ‘Discussion’ section.

Role of the inferior frontal cortex in speech recovery after adult cochlear implant

Though there are no similar data in literature for the predictive role of the early post-implantation activity in the left frontal cortex for the recuperation of post-lingually deaf adult cochlear implant users, Lee et al. (2007) studied whether the pre-implantation fluorodeoxyglucose-PET at rest can predict auditory outcome in prelingually deaf children. Speech scores in their study were also associated with increased initial activity in the left frontal cortex. Although the data obtained in deaf children and adults cannot be directly compared [see Figure 5 in Lee et al., (2007)], the implication of the left frontal cortex in the correlation analysis is independent of the nature (rest or speech processing) and modality (visual or visuo-auditory) of the task, and importantly, independent of the age at which the deafness occurred (at birth or during adulthood). This indicates that the functional implication of the left frontal areas in speech comprehension recovery of cochlear implanted deaf patients occurs at multiple levels.

The left frontal cortical region is involved in numerous speech-related functions, from phonological and semantic processing to short-term memory (Friederici, 2011). However, its impact on the success of rehabilitation through the cochlear implant could be related to its more global implication in diverse cognitive aspects of language acquisition or adaptation (Opitz and Friederici, 2003). In normal hearing subjects exposed to a simulated cochlear implant (vocoding), there is a large set of auditory areas that are sensitive to the degradation of the auditory signal (Strelnikov et al., 2011). However, only the left inferior frontal cortex presents an activity level correlated with the scores of the vocoded signal perception after learning (Eisner et al., 2010). This suggests that the frontal cortex is predictive of cochlear implant outcomes, because this region hosts high-level linguistic functions that are crucial to adapt to the vocoded speech: phoneme perception, segmentation and working memory. Such an hypothesis is in agreement with studies showing a progressive re-activation of Broca’s area (Rouger et al., 2012) depending on the degree of auditory recovery in cochlear implant users (Green et al., 2005; Mortensen et al., 2006).

Visual activity and performance level in speech recovery after adult cochlear implant

One of the most remarkable results of our analysis is that the largest and most significant activity cluster correlated with cochlear implant outcomes is observed in the visual occipital cortex. When measured on an individual basis, there is a high correlation between this cortex and the auditory scores obtained for this occipital activity (r = 0.9). The higher the level of activity in the visual cortex, the higher the auditory proficiency will be after cochlear implantation. This visual region corresponds to the representation of the central visual field (∼3°), which suggests an implication in foveal gazing, such as during lip-reading.

In the present study using three different conditions, the same visual region presents a positive correlation with cochlear implant outcomes. Consequently, the predictive role of the occipital visual cortex is task and modality-independent, which reinforces its role for long-term adaptive strategies. However, there is a crucial difference concerning the predictive role of the visual cortex between post-lingual deaf adults and prelingually deaf children (Lee et al., 2005, 2007). In prelingually deaf adults (Lee et al., 2005), poor cochlear-implant subjects present a higher resting state activity in the visual occipital cortex and fusiform gyrus. Further, using a liberal statistical threshold, the same visual areas were negatively correlated to cochlear implant outcomes (Lee et al., 2007). These results have been interpreted as the consequence of intra- and cross-modal reorganizations that occur during deafness. In prelingually deaf children, the acquisition of speech-reading skills or sign language as sensory substitution strategies for speech comprehension (Tyler et al., 1997) is linked to functional reorganization (Bavelier and Neville, 2002) expressed as activations of the auditory areas by visual linguistic information (Nishimura et al., 1999; Petitto et al., 2000). Thus, it has been proposed that the long-term colonization of the auditory areas by visual speech processing in prelingually deaf adult cochlear-implant users interferes with auditory treatment (Lee et al., 2001).

In the present study, the beneficial commitment of the visual cortex for auditory recuperation after cochlear implantation corresponds to the important role of visual input for speech comprehension in post-lingually deaf patients (Rouger et al., 2007, 2012; Strelnikov et al., 2009, 2010) concerning both phonological (Woodhouse et al., 2009) and lexical access. After language acquisition, many words become audio-visual objects (van Wassenhove et al., 2005), and a long period of deafness may, to some extent, degrade the auditory counterpart of these bimodal objects. During the partial restoration of audition by cochlear implants, the visual counterpart of the audio-visual objects helps decipher the auditory information and finally increases the capacity for auditory discrimination (Barone and Deguine, 2011). Such visuo-auditory synergy is also observed at the neurofunctional level in cochlear implant users, in which, after implantation, there is a progressive increase of activation in the visual areas (Giraud et al., 2001; Strelnikov et al., 2010). Similarly, in normal-hearing subjects, there is evidence at a neuronal level of a facilitatory visual influence on auditory responses (Besle et al., 2004; Arnal et al., 2009; Blank and von Kriegstein, 2012). Such mechanisms could be more efficient in cochlear implant users with a high initial level of activity in the visual cortex after implantation, leading to higher proficiency in auditory recovery.

Finally, the beneficial role of visuo-auditory synergy is furthermore apparent from the fact that high activity in the posterior temporal cortex is associated with high cochlear implant outcomes. The posterior temporal cortex is known for audio-visual integration during speech processing (Wright et al., 2003), such as semantic decisions under cross-modal influence (Kang et al., 2006) or visually-based deciphering of ambiguous auditory phonemes (Kilian-Hutten et al., 2011); it has an amodal role in making sense of environmental sounds and images. Thus, the correlation of scores with this area reflects the initial capacity of patients with cochlear implants for audio-visual integration and amodal semantic processing.

Auditory activity and performance level after adult cochlear implant

An increase of the metabolism in the temporal auditory areas (Lee et al., 2001), which is associated with low cochlear implant outcome has been attributed in prelingually deaf cochlear implanted children to the deleterious effect of the visual cross-modal reorganization. How much the primary auditory cortex is implicated in cross-modal reorganization or whether this reorganization engages auditory areas of higher hierarchical levels, remains unclear (Kral et al., 2003). Although the activity level in the auditory core (BA41–42) was negatively associated with low cochlear implant outcomes in a large data set of prelingually deaf children (Lee et al., 2001), it was either marginally involved (Lee et al., 2007) or not correlated at all in patients with cochlear implants with post-lingual deafness (Giraud and Lee, 2007). In the present analysis of post-lingually deaf adults, A1 as well as the surrounding auditory areas of the temporal plane were not associated with any performance level in auditory recovery 6 months later. Firstly, these results tend to suggest that the spectro-temporal analysis performed at the low stages of auditory processing is not the limiting factor for the recovery of speech comprehension in post-lingually deaf adults. Secondly, it indicates that cross-modal reorganization after deafness probably does not occur at low auditory hierarchical levels such as A1, in agreement with electrophysiological and anatomical research in animals (Kral et al., 2003; Barone et al., 2013) or brain imaging studies in humans (Nishimura et al., 1999; Petitto et al., 2000). Lastly, it is also of importance to take into account that the areas from which predictions of cochlear implant outcomes can be derived are extracted with a similar accuracy using resting conditions or speech-driven conditions. These results highlight the fact that the crucial role in brain plasticity related to the high speech recovery level after cochlear implantation belongs to the compensatory network outside the primary auditory cortex. This long-term reorganization exists in the brain even at rest and is only slightly modified by stimulation with speech.

However, a deleterious impact of cross-modal reorganization in associative auditory areas can be found at the level of the middle STG/STS in both post-lingually deaf adults (present study) and prelingually deaf children (Lee et al., 2007). Brain coordinates indicate that the same area is concerned in both studies. The middle STG/STS is a large cortical region with functionally distinct territories that include visual, auditory and multisensory integration (Beauchamp, 2005). As high initial activity in the middle STG/STS corresponds to the poor results at the later period post-implantation, it follows that its initial activity is caused by something which is deleterious for speech restoration. In our study, the STG/STS cortical locus closely matches the STS region reported in literature as voice-sensitive cortical regions belonging to the temporal voice areas (TVA network) (Belin et al., 2000; Von Kriegstein and Giraud, 2004) and considered as being predominantly an auditory area. The peaks for all the conditions (visual, audio-visual and rest) coincide, within the smoothing error, with the sub-peak, which has been shown to be activated during lip-reading in adult cochlear implant users (Rouger et al., 2012) or by sign language in prelingually deaf adults (Sadato et al., 2004). It is noteworthy that in adults (Rouger et al., 2012), cross-modal reorganization in the STG/STS region decreases progressively as long as the patients are recovering auditory speech comprehension. In consequence, it is probable that the capacity for reversal of the cross-modal reorganization of the STG/STS region is crucial to the ability of cochlear implant users to recover speech comprehension.

Cross-modal reorganization and auditory recovery in pre- and post-lingually deaf patients with cochlear implants

Lengthy debate has occurred regarding the role—deleterious or beneficial—of the visual processing of speech and the capacity of deaf patients to recover auditory speech comprehension after cochlear implantation. In prelingually deaf children, it is now clearly established that the colonization of the auditory areas by visual functions proscribe a restoration of auditory speech processing (Lee et al., 2001). In patients with cochlear implants with post-lingual deafness, vision can be deleterious for non-speech (Doucet et al., 2006; Champoux et al., 2009) or during incongruent or ambiguous audio-visual conditions (Rouger et al., 2008). However, visual speech is important to help decipher the incomplete auditory signal delivered by the implant. Further, early exposure to visual language in early deaf subjects is beneficial to learning a new language suggesting that the crucial factor is more related to the experience of language itself rather than to the modality (Mayberry et al., 2002). However, in case of restoration of spoken language through cochlear implant, in both pre- and post-lingually deaf patients, the functional plasticity of the visual cortex and of the middle STG/STS plays a key role in the capacity of the speech comprehension network to process auditory speech information (Fig. 5B). The visual cortex presents enhanced activity corresponding to intramodal compensatory mechanisms linked to high cochlear implant outcomes (Doucet et al., 2006), whereas the middle STG/STS undergoes cross-modal reorganization by processing visual speech-reading (Rouger et al., 2012). The visual cortex and the STSG/STS regions are functionally related as the activity in one region is inversely correlated to the activity in the other (Fig. 4). Further the degree of intramodal compensation and cross-modal reorganization induced by deafness in these regions (Fig. 5B) will impact the success of rehabilitation through the auditory prosthesis. In prelingually deaf cochlear implant children, a high activity level in both the visual and middle STG/STS is associated with a low level of auditory recovery, leading to a positive correlation between the activity levels in these two regions. The high visual cortex activity in prelingually deaf cochlear implant children may be explained by intra-modal compensation within the visual system and the high STG/STS activity by the cross-modal reorganization of the auditory system.

In post-lingually deaf adult patients with cochlear implants, intra-modal compensation in the visual area and a weak level of cross-modal reorganization of the auditory STG/STS are associated with a high cochlear implant outcome. In consequence, the activity levels in these two regions are inversely correlated. We hypothesize that the order of magnitude of intra- and cross-modal plasticity may depend on the duration of deafness. It may act directly on the reversibility of the functional reorganization leading to a clear distinction between the cochlear implant outcomes of pre- and post-lingually deaf patients with cochlear implants.

It is important to note that this study also implicates the same network (the occipital visual cortex and the STG/STS) as in cross-modal reorganization among congenitally blind subjects (Gougoux et al., 2009), adding further evidence for its global implication in cross-modal compensation after sensory loss.


In post-lingually deaf adult cochlear implant users, the influence of the visual cortex on the efficiency of purely auditory speech perception suggests the existence of some long-term neural facilitation mechanisms that build up a real synergy between the two modalities, such that a better functional level of one modality leads to the better performance of the other (Barone and Deguine, 2011). Such cooperation may be a reflection of the multisensory nature of the perceived world, a feature which is especially present for speech. Thus, predictions about a word originating from the visual modality may facilitate the perception of the word’s auditory counterpart in communicative situations. These visual predictions exist as internal representations at rest.

Additionally, the practical implementation of these findings concerning the inter-modality facilitation in the brain is a necessity to develop the visual capacities of post-lingually deaf adult cochlear implant users during rehabilitation programmes to improve and accelerate the process of restoring their auditory performance.


This work was supported by the ANR Plasmody (ANR- 11-BSHS2-0008 to B.P. and O.D.), The DRCI Toulouse (Direction de la Recherche Clinique et de l’Innovation to K.S.), the region Midi-Pyrénnées (RTT N° 10006537 to K.S. and B.P.) the Fondation pour la Recherche Médicale (to S.L. and J.R.), and the recurrent funding of the CNRS (to O.D. and B.P.).


We would like to thank the CI patients for their participation, ML. Laborde and M. Jucla for their help in stimuli preparation, G. Viallard and H. Gros for their help in data acquisition, C. Marlot for help in bibliography.

sound pressure level
superior temporal gyrus and sulcus


View Abstract