Electroconvulsive therapy (ECT) is the most effective short-term treatment for severe depression (1) but is limited by issues of acceptability, need for anesthesia, seizure induction, and cognitive side effects (2–4). One proposed alternative is repetitive transcranial magnetic stimulation (rTMS). A hand-held coil is used to deliver intense, ultrabrief magnetic pulses that pass unimpeded through the scalp and skull to induce focal secondary electrical stimulation of targeted cortical regions (5). rTMS is nonconvulsive, requires no anesthesia, has a safe side effect profile, and is not associated with cognitive side effects. There has been much interest in rTMS for targeting neuronal circuitry implicated in neuropsychiatric disorders, e.g., the left dorsolateral prefrontal cortex in depression (6, 7).

Initial randomized studies comparing rTMS with ECT in depression suggested that its effectiveness may approach that of ECT, particularly in nondelusional depression (8–11). A decision analysis, based on an assumption of equal effectiveness, led to the conclusion that rTMS would cost less than ECT (12). We wished to compare rTMS with ECT in routine clinical practice and used a pragmatic approach (13) to perform a multicenter randomized, controlled trial to test the equivalence of rTMS with ECT for major depression with a 6-month follow-up.

Patients

Patients were recruited from the South London and Maudsley National Health Service Trust, London, and Pembury Hospital in the Invicta Mental Health Trust, Kent, U.K. Recruitment was between January 2002 and August 2004. The study was a two-group, parallel-design, randomized, controlled trial of rTMS and ECT. The main entry criterion was referral by a consultant psychiatrist for ECT to treat a major depressive episode. Diagnosis was confirmed by using the mood episodes module of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID) (14). Eligible right-handed patients at least 18 years old were invited to participate. The exclusion criteria were inability to have rTMS because of metallic implants or foreign bodies, history of seizures, substance misuse in the previous 6 months, being medically unfit for general anesthesia or ECT, ECT or rTMS in the previous 6 months, dementia or other axis I diagnosis, and inability or refusal to provide informed consent. The patients continued their usual medical care and psychotropic medications; no medication changes were made during the treatment course. Local ethics committees approved the study. After complete description of the study to the subjects, written informed consent was obtained.

Randomization

To ensure allocation concealment, after baseline assessment by trained research workers (S.E., A.M.), the patients were randomly allocated to ECT or rTMS by an independent third party using a protected and concealed computer database containing the randomization list. The randomization was stratified by the patient’s health trust. Subsequent ratings were performed by researchers blind to treatment (G.P., R.P.). The patients were not blind to the allocated treatment but were asked not to discuss the treatment with the raters. To test the blinding, the raters were asked to guess the allocated treatments after the end-of-treatment assessments.

Therapies

rTMS was given by research physicians (S.E., A.M.) using the Magstim Super Rapid stimulator (Magstim Co., Whitland, U.K.) with a figure-eight coil kept cooled on ice, essentially as previously described (8, 10). In the first session, the motor threshold of the abductor pollicis brevis site in the left motor cortex was determined by visual inspection using a method of limits (15). Stimulations were given at 110% of the motor threshold to the left dorsolateral prefrontal cortex, deemed to be located 5 cm anterior to the abductor pollicis brevis in the parasagittal plane. Each session entailed 20 trains at 10 Hz for 5 seconds with 55-second intertrain intervals. A full course comprised 15 daily sessions (total of 15,000 magnetic pulses) administered weekdays, beginning on Monday.

ECT was administered twice weekly with hand-held electrodes according to the guidelines of the Royal College of Psychiatrists; methohexitone (0.75–1.0 mg/kg) was used for anesthesia, and suxamethonium (0.5–1.0 mg/kg) was used as a muscle relaxant (16). A Thymatron DGx device (Somatics, Lake Bluff, Ill.) was used at the South London and Maudsley sites, and the Mecta SR2 (Mecta Corp., Lake Oswego, Ore.) was used at Pembury. Seizure duration was measured by EEG monitoring. Seizure threshold was established by a method of limits at the first session (17). Subsequent treatments were given at 1.5 times the seizure threshold for bilateral frontotemporal ECT and 2.5 times the seizure threshold for right unilateral ECT. Stimulus charge was titrated upward as required during the treatment course according to standard stimulus dosing protocols (16). The number of ECT treatments depended on the patients’ responses as determined by the referring physicians.

Assessment and Outcomes

Baseline assessments were performed before randomization. The outcome measures were repeated at the end of treatment, 2–3 days after the final session, and at follow-up 6 months later. Additional baseline data, obtained by interview and review of hospital records, included age, sex, duration of the current depressive episode, history of depression and ECT, presence of psychotic symptoms (delusions and/or hallucinations) as detected by the SCID, treatment resistance as measured by the number of previous adequate courses of antidepressants and augmentation strategies, and current psychotropic medications.

The primary outcome measure was the end-of-treatment score on the 17-item Hamilton Depression Rating Scale (HAM-D) (18) and the rate of remission; remission was defined as a HAM-D score of 8 or less. Response was defined as a 50% reduction in the baseline HAM-D score at the end of treatment. The secondary outcome measures included self-ratings of depression with both the Beck Depression Inventory-II (19) and aggregated visual analogue mood scales (20) plus a global measure of psychopathology on the Brief Psychiatric Rating Scale (BPRS) (21).

The secondary outcomes also included self-rated and observer-rated measures of side effect symptoms and cognition. Subjective symptoms potentially attributable to ECT or rTMS were assessed with a shortened version of the Columbia ECT Subjective Side Effects Schedule (22, 23), modified to include potential rTMS side effects (e.g., seizure induction, scalp discomfort, hearing loss) and any unpredictable adverse events. Cognitive symptoms embedded within this instrument allowed a self-rating of cognition, consisting of the total number of positive responses to the following five questions: “Have you had trouble recalling people’s names?” “Have you felt confused or disoriented?” “Have you had any memory problems?” “Have you had trouble concentrating?” and “Have you had trouble holding in your memory new things you have learned?”

Changes in cognitive function, due to treatment side effects or response, were recorded. Global cognition was assessed by using the CAMCOG section of the Cambridge Examination for Mental Disorders of the Elderly (24), which also generates a Mini-Mental State Examination (MMSE) score (25). It provides a total score (maximum, 107) plus subscale scores for different aspects of cognition and has been used previously to study cognition in depression (26).

Statistical Analyses

Trial data available around the start of this study demonstrated that rTMS was not better than ECT but that its effectiveness possibly approached that of ECT (8–11). This finding allowed us to review our original calculations for the number of subjects needed. We believed that rTMS would be of clinical interest as an alternative if the percentage reduction from the baseline HAM-D score achieved by rTMS was at least three-quarters that achieved by ECT. Using data from a large series of depressed patients’ responses to ECT (27), we calculated that the maximum acceptable difference between rTMS and ECT change scores would be 18.1 percentage points. We estimated that 22 subjects per treatment group would be required to have 80% power to demonstrate such equivalence in a one-sided equivalence test at a 5% significance level, if the true effects of the two treatments were assumed to be the same.

Outcomes were analyzed on an intention-to-treat basis. HAM-D scores for the two treatment arms were compared by using analysis of covariance (ANCOVA) with HAM-D scores at the end of treatment and 6-month follow-up as the dependent variables and baseline HAM-D score included as a covariate, along with recruitment site to adjust for center effects. The model also included main effects of time (end of treatment or follow-up) and treatment, plus their interaction. End of treatment was defined as the last treatment received within the allocated course, irrespective of the number of treatments. rTMS courses were predetermined to have fixed durations, i.e., 15 weekdays, whereas ECT courses were of variable duration. Ratings obtained within the treatment courses were therefore not included in the analyses. Standard errors that were robust against correlations within subject clusters were used to account for the two repeated measures per subject. If the treatment-by-time interaction tested significant at the 5% level, two post hoc comparisons were performed to compare scores at the end of treatment and follow-up within each treatment arm separately (Bonferroni-adjusted significance level, 2.5%). If the interaction term was not significant, it was excluded from the model and the main effect of treatment was evaluated in order to estimate the treatment effect. Standardized effect sizes were calculated for significant results by dividing the estimated group difference by a measure of background variability (standard deviation of baseline score).

To assess the equivalence of the two interventions for the primary outcome measure, the end-of-treatment estimate of the group difference was transformed into an estimate of group difference in the percentage reduction from the baseline HAM-D score, and a confidence interval (CI) was established for this.

Secondary outcomes were analyzed by using the same ANCOVA model. Baseline and change scores for scores on the HAM-D and Columbia ECT Subjective Side Effects Schedule were correlated by using Spearman’s rank correlation. Binary outcomes at single time points (remission at the end of treatment or relapse at follow-up) were compared in the two groups by using Fisher exact tests. The data were analyzed by using Stata 8.0 (Stata Corp., College Station, Tex.).

Enrollment

The trial profile is shown in Figure 1. Of 260 patients referred for ECT, 107 were eligible depressed patients. The most common reason for exclusion was not consenting to ECT while being formally treated in accordance with the U.K. Mental Health Act 1983. Of those eligible, 46 (43.0%) consented to enter the study. There was no statistical difference in mean age or sex ratio between the eligible patients who consented and those who declined to participate. Five patients in the rTMS group terminated treatment early, having 10 or fewer sessions, because they believed they were not improving, and one patient could not attend the 15th session; all but one agreed to end-of-treatment assessments. The rTMS treatments were well tolerated by all patients, and nobody dropped out because of pain at the stimulation site. None of the ECT patients dropped out at this stage. In the ECT group, 18 (81.8%) had bilateral and four (18.2%) had unilateral ECT. The mean number of rTMS sessions was 13.7 (SD=2.7, range=5–15), while the mean number of ECT sessions was 6.3 (SD=2.5, range=2–10). The mean rTMS dose was 67% (SD=11) of the maximum output of the stimulator. The course durations (days from first to last treatment) were comparable for ECT (mean=22.4, SD=12.7) and rTMS (mean=19.5, SD=6.3).

Rater treatment guesses were unavailable for eight patients. Of the remaining 38, five had inadvertently informed the raters and the raters guessed correctly for 30 (92.1%). The extent of response was the main reason given for the correct guess. After 6 months, 21 (87.5%) of the rTMS patients and 16 (72.7%) of the ECT patients agreed to follow-up. The main reason for not agreeing was unwillingness to be interviewed again. Five (20.8%) of the patients randomly assigned to rTMS (who received nine, 10, 14, 15, and 15 sessions) crossed over to ECT after the end-of-treatment assessments but were initially analyzed in the rTMS group.

Table 1 summarizes demographic and clinical baseline information. At randomization the two treatment groups were similar on most measures.

Primary Outcome

Changes in HAM-D scores over time are illustrated in Figure 2. The posttreatment group difference varied significantly with assessment point (group-by-time interaction: F=6.20, df=1, 45, p=0.02). Post hoc tests showed that the end-of-treatment HAM-D scores were significantly lower in the ECT group than in the rTMS group (F=10.89, df=1, 45, 95% CI for difference=3.40 to 14.05, p=0.002), demonstrating a strong standardized effect size of 1.44. However, at the 6-month follow-up, the HAM-D scores did not differ between groups (F=0.01, df=1, 45, 95% CI= –6.92 to 6.33, p=0.93). At the end of treatment, 13 patients (59.1%) in the ECT group met the remission criterion (HAM-D score of 8 or lower), while only four (16.7%) did in the rTMS group (Fisher’s exact test, p=0.006). Thirteen patients (59.1%) in the ECT group and four (16.7%) in the rTMS group were deemed responders (Fisher’s exact test, p=0.006). Of those who agreed to follow-up assessment after 6 months, six of 12 ECT patients with remissions and two of the four rTMS patients with remissions continued to meet the remission criterion.

Psychosis and older age have been reported to be negative indicators for response to rTMS (8, 28). Anticonvulsant mood stabilizers and benzodiazepines might also affect response (29). We therefore performed analyses examining whether adding interactions between treatment and psychosis, treatment and age, and treatment and anticonvulsants/benzodiazepines had a significant effect on the model for primary outcome. There was no evidence at the 5% level to suggest that the effect on HAM-D score was modified by the interaction of treatment type with psychosis (F=3.76, df=1, 45, p=0.06), with age (F=2.76, df=1, 45, p=0.10), or with anticonvulsant or benzodiazepine treatment (F=0.88, df=1, 45, p=0.35). When the subjects with psychosis were excluded from the analysis, 12 (63.2%) of the 19 subjects in the ECT group and three (15.8%) of the 19 in the rTMS group met the remission criterion at the end of treatment (Fisher’s exact test, p=0.007); 13 subjects (68.4%) in the ECT group and four (21.1%) in the rTMS group were responders (Fisher’s exact test, p=0.02).

In addition to intention-to-treat analyses, a received-treatment analysis was performed as a sensitivity analysis, but this did not affect the primary outcome. This excluded the five patients in the rTMS group who had 10 or fewer rTMS sessions and took into account the crossover of five rTMS patients to the ECT group before the 6-month follow-up.

The mean reduction in score on the HAM-D achieved at the end of treatment, in relation to the adjusted baseline, was 14.1 points for ECT and 5.4 for rTMS. This translates into mean percentage reductions from baseline of 58% and 22%, respectively. The absolute difference in percentage reduction from baseline was therefore 36% (95% CI=14% to 58%). This point estimate lies considerably outside the predefined equivalence range (i.e., up to 18.1 percentage points), and so does almost all of the respective confidence interval, with only a small fraction of the confidence range (from 14% to 18.1%) falling into the predefined equivalence range. The rTMS treatment effect was therefore statistically significantly worse than that of ECT, and it was at least 14 percentage points worse.

Secondary Outcomes

Mood symptoms

Changes in scores on the Beck Depression Inventory, visual analogue mood scales, and BPRS are illustrated in Figure 3. While the Beck scale and BPRS showed patterns similar to the pattern for the HAM-D, there was no statistical evidence that the treatment effect varied over time; there was no significant interaction between group and time for the Beck scale (F=1.39, df=1, 45, p=0.25), aggregate visual analogue mood scales (F=1.66, df=1, 41, p=0.20), or BPRS (F=0.72, df=1, 43, p=0.40). Interaction terms were therefore excluded from the model, and the main effect of treatment arm was evaluated across the end-of-treatment and 6-month follow-up time points. This showed significantly lower scores for ECT than for rTMS on the Beck Depression Inventory (F=7.29, df=1, 45, 95% CI for difference=2.27 to 15.58, p=0.01), visual analogue mood scales (F=17.72, df=1, 41, 95% CI=106.54 to 302.78, p<0.001), and BPRS (F=4.75, df=1, 43, 95% CI=0.47 to 11.69, p=0.03). All of these differences translated into substantial effect sizes for ECT on the standardized scale (Beck Depression Inventory, effect size=0.9; visual analogue scales, effect size=1.8; BPRS, effect size=0.7).

Side effect symptoms and cognition

Scores on the Columbia ECT Subjective Side Effects Schedule at the different time points are shown in Table 2. While there was no significant group-by-time interaction, the ECT group was estimated to have overall significantly lower scores for subjective side effect symptoms after treatment (F=6.00, df=1, 40, 95% CI for difference=0.51 to 5.33, p=0.02), with a standardized effect size of 0.7. The baseline side effect and HAM-D scores were significantly correlated (r^s=0.44, N=43, p=0.003), while the changes in the side effect scores from baseline to the end of treatment strongly correlated with changes in HAM-D score (r^s=0.76, N=36, p<0.001), suggesting that the side effects scale was also measuring depressive symptoms (23). Scores for self-reported cognitive side effects, based on the five-question measure within the Columbia ECT Subjective Side Effects Schedule, were analyzed separately (Table 2), and there was no significant group-by-time interaction or main effect of group.

No significant differences were found, either as a group-by-time interaction or main effect of group, for the total CAMCOG score, MMSE score, or score for verbal fluency or anterograde or retrograde memory on subscales of the Cambridge Examination for Mental Disorders of the Elderly (Table 3). There was a significant group main effect on the subscore for attention and orientation (F=9.36, df=1, 36, 95% CI=0.80 to 3.95, p=0.004), with the ECT group showing mild improvement at the end of treatment but the rTMS group showing a decrease in mean score.

During the 6-month follow-up period, a 76-year-old man in the rTMS group died from previously diagnosed prostatic cancer; he was physically stable during treatment with no evidence of CNS involvement. No other major adverse events were recorded.

This trial showed that 3 weeks of rTMS was not identical to ECT and its effectiveness was insufficient to merit replacement of ECT by rTMS. In addition, ECT was substantially more effective than rTMS in short-term treatment.

Because of the severity of the depression, it would not have been ethically justifiable to have a placebo group. However, similar treatment effects were found on every mood outcome measure. It is interesting that the HAM-D scores in the rTMS group improved during the 6-month follow-up and became similar to those of the ECT group, probably owing to ongoing intensive management. Indeed, five rTMS patients went on to be treated with ECT, but this did not alter the trial result in the received-treatment analysis.

Of 260 patients referred for ECT during the trial, 30.0% did not consent to treatment and were deemed sufficiently ill to warrant involuntary ECT under U.K. mental health legislation. This rate is higher than the proportion (16%) in 2002 who received ECT under mental health legislation in England (30) and reflects increased use of the mental health legislation in inner-city London, e.g., 26% of all admissions (31). However, as rTMS requires a very high degree of cooperation for prolonged periods, it would not have been possible to treat nonconsenting patients. Thus, the present trial reflects the patients referred for ECT whom it was possible to treat with rTMS.

To approximate routine clinical practice, the patients continued their usual medications, including anticonvulsant mood stabilizers and benzodiazepines. These drugs can raise the seizure threshold, but this was accommodated by the ECT protocol (16). The effect of these medications on therapeutic response to rTMS has not been systematically evaluated but, like age and psychosis, did not affect the primary outcome.

Previous randomized comparisons of rTMS and ECT have shown rTMS to be either less effective (8, 9) or not statistically different from ECT (10, 11). In the best-designed study, Grunhaus et al. (11) allocated 20 patients to right unilateral ECT (stimulus dosing protocol using 2.5 times the seizure threshold; mean=10.3 treatments, SD=3.1) and 20 patients to rTMS (20 daily sessions of 20 trains at 90% motor threshold at 10 Hz for 6 seconds). They found a remission rate (HAM-D score of 8 or less) of 30% in both groups. Although the mean baseline HAM-D scores were similar to those in our trial, this remission rate for ECT was one-half that in our study according to the same criterion. It is possible that the form of unilateral ECT used was less effective, as has been previously reported (3), which would account for the apparent equivalence of treatments.

Subjective Side Effects

Some “side effects” of ECT may be symptoms of depression (17, 32). Our findings support this in that there was a strong correlation between baseline HAM-D and side effect scores and also between the change in depression and side effect scores. Also, the ECT group, which experienced significantly greater improvement in depressive symptoms, had lower side effect scores after treatment than the rTMS group.

A systematic review showed that one third of patients reported substantial memory loss after ECT (4). We were therefore surprised there were no significant differences in subjective cognitive complaints at the end of treatment or even 6 months later. If anything, there was a trend for complaints to diminish more in the ECT group than in the rTMS group, probably paralleling the improved mood ratings seen in the ECT group. We know of only one previous randomized study of ECT and rTMS that included subjective side effects, and in that study the scores were improved in both groups (9).

Cognitive Side Effects

Previous studies have shown that ECT can adversely affect cognition (33). In particular, it can interfere with memory, disrupting both new learning and remote memory. In contrast, studies on rTMS have suggested either no adverse effects on cognition (34) or some positive effects independent of improved mood (35).

To track changes in global cognition, Grunhaus et al. used the MMSE in their randomized trials of rTMS and ECT (8, 11) and found no significant difference in scores over time or between groups. We have extended these studies by following patients over 6 months and by using the more detailed CAMCOG portion of the Cambridge Examination for Mental Disorders of the Elderly. We also found no differences over time or between the groups for either the MMSE or CAMCOG scores. In the analysis of CAMCOG subscales, there was a modest but significant improvement in attention and orientation in the ECT group, but no differences were detected on the subscale for anterograde or retrograde memory. However, caution is warranted in interpreting these cognitive findings in light of the high dropout rate and small group size. In addition, it is possible that more sensitive memory assessments may have shown differences between the two treatments.

For example, O’Connor et al. (36) reported that, compared to patients treated with rTMS, those treated with ECT had impaired acquisition and delayed recall of newly learned words 2–4 hours after the final treatment but that these deficits disappeared 2 weeks later. Patients receiving ECT also had impaired retrograde memory for news events of the previous 25 years, and they did not fully recover from this impairment by 2 weeks. Interpretation of this study is limited by the lack of randomization, blinding, and balancing of groups regarding depression severity. In a second comparison study (37), blinded ratings 1 week after course completion showed the ECT group to have more impaired delayed verbal recall plus more deficits in both verbal and visual retrograde memory. While subjective complaints of memory function improved after treatment in the rTMS group, subjective memory function did not improve in the ECT group. Again, interpretation is complicated by a lack of randomization and also by a low therapeutic response to ECT. However, taken together, these studies highlight the potential problem of different types of memory disturbance after ECT and the need for further rigorous research.

Limitations

Loss of rater blinding is a potential source of bias, particularly for our primary outcome, the observer-rated HAM-D. However, we found similar end-of-treatment results with two self-rated mood scales, and these findings were paralleled by the scores on the BPRS as well as the self-rated Columbia ECT Subjective Side Effects Schedule, which could be considered a surrogate measure of depressive symptoms (23). Of the four previously reported randomized trials comparing rTMS with ECT, two were open studies (8, 10) while two (9, 11) were single-blinded but with no information on the success of blinding to allow comparison.

While rTMS was administered for a fixed schedule, ECT continued until treatment response was evident. Although the mean durations of the rTMS and ECT courses were comparable, it could be suggested that many more weeks of rTMS were required (6). However, it has already been reported that little further benefit is derived in continuing rTMS in patients with minimal improvement after 2 weeks of rTMS (8). In the present study, five patients stopped rTMS within 2 weeks because of a perceived lack of benefit. In the remaining 19 patients, who received 3 weeks (14 or 15 sessions) of rTMS, the HAM-D score declined only 19.7% (SD=32.5) from baseline and only four of the 19 met the remission criterion.

The optimal treatment characteristics for rTMS are not known. We chose stimulus intensities greater than those in most of the studies published when recruitment began, in the hope of greater efficacy. Recent studies have used stimulus intensities similar to those in our study but have given more stimuli per session and prolonged the length of the treatment course (7). Identification of the dorsolateral prefrontal site is a problem with most rTMS trials that have adopted the convention of using a fixed distance (i.e., 5 cm) anterior to the left motor cortex abductor pollicis brevis site, which can only be an estimation. One way to overcome this is to use a neuronavigation approach, which entails neuroimaging and can also account for scalp-to-cortex distances to adjust the stimulus intensity (28). This would counter the possibility that rTMS treatments for older patients may not adequately stimulate the prefrontal cortex because of increased distance between the scalp and cortex. In a recent well-designed trial of real versus sham rTMS for treatment-resistant depression, in which this approach was used, the remission rate for real rTMS was 20%, compared to 3% for the sham group (38). Until these treatment variables and methods are systematically compared it is difficult to determine their individual influences on therapeutic response.

Conclusions

Our results indicate that ECT is substantially more effective for severe depression than rTMS and do not support replacement of ECT by rTMS. Several meta-analyses of trials of real versus placebo rTMS for depression have been published (for review, see reference 7). The authors of one of the most recent of these concluded that there was currently insufficient evidence to support routine use of rTMS but that it merited further development (39). It is worth bearing in mind that ECT took several decades to develop and is still not optimal regarding side effects or stimulus delivery. Progress with rTMS has also been slow. We recommend that rTMS methods (e.g., coil placement, stimulus intensity, number of treatments) be more rigorously characterized before further studies are performed to compare it with ECT.