Treatment Outcomes in Depression: Comparison of Remote Treatment Through Telepsychiatry to In-Person Treatment

Telepsychiatry is an increasingly common method of providing expert psychiatric treatment to patients at a distance from the source of care. Its use has been suggested for the treatment of patients in remote locations or in areas where psychiatric expertise is scarce.

Although demonstration telepsychiatry projects since 1959 have shown the usefulness of this modality (1, 2), the high cost of telepsychiatry equipment and transmission, the limited availability of the required equipment, and the difficulty of using the equipment prevented widespread use of telepsychiatry. However, more recent advances in telecommunications have led to the increasing availability of inexpensive, user-friendly telecommunications systems. Thus, during the 1990s, there was a large increase in the number of clinical telepsychiatry programs in many different areas of practice, including child psychiatry (3, 4), geriatric psychiatry (5), forensic psychiatry (6), and rural psychiatry (7–9).

Paralleling this increased clinical utilization has been an increasing number of research projects in telepsychiatry. Most of these studies examined the reliability of telepsychiatry in psychiatric assessment, including assessment of schizophrenia (10), obsessive-compulsive disorder (11), major depression (12), alcohol abuse (13), and cognitive impairment/dementia (14, 15).

Despite the rapid growth in the clinical use of telepsychiatry and the increasing interest in research on this modality, to our knowledge, there have been no large-scale, randomized trials comparing telepsychiatric to in-person treatment of psychiatric illness. The purpose of this study was to compare rates of symptom improvement and remission, patient adherence, patient and psychiatrist satisfaction, and economic effects of remote versus in-person treatment of depression.

Method

Structure of the Study

This study was approved by the Institutional Review Board of the University of Maryland School of Medicine. In this randomized, controlled trial, veterans who were referred to any of three mental health clinics within the Department of Veterans Affairs (VA) Maryland Health Care System were evaluated for participation. The clinics were 1) the Baltimore VA Medical Center located in downtown Baltimore, 2) the Perry Point VA Medical Center located in northeast Maryland, a 1-hour drive from Baltimore, and 3) the Cambridge satellite clinic located on the eastern shore of Maryland, a 90-minute drive from Baltimore. Graduate-trained research assistants contacted potential subjects at or shortly after the time that they were initially referred to one of these clinics. Contacts were made either in person, if the referring clinician contacted the research assistant while the patient was still at the medical center, or by telephone. During this initial contact, the research assistant explained the study, and interested subjects were invited to come to the clinic for an evaluation to determine eligibility for the study. At this visit, the research assistant administered the 24-item Hamilton Depression Rating Scale (16) and the Structured Clinical Interview for DSM-IV (SCID) (17). Patients met the inclusion criteria if they scored 16 or higher on the Hamilton depression scale and met the DSM-IV (SCID) criteria for one of the following five diagnoses: major depressive disorder, dysthymic disorder, adjustment disorder with depressed mood, mood disorder due to a general medical condition, or depressive disorder not otherwise specified. Patients were excluded if they met the criteria for bipolar disorder or schizophrenia at any point in their lifetime or met the criteria for substance abuse or dependence within the past year. They were also excluded if they required hospitalization or if they had been receiving pharmacological treatment for depression for more than a month immediately before the initial visit.

Patients who met the eligibility criteria were offered an opportunity to participate in the study. Research assistants obtained written informed consent after explaining the study to the patient and offering him/her an opportunity to ask questions about the study. Patients who chose to participate were randomly assigned to see a psychiatrist either in person or by means of telepsychiatry (“remote treatment”). In either case, the treatment was carried out with the patient located at the clinic where the patient initially presented for care. A stratified, variable block randomization procedure was used to make treatment assignments on the basis of age (young: <50 years, old: ≥50 years) and depression severity (mild: Hamilton depression scale score <24, severe: Hamilton depression scale score ≥24). Remote treatment occurred in one of the following two arrangements: 1) psychiatrists located at Baltimore saw patients located at Perry Point or Cambridge or 2) psychiatrists located at Perry Point or Cambridge saw patients located at Baltimore. Eight board-certified or board-eligible psychiatrists provided treatment throughout the study. Three were located at Baltimore, three at Perry Point, and two at Cambridge. Each psychiatrist saw patients in both treatment conditions. Remote treatment occurred through the use of personal-computer-based videoconferencing equipment, which consisted of VTEL software (VTEL Corp., Austin, Tex.) and cameras mounted on the monitors. The equipment was connected to integrated services digital network lines, and transmission usually occurred at a rate of 384 kbit/second.

Treatment consisted of eight sessions with a psychiatrist over a 6-month period. The first session occurred immediately after the initial assessment by the research assistant. At this session, the psychiatrist conducted his or her own clinical evaluation. If in the psychiatrist’s opinion the patient did not meet the eligibility criteria, the patient was excluded from the study. Only one patient was excluded through this process, after the psychiatrist diagnosed current substance use disorder during the patient’s initial visit. This patient was deemed ineligible for further participation and was referred to another provider.

Subsequent sessions were scheduled at weeks 1, 3, 7, 11, 15, 19, and 26. Treatment sessions lasted approximately 20 minutes and consisted of antidepressant medication management, psychoeducation, and brief supportive counseling. At each visit, the patient also had a separate meeting with a research assistant during which the patient participated in an interview and completed the self-report measures described in the next section. Subjects were paid $5 per visit for their participation.

Outcome Variables

The major outcome variables in this study were treatment response, treatment adherence, patient satisfaction, psychiatrist satisfaction, and resource consumption or “cost effects.” Treatment response was measured with the 24-item Hamilton depression scale (weeks 0, 7, 15, and 26), Beck Depression Inventory (weeks 0, 1, 3, 7, 11, 15, 19, and 26) (18), Spielberger Trait Anxiety Inventory Scale (weeks 0 and 26) (19), the Spielberger State Anxiety Scale (19) (weeks 0, 1, 3, 7, 11, 15, 19, and 26), Global Assessment of Functioning Scale (GAF) (weeks 0 and 26), Clinical Global Impression (CGI) (20) (weeks 0 and 26), and Medical Outcomes Study 12-Item Short-Form Health Survey (21) (weeks 0 and 26). The presence of a personality disorder was assessed at week 1 with the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (22). Adherence was assessed in terms of dropout rates, time course of dropouts, number of session appointments kept, and pill counts. Patient satisfaction was assessed at weeks 7, 15, and 26 by using a scale that was developed for this study, and psychiatrist satisfaction was assessed at week 26 by using a scale that was developed for this study. In addition, information about demographic characteristics and past and present medical and psychiatric history was collected at week 0.

The cost effect (in 2000 dollars) was measured in two ways: 1) by estimating the marginal costs of operating the telepsychiatry session compared to the in-person session (intervention cost) and 2) by examining whether the telepsychiatry intervention increased or decreased total Veterans Health Administration health care resource consumption for these patients during the 6-month study period (intervention’s cost consequences). Both analyses were done from the provider’s (i.e., Veterans Health Administration) perspective and focused on direct costs to the local medical center. To assess the cost of delivery of the intervention, the operational expenses to the institution for equipment, cabling, access fees, and maintenance fees were prospectively identified, and the specific personnel time and access time used in the clinical portion of each visit were recorded during each visit. The actual times necessary for personnel to conduct the clinical portion of the visit were tracked prospectively at each visit, as was the duration of the remote connection. Depreciated equipment expenses, annual maintenance fees, mean rates for personnel salaries and benefits, and actual access line charges were added to obtain the aggregate consumption of resources per patient visit, patient episode, and study group, and the aggregates were then averaged for comparison between the study groups. Separately, in the analyses of the intervention’s cost consequences, all medical events, including the psychiatry visits in the study, were tracked through the electronic medical records system of the Veterans Health Administration. For each patient, an episode of 6-months’ treatment, including all physician and emergency room visits, admissions, ancillary inputs to care (e.g., radiology and laboratory services), and pharmacy services, was created from enrollment to the actual or expected closeout date. To obtain an unbiased proxy for the cost of these health care events, an external valuation technique was applied (23). We used the national average per diagnosis related group (admissions), per Physician’s Current Procedural Terminology, 4th ed., code (procedures), or per visit, which was derived from published Centers for Medicare & Medicaid Services Medicare rate structures (24).

Statistical Analysis

We assessed the effect of treatment on the severity of depressive symptoms by fitting a mixed model to the repeated follow-up measures in the two treatment groups. An unstructured covariance matrix was specified to account for the dependence of the within-subject measures, and the parameters of the model were estimated by using maximum likelihood. This analysis provided F tests for the main effect of time, treatment group (remote versus in-person treatment), and the interaction of time and treatment group. The time main effect was relevant to the change in severity of depression symptoms over time, an indication of improvement in one or both of the treatment conditions. The primary focus of these analyses was the time-by-treatment group interaction, which was an indicator of whether the change in severity of depressive symptoms over time was influenced by the treatment condition. For the repeated-measures analyses, patients lost to follow-up were retained in the analysis by using their last observable score for all the remaining time points.

The change in depressive symptoms from the beginning to the end of treatment was summarized by a binary variable based on improvement of 50% or more in the Hamilton depression scale score. Remission of depression was summarized by a binary variable based on a final 17-item Hamilton depression scale score of 7 or less. Chi-square tests were used to compare the proportions of subjects in each treatment condition with improvement or remission of depressive symptoms. The proportions of treatment dropouts and of patients who were adherent with medication treatment less than 70% of the time were compared by treatment condition by using chi-square tests. The mean numbers of appointments kept and adherence percentages were compared between groups by using t tests. The time to dropout was the basis for a log rank test used to assess the difference in attrition over time for the two treatment groups.

Patient satisfaction was measured by using a 19-item scale that was developed for this study, and psychiatrist satisfaction was measured by using a 17-item scale that was developed for this study. Each scale consisted of several statements with which the respondents rated their extent of agreement. Statements from the patient satisfaction scale included “The psychiatrist seemed interested in helping me” and “During the course of treatment, I felt confident that I would be able to reach the psychiatrist in case of emergency.” Examples of items on the psychiatrist satisfaction scale included “I felt that I really understood the patient’s problems,” “The patient seemed to have a difficult time understanding what I was saying,” and “I was able to establish a good rapport with the patient.” Each statement was rated on a scale from 1 (strongly agree) to 4 (strongly disagree). (Some items were reverse-scored, depending on the nature of the item.) A total score was calculated by adding up the responses for each question. The distributions of satisfaction scores on both the patient scale and the psychiatrist scale were nearly symmetric, with the means falling about midway in the ranges in all cases. Furthermore, nonparametric (Mann-Whitney) tests gave p values very similar to those derived by using t tests. Therefore, the t test results are reported in this article.

The mean costs of delivering a visit remotely were compared to those of an in-person visit by using a t test. After investigating univariate differences between the study groups for possible consequences of the intervention on total health care expenses, multivariate techniques were used to compare the two groups’ health care resource use during the intervention period. Semilogarithmic ordinary least squares regression of the dependent variable—total health care consumption (in dollars) during the 6-month study period—was employed. Group differences were detected by t test of the estimated group coefficient in regressions where the log-transformed values for total health care costs were regressed on the two-group dichotomous term, with adjustment for the baseline values of age, gender, race, social support, comorbidities, and depression severity.

Power Calculations

The size of the study group was originally chosen to detect small to moderate effect sizes. With the group sizes of 60 that were obtained, there was 80% power for the detection of a standardized difference of 0.5 standard deviation (a moderate effect size) at any given follow-up time point and of an average difference of 0.28 standard deviation (a rather small effect size) over three follow-up time points.

Results

Subject Characteristics

Five hundred sixty-one patients were contacted for possible participation in the study. Of these, 436 were interested in the study and were screened by trained research assistants. Of this group, 131 appeared eligible to participate and were randomly assigned to either remote or in-person treatment. However, 12 of these patients (eight in the remote treatment condition and four in the in-person treatment condition) were ultimately not included in this study. Five of the 12 patients were withdrawn because it was discovered after they had begun the study that they had had an active substance abuse problem during the year before the initial visit. Two patients were excluded because it was discovered after they had begun the study that they had a psychotic disorder and bipolar disorder, respectively. Two patients were excluded because review of their baseline data revealed errors showing that they had not actually met the Hamilton depression scale score eligibility criteria. One patient was unable to complete the baseline assessments and did not return. An additional patient was referred to another provider after being randomly assigned to the remote condition, because the telecommunication equipment was not working properly at the time of the initial visit and the patient required immediate attention. Finally, the patient mentioned earlier (see Method section) was referred out of the study because the psychiatrist to whom he was randomly assigned gave him a diagnosis of an active substance use disorder. Thus, the final study group consisted of 119 patients (105 men and 14 women).

The mean age of the participants was 49.7 years (SD=12.8). Thirty-six percent were African American, 61% were Caucasian, and 3% were Hispanic or Asian. Forty-four percent were married; 37% were divorced, separated, or widowed; and 19% were never married. Fifty percent had more than 12 years of education, 33% were high school graduates, and 17% had less than 12 years of education. Thirty-nine percent were employed full-time, 19% were employed part-time, 13% were unemployed, and 30% were retired or receiving disability. Approximately half (52%) reported a gross income of less than $1,000/month, 28% reported income of $1,000 to $2,000/month, and 19% reported income greater than $2,000/month.

Fifty-nine subjects were randomly assigned to the remote group, and 60 were assigned to the in-person group. No significant differences in age, gender, race, marital status, education, income, or employment status were found between the remote and in-person groups.

Treatment Outcomes

As predicted, patients’ depressive symptoms, as measured by the 24-item Hamilton depression scale, significantly improved over the treatment period (time main effect: F=49.0, df=3, 117, p<0.001), and improvement did not differ by treatment group (time-by-treatment interaction: F=0.4, df=3, 117, n.s.). Improvement in the Hamilton depression scale score was also analyzed in terms of response to treatment (greater than 50% improvement from the first to the last visit). Forty-nine percent of the subjects in the remote group had a response to treatment, compared to 43% in the in-person group (χ²=0.4, df=1, n.s.). Thirty-nine percent of the subjects in the remote group had a remission, as indicated by a final 17-item Hamilton depression scale score of 7 or less, compared to 35% in the in-person group (χ²=0.2, df=1, n.s.).

Results obtained with the Beck Depression Inventory, state anxiety scale, GAF, CGI, and Short-Form Health Survey were similar to those obtained with the Hamilton depression scale. In all cases, scores improved significantly over the course of treatment, and improvement did not differ significantly between the two treatment groups.

As noted earlier in the Method section, inclusion criteria required both a SCID diagnosis of depression by the research assistant and a diagnosis of depression by the psychiatrist. Although the research assistant using the SCID diagnosed major depressive disorder for 106 of the 119 subjects, the psychiatrist gave this diagnosis to only 84 of the 119 subjects. Eighty-two subjects received a diagnosis of major depressive disorder from both the research assistant using the SCID and the treating psychiatrist. The analyses described earlier were repeated with the data from just the 82 subjects who received a diagnosis of major depressive disorder from both raters, and the findings were the same as those reported for the entire study group: no significant differences were found between the in-person and remote conditions.

Adherence to Treatment

Sixteen participants (27%) in the remote group and 18 (30%) in the in-person group dropped out of the study (χ²=0.4, df=1, n.s.). Both groups kept appointments for an average of 6.5 visits during the study period (t=0.2, df=117, n.s.) Furthermore, a time-to-event analysis indicated no significant difference in the time course of the dropouts between the two treatment groups (χ²=0.1, df=1, n.s.).

Medication adherence data were available for 73 subjects. Patients were excluded from this analysis if they had fewer than three visits with complete medication counts. Patients who took at least 70% of the pills they were expected to take were considered adherent, and the others were considered nonadherent. There was no difference in the percentage of adherent patients between the two treatment groups (χ²=0.2, df=1, n.s.).

Satisfaction

There was no difference in patient satisfaction between the remote and in-person groups at visit 4 (t<0.1, df=87, n.s.), visit 6 (t=–0.4, df=74, n.s.), or visit 8 (t=1.3, df=74, n.s.). On average, the scores for patients in both treatment groups fell between the “agree” and “strongly agree” responses on the satisfaction scale, indicating agreement with positive statements about treatment.

Psychiatrist satisfaction scores were compared at the final visit. Since all psychiatrists in the study had seen patients in both treatment conditions, they all provided satisfaction ratings for some patients seen remotely and some seen in person. Psychiatrist satisfaction was greater when patients were treated in person than when patients were treated remotely (t=–2.2, df=79, p<0.05). On average, the scores for psychiatrists in both treatment groups fell between the “agree” and “strongly agree” responses on the satisfaction scale, indicating agreement with positive statements about treatment.

Resource Consumption, or Cost Effects

The estimated marginal costs to the institution were $86.16 for a telepsychiatry session and $63.25 for an in-person treatment session (t=3.2, p<0.001). Thus, the per-session cost of remote treatment was higher than that of in-person treatment. However, when the cost of psychiatrist travel time was factored in and the time-distance effect was modeled, the cost of remote treatment was equal to that of in-person treatment if the psychiatrist had to travel 22 miles from the medical center to the clinic and was less if the psychiatrist had to travel more than 22 miles to the clinic.

Total VA health care resource consumption of each group during the 6-month study period was also compared. The dependent variable of this regression analysis was the log of total health care estimated expense per quarter per patient. The model explained 42% of the variance (model F=4.6, df=7.26, p<0.02) in the patient’s health care expense per quarter. Two independent variables’ coefficients were significant: white patients used a less expensive mix of care (t=–3.5, p<0.0001), and the total number of diagnoses was highly significant in predicting health care expenses (t=29.2, p<0.0001). The remote group was not associated with significantly different overall consumption of Veterans Health Administration health care (t=0.7, n.s.).

Discussion

Although the clinical use of telepsychiatry has increased substantially during the past decade, there has been a paucity of randomized, controlled studies comparing remote to in-person treatment of psychiatric illness. To our knowledge, this is the first large-scale, randomized, controlled study to investigate the use of telepsychiatry in the treatment of depression. We found no significant differences between in-person and telepsychiatric treatment in terms of symptom improvement, remission, treatment adherence, or patient satisfaction.

Over the 6-month treatment period, there was evidence of significant improvement in participants’ depression as measured by both interviewer-administered Hamilton depression scale scores and self-report Beck Depression Inventory scores, regardless of which type of treatment they received. The rates of clinical improvement, as measured by a decrease of at least 50% in the Hamilton depression scale score, and remission, as measured by a final 17-item Hamilton depression scale score of 7 or less, were not significantly different between the two groups. These results suggest that telepsychiatric treatment is comparable to in-person treatment in terms of symptom improvement.

However, in both groups the rates of improvement and remission were lower than those found in some other studies (25). The relatively low response rate in this study may have a number of possible explanations. Many patients in the VA Maryland Health Care System are treated for depression in primary care. It is our impression that patients who are perceived by primary care providers to be more difficult to treat, either initially or because they do not respond quickly to an initial dose of antidepressant, are referred for specialty mental health treatment. Thus, the patients in this study might have had more treatment-resistant depression, compared with patients in many other clinical trials. Furthermore, unlike most clinical trials, our study did not use medical comorbidity as an exclusion criterion. Therefore, it is possible that for some subjects, medical comorbidity contributed to treatment resistance. In fact, data indicate that primary care VA patients nationally have more medical comorbidity than non-VA primary care patients (26).

The patients in this study who were treated remotely were as adherent with treatment as the patients treated in person. Dropout rates did not differ between the two groups of patients. Also, no significant difference between treatment groups was found in the time course of the dropouts. Furthermore, there was no difference in the number of visits between groups, indicating that patients in the remote group were no more likely to miss their appointments than patients in the in-person group. Finally, pill counts indicated no difference in patients’ adherence with psychiatric medication treatment. Patients who received remote treatment were as satisfied with treatment as those treated in person. Both groups reported high levels of satisfaction. Although the psychiatrists in this study reported significantly more satisfaction with in-person than with remote treatment, on average psychiatrists reported high satisfaction with both conditions, so the clinical significance of the higher satisfaction with in-person treatment is probably minimal.

Although remote treatment was not associated with increased overall health care consumption, each remote visit was more expensive to the institution than each in-person visit. However, productivity gains of handling more remote patients per day improved the cost minimization of the intervention, given the costs that would have been incurred if psychiatrists traveled from the medical center to the clinic to provide in-person care. This cost savings from telepsychiatry first occurred if the psychiatrist had to travel more than 22 miles.

This study had several limitations. First, patients needed to come in to the clinic for treatment, so remote treatment did not offer them the convenience of participating in treatment from their home or a site close to their home. It is possible that patient satisfaction with remote treatment may have been even higher if patients had the benefit of being treated from home or a site close to home. A second limitation is that all subjects in both the remote and the in-person conditions had in-person contact with the research assistant at each visit. Although the nature of this contact was not explicitly therapeutic and was devoted to completing rating scales and handling administrative aspects of the study, it is still possible that the subjects received therapeutic benefit from these interactions. Thus, it is possible that without this in-person contact, subjects in the remote group might not have improved as much as they did. A third limitation was that the majority of the participants were male. Therefore, these findings may not be applicable to a female population. Future research should include a more gender-balanced study group. Fourth, the people who chose to be involved in this study may have been in some way different from those we approached who chose not to participate. Perhaps they were more comfortable with technology, more willing to try new things, or more willing to share information about themselves. Thus, these participants may have not been typical of the VA population as a whole. Finally, as with all research in which participants know they are being observed, patients’ self-reports may have been influenced by their awareness of being in a research study. Likewise, since the raters were not blind to treatment condition, their ratings might have been biased in favor of or against telepsychiatric treatment.

To our knowledge, this study is the first to demonstrate that symptom improvement and remission of depression treated by means of telepsychiatry is similar to improvement with in-person treatment. This finding could translate into increased access to psychiatric care for people who live at a great distance from treatment centers, without compromise of the quality of treatment. Psychiatrists practice mainly in large metropolitan areas, and there is a shortage of psychiatrists in rural and exurban regions. Using telepsychiatry from a central location, psychiatrists in metropolitan areas could provide treatment to patients in outlying areas at the clinically required frequency and intensity. In addition, telepsychiatry could be used to treat disabled or elderly people who cannot easily leave their homes. Finally, telepsychiatry could be used to treat patients who otherwise might be reluctant to come to a psychiatric office or clinic. For instance, patients with agoraphobia could be treated in their homes by means of telepsychiatry. In summary, the findings of this study suggest that telepsychiatry could be used to expand health care access without reducing patient satisfaction or adherence and without incurring high costs. At the same time, given the limitations noted earlier, more research is indicated to confirm our findings. In particular, further studies of efficacy are needed, especially given the relatively low response rates and remission rates found in both treatment groups in this study.

Preliminary results presented at the annual meeting of the American Association of Geriatric Psychiatry, New Orleans, March 13–16, 1999; the 18th annual meeting of the VA Health Services Research and Development Service, Washington, D.C., March 22–24, 2000; the 19th annual meeting of the VA Health Services Research and Development Service, Philadelphia, Feb. 14–16, 2001; and the 155th annual meeting of the American Psychiatric Association, Philadelphia, May 18–23, 2002. Received Jan. 23, 2002; revisions received July 24, 2002, and March 26, 2003; accepted Nov. 7, 2003. From the VA Capitol Network Mental Illness Research Education and Clinical Center, Baltimore; the VA Maryland Health Care System; the Departments of Psychiatry, Medicine, and Epidemiology and Preventive Medicine, University of Maryland School of Medicine, Baltimore; the Baltimore Research Education Foundation, Baltimore; the Capitol Network Health Services Research and Development Center and the Mood Disorders Program, Mental Health Service Line, VA Medical Center, Baltimore; Veterans Healthcare Administration Cooperative Studies Programs, Perry Point, Md.; Behavioral Health and Clinical Neurosciences Division, NW Hepatitis C Resource Center, Portland, Ore.; Mood Disorders Center, VA Medical Center, Portland, Ore.; and the Department of Psychiatry, Oregon Health Sciences University, Portland, Ore. Address reprint requests to Dr. Ruskin, Mental Health Service Line (116/MH), VA Maryland Health Care System, 10 North Greene St., Baltimore, MD 21201; [email protected] (e-mail). This material is based on work supported by the VA Office of Research and Development Health Services Research and Development Service, the VA Maryland Health Care System, and the VISN 5 Mental Illness Research, Education and Clinical Center. The authors thank Stacy Baron, Keri Roberts, Kimberly Douchis, David Campbell, Samuel Korobkin, Andrew Rhein, Nicole Valletto, and Tricia Witte for assistance with data collection; treating psychiatrists Arturo Monteiro, Martin Brandes, Eapen Abraham, and Erwin Bachmeister for participation in the study; Heather Grimsley and Brett South for assistance with data analysis; and Sharon Zielinski, Patricia Briggs, and Barbara Scroggins for assistance with computer technology.