Technology and Mental Health: State of the Art for Assessment and Treatment

Technology is ubiquitous in society and now mediates many forms of interpersonal and societal communication. It is no surprise that the numbers of technology-based interventions and strategies for treating psychiatric disorders are rapidly increasing. These technologies include evaluation of nearly all features of psychiatric disorders, including symptoms, cognitive performance, and everyday functioning. In fact, while technology-based assessments and intervention strategies initially were administered in-person at office visits, many of these strategies are now administered remotely using cloud-based applications.

Current technology allows for the structured delivery of material used for assessment and training in cognitive, social cognitive, and functional domains; two-way communication with video, short message services (SMS), such as Twitter and other software platforms, including remote therapy applications; paging using various technologies for assessment and intervention purposes using ecological momentary assessment (EMA) strategies; continuous passive monitoring of location and behavior (including activity and physiological signals such as heart rate and skin conductance); and presentation of reality-based computer simulations. These simulations include both immersive virtual reality (VR) simulations and more static simulations that allow performance assessment and, in some cases, training on veridical representations of technology-based tasks such as shopping, banking, traveling, and placing online orders. The devices on which such applications are now delivered range from computers to tablets to smartphones to wearable devices. Thus, technology in the context of this review refers to an array of different functions (messaging, monitoring) across a number of different platforms and operating systems (Windows, iOS, and Android).

In this review, we describe technology relevant to mental health applications, including both assessment and intervention applications. In the assessment domain, we focus on assessment of cognitive abilities, emotion regulation capacity, functional skills, and clinical symptoms, including thorough sampling of individual symptoms and activities through structured queries or observed experiences. Assessment technologies involve observational strategies, including EMA, paging, and passive measurement, and cues to engage in performance-based assessments in cognitive, social cognitive, or functional domains. We generally focus here on adult populations, with some mention of interventions for attention deficit hyperactivity disorder (ADHD) in children that may also apply to adults.

In the treatment domain, we present information on applications designed to deliver interventions as well as applications that are designed to augment other treatments. Examples of direct treatment delivery include performance-based training in cognition and functional skills, which are available across conditions ranging from ADHD to mild cognitive impairment to substance use disorders. Other therapeutic applications include immersive VR simulations as well as cognitive-behavioral therapy applications. Technology-based augmentation strategies include tools for self-monitoring between therapy sessions, delivery of reminders to reinforce therapeutic goals, and various ways to track adherence to treatments. This leads to a very broad-based review, which itself is only a shadow of the field of mental health assessment and treatment and technological interfaces.

Our review of these applications and technologies includes data on their efficacy (when they are employed as treatments or assessment tools) as well as data on user tolerability. Any disparities between clinical trial results obtained with digital health technologies and outcomes arising from traditional in-person clinical trials require reconciliation and interpretation, and are likely related to factors of 1) real-world engagement challenges for patients and 2) workflow challenges for clinicians (1, 2). Considering data on real-world effectiveness beyond just efficacy data will be critical to ensure that the field makes optimal use of emerging technologies (3).

Technology-Based Assessment of Cognition and Everyday Functioning

Cognitive Assessments

Computerized cognitive assessment strategies have been used for several decades. Multiple testing batteries are currently available, and these have been reviewed in detail elsewhere (4–6). Computerized assessments are appealing for several reasons, including systematic delivery of instructions and collection of responses, as well as automated scoring and norming of response data.

Computerized cognitive testing has been used in multiple clinical trials, and its use in routine clinical practice is also becoming more common. Certain tests have always been available exclusively in a fully computerized format (7). Other assessments, initially developed for administration using paper and pencil (e.g., the Brief Assessment of Cognition [8] and the Wechsler Intelligence Scales) were subsequently released as computerized applications (9, 10) developed to be convergent with the widely used paper versions. A significant advantage offered by many of these computer-based assessment tools is that the tester does not have to be a licensed professional; indeed, subprofessional clinical trainees can acquire the skills required to administer most computerized assessments and collect valid data. However, evaluation of whether results from computer-based cognitive assessments are convergent with the results of traditional in-person assessments remains an important consideration (11). Some recent data suggest substantial challenges with certain tasks, particularly if there is an attempt to sustain fidelity to paper-and-pencil assessments while performing a remote assessment (12). As a result, there is a need for careful consideration of whether all legacy cognitive assessments can be performed remotely.

Functional Capacity Assessments

A variety of computer-based strategies examine the ability to perform skills that are critical for everyday functioning, referred to as functional capacity. Available assessment tools evaluate performance on a range of tasks through structured simulations of everyday activities, veridical simulations of everyday tasks, and VR-based simulations. The stand-alone task batteries usually have a structured assessment sequence with individually scorable tasks and are normable in a manner similar to that applied to responses on neuropsychological tests. While the VR assessments (described below) are commonly more realistic and more flexible, they are, in many cases, less amenable to normative standards. In the stand-alone tests, such as the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) (13), touch-screen responses are used to assess the subject’s ability to perform a sequence of skilled acts using simulation formats (e.g., looking in cabinets for specific target items, developing a shopping list, utilizing public transportation, and navigating a shopping experience in a virtual retail store). In another, the computerized functional skills assessment and training system (CFSAT) (14), the specific components of skilled acts are examined, such as entering a personal identification number on an automatic teller machine screen, selecting the correct ticket choice on a computer kiosk, and using the keypad on a simulated mobile phone to enter responses to a telephone voice menu. Data have consistently shown that performance on these computerized simulations of everyday activities is correlated with cognitive function (12, 13) measured with standard or digital strategies. These assessments have a variety of functions, including use in clinical trials of cognitive enhancement where evidence of functional relevance is required and in clinical settings to directly measure improvements in functional skills in individuals receiving rehabilitative interventions (15, 16).

Remote Delivery of Technology-Based Cognitive and Functional Assessments

As the assessment technology reviewed here is already available for either remote or in-person assessment settings, we briefly address the feasibility of remote delivery of cognitive assessments. Several different formats are used for remote assessment, including tester administration of tasks over a videoconferencing application and remote, exclusively self-administration of all assessments by the subject. There are several challenges inherent in each approach. For exclusively remote, self-administered assessments, the participant needs to be comfortable with, and capable of using, the required technology.

Videoconference administration of tests that were designed to be administered in-person requires consideration of the technological demands of the conferencing application and the ability of participants to use the technology as well as to perform the critical skills. It is certainly possible to perform certain types of cognitive assessments over the telephone (e.g., measurement of verbal responses in tests of working memory or episodic memory, and measurement of auditory processing speed on tasks such as the Oral Trail Making Task [17]). For videoconferencing applications in cases where the participant is asked to perform cognitive tests on the device while simultaneously receiving remote instructions and supervision, the participant need only be able to manage the technology-based delivery of the assessment program, which can possibly be facilitated by another person who is on-site with the participant at the time of testing (18). These challenges may be difficult to eliminate entirely.

In the case of assessments designed for fully remote self-administration, there are other potential challenges. Several such studies have found significantly more missing data than observed with in-person, paper-and-pencil assessments (19, 20). One possibility is that some participants, particularly those with severe mental illness or other forms of cognitive impairment, find the computerized assessments difficult to comprehend and/or are less motivated, particularly without another person present to receive instructions and facilitate subject engagement. We recently validated methods for remote delivery of neurocognition (21) and social cognition (i.e., emotion recognition) (22) testing embedded in an EMA application. Participants were seen in person and trained on the use of the technology at the start of participation. In a sample that demonstrated the baseline ability to utilize this technology, we found that subsequent adherence to the EMA cognitive assessments (75% for neurocognition [N=168] and 80% for emotion recognition [N=86]) was high and data quality was on average excellent. Adherence was not correlated with diagnosis (major depression, bipolar disorder, schizophrenia), age, sex, or presence of psychosis, negative symptoms, or suicidal ideation. Although these data are quite positive, strategies for determination of an individual’s capacity to be assessed remotely seems to be an important clinical topic.

Clinical Virtual Reality

Over the past 25 years, researchers and clinicians have pursued the use of VR as a tool to advance clinical assessment, intervention, and scientific research (23–31). This effort was inspired by the intuitively obvious opportunity for VR environments to address specific challenges inherent in the provision of usual clinical strategies for mental health, rehabilitation, and general medical care. At its core, VR technology, along with other related simulation-based formats (e.g., augmented/mixed reality), offers new capabilities that did not exist a decade ago. Many recently developed VR-based testing, training, teaching, and treatment approaches would be difficult, if not impossible, to deliver without leveraging the power of modern computing, three-dimensional (3D) graphics, body tracking sensors, novel user interfaces, gaming/narrative principles, big data analytics, and artificial intelligence. Such VR-enabling technologies allow for the creation of highly realistic, interactive, engaging, and systematically controllable digital simulation environments. Users can be immersed in VR simulations and interact with content for the purposes of clinical assessment and intervention. VR technology is thus well matched to the requirements of various clinical targets and psychiatric contexts.

Defining Virtual Reality

Since the inception of VR, a large and evolving scientific literature has emerged regarding the outcomes and effects associated with what we now refer to as clinical VR applications that target psychological, cognitive, motor, and functional impairments or symptoms across a wide range of health conditions. Continuing advances in the underlying enabling technologies for creating and delivering VR applications have resulted in their widespread availability as consumer products, sometimes at a very low cost (e.g., Oculus Quest 2).

The concept and definition of VR has been debated over the years. VR has been very generally defined as a way to visualize, manipulate, and interact with technology and complex data in a more naturalistic and intuitive manner (32). From this baseline perspective, VR can be seen as an advanced form of human-computer interaction that allows a user to interact with computers beyond what is typically afforded by standard mouse–keyboard–touchscreen interface devices. An engaged VR user experience can be created through unique combinations of interaction devices, sensory display systems, and the type of content presented in the virtual environment. Thus, there are two common types of VR. The automated observation of these interactions constitutes the assessment components of VR therapies.

Nonimmersive VR is the most basic format and is similar to the experience of playing a video game. Virtual content is delivered on a standard computer monitor, tablet, mobile phone, or television as users interact with 3D computer graphics using a game pad, joystick, mouse, keyboard, or specialized interface devices (e.g., other handheld devices, data gloves, treadmills). Modern computer games that support user interaction and navigation within 3D graphics can be considered to be VR environments. Tasks such as the VRFCAT described above are nonimmersive VR assessment strategies.

Immersive VR integrates head-mounted displays, body-tracking sensors, specialized interface devices, and 3D graphics (33). Users operate within a simulated environment that changes in a natural or intuitive way based on the user’s motion and interaction. The head-mounted display occludes the user’s view of the outside world while head- and body-tracking technology senses the user’s position and movement. These user movement data are rapidly sent to a computing system, which uses the movement and interaction data to update the sensory stimuli, which are presented to the user via the head-mounted display. When users are immersed in computer-generated visual imagery and sounds of a simulated virtual scene, their interaction produces an experience that corresponds to what they would see and hear if the scene were real.

Regardless of the technical approach, the key aim of these immersive systems is to perceptually replace the outside world with the virtual world to psychologically engage users with simulated digital content designed to create a specific user experience. Immersive VR is typically the choice for applications where a controlled stimulus environment is desirable for constraining a user’s perceptual experience to a synthetic world. This format has been often used in clinical VR applications for assessment of anxiety disorder or PTSD severity and subsequent exposure therapy, as distraction for patients undergoing acutely painful medical procedures, and in the physical/cognitive assessment/rehabilitation domain. The research potential—for example, studying neural processes during brain imaging or neurosurgery—are also clear.

In a related domain, recent work has involved the creation of virtual human characters (sometimes called avatars or autonomous agents) that allow users to engage in clinical interactions within both nonimmersive and immersive simulations. The creation of virtual humans has evolved from research showing their clinical usefulness as stimuli for exposure therapy for social phobias (34, 35), for role-play training for social skills in people on the autism spectrum (36–38), for activities for addressing intimate partner violence (39), and for teaching self-compassion in persons with depression (40). More complex virtual humans infused with varying levels of natural language processing and artificial intelligence have shown effectiveness in the role of virtual patients that novice clinicians can use in practicing the skills required for challenging diagnostic interviews (41) and motivational interviewing (42). They have also been created to produce online virtual human health care guides (43, 44) and as clinical interviewers, with automated sensing of facial, gestural, and vocal behaviors that are useful for inferring the state of the user interacting with these virtual human entities (45) and for assessing clinician empathetic behavior (46).

Current VR Clinical Treatment Areas

The field of clinical VR has expanded dramatically as the technology has evolved. Clinical VR has been shown to be effective in fear reduction in persons with specific phobias (e.g., 47, 48), treatment for posttraumatic stress disorder (e.g., 49–52), and cue exposure for addiction treatment and relapse prevention (53–55). VR has also been effective in treating depression (40), paranoid delusions (56), and body image disturbances in patients with eating disorders (27, 28). Cognitive and physical rehabilitation research using VR has produced promising results when applied to navigation and spatial training in children and adults with motor impairments (57), functional skill training and motor rehabilitation in patients with central nervous system dysfunction (e.g., stroke, traumatic brain injury, cerebral palsy, multiple sclerosis) (58), and for the rehabilitation of attention, memory, spatial skills, and other cognitive functions (59, 60).

VR Assets for Advancing Clinical Interventions: Expose, Distract, Motivate, Measure, and Engage

On a very general level, VR leverages core processes that are relevant across a variety of clinical domains. These processes can be summarized as the capacity to expose, distract, motivate, measure, and engage users. Expose refers to clinical applications designed to provide exposure therapy for anxiety disorders and PTSD, to practice social interactions in order to reduce paranoid delusions, and cue-exposure approaches for addiction treatment and relapse prevention. VR used for exposure therapy offers strong evidence, with a recent meta-analysis suggesting efficacy across a wide range of phobias, anxiety, and trauma- and stressor-related disorders (61). Further meta-analysis supports the efficacy of virtual reality for anxiety-related disorders, although the research base is still relatively small (62). Distract refers to methods for distracting attention from painful medical procedures to reduce pain perception, promote reduction of discomfort, and provide respite from bleak hospital settings. A recent systematic review supported the potential of VR for pain management but noted that the studied effects are often for acute pain, and less is known about longitudinal analgesia for chronic pain (63). Motivate refers to the practice of promoting patient adherence to repetitive and sometimes boring or frustrating training tasks that need to be performed for cognitive or physical rehabilitation and chronic pain management by embedding these activities within game-like contexts. Measure underscores the capability that VR simulations provide for quantifying activity and/or performance in response to controlled simulations of fearful experiences. Finally, engage is generally seen as the end result of capturing attention or action that is useful for encouraging participation with clinical applications where users relate to and interact with virtual content as if it were physically real—sometimes referred to as the sense of presence (64). For example, learning the “skill” of achieving a “mindful” state typically requires multiple sessions before the user perceives a rewarding change in their mental or emotional state. VR has been used to create engaging experiences within which users may be more compelled to practice and learn this skill.

The Future of Clinical VR

Scientific support for the clinical use of VR for mental health and rehabilitation has evolved as the costs and complexity of developing VR applications have gone down and the capacity of the technology has increased. A complex VR headset and hand controllers that might have cost tens of thousands of dollars in 2000 now cost well under $800. This trend should be accelerated by recent developments in “standalone” VR headsets (e.g., Oculus Quest, Pico Neo, Vive Focus, etc.). Such low-cost VR display systems do not require a tethered computer, as all the graphic and interaction processing take place onboard the device. These lower-cost devices will promote adoption and enable larger-scale clinical studies that can help build the effectiveness data necessary for VR to build out a solid evidence base for guiding future clinical implementation. As we look to the future, we also see growing clinician awareness, acceptance, and adoption of clinical VR methods. For example, Norcross et al. (65) surveyed 70 psychotherapy experts regarding interventions they predicted to increase in the next decade; VR was ranked 4th out of 45 options, with other computer-supported methods (teletherapy, mobile apps, online cognitive-behavioral therapy self-help) occupying three of the other top five positions. Moreover, the COVID-19 crisis has certainly accelerated the exploration and acceptance of these technologies to amplify access to care, and that interest will likely continue after the pandemic has passed (66). Thus, in view of the current enthusiasm for VR generally across society, and specifically in the clinical community, coupled with emerging scientific support and lower system costs, it is likely that clinical VR applications have the potential to become standard tools for psychiatry researchers and possibly to be utilized more widely by practitioners.

Ecological Momentary Assessment (EMA)

Ecological momentary assessment (EMA), also referred to as the experience sampling method (ESM), has been a tool for understanding fluctuating phenomena and within-person dynamics, and the ubiquity of the smartphone has greatly accelerated the accessibility of this method for clinical applications (67). Programs for the delivery of EMA surveys have become more widely available, and the tools for analysis of intensive longitudinal data have proliferated. At the earlier stages of EMA, the focus was typically on the recording of behaviors (e.g., activity, sleep, smoking) or daily life experiences, such as stressors, through diaries (68–70). The data gathered enabled examination of within-person change, but required user input and did little to reduce the biases inherent to self-report (70). These older assessment strategies had no way to accurately time-stamp the reports that were collected. Anecdotal reports of people arriving 20 minutes early for their appointments and completing 14 days’ worth of assessments are confirmed by the results of research studies comparing reported and observed adherence to paper diary assessments (71).

Personal digital assistants such as early Palm Pilot–like devices automated these processes (72, 73). Moreover, prompts to complete surveys could now be timed and momentary responses could be time-stamped (74). With the translation of EMA to smartphones, surveys could be delivered according to different contexts experienced by the individual (and indexed by the geolocation features of the device), thus enabling more personalized information to be gathered. The ability to tailor probes based on the individual’s momentary state generated a new field of ecological momentary intervention (75), and several trials have evaluated personalized automated interventions that leverage momentary data (76, 77). Some researchers have moved beyond self-reports to intensively repeated objective measures, including brief cognitive tests embedded in the EMA programs as described above (21, 22).

The kinds of questions that researchers have been able to ask with these new tools have led to new insights in fundamental questions in mental health. Sometimes these findings are at odds with prevailing theories. It is commonly believed that smokers relapse because of nicotine withdrawal symptoms. Shiffman et al. (78) evaluated smoking behavior in non-daily smokers and found that negative affect was more important than withdrawal symptoms in relapse, which is critical for understanding which factors to target to sustain smoking cessation. It is commonly believed that suicidal ideation arises from feelings of hopelessness. Kleiman et al. (79) found that suicidal thoughts varied markedly throughout the day and that variation in candidate predictors (e.g., hopelessness) did not predict the emergence of this ideation, a finding that had been produced previously in a hospitalized sample (80). Depp et al. (81) found that social isolation and number of social interactions did not predict onset of suicidal ideation in people with schizophrenia, but that the anticipation of being alone later was associated with an increase in ideation. Granholm et al. (82) found that people with schizophrenia (N=100) spent considerably more time home and alone than healthy control subjects (N=71) and, even when home and alone, engaged in fewer productive behaviors. In a follow-up analysis of this sample, Strassnig et al. (83) found that people with schizophrenia reported fewer activities, spent considerably more time sitting and less time standing, and were considerably more likely to sleep during the daytime hours. However, listening to music and watching television were not differentially common in healthy and schizophrenia participants, suggesting that activities less productive than passive recreation are among the things that were more common in participants with schizophrenia.

More general lifespan questions can also be addressed by EMA. Using a measurement burst design in which bouts of EMA are integrated with a longitudinal follow-up period, Koffer et al. (84) found that older age was associated with greater ability to buffer against the effect of stress on affect.

These are just a few examples from a burgeoning field, highlighting the degree to which active EMA paradigms can be used to advance understanding of the dynamic processes underlying psychiatric diagnoses, extending and sometimes challenging prevailing theories. EMA is a useful strategy to identify targeted features of different conditions on a momentary basis. For example, repeated assessment can identify the proportion of prompts that are answered at home versus away and in the presence of other people versus alone. As these are the central indices of social isolation and social avoidance, the socially relevant impact of negative symptoms in schizophrenia (85) and current depression in mood disorders can be directly indexed. Research suggests excellent correlations between clinical ratings of symptoms from structured interviews and EMA data, while identifying fluctuations in symptoms that are missed by more widely spaced assessments (86, 87). These strategies can also be used to examine health-relevant behaviors in mental health populations, as described above. Given the reduced life expectancy associated with severe mental illness and the high prevalence of metabolic syndrome, EMA can be used to estimate the amount of time spent sitting versus standing or otherwise engaged in active behaviors. Given that contemporary EMA can collect the occurrence of multiple different activities since the last survey, it is quite easy to see whether only one activity has occurred since the last survey or whether participants are engaging in multiple concurrent activities, including physical activities (88). When paired with the passive digital phenotyping described below, a comprehensive EMA assessment can examine location and social context, refine measurements of activity (exercise vs. agitation), detect sleeping during the daytime and not at night, and assess concurrent subjective emotional responses to these activities.

Passive Digital Phenotyping

A more recent breakthrough involves quantifying clinical outcomes using “passive” digital phenotyping (i.e., unobtrusively collecting data via the internal sensors of a smartphone, a wrist-worn smart band, or another device). Passive measures can reduce certain limitations associated with interview- and questionnaire-based clinical assessments (e.g., cognitive impairment, social desirability, cultural biases [89]). Numerous passive measures have been evaluated in psychiatric populations (e.g., geolocation, accelerometry, ambient speech recorded from the environment, phone call and text logs, screen on/off time, social media activity, Bluetooth-based proximity social sensing) (90–96). However, the validity of these passive measures is only beginning to be established. Goldsack et al. (97) proposed the V3 framework for determining the validity of passive digital biomarkers, which involves three components: verification, analytical validation, and clinical validation. These components, as reviewed below, provide a useful heuristic for determining whether the level of validity achieved for various passive measures meets clinical standards.

The first component of the V3 model, verification (i.e., efficacy), is a quality-control step for the device of interest that is performed by the manufacturer. It occurs before testing is conducted on human subjects. The goal is to determine whether the sensor captures data accurately and to verify that the software accurately outputs data within a predetermined range of values. For example, accelerometry could be verified by placing a smart band on an object programmed to accelerate at a prespecified rate. Verification is typically done by device/software manufacturers against a reference standard. However, the results of these tests and the analytic methods supporting the devices are typically not published or made available for evaluation, which presents replication challenges. Additionally, common standards do not exist for verifying passive digital phenotyping sensors of interest, and sensors embedded in different models will often be different. Since devices and sensors may require differing levels of verification (e.g., required accuracy) for various clinical purposes, evaluating verification data is a critical step that should occur before passive digital phenotyping measures are applied in studies in clinical populations. For medical devices, such as medical decision-making software, this process may be handled by the U.S. Food and Drug Administration (FDA) as part of Good Manufacturing Practice (GMP) standards. Making test results and analytic methods underlying devices accessible to researchers will help disentangle whether failures of replication are true problems with reproducibility across clinical populations or simply differences in the technical quality of different devices used in studies.

The second component, analytical validation (i.e., effectiveness), involves behavioral or physiological validation of a device in human subjects in the real world. A key first step in this process is determining whether sample-level data output by the device is properly received and that algorithms calculated on that data perform as expected. The metric resulting from the algorithm, applied in real time or post hoc, should be evaluated against a relevant reference. Although agreed-upon reference standards have not been determined for validating passive digital phenotyping measures, there has been initial analytical validation of some passive measures. For example, phone-based geolocation and accelerometry recorded on the ExpoApp have been validated in relation to a reference wrist-worn actigraph and a travel/activity diary; time in microenvironments and physical activity from the diary demonstrated high agreement with phone-based geolocation and accelerometry measures (98). Huang and Onnela (92) analytically validated a phone accelerometer and gyroscope using a ground-truth standard. They had human participants engage in specific physical activities (e.g., sitting, standing, walking, and ascending and descending stairs) with a phone in their front and back pockets. Behavior was filmed throughout as an objective reference. The sensors accurately predicted video-recorded behavior in the reference standard. One ongoing challenge is that as smartphones are updated with new software and phone models with new sensors, prior validation efforts cannot be assumed to be valid.

The third component, clinical validation (i.e., implementation), involves determining whether the passive digital phenotyping variable of interest adequately predicts a specific clinical outcome within the population of interest. Preliminary evidence for clinical validation exists for several passive measures—although at times results have also been contradictory (99). For example, in bipolar disorder, incipient depressive symptoms have been predicted by changes in the number of outgoing text messages, the duration of incoming phone calls, geolocation-based mobility measures, and vocal features extracted during phone calls. Manic symptoms of bipolar disorder have been predicted by more outgoing texts and calls, acoustic properties of speech extracted during phone calls (e.g., standard deviation of pitch), and increased movement detected via accelerometry (100, 101). Clinically elevated and subthreshold depressive symptoms have been predicted by geolocation-derived measures of circadian rhythm, normalized entropy, and location variance, as well as phone usage frequency and speech-derived audio volume (102–105). Social anxiety has been predicted by reduced movement on accelerometry and fewer outgoing calls and texts (106). Relapse of psychotic disorders has been predicted by geolocation mobility metrics and text/call behavior (90). Negative symptoms of schizophrenia measured via EMA or clinical ratings have been predicted by geolocation-based mobility metrics, voice activity, and actigraphy-based metrics of gesture and activity level (99, 107–110). Combining passive measures with EMA surveys may further enhance clinical validation. For example, Raugh et al. (111) found that the combination of geolocation and EMA surveys was a stronger predictor of clinically rated negative symptoms in schizophrenia than either measure alone. Similarly, Faurholt-Jepsen et al. (101) found that combining vocal acoustic features extracted from phone calls with EMA reports improved the correct classification of mixed or manic mood states in bipolar disorder beyond either measure alone. Henson et al. (112) reported that a combination of EMA and passive data, when analyzed for congruence with anomaly detection methods, was associated with early warnings of relapse in people with schizophrenia. Thus, studies suggest that passive measures are promising tools for measuring clinical outcomes. However, there are numerous inconsistencies regarding the predictive value of specific metrics and measures for classifying individual disorders or symptom states, including geolocation, accelerometry, ambient speech, and ambulatory psychophysiology (113–116). For example, clinical data on sleep did not match sensor report in one study (94), and results are not comparable across studies because of differences in sensors utilized, in the clinical targets, in time frames for calculating associations across assessment modalities (e.g., daily or monthly), and in the populations studied. There are also fundamental differences across studies in methods and analyses, such as controlling for multiple comparisons when examining correlational data.

Clinical validation (i.e., implementation) is of particular concern for using passive measures as outcomes for clinical interventions. Unlike traditional interview- or questionnaire-based clinical outcome measures, standards for the level of psychometric evidence needed to say that a measure is clinically validated have not yet been determined for passive digital phenotyping. Proprietary data collection via devices (e.g., a custom wearable device [117]) and proprietary methods for analysis (e.g., a custom machine learning algorithm [118]) offer both innovation and a challenge to reproducible clinical research. Further complexity arises from the trend toward using more complex analytic methods with passive digital phenotyping because of the multilevel nature of the data. For example, machine learning is an increasingly common tool in the clinical validation process, and studies have employed various algorithms to predict a range of clinical outcomes (e.g., classification, regression, unsupervised clustering) (119). However, common standards for judging the level of psychometric evidence that constitutes clinical validation for machine learning are not yet uniformly applied across the field. Is predictive accuracy of 70% enough to declare clinical validation, or should a higher standard be set (e.g., 90% accuracy) (104, 106)? Similar considerations affect simpler analytic methods, such as simple correlations for passive data aggregated across a range of time (e.g., 1 week) to form a single value that can be correlated with clinical outcomes. It seems important that such aggregated values be adjusted for the extent of daily or time of day variation. These adjusted correlations tend to be statistically significant but lower (r values ∼0.3–0.5) than typical standards for convergent validity that would be applied within clinical rating scales or questionnaires (e.g., r values >0.80) (103, 104, 111). Do these lower correlations reflect inadequate convergent validity, even though they are statistically significant? Or is the lower correlation to be expected (and therefore acceptable) because of the fact that it averages across differences in temporal variation across measures or method variance? We suggest that common guidelines for judging what constitutes clinical validation are clearly needed for passive digital phenotyping. There should also be an effort to ensure that clinical validation studies include a representative sample with diverse individuals to ensure that algorithms are not primarily trained to be accurate in populations whose demographic and personal characteristics do not overlap with the clinical populations of interest and that methodological and analytic approaches are valid and consistent throughout the population.

Feasibility of implementation is the next consideration, and barriers and facilitators such as cost, accessibility, tolerability, ease of use, and data failure rates are among the relevant factors. Few studies have evaluated user experience of interactions with passive measures. However, qualitative studies employing interviews designed to assess patient perceptions have indicated that while many see these technologies as holding promise for clinical detection and self-management, there may also be unintended barriers to use, such as increased stigma or anxiety (120, 121). One would expect that most passive measures would not be viewed by participants as burdensome, given that they are collected unobtrusively by the background sensors of their device and do not require direct participant action. However, there may be some instances where device interface proves problematic in clinical populations. For example, in a study on outpatients with chronic schizophrenia, the participants had considerable difficulty with remedying Bluetooth unpairing of a smart band and smartphone (112). People with schizophrenia found this pairing issue more burdensome than did control subjects. It is also unclear whether certain clinical symptoms interact with the willingness to consent to participating in digital phenotyping studies. For example, by their nature, continuous geolocation and ambient speech monitoring raise questions about privacy and agency. It is unclear whether clinical populations, such as individuals with schizophrenia who have delusions of suspicion, experience such technologies as intrusive and whether they exacerbate symptoms or result in the individual not consenting to participate out of fear of being monitored. Some data suggest that the prevalence of answering prompts while acknowledging concurrent psychotic symptoms is reasonably high (86) and that EMA reports of location have been validated using GPS coordinates (108). More generally, issues of systemic racism and mistrust of how passive digital phenotyping information could be (mis)used by the law enforcement or other systems of power may influence implementation of these methods in participants who are racial minorities. Thus, user experience should be carefully evaluated when administering these technologies in clinical populations. As we mention below, the general issue of access to the Internet and experience with any technology is a barrier that will need continuous attention.

Combinations of EMA and passive digital phenotyping seem likely to improve interventions and assessment. GPS location coordinates provide information about where one is, but not who is with them. Proximity detection can determine whether another individual with a device is present, and ambient sound sampling can tell whether individuals are interacting or are simply in proximity to each other. Smart bands can detect activity but not the motivation for the activity (exercise vs. agitation). Combining mood sampling with geolocation information and EMA can help determine whether social isolation is due to depression or lack of motivation, and facial and vocal affect assessment from participant-captured samples can provide validation information for mood reports. A recent example (122) suggested that the combination of passive phenotyping and EMA prompts was feasible, with multiple different prompted responses collected, in conjunction with data regarding location, psychophysiological responses, and ambulatory acoustics (44 participants with schizophrenia and 19 with bipolar disorder). Thus, an array of different elements of functioning can be captured simultaneously and used to generate a wide-ranging picture of momentary functioning.

A challenge in the domain of passive digital phenotyping is that application developers and scientific utilizers are commonly at the mercy of the manufacturers, who can restrict access to phone features for applications or push out operating system upgrades that cause software to fail. Further, applications that monitor access to social media may also encounter restricted access or requirements that access be granted for each time the application attempts to capture data. This is an area where collaboration with manufacturers will be required.

Adherence Monitoring

One of the major approaches using technology in mental health treatment is in adjuncts to therapies. In particular, treatment adherence monitoring is a clear area of need and has been a focus of both clinical trials and clinical treatments. For example, several studies have used mobile monitoring to check in with patients at high risk for nonadherence, including early-course psychosis patients (123) and patients with bipolar disorder (124). Some applications have been approved as medical devices by the FDA. For example, an application that monitors adherence to aripiprazole was approved by the FDA in 2017 (125) and involves an embedded sensor in a pill. Other strategies used in clinical trials include the use of digital photography to capture the moment of pill taking or other chemical tags that can be detected after a medication is taken (126). Systematic studies are under way to examine the usefulness of these strategies for real-world adherence support.

One of the issues with adherence monitoring is that this cannot be a passive measurement strategy. Individuals whose adherence is monitored, in either research or clinical treatment, need to be fully informed and to agree to this monitoring, and their consent must be valid.

Smartphone Therapeutic Applications

There are many mobile apps designed around the principles of cognitive-behavioral therapy (CBT) or other evidence-based interventions but few randomized controlled trials demonstrating their efficacy for any disorder. There are a huge number of applications available that attempt to promote mindfulness or induce relaxation. As many of these applications are not tested empirically at all, we have focused on the translation of CBT strategies into applications. As CBT has a long history of being systematically manualized, the comparison of efficacy of applications to legacy in-person treatment is facilitated. The majority of existing data on the efficacy of mobile app-based interventions comes from randomized controlled trials assessing symptoms associated with a defined disorder or other mental health outcomes such as stress levels, well-being, and quality of life. There are some data to suggest that smartphone interventions can be effective in reducing depressive symptoms. A 2017 meta-analysis by Firth et al. (127) identified a small number of randomized controlled trials (N=18) that examined the efficacy of smartphone-based interventions in improving symptoms of depression. They found a significant reduction in depressive symptoms with smartphone interventions compared with waiting list or inactive control conditions (g=0.56) and a smaller effect in comparison to active control conditions (g=0.22). The use of interventions based on cognitive-behavioral techniques offered greater benefits for depression than computerized cognitive training applications. In a 2019 meta-analysis of randomized controlled trials assessing the efficacy of app-supported smartphone interventions for mental health disorders, Linardon et al. (128) found that smartphone interventions significantly outperformed control conditions in improving depressive symptoms. Similar to the Firth et al. meta-analysis, the effect size was larger when waiting list (g=0.32) or informational resources (g=0.39) were used as control conditions compared with attention or placebo control conditions, such as checking the weather on the phone (g=0.12). Of the 54 comparisons (smartphone vs. control) analyzed, 26 involved a CBT-based app; however, a subgroup analysis did not show them to be associated with larger effect sizes. CBT is an empirically and meta-analytically supported treatment for depression, but some researchers have suggested a low level of adherence to the core principles of CBT models and identified highly variable usability among CBT-based smartphone interventions as reasons for their lack of superiority (129). A 2021 review of studies of CBT smartphone apps for depression featuring a control group reported that results remain too heterogeneous to recommend for front-line care (130).

Similarly, a small but growing body of data suggest that smartphone interventions may be efficacious in the treatment of anxiety symptoms. Another meta-analysis from 2017 (131), focused on randomized controlled trials involving smartphone-supported interventions to reduce anxiety symptoms and found significantly greater reductions in anxiety scores from smartphone interventions compared with control conditions across nine eligible randomized controlled trials. Effect sizes were significantly greater when studies made use of a waiting-list or inactive control conditions (g=0.45) compared with those that used active control conditions (g=0.19). This discrepancy in effect sizes—like that seen in studies assessing depressive symptoms, as noted above—suggests the complexity of conducting digital mental health research and the possibility of a digital placebo effect by which use of a digital device in itself confers a degree of psychological benefit. The Linardon et al. meta-analysis (128) found 29 studies assessing efficacy in treating generalized anxiety symptoms, with eight studies specifically designed to target generalized anxiety symptoms. Across the 39 comparisons within the identified studies, the pooled effect size (g) was found to be 0.30 and statistically significant across all sensitivity analyses. Subgroup analyses again showed a smaller effect size for comparisons using an active comparison intervention (g=0.09) and a larger effect size with studies that used a CBT-based app, which included 16 of the 39 comparisons analyzed.

An intervention strategy that combines EMA principles with interactive smartphone technology is referred to as “just-in-time adaptive interventions” (132). These strategies involve consistent monitoring of behavior, activities, moods, and symptoms, using EMA strategies, but they also interactively offer interventions in real time. An example of such a strategy is the FOCUS intervention (133), which uses a mix of prompts directed to the participant and self-activated tools. The goal of this class of interventions is to sustain engagement while offering interventions in real time. As noted in several reviews, this strategy is being widely used, but the data are not yet at the stage where global statements about efficacy can be made.

Two other examples of smartphone-based interventions that have FDA-approved elements are recently introduced devices to promote smoking cessation and to reduce opioid abuse. PIVOT is a digital smoking cessation app that includes human coaching with text messages, combined with smartphone-based carbon monoxide (CO) monitoring (134). The CO sensor is an FDA-cleared medical device, and the program includes a multistage intervention following standard human-delivered smoking cessation strategies as well as nicotine supplementation.

R-Set-O (135) is an application that is designed to be paired with buprenorphine treatment for opioid addition. In a randomized clinical trial, 82% of participants who were randomized to the device remained in treatment, compared with 68% of those in treatment as usual.

Abstinence was also higher in the active treatment group (77% vs. 63%). Given the typical attrition rates for opioid use disorder treatment (about 50% or more) (136), these are encouraging results. One of the challenges in these interventions is adherence and engagement. For example, in a naturalistic study of the R-Set-O intervention in which data from 3,144 individuals with opioid use disorder were evaluated, 80% completed at least eight of the 67 possible therapeutic modules, 66% completed half of all modules, and 49% completed all modules (137). Although abstinence rates were quite good (about 65%), there is a clear difference in adherence compared with the randomized controlled trials that led to FDA approval. In a large-scale review of device-based interventions, Linardon and Fuller-Tyszkiewicz (128) reported that adherence was challenging in many of these interventions. The types of strategies that succeeded in increasing adherence were clearly associated with attempts to promote engagement at the outset of the intervention. Interventions that used online enrollment were particularly susceptible to poor adherence and dropout, while in-person and telephone recruitment strategies were better.

Computerized Cognitive Training and Cognitive Remediation Therapy

The core of computerized cognitive training (CCT) is software designed to engage and practice cognitive functions. Cognitive functioning is commonly defined in these applications as the set of abilities that would be measured with neuropsychological assessments and relate consistently to everyday functional outcomes. Some programs are explicitly aimed at a single cognitive domain, while others target an array of domains. A central feature of successful CCT programs is adaptive presentation of training stimuli, such that the level of difficulty tracks the participant’s current performance. The goal is to train increasingly more difficult tasks while ensuring a success rate of about 80% of the target stimuli.

Computerized cognitive training has been widely studied in the past two decades, along with concurrent advances in computer technology, which has allowed for great strides forward in terms of control over the learning environment. Multiple studies have demonstrated CCT’s efficacy for improvement of cognition in multiple populations, with the bulk of the evidence in severe mental illness (138, 139) and supported by large-scale studies of healthy older people (140–142). There is considerably less information outside of schizophrenia, but studies in bipolar disorder (143) and major depression (144) have been published. For evaluation of CCT as a mental health treatment, there are several central considerations. These include the range of efficacy expected, how the intervention needs to be delivered, the dose required, and whether there are specific subpopulations who stand to make the most treatment gains. Further, there are several considerations about concurrent treatments that may be required to translate cognitive gains into improvements in everyday functioning. Finally, remote delivery of cognitive training has been studied in the past with some success.

As described in the meta-analyses, cognitive changes induced with CCT have generally been shown to have minimal efficacy for the improvement of everyday functioning in the absence of a targeted intervention aimed at functional skills. When CCT is combined with structured intervention programs, the term cognitive remediation therapy (CRT) is generally applied. CRT has been shown in meta-analyses to produce both cognitive and functional gains (139). There are multiple approaches to delivering CRT, but they all share common features. The intervention is delivered in person by a trainer, and other skills training is delivered as well, typically with a focus on vocational or social functioning. CCT combined with supported employment programs has proven in multiple studies to provide considerable benefits (e.g., 145), even in previous nonresponders (146). Hence, when delivered in a structured CRT program, the range of expected efficacy includes cognition and everyday functioning. Some studies have also trained social cognitive abilities, leading to improved social outcomes (15), and some have found that combined CCT and computerized social cognitive training (CSCT) lead to more substantial gains than CCT alone (147). However, a recent study using compensatory cognitive training combined with supported employment did not find employment gains (148).

Dosing of CCT has varied considerably across studies. In studies of severe mental illness, doses ranging from 15 to 135 training sessions have been delivered. One factor that may mediate the effect of dose is the extent of training engagement. Several studies have suggested that training engagement predicts the extent of training gains in CCT (149, 150). Even large doses of CCT may be ineffective if participants are not actually participating in the procedure (151). Thus, monitoring of engagement, easily accomplished through the software in most training programs, is clearly recommended. There are insufficient data to draw conclusions regarding the likelihood that training engagement will either improve in poorly engaged patients or worsen in those who are initially engaged.

In terms of specific subpopulations with potential to benefit from CRT, prodromal (152), first-episode (153), and chronic (154) schizophrenia patients show equivalent cognitive gains when trained with a single CCT system. In a reanalysis of a larger randomized trial, patients with a shorter illness duration had a greater cognitive and functional response to a comprehensive CRT program (155). In contrast, several studies of patients with extended institutional stays (156, 157) have suggested that benefits are common and include both cognitive and functional improvements. Similarly, in mood disorders, patients with a history of major depression and treatment resistance, both older and younger, have received benefits from CRT (158, 159). Thus, there are no clear indicators for illness characteristics that define which patients will achieve maximum benefit. Engagement has a much stronger signal than age in research to date.

Some rehabilitation facilities may not have access to computers for all participants, and some participants may prefer to train at home. Although the majority of structured CRT has been studied with in-person training, several studies suggest that home-based CCT can be accomplished with reasonable levels of adherence (70%) and with cognitive and social cognitive benefits (15, 152, 153). Train-at-home studies with nonpsychotic community-based populations have also been conducted (160). A sample of 2,912 older community dwellers participated in an entirely online training program, with evidence of gains in both composite scores on cognitive performance and everyday functioning. The dropout rate was considerably larger than seen in the studies noted above, with rates exceeding 50% at the 6-month follow-up.

A treatment for ADHD recently approved by the FDA, EndeavorRx (generic name, AKL-T01), also used a train-at-home performance-based training intervention (161). In a large-scale trial, AKL-T01 was delivered to children with ADHD in a video game–like interface via at-home play for 25 minutes per day, 5 days per week for 4 weeks. The outcome measure was performance on an ADHD-relevant cognitive task, the Test of Variables of Attention (TOVA) (162). The treatment was significantly superior to a video game control condition. AKL-T01 thus received approval to improve attention function as measured by computer-based testing in children ages 8–12 with primarily inattentive or combined-type ADHD who have a demonstrated attention issue. The sponsors of the treatment clearly state that it is designed to be used as augmentation therapy in addition to other treatments. One reason for this suggestion is that scores on ADHD rating scales did not show improvement after training. Thus, this intervention is very similar in terms of strategy to previous studies using CRT to improve cognitive functioning in other conditions, such as schizophrenia.

Pharmacological Augmentation of CRT

An important recent development been systematic studies of pharmacological augmentation of cognitive training (163). These augmentation strategies have been found to be successful for the use of stimulants (164), guanfacine (165), alertness promotion agents (166), and memantine (167) in schizophrenia, and for vortioxetine in age-related cognitive decline (168). Interestingly, modafinil and memantine have been much less effective as monotherapies for cognitive impairments (169, 170). Other studies are examining compounds that have shown preliminary efficacy as monotherapy treatment to improve cognition in schizophrenia (171) and as an adjunct to CRT (172), and these therapies may have promise as augmentations.

An additional important recent finding in the area of pharmacological augmentation of cognitive training is that of the combination of long-acting injectable antipsychotic medications and CRT. In a study of first-episode patients randomized to either oral or long-acting medications as well as to either CRT or another augmented psychosocial intervention, an important interaction effect was found (173). The combination of long-acting medication and CRT led to considerably greater cognitive gains than seen with CRT and oral medications. Further, the cognitive changes directly translated into functional gains, including work function. As this intervention also included vocational rehabilitation for all participants, the effects of cognitive gains associated with CRT on work outcomes in more chronic patients was reproduced. This is, to our knowledge, the first study demonstrating that clinical stability may be a factor that is associated with the efficacy of CRT.

Level of Evidence and Approval

As digital mental health technologies evolve, so do questions regarding the level of evidence to support their claims of efficacy. In response, some have proposed that digital health technologies may benefit from alternative endpoints and novel study designs in order to best capture their efficacy (174). The FDA’s Digital Health Software Precertification (Pre-Cert) Pilot Program is an attempt to reenvision how it approves such technologies (175), although questions remain about the real-world practicality of this approach, given that it remains a pilot project. In short, Pre-Cert seeks to expedite approval of software as a medical device through preapproving technology developers and using real-world data to assess the performance of the software after approval. Still, as noted above, there are smartphones apps, computer programs, and devices that have all been granted FDA marketing approval through more traditional pathways (section 510(k) and de novo) and trial designs.

There are other levels of FDA clearance for approvable technology. Under FDA guidelines, many technology-based interventions are viewed as general wellness applications, and this is outside the focus of regulation. These interventions include technology targeting adherence to scheduled therapeutic activities other than the act of taking medication, exercise, and certain elements of everyday functioning. These applications are generally similar to CCT applications in that they are not aimed at diagnosing and treating a disease. The FDA explicitly states that devices can be exempt from review because “when not associated with the diagnosis, cure, mitigation, treatment, or prevention of a disease the claim falls outside the scope of the definition of a medical device” (176, p. 8). Further, the FDA has a process referred to as enforcement discretion, in which the FDA considers the device to be a medical device but does not require that it receive a formal approval for use. Example 1 in the FDA guidance states that they do not intend to require approval for:

Thus, devices that do not attempt to replace an approved treatment or attempt to eliminate the need for medical care fall under this heading. Clearance of technology under general wellness applications or medical devices that fall under enforcement discretion are not likely to be eligible for direct insurance reimbursement, although they could be part of other bundled services. While in many clinical settings this would not be relevant because therapeutic activities are not billed on a session-by-session basis, in some practice settings this would be more of a challenge. For example, computerized cognitive training is covered by some insurance plans for neurological conditions, such as persistent traumatic brain injury, but not for psychotic or depressive disorders. Similarly, certain adherence applications have been approved by the FDA, but they are linked only to a single medication because the software actually detects the presence of a chip that is ingested along with the pill. Finally, as noted above, the AKL-T01 application for ADHD was approved only as an adjunctive treatment, not a stand-alone.

As regulation seeks to catch up to the mental health technology space, clinicians and patients must make choices today. Various frameworks have been proposed for such evaluation, including one endorsed by APA (177). Several score-based databases have also emerged, although research suggests low rates of concordance between such scoring systems as well as an inability to update at the rate of technology changes (178). Newer educational initiatives offer to help patients and clinicians make informed decisions based on available data (179). The Federal Trade Commission (FTC) continues to sue technology vendors for false marketing claims (notably Lumosity, in relation to brain training in 2016 [180] and a menstrual period tracking app in 2021 [181]) and offers consumer guidance as well.

Conclusions

Technology-based assessment and intervention strategies are proliferating, and the COVID-19 pandemic has accelerated the process. These strategies are based on technology that is newly developed and continuing to evolve. Technological strategies are likely to allow for expansion of clinical assessment and intervention potential and for clinicians’ ability to deliver more service in the same time frame. Even purportedly nontechnological interventions involve technology today, including electronic health records and video conferences, but this review addresses some of the ways that technology will continue to expand in the immediate future.

Development is faster than validation, and advertisements are less expensive than research. A reasonable idea would be to consider the evidence for using applications, keeping in mind that exaggerated claims are common in the technology area. While some of these claims have been the target of investigations by the FTC, the more common challenge is applications that are marketed without extravagant claims but also without adequate data. As a field, we need to develop our standards for what we utilize now and what we wait until later for.

There are several issues to follow into the future. One is the research–clinical deployment gap. Clearly, many technologies are well validated in research settings but are not as actively used in the clinic. Over time, this situation can change; the case for CCT is a perfect example: the advent of better computer technology and the feasibility of remote administration of training has enabled the expansion of general community access to CCT. This process may have been kicked off by CCT providers who made exaggerated efficacy claims, as described above, but the result is that the general community is quite aware of CCT now.

Another critical issue is access. While both age and socioeconomic status used to be barriers to technology access, many more older people have access to the Internet and use smartphone technology. The lack of access on the part of lower-income and rural populations was clearly highlighted during the COVID-19 pandemic, and until the access disparity is resolved, many people will not be reachable with these interventions. Importantly, these are the same factors that create access barriers to mental services in general; given the promise of technology increasing access to mental health services, increasing access to technology will be a critical first step.

In summary, these technological developments are exciting, and they show efficacy in controlled studies and are increasingly designed to be acceptable to patients. There is likely more to come in this broad area, and assessments and interventions that would have seemed like science fiction in the past are entirely commonplace now.