Size Matters: The Unexpected Challenge of Detecting Linkage in Large Cohorts

This issue of the Journal features an article by Fernando S. Goes, M.D., et al. that presents the results of a familial aggregation and genetic linkage study of mood-incongruent psychotic features in bipolar disorder. The central hypothesis of the study is that individuals with mood-incongruent psychosis represent a genetically relevant subset of all individuals with bipolar disorder, a subset that may have particular genetic overlap with schizophrenia. The cohort used in the Goes et al. study was drawn from the National Institute of Mental Health Genetics Initiative Bipolar Disorder Collaborative project, which represents a set of several thousand subjects who were recruited and ascertained at 10 separate sites and who were assessed using three different versions of the Diagnostic Interview for Genetic Studies. All diagnostic data were recently reviewed, cleaned, and merged to provide a single database containing comparable diagnoses assigned with the highest degree of clinical confidence.

The results of this familial aggregation study provide statistically significant evidence that individuals with mood-incongruent psychosis are more likely to have family members with mood-incongruent psychosis than are individuals with mood-congruent psychotic features or no psychotic features, suggesting that this is in fact a genetically relevant subset of bipolar disorder. The results of the linkage study, however, are less clear-cut, despite using 644 carefully characterized families with 2,899 subjects available for genotyping. There are intriguing suggestive results that coincide with the location of previously identified schizophrenia susceptibility loci, adding some support to the hypothesis that there may be genetic overlap between schizophrenia and bipolar disorder with mood-incongruent psychotic features. But despite the large size of the cohort, no statistically significant linkage to a broad or narrow bipolar phenotype was found, regardless of whether or not mood-incongruent psychosis was included as a covariate in the analysis. To place this study in a broader context, it is one of several published in recent months that have used very large numbers of pedigrees (>400) for linkage analysis of schizophrenia or affective disorders that have failed to produce statistically significant results (1 – 3) . This raises the following question: If modest-sized cohorts of even 25 to 50 families can (and have) generate significant results for these disorders, why are not larger cohorts of more families better?

For oligogenic disorders, disorders that we suspect are caused by several genes acting together, it has been appreciated for a number of years that it is much easier to identify an initial significant linkage to any one of the susceptibility loci than it is to replicate significant linkage to a specific locus (4) . Even a model with a modest number of genes (six) is predicted to require replication cohorts on the order of five to six times the size of the original cohort. To collect such larger cohorts in a timely fashion, studies typically use multiple recruitment sites, a feature common to all of these recent very large linkage studies. Despite the best efforts of the investigators, unknown differences may exist between cohorts recruited at different locations. Simulation studies have demonstrated that pooling of data sets with unappreciated differences in locus heterogeneity can result in a loss of power to detect linkage with both parametric (5) and model-free (6) analysis methods. Simply increasing the absolute number of families exhibiting linkage to a given locus in the total data set may not lead to an increase in the overall linkage signal at that locus if the overall proportion of families exhibiting linkage decreases. The current tendency of failure to detect significant linkage to schizophrenia and affective disorders in this recent series of very large studies may simply be the observation in real data sets of this predicted result.

In clinical terms, we suspect that there are genetic subvariants of schizophrenia and bipolar disorder. In their article, Goes et al. use a well-known clinical distinction, the presence of mood-incongruent psychotic features in bipolar disorder, postulating that this clinical classification reflects a specific genetic etiology. While encouraging, the results were not as definitive as we would hope. Even if this clinical homogeneity does reflect some increased genetic similarity across subjects, it is possible that other uncontrolled sources of genetic variation were present in the cohort. Subtle differences in ethnic background might have been introduced by the use of sites across the country, with the potential for differences in the mix of genes that produces bipolar disorder with mood-incongruent psychotic features among different groups from our diverse population. Different clinics might also attract patients with different levels of illness, some focusing on subjects who are in long-term inpatient settings, with others recruiting more heavily from the community. Clinicians who are accustomed to seeing the wide spectrum of presentation of a single DSM-IV defined illness can appreciate how the breath of these definitions would be an important issue in genetic research.

Given this background, is there some useful framework for understanding the overall literature of linkage studies of psychiatric disorders? In retrospect, the statistical power analyses that informed the design of the earliest linkage studies were optimistic and based on models of few susceptibility loci of relatively strong effect. Most early genome scans would be considered underpowered based on today’s oligogenic models, so the failure of many of those studies to detect significant linkage is not unexpected. However, some of the early studies did identify significant linkages, and while not the typical or generally expected outcome, some of these are undoubtedly cases where being lucky does not mean being wrong. Some earlier studies may have benefited from the potentially greater impact of subject variation on a small sample variation. While a large, random cohort will tend to accurately reflect the population from which it is drawn, small cohorts can exhibit significant deviations by chance. If a relatively small percentage of individuals in the general population have an illness influenced by a specific gene, it will actually be easier to collect a group where the majority of cases are influenced by this gene when the cohort is small rather then large. This is the same principle that makes it much more common for a fair coin to come up heads at least 75% of the time in a series of four tosses than in a series of 40.

In an effort to move to the larger size cohorts that are predicted to be needed to replicate initial linkage findings (4) , it is possible that increased heterogeneity has been introduced into these larger cohorts, resulting in an actual reduction of power as the cohort size increases, if the cohort is analyzed in a traditional manner (5) . Fortunately, a new framework is emerging for appropriately analyzing such data sets without this loss of power, allowing for heterogeneity within subsets as well as differences across subsets (7) . This approach has produced novel linkage peaks when very large cohorts are analyzed as clinically relevant subsets instead of as one large cohort. For example, notable novel linkage peaks were detected for autism when a cohort of 303 families with children in the autism spectrum was analyzed by dividing the cohort into subsets based on diagnosis (autism versus Asperger’s syndrome or pervasive developmental disorder) and degree of delay of onset of phrase speech (8) . The challenge for psychiatrists interested in genetics is to identify the relevant features upon which to subset large cohorts.

A related issue raised by the history of linkage studies of psychiatric disorders is our current approach to the measurement of statistical evidence. A myriad of tests and statistics are currently used and reported. While it may be possible to rank order the level of support for different loci within a given study, making comparisons about the strength of evidence supporting loci across studies becomes more difficult. One welcomed tendency is the increased use of simulation studies to provide an empirical interpretation of the results of any given analysis package, as presented in the article by Goes et al. In addition to providing some safeguard against algorithms that may behave unexpectedly in unusual situations, simulations may provide the only feasible method to correct analyses for multiple nonindependent tests and produce a p value that may be evaluated for study-wide significance. Still, p values as a measure of evidence may not be the most useful metric for linkage studies. I would refer readers interested in considering an alternative framework for statistical evidence to a recent review by Vieland (9) that provides a cogent scientific argument for the need to develop a uniform and absolute measure of statistical evidence for linkage, presented in a manner accessible to nonstatisticians. This framework is the basis for the previously mentioned analysis approach that can merge multiple heterogeneous subsets of large cohorts without an accompanying decrease in power, already successfully applied to the analysis of autism. If unanticipated cohort heterogeneity is the cause of the somewhat surprising lack of significant linkage findings in the recent large studies of schizophrenia and affective disorder, then wider use of such an alternative analysis seems worth serious consideration.