Elsevier

NeuroImage

Volume 100, 15 October 2014, Pages 358-369
NeuroImage

Methodological considerations on tract-based spatial statistics (TBSS)

https://doi.org/10.1016/j.neuroimage.2014.06.021Get rights and content

Highlights

  • We investigate tract-based spatial statistics (TBSS) considering potential pitfalls.

  • TBSS is not tract-specific and we show how this may falsify results.

  • User defined parameters strongly influence the final TBSS-derived results.

  • We provide suggestions that improve the validity and increase the impact of TBSS.

Abstract

Having gained a tremendous amount of popularity since its introduction in 2006, tract-based spatial statistics (TBSS) can now be considered as the standard approach for voxel-based analysis (VBA) of diffusion tensor imaging (DTI) data. Aiming to improve the sensitivity, objectivity, and interpretability of multi-subject DTI studies, TBSS includes a skeletonization step that alleviates residual image misalignment and obviates the need for data smoothing. Although TBSS represents an elegant and user-friendly framework that tackles numerous concerns existing in conventional VBA methods, it has limitations of its own, some of which have already been detailed in recent literature. In this work, we present general methodological considerations on TBSS and report on pitfalls that have not been described previously. In particular, we have identified specific assumptions of TBSS that may not be satisfied under typical conditions. Moreover, we demonstrate that the existence of such violations can severely affect the reliability of TBSS results. With TBSS being used increasingly, it is of paramount importance to acquaint TBSS users with these concerns, such that a well-informed decision can be made as to whether and how to pursue a TBSS analysis. Finally, in addition to raising awareness by providing our new insights, we provide constructive suggestions that could improve the validity and increase the impact of TBSS drastically.

Introduction

Diffusion magnetic resonance imaging (MRI) can provide an insight into the living human brain in health and disease, especially in white matter anatomy, and provides quantitative parameters related to white matter (WM) microstructure (Tournier et al., 2011). Much of the knowledge about changes in WM microstructure that we have gained from diffusion MRI originates from studies that compared such diffusion markers between populations of interest, commonly a healthy control group and a diseased population. The value and impact of such studies is directly tied to the ability of researchers to present results that are unbiased, objective, and anatomically specific. Tract-based spatial statistics (TBSS) (Smith et al., 2006) has become a very popular tool for the evaluation of diffusion tensor imaging (DTI) data in this context.

TBSS pioneered the idea of projecting volumetric data onto a WM skeleton to circumvent the partial volume effect (PVE) and gain statistical power from this dimensionality reduction (Smith et al., 2006). The approach does not require data smoothing and could alleviate many concerns that were raised regarding the conventional voxel-based morphometry (VBM) framework that was previously used in many DTI studies (e.g., Jones et al. (2005)). Although TBSS has advanced the state of the art in diffusion MRI group studies significantly, the increased complexity resulting from adding the skeletonization step reduces overall transparency. In other words, while TBSS is very user-friendly and delivers comprehensive images, it may also obscure several aspects of the raw data that the reader of a study or even the researcher that performed the analysis might not be aware of. With more and more scientists adopting the technique, it is therefore increasingly important to raise awareness of the limitations of the approach. In previous studies, some problems related to TBSS have been investigated. Edden and Jones (2011) reported that the shape of the skeleton as well as the statistical results are rotationally variant. Zalesky (2011) quantitatively assessed the performance of the projection algorithm in moderating registration misalignments and showed that only 10% of post-registration misalignment was corrected by the TBSS projection algorithm. Keihaninejad et al. (2012) demonstrated the dependence of specificity and sensitivity of TBSS results on the registration target and suggest the use of a group-wise atlas as target. Van Hecke et al. (2010) discussed potential pitfalls and limitations of TBSS, like the assumption that the effect of interest occurs in voxels where the local FA is highest. Below we discuss important issues that we address in this study.

One major point of debate is the potentially limited anatomical specificity of TBSS. The technique was introduced as being “tract-based”, in response to the challenge of comparing voxels of “the same part of the same WM tract from each and every subject”, both “in terms of resolving topological variabilities and in terms of the exact alignment of the very fine structures present in such data” (Smith et al., 2006). However, making a distinction between adjacent, differently oriented fiber bundles with similar FA values is challenging and alternative methods are described by Kindlmann et al. (2007) and Yushkevich et al. (2008) to overcome this limitation. Since TBSS only makes use of the FA map and discards the orientation information captured in the diffusion data, two different problems arise. First, complications in terms of anatomical specificity occur in regions where pathways of different structures merge, such as those related to the superior projections of the corpus callosum (CC) and the corona radiata fiber bundles. Without the (long-distance) directional tract information derived from the orientation information, it is virtually impossible to assign the FA values to the same anatomical structure across subjects in a consistent way as the skeletonization step causes these different bundles to collapse on top of each other (see Fig. 1). Furthermore, even in regions where the assignment of voxels to tracts is unambiguous, the tract specificity of the TBSS projection step is unknown. The region where the cingulum bundle (CB) and CC are in close proximity is a good example of this, and in the original TBSS-paper, it has been explicitly stated that the CB and CC are correctly differentiated by the projection algorithm (Smith et al., 2006, page 1494, second paragraph): “The superior part of the cingulum (i.e. above the corpus callosum) is slightly extended across its cross-section in the inferior–superior direction, and well-localized across subjects by virtue of the strong, nearby corpus callosum, and hence the normal projections described above work well (similar issues relate to the fornix)”. However, this was not shown experimentally. Since we question the tract-specificity of TBSS throughout this paper, we do not use the words “tract-center” or “tract” when referring to the skeleton, but “locally maximal FA value”, or “FA-skeleton”, because we think this is a less ambiguous and thus more appropriate expression.

Another factor that plays a central role in the TBSS processing pipeline, and one that may greatly affect the anatomical specificity of TBSS, is the quality of image registration. The mean FA skeleton has been shown to be less “alignment-invariant” than anticipated and alternative skeleton-based approaches that try to address this point have been published, but have not yet reached a comparable level of acceptance (Kindlmann et al., 2007, Yushkevich et al., 2008, Zhang et al., 2010a).

A further point of debate is the robustness and interpretability of TBSS results. The original TBSS paper includes inter-subject and inter-session test–retest results regarding the reproducibility of FA values (Smith et al., 2006). However, the influence of the user in terms of parameter settings and the noise level on the final TBSS result, i.e. the significant maps, has not been shown. Being a fully automated technique, TBSS is generally considered to be largely user-independent. However, there are several parameters that have to be adopted in each TBSS analysis. While this is potentially important to allow a proper adaptation of the method to each specific analysis, many papers vary the parameters without motivating their choice. This is critical, since important aspects of the underlying data, such as SNR or alignment problems, remain unnoticed when looking only at the final result. We anticipate that the influence of different TBSS configuration options on the final result is largely unclear and/or underestimated by TBSS users. One important example is the choice of template in TBSS studies. Many studies use the FMRIB template that is distributed with TBSS. This might be mainly for computational reasons, since the generation of a study-specific target is computationally expensive to obtain, especially in larger populations. However, while the choice of template is known to significantly impact the results of other group analysis methods (Van Hecke et al., 2011), its impact on the final TBSS result is largely unknown. An initial study was performed by Keihaninejad et al. (2012), who demonstrated the positive impact of improved alignment on TBSS by the use of a group-wise atlas construction.

Taken together, although TBSS may provide plausible results, the final significance maps overlaid on the template image may also hide potential methodological imperfections related to the quality and/or analysis of the data. In this paper, a deeper look under the surface of the TBSS framework is provided. We address several methodological aspects of the technique: how unbiased, objective, and anatomically specific are TBSS results? What are major sources of bias, user-dependence, and non-specificity and to what extent do these factors affect the final TBSS result? With the detailed analyses presented in this study, we provide an in-depth investigation of the major pitfalls in analyzing and interpreting data with TBSS. We conclude with suggestions that define good practice when using TBSS and we propose improvements that may further raise the validity and impact of TBSS.

Section snippets

TBSS settings

In all experiments, the TBSS pipeline was applied using the recommended parameters. For the in-vivo datasets a permutation test with n = 5000, corrected for multiple comparisons and threshold-free cluster enhancement (TFCE (Smith and Nichols, 2009)) was used to compare patients and controls, with p = 0.05 as threshold for significance. Unless otherwise stated, an FA threshold of 0.2 was applied and the FMRIB58 template (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FMRIB58_FA) was used as registration

Influence of adjacent WM tracts

The results in this section were obtained from in-vivo dataset I/II. Fig. 2e/f shows examples of voxels misassigned from the CB to the CC and vice versa (white and black arrows). The contribution of one tract to the other is not binary, even on a voxel basis, since the registration and interpolation steps introduce a blurring of the binary segmentations. The blue arrows in Fig. 2e/f, for example, point to yellow voxels, where the original colors green and red overlap. Black skeleton voxels

Discussion

TBSS is by far the most popular approach for performing voxel-wise DTI analyses. It provides dedicated processing steps and deals with smoothing and misalignment issues in diffusion MRI-based group analysis studies. However, it also builds upon a certain set of assumptions that we have investigated in detail in this work. Most TBSS users are well informed about the major processing steps and well aware of their major weaknesses, such as the abandonment of directional information in the

Acknowledgments

This study was partly funded by the German Research Council (DFG, LA 2804/1-1) and the contribution of Chantal Tax is supported by an FC-EW grant (No. 612.001.104) from the Dutch Scientific Foundation (NWO). The authors would like to thank the members of the Utrecht Vascular Cognitive Impairment Study Group for providing the diffusion MRI data (dataset II): University Medical Center Utrecht, The Netherlands, Department of Neurology: E. van den Berg, G.J. Biessels, M. Brundel, W.H. Bouvy, S.M.

References (27)

Cited by (355)

  • A Twin Study of Altered White Matter Heritability in Youth With Autism Spectrum Disorder

    2024, Journal of the American Academy of Child and Adolescent Psychiatry
View all citing articles on Scopus
View full text