Meta Analyses SeriesGraphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial
Introduction
For several medical questions of interest, many treatment options exist for the same indication, and these treatments may have been compared against each other and against placebo/no treatment in various clinical trials. Knowing whether one specific treatment is better than placebo or some other specific comparator is only a fragment of the big picture. Ideally, one would like to know how all the different options rank against each other and how big the differences are in effect size between all the available options. Multiple-treatment meta-analysis (MTM) offers a quantitative method of integrating all the data from all the available comparisons [1], [2], [3]. The methodology and its caveats have been extensively discussed in the statistical literature [4], [5], [6], [7], [8]. MTM applications using a variety of relevant methods appear increasingly in major medical journals on influential medical topics [9], [10], [11], [12], [13], [14]. However, the presentation and interpretation of MTM results remain challenging, especially when there are numerous competing treatments.
As MTM methodology becomes popular, one needs efficient and clinically relevant ways to present the results. Until now, results from such analyses are usually presented with display of all pairwise odds ratios or using a common reference treatment. Effect sizes and uncertainty are important for clinical purposes, and their use is recommended for interpreting results when two treatments are compared. However, for MTM, these effect sizes may be difficult to interpret and use for decision-making purposes, because a large number of such effect sizes is generated for each MTM. It would, thus, be useful to provide alternative summary displays of the data that would be more readily interpretable. Decision making often requires probabilities. Some important questions on probabilities for decision making are as follows: What is the probability that each treatment is the best? What is the probability that each treatment is among the n best options? What is the probability that a specific treatment has an effect that is within an acceptable range of the best treatment?
In some cases, it may suffice to know what the best treatment is. One can then obtain the “probability that each treatment is the best.” However, this may not convey the entire picture. For example, the most effective treatment may be unavailable (e.g., in some developing countries), difficult to apply (e.g., many physicians and medical centers may not be familiar with its use), too expensive, or associated with serious toxicity that outweighs its benefits. Moreover, a treatment may have low probability to be the best but very high probability of being second or third best. Also, two treatments may have similar probabilities to be the best but may have very different ranking thereafter. One would like to see, therefore, what the other alternatives are and how they rank. Furthermore, one would often like to know whether these alternatives still have a high probability of being within an acceptable range from the best in terms of effectiveness. Appropriate and comprehensive presentation of MTM results might informally facilitate decision making and assist in trading effectiveness against applicability, cost, or toxicity.
In this article, we aim to discuss simple graphical and quantitative ways to facilitate presentation and interpretation of results from MTM. We first describe a data set as reported and analyzed by the authors of the original publication. In the Methods section, we present in detail quantitative summaries that can be used to complement the conventional way of presentation using summary effect sizes. We show how to obtain and present results on ranking of all treatments and how to appraise the overall ranks and probabilities of being within an acceptable range. We advocate that Bayesian methodology is the natural way to fit MTM models, as it enables a natural and easy estimation of all measures based on probabilities, ranks, or predictions. In the Results section, the methodology is exemplified by reanalyzing the published network. Finally, we discuss how more informative presentation of the results could improve clarity and interpretation.
Section snippets
Data set
We selected a previously published network of interventions with readily available data that generated some discussion and commentaries on publication [15]. It is a network of antiplatelet regimens given after transient ischemic attack or stroke in preventing serious vascular events. The authors found that “the most powerful antiplatelet regimen […] is the combination of aspirin and dipyridamole,” outperforming placebo, aspirin, thienopyridines, and their combination, after summarizing data
Multiple-treatment meta-analysis model for binary outcomes
The idea underlying MTM methodology is that, for any given comparison between two treatments A and B, direct evidence (coming from studies directly comparing A to B) and indirect evidence (coming from combining studies through an intermediate comparator, e.g., A vs. C and B vs. C studies) can be synthesized into a single effect size. The idea is generalized for a network of comparisons where multiple sources of indirect evidence are synthesized achieving maximum precision for the resulting
Software
The Bayesian model presented earlier is fitted using the WinBUGS software (Medical Research Council UK; http://www.mrc-bsu.cam.ac.uk/bugs) [17]. A version of the code used can be found in https://www.bris.ac.uk/cobm/research/mpes/mixed-treatment-comparisons.html. Additional pieces of the code to estimate the presented numerical summaries, the S-plus software code used to create barplots and cumulative ranking curves and to estimate SUCRA and the data can be found in //www.dhe.med.uoi.gr/software.htm
Numerical and graphical summaries of multiple-treatment meta-analysis results based on pairwise comparisons of effects
We reanalyzed the network for serious vascular events with antiplatelet regimens. Conventional presentation of results in terms of mean odds ratios and 95% credible intervals compared with placebo are summarized in Table 2. These are the means of the random effects distribution. The median (to convey possible skewness) common heterogeneity τ and its 95% credible interval are 0.03 (0, 0.13). Additionally, we estimated all pairwise odds ratios and their respective credible intervals presented by
Discussion
Consideration of many treatments in meta-analysis is becoming increasingly popular, as statisticians, clinicians, and researchers try to consider together and understand the results from multiple trials and comparisons on the same topic. However, the methodology of MTM still remains the realm of expert statisticians, as it requires specialized software and careful consideration of the assumptions underlying a joint analysis of the treatment network. Interpretation of the results from MTM
References (24)
- et al.
A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered
J Clin Epidemiol
(2009) - et al.
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis
Lancet
(2009) - et al.
Immunogenicity and adverse events of avian influenza A H5N1 vaccine in healthy adults: multiple-treatments meta-analysis
Lancet Infect Dis
(2009) - et al.
Outcomes associated with drug-eluting and bare-metal stents: a collaborative network meta-analysis
Lancet
(2007) - et al.
Methods to explain the clinical significance of health status measures
Mayo Clin Proc
(2002) - et al.
Incident diabetes in clinical trials of antihypertensive drugs
Lancet
(2007) - et al.
Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis
Lancet
(2007) - et al.
Evaluation of networks of randomized trials
Stat Methods Med Res
(2008) - et al.
Simultaneous comparison of multiple treatments: combining direct and indirect evidence
BMJ
(2005) - et al.
Combination of direct and indirect evidence in mixed treatment comparisons
Stat Med
(2004)
Assessing evidence inconsistency in mixed treatment comparisons
J Am Stat Assoc
The transitive fallacy for randomized trials: if A bests B and B bests C in separate trials, is A better than C?
BMC Med Res Methodol
Cited by (2531)
Network meta-analysis of the effects of different cognitive trainings on the cognitive function of patients with mild cognitive impairment
2024, Journal of Psychiatric ResearchEfficacy and safety of first-line treatment for metastatic triple-negative breast cancer: A network meta-analysis
2024, Cancer Pathogenesis and TherapyDiagnostic accuracy of tests for tuberculous pericarditis: A network meta-analysis
2024, Indian Journal of Tuberculosis