Statistics from Altmetric.com
The Cochrane Library is a readily accessible and frequently updated source of evidence-based summaries to guide care practices. Over the past 15 years, clinicians and consumers have collaborated to generate several hundred Cochrane reviews of a range of therapies relevant to pregnancy and perinatal health. These reviews have influenced practice in several key areas. Randomised controlled trials of perinatal healthcare interventions are difficult to undertake and consequently often underpowered to exclude modest effects. Cochrane reviews that synthesise data from several trials allow a more precise estimation of effect size. This has proved particularly important in defining the effect of infrequent, though clinically important long-term outcomes. The results of Cochrane reviews have also been used to inform future research strategies, some of which have generated new evidence from large trials. Future challenges for the Cochrane Collaboration included developing more user-friendly interfaces and extending the relevance of reviews to perinatal care in middle- and low-income settings.
“It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials” (Archie Cochrane 1979)1
In response to Archie Cochrane’s challenge, systematic summaries of all available randomised controlled trials relevant to pregnancy, childbirth, and care of the newborn infant were collated in two seminal textbooks: Effective care in pregnancy and childbirth (published in 1989) and Effective care of the newborn infant (published in 1992).2 3 These pioneering efforts were the forerunners of the Cochrane Database of Systematic Reviews – an electronic library of systematically prepared, regularly updated, high-quality summaries of evidence of the effects of interventions on clinically relevant outcomes. The Cochrane Pregnancy and Childbirth Group and Cochrane Neonatal Group were among the first established entities within the Cochrane Collaboration. Since the mid-1990s, hundreds of clinicians and consumers have collaborated to summarise, synthesise and disseminate evidence to inform care practices for pregnant women, their infants, and families (fig 1).
What is a Cochrane review?
Clinicians and consumers who face an overload of information about the effects of treatments need summaries (reviews) to help answer clinical questions or develop practice guidelines. However, traditional narrative reviews in journals or textbooks are subject to various biases and often delay recommending effective interventions or continue to promote potentially harmful or ineffective care.4 Cochrane reviews differ from narrative reviews in several key ways:
A protocol specifying the precise clinical question (participants, interventions and outcomes) as well as the search strategy, inclusion and exclusion criteria, and analysis plan are peer reviewed and published before the review can proceed.
The reviewers undertake to minimise bias by identifying and appraising all potentially eligible studies including unpublished trials, those published in abstract form only, and those published in languages other than English.
The methodological features of the included trials are evaluated critically.
Where possible and appropriate, data synthesis in the form of a meta-analysis is performed to increase the precision of the overall estimate of effect size.
Cochrane reviewers undertake to update their reviews at least every 2 years or more frequently if new trials are found.
Electronic publication facilitates regular revision of reviews in response to new data or to feedback from clinicians and consumers.
How have Cochrane reviews influenced practice?
Timely acknowledgement of effective interventions
The potential for systematic cumulative synthesis of research to influence perinatal clinical practice is illustrated by the history of the adoption of antenatal corticosteroids as a standard of care for women with impending preterm delivery. Following preclinical studies by Liggins and colleagues, randomised controlled trials of antenatal corticosteroids undertaken during the 1970s and 1980s provided consistent evidence of benefit for preterm infants. Despite this, most women with impending preterm delivery during that era did not receive antenatal corticosteroids and further placebo-controlled trials continued to be approved, funded and undertaken. A systematic review published in 1990 confirmed that antenatal corticosteroids substantially reduced neonatal mortality and morbidity without increasing the rate of adverse maternal outcomes.5 The strength of these findings prompted unequivocal recommendations by various national bodies that all women with impending preterm delivery should receive antenatal treatment with corticosteroids.6 This led to dramatic changes in practice – in the Vermont–Oxford Network, antenatal corticosteroid exposure in very low birthweight infants increased from 23.8% in 1991 to 71.6% in 1999.7 If a system for cumulative research synthesis had existed earlier, antenatal corticosteroids may have been adopted as a standard of care long before the mid-1990s.8
More precise estimates of effect size
Surfactant replacement therapy, the second major advance in the management of respiratory distress syndrome and its complications in preterm infants, was rapidly adopted into clinical practice during the 1990s for a variety of reasons: the clear rationale for treatment, the strong evidence from individual trials, and the wide participation of many neonatal centres in the trials that demonstrated its beneficial effects. Systematic overviews of surfactant therapy confirm the effect of surfactant therapy in reducing the risk of morbidity and mortality.9 However, subtle yet important issues of surfactant use such as optimal dosage and timing of administration have been less easily addressed and are still debated.10 These uncertainties have been difficult to resolve because very large trials are needed to exclude very much smaller effect sizes than the trials that simply compared surfactant versus no surfactant therapy. Data synthesis has played an important role in improving the precision of estimates of effect sizes. For example, the trials of prophylactic versus selective surfactant administration have not consistently found statistically significant effects on major outcomes. The Cochrane review, by systematically reviewing and conducting a meta-analysis of all the available data, was able to provide evidence that prophylactic surfactant replacement improved outcomes for very preterm infants considered to be at high risk of respiratory distress syndrome (fig 2).11 Cochrane reviews have also helped to inform research to refine the evidence base for surfactant replacement therapies, for example, highlighting the need to re-examine the role of prophylactic or early surfactant replacement in the context of increasing early use of non-invasive ventilation modalities in extremely preterm infants.12
Abandonment of ineffective treatments
Cochrane reviews have been useful in identifying interventions that are ineffective or harmful. For example, preclinical research studies suggested that antenatal thyrotrophin-releasing hormone might act synergistically with corticosteroids to reduce the risk of respiratory distress syndrome in preterm infants. Despite the biological plausibility of this treatment, plus evidence of effect in animal models and encouraging findings from some trials, the Cochrane systematic review did not show any improvement in important outcomes but did find evidence of important adverse effects for mothers and infants.13 The potential for bias in the review was further reduced because the reviewers identified and included data previously reported in abstract form only and analysed data using “intention-to treat” principles. In part as a result of this review, thyrotrophin-releasing hormone is no longer regarded as an appropriate intervention for clinical practice or further trials.
Focus on all important outcomes
In the mid-1990s, postnatal corticosteroid therapy was commonly prescribed for preterm infants to prevent lung inflammation and injury. Evidence existed that corticosteroids had important short-term benefits such as facilitating earlier endotracheal extubation and reducing the requirement for oxygen supplementation during the neonatal period. However, individual trials of postnatal corticosteroid therapy had generally not been designed or powered to assess long-term outcomes. In the late 1990s, Cochrane reviews that pooled data from several trials found evidence that postnatal corticosteroids, particularly if given in the first week after birth, increased the risk of adverse long-term neurodevelopmental outcomes including cerebral palsy.14 These findings prompted national policymakers to issue strong statements advising against the routine use of systemic corticosteroids for the prevention or treatment of chronic lung disease in very low birthweight infants.15 The use of postnatal corticosteroids has decreased dramatically since the findings of the Cochrane reviews were disseminated.16 17 18 In the Vermont–Oxford Network, postnatal corticosteroid use for very low birthweight infants fell from a peak of 29% in 1997 to ⩽8% by 2005 (fig 3). These Cochrane reviews highlighted the dangers of relying solely on assessments of short-term or surrogate outcomes and demonstrated the need for trials to be powered and structured to assess patient-important long-term outcomes like neurodevelopment.
More recently, Ohlsson and colleagues have systematically reviewed the evidence for the effect of early erythropoietin therapy for preterm infants.19 This Cochrane review found that early erythropoietin treatment had short-term benefits (reduced need for blood transfusion), but that these were of limited clinical importance. Moreover, erythropoietin increased the risk of severe retinopathy of prematurity. Individually, the trials had found inconsistent evidence of an effect on retinopathy of prematurity, but their findings were generally not statistically significant because the trials were underpowered to detect important effect sizes on this relatively uncommon outcome. By combining the data in a meta-analysis, the reviewers were able to estimate that one extra case of severe retinopathy of prematurity would occur in every 20 infants treated with early erythropoietin; a level of harm that for most clinicians and patients outweighs any beneficial effects. It is not yet clear how dissemination of this finding is influencing policy and practice.
Rapid appraisal and synthesis of new evidence
Several trials, including large multicentre trials, of mild systemic hypothermia for reducing delayed neuronal damage in term newborn infants with hypoxic-ischaemic encephalopathy, have been undertaken. Recruitment for such trials is difficult; although hypoxic-ischaemic encephalopathy is an important cause of morbidity and mortality, these cases occur unexpectedly and any given centre will only have a small number of cases. Although limited by relatively small sample sizes, some trials found statistically significant reductions in rates of mortality and neurodisability. However, it is difficult to interpret the data in light of the imprecision of the estimates. The Cochrane review of all the available data provides a more precise estimate of effect on mortality and neurodisability.20 The meta-analyses do indicate benefit, but the reviewers have urged caution since there is a need to complete ongoing trials and long-term follow-up. This review will allow for rapid incorporation of new data from the ongoing trials and allow caregivers to use the most precise estimates of effect in decisions regarding introduction of this new therapy.
“No evidence was found”
“Important research implications are more likely to arise from reviews with uncertain findings or where the benefits and risks are mixed” (Caroline Crowther 2006)21
Many Cochrane reviews conclude that, following an extensive literature search and appraisal, “insufficient trial evidence was found to guide clinical practice”. Although clinicians and patients may find this frustrating, acknowledging clinical uncertainty and thereby informing the research agenda are key components of the evidence-based approach to healthcare. Collaborative quality improvement initiatives such as the WOMBAT collaboration in Australasia (see: http://www.wombatcollaboration.net/) use Cochrane reviews to identify important gaps in the evidence in order to promote and support high-quality randomised clinical trials in the perinatal area. Several recent large randomised controlled trials in perinatal medicine have been undertaken when Cochrane reviews had highlighted important areas of clinical uncertainty. Examples relevant to pregnancy and childbirth management strategies have been well described.21 Recent examples of neonatal interventions include:
The “Benefits of Oxygen Saturation Targeting” (BOOST) and “Pulse Oximetry Saturation Trial for Prevention of Retinopathy of Prematurity” (POST ROP) large, pragmatic, randomised controlled trials to resolve the uncertainty surrounding the most appropriate levels of oxygen haemoglobin saturation targets in preterm infants, were informed by the findings of Cochrane reviews which had highlighted the paucity of high-level evidence needed to guide practice.22
The “Caffeine for Apnoea of Prematurity” (CAP) trial was developed by Schmidt and colleagues because Cochrane reviews had identified insufficient evidence that this widely used intervention caused more benefit than harm.23 Existing trials were small and focused solely on short-term outcomes despite the plausible possibility that caffeine could have adverse effects on neurodevelopment.24 The CAP trial therefore assessed both short-term effects and long-term growth and neurodevelopmental outcomes, and provided for the first time robust evidence that prophylactic caffeine has substantial important beneficial effects for preterm infants without competing harmful effects.25
It is unlikely that future interventions, or modifications of current practices, will have the same major beneficial effects for preterm infants that the introduction of antenatal steroids and exogenous surfactants have had. Each of these interventions reduces mortality by about 40%. Trials large enough to detect much more modest (but clinically important) effects will need to be multicentre and often multinational. However, inter-regional and international differences in research funding and administration can often disrupt the timely development and advancement of such trials. These logistical problems can be obviated partly by investigators in separate networks making a priori agreements about the design and conduct of their individual trials. By agreeing to be broadly consistent with regard to the eligibility criteria for participants, the nature of the interventions, and the outcomes assessment processes, prospective meta-analyses can be planned to increase the overall power of related trials to detect meaningful effects. This approach has been adopted by the POST ROP/BOOST2 collaborative network that is undertaking trials of lower versus higher oxygen saturation targeting in extremely preterm infants in Europe, North America and Australasia.26
The collaborative prospective meta-analysis approach, where trials are no longer regarded in isolation but as part of a wider research effort, has additional advantages:
Clinicians, consumers, ethics approval committees, and funders may support and participate in trials that previously would have been considered underpowered to address a research question.27
Researchers will also be encouraged to avoid assessing inappropriate surrogate or composite outcomes simply to justify sample size expectations.28
If an individual trial’s data monitoring committee is considering whether to cease a trial early because an interim analysis found benefit or harm, then an opportunity exists to cross-check with other monitoring committees thereby avoiding the pitfall of drawing spurious conclusions due to the play of chance.29
Getting evidence into policy and practice
Busy clinicians do not always or consistently access clinical guidelines developed using the highest-quality evidence. Benchmarking and audit studies in perinatal networks frequently reveal marked variation in practice, even when good evidence for specific interventions exists.30 Various strategies, for example, audit and intensive feedback, and the use of local opinion leaders, have been shown to help bridge these gaps between evidence, policy and practice.31 32 The relative effect of these strategies is greatest in settings with low levels of adherence to recommended practice.
For example, despite strong evidence that prophylactic administration of surfactant reduces mortality and respiratory morbidity in very preterm infants, benchmarking studies within the Vermont–Oxford Network in 2000 found that many infants still received delayed treatment.33 The network undertook a cluster randomised trial of a multifaceted intervention that included introducing a guideline via an opinion leader, organising training workshops for quality improvement and audit, and giving performance feedback and ongoing support. At the end of the study period, significantly more infants in the intervention hospitals received prophylactic rather than delayed surfactant.34 In the UK, networks of clinicians and consumers supported by patient advocacy agencies are planning similar initiatives.35
Future challenges and opportunities
Although the Cochrane Collaboration has achieved much during its first 15 years, it faces substantial ongoing challenges to maintain and increase its relevance and impact on clinical care. Several opportunities for improvement and development exist:
Making reviews more user friendly
Even in settings with unlimited access to the Cochrane Library, Cochrane reviews may not be used in guideline development.30 Possible reasons include perceptions that in the current format reviews are too lengthy, detailed, technical, difficult to read, and disparate and disjointed. Cochrane Review Groups are attempting to overcome these barriers, for example, by including “plain language summaries” and structured abstracts containing user-friendly statistics (number needed to treat/harm), and short concluding statements about the principal implications for practice. The Child Health field of the Cochrane Collaboration is also currently exploring the utility of “umbrella” or “bridging” reviews – structured clinical summaries of the findings of a group of related Cochrane reviews published online in familiar journal format (see: http://www.evidence-basedchildhealth.com).
Keeping reviews up to date
A key strength of Cochrane reviews is that they are updated at least every 2 years. In this regard, the Cochrane Neonatal and Childbirth and Pregnancy Review Groups have been victims of their own success to some extent as frequent updating now consumes a considerable fraction of reviewer and editor effort. Review groups have had to adopt strategies to prioritise reviews for updating based on perceived clinical importance and knowledge of availability of new evidence. These have been successful in ensuring that the most topical and clinically relevant reviews are up to date.36
Increasing relevance to middle- and low-income settings
It is critical for low- and middle-income countries to ensure that care practices are evidence based so that scarce resources are used appropriately. The Cochrane Collaboration, through initiatives such as the Effective Health Care Alliance and the “Sea-orchid” consortium, is engaging with various international, national, and local agencies to increase the adoption of evidence-based reproductive and perinatal healthcare in middle- and low-income countries.37 38 Such programmes target generators of evidence, users of evidence, and teachers of evidence in order to ultimately ensure the implementation of effective interventions.
The Cochrane Collaboration during the past 15 years has assembled a unique source of summarised and synthesised high-quality evidence to help improve clinical practice and inform the research agenda for newborn infants and their families. During the next 15 years, the Collaboration will continue to embrace opportunities to ensure that policy and practice during pregnancy, childbirth, and the newborn period are supported by the best available levels of evidence.
Competing interests Roger Soll is co-ordinating editor, and William McGuire is feedback editor of the Cochrane Neonatal Review Group.
Provenance and Peer review Commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.