Article Text
PDF
Abstract
Background Research findings are not rapidly or fully implemented into policies and practice in care.
Objectives To assess whether an ‘active’ strategy was more likely to lead to changes in policy and practice in preterm baby care than traditional information dissemination.
Design Cluster randomised trial.
Participants 180 neonatal units (87 active, 93 control) in England; clinicians from active arm units; babies born <27 weeks gestation.
Control arm Dissemination of research report; slides; information about newborn care position statement.
Active arm As above plus offer to become ‘regional ‘champion’ (attend two workshops, support clinicians to implement research evidence regionally), or attend one workshop, promote implementation of research evidence locally.
Main outcome measures timing of surfactant administration; admission temperature; staffing of resuscitation team present at birth.
Results 48/87 Lead clinicians in the active arm attended one or both workshops. There was no evidence of difference in post-intervention policies between trial arms. Practice outcomes based on babies in the active (169) and control arms (186), in 45 and 49 neonatal units respectively, showed active arm babies were more likely to have been given surfactant on labour ward (RR=1.30; 95% CI 0.99 to1.70); p=0.06); to have a higher temperature on admission to neonatal intensive care unit (mean difference=0.29oC; 95% CI 0.22 to 0.55; p=0.03); and to have had the baby's trunk delivered into a plastic bag (RR=1.27; 95% CI 1.01 to 1.60; p=0.04) than the control group. The effect on having an ‘ideal’ resuscitation team at birth was in the same direction of benefit for the active arm (RR=1.18; 95% CI 0.97 to 1.43; p=0.09). The costs of the intervention were modest.
Conclusions This is the first trial to evaluate methods for transferring information from neonatal research into local policies and practice in England. An active approach to research dissemination is both feasible and cost-effective.
Trial registration Current controlled trials ISRCTN89683698
Statistics from Altmetric.com
Introduction
The findings of health service research are frequently not implemented into policies and practice in the delivery of care.1 Studies show that as many as 30% of patients do not receive acute care of proven effectiveness, and this can rise to 40% for chronic care. Moreover, as many as 30% of patients receive acute care that is unnecesary or potentially harmful.2 Much recent empirical research has focused on strategies to translate research evidence into clinical practice (http://www.rxforchange.ca). A systematic review of interventions to disseminate printed educational materials concluded that such interventions can be effective in improving process outcomes but not patient outcomes.3 Examples of other promising methods include audit and feedback (especially when baseline compliance is low and feedback is delivered more intensively4), local opinion leaders (‘champions’5) and reminders.6 The effects of any of these interventions considered separately are modest to moderate (10–15%). Combined (multifaceted) interventions aiming at acting on different barriers to change do not display a clear dose-response relationship,6 however, it is plausible that multifaceted interventions built upon a careful assessment of barriers may be more effective than single interventions.7
What is already known on this topic
▶ Translation of the clinical benefit of research evidence into policies and practice by traditional dissemination of information methods is very poor.
▶ One North American neonatal trial showed that an intensive, complex (and expensive) approach significantly increased the evidence-based use of artificial surfactant in preterm babies.
▶ No attempt has been made to rectify this gap in applying knowledge in the UK setting using more economical methods.
What this study adds
▶ A cluster randomised trial of UK neonatal units, using innovative ‘active’ knowledge transfer strategies, led to significant changes in practice compared with passive dissemination.
▶ This is the first English trial to evaluate methods for transferring information from neonatal research into local polices and practice. The costs were modest.
▶ This active approach to research dissemination is both feasible and cost-effective in the NHS.
Although most of this evidence is based on research in the care of adults, there had been one randomised controlled trial (RCT) in neonatal care. The Vermont Oxford Network conducted a cluster randomised controlled trial, using a multifaceted collaborative quality improvement intervention (audit and feedback, lectures on reviews of the evidenced-based literature, an interactive training workshop and ongoing faculty support via conference calls and email) to promote evidence-based surfactant treatment for preterm babies born at 23−29 weeks' gestation.8 This intervention was associated with a significant improvement (40%) in the earlier surfactant administration compared to usual dissemination strategies.
The Vermont Oxford trial, like most of the relevant literature, was based in North America. The main aim of the present study is to use the rigour of a RCT in an evaluation comparing the effects of different approaches to knowledge transfer on policy and practice in the care of preterm babies in another setting.
Methods
The methods are reported in the published trial protocol.9 In summary, Project 27/2810 conducted by the UK Confidential Enquiry into Maternal and Child Health (CEMACH) (now Centre for Maternal and Child Enquiries (CMACE)) provided the basis for this trial. Variations in standards of care were reported that may have contributed to the death of preterm babies born at 27 or 28 weeks' gestational age. The results and a dissemination package with Microsoft PowerPoint slides were distributed to all UK neonatal units. To evaluate the impact on policy and practice of this dissemination, CEMACH sent a questionnaire to key UK recipients. Responses were received from 94 out of 262 neonatal/paediatric clinicians (36%), and 86 of the 183 acute hospitals with maternity services (47%). Three-quarters of the respondents said they recalled receiving the dissemination package, and most reported using the slide package, finding it useful for raising awareness of the clinical issues and fostering the initiation and/or consolidation of policy and practice changes.
However, gaps remained between the evidence and current practice. This led to a position statement on ‘Early Care of Premature Babies’ from the British Association of Perinatal Medicine (BAPM).11
The aim of the BLISScluster randomised controlled trial of the Effect of ‘Active Dissemination of Information’ on standards of care for premature babies in England (BEADI) trial was to assess whether an evidence-based innovative ‘active’ knowledge transfer strategy was more likely to lead to changes in policy and practice than the passive dissemination of the report, the slide package and the neonatal position statement on a website.
The intended effect was to change practice and policy at institutional level, therefore a cluster RCT where randomisation is by hospitals is the most appropriate design,12 as this allows the delivery of the intervention to be focused on all staff.
Participants
There were three levels:
Neonatal units: the 180 hospitals in England with neonatal intensive care units (NICUs) were identified at the beginning of 2006 by CEMACH working with the EPICure2 study group (http://www.epicure.ac.uk/epicure-2).
Lead clinicians from the neonatal units (see under Interventions below for further details about their roles).
Babies born at <27 weeks' gestation in England in January 2007–March 2007, identified using the CEMACH (now CMACE) nationwide network for data collection.13
Randomisation
Neonatal units were stratified by designation of level of care within the managed Clinical Neonatal Networks (n=25) and by level of care delivered (level I, II or III), and then ordered alphabetically by name of hospital and imported into statistical computer software Stata V.9 at the London School of Hygiene and Tropical Medicine. The programme generated a series of blocks of varying size (two, four or six) for each stratum and allocated units to control or active intervention randomly within each block.
Interventions
Control arm: Clinicians in the control group had previously been sent the CEMACH Project 27/28 report which identified variations in standards of care that might have contributed to death in preterm babies born at 27 or 28 weeks gestation. In addition, they were informed about recommendations and the position statement on early care of the newborn available on the BAPM website,10 and sent an accompanying slide set specifically tailored for different audiences to aid discussions at local hospital clinical governance meetings.
Active arm (box 1): Clinicians in the active group received the same information but in addition were sent a letter explaining the BEADI study and asking them to volunteer to play one of the following roles:
Regional ‘champions’ who would attend two workshops and then provide ongoing support to clinicians within their Network in implementing the research evidence.
Other lead clinicians who would attend one workshop and work at implementing the research evidence in their own unit.
The intervention
Workshop 1, September 2006 provided regional champions with time to explore: (a) the theory and practice of NHS organisational change; (b) the role of champions as leaders; (c) behaviour change principles and the human dimension of changes and (d) how to develop practical skills to effect and sustain change in order to support health care staff in their workplace. The attendees were supervised by trainers with expertise in organisational change.
At workshop 2, a week later, the regional champions and trainers, were joined by consultants and senior nurses from the ‘active’ units to explore: (a) the research evidence in specified clinical areas summarised in lectures from national clinical leaders; (b) benchmarking of individual policies and practices and (c) introduction to tools and practice of change. The clinical leads were then asked to determine actions to: (a) develop responses to suggested areas of changes locally and (b) the support and processes needed to achieve these changes.
After the workshops, the trial team distributed preliminary benchmarking information to allow participating units to see how practices in their own centres compared to practices in England overall. The team also facilitated further contacts and support via email and telephone over the following 2–3 months.
The workshops used an organisational development cycle14 in which a theoretical approach recognises and uses participants' experiential learning to provide a framework to inform the practical process of planned change and to introduce other approaches to manage change. These approaches are enhanced by practical hints, tips and tools for local use. Within this framework, self-nomination (volunteering) may enhance local communication and coordination, employee motivation and capability in the change process.15
A third group of clinicians in the active arm did not volunteer for either of these roles.
Outcomes
The main outcomes were at the level of unit policy and practice chosen to meet the following criteria: (a) the outcomes needed to be important; (b) there was an evidence base for interventions to address these outcomes; and (c) the outcomes were those collected in the EPICure2 study (see below).
Characteristics of units and pre-intervention policies
Project 27/28 showed that a delay in surfactant administration, hypothermia on admission and inadequately experienced staff present at resuscitation at birth were strongly associated with death. Hence BEADI focused on these three main areas of neonatal care. The wording used to describe these outcomes was consolidated at the second workshop (see box 2).
Data collection
The BEADI study collaborated with the EPICure2 study group and CEMACH to avoid duplicating data collection. Data about pre-intervention unit policies were collected by EPICure2 in early 2006. No information about surfactant policies was available from EPICure2, nor about pre-intervention practices in time for the BEADI study. Post-intervention data about both policies and practice were collected by CEMACH for January 2007–March 2007.
Sample size
The power calculations were based on a two-stage approach for policies (units) and for practice (babies). Based on consultation with experts, we assumed pre-intervention rates of between 35% and 60% for the existence of policies for timing of surfactant administration, early temperature control and skill-mix of the resuscitation team. Data from 130 of the 180 NICUs were available from EPICure2. This number would be sufficient to detect effect sizes (RRs) between 1.4 and 1.6, with 80% power at 5% level of statistical significance (two-sided test).
The power calculations for practice outcomes were based on an estimated 1650 annual admissions for neonatal intensive care from 3500 births of babies <27 weeks in England. Over a three-month period, we therefore expected approximately 850 births and 400 admissions. We used a range of likely intra-cluster correlation coefficients (ICC) from published databases of active dissemination research in previous trials, to estimate the power to detect differences in practice between the trial arms. For example, 400 admissions will have around 80% power to detect a difference in practices from 40% to 55% (5% two-sided significance) with ICC of 0.06.
After completing the initial power calculations, further information became available via the EPICure2 study, and we were able to randomise 180 units. The power calculations were therefore conservative and allowed for losses to follow up.
Statistical analysis
Analysis was based on intention to treat principles, comparing outcomes from all the hospitals allocated to active intervention with those allocated to control. For the policies, the analysis is based on hospitals. Fisher's exact test was used when the expected values in any cells were less than five. Analysis for practice outcomes is based on babies within hospitals, taking appropriate account of the clustering by using the generic Stata survey command to estimate design-based F statistics to assess significance levels and estimating effects using generalised models with standard errors adjusted for inter-group correlation. Subgroup analyses are based on the stratification factors. Exploratory per-protocol analyses compared outcomes from hospitals in the control arm to those in the active arm where a lead clinician attended one or more workshops.
Ethical considerations
Approval for the study was granted by the Multi-centre Research Ethics Committee, and the East London and the City Local Research Ethics Committee (LREC). Consent for intervention was requested post-randomisation from the lead clinicians (and their LREC and Trust research and development departments) for units allocated to the active intervention arm. With the agreement of the EPICure2 Steering Committee, Multi-centre Research Ethics Committee approval was also granted for the download of selected data items to CEMACH from the EPICure2 study. Data about babies were anonymised, but data about centres were coded in order to allow linkage with the CEMACH 2007 data collection, and individual centres were not named in any publications. Although information about BEADI had been posted on the BAPM website, clinicians in the control arm were only asked for their consent to the CEMACH data collection for BEADI towards the end of 2006 to reduce the likelihood of changes in their behaviour related to the study outcomes (a modified Zelen design).16
Results
Of the 180 units, 87 were randomised into the active arm and 93 into the control arm. Pre-intervention policy data were available from EPICure2 for 161 units (78 active (89.7%) and 83 control (89.2%)). The randomised groups were broadly comparable, and nearly all the units already had policies in place for prevention of hypothermia and for the resuscitation team at birth, before the trial began (table 1).
A request to participate was mailed to the 87 clinical leads in the active arm early in 2006. Eleven did not wish to participate, and the LREC or Trust research and developmentdepartment refused permission for a further two units. Representatives from 11 of the remaining 74 units volunteered to act as regional champions and attended both workshops in September and October 2006. Representatives of a further 37 units attended the second workshop only, and 26 did not attend either workshop. No representatives from the control units were invited to or attended either workshop.
At the second workshop, the wording of the key ‘messages’ for the champions and clinical leads to disseminate and support was discussed and provisionally agreed, with further refinements during email follow up over the ensuring weeks (box 2).
Practice aims (as agreed at, and shortly after, the workshops)
All intubated babies (<27 weeks) should receive surfactant within an hour of birth (as early as compatible with safety).
At all births <27 weeks gestational age, the following should be called and be present before the baby is delivered: a consultant paediatrician or a middle grade practitioner and a Senior House Officer or an Advanced Neonatal Nurse Practitioner.
Core temperature on admission to NICU should be ≥ 36°C (taken electronically).
The costs of these workshops totalled £11 485 (consisting of the fees for the behavioural change experts, travel reimbursements, room hire and subsistence).
Information about policies in January 2007–March 2007 was available for 131 units (62 active (71.3%) and 69 control (74.2%)). Although there was no scope for improvement in the percentage of units having a strategy for hypothermia prevention, the percentage where the strategy involved delivering the baby into a plastic bag or wrapping did increase (albeit only marginally) from 89% to 95%, but there was no evidence of a difference between the trial arms (RR 0.98, 95% CI 0.90 to 1.07; p=0.71). Similarly, the percentage where there was a policy about the paediatric staff present at the time of birth increased from 87% to 95%, but the difference between the trial arms, although not statistically significant, favoured the control arm (table 2). This reflected the pre-intervention differences between the groups and, based on the 110 units with information at both time points, 9/51 introduced this policy in the active group compared to 4/59 in the control group.
Policies post-intervention
Information about practice outcomes for January 2007–March 2007 was available from 94 units for 355 babies born between 22 and 26 weeks (169 from 45 units in the active arm, and 186 from 49 units in the control arm). More of the babies in the control arm were male and their mean birth weight was lower than those in the active arm (table 3). A slightly higher proportion of missing data was from the level 2 units. Babies in the active arm were more likely to have been given surfactant on the labour ward (RR=1.30; 95% CI 0.99 to 1.70); p=0.06); to have a higher temperature on admission to NICU (mean difference=0.29oC; 95% CI 0.22 to 0.55; p=0.03); to have had the trunk delivered in a plastic bag (RR=1.27; 95% CI 1.01 to 1.60; p=0.04) than those in the control. The effect on having an ‘ideal’ resuscitation team at birth was in the same direction of benefit for the active arm (RR=1.18; 95% CI 0.97 to 1.43) but did not reach conventional levels of statistical significance (p=0.09) (table 4).
Characteristics of babies post-intervention
Practices post-intervention
Given the high percentage of hospitals adhering to the policies post-intervention, it was not meaningful to undertake the formal planned subgroup analyses based on the stratification factors at policy level, or per-protocol analyses. In terms of practice outcomes, pre-specified subgroup analyses did not find any interactions between arm of the trial and level of care; however formal interaction tests were not undertaken between network and arm of the trial due to the high number of strata (25). In per-protocol analyses, the effect on practice appeared similar or only very slightly higher in the groups in which at least one workshop was attended (78.5% surfactant on labour ward, 79.1% delivery into plastic bag, 67.1% with ‘ideal’ team, and mean temperature 36.4 compared to 78.3 %, 78.8%, 67.7% and 36.5 respectively).
Discussion
This cluster randomised trial is the first trial in neonatal medicine in England to evaluate methods for translating research into local policies and practice. The catchment area was nationwide and the methods were rigorous. Using an innovative ‘active’ knowledge transfer strategy led to significant changes in practice compared with passive dissemination of the research report, a slide package and a ‘position statement’ on a website. The costs of the intervention (including workshop facilitators, travel, room hire and food) were modest.
Limitations related to the lack of prior information about the extent to which the policy and practice outcomes were already in place. This meant that there was little scope for improving the policy outcomes (ceiling effects), and there was less power than planned for the practice outcomes. In addition, a RCT about the use of occlusive plastic wrapping was in progress (http://clinicaltrials.gov/ct2/show/record/NCT00930917?term=occlusive+wrapping&rank=1).
Another potential limitation of the study was the extent of loss to follow up. This was, however, equal in both trial arms and built into the power calculations. Although it appears that a higher proportion of level 2 hospitals may have had missing data compared to levels 1 and 3, the numbers (20 missing hospitals) are too small to draw confident conclusions about implications for generalisability.
There may have been some dilution of effects because even the control arm received information about Project 27/28 with a slide set for further dissemination. Also, although clinicians in the control arm were not formally told about their BEADI allocation until late 2006 (de facto, blinded), it is possible they could have then changed their policies or practice, or amended data from their units in a biased way. This seems unlikely. The active intervention might have been more successful if representatives of more of the units attended either workshop. The level of take up was in part a consequence of the decision to ask for consent only after randomisation (the modified Zelen design16). This may have reduced power, but meant that the initial decision to participate in the study was not biased by knowledge of the allocation.
The intervention might have been more effective if a more intensive approach was used. Although the BEADI team facilitated further contacts and support for the clinicians in the active arm via email and telephone over the following 2–3 months, the intervention was less intensive than in the Vermont Oxford trial which used a tight network of clinicians who worked together and were receptive to quality improvement and benchmarking of their performance.17 In addition, the Vermont Oxford trial used a two-day interactive, collaborative improvement workshop; social networking during the meeting (which has been shown to contribute to the success of collaborative initiatives); interactive networking with good communication between the main centre collecting evidence and detailing it proactively to the different hospitals in the network; and the use of the continuous quality improvement concept applied through the Rapid Cycle Improvement Process (RCIP).18 Two further quality improvement cluster randomised trials were published after the BEADI fieldwork, both also in North America and both using pre-existing networks of clinicians. One focused on benchmarking and did not find a significant effect on bronchopulmonary dysplasia (BPD)-free survival, although the study was confounded by an unexpectedly low rate of BPD in the pre-intervention year, making improvement more difficult.19 Lee et al20 randomised units to quality improvement strategies focusing on either reducing BPD or nosocomial infection. There was a clear benefit for reducing BPD, but not for reducing nosocomial infection (as this also improved in the non-nosocomial infection arm).
The less intensive approach taken in BEADI, however, was more suited to the UK and similar settings where it can be seen as part of continuing professional development within the NHS. The costs (including workshop facilitators, travel, room hire and food) were modest. Recent progress in the role of the Managed Clinical Neonatal Networks, and greater use of internet communications, including conference calls, should be considered when developing this promising approach to translational medicine in the future.
In conclusion, this active approach to research dissemination is both feasible and cost-effective in the NHS.
Acknowledgments
The authors would like to thank all the clinicians who responded to the invitation to participate, and especially those involved in one or more of the workshops; Charlotte Bennett, Ian Laing, David Field for speaking at the second workshop; Bliss and CMACE (formerly CEMACH); and all others described above and below.
References
Footnotes
DA led this research and wrote the first draft of the paper but died in 2008, before it was completed
-
Funding The trial was funded by the Bliss Innovation in Care Programme. Further support, including additional data collection, was provided by CEMACH (now CMACE), who acted as the sponsor. Both Bliss and CEMACH were represented on the Steering Committee for the trial and provided comments on study design; the collection and interpretation of data; the writing of the report and in the decision to submit the article for publication.
-
Competing interests None.
-
Ethics approval This study was conducted with the approval of the Multi-centre Research Ethics Committee, and the East London and the City Local Research Ethics Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.