You are here

Article Text

Article menu

Download PDFPDF

Original research
Core outcomes in neonatology: development of a core outcome set for neonatal research
  1. James William Harrison Webbe1,
  2. James M N Duffy2,
  3. Elsa Afonso3,
  4. Iyad Al-Muzaffar4,
  5. Ginny Brunton5,
  6. Anne Greenough6,
  7. Nigel J Hall7,
  8. Marian Knight8,
  9. Jos M Latour9,10,
  10. Caroline Lee-Davey11,
  11. Neil Marlow12,
  12. Laura Noakes13,
  13. Julie Nycyk14,
  14. Angela Richard-Löndt13,
  15. Ben Wills-Eve15,
  16. Neena Modi16,
  17. Chris Gale1
  1. 1 Academic Neonatal Medicine, Imperial College London, London, UK
  2. 2 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, Oxfordshire, UK
  3. 3 Neonatal Unit, Rosie Hospital, Cambridge, Cambridgeshire, UK
  4. 4 The Neonatal Unit, Royal Glamorgan Hospital, Llantrisant, Rhondda Cynon Taf, UK
  5. 5 UCL Institute of Education Centre for Longitudinal Studies, London, UK
  6. 6 Department of Women and Children's Health, School of Life Sciences, Faculty of Life Sciences and Medicine, King's College London, London, UK
  7. 7 Paediatric Surgery, Southampton General Hospital, Southampton, UK
  8. 8 National Perinatal Epidemiology Unit, Oxford, UK
  9. 9 School of Nursing and Midwifery, Faculty of Health, Education and Society, Plymouth University, Plymouth, Devon, UK
  10. 10 School of Nursing and Midwifery, Faculty of Health Sciences, Curtin University, Perth, Western Australia, Australia
  11. 11 Bliss, London, UK
  12. 12 Institute for Women's Health, University College London, London, UK
  13. 13 Parent of Neonatal Patient, London, UK
  14. 14 Neonatal Unit, Birmingham City Hospital, Birmingham, UK
  15. 15 Former neonatal patient, London, UK
  16. 16 Neonatal Medicine, Imperial College London, London, UK
  1. Correspondence to Dr James William Harrison Webbe, Academic Neonatal Medicine, Imperial College London, London, UK; j.webbe{at}imperial.ac.uk

Abstract

Background Neonatal research evaluates many different outcomes using multiple measures. This can prevent synthesis of trial results in meta-analyses, and selected outcomes may not be relevant to former patients, parents and health professionals.

Objective To define a core outcome set (COS) for research involving infants receiving neonatal care in a high-income setting.

Design Outcomes reported in neonatal trials and qualitative studies were systematically reviewed. Stakeholders were recruited for a three-round international Delphi survey. A consensus meeting was held to confirm the final COS, based on the survey results.

Participants Four hundred and fourteen former patients, parents, healthcare professionals and researchers took part in the eDelphi survey; 173 completed all three rounds. Sixteen stakeholders participated in the consensus meeting.

Results The literature reviews identified 104 outcomes; these were included in round 1. Participants proposed 10 additional outcomes; 114 outcomes were scored in rounds 2 and 3. Round 1 scores showed different stakeholder groups prioritised contrasting outcomes. Twelve outcomes were included in the final COS: survival, sepsis, necrotising enterocolitis, brain injury on imaging, general gross motor ability, general cognitive ability, quality of life, adverse events, visual impairment/blindness, hearing impairment/deafness, retinopathy of prematurity and chronic lung disease/bronchopulmonary dysplasia.

Conclusions and relevance A COS for clinical trials and other research studies involving infants receiving neonatal care in a high-income setting has been identified. This COS for neonatology will help standardise outcome selection in clinical trials and ensure these are relevant to those most affected by neonatal care.

  • neonatology
  • outcomes research
  • evidence based medicine
https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/archdischild-2019-317501

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    What is already known on this topic?

    • Inconsistent reporting of outcomes of limited relevance to former patients, parents and healthcare professionals is an important cause of research waste.

    • There is a lack of evidence to guide many neonatal practices, leading to variation in both the care provided and outcomes for patients.

    • Core outcome sets (agreed, standardised outcomes to be reported by all trials) have been developed in other fields to improve outcome selection and facilitate meta-analysis.

    What this study adds?

    • Former patients, parents, doctors, nurses and researchers show differences in how they prioritise neonatal care outcomes.

    • We have identified 12 outcomes that are important to these stakeholders.

    • If these outcomes are reported in a standardised manner by all neonatal research, this will enhance future evidence synthesis.

    Introduction

    The neonatal period is crucial to long-term health, and neonatal conditions are the leading cause of disability-adjusted life-year loss.1 Preterm birth is a major cause of childhood morbidity2 3 and implicated in the pathogenesis of adult non-communicable diseases.4 Neonatal care is common; in high-resource settings one in ten babies are admitted to a neonatal unit, a proportion that is increasing.5

    Unfortunately there is a paucity of high-quality evidence to guide much neonatal practice, leading to variation in clinical care6 7 and outcomes.8 9 One reason research fails to guide practice is because neonatal meta-analyses rarely provide conclusive recommendations,10 11 commonly because trials have used heterogeneous, non-comparable outcomes.12 13 A further limitation of neonatal and paediatric research is that the outcomes reported are frequently not meaningful to patients and parents.14 15

    One solution is the development of a core outcome set: important outcomes identified by key stakeholders using robust consensus methods.16 A core outcome set could ensure all future research in a field reports a common subset of clinically meaningful outcomes and reduces research waste by facilitating meta-analysis.17 A core outcome set is a minimum set and does not preclude researchers reporting other outcomes where relevant.16 The use of core outcome sets for trials is promoted by journals,18 Cochrane Review Group editors19 and research funders.20 Relevant, standardised outcomes are also crucial for observational research,21 22 benchmarking,23 clinical audit24 and quality improvement studies.25

    Objective

    The objective was to develop a core outcome set for research in neonatology.

    Scope

    The core outcome set has been developed to apply to all research involving babies receiving care on any designation of neonatal unit in a high-income setting, with no limitation by gestational age at birth, birth weight or illness severity. It is intended to apply regardless of the specific population of babies, clinical setting or clinical condition that a particular study addresses. The scope was established at the initial steering group meeting following direction from former patients and parents. The parents and former patients all strongly expressed the view that ‘a sick baby is a sick baby’. They were also clear that while it is possible to separate babies on a neonatal unit by gestation, weight or underlying diagnoses, the outcomes that are most important are universal to all these groups. Research involving babies cared for exclusively on labour or postnatal wards or in the community will be excluded as the majority are healthy needing limited medical input.

    Methods

    We prospectively registered the study with the Core Outcome Measures in Effectiveness Trials (COMET) initiative (Registration number 842)26 and published the study protocol.27 Research ethics approval was not required; the project involved consenting adults completing surveys (online supplementary eFigure 1). We formed a steering group to guide the core outcome set development comprising different disciplines, perspectives and expertise (online supplementary eText 1).

    We followed COMET initiative methodology28 with reference to previous core outcome set development work.29 We identified outcomes reported in neonatal trials and qualitative research and then used these to determine a core outcome set using a consensus process (online supplementary eFigure 2).

    Information sources

    We undertook a prospectively registered systematic review to identify outcomes reported in neonatal clinical trials.30 Randomised controlled trials are widely considered to be the most rigorous method to determine how a treatment affects patients.31 32 We searched Cochrane Controlled Trials Register, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE and Medline from July 2012 to July 2017. Three researchers (SA, SS, JWHW) independently double-screened potentially relevant records based on titles and abstracts and reviewed the full text of selected studies to assess eligibility. Due to the large number of trials identified, only those with over 100 infants in each arm were included. As many trials lead to more than one publication reporting outcomes at different time points, we reviewed all linked publications. Outcomes were extracted and categorised by physiological system.

    We undertook a second, prospectively registered33 review to identify outcomes from qualitative research.34 We searched Applied Social Sciences Index and Abstracts, CINAHL, EMBASE, Medline and PsycINFO from 1997 to 2017. Publications describing qualitative data relating to neonatal care outcomes, reported by former patients, parents or healthcare professionals, were included. Narrative text and grouped outcomes were thematically analysed by physiological system.

    The steering group assessed outcomes identified in the two reviews to produce a final inventory in which duplicated or closely related outcomes were grouped. The inventory informed a three-round, online eDelphi survey which was followed by a consensus meeting.

    Participants

    We recruited participants for the eDelphi from the following groups:

    1. Former patients cared for on a neonatal unit, and parents of neonatal patients, recruited through neonatal charities and social media.

    2. Nurses and allied health professionals (including neonatal nurses, midwives, speech and language therapists, occupational therapists, and physiotherapists), recruited through professional journals and associations.

    3. Doctors (including neonatologists, obstetricians, paediatric surgeons, general paediatricians, community paediatricians and general practitioners), recruited through the Royal College of Paediatrics and Child Health and professional organisations.

    4. Academics and researchers in the neonatal field, recruited through meetings, academic publications and organisations.

    Recruitment was international; participants had to have personal experience of neonatal care or research in a high-income setting. We aimed for 30 participants in each group to achieve a total of 120 participants. The sample size followed guidance35 and previous core outcome set development.36

    Consensus process

    Participants completed a three-round eDelphi survey37 to establish consensus. We ran the eDelphi using DelphiManager software.38 To maximise response rates, the survey was kept as short as possible39 and extensive demographic data were not collected. In each round we asked participants to rank outcomes between 1 and 9 (with 1–3 meaning ‘limited importance for decision making’ and 7–9 meaning ‘critical for decision making’) following the Grading of Recommendations Assessment, Development and Evaluation guidelines40 (figure 1). In round 1, participants could suggest outcomes not identified in the reviews which they felt were important; these outcomes were included in rounds 2 and 3. After each round we collated the results. Before participants reviewed and rescored outcomes in rounds 2 and 3, we presented them with a bar chart showing how each outcome had been scored previously. Each graph combined the scores from all stakeholder groups. We applied predefined consensus criteria to round 3 results.16 Provisional core outcomes were those over 70% of participants in each group scored as ‘critical’ and less than 15% of each group scored as ‘limited importance’. Conversely, if over 70% of participants in each group scored an outcome ‘limited importance’ and less than 15% in each group scored it ‘critical’, it was not included. If neither criterion was met, an outcome was classified as ‘no consensus’.

    Figure 1
    Figure 1

    Example screenshot of eDelphi survey.

    Consensus between groups

    We compared scoring patterns using the first round results to assess agreement between stakeholder groups. The mean scores for each outcome were calculated for each group, and pairwise comparisons were then made between groups. Pearson’s correlation coefficient was calculated for each comparison; differences between coefficients were tested using Fisher’s r-to-z transformation.41

    Attrition analysis

    We undertook an attrition analysis to ensure the eDelphi results had not been distorted by differences in opinion between those who dropped out and those who completed all surveys. We compared two groups: participants who only took part in round 1 (including those who dropped out during this round) and participants who contributed in all rounds. We compared how these groups scored outcomes in round 1. We used Mann-Whitney U to test for differences in scoring with Bonferroni correction for multiple comparisons (corrected to 5% significance). For outcomes where a difference in scoring was identified, we also tested if the different scoring patterns observed would have changed whether the outcome was considered ‘core’ in round 1 (according to the predefined consensus definition), suggesting attrition affected whether the outcome met the criteria for inclusion in the final core outcome set.

    Consensus meeting

    The final prespecified phase was a face-to-face meeting to confirm the final core outcome set based on the eDelphi results. We only invited steering group members and eDelphi participants with additional expertise; the meeting was limited to 16 participants to facilitate discussion.42 The consensus meeting remit was limited to refining the final survey results, no new outcomes were considered, and the eDelphi results were paramount. The consensus group were presented the results of the eDelphi and the attrition analysis. They considered whether the identified core outcomes covered all necessary domains, whether there was overlap between outcomes and whether it would be feasible to expect all trials to record each outcome. They discussed the following outcomes in depth: outcomes that met the consensus definition, ‘borderline’ outcomes that narrowly missed the consensus definition (defined as 70% of at least one stakeholder group scored the outcome as ‘critical’) and any outcomes identified during the attrition analysis. Meeting attendees discussed each outcome, then an anonymous vote was held on the question ‘should the outcome be included in the core outcome set?’ For inclusion in the final set, 70% of attendees had to vote ‘Yes’. We have published the meeting minutes online.43

    Results

    This study was completed according to the study protocol.27 The only deviation occurred during the review of trials: due to the large number of studies identified, only trials with over 100 neonates in each arm were included. The results of this core outcome set development are reported using COS-STAR (Core Outcome Set–STAndards for Reporting) reporting guidelines.44

    In the review of clinical trials we identified 76 large neonatal trials reporting 216 outcomes, and in the qualitative literature review we identified 62 publications with 146 outcomes.34 The steering group reviewed these 362 outcomes, identified 19 duplicates and grouped 239 closely related outcomes. This resulted in a final inventory of 104 outcomes, which were entered into the eDelphi (figure 2) (full list in online supplementary eTable 1). Participants added 10 additional outcomes following the first round (online supplementary eTable 2).

    Figure 2
    Figure 2

    Flow chart of identification and selection of outcomes.

    eDelphi surveys

    We recruited a total of 414 participants from 25 countries across 5 continents (online supplementary eFigure 3). The distribution of participants in different stakeholder groups and their participation during the eDelphi are presented in table 1. Participation in all rounds exceeded our target of 120 participants.

    View this table:
    Table 1

    Stakeholder participation across eDelphi rounds

    Two hundred and sixty participants completed the first round. The mean scores for parents and patients correlated with the scores of nurses and therapists more closely (r=0.83) than with the scores of doctors (r=0.51). The mean scores from doctors correlated most closely with those of researchers (r=0.96). The differences between these correlations were statistically significant (p<0.01). Pairwise comparisons are shown in figure 3.

    Figure 3
    Figure 3

    Scatterplots comparing round 1 mean scores (pairwise comparison between stakeholder groups).

    The final round was completed by 173 participants. The highest scoring outcomes from each stakeholder group are shown in table 2.

    View this table:
    Table 2

    Highest scoring outcomes in round 3 by stakeholder group (outcomes ranked by mean score)

    The prespecified consensus definition was met for 15 outcomes; these were discussed at the consensus meeting along with 21 outcomes ranked as ‘borderline’. The attrition analysis identified a statistically significant difference between scoring for 19 outcomes (online supplementary eTable 3); for 17 there was no difference in whether the outcome would have been included in the core outcome set. The remaining two outcomes were discussed at the consensus meeting to ensure attrition had not distorted the consensus process.

    Consensus meeting

    At the consensus meeting 16 participants representing all stakeholder groups (5 former patients/parents, 3 nurses/therapists, 5 doctors and 3 researchers) discussed and voted on each of the 38 outcomes identified from the eDelphi results. Twelve outcomes were identified for inclusion in the final core outcome set. During discussion the outcome ‘Harm from medical treatment’ was defined as ‘Adverse events’ to allow better alignment with existing classifications of iatrogenic harm. Two outcomes (‘Retinopathy of prematurity’ and ‘Chronic lung disease/bronchopulmonary dysplasia’) relate only to preterm infants and should only be reported by trials involving this group. Meeting minutes and voting results are provided in online supplementary eText 2.

    Core outcome set

    The final core outcome set comprises the following:

    1. Survival.

    2. Sepsis.

    3. Necrotising enterocolitis.

    4. Brain injury on imaging.

    5. Retinopathy of prematurity (preterm only).

    6. General gross motor ability.

    7. General cognitive ability.

    8. Quality of life.

    9. Adverse events.

    10. Visual impairment or blindness.

    11. Hearing impairment or deafness.

    12. Chronic lung disease/bronchopulmonary dysplasia (preterm only).

    (Outcomes were ranked by percentage of round 3 participants who scored each outcome ‘critical for decision making’.)

    Discussion

    Using robust, preregistered consensus methodology, we identified 12 outcomes to be reported in all future trials involving infants receiving care on a neonatal unit in a high-income setting. We hope use of this core outcome set will improve research quality and reduce waste. The core outcome set is a minimum set of outcomes that are so important to all stakeholders that failing to report them will mean that important clinical uncertainties cannot be addressed, both at the level of individual studies and in subsequent meta-analyses.

    This core outcome set complements the work by van’t Hooft et al 45 in which a core outcome set for interventions to prevent preterm birth was identified. This contained maternal and neonatal outcomes, but the scope was limited to antenatal interventions. A number of core outcome sets have been developed in women’s health36; in the newborn period these exist only for gastroschisis46 and Hirschsprung’s disease,47 with work under way for neonatal abstinence syndrome.48 In rheumatology widespread adoption has led to full reporting of the rheumatoid arthritis core outcome set in 80% of relevant trials.49 Similar uptake in neonatal research would reduce barriers to meta-analysis10 and aid translation of research findings into clinical practice.

    A strength of our project was the number of parents and former patients who took part. Our review of trials found no reported involvement of parents or former patients in outcome selection; it is therefore unsurprising that these groups report dissatisfaction with outcomes currently reported in neonatal research.14 In our work former patients and parents scored outcomes by importance and could suggest additional important items. Their priorities differed from other stakeholder groups, emphasising the importance of wide involvement in outcome selection.

    A limitation of our work was attrition during the eDelphi, which occurred despite efforts to optimise response rates.39 The attrition rates in this study are comparable with similar projects.36 Explanations for the attrition include the wide range of outcomes (each survey took 20 min) and that participation was voluntary. Former patients and parents were most likely to drop out, perhaps due to their caring commitments.50 The attrition analysis identified outcomes where dropout could have skewed scoring patterns and distorted results; those identified were discussed further at the consensus meeting. Participant attrition is common during Delphi surveys; steps to minimise attrition are evolving.51 Another limitation is that potential stakeholder groups were not represented (eg, hospital administrators/policy makers). No guidance mandates which groups should be involved in core outcome set development16; our project included all groups included in most core outcome set development.52

    Future work will standardise outcome measures and measurement time points for the outcomes identified. While our review found outcome domains were similar across large neonatal trials, disparate measures and time points meant results were not comparable. Heterogeneity of measures and time points is a known barrier to evidence synthesis.12 Defining outcomes like ‘Adverse events’ or ‘Quality of life’, endpoints we have demonstrated to be important to all stakeholder groups, will allow research to report them consistently. Further input from former patients and parents is needed to ensure that outcome measures reflect their lived experiences.14 Starting in 2020 we will define measures and time points following OMERACT (Outcome Measures in Rheumatology) 2.0 methodology,53 in collaboration with other international efforts.54 55 Other core outcome sets have also been developed or are in development in the field of neonatology46 48; it is important that overlapping core outcome sets are harmonised to avoid the multitude of incomparable outcomes being replaced by multiple incompatible core outcome sets. The aim is that future research will report the core outcome set alongside trial-specific outcomes; trial-specific outcomes will address a particular research question and core outcomes will provide data for meta-analyses, particularly for prospectively planned meta-analyses.56

    While core outcome sets are associated with clinical trials, integration with routine data collection will reduce the burden on researchers, facilitate efficient research and improve quality. This will ensure future audit, benchmarking and quality improvement projects are focused on outcomes important to all.

    Conclusion

    We have identified a core outcome set for neonatal research. Adoption of this set will standardise outcome selection and ensure these are relevant to those most affected by neonatal care. This will help research translate into improved clinical practice, optimising outcomes for neonatal patients.

    Acknowledgments

    We wish to thank BLISS and the RCPCH for helping promote this project, and Susanna Sakonidou and Shohaib Ali for their help completing the systematic review of outcomes reported in clinical trials.

    References

    1. ↵
      2. Kyu HH ,
      3. Abate D ,
      4. Abate KH , et al
      . Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990-2017: a systematic analysis for the global burden of disease study 2017. Lancet 2018;392:1859–922.doi:10.1016/S0140-6736(18)32335-3
      OpenUrlCrossRefPubMed
    2. ↵
      2. Baraldi E ,
      3. Filippone M
      . Chronic lung disease after premature birth. N Engl J Med 2007;357:1946–55.doi:10.1056/NEJMra067279
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. McCormick MC ,
      3. Litt JS ,
      4. Smith VC , et al
      . Prematurity: an overview and public health implications. Annu Rev Public Health 2011;32:367–79.doi:10.1146/annurev-publhealth-090810-182459
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      2. Gluckman PD ,
      3. Hanson MA ,
      4. Cooper C , et al
      . Effect of in utero and early-life conditions on adult health and disease. N Engl J Med 2008;359:61–73.doi:10.1056/NEJMra0708473
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Harrison W ,
      3. Goodman D
      . Epidemiologic trends in neonatal intensive care, 2007-2012. JAMA Pediatr 2015;169:855–62.doi:10.1001/jamapediatrics.2015.1305
      OpenUrl
    6. ↵
      2. Klingenberg C ,
      3. Embleton ND ,
      4. Jacobs SE , et al
      . Enteral feeding practices in very preterm infants: an international survey. Arch Dis Child Fetal Neonatal Ed 2012;97:F56–61.doi:10.1136/adc.2010.204123
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Public Health England
      . The NHS atlas of variation in healthcare: reducing unwarranted variation to increase value and improve quality. London: Public Health England, 2015.
    8. ↵
      2. Helenius K ,
      3. Sjörs G ,
      4. Shah PS , et al
      . Survival in very preterm infants. Obstet Gynecol Surv 2018;73:187–9.doi:10.1097/OGX.0000000000000553
      OpenUrl
    9. ↵
      2. Shah PS ,
      3. Lui K ,
      4. Sjörs G , et al
      . Neonatal outcomes of very low birth weight and very preterm neonates: an international comparison. J Pediatr 2016;177:144–52.doi:10.1016/j.jpeds.2016.04.083
      OpenUrlCrossRefPubMed
    10. ↵
      2. Willhelm C ,
      3. Girisch W ,
      4. Gottschling S , et al
      . Systematic Cochrane reviews in neonatology: a critical appraisal. Pediatr Neonatol 2013;54:261–6.doi:10.1016/j.pedneo.2013.03.002
      OpenUrlCrossRefPubMed
    11. ↵
      2. Ding X ,
      3. Zhu L ,
      4. Zhang R , et al
      . Effects of family-centred care interventions on preterm infants and parents in neonatal intensive care units: a systematic review and meta-analysis of randomised controlled trials. Aust Crit Care 2019;32:63–75.doi:10.1016/j.aucc.2018.10.007
      OpenUrl
    12. ↵
      2. Heneghan C ,
      3. Goldacre B ,
      4. Mahtani KR
      . Why clinical trial outcomes fail to translate into benefits for patients. Trials 2017;18.doi:10.1186/s13063-017-1870-2
    13. ↵
      2. Duffy JMN ,
      3. Ziebland S ,
      4. von Dadelszen P , et al
      . Tackling poorly selected, collected, and reported outcomes in obstetrics and gynecology research. Am J Obstet Gynecol 2019;220:71.e1–4.doi:10.1016/j.ajog.2018.09.023
      OpenUrl
    14. ↵
      2. Janvier A ,
      3. Farlow B ,
      4. Baardsnes J , et al
      . Measuring and communicating meaningful outcomes in neonatology: a family perspective. Semin Perinatol 2016;40:571–7.doi:10.1053/j.semperi.2016.09.009
      OpenUrlPubMed
    15. ↵
      2. Sinha IP ,
      3. Williamson PR ,
      4. Smyth RL
      . Outcomes in clinical trials of inhaled corticosteroids for children with asthma are narrowly focussed on short term disease activity. PLoS One 2009;4:e6276.doi:10.1371/journal.pone.0006276
    16. ↵
      2. Williamson PR ,
      3. Altman DG ,
      4. Bagley H , et al
      . The comet Handbook: version 1.0. Trials 2017;18.doi:10.1186/s13063-017-1978-4
    17. ↵
      2. Kirkham JJ ,
      3. Boers M ,
      4. Tugwell P , et al
      . Outcome measures in rheumatoid arthritis randomised trials over the last 50 years. Trials 2013;14:324.doi:10.1186/1745-6215-14-324
    18. ↵
      2. Khan K
      . The crown initiative Journal editors Invite researchers to develop core outcomes in women's health. Obstet Gynecol 2014;124:487–8.
      OpenUrl
    19. ↵
      2. Kirkham JJ ,
      3. Gargon E ,
      4. Clarke M , et al
      . Can a core outcome set improve the quality of systematic reviews? – a survey of the Co-ordinating editors of cochrane review groups. Trials 2013;14:21.doi:10.1186/1745-6215-14-21
    20. ↵
      1. National Institute for Health Research
      . Patients and the public: support and resources for getting started. Available: https://www.nihr.ac.uk/patients-and-public/how-to-join-in/support-and-resources-for-getting-started.htm
    21. ↵
      2. Fitchett EJA ,
      3. Seale AC ,
      4. Vergnano S , et al
      . Strengthening the reporting of observational studies in epidemiology for newborn infection (STROBE-NI): an extension of the STROBE statement for neonatal infection research. Lancet Infect Dis 2016;16:e202–13.doi:10.1016/S1473-3099(16)30082-2
      OpenUrlCrossRef
    22. ↵
      2. Wynn JL ,
      3. Wong HR ,
      4. Shanley TP , et al
      . Time for a neonatal-specific consensus definition for sepsis. Pediatr Crit Care Med 2014;15:523–8.doi:10.1097/PCC.0000000000000157
      OpenUrlCrossRefPubMed
    23. ↵
      2. Concina VA ,
      3. Samide A ,
      4. Bada H , et al
      . Comparing diagnostic criteria for bronchopulmonary dysplasia (BPD) of Vermont Oxford network (von) to the National Institute of child health and development (Nichd) neonatal research network. J Invest Med 2016;64:604–5.
      OpenUrl
    24. ↵
      2. Kerber KJ ,
      3. Mathai M ,
      4. Lewis G , et al
      . Counting every stillbirth and neonatal death through mortality audit to improve quality of care for every pregnant woman and her baby. BMC Pregnancy Childbirth 2015;15.doi:10.1186/1471-2393-15-S2-S9
    25. ↵
      2. Lachman P ,
      3. Jayadev A ,
      4. Rahi M
      . The case for quality improvement in the neonatal intensive care unit. Early Hum Dev 2014;90:719–23.doi:10.1016/j.earlhumdev.2014.09.003
      OpenUrl
    26. ↵
      1. COMET initiative
      . Comet database, 2018. Available: http://www.comet-initiative.org/studies/details/842?result=true
    27. ↵
      2. Webbe J ,
      3. Brunton G ,
      4. Ali S , et al
      . Developing, implementing and disseminating a core outcome set for neonatal medicine. BMJ Paediatr Open 2017;1.doi:10.1136/bmjpo-2017-000048
    28. ↵
      2. Williamson PR ,
      3. Altman DG ,
      4. Blazeby JM , et al
      . Developing core outcome sets for clinical trials: issues to consider. Trials 2012;13:132.doi:10.1186/1745-6215-13-132
    29. ↵
      2. Duffy JMN ,
      3. van 't Hooft J ,
      4. Gale C , et al
      . A protocol for developing, disseminating, and implementing a core outcome set for pre-eclampsia. Pregnancy Hypertens 2016;6:274–8.doi:10.1016/j.preghy.2016.04.008
      OpenUrl
    30. ↵
    31. ↵
      2. Akobeng AK
      . Understanding randomised controlled trials. Arch Dis Child 2005;90:840–4.doi:10.1136/adc.2004.058222
      OpenUrlAbstract/FREE Full Text
    32. ↵
      2. Gale C ,
      3. McGuire W ,
      4. Juszczak E
      . Randomised controlled trials for informing perinatal care. Neonatology 2019:1–7.doi:10.1159/000499881
    33. ↵
    34. ↵
      2. Webbe J ,
      3. Brunton G ,
      4. Ali S , et al
      . Parent, patient and clinician perceptions of outcomes during and following neonatal care: a systematic review of qualitative research. Bmjpo 2018;2:e000343.doi:10.1136/bmjpo-2018-000343
    35. ↵
      2. Sinha IP ,
      3. Smyth RL ,
      4. Williamson PR
      . Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011;8:e1000393.doi:10.1371/journal.pmed.1000393
    36. ↵
      2. Duffy JMN ,
      3. Rolph R ,
      4. Gale C , et al
      . Core outcome sets in women's and newborn health: a systematic review. BJOG: Int J Obstet Gy 2017;124:1481–9.doi:10.1111/1471-0528.14694
      OpenUrl
    37. ↵
      2. Dalkey N ,
      3. Helmer O
      . An experimental application of the Delphi method to the use of experts. Manage Sci 1963;9:458–67.doi:10.1287/mnsc.9.3.458
      OpenUrlCrossRefWeb of Science
    38. ↵
      1. COMET initiative
      . DelphiManager, 2018. Available: http://www.comet-initiative.org/delphimanager/
    39. ↵
      2. Edwards PJ ,
      3. Roberts I ,
      4. Clarke MJ , et al
      . Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009:MR000008.doi:10.1002/14651858.MR000008.pub4
    40. ↵
      2. Guyatt GH ,
      3. Oxman AD ,
      4. Kunz R , et al
      . Grade guidelines: 2. framing the question and deciding on important outcomes. J Clin Epidemiol 2011;64:395–400.doi:10.1016/j.jclinepi.2010.09.012
      OpenUrlCrossRefPubMed
    41. ↵
      2. Fisher RA
      . Frequency distribution of the values of the correlation coefficient in samples from an Indefinitely large population. Biometrika 1915;10:507–21.doi:10.2307/2331838
      OpenUrlCrossRef
    42. ↵
      1. Harvard Business Review Press ib
      . Running meetings. Boston, MA: Harvard Business Review Press, 2014.
    43. ↵
      1. COIN Steering Group
      . Structured minutes from coin consensus meeting, 2018. Available: http://neoepoch.com/core-outcomes/
    44. ↵
      2. Kirkham JJ ,
      3. Gorst S ,
      4. Altman DG , et al
      . Core outcome Set–STAndards for reporting: the COS-STAR statement. PLoS Med 2016;13:e1002148.doi:10.1371/journal.pmed.1002148
    45. ↵
      2. van ʼt Hooft J ,
      3. Duffy JMN ,
      4. Daly M , et al
      . A core outcome set for evaluation of interventions to prevent preterm birth. Obstet Gynecol 2016;127:49–58.doi:10.1097/AOG.0000000000001195
      OpenUrl
    46. ↵
      2. Allin BSR ,
      3. Hall NJ ,
      4. Ross AR , et al
      . Development of a gastroschisis core outcome set. Arch Dis Child Fetal Neonatal Ed 2019;104:F76–82.doi:10.1136/archdischild-2017-314560
      OpenUrlAbstract/FREE Full Text
    47. ↵
      2. Allin BSR ,
      3. Bradnock T ,
      4. Kenny S , et al
      . NETS 1HD study: development of a Hirschsprung’s disease core outcome set. Arch Dis Child 2017;102:1143–51.doi:10.1136/archdischild-2017-312901
      OpenUrlAbstract/FREE Full Text
    48. ↵
      2. Kelly LE ,
      3. Jansson LM ,
      4. Moulsdale W , et al
      . A core outcome set for neonatal abstinence syndrome: study protocol for a systematic review, parent interviews and a Delphi survey. Trials 2016;17.doi:10.1186/s13063-016-1666-9
    49. ↵
      2. Kirkham JJ ,
      3. Clarke M ,
      4. Williamson PR
      . A methodological approach for assessing the uptake of core outcome sets using ClinicalTrials.gov: findings from a review of randomised controlled trials of rheumatoid arthritis. BMJ 2017;357:j2262.doi:10.1136/bmj.j2262
    50. ↵
      2. Dworkin J ,
      3. Hessel H ,
      4. Gliske K , et al
      . A comparison of three online recruitment strategies for engaging parents. Fam Relat 2016;65:550–61.doi:10.1111/fare.12206
      OpenUrl
    51. ↵
      2. Hall DA ,
      3. Smith H ,
      4. Heffernan E , et al
      . Recruiting and retaining participants in e-Delphi surveys for core outcome set development: evaluating the COMiT'ID study. PLoS One 2018;13:e0201378.doi:10.1371/journal.pone.0201378
    52. ↵
      2. Gargon E ,
      3. Gorst SL ,
      4. Harman NL , et al
      . Choosing important health outcomes for comparative effectiveness research: 4th annual update to a systematic review of core outcome sets for research. PLoS One 2018;13:e0209869.doi:10.1371/journal.pone.0209869
    53. ↵
      2. Boers M ,
      3. Kirwan JR ,
      4. Wells G , et al
      . Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol 2014;67:745–53.doi:10.1016/j.jclinepi.2013.11.013
      OpenUrlCrossRefPubMed
    54. ↵
      1. Critical Path Institute
      . International neonatal Consortium, 2018. Available: https://c-path.org/programs/inc/
    55. ↵
      1. International Consortium for Health Outcomes Measurement
      . International Consortium for health outcomes measurement, 2018. Available: https://www.ichom.org/
    56. ↵
      2. Higgins JPT ,
      3. Green S
      . Cochrane Handbook of systematic reviews of interventions version 5.1. Chichester: Wiley, 2011. http://iclibezp1.cc.ic.ac.uk/login?url=http://onlinelibrary.wiley.com/book/

    Footnotes

    • Twitter @Marianfknight, @carolinemdavey, @NeenaModi1, @DrCGale

    • Contributors CG conceived of this project. JWHW, CG and JMND planned and completed the systematic review of outcomes reported in clinical trials. JWHW, CG and GB planned and completed the systematic review of outcomes identified by patients, parents and other stakeholders. The eDelphi surveys were created and run by JWHW. The results were analysed by JWHW, CG and JMND. MK chaired the consensus meeting. The first draft of the manuscript was written by JWHW; CG, JMND and NMo edited and reviewed the manuscript. It was reviewed, edited and approved by JWHW, CG, EA, IA-M, GB, AG, NJH, MK, JML, CL-D, NMa, LN, JN, AR-L, BW-E and NMo.

    • Funding This study was funded as part of a Medical Research Council (MRC) Clinician Scientist Fellowship awarded to CG (MR/N008405/1), salary support for JWHW from the Portland Hospital and Mason Medical Research Foundation, and unrestricted funds held by NMo. NMa receives part funding from the Department of Health’s NIHR Biomedical Research Centre’s funding scheme at UCLH/UCL. The funding bodies had no involvement in the research or this publication.

    • Competing interests JWHW has received support from Chiesi Pharmaceuticals to attend an educational conference and has received a research grant from Mason Medical Research Foundation. AG has held grants from various manufacturers (Abbott Laboratories, MedImmune) and ventilator manufacturers (SLE). AG has received honoraria for giving lectures and advising various manufacturers (Abbott Laboratories, MedImmune) and ventilator manufacturers (SLE). AG is currently receiving a non-conditional educational grant from SLE. NMa has received consultancy fees from Shire and Novartis and is Chair of the long-term outcomes group for the International Neonatal Consortium, Critical Path Institute. NMo is Director of the Neonatal Data Analysis Unit at Imperial College London. In the last 5 years NMo has served on the Board of Trustees of the Royal College of Paediatrics and Child Health, David Harvey Trust, Medical Women’s Federation and Medact; and is a member of the Nestle Scientific Advisory Board. NMo has received research grants from the British Heart Foundation, Medical Research Council, National Institute of Health Research, Westminster Research Fund, Collaboration for Leadership in Applied Health Research and Care Northwest London, Healthcare Quality Improvement Partnership, Bliss, Prolacta Life Sciences, Chiesi, Shire and HCA International; travel and accommodation expenses from Nutricia, Prolacta, Nestle and Chiesi; honoraria from Ferring Pharmaceuticals and Alexion Pharmaceuticals for contributions to expert advisory boards; and Chiesi for contributing to a lecture programme. CG is part of an international team developing reporting guidance (a CONSORT extension) for clinical trials using cohorts and routinely collected health data. He has received support from Chiesi Pharmaceuticals to attend an educational conference; in the past 5 years he has been investigator on received research grants from Medical Research Council, National Institute of Health Research, Canadian Institutes of Health Research, Department of Health in England, Mason Medical Research Foundation, Westminster Medical School Research Trust and Chiesi Pharmaceuticals.

    • Patient consent for publication Not required.

    • Ethics approval Research ethics approval was not required for this project. The systematic reviews undertaken did not need research ethics approval and the anonymised surveys were completed by consenting adults who opted in in response to adverts.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available upon reasonable request.