You are here

Article Text

Article menu
Original research
Variation in hospital morbidities in an Australian neonatal intensive care unit network
  1. Mohamed E Abdel-Latif1,2,3,
  2. Oyelola Adegboye4,5,
  3. Gen Nowak6,
  4. Faiz Elfaki7,
  5. Barbara Bajuk8,
  6. Kathryn Glass9,
  7. David Harley9,10
  1. 1 Department of Neonatology, Centenary Hospital for Women and Children, Canberra Hospital, Garran, Canberra, ACT, Australia
  2. 2 Department of Public Health, College of Science Health and Engineering, La Trobe University, Bundoora, Melbourne, VIC, Australia
  3. 3 Discipline of Neonatology, School of Medicine and Psychology, College of Health and Medicine, Australian National University, Acton, Canberra, ACT, Australia
  4. 4 Public Health and Tropical Medicine, College of Public Health, Medical and Veterinary Sciences, James Cook University, Townsville, Queensland, Australia
  5. 5 Australian Institute of Tropical Health and Medicine, James Cook University, Townsville, QLD, Australia
  6. 6 Research School of Finance, Actuarial Studies, and Statistics, College of Business and Economics, Australian National University, Acton, Canberra, ACT, Australia
  7. 7 Department of Mathematics, Physics, and Statistics, Qatar University, Doha, Qatar
  8. 8 Critical Care Program, Sydney Children's Hospitals Network, Westmead, Sydney, NSW, Australia
  9. 9 National Centre for Epidemiology and Population Health, Australian National University, Acton, Canberra, ACT, Australia
  10. 10 University of Queensland Centre for Clinical Research (UQCCR), University of Queensland, Brisbane, QLD, Australia
  1. Correspondence to Professor Mohamed E Abdel-Latif, Department of Neonatology, Centenary Hospital for Women and Children, Canberra Hospital, Garran, Canberra, ACT, Australia; abdel-Latif.Mohamed{at}act.gov.au

Abstract

Objective There is an expectation among the public and within the profession that the performance and outcome of neonatal intensive care units (NICUs) should be comparable between centres with a similar setting. This study aims to benchmark and audit performance variation in a regional Australian network of eight NICUs.

Design Cohort study using prospectively collected data.

Setting All eight perinatal centres in New South Wales and the Australian Capital Territory, Australia.

Patients All live-born infants born between 23+0 and 31+6 weeks gestation admitted to one of the tertiary perinatal centres from 2007 to 2020 (n=12 608).

Main outcome measures Early and late confirmed sepsis, intraventricular haemorrhage, medically and surgically treated patent ductus arteriosus, chronic lung disease (CLD), postnatal steroid for CLD, necrotising enterocolitis, retinopathy of prematurity (ROP), surgery for ROP, hospital mortality and home oxygen.

Results NICUs showed variations in maternal and neonatal characteristics and resources. The unadjusted funnel plots for neonatal outcomes showed apparent variation with multiple centres outside the 99.8% control limits of the network values. The hierarchical model-based risk-adjustment accounting for differences in patient characteristics showed that discharged home with oxygen is the only outcome above the 99.8% control limits.

Conclusions Hierarchical model-based risk-adjusted estimates of morbidity rates plotted on funnel plots provide a robust and straightforward visual graphical tool for presenting variations in outcome performance to detect aberrations in healthcare delivery and guide timely intervention. We propose using hierarchical model-based risk adjustment and funnel plots in real or near real-time to detect aberrations and start timely intervention.

  • Neonatology
  • Intensive Care Units, Neonatal
  • Health services research

Data availability statement

Data may be obtained from a third party and are not publicly available. All data relevant to the study are included in the article or uploaded as supplementary information. The data for this study were extracted from The Neonatal Intensive Care Units (NICUs) Data Registry, an ongoing prospective statewide audit of infants admitted to the 10 NICUs in NSW and the ACT during the neonatal period. The data are available on reasonable request from the data custodian, NICUS Data Management Committee, Neonatal Intensive Care Units’ Data Registry, Agency for Clinical Innovation, NSW, Australia. All data relevant to the study are included in the article or uploaded as supplementary information.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/archdischild-2022-324940

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • The characteristics of infants admitted to neonatal intensive care units differ, so comparing unadjusted morbidity rates should be avoided.

    WHAT THIS STUDY ADDS

    • Variations in hospital morbidities estimates plotted on funnel plots provide a powerful visual graphical tool for presenting quality performance data.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • Prospective and timely hierarchical model-based risk adjustment for centre-to-centre variation in morbidities is a useful method to inform hospitals to readily appraise their practices and start timely intervention.

    Background

    There is an increasing need for accurate patient quality, safety and hospital performance measures in healthcare. The public and health professionals expect the performance and outcome of neonatal services should be comparable between centres of similar settings. However, there are challenges in meeting these expectations.1

    Centre-to-centre (CTC) variation in neonatal health outcomes may result from patient characteristic differences (intrinsic factors) rather than centre or service differences (extrinsic factors). Outcome variations related to intrinsic and extrinsic factors are called common-cause and special-cause variations, respectively.2 Differentiating these sources of variability is critical to service improvement.

    We previously published the risk-adjusted CTC variation in mortality rates for preterm infants admitted to New South Wales (NSW) and the Australian Capital Territory (ACT) Neonatal Network (NICUS), Australia.3 This study aims to report the risk-adjusted CTC variation in major neonatal morbidities for infants born <32 weeks and admitted to the eight tertiary neonatal intensive care units (NICUs) in NSW and the ACT Neonatal Network. We also assess the benefits of adjusting CTC variation for population characteristics using hierarchical model-based risk adjustment.

    Methods

    Study design

    This prospective population-based cohort study uses data from all tertiary NICUs in well-defined geographic regions of NSW and the ACT.

    Study centres and network

    A full description of the NSW and the ACT neonatal service organisation and networking, medical and nursing staffing of the collaborating NICUs is available elsewhere.4–6 In summary, there is a network of 10 units within NSW and the ACT. These include eight perinatal centres (referred to as A to H in this study) and two children’s hospitals. Among the perinatal centres, three units (C, G and H) have surgical capabilities to operate on conditions like necrotising enterocolitis (NEC) and diaphragmatic hernia.

    Coordination of in-utero or ex-utero high-risk referrals among the network is assisted by an intranet bed availability bulletin board, redirecting referrals when any particular unit is full or nearly full to reduce the risk of overloading. The NSW Neonatal and Paediatric Emergency Transport Service (NETS) is an integrated centralised transport service covering NSW and the ACT which coordinates the transfer of sick infants and children from non-tertiary to tertiary centres as well as surgical cases from non-surgical (A, B, D, E, F) to surgical units (C, G, H).7 Retrieved premature infants (outborns) are preferentially admitted to the eight perinatal centres instead of the two paediatric hospitals.6 8

    Study participants

    The study population comprised all live-born infants born between 23+0 and 31+6 weeks gestation who were admitted to one of the eight tertiary perinatal centres in NSW and the ACT from 2007 to 2020. As of December 2020, NSW and the ACT had a population of 8 599 314 and approximately 99 752 live births per year.9

    Data source

    Data for this study were obtained from The Neonatal Intensive Care Units' Data Registry, which is an ongoing prospective statewide audit of infants admitted to the 10 units (8 perinatal centres and 2 children’s hospitals) for one of the following reasons: gestation 22+0 to 31+6 weeks, birth weight ≤1500 g, assisted ventilation (mechanical ventilation, continuous positive airway pressure, high flow humidified gas), major surgery (opening of a body cavity), insertion of a central line, exchange transfusion for hyperbilirubinemia or therapeutic hypothermia. In this region and according to the National Health and Medical Research Council recommendations, wherever possible preterm birth at <33 weeks should occur in one of the eight perinatal centres.10 Preterm infants <33 weeks who are born in non-tertiary hospitals are transferred to tertiary centres by NETS.7

    Data from the two children’s hospitals (n=24) were excluded from this study due to the low patient load, as retrieved premature infants (outborns) were preferentially admitted to the eight perinatal centres instead of the two paediatric hospitals.3 This means that the outcome of premature infants in these two hospitals may be affected because of this policy, as shown elsewhere.6 8

    Definitions

    NICUS data definitions and data accuracy have been described elsewhere.11–13 Chronic lung disease (CLD) was defined as the requirement for respiratory support at 36 weeks postmenstrual age.14 Intraventricular haemorrhage (IVH) was graded I–IV by Papile’s classification;15 NEC was staged according to Bell’s classification;16 retinopathy of prematurity (ROP) was staged I–V according to international criteria.17 18 The details laser therapy for ROP ‘Surgery for ROP’ can be found elsewhere.19 Patent ductus arteriosus (PDA) was diagnosed in infants with ‘clinical evidence of left to right shunt documented by continuous murmur, hyperdynamic precordium, bounding pulses, wide pulse pressure, congestive heart failure, increased pulmonary vasculature or cardiomegaly by chest x-ray and/or increased oxygen requirement or ECHO evidence of PDA with documentation of left to right ductal shunting’.20 PDA pharmacological (medical) and surgical management protocols differ between centres.21 Proven sepsis is defined as a clinical picture consistent with sepsis and either a positive bacterial, viral or fungal culture of blood and/or cerebrospinal fluid occurring less than 48 hours from birth (early) or from 48 hours after birth (late).22 Infections with coagulase-negative staphylococci, and other potential contaminants, were included only if the baby was considered clinically septic and there was supporting evidence such as raised white cell count or thrombocytopenia.

    Primary outcome measures

    We selected major neonatal morbidities to be benchmarked. These are shown in online supplemental tables 1–12.

    Statistical analysis

    Statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, North Carolina, USA). Categorical variables were described by frequencies and percentages, while continuous variables were presented as medians and quartiles (25th and 75th percentile).

    We used hierarchical model-based risk adjustment to estimate risk-adjusted CTC variation in major neonatal morbidities for infants born <32 weeks and admitted to the eight NICUs after adjusting for case-mix and the random effect of the centre using the steps below.

    First, we used a multivariable Poisson model to control for antenatal and perinatal variables other than intermediate variables (eg, CLD, IVH, NEC, ROP), as these may be related, directly or indirectly, to the quality of the hospital management and might thus act as intermediate comorbidities through which the effect of the ‘hospital’ is mediated.23 For each outcome/morbidity, the model was used to estimate the expected and predicted risk of morbidity for each patient. The level of statistical significance for model selection was based on a 5% level of significance through a stepwise variable selection approach. The calibration of the model was determined by the Hosmer-Lemeshow goodness-of-fit χ2 test.24 The ability of the model to discriminate between those who had versus those who did not have the morbidity was summarised using the C-statistic. A C-statistic of 0.5 indicates that the model discriminates no better than chance alone, whereas a value of 1.0 indicates perfect discrimination.25 26

    Second, risk-adjusted standardised ratios (RAR) for each outcome/morbidity within each hospital were then computed as the ratio of predicted-to-expected hospital morbidity multiplied by the network’s observed rate.

    Third, a risk-adjusted standardised incidence ratio (RSIR) was produced by dividing the hospital crude rate by RAR. The 95% CI for RSIR were computed using Monte Carlo simulation as described elsewhere.27

    Last, we used funnel plots to provide a visual indication to differentiate between common-cause and special-cause variation in risk-adjusted hospital morbidity among NICUs.28 These plots indicate whether morbidity rates in a NICU differ significantly from the average network rate, assuming only random sampling variation influences the NICU’s rate. A solid horizontal line represents the overall network morbidity rate while the 95% (2.5th percentile represents the lower control limit and 97.5th percentile represents the upper control limit) and 99.8% (0.1th percentile represents the lower control limit, and 99.9th percentile represents the upper control limit) control limits are represented by the curved dotted lines. Assuming differences arise from random sampling variation alone, the chance of the hospital being within limits is 95% for the inner funnels and 99.8% for the outer funnel.

    Results

    Study population

    A total of 12 608 live-born infants <32+0 weeks gestation were admitted to one of the eight tertiary perinatal centres during the study period. The maternal and neonatal characteristics of the study group stratified by admitting hospital are presented in tables 1 and 2. There is variation in patient characteristics (intrinsic factors) between the admitting hospitals. The median (25th and 75th percentile) age of mothers in this study was 31.0 (26.0–35.0) years. The percentage of Indigenous Australians was 7.3% and ranged from 3% (hospital F) to 14.3% (hospital G) (table 1). Three of the eight hospitals have onsite surgical support (table 2). The median (25th and 75th percentile) length of hospital stay among neonatal centres was relatively homogenous, ranging from 51.2 (37.5–71.8) in hospital G to 58.9 (42.9–83.1) in hospital C.

    View this table:
    Table 1

    Maternal characteristics of the study group stratified by admitting hospital A to H

    View this table:
    Table 2

    Neonatal characteristics of the study group stratified by admitting hospital A to H

    Major neonatal morbidities

    Table 3 presents the observed (unadjusted) and hierarchical model-based risk-adjustment estimates of twelve major neonatal morbidities across eight hospitals.

    View this table:
    Table 3

    Major neonatal morbidities of the study group stratified by admitting hospital A to H

    The risk-adjusted estimates presented in table 3 were plotted as funnel plots (online supplemental figures 1–12) to visualise the unadjusted and adjusted estimates. Centres above and below the limits likely indicate special-cause variation, whereas centres within limits indicate common-cause variation. The unadjusted morbidities indicate an apparent variation between the hospitals, especially for NEC, postnatal steroids for CLD, ROP grade III to V, surgery for ROP, hospital mortality and home oxygen. The estimates for these health outcomes were outside the 99.8% control limits of the network values. With 6.35%, hospital H recorded the highest prevalence rate among the network of hospitals in the study for NEC, which is above the upper 99.8% control limit (online supplemental figure 6A). While the overall prevalence of postnatal steroid administration for CLD was 6.96%, hospitals C (9.68%) and F (9.75%) recorded estimates higher than the upper 99.8% control limit of the network prevalence rate (online supplemental figure 8A). There was an overall prevalence of 3.2% for surgery for ROP, with hospital F having a prevalence rate of 4.79%, which is above the upper 99.8% control limit (online supplemental figure 9A).

    To accurately estimate various morbidities, it is essential to consider each hospital’s patient profile and adjust accordingly. Online supplemental tables 1–12 present the patient characteristics incorporated into the hierarchical model-based risk adjustment for models. After accounting for differences in patient characteristics, the resulting risk-adjusted estimates are presented in table 3 and online supplemental figures 1–12 (panel B). In contrast to unadjusted rates, discharge home with oxygen is the only outcome for which some hospitals lie above the 99.8% control limits after adjusting for hospital-level patient characteristics (online supplemental figure 12B). For hospitals B and G, the estimates were above the upper 99.8% control limit, while for hospitals A, E and F, the estimates were below the lower 99.8% control limit.

    Discussion

    We have presented benchmarking for major neonatal morbidities in NSW and the ACT. We used hierarchical model-based risk adjustment rather than the traditional logistic regression for case-mix adjustment. Hierarchical model-based risk adjustments have been shown to have the additional benefit of adjusting for centre sample size and clustering issues and avoiding overestimation of intercentre variability and consequent false outlier classification.27 29

    Of note is that hierarchical and other statistical models do not provide direct guidance on improving quality despite flagging areas for further investigation.30 However, combining this robust statistical modelling with funnel plots is helpful in providing a systematic structure for quality improvement, as discussed below.

    From the adjusted funnel plots with the upper and lower control limits, the eight perinatal centres could be divided into three categories with guidance for appropriate action:2 31 32

    • Category 1, positive special-cause variation: performance of these centres is below the lower control limit. Lessons could be learnt from these centres to improve the performance of other centres.

    • Category 2, common-cause variation: performance is within the control limits. This is most likely the result of factors intrinsic to the centres. The reduction of common-cause variation requires a fundamental change in the underlying process.22 This should be informed by lessons learnt from Category 1. There are no grounds for acting in individual centres in this group.

    • Category 3, negative special-cause variation: performance is above the upper control limit. This is most likely the result of factors extrinsic to the centres, and its reduction requires identification of and action on the special causes.22 These centres need to identify and eliminate the special causes of their poorer results. Again, this should be informed by lessons learnt from Category 1.

    Generally, variations in outcome rates among hospitals may be caused by measurement inaccuracy in assessing the outcome, differences in case mix, sampling variability or differences in hospital clinical practices.25 26 Our hierarchical model-based risk-adjusted approach adjusts for variation arising from differences in the case-mix. In our data, we used prospectively collected statewide data using standardised definitions to preclude any data inaccuracy and sampling variations.

    A proposed strategy to examine the special-cause variation is the pyramid investigation model.33 This model checks five variables: data accuracy, patient case mix, structure and the process of care and carers.2 33 Identifying the exact causes of common-cause and special-cause variation between NICUs in NSW and the ACT needs further research.

    Our study is not without limitations. We adjusted for selected variables, but there may be others that were not collected in the database. The transfer pattern of outborn infants may disadvantage certain hospitals with a higher proportion of outborns. However, our hierarchical model adjusted for 'outborn' and other factors.

    Our analysis demonstrated the utility of adjusted funnel plots for effectively identifying NICUs with high morbidity rates that may require intervention. Similar applications have been shown to improve quality and detect aberrations elsewhere in healthcare settings.2 34 These methods have been in use in the manufacturing industry since the mid-1900s and have greatly improved the quality of products.22 Adjusted funnel plots have two major advantages. First, they focus on the centre that fails relative to the best centre so, enabling a systematic approach to guide improvement.2 34 35 Second, they can be employed in real or semi-real-time to detect aberrations early and act promptly. The method is generalisable for evaluation and performance improvement for NICUs and other similar healthcare settings.

    Data availability statement

    Data may be obtained from a third party and are not publicly available. All data relevant to the study are included in the article or uploaded as supplementary information. The data for this study were extracted from The Neonatal Intensive Care Units (NICUs) Data Registry, an ongoing prospective statewide audit of infants admitted to the 10 NICUs in NSW and the ACT during the neonatal period. The data are available on reasonable request from the data custodian, NICUS Data Management Committee, Neonatal Intensive Care Units’ Data Registry, Agency for Clinical Innovation, NSW, Australia. All data relevant to the study are included in the article or uploaded as supplementary information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    The Australian National University (No 2015/029), the ACT Health Human Research Ethics Committee (No ETH.11.09/1031) and Hunter New England Research Ethics and Governance Unit have approved this study. The Ethics Committee waived the requirement for individual patient consent.

    Acknowledgments

    The authors thank the Directors, the NICUS members and the audit officers of all tertiary units in supporting this collaborative study: NICUS, Dr Himanshu Popat (Chairperson), Barbara Bajuk (Coordinator), Sara Sedgley (CNS); Canberra Hospital, Dr Hazel Carlisle (Director), Melanie Rosin; John Hunter Children's Hospital, Dr Larissa Korostenski (Director), Alissa Argomand, Stacey Leonard; Royal Prince Alfred Hospital, Dr Mark Greenhalgh (Director), Shelley Reid; Liverpool Hospital, Dr Jacqueline Stack (Director), Dr Ian Callander, Kathryn Medlin, Amanda Beesley; Nepean Hospital, Dr Lyn Downe, Mee Fong Chin; The Children's Hospital at Westmead, Professor Nadia Badawi (Director), Karina Rogers; Royal North Shore Hospital, Dr Mary Paradisis (Director), Professor Martin Kluckow, Lyn Barnes; Sydney Children's Hospital, Dr Andrew Numa (Director), Dr Gary Williams, Janelle Young; Westmead Hospital, Dr Melissa Luig (Director), Jane Baird, Gemma Lowe and Royal Hospital for Women, Dr Srinivas Bolisetty (Director), Professor Julee Oei, Diane Cameron. We also thank the babies and their families, the nursing and midwifery, obstetric and medical records staff of the obstetric and children's hospitals in NSW and the ACT.

    References

    1. ↵
      2. Field D ,
      3. Manktelow B ,
      4. Draper ES
      . Bench marking and performance management in neonatal care: easier said than done! Arch Dis Child Fetal Neonatal Ed 2002;87:163F–4.doi:10.1136/fn.87.3.F163
      OpenUrlFREE Full Text
    2. ↵
      2. Mohammed MA ,
      3. Cheng KK ,
      4. Rouse A , et al
      . Bristol, shipman, and clinical governance: shewhart's forgotten lessons. Lancet 2001;357:463–7.doi:10.1016/S0140-6736(00)04019-8 pmid:http://www.ncbi.nlm.nih.gov/pubmed/11273083
      OpenUrlCrossRefPubMedWeb of Science
    3. ↵
      2. Abdel-Latif ME ,
      3. Nowak G ,
      4. Bajuk B , et al
      . Variation in hospital mortality in an Australian neonatal intensive care unit network. Arch Dis Child Fetal Neonatal Ed 2018;103:F331–6. doi:10.1136/archdischild-2017-313222 pmid:29074720
      OpenUrlAbstract/FREE Full Text
    4. ↵
      2. Abdel-Latif ME ,
      3. Bajuk B ,
      4. Oei J , et al
      . Mortality and morbidities among very premature infants admitted after hours in an Australian neonatal intensive care unit network. Pediatrics 2006;117:1632–9.doi:10.1542/peds.2005-1421 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16651317
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. Abdel-Latif ME ,
      3. Bajuk B ,
      4. Oei J , et al
      . Does rural or urban residence make a difference to neonatal outcome in premature birth? A regional study in Australia. Arch Dis Child Fetal Neonatal Ed 2006;91:F251–6.doi:10.1136/adc.2005.090670 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16428354
      OpenUrlAbstract/FREE Full Text
    6. ↵
      2. Lui K ,
      3. Abdel-Latif ME ,
      4. Allgood CL , et al
      . Improved outcomes of extremely premature outborn infants: effects of strategic changes in perinatal and retrieval services. Pediatrics 2006;118:2076–83.doi:10.1542/peds.2006-1540 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17079581
      OpenUrlCrossRefPubMedWeb of Science
    7. ↵
      2. Abdel-Latif ME ,
      3. Berry A
      . Analysis of the retrieval times of a centralised transport service, New South Wales, Australia. Arch Dis Child 2009;94:282–6.doi:10.1136/adc.2007.125211 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18927147
      OpenUrlAbstract/FREE Full Text
    8. ↵
      2. Shah PS ,
      3. Shah V ,
      4. Qiu Z , et al
      . Improved outcomes of outborn preterm infants if admitted to perinatal centers versus freestanding pediatric hospitals. J Pediatr 2005;146:626–31.doi:10.1016/j.jpeds.2005.01.030 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15870665
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. Australian Bureau of Statistics
      . National, state and territory population, 2021. Available: https://www.abs.gov.au/statistics/people/population [Accessed 06 Jan 2022].
    10. ↵
      1. National Health and Medical Research Council
      . Clinical practice guidelines: care around preterm birth. Canberra, Australia: National Health and Medical Research Council, 1997.
    11. ↵
      1. Australian and New Zealand Neonatal Network
      . ANZNN 2022 data dictionary. Sydney, Australia, 2021. https://anznn.net/dataresources/datadictionaries
    12. ↵
      2. Bajuk B
      . Validation of the neonatal intensive care units' data collection. Proceedings of the 5th Annual Conference of the Perinatal Society of Australia and New Zealand; The Perinatal Society of Australia and New Zealand, Canberra, ACT, Australia, 2001.
    13. ↵
      2. Bajuk B
      . Neonatal Intensive Care 'Units' (NICUS) Data Registry Definitions. Randwick, New South Wales, Australia, 2020.
    14. ↵
      2. Shennan AT ,
      3. Dunn MS ,
      4. Ohlsson A , et al
      . Abnormal pulmonary outcomes in premature infants: prediction from oxygen requirement in the neonatal period. Pediatrics 1988;82:527–32.doi:10.1542/peds.82.4.527 pmid:http://www.ncbi.nlm.nih.gov/pubmed/3174313
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      2. Papile LA ,
      3. Burstein J ,
      4. Burstein R , et al
      . Incidence and evolution of subependymal and intraventricular hemorrhage: a study of infants with birth weights less than 1,500 GM. J Pediatr 1978;92:529–34.doi:10.1016/S0022-3476(78)80282-0 pmid:http://www.ncbi.nlm.nih.gov/pubmed/305471
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      2. Bell MJ ,
      3. Ternberg JL ,
      4. Feigin RD , et al
      . Neonatal necrotizing enterocolitis. therapeutic decisions based upon clinical staging. Ann Surg 1978;187:1–7.doi:10.1097/00000658-197801000-00001 pmid:http://www.ncbi.nlm.nih.gov/pubmed/413500
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Flynn JT
      . An international classification of retinopathy of prematurity: clinical experience. Trans Am Ophthalmol Soc 1984;82:218–38.pmid:http://www.ncbi.nlm.nih.gov/pubmed/6549514
      OpenUrlPubMed
    18. ↵
      1. International Committee for the Classification of Retinopathy of Prematurity
      . The International classification of retinopathy of prematurity revisited. Arch Ophthalmol 2005;123:991–9.doi:10.1001/archopht.123.7.991 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16009843
      OpenUrlCrossRefPubMedWeb of Science
    19. ↵
      2. Brown KA ,
      3. Heath Jeffery RC ,
      4. Bajuk B , et al
      . Sight-threatening retinopathy of prematurity: changing trends in treatment. J Pediatr Ophthalmol Strabismus 2016;53:90–5.doi:10.3928/01913913-20160122-03 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27018882
      OpenUrlCrossRefPubMed
    20. ↵
      2. Evans N
      . Current controversies in the diagnosis and treatment of patent ductus arteriosus in preterm infants. Adv Neonatal Care 2003;3:168–77.doi:10.1016/S1536-0903(03)00143-7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/14502524
      OpenUrlCrossRefPubMed
    21. ↵
      2. Hoellering AB ,
      3. Cooke L
      . The management of patent ductus arteriosus in Australia and New Zealand. J Paediatr Child Health 2009;45:204–9.doi:10.1111/j.1440-1754.2008.01461.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/19291140
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      2. Isaacs D ,
      3. Barfield CP ,
      4. Grimwood K , et al
      . Systemic bacterial and fungal infections in infants in Australian neonatal units. Australian study group for neonatal infections. Med J Aust 1995;162:198–201.doi:10.5694/j.1326-5377.1995.tb126024.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/7877542
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      2. Abdel-Latif ME ,
      3. Bajuk B ,
      4. Oei J , et al
      . Population study of neurodevelopmental outcomes of extremely premature infants admitted after office hours. J Paediatr Child Health 2014;50:E45–54.doi:10.1111/jpc.12028 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23252772
      OpenUrlCrossRefPubMed
    24. ↵
      2. Lemeshow S ,
      3. Hosmer DW
      . A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92–106.doi:10.1093/oxfordjournals.aje.a113284 pmid:http://www.ncbi.nlm.nih.gov/pubmed/7055134
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      2. Kleinbaum DG
      . Logistic regression: a self-learning text. New York: Springer-Verlag, 1994.
    26. ↵
      2. Hosmer DW ,
      3. Lemeshow S
      . Applied logistic regression. 2nd. New York: Wiley, 2000.
    27. ↵
      2. Austin PC ,
      3. Alter DA ,
      4. Tu JV
      , . The use of fixed- and random-effects models for classifying hospitals as mortality outliers: a monte carlo assessment. Med Decis Making 2003;23:526–39.doi:10.1177/0272989X03258443 pmid:http://www.ncbi.nlm.nih.gov/pubmed/14672113
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      2. Spiegelhalter DJ
      . Funnel plots for comparing institutional performance. Stat Med 2005;24:1185–202.doi:10.1002/sim.1970 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15568194
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      2. Shahian DM ,
      3. Torchiana DF ,
      4. Shemin RJ , et al
      . Massachusetts cardiac surgery report card: implications of statistical methodology. Ann Thorac Surg 2005;80:2106–13.doi:10.1016/j.athoracsur.2005.06.078 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16305853
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. McNeil BJ ,
      3. Pedersen SH ,
      4. Gatsonis C
      . Current issues in profiling quality of care. Inquiry 1992;29:298–307.pmid:http://www.ncbi.nlm.nih.gov/pubmed/1398901
      OpenUrlPubMedWeb of Science
    31. ↵
      2. Mohammed MA ,
      3. Worthington P ,
      4. Woodall WH
      . Plotting basic control charts: tutorial notes for healthcare practitioners. Qual Saf Health Care 2008;17:137–45.doi:10.1136/qshc.2004.012047 pmid:http://www.ncbi.nlm.nih.gov/pubmed/18385409
      OpenUrlAbstract/FREE Full Text
    32. ↵
      2. Benneyan JC ,
      3. Lloyd RC ,
      4. Plsek PE
      . Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003;12:458–64.doi:10.1136/qhc.12.6.458 pmid:http://www.ncbi.nlm.nih.gov/pubmed/14645763
      OpenUrlAbstract/FREE Full Text
    33. ↵
      2. Mohammed MA ,
      3. Rathbone A ,
      4. Myers P , et al
      . An investigation into general practitioners associated with high patient mortality flagged up through the shipman inquiry: retrospective analysis of routine data. BMJ 2004;328:1474–7.doi:10.1136/bmj.328.7454.1474 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15205291
      OpenUrlAbstract/FREE Full Text
    34. ↵
      2. Berwick DM
      . Controlling variation in health care: a consultation from walter shewhart. Med Care 1991;29:1212–25.doi:10.1097/00005650-199112000-00004 pmid:http://www.ncbi.nlm.nih.gov/pubmed/1745079
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      2. Goldstein H ,
      3. Spiegelhalter DJ
      . League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Soc Ser A Stat Soc 1996;159:385.doi:10.2307/2983325

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @Latifme1

    • Contributors MEA-L: conceptualised and designed the study, contributed to statistical analyses and interpretation, drafted the initial manuscript, led the writing and reviewed and revised the manuscript. OA and FE: designed the statistical plan, carried out the initial analyses and critically reviewed and revised the manuscript. GN: contributed to the statistical interpretation, critically reviewed the statistical methods, analysis and interpretation and critically reviewed and revised the manuscript. BB: coordinated and supervised data collection, retrieved and cleaned the data, contributed to statistical interpretation and critically reviewed and revised the manuscript. KG: contributed to the statistical interpretation critically and reviewed and revised the manuscript. DH: contributed to the study design and statistical interpretation and critically reviewed and revised the manuscript. All authors approved the final manuscript as submitted. MEA-L and OA had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.