Variation in hospital mortality in an Australian neonatal intensive care unit network
=====================================================================================

* Mohamed E Abdel-Latif
* Gen Nowak
* Barbara Bajuk
* Kathryn Glass
* David Harley

## Abstract

**Background** Studying centre-to-centre (CTC) variation in mortality rates is important because inferences about quality of care can be made permitting changes in practice to improve outcomes. However, comparisons between hospitals can be misleading unless there is adjustment for population characteristics and severity of illness.

**Objective** We sought to report the risk-adjusted CTC variation in mortality among preterm infants born <32 weeks and admitted to all eight tertiary neonatal intensive care units (NICUs) in the New South Wales and the Australian Capital Territory Neonatal Network (NICUS), Australia.

**Methods** We analysed routinely collected prospective data for births between 2007 and 2014. Adjusted mortality rates for each NICU were produced using a multiple logistic regression model. Output from this model was used to construct funnel plots.

**Results** A total of 7212 live born infants <32 weeks gestation were admitted consecutively to network NICUs during the study period. NICUs differed in their patient populations and severity of illness.

The overall unadjusted hospital mortality rate for the network was 7.9% (n=572 deaths). This varied from 5.3% in hospital E to 10.4% in hospital C. Adjusted mortality rates showed little CTC variation. No hospital reached the +99.8% control limit level on adjusted funnel plots.

**Conclusion** Characteristics of infants admitted to NICUs differ, and comparing unadjusted mortality rates should be avoided. Logistic regression-derived risk-adjusted mortality rates plotted on funnel plots provide a powerful visual graphical tool for presenting quality performance data. CTC variation is readily identified, permitting hospitals to appraise their practices and start timely intervention.

*   premature
*   mortality
*   variation
*   benchmarking

### What is already known on this topic?

Characteristics of infants admitted to neonatal intensive care units differ, and comparing unadjusted mortality rates should be avoided.

### What this study adds?

*   Logistic regression derived risk-adjusted mortality rates plotted on funnel plots provide a powerful visual graphical tool for presenting quality performance data.

*   Centre-to-centre variation is readily identified, permitting hospitals to appraise their practices and start timely intervention.

## Background

Advances in perinatal and neonatal care have significantly reduced neonatal mortality rates, especially among preterm and low birth weight infants.1 However, there is significant centre-to-centre (CTC) variation in mortality rates among neonatal intensive care units (NICUs).2–12 Studying CTC variation in mortality rates is important because inferences about quality of care can be made which permit changes in practice to improve outcomes.13 However, comparisons between hospitals can be misleading unless there is adjustment for differences in population characteristics and severity of illness.

Our aim was to report the risk-adjusted CTC variation in mortality rates for infants born at <32 weeks and admitted to all eight tertiary NICUs in the New South Wales (NSW) and the Australian Capital Territory (ACT) Neonatal Network (NICUS), Australia. Furthermore, we would like to show the benefits of adjusting CTC variation for population characteristics. NSW is the most populous state of Australia,14 and the ACT, home to the Government of the Commonwealth, is within NSW.

## Methods

We analysed prospectively collected data from a geographically defined area. All infants who were delivered before 32 weeks of gestational age and admitted to one of the eight tertiary perinatal centres in NSW and the ACT between 1 January 2007 and 31 December 2014 were included in the study. Full descriptions of the NSW and ACT neonatal service organisation and networking, medical and nursing staffing of the collaborating NICUs are available elsewhere.15–18 Geographically, the ACT lies within NSW and the state and territory combined have a total population of 7904500 with approximately 106 007 live births per year.19 

## Data source

Data for this study were obtained from The Neonatal Intensive Care Units’ Data Collection, which is an ongoing prospective state and territory wide audit of infants admitted to all eight perinatal centres and two children’s hospitals in NSW and the ACT for one of the following reasons: (1) gestation 22+0 to 31+6 weeks; (2) birth weight ≤1500 g; (3) assisted ventilation (mechanical ventilation, continuous positive airway pressure, high flow humidified gas); (4) major surgery (opening of a body cavity); and/or (5) insertion of a central line, exchange transfusion and/or therapeutic hypothermia. Definitions and accuracy of the database have been described elsewhere.20 In Australia, it has been recommended by the National Health and Medical Research Council that wherever possible preterm birth at <33 weeks should occur in a perinatal centre that has the expertise to care for the woman and her preterm infant.21 In general, preterm infants born at <32 weeks in non-tertiary hospitals are transferred to tertiary centres by means of the Newborn and Paediatric Emergency Transport Service, a transport organisation that functions across the NSW/ACT region.16 17 Data from two tertiary children’s hospitals were not included in this study due to the low patient load (n=11) as retrieved premature infants were preferentially admitted to the eight perinatal centres instead of the two paediatric hospitals.18 

### Outcome measure

The study outcome was hospital mortality (death before discharge home). The principal causes of death were coded using the International Classification of Diseases and Related Health Problems, 10th Revision, Australian Modification.22 

Because some conditions, seizures for example, are rarely the principal cause of death, we used a composite neonatal adverse outcome indicator (NAOI). NAOI has been validated previously.23 

### Statistical analysis

Statistical analyses were performed using *SPSS* (IBM, IBM SPSS Statistics for Windows, V22.0.0.0. Released 2013).

We performed a stepwise multiple logistic regression elimination analysis to establish the independent risk factors associated with mortality, after controlling for *significant* confounding factors (eg, ethnicity) in addition to other predictors (eg, gestational age) identified in other studies. In this multivariate model, we controlled for antenatal and perinatal variables but not *intermediate* variables (eg, intraventricular haemorrhage, retinopathy of prematurity, necrotising enterocolitis, chronic lung disease), as these may be related, directly or indirectly, to the quality of the management and might thus act as *intermediate* comorbidities through which the effect of ‘hospital’ is mediated.24 

We used the multiple logistic regression to estimate the probability of hospital mortality after adjusting for potential confounding factors. Calibration was determined by means of Hosmer-Lemeshow goodness-of-fit χ2 test (high p value suggests better classification).25 The ability of the logistic regression model to discriminate between those who died in hospital and those who survived until discharge was summarised using the C-statistic. A C-statistic of 0.5 indicates that the model discriminates no better than chance alone, whereas a value of 1.0 indicates perfect discrimination.26 27 

The probabilities of death within an NICU were summed to give the hospital’s expected rate of death. Risk-adjusted standardised mortality ratios (SMRs) for each hospital were then produced by dividing the observed number of hospital deaths by the expected number produced by the logistic regression model. The 95% CIs for SMR were calculated by using Miettinen’s modification as described by Rothman *et al*.28 The SMRs were used to calculate the adjusted mortality rates for each hospital.

We used funnel plots to provide visual indication to differentiate between two sources of variation, namely common-cause and special-cause variation, of risk-adjusted hospital mortality among NICUs.29 These plots test whether the rate of mortality of an NICU differs significantly from the network average rate, assuming only random sampling variation rather than bias influences the NICU’s rate. The overall network mortality rate is represented by a solid horizontal line while the ±95% (2σ) and ±99.8% (3σ) control limits are represented by the curved dotted lines. Assuming differences arise from random sampling variation alone, the chance of the hospital being within the limits is 95% for the inner funnel and 99.8% for the outer funnel.

The level of statistical significance for all analyses was set at p<0.05 using two-tailed comparisons. The significance level was not changed when multiple comparisons were performed.

## Results

A total of 7212 live born infants <32+0 weeks gestation were admitted consecutively to one of the eight tertiary perinatal centres during the study period. Maternal and neonatal characteristics stratified by admitting hospital are shown in tables 1 and 2.

View this table:
[Table 1](http://fn.bmj.com/content/103/4/F331/T1)

Table 1 
Maternal characteristics of the study group stratified by admitting hospital A to H

View this table:
[Table 2](http://fn.bmj.com/content/103/4/F331/T2)

Table 2 
Neonatal characteristics of the study group stratified by admitting hospital A to H

The proportion of pregnancies and deliveries involving maternal hypertension, multiple birth, caesarean section and Apgar score <7 at 5 min varied between hospitals. Antenatal steroid was administered to 84.9% to 94.0% of neonates (table 1). Overall, 10.4% of the study population was born in non-tertiary hospitals (outborn). Median gestation at birth and birth weight were similar between hospitals ranging from 27 to 31 completed weeks and 910.0 to 1559.0 g, respectively. Use of surfactant and postnatal steroid showed higher variability between hospitals (table 2).

### Observed and expected mortality and SMR

The observed and expected mortality and SMR stratified by admitting hospital are shown in table 3. The overall hospital mortality rate was 7.9% (n=572 deaths), and SMR was 1.02 (95% CI 0.94 to 1.11) (table 3). Hospital mortality rate varied from 5.3% in hospital E to 10.4% in hospital C. Similarly, expected mortality, SMR and risk-adjusted mortality varied between hospitals (table 3).

View this table:
[Table 3](http://fn.bmj.com/content/103/4/F331/T3)

Table 3 
Neonatal mortality of the study group stratified by admitting hospital A to H

Figure 1 shows the observed mortality rates for all admitting hospitals stratified by gestational age at birth. Hospital C and E exceeded the +99.8% and −99.8% limits, respectively (figure 1A). When mortality was stratified by gestation, hospital C approached the +95% control limit for infants born at 23–26 weeks of gestation (figure 1B).

![Figure 1](http://fn.bmj.com/https://fn.bmj.com/content/fetalneonatal/103/4/F331/F1.medium.gif)

[Figure 1](http://fn.bmj.com/content/103/4/F331/F1)

Figure 1 
Funnel plot showing observed mortality rates in neonatal intensive care units in New South Wales and Australian Capital Territory between 2007 and 2014 for infants born at 23 to 31 (A), 23 to 26 (B), 27 to 28 (C) and 29 to 31 (D) weeks of gestation. Average, 95% and 99.8% lower and upper control limits are shown.

Figure 2 shows the risk-adjusted mortality rates for all admitting hospitals stratified by gestational age at birth. No hospital exceeded the ±99.8% control limits. Hospital C approached the +95% control limit for infants born at 23–31 weeks of gestation (figure 2A). Hospital B and E showed better outcome (ie, exceeding −95% control limit) for infants born at 23–31 weeks of gestation (figure 2A).

![Figure 2](http://fn.bmj.com/https://fn.bmj.com/content/fetalneonatal/103/4/F331/F2.medium.gif)

[Figure 2](http://fn.bmj.com/content/103/4/F331/F2)

Figure 2 
Funnel plot showing risk-adjusted hospital mortality in neonatal intensive care units in New South Wales and Australian Capital Territory for infants born at 23 to 31 (A), 23 to 26 (B), 27 to 28 (C) and 29 to 31 (D) weeks of gestation. Average, 95% and 99.8% lower and upper control limits are shown.

### Predictors of mortality

Lack of antenatal steroid, antenatally diagnosed malformation, mode of birth, male gender, lower gestation, lower birth weight, Apgar score <7 at 5 min and lack of postnatal steroid were significant risk factors for mortality (online supplementary table S1). Of note is that postnatal steroid effect changed from unadjusted to the adjusted model. Postnatal steroid was associated with improved survival once other neonatal factors were adjusted for.

### Supplementary file 1

[[SP1.DOCX]](pending:yes) 

### Validation of the logistic regression model

The logistic regression model used to calculate expected mortality shown in table 3 showed an adequate overall discrimination with a C-statistic (SE) of 0.86 (0.01) (95% CI 0.84 to 0.87). The model fitted the data well (goodness-of-fit χ2 (7df)=5.828; p=0.560).

### Causes of death

The most common causes of death are listed in (online supplementary table S2). Intracerebral and grades 3 and 4 intraventricular haemorrhage (136, 23.8%), sepsis (128, 22.4%), respiratory distress syndrome (85, 14.9%) and necrotising enterocolitis (61, 10.7%) were the most common causes of death.

### Supplementary file 2

[[SP2.DOCX]](pending:yes) 

## Discussion

There is a growing demand for accurate measures of performance variation between hospitals nationally and internationally. However, there is no consensus on what performance indicators should be monitored and how performance is best measured.30 

This study showed that there is little CTC variability in mortality across NICUS Network, Australia. Overall, mortality rates were similar when case mix, severity of illness and number of admissions (NICU volume) were taken into consideration (figure 2).

Ongoing benchmarking of performance is critical for implementation and evaluation of quality improvement initiatives. Within neonatal networks, studying benchmarking results may help address significant variations within the network. This could be simplified by using adjusted measures and funnel plots as we have done. Funnel plots are a simple and powerful graphical method for discriminating between two types of variations. Variation between the units is categorised according to the action needed to reduce it,31 thereby guiding the appropriate action.32 33 Process variation is of two types: *common* (chance) and *special* (assignable) causes of variation. *Common-cause variation*, which is shown as data within the control limits of the chart, is intrinsic to the process. Reduction of common-cause variation requires fundamental changes in the underlying process.31 *Special-cause variation*, which is shown as out-of-control signals in the chart (ie, data point outside the ±99.8% limit), is the result of factors extrinsic to the process, and its reduction therefore requires identification of and action on the special causes.31 Opportunities for improvement in infrastructure and clinical practice could be drawn from hospitals operating below the lower −99.8% limit. In our study, no hospital showed special-cause variation. Detailed framework for improvement using process control is discussed elsewhere.34 

Another advantage of using funnel plots is to use the +95% (2σ) control limits to encourage further investigation. Hospitals exceeding this limit should start early quality improvements to make sure the outcome does not become out of control. Similarly, opportunities for improvement in infrastructure and clinical practice could be drawn from hospitals operating below the lower −95% limit.

CTC variation similar to that which we observed has been reported from other countries including the USA and UK, Canada, Germany, Finland, Switzerland and Japan.2–12 The causes of this variation are difficult to ascertain from this study and deserve further consideration. However, others have suggested that variation could be related to illness severity, different attitudes and thresholds to limiting or withdrawing intensive care measures11 or different management styles among NICUs.2 3 

This study has substantial strengths. It is one of the larger NICU network reports using validated data35 with standardised published definitions.36 All data were coded prospectively, enhancing reliability and accuracy. Our statistical approach is robust as it took case mix and NICU volume into consideration. Our data lacked certain information pertinent to neonatal outcomes, such as fetal compromise37 and sociocultural factors.38 Consequently, there is likely to be some residual confounding despite the high discrimination and goodness of fit of the logistic regression model. Delivery room deaths were not included in our database so we could not detect interhospital variation or determine causes for these deaths. Another limitation is that the maternal record is entered multiple times for ‘multiple and higher order infants’. Therefore, mothers of more than one premature neonate within the study frame appear more than once in table 1 and the number of mothers and neonates is equal (7212). As the data are de-identified, we are unable to account for this. But we believe, particularly given relatively low parity in Australia, that the number of mothers included more than once would be small in the 7-year study period.

## Conclusion and policy implications

Our data showed trends of risk-adjusted CTC mortality variation with a large NICU network. Centre-specific factors other than differences in baseline demographics and NICU volume may be involved. There is an urgent need to delineate these factors. The approach presented in our paper should form the basis for action on neonatal mortality variation and could easily be employed in real or near real time to detect variation early to facilitate timely interventions. In addition, there is a need to study variation in the main morbidities and neurodevelopmental outcomes between these hospitals.

## Acknowledgments

The authors thank the Directors, the New South Wales (NSW) and the Australian Capital Territory (ACT) Neonatal Network (NICUS) members and the audit officers of all tertiary units in supporting this collaborative study: NICUS, Dr Jennifer Bowen (Chairperson), Barbara Bajuk (Coordinator), Sara Sedgley (Research Officer); Canberra Hospital, Dr Hazel Carlisle (Director), Judith Smith; John Hunter Children’s Hospital, Dr Paul Craven (Director), Lynne Cruden, Alissa Argomand; Royal Prince Alfred Hospital, Dr Ingrid Rieger (Director), Dr Girvan Malcolm, Shelley Reid; Liverpool Hospital, Dr Jacqueline Stack (Director), Dr Ian Callander, Kathryn Medlin, Kaye Marcin; Nepean Hospital, Dr Vijay Shindge (Director), Mee Fong Chin, Kerrie Bonser; The Children’s Hospital at Westmead, Prof Nadia Badawi (Co-Director), Dr Robert Halliday (Co-Director), Caroline Karskens; Royal North Shore Hospital, Dr Mary Paradisis, Prof Martin Kluckow, Claire Jacobs; Sydney Children’s Hospital, Dr Andrew Numa (Director), Dr Gary Williams, Janelle Young; Westmead Hospital, Dr Melissa Luig (Director), Jane Baird; and Royal Hospital for Women, Prof Kei Lui (Director), Diane Cameron. The authors also thank the babies and their families, the nursing and midwifery, obstetric and medical records staff of the obstetric and children’s hospitals in NSW and the ACT.

## Footnotes

*   Contributors MEA-L conceptualised and designed the study, carried out the statistical analyses, drafted the initial manuscript and led the writing. GN contributed to the statistical analysis and interpretation. BB retrieved and cleaned the data and contributed to statistical interpretation. KG contributed to the statistical interpretation. DH contributed to the study design and statistical interpretation and led the writing of manuscript. All authors critically reviewed and approved the final manuscript as submitted. All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

*   Competing interests None declared.

*   Ethics approval The Australian National University (No 2015/029), ACT Health Human Research Ethics Committee (No ETH.11.09/1031) and Hunter New England Research Ethics and Governance Unit has approved this study.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Data sharing statement All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. No other data from the study are available.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/) 

## References

1.   Hack M , Wright LL , Shankaran S , et al . Very-low-birth-weight outcomes of the National Institute of Child Health and Human Development Neonatal Network, November 1989 to October 1990. Am J Obstet Gynecol 1995;172:457–64.[doi:10.1016/0002-9378(95)90557-X](http://dx.doi.org/10.1016/0002-9378(95)90557-X) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1016/0002-9378(95)90557-X&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=7856670&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=A1995QJ32800001&link_type=ISI) 

2.   Cotten CM , Oh W , McDonald S , et al . Prolonged hospital stay for extremely premature infants: risk factors, center differences, and the impact of mortality on selecting a best-performing center. J Perinatol 2005;25:650–5.[doi:10.1038/sj.jp.7211369](http://dx.doi.org/10.1038/sj.jp.7211369) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1038/sj.jp.7211369&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=16079906&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 

3.   Vohr BR , Wright LL , Dusick AM , et al . Center differences and outcomes of extremely low birth weight infants. Pediatrics 2004;113:781–9.
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTEzLzQvNzgxIjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.   Horbar JD , Badger GJ , Lewit EM , et al . Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics 1997;99:149–56.
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiOTkvMi8xNDkiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.   Field D , Draper ES . Survival and place of delivery 1994–96. Arch Dis Child Fetal Neonatal Ed 1999;80:F111–4.
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTM6ImZldGFsbmVvbmF0YWwiO3M6NToicmVzaWQiO3M6OToiODAvMi9GMTExIjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

6.   Lee S , Chan K , Ohlson A , et al . Variations in treatment decisions and outcomes of infants. Pediatr Res 1999;45:247A.
    
    

7.   Lee SK , McMillan DD , Ohlsson A , et al . Variations in practice and outcomes in the Canadian NICU network: 1996-1997. Pediatrics 2000;106:1070–9.[doi:10.1542/peds.106.5.1070](http://dx.doi.org/10.1542/peds.106.5.1070) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjEwNi81LzEwNzAiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.   Bartels DB , Wypij D , Wenzlaff P , et al . Hospital volume and neonatal mortality among very low birth weight infants. Pediatrics 2006;117:2206–14.[doi:10.1542/peds.2005-1624](http://dx.doi.org/10.1542/peds.2005-1624) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjExNy82LzIyMDYiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

9.   Tommiska V , Heinonen K , Ikonen S , et al . A national short-term follow-Up study of extremely low birth weight infants born in Finland in 1996-1997. Pediatrics 2001;107:E2.
    
    

10.  Tommiska V , Heinonen K , Lehtonen L , et al . No improvement in outcome of nationwide extremely low birth weight infant populations between 1996-1997 and 1999-2000. Pediatrics 2007;119:29–36.[doi:10.1542/peds.2006-1472](http://dx.doi.org/10.1542/peds.2006-1472) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiMTE5LzEvMjkiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11.  Berger TM , Steurer MA , Woerner A , et al . Trends and centre-to-centre variability in survival rates of very preterm infants (<32 weeks) over a 10-year-period in Switzerland. Arch Dis Child Fetal Neonatal Ed 2012;97:F323–F328.[doi:10.1136/fetalneonatal-2011-301008](http://dx.doi.org/10.1136/fetalneonatal-2011-301008) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTM6ImZldGFsbmVvbmF0YWwiO3M6NToicmVzaWQiO3M6OToiOTcvNS9GMzIzIjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

12.  Kusuda S , Fujimura M , Sakuma I , et al . Morbidity and mortality of infants with very low birth weight in Japan: center variation. Pediatrics 2006;118:e1130–8.[doi:10.1542/peds.2005-2724](http://dx.doi.org/10.1542/peds.2005-2724) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTE6IjExOC80L2UxMTMwIjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

13.  Sankaran K , Chien LY , Walker R , et al . Variations in mortality rates among Canadian neonatal intensive care units. CMAJ 2002;166:173–8.
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjYvMi8xNzMiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

14. Australian Bureau of Statistics. Australian Demographic Statistics: cat. no. 3101.0, ABS, Canberra, 2014. [http://www.abs.gov.au/](http://www.abs.gov.au/). (accessed 9 Jun 2016).
    
    

15.  Abdel-Latif ME , Bajuk B , Oei J , et al . Mortality and morbidities among very premature infants admitted after hours in an Australian neonatal intensive care unit network. Pediatrics 2006;117:1632–9.[doi:10.1542/peds.2005-1421](http://dx.doi.org/10.1542/peds.2005-1421) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjExNy81LzE2MzIiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16.  Rashid A , Bhuta T , Berry A . A regionalised transport service, the way ahead? Arch Dis Child 1999;80:488–92.
    
    [FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI4MC81LzQ4OCI7czo0OiJhdG9tIjtzOjMwOiIvZmV0YWxuZW9uYXRhbC8xMDMvNC9GMzMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17.  Abdel-Latif ME , Berry A . Analysis of the retrieval times of a centralised transport service, New South Wales, Australia. Arch Dis Child 2009;94:282–6.[doi:10.1136/adc.2007.125211](http://dx.doi.org/10.1136/adc.2007.125211) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo4OiI5NC80LzI4MiI7czo0OiJhdG9tIjtzOjMwOiIvZmV0YWxuZW9uYXRhbC8xMDMvNC9GMzMxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18.  Lui K , Abdel-Latif ME , Allgood CL , et al . Improved outcomes of extremely premature outborn infants: effects of strategic changes in perinatal and retrieval services. Pediatrics 2006;118:2076–83.[doi:10.1542/peds.2006-1540](http://dx.doi.org/10.1542/peds.2006-1540) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjExOC81LzIwNzYiO3M6NDoiYXRvbSI7czozMDoiL2ZldGFsbmVvbmF0YWwvMTAzLzQvRjMzMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. Australian Bureau of Statistics. Births, Australia - 2014; cat. no. 3301.0, ABS,Canberra. [http://www.abs.gov.au/](http://www.abs.gov.au/) (accessed9 Jun 2016).
    
    

20.  Bajuk B . Validation of the neonatal intensive care units' data collection. Proceedings of the 5th Annual Conference of the Perinatal Society of Australia and New Zealand. Australia: Canberra, ACT, 2001.
    
    

21. National Health and Medical Research Council. Clinical practice guidelines: Care around preterm birth. Canberra, Australia: National Health and Medical Research Council, 1997.
    
    

22. National Centre for Classification in Health. The international statistical classification of diseases, and related health problems, 10th revision, Australian modification (ICD-10-AM). 3rd ed. Sydney, Australia: National Centre for Classification in Health, Faculty of Health Sciences, The University of Sydney, 2002.
    
    

23.  Lain SJ , Algert CS , Nassar N , et al . Incidence of severe adverse neonatal outcomes: use of a composite indicator in a population cohort. Matern Child Health J 2012;16:600–8.[doi:10.1007/s10995-011-0797-6](http://dx.doi.org/10.1007/s10995-011-0797-6) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1007/s10995-011-0797-6&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=21505774&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=000301795700007&link_type=ISI) 

24.  Abdel-Latif ME , Bajuk B , Oei J , et al . NSW and ACT Neonatal Intensive Care Units Audit Group. Population study of neurodevelopmental outcomes of extremely premature infants admitted after office hours. J Paediatr Child Health 2014;50:E45–E54.[doi:10.1111/jpc.12028](http://dx.doi.org/10.1111/jpc.12028) 
    
    

25.  Lemeshow S , Hosmer DW . A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982;115:92–106.[doi:10.1093/oxfordjournals.aje.a113284](http://dx.doi.org/10.1093/oxfordjournals.aje.a113284) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=7055134&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=A1982MY30800011&link_type=ISI) 

26.  Kleinbaum DG . Logistic regression: a self-learning text. New York: Springer-Verlag, 1994.
    
    

27.  Hosmer DW , Lemeshow S . Applied logistic regression. 2nd ed. New York: Wiley, 2000.
    
    

28.  Rothman KJ , Boice JD . Epidemiologic analysis with a programmable calculator. NIH publication, no. 79-1649. Washington: DC: U.S. Dept. of Health, Education, and Welfare, Public Health Service, National Institutes of Health, 1979.
    
    

29.  Spiegelhalter DJ . Funnel plots for comparing institutional performance. Stat Med 2005;24:1185–202.[doi:10.1002/sim.1970](http://dx.doi.org/10.1002/sim.1970) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1002/sim.1970&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=15568194&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=000228271900004&link_type=ISI) 

30.  Field D . Bench marking and performance management in neonatal care: easier said than done!. Arch Dis Child Fetal Neonatal Ed 2002;87:163F–4.[doi:10.1136/fn.87.3.F163](http://dx.doi.org/10.1136/fn.87.3.F163) 
    
    [FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTM6ImZldGFsbmVvbmF0YWwiO3M6NToicmVzaWQiO3M6OToiODcvMy9GMTYzIjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

31.  Mohammed MA , Cheng KK , Rouse A , et al . Bristol, Shipman, and clinical governance: Shewhart’s forgotten lessons. Lancet 2001;357:463–7.[doi:10.1016/S0140-6736(00)04019-8](http://dx.doi.org/10.1016/S0140-6736(00)04019-8) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(00)04019-8&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=11273083&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=000166865600037&link_type=ISI) 

32.  Mohammed MA , Worthington P , Woodall WH . Plotting basic control charts: tutorial notes for healthcare practitioners. Qual Saf Health Care 2008;17:137–45.[doi:10.1136/qshc.2004.012047](http://dx.doi.org/10.1136/qshc.2004.012047) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjE3LzIvMTM3IjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

33.  Benneyan JC , Lloyd RC , Plsek PE . Statistical process control as a tool for research and healthcare improvement. Qual Saf Health Care 2003;12:458–64.[doi:10.1136/qhc.12.6.458](http://dx.doi.org/10.1136/qhc.12.6.458) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjEyLzYvNDU4IjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

34.  Lilford R , Mohammed MA , Spiegelhalter D , et al . Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363:1147–54.[doi:10.1016/S0140-6736(04)15901-1](http://dx.doi.org/10.1016/S0140-6736(04)15901-1) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1016/S0140-6736(04)15901-1&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=15064036&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 
    
    [Web of Science](http://fn.bmj.com/lookup/external-ref?access_num=000220595500025&link_type=ISI) 

35.  Ford JB , Roberts CL , Algert CS , et al . NICUS group. Using hospital discharge data for determining neonatal morbidity and mortality: a validation study. BMC Health Serv Res 2007;7:188.[doi:10.1186/1472-6963-7-188](http://dx.doi.org/10.1186/1472-6963-7-188) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1186/1472-6963-7-188&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=18021458&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 

36.  Bajuk B . Neonatal Intensive Care Units' (NICUS) Data Collection Definitions. Australia: Westmead, 2011.
    
    

37.  Kaukola T , Räsänen J , Herva R , et al . Suboptimal neurodevelopment in very preterm infants is related to fetal cardiovascular compromise in placental insufficiency. Am J Obstet Gynecol 2005;193:414–20.[doi:10.1016/j.ajog.2004.12.005](http://dx.doi.org/10.1016/j.ajog.2004.12.005) 
    
    [CrossRef](http://fn.bmj.com/lookup/external-ref?access_num=10.1016/j.ajog.2004.12.005&link_type=DOI) 
    
    [PubMed](http://fn.bmj.com/lookup/external-ref?access_num=16098863&link_type=MED&atom=%2Ffetalneonatal%2F103%2F4%2FF331.atom) 

38.  Sommerfelt K , Ellertsen B , Markestad T . Parental factors in cognitive outcome of non-handicapped low birthweight infants. Arch Dis Child Fetal Neonatal Ed 1995;73:F135–F142.[doi:10.1136/fn.73.3.F135](http://dx.doi.org/10.1136/fn.73.3.F135) 
    
    [Abstract/FREE Full Text](http://fn.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTM6ImZldGFsbmVvbmF0YWwiO3M6NToicmVzaWQiO3M6OToiNzMvMy9GMTM1IjtzOjQ6ImF0b20iO3M6MzA6Ii9mZXRhbG5lb25hdGFsLzEwMy80L0YzMzEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9)