Introduction

Neonatal mortality has been a commonly used outcome measure for research into obstetric and neonatal interventions and the quality of perinatal care. However as the neonatal mortality rate has been decreasing, especially amongst infants born after 27 weeks gestation [1], severe morbidity has been suggested as a more relevant outcome [2]. The 5 min Apgar score has been used as an outcome measure of neonatal morbidity [3], however the Apgar score was designed to be a quick and convenient method to report on the status of the newborn infant immediately after birth [4] not an indicator of severe morbidity.

Individual conditions (severe adverse outcome measures), such as seizures or intraventricular haemorrhage, require large study samples to detect often subtle clinical differences between groups of infants at risk. To overcome this, composite neonatal outcomes have been used in randomised controlled trials [5], prospective [6] and retrospective cohort studies [7–9] where the incidence of individual outcomes are low.

Routinely collected or population health datasets (PHDS) such as birth registries and hospital discharge data, are an easily accessible resource to assess neonatal morbidity. Compared to prospective studies or retrospective chart review [10, 11] PHDS potentially provide a less resource-intensive and cost effective method of research. However for PHDS to be used to monitor neonatal morbidity its reliability and validity has to be assured [12, 13].

Studies validating the reporting of neonatal outcomes in PHDS report high positive predictive values (generally greater than 85%) which is important as a high PPV ensures that the majority of neonates identified truly have the condition or procedure. Neonatal outcomes collected in PHDS also generally report moderate to high sensitivities (>75%), indicating good ascertainment of neonatal conditions [14–16], although some individual conditions have been reported to have lower ascertainment. These include pulmonary hypertension (64%) [15], necrotising enterocolitis (62%) [15], respiratory distress (range 50–94%) [14–16], intraventricular haemorrhage (range 52–100%) [14–16], pneumonia (48%) [15], and bacterial sepsis (38%), although the sensitivity of sepsis was increased to 67% when adult sepsis codes were also included [14]. Furthermore, procedures are generally reported better than conditions [17], especially surgical procedures, for example major neonatal surgery has a sensitivity of 91% and a PPV of 95% [15].

The use of a composite indicator helps overcome the under-ascertainment of individual conditions and procedures. Severely ill neonates may have co-morbidities and multiple procedures, so a composite indicator that includes any morbid event increases the chance of identifying those infants with major morbidity [18]. A validated indicator for maternal morbidity using PHDS identified almost 80% of women with a severe maternal morbidity with a positive predictive value of 95% [18].

The aims of this study were: (1) to develop a neonatal adverse outcome indicator (NAOI) to measure severe neonatal morbidity and mortality in population health datasets by using previously validated diagnoses and procedure codes; and.

(2) to assess the face validity of the outcome indicator by examining its association with readmission and infant mortality rates.

Methods

Development and Refinement of Neonatal Adverse Outcome Indicator (NAOI)

The composite outcome indicator, the NAOI, was initially developed based on review of the literature and consultation with a neonatologist (JB). Studies using routinely collected data to measure neonatal morbidity [7, 8] and studies validating the reporting of infant outcomes in PHDS [14–16, 19–21] were reviewed and a comprehensive list of reliably reported diagnoses and procedures indicative of serious adverse neonatal outcomes was compiled. To ensure the indicator captured severe morbidity the components of the indicator were analysed and refined in an iterative process. Where possible we avoided inclusion of factors that may reflect service provision, such as NICU admission, as we planned to use the final indicator to compare neonatal outcomes across obstetric levels of care. Gestational age <32 weeks was included on the basis that it represents a poor obstetric outcome; it also captures morbidity due to conditions such as retinopathy of prematurity which are rare outside of very preterm infants.

Refinement of the initial version of the indicator included an iterative assessment of the incidence of each of the components of the indicator and calculation of the associated rates of hospital readmission to identify those infants with long term morbidity. As a result, less severe, more common neonatal conditions that were poorly predictive of long term morbidity and occurred without other morbidity were not included in the indicator. This mainly affected the transitional neonatal conditions of transient tachypnoea of the newborn, jaundice, low blood sugar and feeding difficulties. Infants with these less severe conditions in isolation had lower hospital readmission rates than those with conditions and procedures included in the NAOI (21.9% for those with transient tachypnea, 23.4% with hypoglycaemia, 22.1% with jaundice and 23.4% for infants receiving enteral nutrition). Ultimately, these conditions were not included unless they were associated with other morbidities or the need for intensive care support. For example, infants with hypoglycaemia who required parenteral support and infants with transient tachypnea requiring ventilatory support were included. Inclusion of the less severe conditions without other morbidity in the NAOI would only slightly increase the sensitivity of indentifying infants with serious morbidity at the expense of including many infants with a less severe spectrum of morbidity, decreasing specificity.

The final list of the components of the indicator is reported in Appendix. Having any one of these conditions or procedures recorded in the infant’s admission before first discharge home (including transfers to other hospitals) qualified as a case to be included in the adverse outcomes indicator. This list of conditions and procedures was applied to the following population health dataset.

Study Population and Datasets

All livebirths with a gestational age of 24 weeks or greater to women residing in New South Wales (NSW), Australia from January 2001 through December 2006 were included in the cohort. NSW is the most populous state in Australia, and with around 90,000 births per year, comprising 34% of all Australian births [22]. Data from the Midwives Data Collection (MDC), a population-based surveillance system of all births in NSW, and the Admitted Patient Data Collection (APDC), an administrative database of all hospital admissions in NSW, were used. These databases have been described previously [23]. The APDC hospital discharge summaries include diagnoses and procedures that are coded for each admission from the medical records according to the 10th revision of the International Classification of Diseases and Related Health Problems, Australian Modification (ICD-10-AM) and the affiliated Australian Classification of Health Interventions (ACHI), respectively. The two databases have been linked, using probabilistic linkage. This linkage enables the infant’s MDC birth data to be linked to their hospital birth admission, and longitudinally linked to subsequent hospital admissions. These data are also linked to Australian Bureau of Statistics (ABS) mortality data, so that all deaths in the first year of life could be identified. Only anonymized data are made available to researchers. Linkage and use of the data was approved by the NSW Population and Health Services Research Ethics Committee.

Maternal factors available on the databases included age, parity, patient in a private or public hospital and smoking status. Socio-economic status was available and categorised into quintiles based on the ABS’s SEIFA (Socio-Economic Indexes for Areas) 2006 index of relative disadvantage by postcode [24]. Infant birthweight, sex, gestational age, Apgar score (1 and 5 min) and level of resuscitation at birth were available from the birth data; while other conditions were determined by searching both the diagnosis and procedure fields on each admission record. Size-for-gestational-age was determined using standard birthweight percentile charts. [25] Small-for-gestational-age (SGA) and large-for-gestational-age (LGA) were defined as <10th percentile and >90th percentile birthweight for gestational age, respectively.

From 2001 to 2006 there were 516,843 infants of at least 24 weeks gestational age, born alive to women residing in NSW with an MDC birth record. However, 9,166 (1.8%) births did not link to any hospital admission record, and 814 (0.2%) births only linked to a post-delivery admission. Homebirths were over-represented in these non-linked records: 5.4% were planned homebirths versus 0.1% of births that did link to a hospital record. Any neonatal morbidity identified among the unlinked records was only established by gestational age, birthweight, resuscitation or mortality criteria. The proportion of missing data for variables from the MDC that were incorporated into the NAOI was small: birthweight 0.01%, gestational age 0.01%, resuscitation 0.09%.

Data Analysis

The frequency and relative risks of the NAOI were calculated for a range of maternal and infant characteristics using the SAS statistical software. In addition, incidence rates were calculated for each condition or procedure included in the composite outcome indicator for early preterm (<34 weeks gestation), late preterm (34–36 weeks) and term (≥37 weeks gestation) infants before first discharge home. Denominators were based on all livebirths and sourced from the MDC.

Longer term neonatal outcomes, including infant mortality and readmission to hospital following the first discharge home were also assessed. Readmissions included day-only stays (patients who are admitted and discharged on the same day and can include emergency room admissions) as well as overnight admissions. Readmissions for certain elective procedures (e.g circumcision, vaccination) were not included in the calculation of readmission rates. The rates of hospital readmission and infant death after the first discharge and before one year of age were calculated for the NAOI and for components of the NAOI.

To investigate the face validity of the NAOI, the association between neonatal morbidity and subsequent infant mortality and hospital readmission were examined. Relative risk (RR) (and 95% confidence intervals (CI)) for hospital readmission or infant death in the first year of life were calculated for infants identified by the NAOI compared to those without the indicator. For comparative purposes, we also investigated the mortality rate in infants with 5 min Apgar score < 7. This analysis was performed on all infants that were discharged and then on a subgroup which excluded infants with severe congenital abnormalities.

Results

Of the 516,843 infants of at least 24 weeks gestational age, 23,726 infants (4.59%) had one or more conditions included in the NAOI. The frequency of infants identified by the NAOI fell sharply by week of gestation. At 32 weeks the rate of adverse outcome was 71.2%, but this fell to 5.1% at 37 weeks (Fig. 1). Morbidity was lowest at 39 weeks (2.0%) and 40 weeks gestation (2.0%), but increased to 2.4% at 41 weeks and 3.2% for gestations of 42 weeks and greater.

Fig. 1
figure 1

Rate of neonatal morbidity by gestational age at delivery

Full size image

The distribution of maternal and infant characteristics by NAOI is shown in Table 1. Nulliparous mothers, smokers and mothers in the lowest quintile for socioeconomic status were associated with increased risk of an infant with an adverse outcome. The median length of stay (LOS) (including any transfer admissions) for infants with an adverse outcome was 7 days (interquartile range [IQR] 4–18 days), while infants who did not have a record of morbidity had a median LOS of 3 days (IQR 2–5). The LOS for preterm infants identified by the NAOI was typically much longer than for term infants identified by the NAOI: 15 days (IQR 7–30) versus 5 days (IQR 3–7) for term infants with an adverse outcome.

Table 1 Frequency of infants with NAOI and association of NAOI with maternal and infant characteristics
Full size table

The frequency of the NAOI and of most of the component conditions of the NAOI is shown in Table 2, for early preterm (<34 weeks), late preterm (34–36 weeks) and term births (≥37 weeks). The incidence of an adverse outcome was much higher in early preterm (81.9 per 100 births) compared to late preterm (18.3 per 100 births) and term births (2.4 per 100 births),partly because all infants less than 32 weeks gestation are included in the NAOI and comprise 52.1% of the early preterm group. The most common indications of an adverse outcome for term and late preterm infants was intravenous fluids followed by ventilatory support and respiratory distress syndrome, while ventilatory support was the most common indication for an adverse outcome for early preterm infants. Respiratory distress syndrome and ventilatory support were highly correlated: 65.4% of infants with a diagnosis of respiratory distress syndrome required resuscitation with intubation and/or subsequent continuous positive airways pressure (CPAP) or mechanical ventilation.

Table 2 Incidence (per 100 births) for conditions and procedures indicative of neonatal morbidity, by gestational age
Full size table

The rates of readmission and infant death in the first year of life following discharge home are shown for conditions included in the NAOI in Table 3. The rate of readmission in infants without any of the neonatal conditions was 15.3%; most NAOI conditions had a readmission rate that was more than double this. Infants with bronchopulmonary dysplasia had the highest hospital readmission rate (58.4%), followed by infants requiring a surgical procedure before discharge home (54.8%) and infants with intraventricular hemorrhage of grade 2 or higher (52.2%). All term infants with a diagnosis of bronchopulmonary dysplasia had received over 95 h of ventilation.

Table 3 Rates of hospital readmission or death among infants discharged home, up to their first birthday, for conditions and procedures indicative of neonatal morbidity
Full size table

The composite neonatal adverse outcome indicator was strongly associated with death subsequent to discharge home (RR = 9.83.0; 95% CI 8.25–11.71). The NAOI identified 31% of deaths in the first year after discharge home, compared with only 7.5% of infants identified with a 5 min Apgar < 7. The strong association between the NAOI and subsequent infant death persisted even when infants with serious malformations were excluded (RR = 6.73; 95% CI 5.43–8.35). With regards to readmissions during the first year of life following initial discharge, there was also a strong association with the composite neonatal outcome indicator (RR = 1.98; 95% CI 1.93–2.02) whereas an Apgar < 7 had a weaker association (RR = 1.53; 95% CI 1.46–1.60). The readmission rate among all infants was substantial (16.0%) as it included day-only admissions with respiratory or other infections, and short-term sleeping and feeding difficulties were the most common reasons for admission to hospital.

Discussion

The neonatal adverse outcome indicator outlined in this study can be used with population health data to identify infants with serious morbidity at birth and was associated with increased risk of death or readmission to hospital in the first year of life. In the past, studies have used various measures such as gestational age, birth weight and/or Apgar scores to represent neonatal morbidity [3, 6, 26, 27], however these measures are often not sensitive enough to differentiate infants with severe neonatal morbidity. To our knowledge this is the first outcome indicator to capture overall severe neonatal morbidity in both term and preterm infants using PHDS. Other validated neonatal outcome indicators use scales to score newborns that rely on data collected from clinical and laboratory records [10, 11]. Severity-of-illness scores (e.g. CRIB, NTISS, SNAP, SNAP-PE) have been developed for prediction of mortality among infants admitted to a NICU but even the simplest of these requires blood pH and respiratory oxygen levels, information not found in population data sets.

Recent studies using population health data have applied composite outcomes to measure neonatal morbidity amongst term, low risk, singleton pregnancies [7, 8]. These studies report higher incidences of composite morbidity after planned Cesarean (9.5% [8] and 11.9% [7] compared to 2.5% in our study) despite reporting low neonatal mortality (0.008 and 0.02% respectively compared to 0.07% for term infants in our study). This is likely to be due to the inclusion of all diagnoses of transient tachypnea and hypoglycaemia. Both of these conditions are common but infrequently lead to admission into a neonatal intensive care unit (NICU) suggesting a lesser degree of morbidity. Our aim is to identify infants with severe adverse outcomes so in the NAOI we included infants with more severe conditions that required invasive treatment; infants with hypoglycaemia who received intravenous fluids and infants with transient tachypnea requiring ventilator support.

Previous studies have included neonates with a hospital stay of five days or greater [8] or those admitted to a NICU [7]. We did not include these factors in our NAOI as we considered them to be related to service provision and less directly linked to specific morbidity. Given we plan to use the NAOI to compare outcomes across hospitals, rates may be biased by the fact that some small hospitals do not have a NICU and length of hospital stay may be affected by local health policy, mother’s mode of delivery or bed availability.

Readmission to hospital and death in the first year of life was used as a way of validating the NAOI and its individual components since severe neonatal morbidity can predispose to longer term morbidity. Although neither readmission nor infant death are perfect measures to validate the indicator they have been widely used as proxies for longer term neonatal morbidity [28–31]. Hospital readmission has been identified as a significant outcome indicator as sicker or more complicated patients have higher rates of readmission [32], however we found a high proportion (15.3%) of infants without a severe morbidity in the neonatal period were also readmitted to hospital in the first year of life. Infant death in the first year of life is also an outcome of severe neonatal morbidity although many infant deaths do not relate to morbidity in the neonatal period. From 1997 to 2001 the leading causes of death of infants in the postnatal period in Australia were sudden infant death syndrome (28%) followed by unspecified congenital malformation of the heart (3.1%) [33]. Our results show that the NAOI had a higher association with hospital readmission or death in the first year of life than a five minute Apgar score of less than seven. Face validity of the NAOI was further demonstrated by the steady decrease at each week of gestation after 32 weeks in the proportion of infants identified by the indicator, dropping to a minimum at full term.

A strength of this study is the use of diagnoses and procedures that have been validated in population health data across numerous jurisdictions including the NSW MDC and APDC databases [14–16, 19–21]. As reported in the validation studies, population health data have few false positives but under-ascertainment of conditions and procedures. Consequently the incidence of individual conditions and procedures reported in this study need to be interpreted with caution. A major advantage of a composite indicator is that it helps overcome this under-ascertainment. Published incidence rates from retrospective chart reviews or cohort studies highlight the potential under-enumeration of particular conditions in our study; such as respiratory distress syndrome (2.85% [16] vs. 1.69%) and grade 3 or 4 intraventricular haemorrhage in preterm infants (1.8% [34] vs. 0.8%). However published incidence rates of 0.18% for seizures, and 0.66% for intubation in term infants [34] were comparable to incidence rates of 0.20% (seizures) and 0.73% (ventilatory support) in our study. The inclusion of procedure codes can also improve identification of morbidity as procedures are generally reported more accurately than diagnoses [15], and the use of adult codes can improve ascertainment without increasing false negatives [14]. We have included adult codes for seizures, pneumonia and sepsis to improve sensitivity of these conditions. The population datasets available for our analysis contained a relatively wide range of fields, including hospital procedures and delivery room resuscitation. Other studies may have different sets of available variables, which will affect comparability to some extent. A limitation of this study is the necessity of creating dichotomous categories to represent severe morbidity at birth out of what is in reality a spectrum of morbidity. For instance, mechanical ventilation and CPAP were included, but not the relatively more common procedure of oxygen supplementation. Another limitation to the use of the NAOI is that there will still be some inter-hospital variation in use of procedures (and the NAOI incidence) due to availability of services. Caution should also be used when extrapolating the NAOI to longer term morbidity, as the NAOI directly represents morbidity in the neonatal period. Some conditions such as birth trauma (primarily localised paralysis due to brachial plexus injury) usually resolved without readmission.

Conclusion

The composite neonatal adverse outcome indicator is a reliable population measure of severe neonatal morbidity utilizing routinely collected data that can easily be applied by other users of population health databases which is an important feature of an indicator [35]. The NAOI, along with the maternal morbidity outcome indicator [18], can also be used to monitor the quality of obstetric and neonatal care in a uniform and cost-effective way. Because it is based on coded hospital data, it can also be used to determine the direct health system costs of severe neonatal morbidity. Finally, with increasing longitudinal linkage of administrative health data, the NAOI can identify infants for follow-up studies of longer term outcomes.