The improved survival of preterm infants has resulted in the emergence of a neonatal population highly vulnerable to healthcare-associated infections. Bloodstream infection (BSI) rates in neonates in hospital range from 10–25% for all neonates to 50% in extremely preterm infants.1 2 Neonatal BSI has a high mortality, increases the short-term and long-term costs of newborn care, and is a major determinant of neurodevelopment outcome.3

Surveillance is a core component of strategies to combat infectious diseases. It is an essential first step towards benchmarking and performance comparison, ascertainment of antibiotic resistance patterns and temporal trends, facilitation of infection control measures, evaluation of novel therapies and provision of baseline data to inform the design of clinical trials. However, the development of robust surveillance targeting the needs of neonates has proved problematic. In England no infection surveillance initiatives have specifically addressed the needs of neonates.

Progress in neonatal BSI surveillance has been hindered by the lack of an agreed case definition. Laboratory diagnostic tests have poor or indeterminate accuracy4 5 and are often not universally available. Obtaining blood cultures from neonates can be difficult, sample volumes are small, and a substantial number are negative or contaminated. Due to antibiotic use and small sample volumes, BSI blood cultures may give false-negative results. Conversely, many positive blood cultures may be contaminants. This is particularly true for coagulase-negative staphylococci (CoNS), which account for about 70% of all positive blood cultures in neonates,6 though only about half of these represent true infection.7 8

Genuine bloodstream infection with CoNS is almost exclusively confined to the preterm neonate under 32 weeks’ gestation, and reflects immunological immaturity. Clinicians usually base a diagnosis of CoNS BSI on the combination of a positive blood culture, clinical deterioration and an acute phase response, commonly C-reactive protein (CRP). However, clinical responses to the onset of BSI in preterm neonates are non-specific and share features with other non-infective conditions.

In clinical practice, the thresholds for making a diagnosis of BSI and starting antibiotic therapy in neonates are justifiably low and diffuse, which poses a problem for surveillance. If data on BSI are to be pooled to make inter-hospital comparisons, the case definition must be precise and unambiguous. Variations in case definition invalidate surveillance, bench marking and the use of BSI as an outcome in clinical trials. Distinguishing between contaminants and true pathogens is a challenge that must be addressed. Our aim was therefore to develop a pragmatic, neonate-specific case definition for BSI suitable for national and international surveillance.

METHODS

Case definition for neonatal BSI

We utilised data collected from a UK multicentre, single blind, randomised controlled trial—PROGRAMS (PROphylactic GRAnulocyte Macrophage colony stimulating factor to reduce Sepsis, mortality and later disability in preterm neonates, ISRCTN42553489). Eligible infants were <10th centile for birth weight and ⩽31 completed weeks gestational age, and trial entry was within 3 days of birth. Data were captured daily to 28 days from trial entry or until death of the infant if earlier. Blood cultures were obtained at the discretion of the attending clinician. Daily data included the results of blood cultures and CRP, and the objective evaluation of 12 predefined clinical signs, widely held to be suggestive of BSI, and selected following discussion with senior UK neonatologists and review of published recommendations.6 9 10 These signs were the acute onset of: increased oxygen requirement or ventilatory support; increase in apnoea/bradycardia; hypotension; glucose intolerance; impaired peripheral perfusion (capillary refill time >3 s/pallor/mottling/core-peripheral temperature gap >2°C); lethargy/irritability/poor handling; temperature instability; ileus/onset of feed intolerance; increase in serum bilirubin; fall in urine output; metabolic acidosis/base deficit <−10 mmol/l; and anticonvulsant therapy. We analysed daily data for each infant recruited into the PROGRAMS trial up to the onset of the first positive blood culture. The prevalence of each of the 12 clinical signs and any acute rise in CRP over every day of observation, regardless of whether or not a blood culture was performed, was determined together with the sensitivity, specificity, odds ratio and positive predictive value of each for a positive blood culture. Data for up to 28 days were analysed for each baby. As repeated observations from the same subject are not necessarily independent, robust sandwich estimates of the standard errors of the regression coefficients were used to take into account the presence of multiple observations for each baby. The estimates of the standard errors were adjusted based on the observed correlations between repeated observations in the same baby.

We then determined the number of clinical signs that most accurately predicted any positive blood culture. We initially performed a multiple logistic regression analysis which allowed different weights to be given to each clinical sign. As the simpler approach giving an equal weight to each sign on the assumption that they were equally important, resulted in similar model performance, we gave a weight of 1 to each clinical sign selected at the 5% significance level on univariate analysis, in the final model. We calculated the total number of clinical signs present for each baby for each day, the predicted probability of any positive blood culture at sequential numbers of clinical signs and derived the sensitivity and specificity for each cut-off. We expressed these data as a receiver operating characteristic (ROC) curve.

As the clinical importance of a blood culture isolate of CoNS or other skin commensal, or a mixed growth may be uncertain, in our analyses we distinguished these from isolates of recognised pathogens that grew in pure culture and were not common skin flora. Given that a substantial proportion would represent contamination rather than true BSI, our intention was that the case definition for CoNS and mixed growth BSI should optimise specificity without unduly compromising sensitivity.

Model validation

Model validation was performed using clinical data captured on admissions to the neonatal unit at Queen Charlotte’s and Hammersmith Hospitals in London, UK, from January 2001 to October 2003. This is a tertiary referral service providing a full range of newborn care. During 2001 and 2002 clinical signs were only recorded on infants with a positive blood culture. In 2003 clinical signs were recorded on all infants where a blood culture, whether positive or negative, was obtained. CRP was measured daily and a positive response was recorded if there was an acute rise within 24 h of obtaining a blood culture.

Data from each infant were only included up to and including the first positive blood culture. We applied the model previously developed based on the total number of clinical signs present, with and without an acute CRP response, to determine the specificity and sensitivity for a positive blood culture. The model was tested in all babies and in babies of ⩽31 weeks’ gestation, as it had been developed in this age group.

Analyses were performed using Stata Release 8.0. Results are presented as the estimated odds ratio and 95% confidence interval (CI) for predicting a positive blood culture. Statistical significance is based on nominal probability levels of 0.05.

Ethics approval

The PROGRAMS trial was approved by the UK South-East Metropolitan Multi-centre Research Ethics Committee. The Hammersmith Hospitals Trust Caldicott Guardian approved the use of the neonatal clinical database.

RESULTS

Model development

Data were analysed from 220 infants recruited to PROGRAMS from 26 neonatal units throughout the UK. There were 99 positive blood cultures and 3954 days of observations in this data set. Figure 1 summarises the number of clinical signs present on days when no blood culture was obtained and on days when a culture was obtained that was negative, or yielded CoNS, a mixed growth, or a recognised pathogen. There is no overlap in the interquartile range for the number of clinical signs when CoNS was isolated and when a recognised pathogen was isolated.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Model development: box and whisker plot of the number of clinical signs present on days when no blood culture (BC) was taken, a negative BC, coagulase-negative staphylococci (CoNS)-positive BC, mixed growth BC and recognised pathogen positive BC. The box indicates the interquartile range, the whiskers indicate the upper and lower fences (maximum length of whiskers is 1.5 times the interquartile range, rolled back to where there is data), points outside the fences are plotted individually and can be regarded as potential outliers; y axis is number of clinical signs.

The prevalence of each clinical sign in the study population, with the odds ratio for a positive blood culture and test of significance for whether the odds ratio was significantly different from 1 is shown in table 1, together with the univariate sensitivities, specificities and positive predictive values. Anticonvulsant therapy (p = 0.8) and jaundice (p = 0.6) were excluded from further consideration as there was no evidence of a relationship, leaving 10 clinical signs.

The sensitivity and specificity using sequential numbers of clinical signs as a cut-off to define a positive test are shown in table 2. The area under the ROC curve is 0.833 (Figure 2). The area under the ROC curve is the probability that a randomly selected day would be correctly classified.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 Model development: ROC curve for the model containing the number of clinical signs.

A cut-off of ⩾2 clinical signs has the best accuracy (sensitivity + specificity  = 152.6) for all blood cultures. Table 3 shows that the sensitivity for a recognised pathogen is the same (85.7%) regardless of whether a cut-off of ⩾2 or ⩾3 clinical signs is used. The sensitivity for CoNS at a cut-off of ⩾2 is 67.1%, and at ⩾3, 48.0%. However a cut-off of ⩾3 clinical signs improves specificity from 82.9% to 91.8%. The addition of a CRP response substantially reduces sensitivity.

Model validation

Blood cultures were obtained on a total of 561 days (maximum of 7 days per infant), for 471 infants admitted to the Queen Charlotte and Hammersmith’s neonatal unit. There were 213 days with a positive blood culture and 348 days with only negative cultures. Table 4 summarises the test result (based on ⩾3 clinical signs, with and without an acute CRP response) in relation to a positive blood culture, for all babies and babies ⩽31 weeks’ gestation. The model performed well in both groups. Sensitivity was greater in babies ⩽31 weeks’ gestational age; specificity was higher in all babies The addition of a CRP response improved specificity but reduced sensitivity. Overall accuracy was better with ⩾3 clinical signs without a CRP response.

Discussion

The growth of a recognised pathogen in pure culture is generally considered a clear case for inclusion in BSI surveillance. Difficulties arise when growth is mixed or when an organism such as CoNS is isolated. We have demonstrated that the acute onset of ⩾3 clearly defined clinical signs predicts accurately a positive blood culture in neonates. We found that the sensitivity for recognised pathogen positive cultures was greater than for CoNS, as would be expected given that a substantial number of CoNS isolates will represent contamination. The incorporation of clinical criteria into the case definition for BSI when organisms other than recognised pathogens are isolated, or when a mixed growth is obtained, would reduce the number of blood cultures that would be falsely considered BSI if the diagnosis were based primarily on microbiological criteria as is the case with the US National Institute of Child Health and Human Development Neonatal Research Network (NICHD) definition. This does not include any requirement for clinical signs, but relies on single or multiple blood cultures, elevated CRP and duration of antibiotic treatment.2 Multiple cultures are impractical in infants, a rise in CRP occurs after the onset of clinical deterioration and duration of antibiotic treatment is notoriously variable among clinicians. The NICHD acknowledges that its method is likely to overestimate the true rate of BSI. Other countries, such as Germany, have developed specific criteria for neonatal BSI surveillance but unlike our study, no attempt has been made to establish the accuracy of the case definitions.11

A satisfactory case definition for BSI should ideally have high sensitivity (the test is positive if infection is present) and high specificity (the test is negative if infection is absent). In practice, this goal has been elusive. A large number of laboratory indices have been evaluated in neonatal medicine and shown to have wide and variable sensitivities and specificities.4 Some are suitable only in a research setting and many are limited in availability through cost, practicability or labour intensity. One of the most widely used is the acute phase protein CRP. Following exposure to an infectious insult, CRP increases slowly, though it may ultimately rise more than 1000-fold. The sensitivity at the time of onset of clinical signs is only of the order of 60% though this may rise by 24–48 h into the illness.12 The CRP response is thus considered a specific but late marker of infection. We found that the addition of a CRP response improved specificity but decreased the sensitivity with which a positive blood culture was identified. In addition CRP is not measured in all neonatal care units and will not necessarily be elevated at the first signs of infection.

Though unexplored in our study, the use of clearly defined clinical criteria presents additional opportunities. In settings where blood cultures are readily available, this may be used to attempt to capture the incidence of culture-negative BSI and in settings where resources are limited, to attempt to quantify BSI without blood cultures.

The strengths of our study are that the initial choice of clinical criteria was based on data from 26 neonatal units throughout the UK, and that the validation component included data captured on infants with negative as well as positive blood cultures. A weakness is that the data for initial model development was only from preterm babies. However, genuine CoNS BSI is virtually confined to preterm infants, clinical responses are clearer the more mature the infant, and the final model was validated in infants of all gestational ages and shown to perform well.

The evidence base for infection control procedures, antibiotic policies and novel anti-infective treatments is limited in the newborn, despite their high-risk status for infection and their vulnerability to its consequences. Neonates are also more likely than any other patient group to be exposed to prolonged use of broad-spectrum antibiotics and are increasingly vulnerable to multiresistant invasive infection. To date neonates have been largely excluded from national initiatives aiming to reduce infection and improve infection-related outcomes. There have been calls for systematic, data-driven decision-making processes to underpin newborn care.13 Surveillance is such a process and an essential first step towards addressing infection control and reducing healthcare-associated infection in the newborn. A case definition for purposes of surveillance should not be confused with criteria for clinical diagnosis or screening. For diagnosis, test sensitivity is a prime consideration; in screening one aims for high specificity. We are doing neither; we are concerned with developing a case definition for infection control and for this reason have selected a test cut-off with the highest accuracy—that is, where the sum of the sensitivity and specificity is maximum. In neonatal medicine clinical diagnosis is a matter of judgement taking into account all available laboratory and contextual data and treatment thresholds are of necessity low. However, a case definition for infection surveillance must be simple, standardised and unambiguous. The case definition we propose, namely the growth of a recognised pathogen in pure culture or, in the case of a mixed growth or growth of a skin commensal, the added requirement for the acute onset of ⩾3 predefined clinical signs, fulfils these criteria and provides a way forward for national and international neonatal BSI surveillance.