Article Text
PDF
Abstract
AIM To investigate the feasibility of developing an objective tool for predicting death and severe disability using routinely available data, including an objective measure of illness severity, in very low birthweight babies.
METHOD A cohort study of 297 premature babies surviving the first three days of life was made. Predictive variables considered included birthweight, gestation, 3 day cranial ultrasound appearances and 3 day CRIB (clinical risk index for babies) score. Models were developed using regression techniques and positive predictive values (PPV) and likelihood ratios (LR) were calculated.
RESULTS On univariate analysis, birthweight, gestation, 3 day CRIB score and 3 day cranial ultrasound appearances were each associated with death. On multivariate analysis, 3 day CRIB score and 3 day cranial ultrasound appearances remained independently associated. A 3 day CRIB score > 4 along with intraventricular haemorrhage (IVH) grade 3 or 4 was associated with a PPV of 64% and an LR of 9.8 (95% confidence limits 3.5, 27.9). Only 3 day CRIB score and 3 day cranial ultrasound appearances were associated with severe disability on univariate analysis. Both remained independently associated on multivariate analysis. A 3 day CRIB score > 4 along with an IVH grade of 3 or 4 was associated with a PPV of 60% and an LR of 24.2 (95% CI 4.4, 133.3).
CONCLUSION Incorporating objective measures of illness severity may improve current prediction of death and disability in premature infants.
- severity of illness index
- prematurity
- CRIB score
- positive predictive values
- likelihood ratios
Statistics from Altmetric.com
Neonatologists frequently estimate the likely prognosis for very premature infants. However, many neonatal deaths occur within the first two or three days of life usually because the baby succumbs despite continuing intensive care or because continuing treatment is electively withdrawn.1 As a result, these babies do not tend to present the same prognostic dilemmas as those who survive the immediate newborn period. For those who do survive the first three days, two questions are crucial: what are the chances that this baby will still die; and if this baby survives, what are the chances of severe disability? These questions usually arise when discussing prognosis with parents2 or when considering withdrawal of intensive treatment.3-5
Currently, clinicians amalgamate prognostic data from a variety of sources when predicting the outcome for any given baby.6-9 Simple objective risk factors such as birthweight and gestational age at birth are often used10as well as objective evidence of brain damage obtained from the appearances on cranial ultrasound scans.11 Over the past decade several models and scoring systems have also been described.12-16 However, recent evidence suggests that birthweight may not be an accurate guide to prognosis beyond the first few days of life1 and most of the currently available scoring systems are too complex for use in routine clinical practice. Illness severity can also be an independent predictor of mortality17 and although neonatologists routinely use their clinical experience to assess subjectively just how sick any given baby might be, little work has been done to examine how objective measures of illness severity such as SNAP (Score for Neonatal Acute Physiology) or CRIB18 19 might be used in this context. We therefore investigated the possibility of developing a simple predictive model incorporating a measure of illness severity that might provide clinicians with an objective estimate of :(a) the risk of death before discharge; or (b) the risk of severe disability at 18 months in premature babies surviving the first three days of neonatal intensive care. We deliberately explored only the use of data that areroutinely available and simple: birthweight; gestational age at birth; CRIB score on day 3 of life, a simple valid and reliable measure of clinical risk and illness severity18 20; and appearances on cranial ultrasound scans on or around day 3 of life.
Method
This was a retrospective cohort study in infants weighing ⩽ 1500 g at birth or born at < 31 weeks of gestation between 1988 and 1990 in six Scottish hospitals. Infants born by caesarean section were included but not those born outside the target hospitals and subsequently transferred into one of them. Three of the hospitals were regarded as tertiary referral centres. Data were abstracted from the medical records of each infant: birthweight; gestation; presence of non-lethal congenital malformation; the lowest appropriate concentration of inspired oxygen between 48 and 72 hours; the highest appropriate concentration of inspired oxygen between 48 and 72 hours; and the worst base deficit between 48 and 72 hours. A CRIB score relating to day 3 of life, CRIB72, was then calculated according to the original scoring system.18 Where they were available, data reporting the cranial ultrasound appearances on or around day 3 of life were also recorded: a variety of ultrasound machines were used across the study centres although all images were reported according to the Papile classification.21 Data on disability were collected for each surviving infant at a corrected age of 18 months by mailing a questionnaire to the child’s health visitor.22 For the purposes of this study, we have defined a child with severe disability as one who cannot sit unsupported by 18 months and/or is blind, and/or is deaf, requiring bilateral hearing aids, and/or is more than 12 months behind in any other field of development.23Specific aspects of fine motor control, communication, or respiratory disease were not considered because of difficulties in case definition. Disability was therefore a dichotomous outcome: a child was either severely disabled at 18 months according to our definition or was not.
Predictive variables considered were birthweight, gestation, appearances on cranial ultrasound scans and CRIB score on day 3 of life. Birthweight, gestation, and CRIB score were regarded as continuous variables. Appearances on cranial ultrasound scan were treated either as ordinal data (grades 0–4) or, for simplicity, converted into a dichotomous outcome (grade 0–2 vs grades 3 or 4).
In those infants surviving until the end of day 3, predictive models for (i) subsequent death before hospital discharge and (ii) severe disability at 18 months in survivors were fitted to the data using regression techniques.24 Each model was assessed by creating receiver operating characteristic (ROC) curves and calculating the area below the curve, Az,25 as well as by the Hosmer-Lemeshow test of goodness of fit.26 Models were compared by comparing the areas under the respective ROC curves.27
For use in clinical practice, models were transformed into 8 × 2 tables and positive predictive values and likelihood ratios were calculated along with their 95% confidence intervals.28The positive predictive value here represents the number of infants who have a positive “test” and actually experience the outcome of interest (true positives) divided by the total number of infants with a positive “test.” The likelihood ratio is defined as the ratio of true positive “tests” to false positive “tests.”
Results
Four hundred and twenty eight infants were born in the six units over the study period. Forty six infants died in the first 72 hours of life, leaving 382 babies to contribute data to the analyses. Eighty five of these infants had either missing CRIB data or missing ultrasound data, leaving 297 infants contributing to the models predicting death. Of 254 infants surviving to age 2 with CRIB scores and ultrasound data, 14 were not assessed at 18 months, leaving 240 infants for analysis of models predicting severe disability. The characteristics of babies with and without missing data are shown in table 1.
Characteristics of study infants (those infants excluded because of missing data are compared with those with complete data)
Forty three babies died after 3 days of life: median age at death day 13, range day 4 to day 308. The common pathologies leading to death included respiratory distress syndrome (n=22), intraventricular haemorrhage (n=15), and pulmonary interstitial emphysema (n=5). Of the 14 severely disabled children included in the analysis, five were blind, four were deaf, seven could not sit unsupported at 18 months and nine had clinically significant delay in another area of development. Most cranial ultrasound scans were performed on day 3 of life: mode day 3 of life, median day 3 of life (quartiles day 2 of life, day 5 of life). Cases were excluded because the notes could not be traced (n=12), the physiological data required to calculate the CRIB score were missing (n=21), no cranial ultrasound scan had been performed (n=77) or follow up data were not available (n=22).
MODEL PREDICTING DEATH BEFORE DISCHARGE FROM THE NEONATAL UNIT
On univariate analysis, the risk of death was significantly associated with birthweight, gestation, CRIB72 and ultrasound appearances; p < 0.05. On multiple logistic regression, CRIB72 (p < 0.0001) and ultrasound appearances (p = 0.0035) remained independently significant while birthweight (p = 0.9345) and gestation (p = 0.1778) were no longer significantly associated with the risk of death. The univariate models using birthweight and gestational age at birth and the multivariate model using CRIB 72 and ultrasound appearances are described in table 2. All three models achieved satisfactory goodness of fit using the Hosmer-Lemeshow χ2statistic.26 In addition, the area under the curve associated with the model combining CRIB72 and ultrasound appearance was significantly greater than the area associated with both the model based on birth weight alone (p = 0.0011) and the model based on gestation alone (p = 0.0436).
Accuracy and validity of models predicting death before discharge
Table 3 shows the positive predictive values and likelihood ratios associated with the model based on CRIB72 and ultrasound appearances.
Positive predictive values (PPV) and likelihood ratios (LR) associated with model predicting death before discharge
MODEL PREDICTING SEVERE DISABILITY AT 18 MONTHS CORRECTED AGE
On univariate analysis, CRIB72 (p = 0.0051) and ultrasound appearance (p = 0.0193) were both significantly associated with the risk of being severely disabled at 18 months. Neither birthweight (p = 0.0755) nor gestation (p = 0.0719) were associated with this outcome. On multiple regression analysis, both CRIB72 (p = 0.0293) and ultrasound appearance (p = 0.0405) remained independently associated with the risk of severe disability: Log odds severe disability = 3.5871 −(0.1882×CRIB72)−(1.5241×USS); Az(SE) = 0.675411 (0.133221); Hosmer-Lemeshow χ2 = 5.089515; p = 0.89. The corresponding predictive values and likelihood ratios for this model are shown in table 4.
Positive predictive values (PPV) and likelihood ratios (LR) associated with model predicting severe disability at 2 years
Discussion
In this cohort 48% of the babies who died before discharge from the neonatal unit did so within the first 72 hours of birth. This is a smaller proportion than was found in the study by Meadow et al 1 probably because our population was heavier and more mature, but still justifies our decision to concentrate on developing a model(s) predicting the outcome for very low birthweight or preterm babies only if they have survived the first three days of life. We also concentrated on developing models that might help provide answers to highly relevant clinical questions:(1) what are the chances that this baby will survive to be discharged home; and (2) if this baby does survive to go home, what are the chances that s/he will be severely disabled? This approach to predicting outcome has not to our knowledge been reported elsewhere, although others have tried to predict outcome before birth,12 at or shortly after birth,14-16 and at discharge home.13
We believe we have gone some way towards demonstrating the feasibility of developing simple clinically useful models. We specifically tried to identify those infants we considered to have severedisability as it is a concept that clinicians, parents, and lay people readily understand, and it seems to be clinically and practically most relevant when considering prognosis.29-32 Cranial ultrasonography is a valid and reliable tool for assessing brain damage33 34 and is an accepted routine investigation in neonatal practice; and the CRIB score is a simple, robust instrument for measuring clinical risk and illness severity which is calculated easily using only six variables, all of which are usually routinely available.35-37 However, certain methodological issues prevent the immediate application of our models. Although the infants with missing data were similar in their characteristics and outcomes, we cannot be sure what effect their exclusion had on the overall results. Neonatal practice has also changed dramatically over the past 10 years with the introduction of antenatal steroids and surfactant treatment, and although CRIB remains a valid score (G Parry, personal communication), it is not clear what effect this might have on the models presented here. It is also recognised that treatment may sometimes be electively withdrawn and that this practice can be influenced by cranial ultrasound appearances.38 Without being able to adjust for this practice, our results might, therefore, overestimate the predictive capacity of the models presented. A prospective study that takes these issues into account and validates the models described39is therefore needed.
As there is good evidence that neonatal survival has improved,40 the positive (and negative) predictive values from this study will have changed and there would be advantages to considering the likelihood ratio rather than the predictive value when trying to predict outcome. Assuming the validity of the models is maintained and the current death rate and prevalence of disability for any given population are known, the risk of either outcome can be extrapolated using the likelihood ratio along with a simple nomogram for Bayes’s theorem.28 41 It is worth noting, however, that likelihood ratios between 0.01 and 10 add relatively little to what is known before the results of any “test” are available.28 Only the likelihood ratio associated with a CRIB score > 4 plus IVH grade 3 or 4, when predicting either death or severe disability, is big enough (9.8 and 24.2 respectively) to alter significantly the post test probability.
Our results suggest that assessment of illness severity permits a more accurate estimate of prognosis than use of birthweight, gestation, or cranial ultrasound scans alone. However, no single piece of evidence can provide a definitive estimate of prognosis for individual babies.42 Any score is based on population characteristics that may not pertain to the individual baby, relates to outcome data that are by definition already “out of date,” and, when derived from relatively small cohorts, will give estimates which lie within wide confidence limits. Clinicians’ clinical judgement is a powerful tool in its own right43 and scoring systems can only provide an additional insight into prognosis. They should never be used in isolation to make clinical decisions, particularly about withdrawal of active treatment, and we would encourage clinicians to continue to use an holistic approach.