Prediction of severe retinopathy of prematurity using the WINROP algorithm in a birth cohort in South East Scotland
- Chinthika Piyasena1,
- Catherine Dhaliwal2,
- Heather Russell3,
- Ann Hellstrom4,
- Chatarina Löfqvist5,
- Ben J Stenson6,
- Brian W Fleck7
- 1Neonatal Unit, Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, Edinburgh, UK
- 2Department of Histopathology, Royal Infirmary of Edinburgh, Edinburgh, UK
- 3Department of Ophthalmology, Gold Coast Hospital, Southport, Queensland, Australia
- 4Department of Pediatric Ophthalmology, Sahlgrenska Academy, The Queen Silvia Children's Hospital, Göteborg, Sweden
- 5Department of Ophthalmology, Institute of Neuroscience and Physiology, The Sahlgrenska Academy, Göteborg, Sweden
- 6Neonatal Unit, Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, Edinburgh, UK
- 7Department of Ophthalmology, Princess Alexandra Eye Pavilion, Edinburgh, UK
- Correspondence to Dr Chinthika Piyasena, Neonatal Unit, Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, 51 Little France Crescent, Edinburgh EH16 4SA, UK;
- Received 19 March 2013
- Revised 24 July 2013
- Accepted 30 July 2013
- Published Online First 28 August 2013
Purpose We tested the ability of the ‘Weight, IGF-1, Neonatal Retinopathy of Prematurity (WINROP)’ clinical algorithm to detect preterm infants at risk of severe Retinopathy of Prematurity (ROP) in a birth cohort in the South East of Scotland. In particular, we asked the question: ‘are weekly weight measurements essential when using the WINROP algorithm?’
Study design This was a retrospective cohort study. Anonymised clinical data were uploaded to the online WINROP site, and infants at risk of developing severe ROP were identified. The results using WINROP were compared with the actual ROP screening outcomes. Infants with incomplete weight data were included in the whole group, but were excluded from a subgroup analysis of infants with complete weight data. In addition, data were manipulated to test whether missing weight data points in the early neonatal period would lead to loss of sensitivity of the algorithm.
Results The WINROP algorithm had 73% sensitivity for detecting infants at risk of severe ROP when all infants were included and 87% when the complete weight data subgroup was analysed. Manipulation of data from the complete weight data subgroup demonstrated that one or two missing weight data points in the early postnatal period lead to loss of sensitivity performance by WINROP.
Implications The WINROP program offers a non-invasive method of identifying infants at high risk of severe ROP and also identifying those not at risk. However, for WINROP to function optimally, it has to be used as recommended and designed, namely weekly body weight measurements are required.
What is already known on this topic
The Weight, IGF-1, Neonatal Retinopathy of Prematurity (WINROP) clinical algorithm has been shown to reliably identify preterm infants at high risk of developing severe ROP.
The WINROP algorithm has been validated in a number of countries including Sweden, North America and Switzerland, but not to date in the UK.
WINROP is designed to use weekly postnatal weight data.
What this study adds
The WINROP algorithm has been validated in a cohort of preterm infants in the UK, with sensitivity for predicting severe ROP of 87.5%.
WINROP sensitivity for detecting severe ROP was reduced when infants with incomplete weight data were included in the cohort.
For WINROP to function optimally, it has to be used as recommended and designed, namely weekly body weight measurements are required.
Retinopathy of prematurity (ROP) is a leading cause of preventable blindness, worldwide.1 Timely treatment is normally, though not always, effective in preventing blindness.2 It is vital that those infants at risk of blindness due to ROP are examined at the correct times. Well-developed screening guidelines are in place.3 ROP is classified according to the International Classification of ROP, with vascular development in three retinal zones (zones 1–3), five stages of disease severity (stage 1–5) and the presence of abnormally dilated and/or tortuous posterior retinal blood vessels (‘plus’ disease).4 The smaller and more immature an infant is at birth, the greater the likelihood of developing severe retinopathy.5 The current UK national screening guidelines state that it is good practice for all infants born at less than 1501 g and/or less than 32 weeks’ gestational age to have routine eye examinations.3 These examinations are technically difficult and must be performed by an experienced ophthalmologist. The first examination is performed 4–6 weeks after birth, and examinations continue 1–2 weeks until the retina is vascularised into zone 3, usually after 36 completed weeks postmenstrual age (PMA).3 The examinations are distressing for the infant.6 ,7 Only a small minority of screening examinations result in a decision to treat.3 Changes that could make the screening process more efficient, by improved targeting of examinations are needed.
The current UK screening guideline is based solely on risk factors at birth that is, birth weight and gestational age.3 Postnatal factors are not taken into account. Poor postnatal weight gain is a recognised risk factor for the development of severe ROP.8 ,9 A computerised clinical algorithm ‘Weight Insulin-like growth factor (IGF-1) Neonatal Retinopathy Of Prematurity’—‘Weight, IGF-1, Neonatal Retinopathy of Prematurity (WINROP)’ was developed in Sweden, using weekly weight measurements and levels of IGF-1 to determine the risk of developing severe ROP.9 Other similar risk models have been developed.10 ,11 WINROP was designed to ‘alarm’ whether an infant’s weekly weight and IGF-1 measurements did not increase sufficiently. More recent work has demonstrated that weekly weight measurements alone, without the need for IGF-1 measurements, are sufficient to generate accurate prediction of the development of severe ROP.12 Using WINROP to target the intensity of the screening programme for infants at risk of developing severe ROP, with more frequent examinations for those who ‘alarm’ and less frequent examinations for those who do not ‘alarm’ has been suggested.13
WINROP was validated in Sweden with a sensitivity of 100%.14 The programme has been further validated in a number of countries. Sensitivity for the prediction of severe ROP was 90% in Switzerland,15 90.5% in Brazil,16 100% and 98.6% in North America13 ,17 and 84.7% in Mexico.18
Reports validating WINROP have used weekly weight measurements, as per the recommendations and design of the WINROP program (http://www.winrop.com). We have assessed the WINROP algorithm in a UK setting, where in some hospitals, including our own, infants are not always weighed weekly.
Patients and methods
The study took place in the Simpson Centre for Reproductive Health, Edinburgh, between December 1999 and June 2009. The hospital has around 7000 births per year, and the neonatal unit has around 800 admissions per year.
The inclusion criteria were all infants born before 32 weeks’ gestational age and cared for in the neonatal unit during the study period.
The exclusion criteria were:
Infants for whom the minimum data set of weights was not available. We defined this as birth weight, a weight for the 34th or 35th week PMA and at least one other weight between these times.
The presence of a clinical condition that could cause a disproportionately high body weight—for example, hydrocephalus.
Infants who did not have complete retinopathy screening. The criteria for discontinuing screening were: ROP follow-up until ROP in zone 3 was regressing on two consecutive examinations, retinal vascularisation in zone 3 was reached or treatment was performed.
Transfer out of the unit before 34 weeks PMA.
Transfer into the unit after 34 weeks PMA.
All infants had ROP screening performed by a consultant ophthalmologist. ROP was classified according to the International Classification of ROP.4 From 1999 to December 2003, ‘threshold’ ROP as defined by the CryoROP study was treated.19 From January 2004, Type 1 ROP as defined by the Early Treatment for Retinopathy of Prematurity (ETROP) study was treated.2 Infants greater than or equal to 32 weeks’ gestational age but less than 1501 g birth weight were also screened in the UK, but were not included in this study, in keeping with previously published case series evaluating WINROP.13
Anonymised data (date of birth, gestational age, birth weight, gender, weight measurements and dates, results of retinopathy screening) were collected from the neonatal unit electronic patient records and from screening records held by the ophthalmologists. Gestational age was calculated using the hierarchy: (1) antenatal ultrasound measurement and (2) maternal estimate of last menstrual period.
Serial weight measurements, approximately every 7 days, were entered into the secure online WINROP data system until a high-risk alarm occurred or until the infant reached 34 weeks PMA. If a high-risk alarm did not occur, the infant remained low risk. One infant did not have a weight recorded for the 34th week, but fulfilled all other criteria and the weight recorded for the 35th week was used.
The result (low risk, or high-risk alarm) and the timing of a high-risk alarm were recorded for each infant. The result was compared with the maximum stage of retinopathy that was reached. The retinopathy was classified as: no ROP, mild ROP (stage 1 or 2 ROP in zone 2 or 3, without plus disease) or severe ROP (any ROP in zone 1, stage 2 ROP in zone 2 with plus disease or any stage 3 ROP). This classification was chosen in order to be consistent with that used in previous evaluations of WINROP.17
The subgroup of infants who had complete weight data with a weight measurement for each week of life from birth until 34 weeks PMA were subject to an additional, separate analysis. In order to test the effect of incomplete weight data on WINROP performance, data from the complete weight data subgroup were then manipulated by excluding some weight data from upload to the WINROP algorithm.
The study was assessed by the scientific officer of the regional research ethics service, and a decision was made that National Health Service (NHS) ethical review was not needed, as the study only used data acquired as part of usual care.
The flow chart for patients included in the study is given in figure 1. Six hundred and ninety-three infants were included in the study.
The characteristics of all infants, and of the complete weight data subgroup, are shown in table 1. The subgroup with complete weight data was more mature at birth, and relatively fewer developed ROP or were treated for ROP.
ROP and WINROP outcomes
Complete weight data subgroup
Of the 410 infants who had complete weekly weight data, 16 (4%) developed severe ROP. Fourteen of the 16 infants who developed severe ROP triggered a high-risk alarm. The median time from birth to alarm was 9 days (range 1–44). The median time from alarm to the time when severe ROP was clinically diagnosed was 42 days (range 14–68).
Two infants were treated for severe ROP but did not trigger a high-risk alarm. One was a male infant born at 26+0 weeks’ gestation with a birth weight of 1045 g (78th centile). He had severe ROP diagnosed and treated on day 52 chronological age at a PMA of 33+2 weeks. He had a Grade III intraventricular haemorrhage (IVH) but no other neonatal morbidity. He was enrolled in the Neonatal Insulin Replacement Therapy in Europe Study (NIRTURE) trial20 and was randomised to receive insulin therapy for the first seven days of life. The second was a female infant born at 27+4 weeks’ gestation with a birth weight of 1295 g (90th centile). She had severe ROP diagnosed on day 81 chronological age at a PMA of 39+0 and received laser treatment on day 109 at a PMA age of 43+0. She had a grade IV IVH with ventricular dilatation that was not deemed to require a shunt. Both infants had stage 3 ROP in zone 2, with plus disease, at the time of treatment.
Three hundred and one of 693 infants received a high-risk alarm at a median age of 19 days (range 1–67 days). Sixty-two of 85 (73%) infants with severe ROP triggered a high-risk alarm prior to a clinical diagnosis of severe ROP. The median time from birth to alarm was 22.5 days (range 1–62 days). The median time from the alarm to the time when severe ROP was clinically diagnosed was 41 days (range 12–81 days). Thirty-eight of these 62 infants had laser treatment. The median time from the alarm to treatment was 45.5 days (range 17–89 days).
The remaining 23 infants who developed severe ROP were not identified by WINROP. Seventeen of these infants received laser treatment. Two infants who were not identified by WINROP triggered a high-risk alarm 1 to 2 weeks after the clinical diagnosis of severe ROP. The infants with severe ROP not predicted by WINROP tended to have a greater weight for gestation at birth than the infants who triggered a high-risk alarm. Their median birth weight z score was 0.32 compared with a median score of –0.76 for the 301 infants that triggered a high-risk alarm (p<0.05). Two of the infants not identified by WINROP had generalised oedema following necrotising enterocolitis.
Nineteen infants triggered a high-risk alarm at birth. These were growth-restricted infants with a median birth weight z score of −2.32. Six of these infants developed severe ROP.
Performance of WINROP in predicting severe ROP
The sensitivity, specificity, positive predictive value and negative predictive value of WINROP in predicting severe ROP in all infants and in the complete weight data subgroup are shown in table 2.
WINROP correctly identified 48 infants with severe ROP with incomplete weight data and failed to identify 21 infants with severe ROP with incomplete weight data. We examined the effect of fewer weight measurements on the accuracy of WINROP by manipulating data from the 14 infants with severe ROP and complete weight data who triggered an alarm. If the first weight measurement after birth weight (week 1) was omitted, WINROP correctly identified 11/14 infants (25–28 weeks’ gestational age) though unsurprisingly 4.8 days later. Three infants (28–31 weeks’ gestational age) did not trigger an alarm. Omitting the first and the second weight measurements after birth weight (weeks 1 and 2), WINROP identified 10/14 infants (25–28 weeks’ gestational age), though 10.1 days later, and missed four infants (28–31 weeks’ gestational age). Two of 14 infants were missed when weight measurements for weeks 2 or 3 were withheld (28 and 29 weeks’ gestational age), but there was no average delay in the timing of the alarm for the remaining 12 infants (25–31 weeks’ gestational age). Missed weight data points therefore resulted in a reduced sensitivity by WINROP to trigger a high-risk alarm.
We have shown that the WINROP algorithm had 73% sensitivity for detecting infants at risk of severe ROP when all infants were included and 87.5% when the complete weight data subgroup was analysed. Manipulation of data from the complete weight data subgroup demonstrated that one or two missing weight data points in the early postnatal period lead to loss of sensitivity performance by WINROP.
The quality of our study was limited by incomplete weight data in the cohort and by relatively small number of infants who developed severe ROP in the complete weight data subgroup. Nevertheless, important issues have been raised by our study.
Incomplete weight data in our cohort resulted in a reduced sensitivity by WINROP to trigger high-risk alarms. The sensitivity of WINROP to predict the development of severe ROP was 72.94% in the whole group. We confirmed the effect of missing weight data by manipulating data from our complete weight data subgroup. Missing weight data points, especially at 1 week and 2 weeks after birth, resulted in fewer high-risk alarms by WINROP in infants who went on to develop severe ROP. While this should not be surprising, it is an important point when using WINROP to predict the development of severe ROP. In our hospital, neonates are not routinely weighed every week. Weight measurement of infants when they are critically ill is poorly standardised. Allowing for the presence of arterial and venous lines and endotracheal tube adds measurement uncertainty, and associated handling means that the risk of accidental extubation or loss of vascular access cannot be eliminated. In the absence of pressing concerns about fluid balance, weighing is sometimes deferred until the infant is more stable. The value of weekly body weight data in order to use the WINROP program must be weighed against the risks associated with weight measurement.
The sensitivity of WINROP in detecting severe ROP in the subgroup who had complete weight data was similar to that of the Swiss,15 Brazilian16 and Mexican18 cohorts. While the reported sensitivity of WINROP thus far for detecting severe ROP is high, it is unrealistic to expect a sensitivity of 100% in all contexts. When weight data were complete, ROP that met treatment criteria was missed in 2/16 of our infants. We found no documentation, though acknowledging the limitation of retrospective case note review, of generalised oedema in either of the two infants. In the male infant, there were no clear clinical features that might have contributed to a negative alarm. However, this infant had been enrolled into the NIRTURE trial and was randomised to receive insulin therapy for the first 7 days of life. Insulin therapy is known to increase IGF-1 levels and result in less weight loss during the first week of life,20 ,21 thereby enhancing postnatal weight gain. It is conceivable that his weight at 1 week postnatal was greater than it would have been had insulin intervention not been given. Weight at 1 week postnatal is an important data point in the WINROP algorithm, and it is possible that insulin intervention leads to the failure of WINROP to trigger a high-risk alarm. In the female infant, the presence of hydrocephalus, though not requiring a ventricular shunt may have contributed towards non-physiologic weight gain. This combined with being large for gestational age may have caused WINROP to not trigger an alarm. It should be emphasised that WINROP is not suggested as an alternative to clinical eye examination, but as an adjunct to improve the efficiency of ROP screening programmes.13
In the infants with severe ROP correctly identified by WINROP, the alarm occurred at a median time of approximately 3 weeks after birth and a median time of approximately 6 weeks before severe ROP became evident clinically. Six infants who developed severe ROP had a WINROP high-risk alarm at birth, as they were very small for gestational age (SGA). SGA infants are known to be at increased risk of severe ROP,5 and the WINROP program is designed to produce a high-risk alarm at birth in infants who are very SGA. Recognition of infants at risk of severe ROP several weeks before the onset of ROP may provide an opportunity to explore postnatal preventive strategies. Early nutritional interventions (enteral and parenteral) might modify the course of ROP.22 ,23 Supplementation with IGF-1 and essential fatty acids is being investigated.24 Advance warning of the possible need for ophthalmic intervention is helpful to clinicians and parents.
The successful use of postnatal weight gain in the WINROP program opens up the question of whether other postnatal factors might be included in a clinical risk model. Postnatal factors known to be associated with an increased risk of the development of severe ROP include infection25 and intraventricular haemorrhage.26 Ethnicity should also be considered in risk models.11
In conclusion, while two cases in our series were not identified as at risk of severe ROP, the WINROP program nevertheless offers a non-invasive method of identifying infants at high risk of severe ROP and also identifying those at low risk. However, for WINROP to function optimally, it has to be used as recommended and designed, namely requiring weekly body weight measurements.
Contributors CP collected neonatal data and uploaded this data onto the WINROP website; analysed the study data; and wrote the initial draft of the paper. CD and HR compiled the ophthalmology data. AH and CL developed the WINROP algorithm, assisted in our use of the WINROP website, contributed advice throughout the project and contributed to the final draft of the paper. BJS lead all neonatology aspects of the study and contributed to the final draft of the paper. BWF conceived the study, lead all ophthalmology aspects of the study and wrote the final draft of the paper.
Competing interests Access to the WINROP system online is provided without charge to any qualified participant. PremaCure AB has rights to the WINROP system. Drs Lofqvist and Hellstrom own shares in a company controlling PremaCure AB.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data sharing permitted by NHS Lothian for this study.