Article Text

PDF

Validation of the continuous glucose monitoring sensor in preterm infants
  1. K Beardsall1,2,
  2. S Vanhaesebrouck3,
  3. A L Ogilvy-Stuart2,
  4. C Vanhole3,
  5. M vanWeissenbruch4,
  6. P Midgley5,
  7. M Thio6,
  8. L Cornette7,
  9. I Ossuetta8,
  10. C R Palmer9,
  11. I Iglesias6,
  12. M de Jong4,
  13. B Gill7,
  14. F de Zegher3,
  15. D B Dunger1
  1. 1Department of Paediatrics, University of Cambridge, Cambridge University Hospitals NHS Foundation Trust, Hills Road, UK
  2. 2Neonatal Unit, Cambridge University Hospitals NHS Foundation Trust, Hills Road, UK
  3. 3Department of Woman and Child, University of Leuven, Leuven, Belgium
  4. 4VU University Medical Center, Amsterdam, The Netherlands
  5. 5Simpson Centre for Reproductive Health, Royal Infirmary of Edinburgh, 51 Little France Crescent, Edinburgh, Scotland
  6. 6Hospital Universitari Sant Joan de Déu, Passeig Sant Joan de Deu, Esplugues, Barcelona
  7. 7Leeds General Infirmary, Great George Street, Leeds, UK
  8. 8Luton and Dunstable Hospital, Luton, UK
  9. 9Centre for Applied Medical Statistics, Department of Public Health and Primary Care, University of Cambridge Forvie Site, Institute of Public Health, Robinson Way, UK
  1. Correspondence to K Beardsall, Neonatal Unit, Rosie Maternity, Addenbrookes Hospital, Hills Road, Cambridge CB2 2QQ, UK; kb274{at}cam.ac.uk

Abstract

Objective Recent studies have highlighted the need for improved methods of monitoring glucose control in intensive care to reduce hyperglycaemia, without increasing the risk of hypoglycaemia. Continuous glucose monitoring is increasingly used in children with diabetes, but there are little data regarding its use in the preterm infant, particularly at extremes of glucose levels and over prolonged periods. This study aimed to assess the accuracy of the continuous glucose monitoring sensor (CGMS) across the glucose profile, and to determine whether there was any deterioration over a 7 day period.

Design Prospectively collected CGMS data from the NIRTURE Trial was compared with the data obtained simultaneously using point of care glucose monitors.

Setting An international multicentre randomised controlled trial.

Patients One hundred and eighty-eight very low birth weight control infants.

Outcome measures Optimal accuracy, performance goals (American Diabetes Association consensus), Bland Altman, Error Grid analyses and accuracy.

Results The mean (SD) duration of CGMS recordings was 156.18 (29) h (6.5 days), with a total of 5207 paired glucose levels. CGMS data correlated well with point of care devices (r=0.94), with minimal bias. It met the Clarke Error Grid and Consensus Grid criteria for clinical significance. Accuracy of single readings to detect set thresholds of hypoglycaemia, or hyperglycaemia was poor. There was no deterioration over time from insertion.

Conclusions CGMS can provide information on trends in glucose control, and guidance on the need for blood glucose assessment. This highlights the potential use of CGMS in optimising glucose control in preterm infants.

View Full Text

Statistics from Altmetric.com

What is already known on this topic

  • The CGMS is well tolerated in adults and young children with diabetes but has not been validated for use in the newborn.

  • CGMS can be used in children with diabetes to improve glucose control but its use in intensive care is limited.

  • CGMS correlates well with point of care devices over short periods but there is limited data regarding the accuracy of CGM sensors after 3 days.

What this study adds

  • That CGM sensors are well tolerated in very low birth weight infants requiring intensive care during the first week of life.

  • That CGM data correlates well with point of care devices, with minimal bias, and the accuracy does not deteriorate over a 7 day period.

  • CGMs monitoring could be used to detect trends in glucose levels and thus risk of both hyperglycaemia and hypoglycaemia.

Introduction

Hyperglycaemia is a common problem in neonatal intensive care and has been associated with increased mortality and morbidity.1 ,2 Tight glucose control has been shown to be beneficial to subgroups of intensive care patients,3 however the benefits of reducing hyperglycaemia have to be balanced against the risk of hypoglycaemia.4 This is particularly relevant in preterm infants requiring intensive care when wide fluctuations in blood glucose are common5 and the developing brain is likely to be more vulnerable to both hypoglycaemia and hyperglycaemic insults. In these small babies, glucose monitoring clinically relies on intermittent arterial sampling or heel pricks and is not performed as frequently as in paediatric or adult intensive care. Continuous glucose monitoring has been developed for use in the management of diabetes mellitus and has been found to help to improve glucose control.6 There is therefore increasing interest, and controversy, in the clinical use of continuous glucose monitoring sensors (CGMS) to help in the management of glucose control in intensive care7 ,8 and in new born infants at risk of both hyperglycaemia and hypoglycaemia.9 However, there have been concerns about its accuracy at detecting hypoglycaemia.8 A pilot study of CGMS in preterm babies demonstrated it to be well tolerated in these infants.10 However, there are limited data available regarding its accuracy at extremes of glucose control, or over prolonged periods in preterm infants requiring intensive care. CGMS data collected, using the CGMS System Gold (Medtronic, Minneapolis, Minnesota, USA), as part of the NIRTURE clinical trial11 provided the opportunity to assess the CGMS, both at extremes of glucose control, and to determine if there was any deterioration in performance with time from insertion.

Methods

Subjects and measurements

The subjects were the control infants participating in the Neonatal Insulin Replacement Therapy in Europe Trial (NIRTURE ISRCTN78428828). This was an international multi-centre study investigating the role of early insulin in VLBW infants.11 Ethical and regulatory authority approval (Eudract No 2004-002170-34) were obtained for each centre, and the protocol is in the public domain.12 Infants were recruited from 8 European centres between February 2005 and August 2007. Written informed parental consent was obtained prior to recruitment which was within 24 h of birth. Exclusion criteria included maternal diabetes and major fetal congenital abnormalities. Blood glucose measurements were undertaken for the purposes of the study a minimum of 3 times a day. Whenever possible, these samples were taken to coincide with blood sampling for clinical reasons such as the taking of bloods for blood gas analyses or routine daily bloods, not at preset times. In addition, many babies had additional blood glucose measurements taken for clinical reasons. The clinical need for blood sampling was not simply for assessment of glucose levels but also for example with blood gas analyses. There will have been an increased sampling if there were concerns about blood glucose levels but this would only help to identify accuracy around particularly important clinical thresholds.

Infants were followed to the expected date of delivery (EDD).

The continuous glucose monitoring device (CGMS)

The subcutaneous glucose sensor was inserted into the lateral thigh within 24 h of birth, from which continuous measurements were recorded for 7 days. Use of the CGM (Medtronic) in the preterm infant has been described.10 In essence, the device is composed of a disposable glucose oxidase-based platinum electrode sensor that is inserted subcutaneously. This sensor catalyses interstitial glucose, and this is converted to an averaged glucose value every 5 min. Glucose values outside the range of 2.2 to 24.0 mmol/l (40–430 mg/dl) are recorded as <2.2 mmol/l (40 mg/dl) or >24 mmol/l (430 mg/dl), respectively. The CGMS was calibrated in each centre using the results of the point of care (PoC) blood glucose analyses of samples that were used for clinical management. The meter would not calibrate if samples were outside range and requested a repeat sample to be taken to check calibration. The samples were a combination of arterial or capillary samples. The measurements were undertaken on a number of different point of care devices: Leuven, Genk and Leeds, Radiometer (Radiometer Medical ApS, Denmark); Cambridge and Barcelona, Bayer Elite XL (Bayer, Germany); Luton, Medisense (Abbott, Illinois, USA), Amsterdam, HemoCue Glucose Analyser (HemoCue AB); and Edinburgh, Yellow Springs Instrument (YSI (UK), Hampshire, England). The CGM data were downloaded at completion of the 7 day study period with the data previously being blinded to the clinical and research teams. All CGMS profiles were examined to identify any periods when there was insufficient calibration of the sensor to be confident about sensor glucose levels or periods when the sensor was ‘failing’ prior to removal. This involved removing data immediately prior to removal of a sensor when the current (ISIG) value was <10 in accordance with the recommendations from Medtronic. The mean (SD) duration of CGMS recordings was 156.18 (29) h (6.5 days). In total, a period of 2.95 days (0.2% of the data) were removed from the analyses, representing data from 9 babies median (range) 8.75 (2.33 to 11.75) h of data.

Statistical analyses

There are currently no criteria or guidelines for assessment of accuracy or performance of continuous glucose monitoring devices and so we applied a number of previously established methodologies for assessment of new analytical methods as listed below. Performance of the CGMS was evaluated against blood glucose results obtained simultaneously using PoC devices.

Optimum accuracy

Criteria for optimal accuracy defined by Medtronic Technology have been established as a correlation between the sensor and the meter readings of at least 0.79, and a mean absolute error of not more than 28%.13 For each day of data collection, the correlation coefficient between the meter readings and the glucose sensor were calculated. The mean absolute error was calculated by taking the absolute difference from the PoC value, and then dividing by the PoC value, and then averaging all the pairs of data.

Performance goals of the American Diabetes Association (ADA) consensus

Based on the ADA consensus statement, we used the recommended performance goal of a total analytical error of <10% for all glucose values.14

Bland Altman analyses

To assess agreement between the monitor and the meter, we used the Bland Altman Method.15 To determine the limits of agreement, we calculated the average difference between the two measurements ± 1.96 SD of the difference.

Clarke Error Grid

The Clarke Error Grid separates the scatter plot of data into five zones of clinical significance.16 The zones are defined according to the presence of error in accuracy of the measurement combined with the severity of the consequence of subsequent treatment error. A represents absence of error, B where the two methods disagree by >20% but this does not lead to treatment error. Zone C, D and E represent increasingly large and potentially harmful discrepancies between the evaluation and reference methods. If a new method has a percentage (>95%) in zones A and B then it is considered clinically acceptable.

Consensus Error Grid

The Consensus Error Grid separates the scatter plot of data into 5 zones of clinical significance with the zones defined by the Consensus opinion of a panel of 100 clinicians and differs from the Clarke Error Grid in that the zones are in continuity.17

Accuracy

We analysed the specificity and sensitivity of the CGMS to correctly detect incidences of hyperglycaemia (defined as blood glucose >10 mmol/l) and hypoglycaemia (defined as blood glucose <2.6 mmol/l), with respect to the comparative blood glucose level determined by PoC glucose measurements.

Results

CGMS data were available from 188 control infants, there were 4 controls, where CGMS data were not available, due to a combination of early death, or sensor failure. The characteristics of the cohort are shown in table 13. The mean (SD) duration of CGMS recordings was 156.18 (29) h (6.5 days), with a total of 5207 paired glucose levels (CGMS and point of care glucose level). There were 584 CGMS readings <2.6 mmol/l and 680 readings >10 mmol/l. Tolerance of the sensor was excellent with no reports of side effects at the site of insertion, no evidence of infection, oedema or problems of skin debridement with removal.

Table 1

Clinical characteristics of study babies

Optimum accuracy

The correlation between sensor (CGMS) and PoC glucose readings = 0.94 with mean absolute error of 8.8%. The correlation did not significantly differ during the week but was lowest on day 1 (R2=0.91) and highest on day 4 (R2=0.96). The data easily met the optimal accuracy criteria defined by Medtronic, with a mean absolute error ranged from 6.9 to 12.7%. There was no statistically significant increase in error over the 7 day study period with the error being highest on day 1 (table 2).

Table 2

Mean absolute per cent difference between sensor (CGMS) and PoC glucose measurements and per cent of values within the 10% reference range (ADA criteria).

Performance goals

The CGMS failed to meet the ADA criteria of <10% error for all values (table 2). The error was particularly marked at lower glucose levels when >50% of readings had an error in excess of 10%. The mean absolute error did not increase over the 7 day study period, with the largest error being on day 1.

Bias and performance at extremes

The Bland Altman analyses for all glucose measurements demonstrated that the CGMS marginally under reads but that there is essential no bias (figure 1). The mean (95% CI) difference is −0.02 (−1.6 to 1.7) mmol/l. However, for readings <4.0 mmol/ there is slight bias to over read. In contrast for glucose levels >10 mmol/l there was a slight bias to under read.

Figure 1

Bland Altman: CGMS versus PoC. Bland Altman plot for the CGMS glucose versus the PoC glucose level (mmol/l). Horizontal axis represents the mean of the sensor (CGMS) and point of care glucose measurement (mmol/l). Vertical axis represents the difference between the sensor (CGMS) and the point of care glucose measure (mmol/l).

Clinical significance: error grid analyses

Table 3 provides a comparison of Clarke Error Grid scores for each day. These demonstrate that, based on the established criteria that >95% of readings must be in either zone A or B, the CGMS met the criteria on all study days. The least values in zone A was on day 1. The Clarke Error Grids for day 6 is shown in figure 2. When plotted using the Consensus error grid, the CGMS results also met the criteria for clinical acceptability (data not shown).

Figure 2

Clarke Error Grid analyses day 6. Horizontal axis represents point of care glucose level; vertical axis represents sensor (CGMS) glucose reading. Region A: values within 20% of the reference sensor, Region B: values outside of 20% but would not lead to inappropriate treatment, Region C: values leading to unnecessary treatment, Region D: values indicating a potentially dangerous failure to detect hypoglycemia or hyperglycemia, Region E values that would confuse treatment of hypoglycemia for hyperglycemia and vice-versa.

Table 3

Comparison of Clarke Error Grid scores for each day

Sensitivity and specificity

The sensitivity of the CGMS to detect hyperglycaemia (>10 mmol/l) was 88% with specificity 98%, and a positive predictive value of 90% and negative predictive value of 98%. The sensitivity of the CGMS to detect hypoglycaemia (glucose <2.6 mmol/l) was 17% and the specificity was 100%, with a positive predictive values of 40% and negative predictive value of 99%.

Discussion

This is the first study to investigate the use of CGMS in a large population of preterm infants. It demonstrated the sensors to be well tolerated without the problems of sensor failure that have previously been reported,18 ,19 and that CGMS accuracy in the preterm infant is similar to that in the diabetic population,19 with the device maintaining performance over an extended period of up to 7 days.

However the CGMS failed to meet the ADA performance criteria, which is in accordance with the previous reports in other populations.19 The mean absolute error was greatest on day 1, which may be related to the more rapidly changing glucose levels on day 1, as babies are undergoing the metabolic transition from in utero to ex utero life. At this time, as CGMS readings tend to lag behind blood glucose levels, and with rapid fluctuations in glucose levels, this may lead to more significant errors occurring.20 Alternatively the error may be due to limited data for calibration, as calibration is dependent on blood glucose readings entered into the device within a 24 h period. Therefore on day 1, the CGMS has less recent data to use for calibration potentially leading to reduced accuracy.

The CGMS met the criteria for clinical accuracy based on the Error Grid analyses, but these zones were defined in the context of management of patients with diabetes, and the use of point of care glucose monitoring devices. Therefore, the interpretation of these results in the setting of neonates in intensive care, where interventional glucose levels are different needs to be cautious. The lower limit of the plot lies at 4 mmol/l whereas in the unwell newborn hypoglycaemia is commonly defined as <2.6 mmol/ and clinical management is dependent on accurate results at these lower thresholds. Therefore, the interpretation of clinical significance would be improved by a plot specifically designed for the clinical significance of glucose levels in the newborn or preterm infant. The Consensus grid has the advantage of considering the significance of glucose values down to levels of 2.8 mmol/l, whereas the lowest band for the Clarke Error Grid is 5 mmol/l. This is significant in that the clinical threshold for hypoglycaemia in the neonate is usually considered to be 2.6 mmol/l. We have considered developing a similar grid for the newborn and are in discussions regarding this. However as hypoglycaemia may be a normal physiological phenomenon, clinical practice is strongly biased towards the clinical context and condition of the baby rather than single readings.

Our Bland Altman analyses for all glucose levels demonstrated that the CGMS marginally under reads, as had been reported in the literature.21 ,22 However at low glucose levels, this was not the case with a tendency to over read. This combined with low sensitivity and specificity for hypoglycaemia means caution must be advised regarding the accuracy of individual readings to diagnose hypoglycaemia. However, the value and potential uses of continuous glucose monitoring in the future should not be based on accuracy of single readings but on the ability of it to warn of changes in patterns and trends of glucose control without the need for repeated blood sampling. As such its use in clinical practice should be to provide additional information to guide management and on the need for assessment of true blood glucose at critical clinical thresholds in a more timely fashion.

In addition, newer CGM systems with modified sensors and monitors are now thought to have better accuracy, particularly at lower glucose levels. Importantly, the clinical advantages of the use of CGMS in the intensive care setting would not be to diagnose hypoglycaemia, but to provide continuous information regarding trends in glucose levels. This could then guide clinical management and the need for accurate glucose monitoring at specific treatment thresholds. This could be of particular benefit in preterm babies when it is beneficial to minimise handling and reduce invasive blood sampling.

One of the weaknesses of this study is the comparison of CGMS measurements against different PoC methodologies for glucose analyses, and using both arterial and capillary blood samples. It would have been better to compare the CGMS measurement against a gold standard: a measurement taken on only one blood sample type, and measured on one specific well validated glucose analyser. However, this was not possible within the clinical trial setting. However, the data provide a comparison of the CGMS with what is currently being used for clinical management of patients. In addition, the CGMS updates its recorded glucose data every 5 min, therefore in the clinical setting any delay in entering the blood glucose value into the monitor could result in an artefactual difference in glucose levels between point of care and CGMS.

It also must be remembered that all these assessment criteria for the CGM are designed for assessment of point of care meters which give single readings on which clinical management is based. These methods are not designed for assessing the performance of devices which collect a wealth of continuous data where trends are more important than absolute values.23 The potential clinical advantage being in providing early warning of trends to threshold levels of glucose when true blood glucose assessment may be needed. Alternative strategies for assessment of continuous monitoring devices have been suggested but are not well established. We did not attempt to assess the impact of rate of change of glucose levels or the impact of insulin infusions on the accuracy of readings but this would benefit from further analyses.

This study demonstrates that the CGMS is well tolerated in the preterm infant. Glucose results correlate well with PoC devices with minimal bias. The CGMS provides much more detailed continuous data and therefore although single readings may not be accurate in the diagnosis of hypoglycaemia the device can help to detect periods of both hyperglycaemia and hypoglycaemia. They provide a valuable research tool for assessment of glucose control in the newborn. In the future the more detailed information on trends in glucose levels may help to guide clinical management but real time devices need to be validated in clinical practice before being adopted as part of routine clinical care.

Acknowledgments

The NIRTURE study was sponsored by Cambridge University Hospitals NHS Foundation Trust, Novo Nordisk provided the insulin aspart, and Medtronic provided the continuous glucose monitoring system and sensors. Funding was provided by an unrestricted grant from Novo Nordisk, with additional financial support from Medtronic, Leeds General Infirmary research fund, Clinicip Consortium, University of Cambridge Department of Paediatrics, the NIHR Cambridge Comprehensive Biomedical Research Centre and NHS Research and Development. NIRTURE was adopted by UK Medicines for Children Research Network. The authors are grateful for the time and support provided by the Trial Steering Committee which included Professor Kate Costeloe, Professor Peter Brocklehurst, Professor Tim Cole, Dr Robert Tasker, Alison Baillie, Cheryl France and the Data Monitoring Committee Professor Mike Preece, Professor Diana Elbourne and Dr Edmund Hey. The authors are grateful to the staff in the CTU, Mark Wilson, Heather Withers, Diane Picton, and Barry Widmer. To those involved in patient recruitment including research fellows Roy McDougal and Imran Ahmed, research nurses Lynn Thomson, Yvonne Millar, Anneke Cranendonk and Lisa Auty, Claire Theyskens. ME Wilinska assisted in producing the Consensus graphs. The authors thank all the staff who cared for the babies and the families who agreed to participate.

References

View Abstract

Footnotes

  • Funding NovoNordisk, Clinicip consortium, Medtronic, Leeds General Infirmary.

  • Competing interests None.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.