Aim: To determine the sensitivity and specificity of three newly developed pulse oximeters in the detection of hyperoxaemia, defined as an arterial partial pressure of oxygen (Pao2) of > 80 mm Hg.
Methods: Spo2 readings from three oximeters (Agilent Viridia (AgV), Masimo SET (MaS), Nellcor Oxismart (NeO)) were documented in 56 infants (median gestational age at birth 35.5 weeks, range 24–41) whenever an arterial blood gas was taken for clinical purposes. Blood samples were analysed within one minute in a Radiometer ABL 505 blood gas analyser and OSM3 co-oximeter.
Results: Between 280 and 291 blood gases were analysed for each instrument; 105–112 showed a Pao2 > 80 mm Hg. At an upper alarm limit of 95%, the three instruments detected hyperoxaemia with 93–95% sensitivity. Specificity at this alarm level ranged from 26 to 45%. The mean (SD) difference between arterial oxygen saturation and Spo2 (bias) was −0.25 (2.5)% for AgV, −0.06 (2.5)% for MaS, and −0.91 (2.6)% for NeO (p < 0.01, NeO v AgV and MaS).
Conclusion: These instruments detected hyperoxaemia with sufficient sensitivity at an upper alarm limit of 95%, but showed differences in their specificity, which was probably related to differences in measurement bias.
Statistics from Altmetric.com
- AgV, Agilent Viridia
- MaS, Masimo SET
- NeO, Nellcor Oxismart
- Pao2, arterial partial pressure of oxygen
- Sao2, arterial oxygen saturation
- pulse oximeter saturation
Determination of the partial pressure of oxygen in arterial blood (Pao2) is regarded as the optimum method for assessing oxygenation.1 In clinical practice, however, oxygenation is increasingly monitored by pulse oximetry, mainly because this technique provides immediate, continuous, and non-invasive information without the need for calibration or sensor replacement.1 Nevertheless, it has two major disadvantages: a high rate of false alarms, most of which are caused by motion artefact, and a reduced sensitivity to hyperoxaemia.1,2 The latter is due to the shape of the oxygen dissociation curve, where large changes in the upper range of Pao2 are associated with small changes in arterial oxygen saturation (Sao2). This is unfortunate, because reliable detection of hyperoxaemia, usually defined as a Pao2 > 80 mm Hg, is crucial for the avoidance of retinopathy of prematurity and other oxygen related diseases.3–5 As software algorithms and probe calibrations differ between instruments and sensors, there may be differences in the upper alarm limit used with an individual manufacturer’s pulse oximeter to optimally detect hyperoxaemia.1,6
Recent developments in pulse oximetry have resulted in a dramatic reduction in false alarm rates,7 which makes these instruments particularly attractive for the neonatal intensive care unit. Nothing, however, is known about their ability to detect hyperoxaemia. We set out to determine the sensitivity and specificity of three recently available pulse oximeters in the detection of hyperoxaemia.
PATIENTS AND METHODS
Fifty six term and preterm infants (33 boys) were enrolled after informed parental consent had been obtained. These infants had been admitted to the neonatal intensive care unit at Hannover Medical School for various reasons (extreme prematurity (18), congenital malformations (21), metabolic disorders (6), respiratory failure in near term infants (7), asphyxia (3), and morbus haemolyticus neonatorum (1)) and had indwelling arterial lines (right or left radial artery) for clinical reasons. Their median gestational age was 35.5 weeks (range 24–41), birth weight 2480 g (range 370–4680), age at study 6 days (range 1–149), and study weight 2680 g (range 430–5800). Forty two infants were mechanically ventilated, and five received continuous positive airway pressure through a nasopharyngeal tube; 48 infants received additional inspired oxygen. Infants with a patent arterial duct, as shown by colour Doppler echocardiography, who had a preductal arterial line were not enrolled. In addition to standard monitoring equipment, which included a pulse oximeter, 46 infants had one and 10 infants had two additional sensors attached to a hand and/or foot; the clinical characteristics of the infants in these subgroups were similar. Instruments used in this study were the Agilent Viridia M3 (AgV; software version B; Agilent, Böblingen, Germany), Masimo SET (MaS; software version 1.13; Masimo, Irvine, California, USA), and Nellcor N3000 Oxismart (NeO; software version 3.04, Mallinckrodt, Pleasanton, California, USA). Sensors used were the Nellcor N20 with NeO and AgV and the Masimo NeoPT with MaS. Care was taken that sensors had good contact with the skin and were shielded against each other and ambient light by opaque foam wraps. Averaging times were 10 seconds for AgV, 8 seconds for MaS, and variable—that is, dependent on signal quality—for NeO.
Whenever an arterial blood sample was taken for clinical purposes, the Spo2 readings on the oximeters were recorded at the precise moment the blood was drawn into the syringe for analysis. Measurements were not timed specifically to include high Pao2 values or to exclude periods with motion and/or differences between pulse and heart rate,6 but Spo2 readings had to be stable for at least 20 seconds before the arterial blood sample was drawn; this was to exclude the possibility that any potential differences between instruments were caused by their variable averaging times. Blood samples were analysed within one minute: Pao2 was measured with a standard blood gas analyser (Radiometer ABL 505; Radiometer, Copenhagen, Denmark), and functional Sao2 with a co-oximeter (Radiometer OSM 3). Co-oximeter measurements were corrected for fetal haemoglobin. Sao2 values were plotted against the difference between arterial and pulse oximeter saturation measurements (Sao2−Spo2), and the mean (bias) and standard deviations (precision) of these differences calculated for each instrument.8,9 Sensitivity was calculated as the proportion of instances with a Pao2 > 80 mm Hg that were associated with an Spo2 value above the threshold, divided by all instances with Pao2 > 80 mm Hg. Specificity was calculated as the proportion of Pao2 readings ≤ 80 mm Hg associated with an Spo2 value below the threshold, divided by the number of all instances with a Pao2 ≤ 80 mm Hg. Statistical analysis was performed using the two sided t test. The study protocol was approved by the ethics committee of Hannover Medical School.
A total of 280 Spo2/Sao2/Pao2 determinations were performed for AgV, and 291 each for MaS and NeO; 105 (112 for AgV) in 27 (24) patients showed a Pao2 > 80 mm Hg (fig 1). A median of 10 measurements (range 2–15) was documented in each infant, with a median of three measurements (range 1–13) showing a Pao2 > 80 mm Hg. Table 1 shows sensitivity and specificity at various potential upper alarm limits. At an upper alarm limit of 95%, all three instruments detected 93–95% of hyperoxaemic episodes—that is, sensitivity to hyperoxaemia was comparable between instruments. Specificity at this threshold value, however, was more variable, ranging from 26% (NeO) to 45% (MaS).
The two highest Pao2 values that were > 80 mm Hg, but associated with an Spo2 ≤ 95% (false negative for hyperoxaemia) were 144 and 92 mm Hg for AgV, 169 and 98 mm Hg for MaS, and 141 and 95 mm Hg for NeO. The lowest Pao2 with Spo2 > 95% (false positive for hyperoxaemia) was 46 mm Hg for AgV, 56 mm Hg for MaS, and 50 mm Hg for NeO (fig 1).
Measurement precision, defined as the standard deviation of the difference between Sao2 and Spo2, was similar between instruments (2.5% for both AgV and MaS, 2.6% for NeO), while bias was smaller with the MaS (mean, −0.06%) and AgV (−0.25%) than with the NeO instrument (−0.91%, p < 0.01 for NeO v AgV and MaS).
The three pulse oximeters investigated in this study were chosen because they were shown to7 or purportedly have a relatively low false alarm rate, making them potentially interesting for use in neonates, in whom false alarms are a major problem.2 Avoidance of hyperoxaemia is particularly important in this age group. As previous studies have shown major differences in the upper alarm limit that has to be used if priority is given to the avoidance of hyperoxaemia,1,6,10,11 we wanted to know whether such differences still exist with current instruments. We found that all three instruments detected hyperoxaemia, defined as a Pao2 > 80 mm Hg, with sufficient sensitivity (93–95%) if the upper alarm limit was set at 95%.
Because we aimed to have a substantial proportion of hyperoxaemic values, we did not restrict our study to preterm infants, as did Bucher et al,6 but also included term infants. As Sao2 measurements were corrected for the proportion of fetal haemoglobin, and pulse oximeters are not influenced by fetal haemoglobin, we are confident that the decision to include term infants did not affect our results. Also, the number of measurements varied between infants. This was because blood gases were only taken for clinical purposes, and arterial lines were removed at the discretion of the attending neonatologist. There was nothing to suggest, however, that this decision in any way biased our results.
The sensitivity of these pulse oximeters was within the same range as that of some conventional instruments,6,10–12 with only minimal differences between instruments. This is probably due to the fact that all instruments measure functional rather than fractional oxygen saturation, which is in contrast with an earlier study,6 which found that an instrument that measured fractional saturation had to be used with an upper alarm limit of 88% if hyperoxaemia were to be detected reliably. With regard to specificity, however, the MaS seemed to perform better than the other two instruments, which may be related to differences in measurement bias. Although these differences were small (< 1%), they may still be relevant, as small changes in Sao2 may be associated with large changes in Pao2 in the hyperoxic range.
As shown in table 1, both sensitivity and specificity depend on the alarm limit used. Sensitivity can be increased by decreasing the upper alarm limit, but the specificity, which is already low, will then decrease even further. This carries the risk of keeping infants hypoxaemic if priority is given to the avoidance of hyperoxaemia. This is because a wide range of Pao2 values may be associated with an Spo2 above the upper alarm limit (fig 1). In this study, the lowest Pao2 value measured with an Spo2 > 95%, however, was always above 45 mm Hg, which is somewhat reassuring.
Although our findings largely reflect blood physiology, i.e. the shape of the O2 dissociation curve, one might expect that the new generation instruments investigated here would have performed better than their first generation counterparts. Their major strength, however, is their improved handling of conditions with low signal to noise ratios, for which they apply frequency domain analysis (AgV), adaptive filtering (NeO), or a combination of frequency and time domain analysis and adaptive filtering (MaS).13 Such conditions, however, although not specifically excluded, occur comparatively rarely and are thus unlikely to have significantly affected our results.
A principal problem with a study validating medical instruments is that software algorithms are constantly updated by manufacturers. For example, Nellcor’s Oxismart technology has already been replaced by Oxismart/XL, and Masimo had two major software upgrades since initiation of this study. All three manufacturers, however, assured us that, although handling of motion and/or low perfusion has changed with these software upgrades, the “lookup tables” used for deriving the displayed saturation values from the red/infrared ratios that are actually measured have not.
In conclusion, this study has shown that all three pulse oximeters investigated are similarly sensitive in the detection of hyperoxaemia, but differ somewhat in their specificity, probably reflecting differences in measurement bias. Nevertheless, despite these encouraging results, we do not recommend using pulse oximetry as the sole means of monitoring oxygenation in the neonatal intensive care unit. We must also warn against uncritically applying the definition used for hyperoxaemia in this study, although valid for preterm infants receiving additional inspired oxygen, to healthy neonates breathing room air at sea level, in whom both an Pao2 above 80 mm Hg and an Spo2 above 95% are part of normal infant physiology.14,15
We are grateful to the medical and nursing staff of the neonatal intensive care unit at Hannover Medical School for their help with this study and to Agilent, Masimo, and Nellcor for providing the instruments investigated in this study.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.