Article Text
PDF
Abstract
Objective To undertake a cost-effectiveness analysis that compares pulse oximetry as an adjunct to clinical examination with clinical examination alone in newborn screening for congenital heart defects (CHDs).
Design Model-based economic evaluation using accuracy and cost data from a primary study supplemented from published sources taking an NHS perspective.
Setting Six large maternity units in the UK.
Patients 20 055 newborn infants prior to discharge from hospital.
Intervention Pulse oximetry as an adjunct to clinical examination.
Main outcome measure Cost effectiveness based on incremental cost per timely diagnosis.
Results Pulse oximetry as an adjunct to clinical examination is twice as costly but provides a timely diagnosis to almost 30 additional cases of CHD per 100 000 live births compared with a modelled strategy of clinical examination alone. The incremental cost-effectiveness ratio for this strategy compared with clinical examination alone is approximately £24 000 per case of timely diagnosis in a population in which antenatal screening for CHDs already exists. The probabilistic sensitivity analysis suggests that at a willingness-to-pay (WTP) threshold of £100 000, the probability of ‘pulse oximetry as an adjunct to clinical examination’ being cost effective is more than 90%. Such a WTP threshold is plausible if a newborn with timely diagnosis of a CHD gained just five quality-adjusted life years, even when treatment costs are taken into consideration.
Conclusion Pulse oximetry as an adjunct to current routine practice of clinical examination alone is likely to be considered a cost-effective strategy in the light of currently accepted thresholds.
Statistics from Altmetric.com
Introduction
Congenital heart defects (CHDs) are the most common group of congenital malformations and one of the leading causes of infant death in the developed world, accounting for more deaths than any other type of malformation.1,–,4
Current routine screening for CHDs includes a mid-trimester anomaly ultrasound scan and a postnatal clinical examination. Both investigations have a relatively low detection rate and a number of babies are discharged from hospital and may die before a CHD is diagnosed.4
Most newborns with a CHD can be stabilised and treated. However, if critical defects are not detected sufficiently early then cardiovascular collapse and death are potential sequelae. Timely recognition of these conditions is likely to improve outcome and the accuracy of pulse oximetry screening in this respect has been investigated.3 A recent large UK study has provided further evidence that pulse oximetry may be useful in detecting CHDs in apparently healthy newborns.4
What is already known on this topic
▶ Congenital heart defects (CHDs) are the most common group of congenital malformations with a reported incidence of between 4 and 10 per 1000 live born infants.
▶ CHDs are one of the leading causes of infant death in the developed world.
▶ Timely recognition of these conditions is likely to improve the outcomes for infants.
What this study adds
▶ Pulse oximetry adds value to existing screening procedures and is likely to identify cases of critical CHDs which would otherwise go undetected.
▶ Pulse oximetry as an adjunct to current practice is a cost-effective addition according to acceptable UK thresholds.
In this article we report the results of a model-based economic evaluation including data from this study alongside which primary data on costs and resource use were collected prospectively.4 The objective of the economic evaluation is to compare the cost and cost effectiveness of pulse oximetry as an adjunct to current practice with current practice alone based on a principal outcome of cost per timely diagnosis.
Methods
The test accuracy study is reported elsewhere.4 Briefly, the study was performed in six large maternity units with delayed verification in test-negative cases. The index test of pulse oximetry was performed in 20 055 newborns prior to discharge from hospital. Newborns not achieving predetermined oxygen saturation thresholds underwent the reference standard of echocardiography. All other infants were followed up until the age of 12 months by examining regional and national congenital anomaly and cardiac registries and clinical follow-up. The study compared the accuracy of the index test in detecting CHDs from predefined definitions of severity of CHDs (table 1). Our objective was to identify serious, critical and significant CHDs from these categories to provide a timely diagnosis. For the economic analysis, we developed a decision analytic model, using as a starting point the model developed by Knowles et al.5
Prevalence data by disease category used to populate the models
Knowles et al carried out a model-based economic evaluation to explore the cost effectiveness of pulse oximetry using published sources but acknowledged that primary data on test accuracy and costs were required.5
Screening strategies
All infants enrolled in the primary study underwent pulse oximetry but we also considered the likely pathway the infant would follow in the absence of pulse oximetry, according to current practice.
The model strategy referred to as clinical examination (figure 1) represents routine practice and refers to the clinical pathway followed if pulse oximetry had not been carried out (comparator strategy). Typically, all infants receive a routine clinical examination by a trained professional before discharge from hospital. If the clinical examination result is abnormal the infant will receive further tests depending on the abnormality or concern raised. If a CHD is suspected, the infant will usually require a diagnostic echocardiogram.
Model structure. CE, clinical examination; CHD, congenital heart defect; PO, pulse oximetry.
The model strategy referred to as pulse oximetry as an adjunct to clinical examination is the intervention strategy. Infants for whom pulse oximetry is considered normal (test negative) will receive the same clinical examination and will, in the model, follow the same pathway as they would had they not undergone pulse oximetry. If at any stage the clinical examination provides an abnormal result which is suggestive of a CHD, the infant will be sent for a diagnostic echocardiogram. If a non-CHD abnormality is suspected the infant will be sent for alternative appropriate tests.
All infants with an abnormal pulse oximetry test result (test positive) receive an expedited clinical examination (ECE) which is primarily checking for signs of a CHD and is usually undertaken sooner than the routine clinical examination. If the ECE shows an abnormality (test positive) the infant will undergo diagnostic echocardiogram. If the ECE suggests no CHD-related abnormality (test negative) the infant will undergo a second pulse oximetry test. If this is still abnormal (test positive) a diagnostic echocardiogram will be performed.
If the second pulse oximetry test is normal (test negative) infants receive the remaining component of the routine clinical examination that was not part of the ECE, that is, the non-cardiac aspects (such as checking the eyes, hips etc), and continue to follow routine care as described in the comparator strategy, receiving non-CHD related interventions if appropriate.
In both strategies, the diagnostic echocardiogram is confirmatory and any infant with a CHD will receive appropriate treatment.
Model structure
A decision tree model was developed by TreeAge Sofware Inc (Williamstown, Massachusetts, USA) to represent the alternative strategies. The pathways of the model represent, as far as possible, the clinical procedures carried out in the study. The decision tree structure is presented in figure 1.
Newborns with a CHD present are assumed to have the CHD confirmed by echocardiogram assessment. If the diagnosis of a CHD is missed in the neonatal period it could be confirmed by follow-up on the congenital anomaly or other clinical registers. The model time horizon is a period of 1 year, which is the year of follow-up in the study.
Clinical data used in the model
The decision model was populated with prevalence data from the primary accuracy study.4 The population prevalence is presented in subgroups according to CHD severity and excludes infants in whom CHD was suspected following antenatal ultrasound screening and subsequent fetal echocardiogram. These prevalence data are presented in table 1.
The recommended threshold for a positive pulse oximetry test is preductal or postductal saturation <95% or a differential of >2% between the two; this was used in the base case of the model.4 The test accuracy data used for the model were drawn as far as possible from the current study but are supplemented if necessary by data from Knowles et al.5 However, the accuracy study did not have a control arm and the ‘clinical examination alone’ strategy, which represents routine treatment, was always performed with knowledge of the pulse oximetry result. It was not possible to estimate the accuracy of clinical examination alone in the current primary study because the clinicians were not blind to the result, and so it was estimated in the following way: the probability of a newborn having an abnormal ECE following an abnormal pulse oximetry result was recorded in the current study. The critical and serious (major CHD) cases identified by clinical examination after a normal pulse oximetry test were also recorded. However, data on clinical examination findings were not collected for significant and non-significant subgroups, nor the false positives following a normal pulse oximetry result, so these data were supplemented from Knowles et al.5 Thus the sensitivity and specificity for ‘clinical examination alone’ were estimated by taking a weighted average of the ECE (which followed an abnormal pulse oximetry test) and the routine clinical examination (which followed a normal pulse oximetry test). This approach followed the method described in Knowles et al5 and is based on the methods used by Wren et al 2000.6
Costs and resource use data
We carried out a time and motion study. Staff from all study sites were asked to record the time taken to carry out the test during a single week towards the end of the study. The week was chosen to represent a time when the study was in full swing, having overcome any teething problems or training issues. All centres completed and returned their results for the tests carried out that week. In total, 312 completed forms were received. Overall, the pulse oximetry test was carried out with an average duration of 6.9 min. The minimum recorded time to carry out the test was 1 min and the maximum amount of time was 30 min (median 5 min). The time recorded to complete the pulse oximetry test relates only to the time taken to undertake the test as consent had been given prior to the birth.
The total cost of carrying out the pulse oximetry test including staff time, equipment and assuming a midwife carried out all tests was estimated to be £6.24. Use of the mean is appropriate to capture the outliers.7 8 Table 2 presents the unit costs used in the model compared with the costs used in the Knowles' study appropriately inflated.
Unit costs and resource use for current model-based analysis and Knowles et al (2005)5
Much of the remaining required cost data used in the model were from secondary costs presented in Knowles et al.5 All cost data used, from published sources or the current study, are reported in 2009 UK sterling (£) prices, having been appropriately inflated using the Health and Community Health Services pay and price index.9
The time taken for the ECE was recorded in the study. We assumed the routine clinical examination would be carried out by a Specialty Trainee 2 and would take the same average time of 8.57 min to complete as with the ECE. The cost of the clinical examination was estimated at £5.43.
Analysis
The model was constructed to investigate the cost effectiveness of the screening strategies: pulse oximetry as an adjunct to clinical examination versus clinical examination alone. The analyses were carried out from an NHS perspective presented in terms of incremental cost-effectiveness ratios (ICERs) and based on outcome of cost per timely diagnosis. ‘Timely diagnosis’ of CHDs refers to diagnosis confirmed by echocardiogram before preoperative collapse or death of the infant.
Two sensitivity analyses were carried out in which the threshold for an abnormal pulse oximetry was changed from that used in the study to thresholds employed in other studies. Two additional analyses in which an aspect of the input costs was changed were also carried out. The impact of all these changes on the ICER is considered.
Changing the thresholds
First, the threshold was changed to <95% in both limbs or a differential of >3%, as used by de-Wahl Granelli et al.10 Second, the threshold was changed to <95% for postductal saturations alone, as used in other published studies.11 12
Changing an aspect of the costs
First, the cost was changed to include hospitals without echocardiography available on site. The cost for echocardiography assessment was £115.57.5 We doubled the cost of the echocardiography, the rationale being to account for those hospitals without available echocardiography and thus an additional cost of travel. Second, the cost of the pulse oximetry test was changed. The median duration for carrying out the test was 5 min. Using the median instead of the mean duration (6.9 min) changed the total cost of carrying out the test from £6.24 to £4.68.
Results
The modelled clinical examination alone strategy was estimated to detect approximately 91.5 additional cases of clinically significant CHD per 100 000 live births, at an estimated cost of £614 000 for the strategy. The intervention strategy of pulse oximetry as an adjunct to clinical examination would detect 121.4 cases of CHD per 100 000 live births at a cost of £1 358 800 for the strategy. Thus an additional cost of £744 700 would be required to detect approximately 30 additional cases (29.9 cases in table 3) of a timely diagnosis per 100 000 live births. The ICER which represents the additional cost per additional case of a timely diagnosed case of CHD per 100 000 live births is approximately £24 900 (table 3).
Summary of base-case results and deterministic sensitivity analysis results
The results for the analysis are presented in a cost-effectiveness acceptability curve (figure 2). The curve presents the probability that a screening strategy is cost effective at society's willingness to pay (WTP) for a timely diagnosis of a clinically significant CHD. Interpretation of the results presented in cost per quality-adjusted life years (QALYs) is straightforward for decision makers because it is the acceptable threshold used by decision-making institutions. To interpret the results in the absence of QALYs, we consider the following. The WTP threshold used by the National Institute for Health and Clinical Excellence (NICE) in the UK is £20 000 per QALY; this means that below and up to this threshold society is assumed to be willing to pay £20 000 per QALY for a year of life in full health. Thus from the diagram, for society to be willing to pay £100 000, a newborn with timely diagnosis of a CHD would need to gain just five QALYs. The technology used to treat these conditions is advancing all the time. High-quality multicentre follow-up studies are lacking but indications are that the majority of these children reach early adulthood in good health.5
Cost-effectiveness acceptability curve
From figure 2, we can see that at the £100 000 WTP threshold, the probability that ‘pulse oximetry as an adjunct to clinical examination’ would be cost effective is considerably greater than 90%. So an infant who receives a timely diagnosis as a result of the strategy of ‘pulse oximetry as an adjunct to clinical examination’ would only have to achieve five QALYs to reach a 90% chance of the strategy being considered cost effective.
Deterministic sensitivity analysis
In the first set of deterministic sensitivity analyses, we changed the pulse oximetry thresholds used to determine a positive case. For the first threshold change, the relative cost effectiveness for the intervention of pulse oximetry as an adjunct to clinical examination compared with modelled clinical examination alone appeared slightly more favourable. However, for the second threshold change, the cost effectiveness of the intervention was slightly less favourable. It should be stressed that no inference should be made from these sensitivity analyses about what the appropriate cut-off should be (table 3).
When costs were changed, for the first cost change there was some sensitivity to the increased cost of a echocardiogram but in line with intuition given the increase (table 3). For the second cost change, the results of the one-way deterministic sensitivity analysis using the median instead of the mean costs for the test show an expected favourable effect on the ICER, which is reduced to £19 500. Using the median, as opposed to the mean duration, would risk underestimating the true cost of carrying out the test.8
Discussion
Pulse oximetry as an adjunct to clinical examination alone is estimated to cost an additional £744 700 required to detect approximately 30 additional cases (29.9 cases in table 3) of a timely diagnosis per 100 000 live births compared with clinical examination alone. The ICER of this intervention compared with routine clinical screening is approximately £24 000 per case of timely diagnosis in a population in which antenatal screening for CHD already exists. The probabilistic sensitivity analysis suggests that at a WTP threshold of £100 000, the probability of this intervention being cost-effective is more than 90%. Such a WTP threshold is plausible if a newborn with timely diagnosis of a CHD gained just five QALYs, even when treatment costs are taken into consideration.
The costs presented in this analysis are likely to represent the upper limit of carrying out pulse oximetry screening for a number of reasons. First, in our analysis the test was always assumed to be carried out by a midwife, although healthcare assistants whose time is relatively less costly can perform the test. Second, the average cost of the test is influenced by the presence of a few outliers: a test duration of 30 min was reported only once; 20 min was the next longest duration and on the whole most (71%) staff took less than 10 min to carry out the test and this is likely to improve over time. Finally, our analysis assumes additional knock-on tests such as echocardiograms are carried out by a consultant, but a trained technician can also perform these tests. Overall, over time the costs associated with testing are likely to fall.
The strength of this analysis is that it is based on accuracy and cost data from a primary study. Cost and resource use data collected via a time and motion study represent the first primary cost data for use of the pulse oximetry test in a UK setting. A limitation is that, in the pragmatic design of the study, it was not possible to measure the accuracy of ‘clinical examination alone’ since it was always performed after a negative pulse oximetry test and it was not possible for the clinicians to be blind to the result of the pulse oximetry test. However, this limitation has been overcome by using appropriate statistical methods and sensitivity analysis.
Knowles et al5 carried out a model-based economic evaluation to assess the cost effectiveness of pulse oximetry using an outcome of cost per timely diagnosis, but based entirely on secondary data for accuracy and costs. The key difference related to the time assumed for carrying out the test. Our estimated ICERs are higher than those estimated by Knowles et al,5 but differences are accounted for by assumptions about cost and resource use.
The techniques involved in treating CHDs are advancing so evidence on the life expectancy and quality of life of the infants, as they develop through childhood and young adulthood, is improving all the time. The early evidence suggests that the majority1 5 of these infants survive into young adulthood at least, and with good quality of life. If we assume that they achieve as few as just 5 years worth of full health overall, which could allow for discounting, then the average costs per QALY associated with the timely diagnosis would be approximately £5000. This falls well within the acceptable thresholds used by decision makers such as NICE. Treatment costs are not included in this ICER. Although treatment costs will be incurred for the infants identified, if CHDs were not detected with pulse oximetry in the neonatal period but became apparent later, treatment costs would still be necessary. Furthermore, in the absence of a timely diagnosis, treatment costs are likely to be higher while survival prospects risk being poorer.
Our colleagues have reported that ‘Pulse oximetry is a safe, non-invasive, feasible, reasonably accurate test, which has a sensitivity superior to that of antenatal screening and clinical examination. This study enhances the strong body of evidence which indicate the clear potential benefits of introducing predischarge pulse oximetry screening as a routine procedure.’4 Our paper suggests pulse oximetry is also likely to be considered cost effective.
Acknowledgments
The main phase of this trial was funded by a UK National Institute for Health Research Health Technology Assessment Programme and will be published in full in the Health Technology Assessment journal series. The views and opinions expressed are those of the authors and do not necessarily reflect those of the Department of Health. The authors thank the members of the joint Steering/Data Monitoring Committee for their assistance throughout the project: Dr Gerben ter Riet (Chair; Academisch Medisch Centrum, Universiteit van Amsterdam), Mrs Suzie Hutchinson (Little Hearts Matter), Dr Carole Cummins (University of Birmingham), Dr Sam Richmond (Sunderland Royal Hospital) and Dr Stavros Petrou (University of Oxford).
Footnotes
-
Funding This review was funded by the National Institute for Health Research Health Technology Assessment Programme (06/06/03).
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.