Article Text
PDF
Abstract
Background Continuous quality improvement (CQI) collaboration has not eliminated the morbidity variability seen among neonatal intensive care units (NICUs). Factors other than inconstant application of potentially better practices (PBPs) might explain divergent proficiency.
Objective Measure a composite morbidity score and determine whether cultural, environmental and cognitive factors distinguish high proficiency from lower proficiency NICUs.
Design/methods Retrospective analysis using a risk-adjusted composite morbidity score (Benefit Metric) and cultural survey focusing on very low birth weight (VLBW) infants from 39 NICUs, years 2000–2014. The Benefit Metric and yearly variance from the group mean was rank-ordered by NICU. A comprehensive survey was completed by each NICU exploring whether morbidity variance correlated with CQI methodology, cultural, environmental and/or cognitive characteristics.
Results 58 272 VLBW infants were included, mean (SD) age 28.2 (3.0) weeks, birth weight 1031 (301) g. The 39 NICU groups’ Benefit Metric improved 40%, from 80 in 2000 to 112 in 2014 (P<0.001). 14 NICUs had composite morbidity scores significantly better than the group, 16 did not differ and 9 scored below the group mean. The 14 highest performing NICUs were characterised by more effective team work, superior morale, greater problem-solving expectations of providers, enhanced learning opportunities, knowledge of CQI fundamentals and more generous staffing.
Conclusion Cultural, environmental and cognitive characteristics vary among NICUs perhaps more than traditional CQI methodology and PBPs, possibly explaining the inconstancy of VLBW infant morbidity reduction efforts. High proficiency NICUs foster spirited team work and camaraderie, sustained learning opportunities and support of favourable staffing that allows problem solving and widespread involvement in CQI activities.
- composite morbidity score
- continued quality improvement
- very low birth weight infants
- potentially better practices
Statistics from Altmetric.com
- composite morbidity score
- continued quality improvement
- very low birth weight infants
- potentially better practices
What is known about this subject?
Despite risk adjustment and widespread availability of continuous quality improvement (CQI) methodology, significant variability in premature infant morbidity rates persists among neonatal intensive care units (NICUs).
Even NICUs that collaborate closely within formal networks and share potentially better practices have widely variant morbidity rates.
NICU cultural features, environment and human factors related to CQI are largely unmeasured and not reported in published investigations.
What this study adds?
We used an established risk-adjusted composite morbidity score in premature infants to rank order 39 NICUs according to proficiency over a 15-year period.
An environmental survey was completed at each NICU that explored each NICU’s cultural and cognitive milieu, educational efforts and CQI methods.
High proficiency NICUs foster human factors like spirited team work, camaraderie, sustained learning opportunities and employ favourable staffing models that allow problem solving and widespread involvement in CQI activities.
Introduction
‘All happy families are alike; each unhappy family is unhappy in its’ own way’, so writes Leo Tolstoy in the epic first line of Anna Karenina. Is this classic kindred insight relevant to our work family in the neonatal intensive care unit (NICU)? Widespread dissemination of continuous quality improvement (CQI) methodology has reduced certain morbidities in the very low birth weight (VLBW) infant population,1 2 although the precise role specific potentially better practices (PBPs) play in lowering the morbidity burden remains uncertain. Persistent variance in morbidity rates after risk adjustment is perplexing because the same CQI methodology, PBPs and evidence-based medicine guidelines are available to all NICUs, especially members of organised collaborations like the Vermont Oxford Network (VON). The familiar rejoinder—but not every NICU adopts the right PBPs, nor regularly measures compliance, so of course outcomes vary—is a partially true but unsatisfactory response. It is well known that NICUs working closely together collectively agreeing on the same PBPs and CQI tools, report widely divergent morbidity rates,3–5 so the question is why?
Our long-time CQI workgroup developed the first risk-adjusted composite morbidity and mortality score (the Benefit Metric) for VLBW infants, a practical tool for measuring overall CQI performance to study whole cultures of excellence.3 In our eight hospital collaborations, some NICUs consistently outperformed others despite common CQI methods and PBPs. This investigation follows up our initial study with an expanded group of 39 North American NICUs using an exploratory cultural survey to investigate variant morbidity reduction efforts. Do highly proficient NICUs (ranked by the Benefit Metric) share traditional CQI tools and PBPs and cultural, environmental, social and/or cognitive characteristics heretofore not appreciated by traditional quality improvement studies?
Methods
We conducted a retrospective review of each NICUs’ VON yearly VLBW infant report, in which 39 NICUs across North America participated (see additional contributions). Eligible infants included all VLBW infants born 401–1500 g from years 2000 to 2014. Authorised users at VON hospitals have access to their NICU patient-level data and aggregate data for the entire VON through the Nightingale Internet Reporting System. An authorised individual at each participating hospital downloaded their deidentified data and provided it to the authors. The VON did not provide any individual hospital or patient-level data for this study and played no role in the design, conduct, analysis or reporting of this research.
Formulation of the risk-adjusted composite morbidity score is detailed in our previous manuscript.3 The Benefit Metric is a proficiency score that weighs the eight major VLBW infant morbidities equally: chronic lung disease, grades 3–4 intraventricular haemorrhage, periventricular leucomalacia, stages 3–4 retinopathy of prematurity, any late infection, necrotising enterocolitis, focal intestinal perforation and discharge weight <10th percentile as defined in the VON Manual of Operations.6 Online supplementary figure describes the Benefit Metric that includes a mortality deduction and uses standard VON risk adjusters. After calculation of the yearly Benefit Metric for each NICU, a group mean was determined for years 2000–2014, and each NICU’s result was compared against the average. The 39 NICUs were partitioned into three proficiency groups: those with Benefit Metric cumulative scores significantly above (group A green), at (group B yellow) or below the mean (group C red).
Supplementary file 1
Without knowing the rank order of their cumulative composite morbidity score, each NICU completed a 103-question survey describing their facility relevant to CQI efforts from years 2000 to 2014. The survey was formulated by the coauthors with input from the VON POD group,3 then circulated to four independent NICU CQI experts for revision (modified Delphi method). The survey was completed at each site by a multidisciplinary group of at least four to five NICU providers answering by consensus: a neonatologist(s), bedside nurse, neonatal nurse practitioner and/or respiratory therapist. Each multidisciplinary team completed the survey by early 2016 and was instructed to answer each question as generally applying to years 2000–2014, although they could use group discretion if a survey question was particularly relevant to years 2012–2014. The survey included 39 inquiries focused on CQI methodology and medical therapies, 22 related to NICU staffing models, 8 physical structure queries, 30 addressed cultural, social and/or cognitive characteristics and 4 were sham questions. We hypothesised that higher proficiency NICUs as defined by the Benefit Metric would have similar survey results and particularly regarding those inquiries related to culture, social and cognitive characteristics.
For each NICU, their Benefit Metric difference from the group mean was calculated yearly. The weighted mean difference over the study period and the 95% CIs were calculated (number of admissions as the weight). Based on this mean difference, the 39 NICUs were ranked as follows: (A) NICUs with the lower 95% CI above zero (significantly better than the group mean) were ranked by this lower 95% confidence limit (group A green), (B) NICUs with 95% CIs containing zero (not different from the group mean) were ranked by the Benefit Metric point estimate (group B yellow), (C) NICUs with the upper 95% CIs below zero (significantly below the group mean) were ranked by this upper 95% confidence limit (group C red).
To identify any particular survey answer that might distinguish group A NICUs from groups B and C, we used the following guide for the 103 inquiries: (A) were the majority (>50%) of group A answers similar?; (B) was the group A particular response more frequent than group B and group C?; and (C) if yes to both, the particular group A answer had to be at least 15% or more frequent compared with group B and group C when the answer was also more frequent in group B than Group C (absolute percentage, for example, 60% vs 45%), or the group A answer had to be at least 30% more frequent compared with group B and group C combined when group B and group C answers were similar (absolute percentage, for example, 67% vs 37%). Some questions of obviously similar natures were aggregated (taking the mean) and reassessed as a composite answer. Linear regression was used to examine Benefit Metric fluctuation over time. All data analyses were performed using R V.3.1.0 statistical program (R Foundation for Statistical Computing, Vienna, Austria).
Results
Table 1 describes the characteristics of the VLBW infants from the 39 NICUs. Figure 1 illustrates the individual NICU composite morbidity score ranking, the 14 most proficient NICUs (group A green) had yearly composite morbidity scores significantly better than the mean, 16 NICUs (group B yellow) were not significantly different and 9 NICUs (group C red) were significantly below the mean. Figure 2 shows the whole group 39 NICU Benefit Metric improved from 80 in year 2000 to 112 in 2014, a significant reduction in the risk-adjusted VLBW infant morbidity burden of 40% (P<0.001). Figure 2 also shows proficiency groups A, B and C separately; all three groups significantly improved over 15 years (P<0.001); the rate of improvement (line slope) was greatest for group B (P=0.01).
VLBW infant characteristics at the 39 NICUs, born 2000–2014
Thirty-nine NICUs ranked by the Benefit Metric composite morbidity score: high proficiency group A green circlesare significantly greater, group B yellow circlesare not significantly different and lower proficiency group C red circlesare significantly less than the group mean. NICU, neonatal intensive care unit; VLBW, very low birth weight.
Thirty-nine NICUs whole-group Benefit Metric over 15 years and the three proficiency groups plotted individually. All four lines demonstrate significant composite morbidity reduction (see Results). NICU, neonatal intensive care unit.
Figure 3 illustrates for each NICU the relationship between the cumulative 15-year Benefit Metric above or below the group mean (from figure 1) and whether each NICU improved over that period (from the line slopes in figure 2); the most desirable position being the upper right quadrant, least desirable being the lower left quadrant. The specific culture and environment survey descriptors shared in common by the 14 high proficiency group A green NICUs are summarised in box. Online supplementary table shows the entire 103 question survey sorted by category and then colour coded to clarify those questions/answers that either distinguished the high proficiency group A NICUs, or were common to all or nearly all NICUs, or demonstrated no discernable pattern of answers among all NICUs.
Supplementary file 2
![[SP1.jpg]](https://fn.bmj.com/content/fetalneonatal/104/1/F13/DC1/embed/inline-supplementary-material-1.jpg?download=true){kind=link}
Relationship between the cumulative 15-year Benefit Metric from figure 1 that are above or below the group mean (x-axis) and whether each NICU improved over 15 years derived from the line slopes in figure 2 (y-axis). Upper right quadrant indicates both a better and improving composite morbidity score. NICU, neonatal intensive care units.
Discussion
Our large collaborative CQI investigation suggests Tolstoy’s kindred insight might be relevant to neonatology—‘happy NICU families’ are similar; they share generally unmeasured but perhaps indispensable cultural, social and cognitive traits, as well as staffing models that might influence multidimensional CQI interactions. Put another way: Every NICU is perfectly designed to produce the results it gets,7 and Context+Methods=Outcomes.8 Context is typically not characterised in CQI publications9 10; we believe linking an objective composite morbidity metric to ‘culture’ is crucial because variability in proficiency between NICUs cannot be fully explained by traditional risk adjustment, CQI methodology or PBPs. Scrutiny of high-performance, low-morbidity NICUs is of intense interest because if a VLBW infant is discharged home without any major morbidity, the likelihood of good long-term health and neurodevelopment is ≥90%.11–14 Admittedly, we have cast a wide net hoping to find insights into why some NICUs consistently outperform others, but believe there is instructive signal amidst the noise from our survey inclusive of 58 272 VLBW infants spanning 15 years of CQI efforts in 39 NICUs.
First, the 40% overall improvement in the Benefit Metric over 15 years (figure 1) is a tremendous accomplishment reflective of multiple morbidity reduction efforts and collaborative CQI efforts.1–3 Whether this translates to a similar enhancement of long-term health remains to be determined, but there is good evidence to believe such.11–13 Second, our findings suggest minimal-to-no correlation between the risk adjusted composite morbidity metric (a proficiency score) and many established CQI methodologies and PBPs (colour codes red, royal blue and purple in the online supplementary table). This is not surprising: (A) all 39 NICUs are VON members and participate in formal CQI activities, so for example, we would expect the widespread use of checklists, central line teams, donor breast milk and feeding guidelines, and (B) our survey did not determine precise compliance with specific CQI methods and PBPs, thus we could not consistently ascertain why common CQI practices or ‘culture’ features did not correlate with morbidity reduction.
Third, some CQI descriptors inversely correlated with composite morbidity reduction (box and online supplementary table). We speculate: (A) do primary nursing models and formal palliative care teams diffuse personal ‘ownership’ and lessen individual responsibility for infant outcomes?; (B) does singling out NICU workers for recognition unwittingly create hierarchy and resentment?; (C) do fewer NICU windows foster more staff focus?; and (D) the relocation of a particular individual(s) having a positive effect on NICU CQI is not surprising, just seldom quantified in CQI publications.7–10
Culture and environment survey summary descriptors of the particular shared characteristics of the 14 high proficiency group A green circle neonatal intensive care unit (NICUs) in figure 1. Numbers after each descriptor refer to the particular survey question listed in the online supplementary table
Continuous quality improvement features and medical therapies
Neonatologists, nurses, neonatal nurse practitioners and respiratory therapists felt there was sufficient time and effort allotted to make sure the NICU staff understood and contributed to practice changes.4 5
Higher percentage of neonatologists could define the concept of a Plan-Do-Study-Act cycle.1
More overall willingness to adopt new or advanced therapies.2
NICU had a regular, formal teaching conference.3
NICU conducted simulation training, for example, mock codes, to standardise care.6
No formal palliative team.7
Staffing models
Neonatologists rounded on fewer infants per day, plus a greater nurse: infant ratio for levels 2 and 3 infants.3–5
Less nurse turnover.6
Neonatal nurse practitioners rounded on very low birth weight (VLBW) infants.1
No primary nursing.8
Dedicated 24-hour delivery room resuscitation team.2
Paediatric and family medicine trainees did not round on VLBW infants7
Physical structure
Cultural, social and cognitive features
Overall morale rated higher by neonatologists, nurses and neonatal nurse practitioners.6
Greater atmosphere of NICU team work.1
Neonatologist and nurse daily work expectations included direct patient care and encouraged problem solving to determine how to do the job better.2 3
Positive effect of certain providers no longer working in the NICU.5
Parent room or family resource library in the NICU.4
No NICU provider special recognition celebrations or ‘Employee of the Month’ events.7
Sham questions
None.
Fourth, the characteristics common to the most proficient group A NICUs (box and the online supplementary table) include several descriptors of more effective team work, higher morale, greater problem-solving expectations of providers, enhanced learning opportunities, knowledge of CQI fundamentals and favourable staffing models. Horse or cart? Are higher performing group A NICUs ‘happier’ team-oriented cultures because of their intrinsic low morbidity rates, or did proficient group A NICUs first establish positive cultural, cognitive and problem-solving habits then subsequently reduce their VLBW infant morbidities? Fifth, our survey emphasises the need for thoughtful study of NICU staffing models because proficient group A NICUs were characterised by more favourable indices of nurse and neonatologist to infant ratios and less use of trainees rounding. NICU staffing paradigms are rarely detailed in randomised controlled trials or CQI reports, yet workplace personnel mixtures can be a critical variable that significantly affect morbidities.15 16
CQI networks like the VON have reported impressive improvement in certain VLBW infant morbidities and naturally this has been attributed to participation in CQI collaboratives and embracement of standardised methods and PBPs.1–3 However, several complex challenges remain: (A) the twofold to fourfold variation in top and bottom quartiles for major morbidities like stages 3–4 retinopathy of prematurity and chronic lung disease persists despite ready availability of CQI tools for all,17 (B) NICUs endorse PBPs but do not consistently document compliance with such, making it difficult to ascertain cause and effect,18 (C) some important morbidities resist traditional CQI methodology (chronic lung disease and necrotising enterocolitis), suggesting crucial biological, cognitive and/or cultural limitations are yet to be elucidated1 and (D) value (health benefit divided by cost) is infrequently reported in CQI publications, adding to the perplexity of astute resource utilisation in this era of increasing financial constraints.3 17 18
Comprehensive reviews provide surprisingly limited support for the overall effectiveness of CQI collaboratives.19 20 Thus, we attempted to enhance study of CQI tools and context and culture, the behavioural and cognitive aspects of interprofessional NICU care that likely influence outcomes. Decisions to adopt particular interventions involve human factors and value judgements, simply resorting to hierarchies of evidence and top-down implementation seldom sustains improvement.7 21–23 Our long-time collaboration with other NICUs in the VON suggests a moral component to CQI,3 18 24 25 because we are all trying to foster NICU cultures that: (A) establish the most healthful PBPs for premature infants, (B) implement these methods reliably, (C) monitor carefully for unintended consequences and (D) incorporate family preferences via shared decision making. Being mindful of societal priorities and resource restraints, neonatology needs refined CQI strategies that scrutinise low value care and encourage high value care, perhaps the principal challenge of 21st-century medicine.26–28
Limitations of our study are several: (1) our risk-adjusted composite morbidity score is methodologically robust, but it is precarious to link this objective CQI proficiency metric to a culture and environment survey that was dependent on recall and subjective assessments of social and cognitive measures; (2) our goal was to capture consensus judgement over a meaningful time period of CQI activities, ‘culture’ generally does not change rapidly, so a multidisciplinary group completed the survey together to capture essence. Many of the questions involved summary descriptors over a 15-year time span not necessarily identifying individual provider style nor whether variables were changing significantly over the most recent 3 years; (3) our survey contained ‘human factor’ judgements inherently difficult to quantitate, some imprecision surely accompanies culture and cognitive appraisals; (4) our designation of what defined significant differences between a survey answer in groups A, B and C was an arbitrary but reasonable gestalt judgement of subjective variables typically used in the social sciences29; and (5) the relationship between survey answers and the Benefit Metric score is correlation not necessarily causation.
Conclusion
High proficiency NICUs, as ranked by our risk-adjusted, composite VLBW infant morbidity score, share important cultural, social and cognitive characteristics heretofore not appreciated by traditional CQI analysis including more effective team work, higher morale, greater problem-solving expectations of providers, enhanced learning opportunities, knowledge of CQI fundamentals and favourable staffing models. Broader investigations of whole NICU cultures of excellence that encompass human factors and context, in addition to the refinement of PBPs, might help us understand and promote the triple aim of better health, satisfaction and value.
References
Footnotes
Contributors JWK and LW had full access to the data and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: all authors. Acquisition, analysis and/or interpretation of the data: JWK and LW. Drafting of the manuscript: JWK. Critical revision of the manuscript for important intellectual content: all authors. Statistical analysis: JWK and LW. Obtained funding: JWK. Administrative, technical or material support: JWK. Study supervision: JWK.
Funding This investigation was supported by the Providence Health and Services Foundation and Northwest Newborn Specialists PC.
Disclaimer The Vermont Oxford Network played no role in the design, analysis, interpretation, or reporting of this research. The views, conclusions, and opinions expressed are solely those of the authors and do not represent the Vermont Oxford Network.
Competing interests None declared.
Ethics approval The institutional review board at each participating hospital approved this investigation.
Provenance and peer review Not commissioned; externally peer reviewed.
Collaborators This investigation was conducted in collaboration with multidisciplinary NICU teams from the following institutions: Asante Rogue Regional Medical Center, Medford, OR, Baylor All Saints Medical Center, Fort Worth, TX, Capital Health Medical Center, Pennington, NJ, Children’s Hospital of San Antonio, San Antonio, TX, Community Medical Center, Missoula, MT, East Tennessee Children’s Hospital, Knoxville, TN, Helen DeVos Children’s Hospital, Grand Rapids, MI, Kadlec Regional Medical Center, Richland, WA, Kapiolani Medical Center, Honolulu, HI, Legacy Randall Children’s Hospital, Portland, OR, Legacy Salmon Creek Hospital, Vancouver, WA, Lucille Packard Children’s Hospital Stanford, Palo Alto, CA, Maricopa Integrated Health System, Phoenix, AZ, Meriter Hospital, Madison, WI, Methodist Children’s Hospital of San Antonio, San Antonio, TX, Miami Valley Hospital, Dayton, OH, Minneapolis Children’s Hospital, Minneapolis, MN, North Central Baptist Hospital, San Antonio, TX, Northside Hospital, Atlanta, GA, Oregon Health Sciences University, Portland, OR, PeaceHealth Sacred Heart Hospital, Eugene, OR, Providence Alaska Medical Center, Anchorage, AK, Providence Little Company of Mary Medical Center, Torrance, CA, Providence Sacred Heart Medical Center and Children’s Hospital, Spokane, WA, Providence St. Joseph Medical Center, Burbank, CA, Providence St. Vincent Medical Center, Portland, OR, Providence Tarzana Medical Center, Los Angeles, CA, St.Barnabas Hospital, Livingston, NJ, St. Charles Medical Center, Bend, OR, St. Luke’s Hospital, Boise, ID, St. Paul Children’s Hospital, St. Paul, MN, St. Vincent Healthcare, Billings, MT, Salem Hospital, Salem, OR, Sharp Mary Birch Hospital for Women and Newborns, San Diego, CA, Swedish Medical Center, Seattle, WA, Sunnybrook Health Sciences Center, Toronto, CA, University of Michigan Hospital and Health Center, Ann Arbor, MI, University of Washington Medical Center, Seattle WA, Winnie Palmer Hospital for Women and Babies, Orlando, FL.