Audit is one of the key components of any clinical governance strategy1 and is used throughout the NHS for ensuring maintenance of standards. The traditional model of medical audit is often unsatisfactory for junior staff in 6-month posts as there is insufficient time to see changes implemented, a re-audit carried out and “closure of the audit loop”. They thus get little ownership of, or satisfaction from, their audit, which has been shown to be important2 if we are to avoid the danger of audit becoming a chore instead of a useful tool for improving practice.

We describe the use of “random safety audits” which overcome many of the negative aspects of the traditional audit. They are adapted from industry3 where they have been very effective process audits: checklists are compiled for each of a number of pre-identified error-prone activities. To perform the audit, a checklist is chosen at random and the auditors then go to that point in the process to directly engage staff in an immediate review of the work in progress relative to the checklist endpoints. The idea in industry is that this will identify error and error-prone situations and increase safety awareness of workers “on the shop floor”.

This method of auditing is attractive in clinical practice for many reasons. First, the audits are in “real time”, assessing actual practice. Second, the immediacy of feedback allows for immediate awareness and change in practice where necessary. A formal action plan can be made and circulated and the standard can be re-audited within a short timeframe — typically weeks. Ursprung and colleagues have shown that it is feasible to adapt this process for neonatal intensive care unit (NICU) practice.4

Our aim was to adapt random safety audits for use in our own neonatal unit and to analyse the effects on practice, focusing particularly on issues related to nosocomial infection.

METHOD

We chose 15 audit standards: 11 that were part of our infection control strategy and four routine care standards that were considered suitable for this method of audit (table 1).

The standards were chosen by the medical staff in consultation with the nursing team to ensure that the audit standards were correct reflections of what was recognised to be best practice.

Southmead is a Level 3 NICU with seven intensive care cots, five high-dependency and 14 special care cots. The audits were performed by medical and nursing staff working within the unit.

In order to ease the process and reduce the time taken to perform and report the audits, we designed clear, simple data collection tables and found that even complicated guidelines could be translated into a simple-to-use table. Figure 1 shows the lipid prescription guideline and the table designed for the audit, as an example.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Transformation of a complicated lipid prescription guideline (inset) into an easy-to-use audit table. DIC, disseminated intravascular coagulation; TPN, total parenteral nutrition; VLBW, very low birth weight.

Two standards were audited each week during the grand round when the greatest number of staff would be present. Since this was always the same day of the week, staff were aware when audits would be taking place but not which topics were being performed. Standards to be audited were chosen in a non-random manner from the 15 topics. Each was audited once before repeating the cycle of 15 audits. After the first cycle of 15 audits, staff were aware of the topics to be audited, but not which audits were being carried out on a particular day.

If there was non-compliance with standards this was discussed at the time with the relevant staff member. Feedback was given in a non-judgemental way and designed to encourage compliance with the standard rather than resentment. All staff were treated in the same way. We also tried to engage the staff in discussing the reasons for non-compliance to encourage greater ownership of the process.

As well as the immediate feedback, at the end of the ward round, we summarised the results and disseminated them by personally informing the staff, and by use of a template poster designed for displaying results on a designated audit notice board. For each topic, we had a laminated eye-catching photograph which could be stuck onto the poster to attract attention of those walking past. Results were also summarised in the staff communication book by the end of the day to try and ensure wide dissemination of information.

Any changes to policy were discussed at the monthly neonatal unit audit meeting.

The audits were approved by the North Bristol Trust Audit Department.

RESULTS

In 6 months we completed three cycles of auditing 15 standards making a total of 45 audits. Unfortunately, one set of results for the leaning topic were mislaid and therefore we report complete results for 14 topics.

The results of the audits of infection control standards showed compliance in the first cycle ranging from 20% to 100% (see table 2). These figures improved or remained the same in all but one of the standards (fig 2). The overall improvement in performance expressed as a median and range is shown in fig 3.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2 The results of random safety audits on infection control standards.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3 Median and range of results for percentage compliance with infection control standards.

The results of the audits of routine care standards were more varied and are presented in table 3.

DISCUSSION

We have shown that random safety audits can improve compliance with unit guidelines and protocols. We had greatest success with our infection control standards. The only infection control standard that appeared to show a marked fall in compliance was lipid prescribing. However the fall from 100% compliance in the second audit to 66% compliance in the third audit represented small numbers of 3/3 and 2/3. At the time of the audits, our unit particularly focused on infection control, and the introduction of random safety audits was part of this campaign. We had chosen to focus on infection control as a result of benchmarking in the Vermont Oxford Network and the idea for random safety audits came from the VON 5th Annual Quality Congress in 2004. This pre-existing emphasis may partly explain the greater improvement in compliance as compared with the routine care standards. In the context of infection control, Kilbride et al5 have described the importance of a “habit for change” and a “habit for systems thinking”, that is, considering the small processes that contribute to the overall care. We already had a habit for change as part of our infection control campaign and the random safety audits were part of the “habit for systems thinking”.

Compliance with routine care standards did not show the sustained improvement of the infection control standards. Although the actual numbers in the results seemed more disappointing, the process still had many positive outcomes. Guidelines were discussed and plans made to clarify and formalise changes, such as in oxygen saturation limits and first-day checks. For example, the reason for incomplete first-day checks was often that the infant was too unstable for complete examination on admission. Therefore we planned to review first-day checks on a weekly basis. Confusion in the oxygen saturation guideline was highlighted and this was in the process of review. These discussions led to greater understanding of our practice and what our guidelines should be.

Another possible reason for poor compliance in the routine standards was that they were more focused on junior medical staff work — compared to the infection control standards that were multidisciplinary — and there was a change-over of junior staff between the first and second audits. We feel that the quick turn-around and re-auditing by random safety audits had an important role to play here in highlighting areas for improvement and in stressing our commitment to maintenance of standards especially for those who are new to the unit and in short-term jobs.

There are possible limitations to using random safety audits. One is that it requires dedicated staff to maintain the momentum. However, our experience is that these audits are well received and considered to be a very effective form of audit and therefore, we hope that they would be embraced by other units or departments. Although we concentrated on neonatal topics, Ursprung et al4 showed that they could be adapted for more general measures including the processes of ordering and obtaining results of investigations, communication problems and equipment problems. A second concern may be that they are time-consuming but we found that if well prepared, the process of doing the audits was very quick. Dissemination of results also took little time once the results poster was designed and photographs of the audits were obtained. The most time-consuming area will be collating the results to show trends over time but this is a worthwhile investment for improving patient safety.

Another potential limitation is that staff may have looked to see what was being audited and then tried to improve their performance prior to being audited. To minimise this when performing audits of the environment, we tried to complete audits of all of the relevant areas prior to giving feedback. Although we may not have been able to completely rule out any quick alterations prior to our audit, at the very least, the process of performing the audit reminded the staff of what the guidelines should have been and produced a positive change in behaviour.

Perhaps the greatest concern is that random safety audits may be punitive. This should not be the case if they are done in the correct spirit of working together to improve the care of patients. The NHS aims to foster a positive environment to enhance working practices rather than one where blame is apportioned.6 We did not experience adverse reactions from the staff. Indeed our experience was that the medical and nursing staff were very supportive of this style of audit and that they generated informal conversation both before and after the audits. Concerns regarding the audits were generally regarding the clarity of guidelines and so the audits provided a forum for re-writing of guidelines where appropriate. Because of the immediate feedback and the demonstration of improvement in adherence to guidelines, medical staff in particular found this form of audit to be logical and appealing.

We did not directly involve families in these audits but they were aware of the processes, particularly regarding hand washing and jewellery, and reacted positively to the overall aim of improving quality of care.

In summary the benefits to our unit were: increased awareness of infection control measures, improvement in infection control practices, continued emphasis on infection control and general good clinical practice, clarification of guidelines and improved team working. We would strongly recommend this form of audit to other units.