Article Text
PDF
Abstract
AIM To determine whether existing information and surveillance systems can be used to provide follow up data on groups of infants at increased risk of disability—for example, the survivors of neonatal intensive care.
METHODS A survey was made of maternity, neonatal, and community child health information systems and surveillance programmes in the Trent Regional Health Authority. Children known to have received neonatal intensive care in Trent between 1 August 1992 and 31 July 1993, and a random sample of normal children in two health districts (data quality check) were included. A data linkage study was made to determine whether follow up information about a random sample of infants, known to be at increased risk of poor outcome, could be identified on community child health databases. Two widely accepted datasets (birth and 2 years) were used as standards for this exercise. The quality of data was audited.
RESULTS All clinical items of the birth minimum dataset were routinely recorded by at least one agency in each health district in Trent. Of the descriptive items, only the mother’s age on leaving full time education was not collected. At 2 years, all clinical items were collected as part of the routine surveillance programme, but data were recorded using a system which severely limited interpretation. Data quality, in terms of the number of errors introduced at data entry, was very good with only 1.1% of the check items (4/368) incorrectly recorded. Only two districts had organised electronic transfer of data between maternity, neonatal, and community child health systems. The mother’s NHS number, although available, was not routinely recorded by any system. The NHS number of the infant was routinely collected by six out of 12 community paediatric services. Data linkage was attempted in six districts with appropriate community child health databases. Just over 70% of the intensive care sample was successfully linked with follow up information on child health systems.
CONCLUSIONS The existing programmes for routine child surveillance could provide outcome data for high risk groups of infants, such as the survivors of neonatal intensive care. However, the present coding system used for data entry is inadequate. Furthermore, rates of identification, without the use of a unique identifier (NHS number) for each subject, are currently insufficient for monitoring health status in later life.
- outcome data
- information systems
- surveillance programmes
- intensive care
Statistics from Altmetric.com
The specialty of neonatal intensive care has developed in just over 25 years. During this period there has been a rapid growth in provision, accompanied by a dramatic improvement in the survival of premature infants.1 The service is broadly seen as a success. This situation has created considerable difficulties for the Health Service as it has come under increased pressure to fund provision throughout this period. A series of government bodies reviewed the service between 1970 and 1991, considering mainly the issue of the gap between supply and demand.2-10
The NHS reforms of 1991 introduced many changes, in particular value for money and cost effective health care became much more important. For neonatal intensive care, this change of emphasis meant that those responsible for purchasing health care wished to see not simply figures for short term survival but handicap rates among survivors in later childhood. As a consequence, official reports since 1991 have all emphasised the need for neonatal units and or health districts to produce late morbidity data. The Audit Commission (1993)11in particular highlighted the lack of comprehensive statistics of this type. In general, the Audit Commission found published reports related either to a single unit12 (and hence were difficult to assess because of potential referral bias), or to a geographical population for a short period as part of a special study.13 Investigations of this latter type provide a realistic insight into the effects of neonatal care, but without continuing data over time and more comprehensive geographical coverage, that insight is necessarily limited.
Community child health departments have existed throughout the country for many years with a particular remit in the areas of preventive child health and surveillance. The National Child Health Computing Committee was established in 1977 to try and oversee the introduction of computer technology in relation to community child health. However, this group did not have responsibility for the development of a comprehensive national system, each district being responsible for its own local service.
It was against this background that the National Perinatal Epidemiology Unit and Oxford Regional Health Authority organised a multidisciplinary workshop in 199314 to address the issue of how ongoing data might be obtained about the health status of children who had received neonatal intensive care. At this and follow up meetings, a core dataset relating to birth and including information about the child’s health status at the age of 2 years was determined. However, opinion was divided over two related issues. The first was whether the minimum datasets should be used for monitoring specific groups of babies (for example, all babies born less than or equal to 32 weeks of gestation) or whether it should be part of pre-school child health screening programmes for the population as a whole. There were also different views about whether routine systems could ever supply data of sufficient quality to allow a particular group to be reliably monitored or whether special studies would always be required. One major difficulty was the uncertainty about what could be obtained from the existing surveillance schemes.
It was in response to these circumstances that the Trent Neonatal Follow up project was developed with the intention of addressing three specific questions:
- 1
- Were routine systems able to provide the information included in the agreed core datasets for survivors of neonatal intensive care?
- 2
- Using existing mechanisms, was it possible to track the survivors of neonatal intensive care through the routine child health surveillance systems?
- 3
- What was the quality of the data on the child health systems?
Methods
The investigation was carried out by a research health visitor and a computer scientist. The project was based in the Trent Health Region which has a population of approximately 4.6 million and a birth rate of about 60 000 a year. At the time of the study there were 16 hospitals with both delivery and neonatal facilities and 12 community paediatric services. The community paediatric services each covered a geographical patch identical to the area of responsibility of the local purchasing authority/district health authority. Work was carried out between March 1994 and March 1996.
SURVEY OF INFORMATION AVAILABLE ON ROUTINE SYSTEMS
Relevant professional and administrative staff were contacted in each maternity, neonatal, and community child health service. For the purposes of this study, the two datasets produced following the National Perinatal Epidemiology Unit workshop were taken to represent a gold standard, both in terms of timing and content. Using a semi-structured interview the following were determined at each contact:
- 1
- Which items contained in the minimum datasets were collected routinely?
- 2
- Were relevant data items stored on computer?
- 3
- Where information was stored on computer, was this transferred automatically to any other part of the service?
- 4
- Were the data stored in such a way as to allow analysis e.g. trends over time, by sub groups of children?
- 5
- Who was responsible for collecting the follow up data?
- 6
- What form of “identifiers” did the systems use?
IDENTIFICATION OF A SAMPLE OF INTENSIVE CARE SURVIVORS ON ROUTINE CHILD HEALTH SYSTEMS
Initial enquiries determined the extent to which NHS number, or any other “identifier” was routinely used to link the records of a single child within the various elements of the maternity, neonatal, and community child health services. However, identifiers were rarely available and we therefore tested the feasibility of tracking the survivors of neonatal intensive care through community databases, using an identifier derived from combinations of date of birth, birthweight, and gestation.
A register of all babies receiving neonatal intensive care has existed in Trent since 1990 (Trent Neonatal Survey). From this database it was possible to take a random sample of 50 premature infants born between the beginning of August 1992 and the end of July 1993 for six health districts of residence. The six districts were chosen because they had surveillance data stored on computer with software that permitted specific enquiries to be undertaken. In two of the six districts the sample was smaller because fewer than 50 preterm babies had received neonatal intensive care. Each of the six community services provided a simple report containing basic information including name, date of birth, birthweight, and gestation for all children in the relevant age band.
Attempts to link the cases identified by the neonatal survey with the surveillance data were made using two separate approaches. The first used date of birth, birthweight, and gestation as a composite identifier and the second date of birth and birth weight only. For each of these derived identifiers, two sets of criteria were used in trying to achieve a match. Initial attempts for each child used the data in its original form in each dataset (exact match). Where no match was obtained, further tests were carried out in an attempt to establish links that had failed because of erroneous birthweight or gestation recordings. A range of 0.1 kg was allowed for birthweights, and 2 weeks for gestation values (range match). Clerical scrutiny was used in an attempt to resolve any unlinked (or ambiguously linked) records.
The results for each child in the sample were then classified manually, with the aid of other information carried by the two datasets, into one of five categories: unique match (definite) if linked records also had the same first or last name; unique match (probable) if there were no common names but equally no apparent signs of belonging to different babies; wrong match if linked records showed clear signs that they belonged to different babies—for example, names indicated different gender; non-unique match if a neonatal record matched more than one child health record; and failed match if a neonatal record did not match with any child health record.
DATA QUALITY CHECK
An audit of data preparation and data entry was carried out in two “volunteer” districts. One hundred children who had recently undergone their 18 to 24 month check were selected at random from lists supplied by two community child health services with computerised surveillance records. Each child’s GP was sent an introductory letter and, provided he or she felt it appropriate, the child’s family was approached. Those that consented to take part were then seen at home. During the interview the research health visitor was able to validate information about the check held by the community child health service in respect of eight specific items included in the minimum dataset. Where necessary, the parent held record and review of the child were also used.
Results
Table 1 shows the information systems that were identified holding data relevant to the minimum datasets. Within only two health districts did information pass electronically between maternity, neonatal, and community systems. In five of the remaining ten districts the community child health service was able to access demographic (but not clinical) data from the local hospital.
Information systems identified
Table 2 summarises those items of the “birth” minimum dataset which were routinely collected and stored on a database. Some items were collected and stored four separate times within a single locality. Only one of the items, “Mother’s age when last in full-time education,” was not present on any system. Mothers’ support status was normally entered in the notes at booking but recorded in only five of the nine maternity databases. Mothers’ NHS numbers were not routinely recorded on any system. Six community services recorded the infant’s NHS number when it became available. This was achieved with the help of the local Registrar for births and deaths who had an arrangement to notify the relevant community service of the number directly.
Birth minimum dataset
COMMUNITY DATABASES
Each district based community child health service had a database containing demographic details of their population and capable of underpinning the local immunisation programme. Ten of the 12 had systems capable of recording surveillance data. Although each community child health service continued to operate basic child health surveillance programmes, only eight out of 10 with the capability entered this on computer. None of these provided regular feedback to those who carried out the checks but six used the data for ad hoc audits (especially in relation to hearing loss) and also to supply data for the special educational needs register. Uptake rates in the 12 districts for checks at 18 to 24 months were estimated to range from 79% to 97%.
Every community child health service aimed to provide an examination of all children between 18 months and 2 years. In 10 of the 12 districts health visitors were responsible for this check, with the remainder relying on either GPs or clinical medical officers. Every aspect of development covered by the minimum dataset for 2 years was assessed by each of the community services. The findings were interpreted and documented in all districts using the following codes: satisfactory; problem; observe; treated; referred; or not examined. Not every district used all of these codes; some relied on a combination of only four.
There was little or no attempt to standardise the interpretation of these terms. For example, a child with little expressive language at 18 months could be classified as “satisfactory,” “problem,” “observe,” or “referred” by different professionals. Nor were records updated. Therefore, a child referred for a further opinion and found to be normal remained with “referred” on his or her record. The districts varied in the way in which they updated information about congenital anomalies on the child health database: some did not update this after entry of birth notification data; others updated as notified.
RESULTS OF DATA LINKAGE EXERCISE
Fig 1 shows the outcome of the data linkage exercise, using both exact and range matching techniques. The results are based on data received from six health districts (266 children) with appropriate systems.
Results of data linkage exercise averaged for all districts.
Tests using the identifier date of birth, birthweight, and gestation produced “error free” linking, in that no wrong or non-unique matches were made. However, the overall match rate for all six districts of 53.9% (rising to 71.3% using “range matching”) was disappointing.
The greatest match rate was achieved using “range” matching with the identifier date of birth, and birthweight, where outcome data were traced for an average 80% of children. This method did, however, lead to a small number of wrong (2%) and non-unique (2.3%) matches.
RESULTS OF DATA QUALITY
A random sample of 100 children in two districts were initially selected. Responses are shown in table 3.
Responses of random sample of 100 children
Forty six children were seen at home and the findings compared with information about their earlier examination held on the community child health database. From a total of 368 individual recordings of items covered by the Oxford minimum dataset, only four (1.1%) were incorrectly recorded.
Discussion
Among neonatologists there has always been a general acceptance of the importance of outcome data in assessing the impact of specialist neonatal care. The health service reforms have given a new prominence to this goal. The investigations presented here indicate that virtually all of the relevant information is being collected: the maternal and demographic data sometimes several times over. However, several problems prevent the various data items about individual babies being collated.
Within hospitals the Patient Administration Systems (PAS) are based on Korner datasets and have little clinical information of relevance to neonatal care. Specialist maternity and neonatal systems are potentially much more useful because they have been set up by those with the relevant clinical interests. However, only one Health Authority in collaboration with local Trusts had planned an information system which allowed information to flow from one part of the service to another electronically.
The focus of community child health computing departments has traditionally been administrative. Much of the success of the national immunisation programme is due to the efficiency of these organisations in keeping up to date demographic details about the child population. Clearly, surveillance data are more difficult to deal with as the information often requires some form of interpretation. The present approach using codes based on the terms satisfactory, problem, observe, etc., is inadequate. As a result the present data from surveillance checks are uninterpretable outside the service in which they originated, and even then their application seems to be very limited. The computer software is, in some cases, inflexible, giving little opportunity for results to be fed back to individual contributors. Children moving in or out of the locality cause further problems. The exact timing and content of the various assessments lacks standardisation and hence it may not be possible to update computer surveillance records in the new area. These difficulties probably explain why one third of Trent districts collect paper records of assessment but do not enter the data on computer. Others pay for data entry but either use the information very infrequently or not at all.
It is important to distinguish between detailed specialist follow up (developmental examination) designed predominantly for the benefit of the individual patient15 and “surveillance” (where the concept is of assessment of the whole population).16 The former is offered by seven units in Trent, although the exact content varies. As infants are included using a unit (and not geographical) base the data are of little help in assessing the broader impact of neonatal care. In some cases, however, the information is made available to the relevant community child health service to aid in the management of the child at home and school.
From this study it is clear that all of the clinical items in the birth minimum dataset are already being collected. Similarly, all of the areas of development identified within the two year minimum dataset are currently being assessed by each of the Trent community child health services between 18 months and 2 years. Take up rates are high. In the community child health services the process of data preparation and entry seemed to take place efficiently and without the introduction of excessive errors. We have no reason to believe that the low response rate of GPs and parents (46%) introduced any element of bias. However, we are involved in further work to assess the accuracy of data both in a larger cohort and in a group of children where a higher rate of abnormalities might be expected, issues not fully covered in the investigation presented here. These various studies have led us to believe that the information necessary to provide outcome data about the survivors of neonatal care is potentially available without significant further expenditure. Before this can take place, however, several important issues remain to be addressed.
Far greater standardisation of data collection methods is required. For hospitals the relevant information is largely collected for clinical interest. A contractual requirement to make the birth minimum dataset available on all infants who receive neonatal intensive care would provide a different focus but should present little difficulty. For community child health services it would be necessary to try and harmonise the timing of the 18 month to two year check (perhaps to two years) and also to standardise the description of data items so that information on impairment of function can easily be extracted. A commitment in each community contract to provide two year outcome data in this standardised format for at least 95% (for example) of the infants in the district who had received neonatal intensive care would give the process the necessary priority status.
One of the single biggest problems to be overcome is that of a unique identifier. The data from Trent indicate that although a large proportion of children can be traced between different databases using items such as birthweight and gestation, the linkage is not 100%. This shortfall is known to be due, at least in part, to children who have moved. However, without a means of identifying the new address the data remain inadequate.17 It is clear that six (50%) of the Trent districts have a system for entering the NHS number on to the community child health database. If purchasers insisted through their contracts that: all districts adopt a similar approach; and that the NHS number be included in the record of all children who move into a district, many of the problems of tracing infants would be resolved. Plans to revise the NHS number will have a limited impact while the present system, of delaying the allocation of NHS number until birth registration, remains.
The above changes should be able to be achieved within existing budgets. Each locality would clearly need someone to coordinate the exchange of data between community and hospital based services and to deal with enquiries from other areas of the country, but such individuals already exist but they struggle without the benefit of a national framework.
With these steps it would be possible to move forward. It would be important for results to be collated for both individual units and purchasing authorities as hospitals will want to be able to look at their performance in total, including inborn infants and those transferred after delivery, while health authorities will have more interest in the results for their population as a whole irrespective of where the care was given. The results of an individual unit would need too much interpretation to be published in a crude league table, but health authorities are now, in general, sufficiently large to permit annual results to be meaningfully expressed—for example, normal survivors under 1500 g birthweight.
None of the above would curtail existing activity, but it should introduce a new sense of purpose for some areas of data collection. At present information of this type is hardly used at all, so the publication of reports and feedback to contributors should improve the quality of surveillance for all children. The approach, of formalising data collection in a contractual framework, need not be limited to the survivors of neonatal intensive care. Babies born from a particular locality, or after a certain obstetric intervention, could also be targeted in this way. Even if the present population screening programmes are abandoned these limited checks of at risks groups are likely to remain worth while. This is true not only because of the need for outcome information but also to inform decisions about the planning of provision for later life.
The National Perinatal Epidemiology Unit/Oxford Regional Health Authority conference identified wide areas of agreement between relevant professional bodies on ways to improve health care for young children. However, no executive body now exists with a role to develop such ideas. This report indicates that many improvements could be made using existing resources. Surely we must find a way to seize this opportunity.
Acknowledgments
We thank the many NHS staff throughout Trent who cooperated with this study. The work was funded by the Trent office of NHS Research and Development.