AIM To determine whether those most easily reviewed in a population prevalence study differ from those followed up only with difficulty.
METHODS All babies born before 32 weeks of gestation in the North of England in 1983, 1990, and 1991 were traced, and all the survivors assessed at two years by one of two independent clinicians.
RESULTS 818 of the 1138 live born babies survived to discharge. There was some non-significant, excess disability in the 5% of long term survivors who were difficult to trace because of social mobility, but eight times as much severe disability in the 1% (9/796) in care and in the 5% (38/796) whose parents initially failed to keep a series of home or hospital appointments for interview, and five times as much emergent disability in the 2.7% (22/818) who died after discharge but before their second birthday. Had the babies who were seen without difficulty been considered representative of all the babies surviving to discharge, the reported disability rate would have been two thirds what it really was (6.9% instead of 11.0%).
CONCLUSIONS Population prevalence studies that ignore those who seem reluctant to cooperate risk serious ascertainment bias.
Children seen for review without difficulty in any study of community prevalence are seldom representative of those who are not assessed
Failure to review those who prove difficult to trace introduces less bias than failure to review those who prove difficult to review once traced
Studies where it is not possible to see some children for assessment might usefully include a calculation of what the total prevalence would be if there was a fivefold difference in the proportion with the condition in question among the children who were not seen
Parental support for any such study can nearly always be obtained if an effort is made to make the family feel, from the outset, that they are partners in a collaborative exercise of genuine worth
- prevalence studies
Statistics from Altmetric.com
The subsequent progress of infants who may have been exposed to perinatal hazard is an important component of clinical audit. Much has been written about the need to improve the design, conduct, and analysis of such studies,1 and it is generally accepted that loss to follow up could bias the conclusions reached. There is, however, very little evidence to show that bias does actually occur, and too little information to assess the potential magnitude of this problem.
A collaborative population based study was undertaken of all the babies of less than 32 weeks gestation who survived to discharge from the 20 maternity units serving women normally resident in the North of England after delivery in 1983.2 A second, similar study was mounted in 1990 and 1991.3 The North is a well defined, medically self sufficient, 15 415 km2 region, with five recognised neonatal referral units serving urban Tyne- and Teesside, and a large rural hinterland with a total population of 3 100 000 and about 40 000 births a year.4 It contains one of the most densely, and most sparsely, populated counties; economic standards are equally variable, but average earnings are among the lowest in the country. No other region in England has a higher unemployment rate.
Multiple sources of ascertainment were used to identify all eligible babies, including each hospital’s delivery suite and neonatal unit registers, all national birth and death registration returns, and the Regional Health Authority’s community child health register. Gestation was then calculated by survey staff from the mother’s menstrual history, as recorded in the obstetric notes unless this was uncertain, or unless it differed from the estimate obtained from at least one reliable ultrasound scan undertaken before 20 weeks of gestation by at least 14 days. Reliable ultrasound information on which to cross validate the menstrual information was available for 97% of the pregnancies, and in all but a few this was a maternity unit measurement of biparietal diameter at between 14 and 18 weeks.
Every child was traced and all the survivors seen for detailed assessment two years after birth by an experienced and independent paediatrician, who was unaware at that time of the child’s neonatal history. Children were seen at hospital, at home, or in a clinic as seemed most appropriate anywhere in the UK. Prior guidance from the family doctor and of the local paediatrician responsible for initial post discharge follow up was obtained in formulating the initial approach to each family. All the parents of children born in 1990–1 had been aware from the outset that their children were involved in a collaborative, region wide, nurse coordinated research study.
A quantitative measure of development was obtained at two years using the Griffiths Mental and Developmental scales,5 and the child was classified as having a severe disability using criteria laid down in a recent national consensus report6 (with one minor amendment to allow for the fact that it was impossible to judge how completely aids were correcting any visual or auditory impairment7). All the children born in 1983 were also seen a second time when 6 years old so that intelligence could be measured using the British Ability Scales8 and any sensorimotor disability reassessed. Children were deemed to have an abnormal developmental or intelligence quotient when this was more than three standard deviations below the norm for all the preterm babies in the study without serious sensorimotor disability.
Separate information on progress was also collected at 2 years from each child’s personal health visitor (an experienced community based nurse with a free, statutory, area wide health education remit). Permission to do this was obtained from the parents at the start of the study.
Children were classified as hard to see if their parents failed to keep more than two of the planned review appointments at 2 years, had left the region, moved house more than twice within the region, changed both the baby’s names, or had the baby fostered or adopted before the age of 2.
In each of these two cohort studies it proved difficult to see 11% of the survivors for detailed assessment at the age of 2. Social mobility was the main reason why many children were only seen with difficulty, but parental behaviour (failure to keep previously agreed appointments without explanation) suggested that some parents were reluctant to have their children examined, and children who had been adopted, or who were in care, were included in this group because social service agencies were very reluctant to approve access to these children (even when survey staff were aware of their whereabouts). A distinction has been made between children who were difficult to trace and children who were difficult to see when traced, in the analysis that follows, after this subclassification had been independently confirmed by a clinician unaware of the child’s developmental status at follow up.
CHILDREN BORN IN 1983
Two hundred and thirty of the 356 babies born alive at 23 to 31 weeks of gestation in 1983 were still alive two years later. Every child was seen for assessment at that time, but 26 (11%) were only seen with difficulty. In 14 cases multiple changes of address (with or without changes of name) made tracing difficult, while in a further 12 parental behaviour suggested a reluctance to let the child be examined (table 1). Nine of these 26 children (34.6%) showed signs of severe disability,6 but only nine (4.4%) of the other 204 children (p<0.01).9
All these children were seen for a second time by the same clinician (UW) without further difficulty when they were 6 years old so that their intellectual as well as their sensorimotor abilities could be assessed. One very disabled survivor (with myotonic dystrophy) had died; the findings in the other 229 long term survivors are shown in table 2. There was five times as much sensorimotor or cognitive disability among the survivors initially only seen with difficulty as there was among the 89% who had been traced and seen at two years without substantial effort.
CHILDREN BORN IN 1990–1
Five hundred and sixty six of the 782 babies born alive at 23 to 31 weeks of gestation in 1990 and 1991 were still alive two years later. Every child was seen for assessment at that time, but 61 (11%) were only seen with difficulty. All but one were seen by the same clinician (WT). Sixty four of the survivors were disabled, and there was evidence that seven of the 16 who died after their discharge home from neonatal care would have been disabled had they survived. There was five times as much disability among the 61 children only seen with difficulty as there was among the other survivors (p < 0.001). Disability was particularly common in the children of parents who seemed reluctant to have their child examined (table 1). Had the babies who were seen without difficulty (87% of those discharged home alive) been considered representative of all the babies surviving to discharge, the disability rate would have been taken to be two thirds of what it actually was.
A quantitative assessment was also made of the developmental progress of each individual survivor at 2 years; 40% of the children with a developmental quotient that was more than 2 standard deviations below normal proved difficult to see for review (fig 1). Similar findings were encountered among the children born in 1983.9
It is instructive to see what would have been achieved had other methods of ascertainment been used. Although collaborative follow up was planned from the outset, as part of the 1990–1 study, 31% (20/64) of the severely disabled children had stopped attending the clinic run by the consultant paediatrician responsible for the child’s initial follow up after discharge two years later, and little was known locally about the current clinical status of these children. Even fewer of the healthy children were still being seen.
An independent progress report was requested, as part of the present study, from the local health visitor shortly before each child’s second birthday.10 Reports were eventually received on 86.9% (436/502) of the survivors who were later judged, on medical review, to be free from disability, and on 89.1% (57/64) of those with severe disability. Although a small number of questions had been left unanswered in some of these returns, the reports were (with two exceptions) good enough to determine whether there was severe disability. Reports were obtained on 80% (28/35) of the children whose families seemed reluctant to agree to medical review. Even with this dual approach, the outcome for six of the disabled long term survivors would have still gone undocumented, had other means not been found to ascertain outcome at 2 years.
A preliminary letter commenting on the bias that is liable to develop when some neonatal survivors are lost to follow up was published in 1989 when the result of the initial two year assessment of the children born in 1983 first became available.9 A second detailed review, undertaken when these children were 6, has confirmed that disability was particularly common in those babies who initially proved hard to see for assessment, while a second larger study of all the children born in 1990–1 has now provided independent confirmation of these findings.
Two other published studies have encountered a similar phenomenon, but both had to use indirect methods to deduce the probable disability rate among the children not returning for follow up, precisely because these children were never traced or seen again.11 12 Another study, 25 years ago, showed that those who were only evaluated as a result of a home visit were twice as likely to be disabled as those who kept a hospital follow up appointment.13 Other research reports have shown that young and single mothers are less likely to bring their children back for review, especially when this is being offered from a centre without neonatal intensive care facilities, and that it is the children of these mothers who are most likely to have disability at follow up.14 Such studies have failed to find any other important socioeconomic differences between the families who returned for follow up and those who did not.15 16Twenty years ago, research using multiple methods of ascertainment to study the epidemiology of behavioural deviance in later childhood showed that selective loss of information could lead to a biased study outcome.17
Adults behave no differently: the patients who were lost to follow up after total hip replacement in one recent study had had significantly more pain and been significantly less satisfied with the result of the operation than those who continued to attend.18 The women who were most difficult to trace after amniocentesis or chorion villus sampling in another recent population based follow up study were those with the worst pregnancy outcome.19
Family mobility can make subsequent assessment difficult, and young disadvantaged families are often particularly mobile, but this proved to be a cause of only minor potential ascertainment bias in the present studies. Movement out of the region or country increased with time, but was only an issue for 1.5% (12/796) of the surviving children at two years. Such problems were minimised in the second cohort by securing the prospective support of relatives, particularly in those families where the child was not always registered for care with a general practitioner.
When contact is lost with a family it may not even be possible to confirm that the child is still alive. Five per cent (57/1138) of the live born babies in the two studies reported here died when they were more than 27 days old, and 22 of these died after discharge from hospital but before they were 2 years old. Over a third of the latter would have been at high risk of severe disability had they survived (table 1).
Much more disability was found where the survey team found it difficult to obtain an assessment after contact was established with the family, and this proved as important a problem in the second study as the first. Multiple factors may have been operating here. In some cases there was clear resentment of all social (authoritarian) intrusion, and in others frank hostility towards the medical profession, sometimes generated, at least in part, by the nature of the child’s disability, the way it had first been revealed to the family, or subsequently handled. In other cases an unsupported mother seemed so stressed by the burden of caring for a very disabled child that she was reluctant to tolerate yet another visit or interview when she could see no immediate benefit to the child. Some families seemed very disorganised, and eight were seen only after more than five failed appointments. In three families, where nobody could be found at home at the prearranged time on three separate occasions, contact was eventually achieved by arriving for the next arranged home assessment one hour early. Denial of developmental delay was not encountered, and several families expressed gratitude that their child had finally had a detailed and comprehensive assessment, and that some of their concerns and questions had been openly addressed. Once trust had been won in this way, subsequent reassessment at six years was never a problem. Others have shown, over many years, the importance of ensuring that families feel that they are genuine partners in a shared endeavour.20
Resistance from social workers has been an almost insuperable hurdle to the follow up of fostered and adopted children in almost every previous published study. The importance of breaching this barrier was shown when we established that five of the 10 children in care at the age of 2 were severely disabled.
Health visitors’ reports can provide a reliable way of obtaining a relatively unbiased estimate of the proportion of survivors with severe disability,10 but they cannot provide detailed information on the origin and nature of any disability, or its exact extent. They can, however, be used to identify a subset of children for further, more detailed, evaluation. Nevertheless, such reports can only be requested if the child’s whereabouts are known, and changes of name and excessive social mobility (irrespective of whether there was also reluctance to accept assessment), made that difficult in 47 (5.9%) of these families, 10 of whom had a disabled child.
We conclude that the people seen without difficulty in any community based prevalence study may be totally unrepresentative of those who are not seen. Archie Cochrane sensed this in the pioneering studies he did in South Wales in the 1950s,20 but the magnitude of the possible ascertainment bias has, until now, been very poorly defined. It must be a matter for concern that, in over half the studies included in one recent international overview of the long term prognosis for the very low birthweight baby,21 more than 10% of all study children had been “lost to follow-up.” Studies that fail to win the voluntary support of all concerned risk coming to a misleading conclusion.
We acknowledge the contribution of the senior neonatal nurses whose commitment to the Northern Neonatal Nursing Initiative7 secured the support of the families to the prospective study of all the children born in 1990 and 1991, and of the health visitors and of our paediatric colleagues for their assessments of these children. We are grateful to Dr Sam Richmond for his encouragement, and for his many contributions to this study.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.