Send letter to journal:
Re: Test weighing does not work

Dear Editor,

I found this an interesting article. When one feeds babies with a measured amount of milk, weighs each baby before and after the feed, and compares the change in the baby's weight with the amount of milk ingested - there is no connection between the measurements. However, for the population of babies as a whole, the measurements appear to fit. What can the jobbing neonatologist draw from this?

Although apparently counter intuitive, if we think back to our A Level Physics studies, then the conundrum becomes easier to understand. Each measurement we make carries with it an inaccuracy, or error: when we weigh the babies and subtract one measurement from the other, the error in the result is the sum of the errors in each individual measurement.

I would be surprised if weighing instruments in neonatal units were more accurate than, say 0.5%. Indeed, I would suggest that this accuracy would represent top class weighing scales, accurately maintained.

The babies in the study by Savenije & Brand weighed 2045 - 3470 g. I note, incidentally, the use of a terminal 5, or zero, suggesting that the authors feel we can weigh the babies to +/- 5 grams - this would be an accuracy of better than 0.2%. Although this is what other neonatal practitioners also imply when they quote newborn weights, I would suggest that it is optimistic. The median birth weight was 2747 g - if we accept an 0.5% error, this would be +/- approx 15 g.

The measurement the authors are interested in is the difference between the measurement before the feed and that after the feed, so this is known only to an accuracy of +/- 30 g, approximately 30 mls of milk. This inaccuracy is, though, similar to the actual amounts given to the babies (24.5 to 45 ml). Perusing Fig 1, it seems that what I am saying is basically substantiated - the errors largely are 10 mls or less, but a number range up to about 30 mls.

If we make a measurement, and the error in the measurement is of the same order as the size of the measurement itself, then the measurement is not useful. On the other hand, for the whole population of babies, the errors evened out - again, this is not too surprising.

The authors did attempt to cover this point - their standard weights were accurately weighed, it seems, to within 1 g. Despite this, I suggest that my reasoning supplies the answer to why test weighing remains a waste of time.

It should also be noted that, in the 'real world' of the usual newborn nursery, an accuracy in weighing (or any other measurement) of 0.5% accuracy is probably markedly better than can be achieved, making test weighing even more of a waste of time.

Send letter to journal:
Re: Test weighing has previously been shown to be unreliable

Dear Editor,

Send letter to journal:
Re: Test-Weighing for Term and Premature Infants is an Accurate Procedure

Dear Editor,

This letter is in response to “Accuracy and Precision of Test Weighing to Assess Milk Intake in Newborn Infants: 2006;91;F 330-332 (1), in which the investigators conclude that test-weighing is too imprecise for routine clinical use. This conclusion is contrary to a series of very well-controlled studies on test-weighing in term and premature infants. Our concerns with the conclusions of this study are as follows.

First, the investigators’ use of the term “precision” is incorrect. The precision of a measure (also known as its reliability) is the ability of a measurement to be reproduced consistently; precision refers to the repeatability of a measurement. To obtain a measure of precision, the object of interest must be measured more than once under the same circumstances. An assessment of the precision of test-weighing would entail obtaining repeated measurements of the infants’ weights and/or the milk volume before and after feedings such as performed in previous studies of the reliability of test-weighing (2). However, these investigators did not perform repeated infant or milk weights, so they did not measure precision. Thus, their claim that test weights are imprecise is incorrect. Similarly, the investigators’ conclusion that their data demonstrate the accuracy of test-weighing is incorrect. Instead, their data reveal large and clinically important differences between the actual volume consumed and the volume estimated by test-weighing in many of the cases. Thus, the test-weighing procedure, as reported in their investigation, did not yield accurate results.

The large measurement error reported in this study is inconsistent with the results of previously published research and is likely due the lack of research control in this study. The lack of control is evident in several aspects of the procedure, including the administration of infant feedings, infant weighing procedures, evaluation of the volume of milk consumed and the choice of an infant scale. In previous studies on this topic, infants’ clothing and equipment have been standardized and controlled, and the clinicians are well-prepared for their role in the procedures. Previously published research has also demonstrated that the reliability (precision) of infant weighing procedures is significantly affected by the presence of equipment such as IVs and oxygen nasal cannulae (3), so the management of that equipment during the test weighing procedure requires careful control. Additionally, in previous studies, infants were removed from analysis when regurgitation or spitting up resulted in milk being spilled outside the clothing that was a part of the pre- and post-weights. Obviously, when actual milk consumed is not measured as a part of the test-weight, the estimate will be inaccurate. Additionally, the scale used to obtain the test weights was not adequately described. Although the investigators tested the repeatability of weights obtained on the scales using 1500 and 4000 g weights, they did not report whether the scale had the capability to accurately measure small weights – such as oral intakes as small as 1-2 g. The scales used in previous studies were specifically designed to detect these small differences in weight. Finally, the practice of using syringes to measure milk volume rather than weighing the milk before and after feeding is puzzling. All of these factors may have contributed to the error in the test-weight estimates in the current study.

Finally, there are incorrect statements in this research report. The authors suggest that previous investigators have not adequately quantified the precision and accuracy of test-weighing, and have reported only correlation coefficients to describe the accuracy of test weights. This is simply incorrect; our studies of test-weighing include numerous statistics appropriate for quantifying the magnitude of error in physical measures such as weight (4). The statistics reported in those studies included the mean differences, standard deviation of the net differences, mean absolute differences, maximal differences, percentage of differences exceeding 5 g, and the overall percentage of error in the measurement, calculated as (((|actual-estimated values|) / actual value)*100) (5; 6). The investigators also incorrectly assert that “differences of up to 30 ml” have been reported by all previous studies. This is also incorrect; in our 1990 publication addressing the accuracy of test-weights for premature infants (5), the maximum difference between the actual and estimated values for the electronic scale was 10 ml, and only 6.25% of the differences exceeded 5 ml.

In summary, test-weighing, when performed with standard research controls and electronic scales that weigh to the nearest 1 to 2 g, has been demonstrated to be accurate in well-controlled clinical trials and has been endorsed by the World Health Organization as a method of accurately estimating intake. The lower accuracy in measures reported by these investigators underscore the need to carefully select a scale and control for the procedures used for test-weighing in the clinical setting to the extent possible, but do not indicate that test-weighing is too inaccurate for clinical use.

Paula P. Meier, DNSC, RN
Janet L. Engstrom, PhD, RNC, CNM

Rush University Medical Center
Chicago
Illinois
United States

References:

1. Savenije OEM, Brand PLP. Accuracy and precision of test weighing to assess milk intake in newborn infants. Arch Dis Child Fetal Neonatal Ed 2006; 91 (5): F330-332.

2. Kavanaugh K, Engstrom JL, Meier PP, Lysakowski TY. How reliable are scales for weighing preterm infants. Neonatal Network 1990;9(3): 29- 32.

3. Engstrom JL, Kavanugh K, Meier PP, Boles E, Hernandez J, Wheeler D, Chuffo R. Reliability of in-bed weighing procedures for critically ill infants. Neonatal Network 1995;14(5):27-33.

4. Engstrom JL. Assessment of the reliability of physical measures. Research in Nursing and Health 1988;11:383-389.

5. Meier PP, Lysakowski, TY, Engstrom JL, Kavanaugh KL, Mangurten H. The accuracy of test weighing for preterm infants. Journal of Pediatric Gastroenterology and Nutrition 1990;10:62-5.

Send letter to journal:
Re: Correct or incorrect? That's the question

Dear Editor,

Drs Meier and Engstrom raise a number of issues regarding our paper and its conclusions. Their first concern is our use of the terms "precision" and "accuracy", which they claim are incorrect. Although it is true that the term "precision" can be used to capture repeatability of a measurement (as Meier and Engstrom have reported themselves), it can be (and has been) also used in the fashion that we describe.(1)Meier and Engstrom's assertion that our definition is incorrect is, therefore, incorrect - it's just different. The same goes for our definition of accuracy. It is correct that our results differ from previous studies, including those of drs Meier and Engstrom. We agree that this may be due to different measurement conditions. Whilst test weighing may perform reasonably well under strictly controlled conditions (as Meier and Engstrom have shown) it is unlikely that such conditions can be implemented in practice. It is not the performance of the test weighing under laboratory conditions that is important, it is its performance "in the field" (i.e., on a busy newborn ward under daily practical conditions)that counts. As our results clearly show, test weighing is an unreliable procedure to estimate milk intake under those conditions. This has been recognized by other eLetter writers. The scale we used was described carefully, with brand name, type number, design aim (to weigh infants in single grams) and measurement characteristics (we reported the repeatability - or precision, if we follow Meier and Engstrom's definition - of measurements which was actually VERY good, with an SD of repeated measurements of < 1 g, or < 0.5%). This should reassure drs Meier and Engstrom's that this scale was, indeed, designed to measure reliably down to the single gram. The measurement characteristics of our scale are not inferior to the scales used by Meier and Engstrom, which, in their words, were "specifically designed to detect such small differences in weight". The Royal Dutch Pharmaceutical Society, whose published guidelines we followed, apparently uses stricter standards for weighing purposes than drs Meier and Engstrom do. It would be shortsighted to call the American standard "incorrect" - it's just different. Although drs Meier and Engstrom correctly raise the point that test weighing may be reliable under strictly controlled conditions, our results clearly show that it is not in daily clinical practice. That does not justify the qualification that our results are "incorrect" or theirs are correct. They're just different, and complementary. We believe that our results justify the abandoning of test weighing in daily clinical practice, and it would appear from the other letters that this view is endorsed by others.

Zwolle, March 2007

Paul Brand

Reference: