We can learn what is achievable with current technologies by comparing our neonatal intensive care unit outcomes with others. Because neonatal intensive care units may vary with respect to their case-mix, risk adjustment is essential to making fair comparisons in any research that does not equalize risks through randomization. Risk adjustment first requires strict definition of each specific outcome. Then each risk factor is measured and weighted accordingly. Severity of illness scores are a special form of risk adjustment. The leading newborn illness severity scores rely on physiology-based items from bedside vital signs and laboratory tests. The mechanics of score development are discussed including item selection, definition, collection, and potential biases. The process of weighting risk factors usually involves building multivariate models. Issues of derivation, validation, discrimination, calibration, and reliability affect the utility of all scores. Once a comparison is appropriately risk-adjusted, there are important cautions about interpretation, including the source of the reference (benchmark) population, sample size, and biases from incomplete risk adjustment. Nonetheless, these findings can spur quality improvement efforts that can lead to dramatic, system-wide improvements in outcomes.