The Number Needed to Treat (NNT) is becoming increasingly popular as a measure to compare treatment outcome between two groups of patients. The NNT is suitable to compare two proportions and can be used, for example, in randomized controlled trials to compare mortality in an experimental group with mortality in a control group. One of the main reasons for the popularity of the NNT is that it is a measure which is easy to interpret and explain. For example, to study the difference in mortality between two different treatments, an NNT of two means that two patients need to be treated with a new treatment instead of an old treatment for one additional patient to survive. The NNT is calculated by taking the inverse of the difference between two proportions:

NNT= 1/(p₁-p₂)

This paper will compare the characteristics of the NNT with the characteristics of a number of other measures and the discuss the use and limitations of the NNT for use in research. This discussion is partly based on Reynolds (1977), who in his book discusses the properties and requirements of measures of association for crosstabulations.

The characteristics of the NNT can be compared with those of other measures, such as the correlation coefficient and the odds-ratio. The correlation coefficient is often seen as an ideal measure. The correlation coefficient has an un-interrupted range and it has, at -1 and +1, two clearly specified points were total negative or positive association is accomplished. The value 0 is the point of zero association and the boundary between positive and negative association. The correlation coefficient is symmetrical, producing directly comparable values for positive and negative associations of similar strength. The correlation coefficient can be intuitively understood in terms of explained variance.

Compared with the correlation coefficient the odds ratio is considered to be less ideal. It has an un-interrupted range with 0 as the clearly specified value for total negative association and one as the point of no-association. Infinity is the point for total positive association, not an ideal value to assign to something which can be relatively easily defined empirically. The odds ratio is not symmetrical, however, it does not take much effort to directly compare negative and positive associations if required. The odds-ratio can be understood as the ratio of two odds although interpreting the ratio in this way requires some experience.

The NNT has an interrupted range whereby values between -1 and +1 are not valid. The NNT is symmetrical, for example, if in a clinical trial in the control group mortality would be 10% while in the experimental group mortality would be 5% the NNT would be 20 [1/(.1-.05)]. If the same data is used in terms of survival, 90% survives in the control group against 95% in the experimental group, this gives an NNT of -20 [1/(.9-.95)]. The NNT for total negative and positive associations equals minus one and one respectively, and this is the case when each additional patient treated improves according to the study definition. One of the main problem with the NNT is that the numerical value for no association can equal both plus and minus infinity. This is if an infinite number of patients would have to be treated, or not treated, to get one improving. Contrary to the correlation co-efficient and the odds-ratio the point of no association does not at the same time define the boundary between positive and negative associations. Of the three measures discussed the NNT is easiest to interpret.

The fact that the NNT reaches no association at infinity means that the NNT cannot be directly used in testing for statistical significance. To do this the value of an NNT should have to be tested against infinity, which is impossible. In research the problem seems to have been overcome by first proving statistical significance by using a t-test or Chi-square test, following which the NNT is used to explain the findings, a valuable and valid procedure. Increasingly, however, the NNT is now presented with confidence intervals. But to use a confidence interval around the NNT validly the confidence interval should be differently used and interpreted and cannot be compared with a confidence interval around, for example, a correlation coefficient, an odds ratio or the difference between two means.

The use of a confidence interval has mostly two purposes. First, as a descriptive measure with the aim of giving an insight in the precision of an estimate, wider intervals indicating less precision. Secondly, to test for statistical significance, statistical significance declared if the confidence interval does not incorporate the value which indicates no association. For example, for the odds ratio, if the odds ratio is larger than one but the confidence interval around the odds ratio incorporates values between one and zero as well as values higher than one, it is considered that it is likely that a relationship is positive, but, it might not be much less likely that the relationship is negative. In terms of treatments, the new treatment might have a good likelihood that it is more beneficial compared with the old treatment, but it might not be much less likely that the old treatment is more beneficial. In this case the conclusion is that the association between treatment and outcome is not statistically significant. Similarly, if a confidence interval for the correlation coefficient contains the value zero, it is considered that an association is not statistically significant.

This hypothesis testing approach is not possible for the NNT. To test for significance using the NNT requires that the confidence interval for the NNT would incorporate infinity. Also, the NNT does not provide an uninterrupted range of values representing both negative and positive associations. However, although these limitations of the NNT seem obvious, two methods for calculating confidence intervals for the NNT have been proposed.

The first method is proposed by Cook and Sacket (1995) and based on calculating first the confidence interval for the difference between the two proportions. The result of this calculation is 'inverted' (ie: 1/CI) to give a confidence interval for the NNT.

The second method is proposed by Schulzer and Mancini (1996) and based on the fact that a waiting time model can be applied to the NNT. Consider that one has to, on average, wait NNT-1 patients to observe one patient in which the treatment is successful. In this case the variance around the NNT follows the geometrical distribution and a standard error considering the number of observations can be obtained. The formulae for calculating the upper bound for the confidence interval around the NNT is given in formulae 1 for the method suggested for Cook and Sacket and in formulae 2 for the method suggested by Schulzer and Mancini.

If the data concerns a relatively large number of cases, about 1000 in each group, the two methods produce similar results. However, for smaller numbers of cases the confidence interval for the NNT is much wider using the method suggested by Cook and Sacket compared with the method suggested by Schulzer and Mancini. If the association is not statistically significant the method suggested by Cook and Sacket produces a confidence interval which does not contain the value of the NNT. The method suggested by Schulzer and Mancini contains values in the invalid range between -1 and +1.

DISCUSSION

Although the NNT is now quite widely used there remain a large number of questions to be answered with regard to the formal properties of this measure of association. As long as the NNT is used only for description there is no problem, the NNT is easy to interpret and the link the NNT can provide with practice, particularly with cost benefit analysis, is appealing and practical. However, confidence intervals for the NNT are now being presented and the validity of this seems questionable. It is doubtful that the problems are widely known among readers of medical journals and the approach to present confidence intervals for the NNT only in cases of statistical significance seems to be an invitation to commit publication bias. Then there is the problem that two methods to calculate the confidence interval are proposed. Both provide estimates, however, the two methods of estimation provide very different results.

One wonders if it is not a better idea to be more cautious in the use of the NNT. The measure on which the NNT is based, the absolute difference between two proportions, is one of the best researched in statistics and about as easy to interpret as the NNT. Maybe the NNT should be used only sparingly and preferably only in cases were its use is obvious, for example, in the case of cost comparisons between treatments. Preference should be given to the difference between means with associated confidence intervals and tests of statistical significance.

Armitage P, Berry G. Statistical methods in medical research. Oxford Blackwell Scientific Publications Pty Ltd, 1971.

Cook R J, Sackett D L. The number needed to treat: a clinically useful measure of treatment effect. Br Med J 1995; 310: 542-454.

Reynolds H T. The analysis of cross-classifications. New York, London: Free Press, Collier Macmillan, 1977.

Schultzer M, Mancini G B J. 'Unqualified success' and 'unmitigated failure'; Number-Needed -to-Treat Related concepts for assessing treatment efficacy in the presence of treatment induced adverse effects. Int J Epidemiol 1996; 25(4):704-712.

TOP of page

Compare Car Rentals!
Help SISA and compare two rental cars!
An easy way to find the best option.
www.quantitativeskills.com