A Nature report about a new, enzymatic assay of Mycobacterium tuberculosis is (inadvertently) mostly a stark reminder that the false positive and the false negative rates are both important for evaluating an assay’s performance. Superficially, no doubt the assay has advantages: it does not require PCR, prolonged bacterial culture or microscopy, and delivers a result in half an hour unlike current standard methods. It is also sensitive: in a test it flagged all samples positive which microscopy found. Microscopy missed 50% of all positive samples, but even of those missed by microscopy the new method flagged 80% positive. Overall the assay recognises 90% of the Tb+ cases as such.

Despite of these advantages this is not yet a promising method. There is a 27 % false positive rate, i. e. the assay flags a quarter of all tested patients as Tb positive even though there are no Tb-causing bacteria in their samples. This is a problem because only 2-400 / 100 000 people get tuberculosis in any country of the world (World Bank). The new test flags about 27 000 positive out of those 99 600 healthy persons in the population.

If we compare the performance of this test to another “test” which just randomly picks people from the general populace and then says they are Tb positive, then the new test performs only 3.3 times better. This assumes 400 / 100 000 infected with tuberculosis. Within the group of people diagnosed as Tb+ with the new test the rate of actual Tb+ infections is:

(Tb+ & test +) /  [(Tb- & test +) + (Tb+ & test +)] =

0.9 * 0.004 / [(0.27*(1-0.004)) + 0.9 * 0.004)] = 0.0132

Comparing this to the Tb+ – rate in the general population of 0.004 (expected if randomly sampling from the population) gives the enrichment of 3.4-fold:

0.0132 / 0.004 = 3.3

This gets worse if the infection rate in the population is less.

It remained unclear from the article whether the false positive rate is caused by the test responding to mycobacteria almost in general, not only to the target pathogen Mycobacterium tuberculosis. It remained also doubtful whether there still might be a good use case. If, as has been proposed, the new method is used for prescreening before other methods like PCR, then this approach saves 2/3 of the tests. The prescreening however will miss some infections, and therefore it is worth keeping in mind that with prescreening the false negative rate will increase up to threefold if a recent evaluation of PCR specificity is to be trusted. Therefore this use case seems another risky proposition in a not well thought out report.