10 December 2010

Number Needed to Hit

In the comments on my previous post on Dice and Information, I mentioned a statistic called Number Needed to Treat (NNT). This is used in epidemiology and medical science as a way to compare the benefit and cost of two treatment. For example, this might be two different drugs that might be used to treat an illness. Here "benefit" means any beneficial effect, maybe something major like preventing fatal heart disease, or something minor like prevention of allergy symptoms. For this discussion I will assume it is a BIG benefit, like saving a life.

Benefit: Suppose that in a clinical trial running one year, 80% of patients on drug A benefit from the drug, and 82% of patients on drug B benefit equally. Now drug B is better by 2%, but what does that mean? If we treat 100 patient on each drug for one year, we expect 80 "benefactors" from Drug A and 82 benefactor from drug B, a difference of two patients saved every year for every 100 treated, or one saved for every 50 patients treated (for one year).

A shorter way to calculate this is NNT = 1/(0.82-0.80) 1/(0.02) = 50 patients "needed to treat" for one to benefit. Make sure to subtract the smaller probability from the larger, or you up with a negative number.

Cost: Now suppose that drug B costs $10,000 per year, compared to $1000 per year for drug A; Is drug B worth ten times the expense? NNT helps answer that question. Consider that treating 50 patients for one year (the NNT) would cost $250,000 on drug B and and $50,000 on drug A - a difference of $200,000 for one life saved each year. Now is drug B worth the cost? If the benefit is strong, like not dying, then it may be worthwhile. It certainly is if you are the 1-in-50 patient, but you can't know that is advance. Also, resources are limited. It is not good practice to make patients pay large expenses, or tie-up hospital resources, for treatments with very little benefit.

If anyone is wondering, this is not Obama-Care Death-Panel stuff, this is a serious sort of decision that determines what doctors call "Best Practice".

Well so far this has been pretty boring, but now we can take the idea behind NNT and turn it into a statistic for games. 

If we replace the probability a treatment will be effective with the probability of a successful "to-hit" roll, we get a measure of Number or rolls Needed to Hit (NNH). This is now the number of to-hit attempts you will need to make, with some higher probability of success, in order to hit one more target on average. In the sniper rifle example from the previous comments, the difference between 80% and 82% means one more hit out of every 50 shots fired. This 2% gain could be useful from the perspective of a first-person-shooter game, or even critical to ensure taking out an important target, but on a larger scale other things could matter more. For instance, if the more accurate rifle weighs more, or costs more, and this is reflected in game usage and costs, then 2% gain in accuracy might not be worthwhile.

GBR Giant Battling Robots Favicon
Post a Comment