User:Starwed/Spading

Applied statistics for Spades

A lot of the time we want to know the rate at which something happens: How often does a Yeti get the jump on me? How often does deodorant drop? How often do I critical hit a monster? The way to spade this is, of course, to spend many adventures, and record exactly how often it happens. That bits easy, if sometimes time consuming. But then you need to know the statistical power of your data.

This page won't go into the mathematics of any of this; it's just going to provide results. For more information, you can see the following sources:

• wikipedia:Margin of error
• Page 12 of: An Introduction to Stochastic Processes in Physics; Don S Lemons. (You can find this at google books.)

Standard Error

...even the most stupid of men, by some instinct of nature, by himself and without any instruction (which is a remarkable thing), is convinced that the more observations have been made, the less danger there is of wandering from one's goal - Jacob Bernoulli

The standard error gives a measure of how close the observed probability (p) is likely to the actual rate. We can be 68% certain that the real value is within 1.0 standard errors of the observed rate, 95% certain that it lies within 1.96, and 99% certain that it lies within 2.58. The formula is:

Standard error =

As expected, the more observations made (n) the smaller it will be. But it also depends on p * (1-p), which is largest when p=.5. So the further the observed rate is from 50%, the less observations you need to get a good confidence interval on the rate.

Item drops

Turns out that most item drop rates are guaranteed to be integers, which means there's a better way to spade those!

Sometimes what you observe directly might not actually be what you're trying to spade; a good example is item drop rate. You might want to see how an effect modifies item drops, or you might want to know the base drop rate (r) of an item. The observed probability of a drop (p) will be:

p = r * (1 + b)

Where r is the base drop rate and b the total item bonus.

Suppose you're trying to spade out the base drop rate of an item. Then solving the above equation for r:

r = p / (1 + b)

Now, this means that the standard error for r will usually be smaller than p:

StdErr_r = StdErr_p / (1+b)

Whereas if you're trying to calculate the bonus (b) of some unknown effect, the formula will be

b_unknown = p/r - (1 + b_known)

And the standard error will then be bigger than that of p:

StdErr_b = StdErr_p / r

Combining Measurements

It might be the case that you measure a particular quantity several times using a different method. It's possible to combine these several measurements into a single measurement with a smaller standard error than any one by itself, using a weighted mean.

The more certain a particular measurement is, the more weight we want to give it when calculating the average. The correct weight to use is something proportional to the inverse variance: 1/StdErr^2. For more information see wikipedia.