Bootstrap (resampling)
Also known as: Resampling · Efron's bootstrap · Resampling with replacement
A technique that estimates the uncertainty of a metric by resampling the data itself with replacement hundreds of times and watching how much the result varies. It is used when there is no closed-form formula for the margin of error.
Legal basis
Efron (1979) — Bootstrap Methods, Annals of Statistics
For the margin of error of a mean there is a formula; for the Gini, the lift or the difference between two curves, often there is none. Bootstrap solves this without a formula: it draws, with replacement, a new sample of the same size from the observed data, recomputes the metric, and repeats the process — typically 1,000 times.
The distribution of those thousands of results is the metric's own uncertainty: the 2.5th and 97.5th percentiles give a 95% confidence interval. Proposed by Bradley Efron in 1979, it is the standard way to put an error bar on almost any number when theory does not offer one.
Frequently asked questions
Why use bootstrap instead of a formula?
Because metrics like Gini, lift and the difference between curves have no simple closed-form formula for the margin of error. Bootstrap estimates that uncertainty by resampling the data with replacement and watching how much the result varies.