Bootstrap Resampling

A statistic calculated from one dataset is only an estimate. Another sample from the same population could produce a different mean, correlation or evaluation score. The bootstrap approximates aspects of that sampling variation using the observations already available. In the ordinary nonparametric bootstrap for independent observations, draw a sample of the original size with replacement. Each draw can select any original observation again, so some observations appear more than once and others are omitted. Calculate the statistic on that sample. Repeat the process many times to produce a distribution of bootstrap statistics. For an arithmetic illustration, the values 2, 4 and 9 have a mean of five. A possible resample is 2, 2 and 9, whose mean is about 4.33. One resample does not establish an uncertainty interval. The variation across many appropriately constructed resamples is what makes the procedure useful. A percentile interval takes endpoints from percentiles of the bootstrap distribution. Other methods, including the bias-corrected and accelerated method supported by SciPy, adjust the construction differently. The chosen method and its assumptions matter, particularly for small samples or difficult statistics. For paired data, preserve the pairing. When comparing model predictions on the same cases, resample each case with its outcome and both predictions. Breaking those relationships changes the question being evaluated. The bootstrap cannot create evidence about people or conditions missing from the original sample. It also does not automatically correct selection bias or dependence between observations. Repeated measurements may require resampling groups; time series may require a suitable block method. Choose the resampling unit to reflect the data-generating process before interpreting the apparent precision.