Sample test
Also known as: Proof of concept · PoC · Pilot · Backtest · Validation test
Validation of the score on a sample of your own historical base, with success criteria fixed in advance, to measure the real gain without touching the operation.
At Zarv
Validation in 2 phases — pilot and expansion — on historical data, with no change to operations.
Before switching a score on in production, a sample test answers a business question with data you already hold: on this portfolio, does this score separate risk enough to justify the decision? You analyze a sample of the historical base — cases with and without the event — and measure Gini, decile lift and incremental lift over the current model, each with a confidence interval.
What makes the result objective is fixing the success criteria before seeing the data, and running in two phases: a smaller pilot that has to hit the criterion, and only then the expansion. Nothing in the operation changes during the test — it measures what the score would have done, not what it changes going forward.
Frequently asked questions
What is a score sample test?
It is a validation of the score on a sample of the company's own historical base, with success criteria defined before the test, to measure the real gain (Gini, lift, incremental lift) without altering the operation.