The brewer who proved the cheaper batch was just as good.
We named Gosset after the man who invented the test for the question our customers ask about AI models.
Is the cheaper barley just as good?
William Sealy Gosset joined the Guinness brewery in Dublin as a young chemist. The brewery kept asking him whether switching to a different, cheaper barley would change the beer. Every test batch cost time and money, so he only ever had a handful of results to judge from.
A test built for small samples
The statistics of the day assumed large samples and gave confident, often wrong answers on a few. Gosset worked out the maths for small ones: how sure you can be that a difference is real and not chance. It is now called the t-test, and it is used in science and medicine every day.
The work, not the name
Guinness did not let its researchers publish under their own names. So Gosset published "The probable error of a mean" in the journal Biometrika under a pen name: Student. The method became famous while he stayed out of view, and he later became Head Brewer of Guinness.
Is the cheaper model just as good for my app?
New AI models ship every week and prices keep changing. We answer the question the way Gosset would: take a small sample of your real requests, run them more than once to separate chance from real change, and show the evidence before anything changes.
Like Student, we would rather the work speak for itself.
Source: Student (1908), "The probable error of a mean", Biometrika 6(1). Gosset is not affiliated with Guinness or Champagne Gosset.