1$FreeBSD$ 2 3A small tool to do the statistics legwork on benchmarks etc. 4 5Prepare your data into two files, one number per line 6run 7 ./ministat data_before data_after 8 9and see what it says. 10 11You need at least three data points in each data set, but the more 12you have the better your result generally gets. 13 14Here are two typical outputs: 15 16x _1 17+ _2 18+--------------------------------------------------------------------------+ 19|x + x+ x x x + ++ | 20| |_________|______AM_______________|__A___________M_______________|| 21+--------------------------------------------------------------------------+ 22 N Min Max Median Avg Stddev 23x 5 36060 36138 36107 36105.6 31.165686 24+ 5 36084 36187 36163 36142.6 49.952978 25No difference proven at 95.0% confidence 26 27Here nothing can be concluded from the numbers. It _may_ be possible to 28prove something if many more measurements are made, but with only five 29measurements, nothing is proven. 30 31 32x _1 33+ _2 34+--------------------------------------------------------------------------+ 35| + | 36| x + +| 37|x x x x + +| 38| |_______________A_____M_________| |_M___A____| | 39+--------------------------------------------------------------------------+ 40 N Min Max Median Avg Stddev 41x 5 0.133 0.137 0.136 0.1354 0.0015165751 42+ 5 0.139 0.14 0.139 0.1394 0.00054772256 43Difference at 95.0% confidence 44 0.004 +/- 0.00166288 45 2.95421% +/- 1.22812% 46 (Student's t, pooled s = 0.00114018) 47 48Here we have a clearcut difference, not very big, but clear and unambiguous. 49 50 51