1 2A small tool to do the statistics legwork on benchmarks etc. 3 4Prepare your data into two files, one number per line 5run 6 ./ministat data_before data_after 7 8and see what it says. 9 10You need at least three data points in each data set, but the more 11you have the better your result generally gets. 12 13Here are two typical outputs: 14 15x _1 16+ _2 17+--------------------------------------------------------------------------+ 18|x + x+ x x x + ++ | 19| |_________|______AM_______________|__A___________M_______________|| 20+--------------------------------------------------------------------------+ 21 N Min Max Median Avg Stddev 22x 5 36060 36138 36107 36105.6 31.165686 23+ 5 36084 36187 36163 36142.6 49.952978 24No difference proven at 95.0% confidence 25 26Here nothing can be concluded from the numbers. It _may_ be possible to 27prove something if many more measurements are made, but with only five 28measurements, nothing is proven. 29 30 31x _1 32+ _2 33+--------------------------------------------------------------------------+ 34| + | 35| x + +| 36|x x x x + +| 37| |_______________A_____M_________| |_M___A____| | 38+--------------------------------------------------------------------------+ 39 N Min Max Median Avg Stddev 40x 5 0.133 0.137 0.136 0.1354 0.0015165751 41+ 5 0.139 0.14 0.139 0.1394 0.00054772256 42Difference at 95.0% confidence 43 0.004 +/- 0.00166288 44 2.95421% +/- 1.22812% 45 (Student's t, pooled s = 0.00114018) 46 47Here we have a clearcut difference, not very big, but clear and unambiguous. 48 49 50