README
1A small tool to do the statistics legwork on benchmarks etc.
2
3Prepare your data into two files, one number per line
4run
5 ./ministat data_before data_after
6
7and see what it says.
8
9You need at least three data points in each data set, but the more
10you have the better your result generally gets.
11
12Here are two typical outputs:
13
14x _1
15+ _2
16+--------------------------------------------------------------------------+
17|x + x+ x x x + ++ |
18| |_________|______AM_______________|__A___________M_______________||
19+--------------------------------------------------------------------------+
20 N Min Max Median Avg Stddev
21x 5 36060 36138 36107 36105.6 31.165686
22+ 5 36084 36187 36163 36142.6 49.952978
23No difference proven at 95.0% confidence
24
25Here nothing can be concluded from the numbers. It _may_ be possible to
26prove something if many more measurements are made, but with only five
27measurements, nothing is proven.
28
29
30x _1
31+ _2
32+--------------------------------------------------------------------------+
33| + |
34| x + +|
35|x x x x + +|
36| |_______________A_____M_________| |_M___A____| |
37+--------------------------------------------------------------------------+
38 N Min Max Median Avg Stddev
39x 5 0.133 0.137 0.136 0.1354 0.0015165751
40+ 5 0.139 0.14 0.139 0.1394 0.00054772256
41Difference at 95.0% confidence
42 0.004 +/- 0.00166288
43 2.95421% +/- 1.26027%
44 (Student's t, pooled s = 0.00114018)
45
46Here we have a clearcut difference, not very big, but clear and unambiguous.
47