Herbie results

Output vs Input Accuracy

Each point represents a Herbie run below. Its horizontal position shows initial accuracy, and vertical position shows final accuracy. Points above the line are improved by Herbie.

Accuracy vs Cost

A joint cost-accuracy pareto curve for the Herbie runs below. Accuracy is on the vertical axis, and cost is on the vertical axis. Down and to the left is better. The initial programs are shown by the red square.

Test	Start	Result ?	Target ?	Time
arccos	0.02%	0.02%		4.9s	»
Rump's expression from Stadtherr's award speech	90.84%	90.84%		0.8s	»
Rosa's TurbineBenchmark	19.49%	0.42%		16.6s	»
Rosa's FloatVsDoubleBenchmark	0.81%	0.43%		37.5s	»
Rosa's DopplerBench	28.16%	2.1%		11.1s	»
Rosa's Benchmark	0.39%	0.34%		6.2s	»
Radioactive exchange between two surfaces	0%	0%		11.0s	»
NMSE Section 6.1 mentioned, B	22.62%	0.38%		13.0s	»
NMSE Section 6.1 mentioned, A	46.8%	0.87%		15.0s	»
Kahan's Unum-Targeted Monster				4.5s	»
Kahan's Monster				2.1s	»
Kahan p13 Example 3	0.03%	0.02%		8.7s	»
Kahan p13 Example 2	0.04%	0.02%		13.2s	»
Kahan p13 Example 1	0.03%	0.16%		12.5s	»
From Warwick Tucker's Validated Numerics	90.84%	0%		1.9s	»
From Rump in a 1983 paper, rewritten	96.87%	0%		3.5s	»
From Rump in a 1983 paper	81.25%	0%		3.1s	»
ENA, Section 1.4, Mentioned, B	12.23%	0.39%		8.1s	»
ENA, Section 1.4, Exercise 4b, n=5	12.01%	0.5%		12.7s	»
ENA, Section 1.4, Exercise 4b, n=2	24.95%	0.02%		6.1s	»
ENA, Section 1.4, Exercise 1	5.52%	0.61%		8.5s	»
x / (x^2 + 1)	23.1%	0.03%	0.14%	2.5s	»
simple fma test	70.49%	0%	0%	3.2s	»
ln(1 + x)	60.26%	0%	0.38%	3.1s	»
fma_test2	100%	0.46%	0.46%	10.6s	»
fma_test1	96.63%	0.49%	79.01%	3.3s	»
The quadratic formula (r2)	54.39%	16.77%	33.45%	17.6s	»
The quadratic formula (r1)	53.05%	16.31%	32.88%	19.0s	»
Rectangular parallelepiped of dimension a×b×c	0%	0%	0%	1.3s	»
Kahan's exp quotient	62.19%	0.03%	62.78%	3.2s	»
Kahan p9 Example	32.4%	0.04%	0.09%	16.1s	»
Expression, p6	5.64%	0%	5.93%	11.9s	»
Expression, p14	0.07%	0.04%	0.06%	7.0s	»
Expression 4, p15	0.03%	0.02%	0.02%	1.9s	»
Expression 3, p15	0.02%	0%	0.04%	2.4s	»
Expression 2, p15	0%	0%	0.03%	1.7s	»
Expression 1, p15	0.63%	0.38%	0.38%	9.1s	»
ENA, Section 1.4, Mentioned, A	47.74%	0.01%	0.02%	5.1s	»
ENA, Section 1.4, Exercise 4d	39.23%	0.45%	0.45%	7.6s	»
ENA, Section 1.4, Exercise 4a	46.96%	0.44%	1.26%	18.2s	»
Difference of squares	0.01%	0%	0%	1.5s	»
Compound Interest	75.36%	17.78%	75.04%	18.3s	»
Complex division, real part	40.48%	6.36%	0.63%	18.0s	»
Complex division, imag part	41.18%	1.07%	0.7%	19.3s	»
Area of a triangle				0.3s	»

Test

Start

Result ?

Target ?

Time

arccos

0.02%

4.9s

Rump's expression from Stadtherr's award speech

90.84%

0.8s

Rosa's TurbineBenchmark

19.49%

0.42%

16.6s

Rosa's FloatVsDoubleBenchmark

0.81%

0.43%

37.5s

Rosa's DopplerBench

28.16%

2.1%

11.1s

Rosa's Benchmark

0.39%

0.34%

6.2s

Radioactive exchange between two surfaces

11.0s

NMSE Section 6.1 mentioned, B

22.62%

0.38%

13.0s

NMSE Section 6.1 mentioned, A

46.8%

0.87%

15.0s

Kahan's Unum-Targeted Monster

4.5s

Kahan's Monster

2.1s

Kahan p13 Example 3

0.03%

0.02%

8.7s

Kahan p13 Example 2

0.04%

0.02%

13.2s

Kahan p13 Example 1

0.03%

0.16%

12.5s

From Warwick Tucker's Validated Numerics

90.84%

1.9s

From Rump in a 1983 paper, rewritten

96.87%

3.5s

From Rump in a 1983 paper

81.25%

3.1s

ENA, Section 1.4, Mentioned, B

12.23%

0.39%

8.1s

ENA, Section 1.4, Exercise 4b, n=5

12.01%

0.5%

12.7s

ENA, Section 1.4, Exercise 4b, n=2

24.95%

0.02%

6.1s

ENA, Section 1.4, Exercise 1

5.52%

0.61%

8.5s

x / (x^2 + 1)

23.1%

0.03%

0.14%

2.5s

simple fma test

70.49%

3.2s

ln(1 + x)

60.26%

0.38%

3.1s

fma_test2

100%

0.46%

10.6s

fma_test1

96.63%

0.49%

79.01%

3.3s

The quadratic formula (r2)

54.39%

16.77%

33.45%

17.6s

The quadratic formula (r1)

53.05%

16.31%

32.88%

19.0s

Rectangular parallelepiped of dimension a×b×c

1.3s

Kahan's exp quotient

62.19%

0.03%

62.78%

3.2s

Kahan p9 Example

32.4%

0.04%

0.09%

16.1s

Expression, p6

5.64%

5.93%

11.9s

Expression, p14

0.07%

0.04%

0.06%

7.0s

Expression 4, p15

0.03%

0.02%

1.9s

Expression 3, p15

0.02%

0.04%

2.4s

Expression 2, p15

0.03%

1.7s

Expression 1, p15

0.63%

0.38%

9.1s

ENA, Section 1.4, Mentioned, A

47.74%

0.01%

0.02%

5.1s

ENA, Section 1.4, Exercise 4d

39.23%

0.45%

7.6s

ENA, Section 1.4, Exercise 4a

46.96%

0.44%

1.26%

18.2s

Difference of squares

0.01%

1.5s

Compound Interest

75.36%

17.78%

75.04%

18.3s

Complex division, real part

40.48%

6.36%

0.63%

18.0s

Complex division, imag part

41.18%

1.07%

0.7%

19.3s

Area of a triangle

0.3s