Herbie results

Output vs Input Accuracy

Each point represents a Herbie run below. Its horizontal position shows initial accuracy, and vertical position shows final accuracy. Points above the line are improved by Herbie.

Accuracy vs Cost

A joint cost-accuracy pareto curve for the Herbie runs below. Accuracy is on the vertical axis, and cost is on the vertical axis. Down and to the left is better. The initial programs are shown by the red square.

Test	Start	Result ?	Target ?	Time
arccos	100.0%	100.0%		4.8s	»
Rump's expression from Stadtherr's award speech	9.2%	9.2%		0.8s	»
Rosa's TurbineBenchmark	79.9%	99.7%		19.3s	»
Rosa's FloatVsDoubleBenchmark	99.2%	99.6%		39.0s	»
Rosa's DopplerBench	71.9%	98.1%		11.1s	»
Rosa's Benchmark	99.6%	99.7%		8.5s	»
Radioactive exchange between two surfaces	100.0%	100.0%		1.7s	»
NMSE Section 6.1 mentioned, B	77.5%	99.6%		13.8s	»
NMSE Section 6.1 mentioned, A	54.9%	99.1%		17.0s	»
Kahan's Unum-Targeted Monster				4.7s	»
Kahan's Monster				2.3s	»
Kahan p13 Example 3	100.0%	100.0%		7.3s	»
Kahan p13 Example 2	100.0%	100.0%		12.8s	»
Kahan p13 Example 1	99.9%	99.9%		16.0s	»
From Warwick Tucker's Validated Numerics	9.2%	100.0%		2.2s	»
From Rump in a 1983 paper, rewritten	3.1%	100.0%		3.9s	»
From Rump in a 1983 paper	18.8%	100.0%		3.6s	»
ENA, Section 1.4, Mentioned, B	87.9%	99.6%		9.4s	»
ENA, Section 1.4, Exercise 4b, n=5	88.4%	99.4%		14.4s	»
ENA, Section 1.4, Exercise 4b, n=2	75.5%	100.0%		7.7s	»
ENA, Section 1.4, Exercise 1	94.5%	99.5%		9.6s	»
x / (x^2 + 1)	76.3%	99.9%	99.8%	1.7s	»
simple fma test	29.7%	100.0%	100.0%	3.7s	»
ln(1 + x)	39.7%	100.0%	99.6%	3.6s	»
fma_test2	0.0%	99.5%	99.5%	10.6s	»
fma_test1	3.4%	99.5%	21.0%	3.4s	»
The quadratic formula (r2)	46.2%	82.8%	67.2%	22.9s	»
The quadratic formula (r1)	46.7%	83.5%	66.7%	20.2s	»
Rectangular parallelepiped of dimension a×b×c	100.0%	100.0%	100.0%	1.4s	»
Kahan's exp quotient	37.1%	100.0%	36.4%	1.9s	»
Kahan p9 Example	67.5%	99.9%	99.9%	16.4s	»
Expression, p6	94.3%	100.0%	94.0%	7.6s	»
Expression, p14	99.9%	99.9%	99.9%	7.3s	»
Expression 4, p15	100.0%	100.0%	100.0%	4.1s	»
Expression 3, p15	100.0%	100.0%	100.0%	2.4s	»
Expression 2, p15	100.0%	100.0%	100.0%	0.8s	»
Expression 1, p15	99.4%	99.6%	99.6%	9.7s	»
ENA, Section 1.4, Mentioned, A	53.0%	100.0%	100.0%	7.7s	»
ENA, Section 1.4, Exercise 4d	61.6%	99.5%	99.5%	9.3s	»
ENA, Section 1.4, Exercise 4a	53.3%	99.5%	98.8%	19.8s	»
Difference of squares	100.0%	100.0%	100.0%	1.5s	»
Compound Interest	24.7%	99.1%	25.9%	22.1s	»
Complex division, real part	59.3%	99.0%	99.4%	14.4s	»
Complex division, imag part	59.6%	98.4%	99.3%	13.5s	»
Area of a triangle				0.2s	»

Test

Start

Result ?

Target ?

Time

arccos

100.0%

4.8s

Rump's expression from Stadtherr's award speech

9.2%

0.8s

Rosa's TurbineBenchmark

79.9%

99.7%

19.3s

Rosa's FloatVsDoubleBenchmark

99.2%

99.6%

39.0s

Rosa's DopplerBench

71.9%

98.1%

11.1s

Rosa's Benchmark

99.6%

99.7%

8.5s

Radioactive exchange between two surfaces

100.0%

1.7s

NMSE Section 6.1 mentioned, B

77.5%

99.6%

13.8s

NMSE Section 6.1 mentioned, A

54.9%

99.1%

17.0s

Kahan's Unum-Targeted Monster

4.7s

Kahan's Monster

2.3s

Kahan p13 Example 3

100.0%

7.3s

Kahan p13 Example 2

100.0%

12.8s

Kahan p13 Example 1

99.9%

16.0s

From Warwick Tucker's Validated Numerics

9.2%

100.0%

2.2s

From Rump in a 1983 paper, rewritten

3.1%

100.0%

3.9s

From Rump in a 1983 paper

18.8%

100.0%

3.6s

ENA, Section 1.4, Mentioned, B

87.9%

99.6%

9.4s

ENA, Section 1.4, Exercise 4b, n=5

88.4%

99.4%

14.4s

ENA, Section 1.4, Exercise 4b, n=2

75.5%

100.0%

7.7s

ENA, Section 1.4, Exercise 1

94.5%

99.5%

9.6s

x / (x^2 + 1)

76.3%

99.9%

99.8%

1.7s

simple fma test

29.7%

100.0%

3.7s

ln(1 + x)

39.7%

100.0%

99.6%

3.6s

fma_test2

0.0%

99.5%

10.6s

fma_test1

3.4%

99.5%

21.0%

3.4s

The quadratic formula (r2)

46.2%

82.8%

67.2%

22.9s

The quadratic formula (r1)

46.7%

83.5%

66.7%

20.2s

Rectangular parallelepiped of dimension a×b×c

100.0%

1.4s

Kahan's exp quotient

37.1%

100.0%

36.4%

1.9s

Kahan p9 Example

67.5%

99.9%

16.4s

Expression, p6

94.3%

100.0%

94.0%

7.6s

Expression, p14

99.9%

7.3s

Expression 4, p15

100.0%

4.1s

Expression 3, p15

100.0%

2.4s

Expression 2, p15

100.0%

0.8s

Expression 1, p15

99.4%

99.6%

9.7s

ENA, Section 1.4, Mentioned, A

53.0%

100.0%

7.7s

ENA, Section 1.4, Exercise 4d

61.6%

99.5%

9.3s

ENA, Section 1.4, Exercise 4a

53.3%

99.5%

98.8%

19.8s

Difference of squares

100.0%

1.5s

Compound Interest

24.7%

99.1%

25.9%

22.1s

Complex division, real part

59.3%

99.0%

99.4%

14.4s

Complex division, imag part

59.6%

98.4%

99.3%

13.5s

Area of a triangle

0.2s