Herbie results

Output vs Input Accuracy

Each point represents a Herbie run below. Its horizontal position shows initial accuracy, and vertical position shows final accuracy. Points above the line are improved by Herbie.

Accuracy vs Cost

A joint cost-accuracy pareto curve for the Herbie runs below. Accuracy is on the vertical axis, and cost is on the vertical axis. Down and to the left is better. The initial programs are shown by the red square.

Test	Start	Result ?	Target ?	Time
symmetry log of sum of exp	29.3	1.0		16.0s	»
raw-angle from scale-rotated-ellipse				2.5min	»
b parameter of renormalized beta distribution	0.1	0.2		7.4s	»
b from scale-rotated-ellipse	64.0	37.5		2.4min	»
ab-angle->ABCF D	16.1	0.3		4.3s	»
ab-angle->ABCF C	20.4	20.4		18.4s	»
ab-angle->ABCF B	31.4	21.5		21.0s	»
ab-angle->ABCF A	20.3	20.3		18.8s	»
a parameter of renormalized beta distribution	0.2	0.3		7.7s	»
a from scale-rotated-ellipse	63.4	46.6		2.4min	»
Trigonometry B	0.3	0.3		12.6s	»
Trigonometry A	0.1	0.1		9.7s	»
Spherical law of cosines	16.6	3.8		37.8s	»
Simplification of discriminant from scale-rotated-ellipse	41.4	6.0		1.3min	»
Quadratic roots, wide range	52.7	0.1		15.7s	»
Quadratic roots, narrow range	28.3	0.3		21.2s	»
Quadratic roots, medium range	43.8	2.9		23.8s	»
Quadratic roots, full range	34.2	10.1		16.7s	»
Midpoint on a great circle	0.8	0.2		22.2s	»
Logistic function from Lakshay Garg	29.2	0.6		11.5s	»
Hyperbolic tangent	58.2	0.0		3.4s	»
Hyperbolic sine	58.1	0.0		5.1s	»
Hyperbolic secant	0.0	0.0		3.0s	»
Hyperbolic arc-cosine	32.5	0.3		3.3s	»
Hyperbolic arc-(co)tangent	58.6	0.0		5.6s	»
Hyperbolic arc-(co)secant	0.1	0.1		6.8s	»
Example from Robby	0.1	0.1		18.4s	»
Example 2 from Robby	0.1	0.1		19.8s	»
Equirectangular approximation to distance on a great circle	39.0	3.8		18.0s	»
Eccentricity of an ellipse	14.5	0.0		4.0s	»
Distance on a great circle	23.7	13.6		2.0min	»
Destination given bearing on a great circle	0.2	0.2		32.2s	»
Cubic critical, wide range	52.8	0.1		15.6s	»
Cubic critical, narrow range	28.4	0.3		21.2s	»
Cubic critical, medium range	43.9	0.2		18.4s	»
Cubic critical	34.2	6.7		26.6s	»
Bearing on a great circle	13.1	0.2		51.3s	»
Asymptote C	29.6	0.0		8.9s	»
Asymptote B	0.0	0.0		5.6s	»
Asymptote A	14.5	0.4		3.8s	»
ABCF->ab-angle b	52.5	38.9		55.3s	»
ABCF->ab-angle angle	30.0	11.1		17.4s	»
ABCF->ab-angle a	52.6	33.3		54.6s	»
2-ancestry mixing, zero discriminant	15.8	0.8		6.9s	»
2-ancestry mixing, positive discriminant	35.6	2.8		12.3s	»
2-ancestry mixing, negative discriminant	1.0	0.1		10.2s	»
Quotient of sum of exps	0.8	1.2	0.0	7.8s	»
Quotient of products	11.5	2.6	10.7	4.3s	»
Logistic regression 2	0.5	0.5	0.1	9.5s	»
Hyperbolic arcsine	53.0	0.2	45.3	6.9s	»
Harley's example				2.5min	»
Exp of sum of logs	5.7	0	0	1.5s	»
(- (/ x0 (- 1 x1)) x0)				0.0s	»

Test

Start

Result ?

Target ?

Time

symmetry log of sum of exp

29.3

1.0

16.0s

raw-angle from scale-rotated-ellipse

2.5min

b parameter of renormalized beta distribution

0.1

0.2

7.4s

b from scale-rotated-ellipse

64.0

37.5

2.4min

ab-angle->ABCF D

16.1

0.3

4.3s

ab-angle->ABCF C

20.4

18.4s

ab-angle->ABCF B

31.4

21.5

21.0s

ab-angle->ABCF A

20.3

18.8s

a parameter of renormalized beta distribution

0.2

0.3

7.7s

a from scale-rotated-ellipse

63.4

46.6

2.4min

Trigonometry B

0.3

12.6s

Trigonometry A

0.1

9.7s

Spherical law of cosines

16.6

3.8

37.8s

Simplification of discriminant from scale-rotated-ellipse

41.4

6.0

1.3min

Quadratic roots, wide range

52.7

0.1

15.7s

Quadratic roots, narrow range

28.3

0.3

21.2s

Quadratic roots, medium range

43.8

2.9

23.8s

Quadratic roots, full range

34.2

10.1

16.7s

Midpoint on a great circle

0.8

0.2

22.2s

Logistic function from Lakshay Garg

29.2

0.6

11.5s

Hyperbolic tangent

58.2

0.0

3.4s

Hyperbolic sine

58.1

0.0

5.1s

Hyperbolic secant

0.0

3.0s

Hyperbolic arc-cosine

32.5

0.3

3.3s

Hyperbolic arc-(co)tangent

58.6

0.0

5.6s

Hyperbolic arc-(co)secant

0.1

6.8s

Example from Robby

0.1

18.4s

Example 2 from Robby

0.1

19.8s

Equirectangular approximation to distance on a great circle

39.0

3.8

18.0s

Eccentricity of an ellipse

14.5

0.0

4.0s

Distance on a great circle

23.7

13.6

2.0min

Destination given bearing on a great circle

0.2

32.2s

Cubic critical, wide range

52.8

0.1

15.6s

Cubic critical, narrow range

28.4

0.3

21.2s

Cubic critical, medium range

43.9

0.2

18.4s

Cubic critical

34.2

6.7

26.6s

Bearing on a great circle

13.1

0.2

51.3s

Asymptote C

29.6

0.0

8.9s

Asymptote B

0.0

5.6s

Asymptote A

14.5

0.4

3.8s

ABCF->ab-angle b

52.5

38.9

55.3s

ABCF->ab-angle angle

30.0

11.1

17.4s

ABCF->ab-angle a

52.6

33.3

54.6s

2-ancestry mixing, zero discriminant

15.8

0.8

6.9s

2-ancestry mixing, positive discriminant

35.6

2.8

12.3s

2-ancestry mixing, negative discriminant

1.0

0.1

10.2s

Quotient of sum of exps

0.8

1.2

0.0

7.8s

Quotient of products

11.5

2.6

10.7

4.3s

Logistic regression 2

0.5

0.1

9.5s

Hyperbolic arcsine

53.0

0.2

45.3

6.9s

Harley's example

2.5min

Exp of sum of logs

5.7

1.5s

(- (/ x0 (- 1 x1)) x0)

0.0s