Herbie results

Output vs Input Accuracy

Each point represents a Herbie run below. Its horizontal position shows initial accuracy, and vertical position shows final accuracy. Points above the line are improved by Herbie.

Accuracy vs Cost

A joint cost-accuracy pareto curve for the Herbie runs below. Accuracy is on the vertical axis, and cost is on the vertical axis. Down and to the left is better. The initial programs are shown by the red square.

Test	Start	Result ?	Target ?	Time
powComplex, real part	33.3	5.6		28.1s	»
powComplex, imaginary part	33.4	17.0		51.2s	»
normal distribution	0.4	0.3		19.7s	»
math.square on complex, real part	0.0	0.0		1.2s	»
math.square on complex, imaginary part	0.0	0.0		1.8s	»
math.sqrt on complex, imaginary part, im greater than 0 branch	38.7	14.5		20.3s	»
math.sin on complex, real part	0.0	0.0		25.9s	»
math.log10 on complex, real part	32.1	8.1		32.1s	»
math.log10 on complex, imaginary part	0.8	0.1		23.6s	»
math.log/2 on complex, real part	32.1	8.2		19.5s	»
math.log/2 on complex, imaginary part	32.0	0.3		17.8s	»
math.log/1 on complex, real part	31.8	7.7		2.7s	»
math.log/1 on complex, imaginary part	0	0		1.1s	»
math.exp on complex, real part	0.0	0.0		7.2s	»
math.exp on complex, imaginary part	0.0	0.0		8.8s	»
math.cos on complex, real part	0.0	0.1		13.0s	»
math.arg on complex	0	0		0.9s	»
math.abs on complex (squared)	0.0	0.0		1.2s	»
math.abs on complex	31.7	8.0		4.5s	»
_multiplyComplex, real part	0.0	0.0		6.4s	»
_multiplyComplex, imaginary part	0.0	0.0		5.9s	»
_divideComplex, real part	26.3	14.9		25.2s	»
_divideComplex, imaginary part	25.4	14.7		24.5s	»
Rust f64::atanh	0.0	0.0		2.3s	»
Rust f32::atanh	0.1	0.1		12.8s	»
Octave 3.8, oct_fill_randg	0.1	0.1		20.0s	»
Octave 3.8, jcobi/4, as called	46.9	0.0		4.3s	»
Octave 3.8, jcobi/4	53.8	9.3		40.7s	»
Octave 3.8, jcobi/3	3.4	0.3		32.7s	»
Octave 3.8, jcobi/2	24.6	1.5		31.0s	»
Octave 3.8, jcobi/1	16.4	0.5		14.3s	»
Jmat.Real.lambertw, estimator	0.3	0.3		7.9s	»
Jmat.Real.gamma, branch z less than 0.5				2.5min	»
Jmat.Real.gamma, branch z greater than 0.5	3.8	2.1		36.1s	»
Jmat.Real.erfi, branch x less than or equal to 0.5	0.2	0.2		11.7s	»
Jmat.Real.erfi, branch x greater than or equal to 5	2.8	2.6		25.5s	»
Jmat.Real.erf	13.4	13.4		18.7s	»
Jmat.Real.dawson	29.3	0.0		18.2s	»
math.sqrt on complex, real part	38.6	31.7	33.3	9.8s	»
math.sin on complex, imaginary part	58.0	0.6	0.2	16.0s	»
math.cube on real	0.1	0.0	0.0	6.5s	»
math.cube on complex, real part	7.3	0.3	0.3	13.3s	»
math.cube on complex, imaginary part	7.8	0.3	0.2	10.6s	»
math.cos on complex, imaginary part	43.0	0.7	0.3	22.7s	»
Rust f64::asinh	45.8	29.6	0.0	6.9s	»
Rust f64::acosh	31.4	0.4	0.1	7.1s	»
Rust f32::asinh	20.3	12.4	0.1	12.5s	»
Rust f32::acosh	15.8	0.4	0.3	10.1s	»
Jmat.Real.lambertw, newton loop step	13.7	1.7	13.0	36.3s	»
Ian Simplification	59.7	58.8	0	25.5s	»
FastMath test5	0.1	0	0	33.0s	»
FastMath test3	0.1	0.1	0.1	7.4s	»
FastMath test2	0.2	0.0	0.0	3.1s	»
FastMath test1	0.3	0	0	2.3s	»
FastMath repmul	0.1	0.0	0.0	17.4s	»
FastMath dist4	0.0	0.0	0.0	18.2s	»
FastMath dist3	0.0	0.0	0.0	14.1s	»
FastMath dist	0.0	0.0	0.0	2.8s	»

Test

Start

Result ?

Target ?

Time

powComplex, real part

33.3

5.6

28.1s

powComplex, imaginary part

33.4

17.0

51.2s

normal distribution

0.4

0.3

19.7s

math.square on complex, real part

0.0

1.2s

math.square on complex, imaginary part

0.0

1.8s

math.sqrt on complex, imaginary part, im greater than 0 branch

38.7

14.5

20.3s

math.sin on complex, real part

0.0

25.9s

math.log10 on complex, real part

32.1

8.1

32.1s

math.log10 on complex, imaginary part

0.8

0.1

23.6s

math.log/2 on complex, real part

32.1

8.2

19.5s

math.log/2 on complex, imaginary part

32.0

0.3

17.8s

math.log/1 on complex, real part

31.8

7.7

2.7s

math.log/1 on complex, imaginary part

1.1s

math.exp on complex, real part

0.0

7.2s

math.exp on complex, imaginary part

0.0

8.8s

math.cos on complex, real part

0.0

0.1

13.0s

math.arg on complex

0.9s

math.abs on complex (squared)

0.0

1.2s

math.abs on complex

31.7

8.0

4.5s

_multiplyComplex, real part

0.0

6.4s

_multiplyComplex, imaginary part

0.0

5.9s

_divideComplex, real part

26.3

14.9

25.2s

_divideComplex, imaginary part

25.4

14.7

24.5s

Rust f64::atanh

0.0

2.3s

Rust f32::atanh

0.1

12.8s

Octave 3.8, oct_fill_randg

0.1

20.0s

Octave 3.8, jcobi/4, as called

46.9

0.0

4.3s

Octave 3.8, jcobi/4

53.8

9.3

40.7s

Octave 3.8, jcobi/3

3.4

0.3

32.7s

Octave 3.8, jcobi/2

24.6

1.5

31.0s

Octave 3.8, jcobi/1

16.4

0.5

14.3s

Jmat.Real.lambertw, estimator

0.3

7.9s

Jmat.Real.gamma, branch z less than 0.5

2.5min

Jmat.Real.gamma, branch z greater than 0.5

3.8

2.1

36.1s

Jmat.Real.erfi, branch x less than or equal to 0.5

0.2

11.7s

Jmat.Real.erfi, branch x greater than or equal to 5

2.8

2.6

25.5s

Jmat.Real.erf

13.4

18.7s

Jmat.Real.dawson

29.3

0.0

18.2s

math.sqrt on complex, real part

38.6

31.7

33.3

9.8s

math.sin on complex, imaginary part

58.0

0.6

0.2

16.0s

math.cube on real

0.1

0.0

6.5s

math.cube on complex, real part

7.3

0.3

13.3s

math.cube on complex, imaginary part

7.8

0.3

0.2

10.6s

math.cos on complex, imaginary part

43.0

0.7

0.3

22.7s

Rust f64::asinh

45.8

29.6

0.0

6.9s

Rust f64::acosh

31.4

0.4

0.1

7.1s

Rust f32::asinh

20.3

12.4

0.1

12.5s

Rust f32::acosh

15.8

0.4

0.3

10.1s

Jmat.Real.lambertw, newton loop step

13.7

1.7

13.0

36.3s

Ian Simplification

59.7

58.8

25.5s

FastMath test5

0.1

33.0s

FastMath test3

0.1

7.4s

FastMath test2

0.2

0.0

3.1s

FastMath test1

0.3

2.3s

FastMath repmul

0.1

0.0

17.4s

FastMath dist4

0.0

18.2s

FastMath dist3

0.0

14.1s

FastMath dist

0.0

2.8s