Metrics for Herbie run

Date:	Saturday, May 25th, 2024
Commit:	445b0ec4 on explanations-trifurcate
Hostname:	nightly with Racket 8.11.1
Seed:	2024146
Parameters:	256 points for 4 iterations
Flags:	`localize:costslocalize:errorsreduce:regimesreduce:avg-errorreduce:binary-searchreduce:branch-expressionssetup:simplifysetup:searchrules:arithmeticrules:polynomialsrules:fractionsrules:exponentsrules:trigonometryrules:hyperbolicrules:numericsrules:specialrules:boolsrules:branchesgenerate:rrgenerate:taylorgenerate:simplifygenerate:proofs` default

sample2.7min (46.6%)

Results

40.4s	55334×	2	valid
32.1s	142065×	0	precondition
29.2s	116287×	1	valid
26.5s	7065×	5	exit
6.6s	56835×	0	valid
3.8s	2712×	3	valid
554.0ms	4108×	0	unsamplable
517.0ms	4110×	0	invalid

Precisions

Click to see histograms. Total time spent on operations: 2.0min

ival-pow: 25.6s (21.8% of total)

ival-exp: 12.4s (10.5% of total)

ival-tan: 12.4s (10.5% of total)

ival-mult: 10.2s (8.7% of total)

ival-sub: 7.4s (6.3% of total)

...c/correct-round.rkt:121:19: 6.5s (5.5% of total)

backward-pass: 5.8s (4.9% of total)

ival-add: 5.5s (4.6% of total)

ival-log: 5.3s (4.5% of total)

ival-div: 4.5s (3.8% of total)

ival-sin: 3.9s (3.3% of total)

ival-<=: 3.9s (3.3% of total)

ival-cos: 3.9s (3.3% of total)

ival-fabs: 3.1s (2.6% of total)

ival-<: 1.9s (1.6% of total)

ival-sqrt: 1.4s (1.2% of total)

ival->: 922.0ms (0.8% of total)

ival-and: 735.0ms (0.6% of total)

const: 641.0ms (0.5% of total)

ival-fmin: 498.0ms (0.4% of total)

ival-neg: 467.0ms (0.4% of total)

ival-atan: 390.0ms (0.3% of total)

ival-cbrt: 311.0ms (0.3% of total)

Bogosity

simplify49.4s (14.2%)

Algorithm

241×	egg-herbie

Rules

85139×	`fma-define`
77767×	`fma-neg`
50439×	`distribute-lft-in`
44889×	`times-frac`
44605×	`distribute-rgt-in`

Stop Event

75×	saturated
164×	node limit
2×	unsound

Counts

57539 → 59416

localize27.2s (7.8%)

Results

7.0s	8045×	1	valid
6.6s	4850×	2	valid
4.8s	12088×	0	valid
1.8s	389×	5	exit
1.1s	483×	3	valid
495.0ms	1501×	0	invalid
32.0ms	15×	2	invalid
21.0ms	20×	1	invalid
7.0ms	1×	4	valid

Compiler

Compiled 17913 to 2585 computations (85.6% saved)

Precisions

Click to see histograms. Total time spent on operations: 14.0s

ival-pow: 2.2s (15.4% of total)

backward-pass: 2.1s (14.7% of total)

ival-mult: 1.8s (13% of total)

ival-div: 1.4s (10.2% of total)

ival-add: 1.2s (8.4% of total)

ival-tan: 772.0ms (5.5% of total)

ival-log: 695.0ms (5% of total)

...c/correct-round.rkt:121:19: 626.0ms (4.5% of total)

ival-sub: 596.0ms (4.2% of total)

ival-cos: 555.0ms (4% of total)

ival-fma: 335.0ms (2.4% of total)

ival-sin: 331.0ms (2.4% of total)

ival-exp: 290.0ms (2.1% of total)

ival-sqrt: 224.0ms (1.6% of total)

ival-cbrt: 213.0ms (1.5% of total)

ival-log1p: 205.0ms (1.5% of total)

ival-neg: 109.0ms (0.8% of total)

const: 107.0ms (0.8% of total)

ival-cosh: 101.0ms (0.7% of total)

ival-hypot: 92.0ms (0.7% of total)

ival-expm1: 66.0ms (0.5% of total)

ival-atan: 38.0ms (0.3% of total)

ival-atan2: 21.0ms (0.1% of total)

ival-e: 6.0ms (0% of total)

soundness24.6s (7.1%)

Rules

30174×	`log1p-expm1-u`
29123×	`fma-define`
25263×	`fma-neg`
22123×	`expm1-log1p-u`
18692×	`associate-r`

Stop Event

9×	saturated
105×	node limit
23×	fuel
1×	unsound
5×	done

Compiler

Compiled 18310 to 5684 computations (69% saved)

rewrite24.5s (7.1%)

Algorithm

106×	batch-egg-rewrite

Rules

65083×	`log1p-expm1-u`
49523×	`expm1-log1p-u`
23106×	`pow1`
22421×	`add-exp-log`
22042×	`add-log-exp`

Stop Event

106×	node limit

Counts

1259 → 38834

prune16.6s (4.8%)

Counts: 80876 → 2237
Compiler: Compiled 72917 to 41691 computations (42.8% saved)

eval14.8s (4.3%)

Compiler: Compiled 1899312 to 304598 computations (84% saved)

explain7.7s (2.2%)

Explanations

Click to see full explanations table

Operator	Subexpression	Explanation	Count
`-.f64`	`#f`	cancellation	4240	35
`log.f64`	`#f`	sensitivity	761	2
`/.f64`	`#f`	u/u	432	0
`sqrt.f64`	`#f`	oflow-rescue	401	0
`+.f64`	`#f`	cancellation	303	0
`/.f64`	`#f`	u/n	114	0
`pow.f64`	`(pow.f64 (+.f64 x #s(literal 1 binary64)) (/.f64 #s(literal 1 binary64) n))`	sensitivity	58	3
`/.f64`	`(/.f64 (.f64 eps (-.f64 (exp.f64 (.f64 (+.f64 a b) eps)) #s(literal 1 binary64))) (.f64 (-.f64 (exp.f64 (.f64 a eps)) #s(literal 1 binary64)) (-.f64 (exp.f64 (*.f64 b eps)) #s(literal 1 binary64))))`	n/u	23	0
↳	`(*.f64 b eps)`	underflow	137
↳	`(-.f64 (exp.f64 (*.f64 a eps)) #s(literal 1 binary64))`	underflow	131
↳	`(-.f64 (exp.f64 (*.f64 b eps)) #s(literal 1 binary64))`	underflow	137
↳	`(.f64 (-.f64 (exp.f64 (.f64 a eps)) #s(literal 1 binary64)) (-.f64 (exp.f64 (*.f64 b eps)) #s(literal 1 binary64)))`	underflow	229
↳	`(*.f64 a eps)`	underflow	131
`-.f64`	`#f`	nan-rescue	13	0
`sqrt.f64`	`#f`	uflow-rescue	12	0
`cos.f64`	`(cos.f64 (+.f64 x eps))`	sensitivity	2	0
`/.f64`	`(/.f64 (-.f64 (exp.f64 (*.f64 #s(literal 2 binary64) x)) #s(literal 1 binary64)) (-.f64 (exp.f64 x) #s(literal 1 binary64)))`	o/o	2	0
↳	`(exp.f64 (*.f64 #s(literal 2 binary64) x))`	overflow	2
↳	`(-.f64 (exp.f64 (*.f64 #s(literal 2 binary64) x)) #s(literal 1 binary64))`	overflow	2
↳	`(exp.f64 x)`	overflow	2
↳	`(-.f64 (exp.f64 x) #s(literal 1 binary64))`	overflow	2
`sin.f64`	`(sin.f64 (+.f64 x eps))`	sensitivity	2	0
`tan.f64`	`(tan.f64 (+.f64 x eps))`	sensitivity	2	0
`/.f64`	`(/.f64 (-.f64 #s(literal 1 binary64) (cos.f64 x)) (*.f64 x x))`	n/o	1	0
↳	`(*.f64 x x)`	overflow	70
`exp.f64`	`(exp.f64 (*.f64 a x))`	sensitivity	1	0

Confusion

	Predicted +	Predicted -
+	5204	21
-	138	1805

Precision

0.974166978659678

Recall

0.9959808612440192

Confusion?

	Predicted +	Predicted Maybe	Predicted -
+	5204	14	7
-	138	23	1782

Precision?

0.9700687860197063

Recall?

0.9986602870813397

Freqs

test

number	freq
0	1826
1	4538
2	644
3	99
4	61

Total Confusion?

	Predicted +	Predicted Maybe	Predicted -
+	28	0	0
-	0	0	0

Precision?

1.0

Recall?

1.0

Results

2.7s	3394×	2	valid
1.7s	7272×	1	valid
352.0ms	3490×	0	valid
205.0ms	180×	3	valid

Compiler

Compiled 2552 to 924 computations (63.8% saved)

Precisions

Click to see histograms. Total time spent on operations: 3.7s

ival-exp: 664.0ms (17.9% of total)

ival-tan: 656.0ms (17.7% of total)

ival-log: 375.0ms (10.1% of total)

ival-cos: 319.0ms (8.6% of total)

backward-pass: 307.0ms (8.3% of total)

ival-sub: 274.0ms (7.4% of total)

ival-sin: 250.0ms (6.7% of total)

ival-div: 238.0ms (6.4% of total)

ival-add: 169.0ms (4.6% of total)

ival-mult: 151.0ms (4.1% of total)

ival-pow: 115.0ms (3.1% of total)

ival-sqrt: 77.0ms (2.1% of total)

ival-cbrt: 33.0ms (0.9% of total)

const: 32.0ms (0.9% of total)

ival-neg: 23.0ms (0.6% of total)

ival-atan: 23.0ms (0.6% of total)

series5.6s (1.6%)

Counts

1259 → 18705

Calls

5097 calls:

Time	Variable		Point	Expression
109.0ms	eps	@	0	(log (/ (- 1 eps) (+ 1 eps)))
101.0ms	eps	@	-inf	(log (- (tan (+ x eps)) (tan x)))
89.0ms	n	@	0	(* n (log n))
78.0ms	x	@	inf	(log (- (tan (+ x eps)) (tan x)))
57.0ms	x	@	-inf	(log (- (tan (+ x eps)) (tan x)))

preprocess4.8s (1.4%)

Algorithm

28×	egg-herbie

Rules

8308×	`fma-neg`
6663×	`fma-define`
4279×	`div-sub`
2669×	`sub-neg`
2621×	`times-frac`

Stop Event

23×	saturated
5×	node limit

Compiler

Compiled 9480 to 3736 computations (60.6% saved)

analyze4.1s (1.2%)

Algorithm

28×

Search

Probability	Valid	Unknown	Precondition	Infinite	Domain	Can't	Iter
0%	0%	61.1%	38.9%	0%	0%	0%	0
6.1%	3.7%	57.4%	38.9%	0%	0%	0%	1
16.5%	10.1%	51%	38.9%	0%	0%	0%	2
35.3%	20.6%	37.8%	41.6%	0%	0%	0%	3
46.1%	26.9%	31.4%	41.7%	0%	0%	0%	4
54.8%	31.5%	26%	42.4%	0%	0%	0%	5
57.1%	32.5%	24.4%	43.1%	0%	0%	0%	6
61.5%	33.9%	21.3%	44%	0%	0.8%	0.1%	7
64.6%	35.3%	19.3%	44.5%	0%	0.8%	0.1%	8
67.1%	36%	17.6%	44.9%	0%	1.2%	0.3%	9
69.1%	36.5%	16.3%	45.2%	0%	1.7%	0.3%	10
71.7%	37.5%	14.8%	45.4%	0%	1.9%	0.4%	11
72.8%	37.8%	14.1%	45.6%	0%	2.2%	0.4%	12

Compiler

Compiled 498 to 322 computations (35.3% saved)

Precisions

Click to see histograms. Total time spent on operations: 2.4s

ival-fabs: 479.0ms (19.7% of total)

ival-fmin: 222.0ms (9.1% of total)

ival-sub: 199.0ms (8.2% of total)

ival-mult: 198.0ms (8.2% of total)

ival-pow: 181.0ms (7.5% of total)

ival-log: 162.0ms (6.7% of total)

...c/correct-round.rkt:121:19: 158.0ms (6.5% of total)

ival-add: 150.0ms (6.2% of total)

ival-div: 101.0ms (4.2% of total)

ival-<=: 86.0ms (3.5% of total)

ival-neg: 83.0ms (3.4% of total)

ival-sqrt: 73.0ms (3% of total)

ival-sin: 73.0ms (3% of total)

ival-exp: 71.0ms (2.9% of total)

ival-tan: 65.0ms (2.7% of total)

ival-cos: 62.0ms (2.6% of total)

const: 21.0ms (0.9% of total)

ival->: 14.0ms (0.6% of total)

ival-and: 13.0ms (0.5% of total)

ival-<: 11.0ms (0.5% of total)

backward-pass: 7.0ms (0.3% of total)

ival-atan: 0.0ms (0% of total)

ival-cbrt: 0.0ms (0% of total)

regimes3.6s (1%)

Counts

4271 → 313

Calls

43 calls:

923.0ms	x
303.0ms	a
217.0ms	(/.f64 #s(literal 1 binary64) n)
201.0ms	b_2
183.0ms	b

Compiler

Compiled 1474 to 985 computations (33.2% saved)

bsearch2.5s (0.7%)

Algorithm

84×	binary-search
31×	left-value

Stop Event

6×	predicate-same
78×	narrow-enough

Results

841.0ms	7574×	0	valid
498.0ms	626×	2	valid
397.0ms	1928×	1	valid
139.0ms	1026×	0	invalid
8.0ms	61×	0	unsamplable

Compiler

Compiled 15167 to 11111 computations (26.7% saved)

Precisions

Click to see histograms. Total time spent on operations: 1.5s

ival-mult: 314.0ms (21.4% of total)

ival-sub: 254.0ms (17.3% of total)

ival-pow: 215.0ms (14.7% of total)

ival-div: 167.0ms (11.4% of total)

ival-add: 142.0ms (9.7% of total)

ival-sqrt: 124.0ms (8.5% of total)

ival-exp: 80.0ms (5.5% of total)

ival-neg: 59.0ms (4% of total)

backward-pass: 51.0ms (3.5% of total)

ival-cos: 22.0ms (1.5% of total)

const: 18.0ms (1.2% of total)

ival-cbrt: 11.0ms (0.8% of total)

ival-sin: 7.0ms (0.5% of total)

ival-log: 1.0ms (0.1% of total)

Herbie run

sample2.7min (46.6%)

simplify49.4s (14.2%)

localize27.2s (7.8%)

soundness24.6s (7.1%)

rewrite24.5s (7.1%)

prune16.6s (4.8%)

eval14.8s (4.3%)

explain7.7s (2.2%)

series5.6s (1.6%)

preprocess4.8s (1.4%)

analyze4.1s (1.2%)

regimes3.6s (1%)

bsearch2.5s (0.7%)

end0.0ms (0%)

Profiling