Metrics for Herbie run

Date:	Wednesday, March 20th, 2024
Commit:	23dc2d0f on exact-hallucinations
Hostname:	nightly with Racket 8.11.1
Seed:	2024080
Parameters:	256 points for 4 iterations
Flags:	`reduce:regimesreduce:avg-errorreduce:binary-searchreduce:branch-expressionssetup:simplifysetup:searchrules:arithmeticrules:polynomialsrules:fractionsrules:exponentsrules:trigonometryrules:hyperbolicrules:numericsrules:specialrules:boolsrules:branchesgenerate:rrgenerate:taylorgenerate:simplifygenerate:proofs` default

sample3.0min (45.5%)

Results

32.3s	142266×	256	precondition
14.8s	90436×	256	valid
29.9s	57747×	1024	valid
9.7s	34509×	512	valid
42.3s	33844×	2048	valid
34.0s	7130×	8192	exit
847.0ms	4226×	256	infinite
641.0ms	4218×	256	unsamplable
526.0ms	4046×	256	invalid
1.1s	1849×	4096	valid
96.0ms	173×	512	infinite
147.0ms	128×	1024	infinite

Precisions

Click to see histograms. Total time spent on operations: 2.0min

Operation ival-pow, time spent: 32.3s, 26.0% of total-time

Operation ival-tan, time spent: 13.9s, 11.0% of total-time

Operation ival-exp, time spent: 12.6s, 10.0% of total-time

Operation ival-mult, time spent: 10.4s, 8.0% of total-time

Operation ival-sub, time spent: 7.5s, 6.0% of total-time

Operation ival-div, time spent: 6.2s, 5.0% of total-time

Operation ival-log, time spent: 6.1s, 5.0% of total-time

Operation ival-cos, time spent: 5.0s, 4.0% of total-time

Operation ival-<=, time spent: 4.9s, 4.0% of total-time

Operation ival-add, time spent: 4.9s, 4.0% of total-time

Operation ival-sin, time spent: 4.3s, 4.0% of total-time

Operation ival-fabs, time spent: 3.8s, 3.0% of total-time

Operation ival-<, time spent: 2.4s, 2.0% of total-time

Operation ival-sqrt, time spent: 1.9s, 2.0% of total-time

Operation const, time spent: 1.6s, 1.0% of total-time

Operation ival-and, time spent: 1.0s, 1.0% of total-time

Operation ival-fmin, time spent: 921.0ms, 1.0% of total-time

Operation ival->, time spent: 857.0ms, 1.0% of total-time

Operation ival-atan, time spent: 649.0ms, 1.0% of total-time

Operation ival-neg, time spent: 522.0ms, 0.0% of total-time

Operation ival-cbrt, time spent: 326.0ms, 0.0% of total-time

Bogosity

localize1.2min (18.1%)

Compiler: Compiled 17277 to 10339 computations (40.2% saved)
Precisions: Click to see histograms. Total time spent on operations: 53.1s
Operation ival-pow, time spent: 14.2s, 27.0% of total-time
Operation ival-mult, time spent: 6.9s, 13.0% of total-time
Operation ival-div, time spent: 6.1s, 11.0% of total-time
Operation ival-add, time spent: 5.0s, 9.0% of total-time
Operation composed, time spent: 4.8s, 9.0% of total-time
Operation ival-log, time spent: 3.3s, 6.0% of total-time
Operation ival-exp, time spent: 2.6s, 5.0% of total-time
Operation ival-sub, time spent: 2.2s, 4.0% of total-time
Operation ival-sin, time spent: 2.0s, 4.0% of total-time
Operation ival-tan, time spent: 1.6s, 3.0% of total-time
Operation const, time spent: 1.2s, 2.0% of total-time
Operation ival-sqrt, time spent: 1.0s, 2.0% of total-time
Operation ival-cbrt, time spent: 827.0ms, 2.0% of total-time
Operation ival-cos, time spent: 741.0ms, 1.0% of total-time
Operation ival-neg, time spent: 386.0ms, 1.0% of total-time
Operation ival-cosh, time spent: 88.0ms, 0.0% of total-time
Operation ival-atan2, time spent: 51.0ms, 0.0% of total-time
Operation ival-atan, time spent: 19.0ms, 0.0% of total-time

eval32.4s (8.1%)

Compiler: Compiled 1462798 to 963035 computations (34.2% saved)

simplify25.0s (6.2%)

Algorithm

122×	egg-herbie

Rules

59986×	`fma-define`
55374×	`fma-neg`
52034×	`distribute-lft-in`
47856×	`*-commutative`
47852×	`distribute-rgt-in`

Stop Event

28×	saturated
92×	node limit
20×	fuel
2×	unsound
7×	done

Counts

38011 → 40621

Compiler

Compiled 3039 to 2159 computations (29% saved)

soundness22.7s (5.6%)

Rules

58800×	`fma-define`
48330×	`fma-neg`
35420×	`associate-r`
31840×	`distribute-lft-in`
31088×	`distribute-rgt-in`

Stop Event

7×	saturated
102×	node limit
1×	unsound

Compiler

Compiled 16798 to 10953 computations (34.8% saved)

rewrite19.5s (4.9%)

Algorithm

95×	batch-egg-rewrite

Rules

60423×	`log1p-expm1-u`
47998×	`expm1-log1p-u`
35738×	`prod-diff`
29630×	`fma-define`
28324×	`log-prod`

Stop Event

95×	node limit

Counts

731 → 24998

preprocess15.2s (3.8%)

Algorithm

27×	egg-herbie

Rules

16450×	`fma-neg`
12928×	`fma-define`
8464×	`div-sub`
5288×	`sub-neg`
5148×	`times-frac`

Explanations

Click to see full explanations table

Operator	Subexpression	Explanation	Count
`cos.f64`	`(cos.f64 (+.f64 x eps))`	sensitivity	1	0
`sqrt.f64`	`#f`	uflow-rescue	16	0
`/.f64`	`(/.f64 (-.f64 1 (cos.f64 x)) (*.f64 x x))`	n/o	6	0
↳	`(*.f64 x x)`	overflow	68
`pow.f64`	`(pow.f64 (+.f64 x 1) (/.f64 1 n))`	sensitivity	64	1
`exp.f64`	`#f`	sensitivity	3	2
`sqrt.f64`	`#f`	oflow-rescue	428	0
`/.f64`	`(/.f64 (.f64 eps (-.f64 (exp.f64 (.f64 (+.f64 a b) eps)) 1)) (.f64 (-.f64 (exp.f64 (.f64 a eps)) 1) (-.f64 (exp.f64 (*.f64 b eps)) 1)))`	n/u	33	0
↳	`(-.f64 (exp.f64 (*.f64 b eps)) 1)`	underflow	150
↳	`(*.f64 b eps)`	underflow	150
↳	`(-.f64 (exp.f64 (*.f64 a eps)) 1)`	underflow	150
↳	`(.f64 (-.f64 (exp.f64 (.f64 a eps)) 1) (-.f64 (exp.f64 (*.f64 b eps)) 1))`	underflow	240
↳	`(*.f64 a eps)`	underflow	150
`log.f64`	`#f`	sensitivity	765	0
`/.f64`	`(/.f64 (-.f64 (exp.f64 (*.f64 2 x)) 1) (-.f64 (exp.f64 x) 1))`	o/o	1	0
↳	`(-.f64 (exp.f64 (*.f64 2 x)) 1)`	overflow	1
↳	`(exp.f64 (*.f64 2 x))`	overflow	1
↳	`(exp.f64 x)`	overflow	1
↳	`(-.f64 (exp.f64 x) 1)`	overflow	1
`sin.f64`	`(sin.f64 (+.f64 x eps))`	sensitivity	2	0
`/.f64`	`#f`	u/u	413	0
`-.f64`	`#f`	nan-rescue	16	0
`-.f64`	`#f`	cancellation	4092	33
`+.f64`	`#f`	cancellation	359	0
`tan.f64`	`(tan.f64 (+.f64 x eps))`	sensitivity	2	0
`/.f64`	`#f`	u/n	48	0

Confusion

	Predicted +	Predicted -
+	5083	7
-	140	1682

Precision

0.9731954815240283

Recall

0.9986247544204322

Confusion?

	Predicted +	Predicted Maybe	Predicted -
+	5083	7	0
-	140	26	1656

Precision?

0.9684170471841704

Recall?

1.0

Stop Event

22×	saturated
5×	node limit

Compiler

Compiled 11118 to 6898 computations (38% saved)

Precisions

Click to see histograms. Total time spent on operations: 5.1s

Operation ival-tan, time spent: 849.0ms, 17.0% of total-time

Operation ival-log, time spent: 684.0ms, 14.0% of total-time

Operation ival-exp, time spent: 626.0ms, 12.0% of total-time

Operation ival-pow, time spent: 546.0ms, 11.0% of total-time

Operation ival-sub, time spent: 465.0ms, 9.0% of total-time

Operation ival-mult, time spent: 370.0ms, 7.0% of total-time

Operation ival-div, time spent: 360.0ms, 7.0% of total-time

Operation ival-add, time spent: 357.0ms, 7.0% of total-time

Operation ival-cos, time spent: 233.0ms, 5.0% of total-time

Operation ival-sqrt, time spent: 175.0ms, 3.0% of total-time

Operation ival-sin, time spent: 169.0ms, 3.0% of total-time

Operation const, time spent: 128.0ms, 3.0% of total-time

Operation ival-atan, time spent: 42.0ms, 1.0% of total-time

Operation ival-neg, time spent: 28.0ms, 1.0% of total-time

Operation ival-cbrt, time spent: 28.0ms, 1.0% of total-time

prune12.8s (3.2%)

Counts: 55720 → 1903
Compiler: Compiled 65078 to 46486 computations (28.6% saved)

regimes7.5s (1.9%)

Counts

3021 → 257

Calls

42 calls:

1.8s	x
845.0ms	b
791.0ms	(/.f64 1 n)
773.0ms	a
760.0ms	c

Compiler

Compiled 1486 to 1040 computations (30% saved)

series4.9s (1.2%)

Counts

731 → 13013

Calls

3450 calls:

Time	Variable		Point	Expression
174.0ms	c	@	-inf	(/ (- (neg b_2) (sqrt (- (* b_2 b_2) (* a c)))) a)
170.0ms	x	@	inf	(* x (+ 1/3 (* -1/4 x)))
148.0ms	x	@	inf	(log (- (tan (+ x eps)) (tan x)))
104.0ms	eps	@	-inf	(log (- (tan (+ x eps)) (tan x)))
76.0ms	n	@	0	(/ (log x) n)

analyze3.3s (0.8%)

Algorithm

28×

Search

Probability	Valid	Unknown	Precondition	Infinite	Domain	Can't	Iter
0%	0%	61.1%	38.9%	0%	0%	0%	0
19.3%	11.1%	46.5%	38.9%	0%	3.6%	0%	1
27%	15.5%	42%	38.9%	0%	3.6%	0%	2
41.8%	22.9%	31.9%	41.6%	0%	3.6%	0%	3
50.2%	27.5%	27.3%	41.7%	0%	3.6%	0%	4
57.3%	30.9%	23.1%	42.4%	0%	3.6%	0%	5
59.5%	31.7%	21.6%	43.1%	0%	3.6%	0%	6
63.4%	32.7%	18.9%	44%	0%	4.3%	0.1%	7
66.6%	34%	17.1%	44.5%	0%	4.4%	0.1%	8
69.1%	34.6%	15.5%	44.9%	0%	4.8%	0.3%	9
71.2%	35.1%	14.1%	45.2%	0%	5.3%	0.3%	10
74%	36%	12.7%	45.4%	0%	5.5%	0.4%	11
75.2%	36.3%	11.9%	45.6%	0%	5.8%	0.4%	12

Compiler

Compiled 505 to 355 computations (29.7% saved)

Precisions

Click to see histograms. Total time spent on operations: 1.8s

Operation ival-mult, time spent: 206.0ms, 11.0% of total-time

Operation ival-sub, time spent: 196.0ms, 11.0% of total-time

Operation ival-pow, time spent: 194.0ms, 11.0% of total-time

Operation ival-tan, time spent: 183.0ms, 10.0% of total-time

Operation ival-log, time spent: 165.0ms, 9.0% of total-time

Operation ival-div, time spent: 134.0ms, 7.0% of total-time

Operation ival-sin, time spent: 126.0ms, 7.0% of total-time

Operation ival-<=, time spent: 111.0ms, 6.0% of total-time

Operation ival-neg, time spent: 111.0ms, 6.0% of total-time

Operation ival-exp, time spent: 92.0ms, 5.0% of total-time

Operation ival-add, time spent: 66.0ms, 4.0% of total-time

Operation ival-sqrt, time spent: 60.0ms, 3.0% of total-time

Operation ival-fabs, time spent: 50.0ms, 3.0% of total-time

Operation const, time spent: 44.0ms, 2.0% of total-time

Operation ival-and, time spent: 14.0ms, 1.0% of total-time

Operation ival->, time spent: 14.0ms, 1.0% of total-time

Operation ival-cos, time spent: 12.0ms, 1.0% of total-time

Operation ival-<, time spent: 11.0ms, 1.0% of total-time

Operation ival-fmin, time spent: 7.0ms, 0.0% of total-time

Operation ival-atan, time spent: 0.0ms, 0.0% of total-time

Operation ...c/syntax/syntax.rkt:185:8, time spent: 0.0ms, 0.0% of total-time

Operation ival-cbrt, time spent: 0.0ms, 0.0% of total-time

bsearch2.8s (0.7%)

Algorithm

69×	binary-search
26×	left-value

Stop Event

3×	predicate-same
66×	narrow-enough

Results

936.0ms	5573×	256	valid
239.0ms	1610×	256	invalid
667.0ms	1246×	1024	valid
257.0ms	628×	512	valid
146.0ms	558×	256	infinite
183.0ms	229×	2048	valid
55.0ms	57×	1024	infinite
16.0ms	29×	512	infinite
5.0ms	26×	256	unsamplable

Compiler

Compiled 14315 to 10494 computations (26.7% saved)

Precisions

Click to see histograms. Total time spent on operations: 1.5s

Operation ival-pow, time spent: 664.0ms, 45.0% of total-time

Operation ival-mult, time spent: 251.0ms, 17.0% of total-time

Operation ival-div, time spent: 150.0ms, 10.0% of total-time

Operation ival-sub, time spent: 107.0ms, 7.0% of total-time

Operation ival-sqrt, time spent: 79.0ms, 5.0% of total-time

Operation ival-add, time spent: 56.0ms, 4.0% of total-time

Operation ival-exp, time spent: 55.0ms, 4.0% of total-time

Operation ival-neg, time spent: 50.0ms, 3.0% of total-time

Operation const, time spent: 28.0ms, 2.0% of total-time

Operation ival-cos, time spent: 17.0ms, 1.0% of total-time

Operation ival-cbrt, time spent: 16.0ms, 1.0% of total-time

Herbie run

sample3.0min (45.5%)

localize1.2min (18.1%)

eval32.4s (8.1%)

simplify25.0s (6.2%)

soundness22.7s (5.6%)

rewrite19.5s (4.9%)

preprocess15.2s (3.8%)

prune12.8s (3.2%)

regimes7.5s (1.9%)

series4.9s (1.2%)

analyze3.3s (0.8%)

bsearch2.8s (0.7%)

end4.0ms (0%)

Profiling