Metrics for Herbie run

Date:	Friday, April 5th, 2024
Commit:	0a13acd6 on explanations-eval-freq
Hostname:	nightly with Racket 8.11.1
Seed:	2024096
Parameters:	256 points for 4 iterations
Flags:	`reduce:regimesreduce:avg-errorreduce:binary-searchreduce:branch-expressionssetup:simplifysetup:searchrules:arithmeticrules:polynomialsrules:fractionsrules:exponentsrules:trigonometryrules:hyperbolicrules:numericsrules:specialrules:boolsrules:branchesgenerate:rrgenerate:taylorgenerate:simplifygenerate:proofs` default

sample8.2min (29.7%)

Results

1.4min	298854×	256	valid
41.0s	45526×	512	valid
6.1s	27818×	256	invalid
7.2s	27416×	256	infinite
7.4s	26506×	256	unsamplable
22.4s	25418×	1024	valid
2.7min	16687×	8192	exit
32.9s	10969×	2048	valid
56.2s	7478×	4096	valid
6.0s	5908×	512	infinite
1.5s	1694×	1024	invalid
1.4s	1194×	2048	invalid
866.0ms	1171×	512	invalid
20.2s	1134×	8192	valid
104.0ms	901×	256	precondition
4.4s	687×	2048	infinite
2.2s	537×	1024	infinite
4.5s	404×	4096	infinite
6.0ms	7×	512	unsamplable
8.0ms	4×	1024	unsamplable
6.0ms	1×	2048	unsamplable

Precisions

Click to see histograms. Total time spent on operations: 6.3min

Operation ival-pow, time spent: 2.6min, 41.0% of total-time

Operation ival-sin, time spent: 47.8s, 13.0% of total-time

Operation ival-mult, time spent: 43.2s, 11.0% of total-time

Operation ival-cos, time spent: 35.0s, 9.0% of total-time

Operation ival-div, time spent: 32.7s, 9.0% of total-time

Operation ival-exp, time spent: 11.1s, 3.0% of total-time

Operation ival-sub, time spent: 10.2s, 3.0% of total-time

Operation ival-add, time spent: 10.1s, 3.0% of total-time

Operation ival-log, time spent: 9.8s, 3.0% of total-time

Operation ival-sqrt, time spent: 5.3s, 1.0% of total-time

Operation const, time spent: 3.9s, 1.0% of total-time

Operation ival-<, time spent: 3.7s, 1.0% of total-time

Operation ival-neg, time spent: 3.2s, 1.0% of total-time

Operation ival-atan2, time spent: 1.5s, 0.0% of total-time

Operation ival-tan, time spent: 1.1s, 0.0% of total-time

Operation ival-pi, time spent: 1.1s, 0.0% of total-time

Operation ival-atan, time spent: 1.0s, 0.0% of total-time

Operation ival-acos, time spent: 867.0ms, 0.0% of total-time

Operation ival-and, time spent: 551.0ms, 0.0% of total-time

Operation ival-asin, time spent: 448.0ms, 0.0% of total-time

Operation ival-<=, time spent: 420.0ms, 0.0% of total-time

Operation ival-fabs, time spent: 295.0ms, 0.0% of total-time

Operation ival-cbrt, time spent: 221.0ms, 0.0% of total-time

Operation ival-==, time spent: 2.0ms, 0.0% of total-time

Operation ival-or, time spent: 0.0ms, 0.0% of total-time

Bogosity

localize7.7min (28.1%)

Results

Compiler

Compiled 88448 to 49014 computations (44.6% saved)

Precisions

Click to see histograms. Total time spent on operations: 6.6min

Operation ival-sin, time spent: 2.3min, 35.0% of total-time

Operation ival-mult, time spent: 1.0min, 15.0% of total-time

Operation ival-pow, time spent: 49.0s, 12.0% of total-time

Operation ival-cos, time spent: 44.6s, 11.0% of total-time

Operation ival-div, time spent: 27.5s, 7.0% of total-time

Operation composed, time spent: 25.3s, 6.0% of total-time

Operation ival-log, time spent: 10.7s, 3.0% of total-time

Operation ival-exp, time spent: 9.9s, 2.0% of total-time

Operation ival-add, time spent: 8.7s, 2.0% of total-time

Operation ival-sub, time spent: 6.1s, 2.0% of total-time

Operation ival-sqrt, time spent: 5.4s, 1.0% of total-time

Operation const, time spent: 4.5s, 1.0% of total-time

Operation ival-atan2, time spent: 1.8s, 0.0% of total-time

Operation ival-cbrt, time spent: 1.7s, 0.0% of total-time

Operation ival-atan, time spent: 727.0ms, 0.0% of total-time

Operation ival-neg, time spent: 604.0ms, 0.0% of total-time

Operation ival-tan, time spent: 536.0ms, 0.0% of total-time

Operation ival-acos, time spent: 504.0ms, 0.0% of total-time

Operation ival-pi, time spent: 235.0ms, 0.0% of total-time

Operation ival-fabs, time spent: 105.0ms, 0.0% of total-time

Operation ival-asin, time spent: 51.0ms, 0.0% of total-time

eval2.4min (8.9%)

Compiler: Compiled 7119886 to 4104636 computations (42.3% saved)

preprocess2.1min (7.7%)

Algorithm

51×	egg-herbie

Rules

63000×	`fma-define`
37664×	`fma-neg`
25928×	`div-sub`
22916×	`unsub-neg`
22266×	`sub-neg`

Explanations

Click to see full explanations table

Operator	Subexpression	Explanation	Count
`cos.f64`	`#f`	sensitivity	1934	0
`+.f64`	`#f`	cancellation	1789	32
`sin.f64`	`#f`	sensitivity	1690	4
`sqrt.f64`	`#f`	oflow-rescue	1344	0
`-.f64`	`#f`	cancellation	961	12
`-.f64`	`#f`	nan-rescue	639	0
`log.f64`	`#f`	sensitivity	587	1
`/.f64`	`#f`	o/n	487	0
`/.f64`	`#f`	o/o	347	0
`/.f64`	`#f`	u/u	310	0
`*.f64`	`#f`	*no**	203	0
`exp.f64`	`#f`	sensitivity	171	16
`sqrt.f64`	`#f`	uflow-rescue	148	0
`/.f64`	`#f`	u/n	132	0
`/.f64`	`#f`	n/o	115	0
`*.f64`	`#f`	*nu**	99	0
`cbrt.f64`	`#f`	oflow-rescue	84	0
`cbrt.f64`	`#f`	uflow-rescue	45	0
`/.f64`	`#f`	n/u	36	0
`*.f64`	`#f`	*uo**	27	0
`*.f64`	`#f`	*ou**	27	0
`acos.f64`	`(acos.f64 (+.f64 (.f64 (sin.f64 phi1) (sin.f64 phi2)) (.f64 (*.f64 (cos.f64 phi1) (cos.f64 phi2)) (cos.f64 (-.f64 lambda1 lambda2)))))`	sensitivity	19	3
`log.f64`	`#f`	oflow-rescue	6	0
`asin.f64`	`(asin.f64 (+.f64 (.f64 (sin.f64 phi1) (cos.f64 delta)) (.f64 (*.f64 (cos.f64 phi1) (sin.f64 delta)) (cos.f64 theta))))`	sensitivity	1	1
`log.f64`	`(log.f64 (+.f64 (exp.f64 a) (exp.f64 b)))`	uflow-rescue	1	0
↳	`(+.f64 (exp.f64 a) (exp.f64 b))`	underflow	1
↳	`(exp.f64 a)`	underflow	128
↳	`(exp.f64 b)`	underflow	1

Confusion

	Predicted +	Predicted -
+	5750	51
-	1380	5875

Precision

0.8064516129032258

Recall

0.9912084123426995

Confusion?

	Predicted +	Predicted Maybe	Predicted -
+	5750	21	30
-	1380	33	5842

Precision?

0.8033129175946548

Recall?

0.9948284778486468

Stop Event

21×	saturated
30×	node limit

Compiler

Compiled 79494 to 48662 computations (38.8% saved)

Precisions

Click to see histograms. Total time spent on operations: 1.3min

Operation ival-sin, time spent: 18.7s, 23.0% of total-time

Operation ival-cos, time spent: 16.2s, 20.0% of total-time

Operation ival-mult, time spent: 14.7s, 18.0% of total-time

Operation ival-div, time spent: 12.3s, 15.0% of total-time

Operation ival-pow, time spent: 10.5s, 13.0% of total-time

Operation ival-add, time spent: 2.0s, 2.0% of total-time

Operation ival-sub, time spent: 1.7s, 2.0% of total-time

Operation const, time spent: 1.0s, 1.0% of total-time

Operation ival-sqrt, time spent: 863.0ms, 1.0% of total-time

Operation ival-log, time spent: 657.0ms, 1.0% of total-time

Operation ival-neg, time spent: 404.0ms, 1.0% of total-time

Operation ival-tan, time spent: 314.0ms, 0.0% of total-time

Operation ival-exp, time spent: 304.0ms, 0.0% of total-time

Operation ival-atan, time spent: 219.0ms, 0.0% of total-time

Operation ival-atan2, time spent: 201.0ms, 0.0% of total-time

Operation ival-pi, time spent: 92.0ms, 0.0% of total-time

Operation ival-cbrt, time spent: 77.0ms, 0.0% of total-time

Operation ival-acos, time spent: 54.0ms, 0.0% of total-time

Operation ival-asin, time spent: 31.0ms, 0.0% of total-time

Operation ival-fabs, time spent: 15.0ms, 0.0% of total-time

regimes1.8min (6.4%)

Counts

18631 → 806

Calls

138 calls:

9.8s	phi2
9.1s	lambda1
8.4s	phi1
7.4s	lambda2
5.3s	b

Compiler

Compiled 10682 to 7880 computations (26.2% saved)

simplify1.3min (4.6%)

Algorithm

233×	egg-herbie

Rules

152030×	`fma-define`
109430×	`distribute-lft-in`
108056×	`associate-r`
105156×	`*-commutative`
100372×	`fma-neg`

Stop Event

51×	saturated
39×	fuel
180×	node limit
2×	unsound
10×	done

Counts

92237 → 78970

Compiler

Compiled 15789 to 10790 computations (31.7% saved)

soundness1.1min (4.1%)

Rules

170208×	`fma-define`
99396×	`fma-neg`
98144×	`associate-r`
91836×	`distribute-lft-in`
81388×	`distribute-rgt-in`

Stop Event

15×	saturated
223×	node limit
3×	unsound

Compiler

Compiled 45825 to 27636 computations (39.7% saved)

prune55.7s (3.4%)

Counts: 165067 → 6762
Compiler: Compiled 436667 to 303302 computations (30.5% saved)

rewrite40.3s (2.4%)

Algorithm

184×	batch-egg-rewrite

Rules

122049×	`log1p-expm1-u`
87769×	`expm1-log1p-u`
57922×	`log-prod`
49082×	`prod-diff`
46275×	`pow1`

Stop Event

181×	node limit
3×	unsound

Counts

1781 → 51122

analyze37.5s (2.3%)

Algorithm

53×

Search

Probability	Valid	Unknown	Precondition	Infinite	Domain	Can't	Iter
0%	0%	79.4%	20.6%	0%	0%	0%	0
14.8%	11.8%	67.6%	20.6%	0%	0%	0%	1
22.6%	17.7%	60.4%	21%	0%	0.9%	0%	2
32.7%	24.7%	50.8%	21.2%	0%	3.3%	0%	3
39.8%	29.8%	45%	21.2%	0%	4%	0%	4
45.4%	33.8%	40.6%	21.2%	0%	4.5%	0%	5
48.4%	35.7%	38%	21.2%	0%	5.1%	0%	6
51.8%	37.8%	35.2%	21.2%	0%	5.7%	0.1%	7
53.5%	38.9%	33.8%	21.3%	0%	6%	0.1%	8
56.4%	40.6%	31.4%	21.3%	0%	6.6%	0.2%	9
57.7%	41.4%	30.3%	21.3%	0%	6.8%	0.2%	10
59.8%	42.7%	28.7%	21.3%	0%	7.1%	0.3%	11
61.3%	43.5%	27.5%	21.3%	0%	7.5%	0.3%	12

Compiler

Compiled 1919 to 1343 computations (30% saved)

Precisions

Click to see histograms. Total time spent on operations: 26.7s

Operation ival-mult, time spent: 7.7s, 29.0% of total-time

Operation ival-sin, time spent: 3.9s, 14.0% of total-time

Operation ival-pow, time spent: 3.8s, 14.0% of total-time

Operation ival-div, time spent: 3.7s, 14.0% of total-time

Operation ival-cos, time spent: 2.7s, 10.0% of total-time

Operation ival-add, time spent: 1.2s, 5.0% of total-time

Operation ival-sub, time spent: 1.1s, 4.0% of total-time

Operation ival-sqrt, time spent: 489.0ms, 2.0% of total-time

Operation const, time spent: 434.0ms, 2.0% of total-time

Operation ival-<, time spent: 408.0ms, 2.0% of total-time

Operation ival-neg, time spent: 302.0ms, 1.0% of total-time

Operation ival-exp, time spent: 260.0ms, 1.0% of total-time

Operation ival-atan2, time spent: 155.0ms, 1.0% of total-time

Operation ival-pi, time spent: 111.0ms, 0.0% of total-time

Operation ival-tan, time spent: 93.0ms, 0.0% of total-time

Operation ival-atan, time spent: 70.0ms, 0.0% of total-time

Operation ival-log, time spent: 64.0ms, 0.0% of total-time

Operation ival-asin, time spent: 46.0ms, 0.0% of total-time

Operation ival-and, time spent: 45.0ms, 0.0% of total-time

Operation ival-acos, time spent: 44.0ms, 0.0% of total-time

Operation ival-fabs, time spent: 37.0ms, 0.0% of total-time

Operation ival-cbrt, time spent: 33.0ms, 0.0% of total-time

Operation ival-<=, time spent: 4.0ms, 0.0% of total-time

Operation ival-or, time spent: 0.0ms, 0.0% of total-time

Operation ival-==, time spent: 0.0ms, 0.0% of total-time

bsearch21.8s (1.3%)

Algorithm

273×	binary-search
91×	left-value

Stop Event

7×	predicate-same
266×	narrow-enough

Results

7.0s	21935×	256	valid
8.0s	7381×	512	valid
1.0s	1237×	2048	valid
194.0ms	1223×	256	invalid
1.1s	986×	4096	valid
591.0ms	874×	1024	valid
352.0ms	644×	256	infinite
524.0ms	175×	512	infinite

Compiler

Compiled 104031 to 77979 computations (25% saved)

Precisions

Click to see histograms. Total time spent on operations: 13.7s

Operation ival-sin, time spent: 2.6s, 19.0% of total-time

Operation ival-cos, time spent: 2.5s, 18.0% of total-time

Operation ival-mult, time spent: 2.4s, 17.0% of total-time

Operation ival-pow, time spent: 1.4s, 10.0% of total-time

Operation ival-div, time spent: 1.4s, 10.0% of total-time

Operation ival-sub, time spent: 715.0ms, 5.0% of total-time

Operation ival-add, time spent: 632.0ms, 5.0% of total-time

Operation ival-sqrt, time spent: 459.0ms, 3.0% of total-time

Operation ival-atan2, time spent: 417.0ms, 3.0% of total-time

Operation ival-atan, time spent: 337.0ms, 2.0% of total-time

Operation ival-acos, time spent: 321.0ms, 2.0% of total-time

Operation const, time spent: 204.0ms, 1.0% of total-time

Operation ival-neg, time spent: 96.0ms, 1.0% of total-time

Operation ival-pi, time spent: 88.0ms, 1.0% of total-time

Operation ival-log, time spent: 73.0ms, 1.0% of total-time

Operation ival-exp, time spent: 58.0ms, 0.0% of total-time

Operation ival-asin, time spent: 27.0ms, 0.0% of total-time

Operation ival-tan, time spent: 11.0ms, 0.0% of total-time

Operation ival-fabs, time spent: 3.0ms, 0.0% of total-time

series18.8s (1.1%)

Counts

1781 → 41115

Calls

10125 calls:

Time	Variable		Point	Expression
308.0ms	lambda2	@	inf	(log (+ 1 (acos (+ (* (* (cos phi1) (cos phi2)) (cos (- lambda2 lambda1))) (* (sin phi1) (sin phi2))))))
305.0ms	c	@	-inf	(sqrt (- (* b b) (* (* 4 a) c)))
275.0ms	ew	@	-inf	(pow (cbrt (* ew (sin t))) 3)
177.0ms	phi2	@	-inf	(log (* R (acos (+ (* (* (cos phi1) (cos phi2)) (cos (- lambda2 lambda1))) (* (sin phi1) phi2)))))
155.0ms	b	@	inf	(* (neg (* x-scale (sqrt 8))) (* (sqrt 2) (sqrt (+ (* (* a (+ 1 (* -1/64800 (* (pow angle 2) (pow (PI) 2))))) (* a (+ 1 (* -1/64800 (* (pow angle 2) (pow (PI) 2)))))) (* (* b (sin (* 1/180 (* angle (PI))))) (* b (sin (* 1/180 (* angle (PI))))))))))

Herbie run

sample8.2min (29.7%)

localize7.7min (28.1%)

eval2.4min (8.9%)

preprocess2.1min (7.7%)

regimes1.8min (6.4%)

simplify1.3min (4.6%)

soundness1.1min (4.1%)

prune55.7s (3.4%)

rewrite40.3s (2.4%)

analyze37.5s (2.3%)

bsearch21.8s (1.3%)

series18.8s (1.1%)

end34.0ms (0%)

Profiling