Metrics for Herbie run

Date:	Wednesday, March 20th, 2024
Commit:	23dc2d0f on exact-hallucinations
Hostname:	nightly with Racket 8.11.1
Seed:	2024080
Parameters:	256 points for 4 iterations
Flags:	`reduce:regimesreduce:avg-errorreduce:binary-searchreduce:branch-expressionssetup:simplifysetup:searchrules:arithmeticrules:polynomialsrules:fractionsrules:exponentsrules:trigonometryrules:hyperbolicrules:numericsrules:specialrules:boolsrules:branchesgenerate:rrgenerate:taylorgenerate:simplifygenerate:proofs` default

sample1.3min (27%)

Results

16.8s	116004×	256	valid
9.5s	23324×	512	valid
32.6s	16792×	8192	exit
5.8s	16031×	1024	valid
1.1s	4976×	256	infinite
4.3s	4783×	2048	valid
554.0ms	4067×	256	invalid
314.0ms	1399×	256	unsamplable
1.0ms	2×	512	infinite

Precisions

Click to see histograms. Total time spent on operations: 50.2s

Operation ival-pow, time spent: 13.6s, 27.0% of total-time

Operation ival-exp, time spent: 9.1s, 18.0% of total-time

Operation ival-log, time spent: 6.4s, 13.0% of total-time

Operation ival-cos, time spent: 3.8s, 8.0% of total-time

Operation ival-mult, time spent: 2.3s, 5.0% of total-time

Operation ival-<=, time spent: 1.8s, 4.0% of total-time

Operation ival-sin, time spent: 1.8s, 4.0% of total-time

Operation ival-sinh, time spent: 1.6s, 3.0% of total-time

Operation ival-sqrt, time spent: 1.4s, 3.0% of total-time

Operation ival-add, time spent: 1.3s, 3.0% of total-time

Operation ival-div, time spent: 1.3s, 3.0% of total-time

Operation ival-sub, time spent: 1.2s, 2.0% of total-time

Operation ival-tan, time spent: 903.0ms, 2.0% of total-time

Operation ival-fmod, time spent: 801.0ms, 2.0% of total-time

Operation ival-neg, time spent: 687.0ms, 1.0% of total-time

Operation ival-acos, time spent: 661.0ms, 1.0% of total-time

Operation const, time spent: 558.0ms, 1.0% of total-time

Operation ival-and, time spent: 282.0ms, 1.0% of total-time

Operation ival-==, time spent: 182.0ms, 0.0% of total-time

Operation ival-<, time spent: 178.0ms, 0.0% of total-time

Operation ival-or, time spent: 87.0ms, 0.0% of total-time

Operation ival-atan, time spent: 84.0ms, 0.0% of total-time

Operation ival->=, time spent: 48.0ms, 0.0% of total-time

Operation ival->, time spent: 33.0ms, 0.0% of total-time

Bogosity

localize39.1s (14.1%)

Compiler: Compiled 9492 to 4828 computations (49.1% saved)
Precisions: Click to see histograms. Total time spent on operations: 24.0s
Operation ival-pow, time spent: 4.6s, 19.0% of total-time
Operation ival-mult, time spent: 3.9s, 16.0% of total-time
Operation ival-log, time spent: 3.1s, 13.0% of total-time
Operation composed, time spent: 2.4s, 10.0% of total-time
Operation ival-add, time spent: 1.6s, 7.0% of total-time
Operation ival-asin, time spent: 1.2s, 5.0% of total-time
Operation ival-cos, time spent: 1.2s, 5.0% of total-time
Operation ival-sub, time spent: 1.1s, 5.0% of total-time
Operation ival-div, time spent: 816.0ms, 3.0% of total-time
Operation ival-exp, time spent: 691.0ms, 3.0% of total-time
Operation ival-acos, time spent: 639.0ms, 3.0% of total-time
Operation ival-sqrt, time spent: 611.0ms, 3.0% of total-time
Operation ival-fmod, time spent: 540.0ms, 2.0% of total-time
Operation const, time spent: 514.0ms, 2.0% of total-time
Operation ival-sin, time spent: 353.0ms, 1.0% of total-time
Operation ival-tan, time spent: 242.0ms, 1.0% of total-time
Operation ival-cbrt, time spent: 216.0ms, 1.0% of total-time
Operation ival-neg, time spent: 139.0ms, 1.0% of total-time
Operation ival-e, time spent: 87.0ms, 0.0% of total-time
Operation ival-sinh, time spent: 47.0ms, 0.0% of total-time
Operation ival-pi, time spent: 13.0ms, 0.0% of total-time
Operation ival-fabs, time spent: 5.0ms, 0.0% of total-time
Operation ival-atan, time spent: 4.0ms, 0.0% of total-time

analyze32.3s (11.6%)

Algorithm

20×

Search

Probability	Valid	Unknown	Precondition	Infinite	Domain	Can't	Iter
0%	0%	73.7%	26.3%	0%	0%	0%	0
22.1%	16.3%	57.5%	26.3%	0%	0%	0%	1
44.1%	32.5%	41.2%	26.3%	0%	0%	0%	2
52.6%	38.7%	35%	26.3%	0%	0%	0%	3
61.9%	45.6%	28.1%	26.3%	0%	0%	0%	4
65.2%	47.5%	25.3%	26.3%	0%	0.9%	0%	5
66.8%	48.4%	24%	26.3%	0%	1.1%	0.2%	6
71.1%	51.1%	20.8%	26.3%	0%	1.6%	0.2%	7
72.1%	51.6%	20%	26.3%	0%	1.8%	0.4%	8
74.4%	53%	18.2%	26.3%	0%	2%	0.4%	9
75.8%	53.9%	17.2%	26.3%	0%	2.1%	0.5%	10
76.6%	54.3%	16.6%	26.3%	0%	2.3%	0.5%	11
77.3%	54.8%	16.1%	26.3%	0%	2.3%	0.5%	12

Compiler

Compiled 295 to 210 computations (28.8% saved)

Precisions

Click to see histograms. Total time spent on operations: 21.5s

Operation ival-<=, time spent: 12.5s, 58.0% of total-time

Operation ival-add, time spent: 4.4s, 20.0% of total-time

Operation ival-tan, time spent: 2.3s, 11.0% of total-time

Operation ival-and, time spent: 350.0ms, 2.0% of total-time

Operation ival-sub, time spent: 302.0ms, 1.0% of total-time

Operation ival-==, time spent: 295.0ms, 1.0% of total-time

Operation const, time spent: 270.0ms, 1.0% of total-time

Operation ival-cos, time spent: 186.0ms, 1.0% of total-time

Operation ival-or, time spent: 178.0ms, 1.0% of total-time

Operation ival-div, time spent: 153.0ms, 1.0% of total-time

Operation ival-mult, time spent: 148.0ms, 1.0% of total-time

Operation ival-pow, time spent: 132.0ms, 1.0% of total-time

Operation ival-sin, time spent: 100.0ms, 0.0% of total-time

Operation ival-exp, time spent: 53.0ms, 0.0% of total-time

Operation ival-fmod, time spent: 45.0ms, 0.0% of total-time

Operation ival-neg, time spent: 25.0ms, 0.0% of total-time

Operation ival-log, time spent: 24.0ms, 0.0% of total-time

Operation ival-sqrt, time spent: 20.0ms, 0.0% of total-time

Operation ival-sinh, time spent: 14.0ms, 0.0% of total-time

Operation ival-atan, time spent: 1.0ms, 0.0% of total-time

Operation ival->=, time spent: 1.0ms, 0.0% of total-time

Operation ival-acos, time spent: 0.0ms, 0.0% of total-time

Operation ival->, time spent: 0.0ms, 0.0% of total-time

Operation ival-<, time spent: 0.0ms, 0.0% of total-time

soundness32.3s (11.6%)

Rules

28910×	`fma-define`
27618×	`fma-neg`
23730×	`associate-r`
19722×	`times-frac`
18666×	`associate-l`

Stop Event

6×	saturated
60×	node limit
2×	unsound

Compiler

Compiled 11135 to 5762 computations (48.3% saved)

eval23.9s (8.6%)

Compiler: Compiled 997328 to 567118 computations (43.1% saved)

rewrite17.0s (6.1%)

Algorithm

67×	batch-egg-rewrite

Rules

34863×	`log1p-expm1-u`
30710×	`expm1-log1p-u`
26354×	`unpow-prod-down`
21892×	`log-prod`
18126×	`associate-r`

Stop Event

67×	node limit

Counts

477 → 16992

simplify16.9s (6.1%)

Algorithm

86×	egg-herbie

Rules

44558×	`fma-define`
40312×	`fma-neg`
34092×	`distribute-lft-in`
31634×	`associate-r`
29710×	`distribute-rgt-in`

Stop Event

19×	saturated
62×	node limit
11×	fuel
5×	unsound
8×	done

Counts

25725 → 21949

Compiler

Compiled 1636 to 1107 computations (32.3% saved)

series12.0s (4.3%)

Counts

477 → 8733

Calls

2091 calls:

Time	Variable		Point	Expression
3.7s	lo	@	0	(+ 1 (/ lo hi))
957.0ms	x	@	inf	(* (+ (/ hi lo) 1) (/ (- hi x) lo))
213.0ms	a	@	-inf	(* x (+ 1 (/ (- (tan (+ y z)) (tan a)) x)))
182.0ms	y	@	-inf	(log (- (/ (sin y) (cos y)) (- (tan a) x)))
159.0ms	y	@	inf	(log (- (/ (sin y) (cos y)) (- (tan a) x)))

prune10.1s (3.6%)

Counts: 41076 → 1729
Compiler: Compiled 59138 to 38518 computations (34.9% saved)

regimes10.0s (3.6%)

Counts

3273 → 155

Calls

55 calls:

1.5s	b
1.3s	a
1.2s	x
1.1s	r
623.0ms	(/.f64 (sin.f64 b) (cos.f64 (+.f64 a b)))

Compiler

Compiled 1698 to 1203 computations (29.2% saved)

preprocess8.0s (2.9%)

Algorithm

20×	egg-herbie

Rules

10358×	`fma-neg`
10108×	`fma-define`
3250×	`distribute-rgt-in`
3210×	`unsub-neg`
3162×	`distribute-lft-in`

Explanations

Click to see full explanations table

Operator	Subexpression	Explanation	Count
`cos.f64`	`#f`	sensitivity	521	0
`sqrt.f64`	`#f`	uflow-rescue	314	0
`pow.f64`	`(pow.f64 l (exp.f64 w))`	sensitivity	125	16
`/.f64`	`(/.f64 (-.f64 x lo) (-.f64 hi lo))`	n/o	256	0
↳	`(-.f64 hi lo)`	overflow	256
`*.f64`	`#f`	*no**	51	0
`acos.f64`	`(acos.f64 (-.f64 1 x))`	sensitivity	255	0
`sqrt.f64`	`#f`	oflow-rescue	571	0
`log.f64`	`(log.f64 (/.f64 (sinh.f64 x) x))`	sensitivity	249	0
`log.f64`	`(log.f64 (/.f64 (sinh.f64 x) x))`	oflow-rescue	5	0
↳	`(sinh.f64 x)`	overflow	5
↳	`(/.f64 (sinh.f64 x) x)`	overflow	5
`-.f64`	`#f`	cancellation	338	0
`-.f64`	`(-.f64 (.f64 a a) (.f64 b b))`	nan-rescue	14	0
↳	`(*.f64 a a)`	overflow	116
↳	`(*.f64 b b)`	overflow	14
`*.f64`	`#f`	*nu**	20	0
`tan.f64`	`(tan.f64 (+.f64 y z))`	sensitivity	191	1
`*.f64`	`(.f64 (pow.f64 c 2) (.f64 (*.f64 x (pow.f64 s 2)) x))`	*uo**	33	0
↳	`(pow.f64 c 2)`	underflow	101
↳	`(*.f64 x (pow.f64 s 2))`	overflow	109
↳	`(pow.f64 s 2)`	overflow	112
↳	`(.f64 (.f64 x (pow.f64 s 2)) x)`	overflow	110

Confusion

	Predicted +	Predicted -
+	2180	279
-	698	1707

Precision

0.7574704656011119

Recall

0.8865392435949573

Confusion?

	Predicted +	Predicted Maybe	Predicted -
+	2180	0	279
-	698	17	1690

Precision?

0.7530224525043178

Recall?

0.8865392435949573

Stop Event

18×	saturated
2×	node limit

Compiler

Compiled 9244 to 5482 computations (40.7% saved)

Precisions

Click to see histograms. Total time spent on operations: 1.6s

Operation ival-mult, time spent: 370.0ms, 23.0% of total-time

Operation ival-cos, time spent: 257.0ms, 16.0% of total-time

Operation ival-pow, time spent: 229.0ms, 14.0% of total-time

Operation ival-log, time spent: 145.0ms, 9.0% of total-time

Operation ival-sqrt, time spent: 84.0ms, 5.0% of total-time

Operation ival-sin, time spent: 80.0ms, 5.0% of total-time

Operation ival-add, time spent: 71.0ms, 4.0% of total-time

Operation ival-exp, time spent: 69.0ms, 4.0% of total-time

Operation ival-sub, time spent: 66.0ms, 4.0% of total-time

Operation ival-tan, time spent: 51.0ms, 3.0% of total-time

Operation ival-sinh, time spent: 50.0ms, 3.0% of total-time

Operation ival-div, time spent: 41.0ms, 3.0% of total-time

Operation ival-fmod, time spent: 36.0ms, 2.0% of total-time

Operation ival-acos, time spent: 35.0ms, 2.0% of total-time

Operation const, time spent: 33.0ms, 2.0% of total-time

Operation ival-neg, time spent: 11.0ms, 1.0% of total-time

Operation ival-atan, time spent: 4.0ms, 0.0% of total-time

bsearch1.2s (0.4%)

Algorithm

37×	binary-search
3×	left-value

Stop Event

1×	predicate-same
36×	narrow-enough

Results

575.0ms	3021×	256	valid
388.0ms	1181×	512	valid
42.0ms	262×	256	infinite

Compiler

Compiled 5943 to 4417 computations (25.7% saved)

Precisions

Click to see histograms. Total time spent on operations: 567.0ms

Operation ival-cos, time spent: 165.0ms, 29.0% of total-time

Operation ival-sin, time spent: 128.0ms, 23.0% of total-time

Operation ival-div, time spent: 78.0ms, 14.0% of total-time

Operation ival-mult, time spent: 47.0ms, 8.0% of total-time

Operation ival-pow, time spent: 39.0ms, 7.0% of total-time

Operation ival-tan, time spent: 36.0ms, 6.0% of total-time

Operation ival-add, time spent: 33.0ms, 6.0% of total-time

Operation ival-exp, time spent: 21.0ms, 4.0% of total-time

Operation const, time spent: 7.0ms, 1.0% of total-time

Operation ival-neg, time spent: 6.0ms, 1.0% of total-time

Operation ival-sub, time spent: 5.0ms, 1.0% of total-time

Operation ival-acos, time spent: 5.0ms, 1.0% of total-time

Herbie run

sample1.3min (27%)

localize39.1s (14.1%)

analyze32.3s (11.6%)

soundness32.3s (11.6%)

eval23.9s (8.6%)

rewrite17.0s (6.1%)

simplify16.9s (6.1%)

series12.0s (4.3%)

prune10.1s (3.6%)

regimes10.0s (3.6%)

preprocess8.0s (2.9%)

bsearch1.2s (0.4%)

end2.0ms (0%)

Profiling