Time: 15.8min
Passed: 17/20
Tests: 31
Bits: 272/471

Output vs Input Accuracy

Each point represents a Herbie run below. Its horizontal position shows initial accuracy, and vertical position shows final accuracy. Points above the line are improved by Herbie.

Accuracy vs Cost

A joint cost-accuracy pareto curve for the Herbie runs below. Accuracy is on the vertical axis, and cost is on the vertical axis. Down and to the left is better. The initial programs are shown by the red square.
TestStartResult ?Target ?Time
VandenBroeck and Keller, Equation (6)16.54.933.9s»
VandenBroeck and Keller, Equation (24)0.20.237.1s»
VandenBroeck and Keller, Equation (23)13.50.31.1min»
VandenBroeck and Keller, Equation (20)61.62.251.4s»
Toniolo and Linder, Equation (7)43.411.446.6s»
Toniolo and Linder, Equation (3b), real4.33.247.4s»
Toniolo and Linder, Equation (3a)1.11.140.5s»
Toniolo and Linder, Equation (2)10.30.927.2s»
Toniolo and Linder, Equation (13)34.232.337.4s»
Toniolo and Linder, Equation (10-)47.021.741.1s»
Toniolo and Linder, Equation (10+)32.723.333.0s»
Migdal et al, Equation (64)0.50.523.9s»
Migdal et al, Equation (51)0.50.518.5s»
Maksimov and Kolovsky, Equation (4)18.10.530.1s»
Maksimov and Kolovsky, Equation (32)16.01.323.4s»
Maksimov and Kolovsky, Equation (3)18.09.027.2s»
Henrywood and Agarwal, Equation (9a)14.410.117.0s»
Henrywood and Agarwal, Equation (3)19.019.08.5s»
Henrywood and Agarwal, Equation (13)59.626.248.5s»
Henrywood and Agarwal, Equation (12)26.720.346.2s»
Given's Rotation SVD example, simplified15.80.229.4s»
Falkner and Boettcher, Equation (22+)1.00.69.8s»
Falkner and Boettcher, Equation (20:1,3)0.50.424.4s»
Falkner and Boettcher, Appendix B, 20.00.010.1s»
Falkner and Boettcher, Appendix B, 10.60.625.4s»
Falkner and Boettcher, Appendix A2.22.221.2s»
Bulmash initializePoisson0.00.01.3min»
Bouland and Aaronson, Equation (26)0.20.23.9s»
Bouland and Aaronson, Equation (25)0.20.217.1s»
Bouland and Aaronson, Equation (24)0.20.211.9s»
Given's Rotation SVD example13.16.513.113.6s»