Profiling results ----------------- Total cpu time observed: 775372ms (out of 778352ms) Number of samples taken: 2282 (once every 340ms) (Hiding functions with self<1.0% and local<2.0%: 17 of 289 hidden) ========================================================================================== Caller Idx Total Self Name+src Local% ms(pct) ms(pct) Callee ========================================================================================== ??? [79] 100.0% [1] 654226(84.4%) 0(0.0%) profile-thunk16 ...t/pkgs/profile-lib/main.rkt:9:0 run [23] 100.0% ------------------------------------------------------------------------------------------ ??? [79] 100.0% [2] 501686(64.7%) 0(0.0%) run-improve50 ...me-testing/src/mainloop.rkt:235:0 for-loop [24] 93.6% get-final-combination [32] 5.9% loop [100] 0.3% setup-alt-simplified [38] 0.1% prepare-points [101] 0.1% ------------------------------------------------------------------------------------------ ??? [180] 100.0% [3] 451834(58.3%) 1002(0.1%) iterate-egraph!13 ...g/src/core/simplify.rkt:105:0 one-iter [25] 98.7% map-enodes [34] 1.1% ------------------------------------------------------------------------------------------ ??? [180] 100.0% [4] 51216(6.6%) 0(0.0%) loop ...regime-testing/src/core/simplify.rkt:237:2 pass [27] 100.0% ------------------------------------------------------------------------------------------ ??? [174] 100.0% [5] 42836(5.5%) 0(0.0%) approximate9 ...e-testing/src/core/taylor.rkt:12:0 loop [28] 95.3% f45 [162] 4.0% taylor-invert [17] 0.8% ------------------------------------------------------------------------------------------ loop [192] 100.0% [6] 40862(5.3%) 0(0.0%) ??? ...herbie/regime-testing/src/points.rkt:293:24 eval-errors [26] 100.0% ------------------------------------------------------------------------------------------ map [168] 5.0% loop [192] 95.0% [7] 40180(5.2%) 312(0.0%) ??? ...herbie/regime-testing/src/points.rkt:289:26 ??? [55] 91.0% ??? [174] 6.6% ulp-difference [62] 1.6% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [8] 37324(4.8%) 0(0.0%) ??? ...ie/regime-testing/src/core/taylor.rkt:44:15 ??? [47] 93.8% hash-ref! [216] 6.2% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [9] 35372(4.6%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:285:26 for-loop [29] 100.0% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [10] 32676(4.2%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:295:23 for-loop [30] 100.0% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [11] 29660(3.8%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:322:33 for-loop [31] 100.0% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [12] 16154(2.1%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:392:23 for-loop [33] 100.0% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [13] 15888(2.0%) 0(0.0%) ??? ...ie/regime-testing/src/core/taylor.rkt:67:15 ??? [174] 89.3% hash-ref! [216] 10.7% ------------------------------------------------------------------------------------------ free-variables [272] 3.0% substitute-e [72] 18.2% match-e [56] 78.7% [14] 10952(1.4%) 336(0.0%) variable? ...e-testing/src/syntax/syntax.rkt:665:0 ??? [132] 96.9% ------------------------------------------------------------------------------------------ copying-mergesort [264] 100.0% [15] 4680(0.6%) 0(0.0%) jloop ...t/collects/racket/private/sort.rkt:121:23 ??? [55] 93.0% ??? [174] 7.0% ------------------------------------------------------------------------------------------ map-enodes [34] 14.0% loop [192] 86.0% [16] 4580(0.6%) 312(0.0%) for-loop ...me-testing/src/core/simplify.rkt:195:2 setfindf [35] 56.2% eval-const-expr [37] 15.2% compose [244] 14.0% curried [271] 7.8% ------------------------------------------------------------------------------------------ approximate9 [5] 8.8% f45 [162] 91.2% [17] 3802(0.5%) 0(0.0%) taylor-invert ...testing/src/core/taylor.rkt:312:0 simplify [46] 54.1% first-nonzero-exp [36] 45.9% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [18] 1322(0.2%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:338:33 hash-ref! [216] 87.7% for-loop [40] 12.3% ------------------------------------------------------------------------------------------ rewriter [103] 100.0% [19] 1290(0.2%) 0(0.0%) *rules* ...gime-testing/src/syntax/rules.rkt:565:0 for-loop [197] 100.0% ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [20] 1002(0.1%) 348(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:483:23 for-loop [39] 65.3% ------------------------------------------------------------------------------------------ f45 [162] 100.0% [21] 648(0.1%) 0(0.0%) taylor ...regime-testing/src/core/taylor.rkt:163:0 debug-print [64] 50.0% ??? [132] 50.0% ------------------------------------------------------------------------------------------ parse-loop12 [187] 1.4% loop [22] 98.6% [22] 346(0.0%) 0(0.0%) loop ...cket/collects/racket/private/map.rkt:36:19 loop [22] 98.6% ??? [174] 1.4% ------------------------------------------------------------------------------------------ profile-thunk16 [1] 99.7% [23] 656142(84.6%) 0(0.0%) run ...share/racket/pkgs/profile-lib/main.rkt:39:2 ??? [79] 100.0% ------------------------------------------------------------------------------------------ run-improve50 [2] 100.0% [24] 469684(60.6%) 0(0.0%) for-loop ...regime-testing/src/mainloop.rkt:251:10 run-iter! [41] 100.0% ------------------------------------------------------------------------------------------ iterate-egraph!13 [3] 100.0% [25] 445914(57.5%) 3352(0.4%) one-iter ...me-testing/src/core/simplify.rkt:116:0 for-loop [42] 72.2% for-loop [43] 26.9% ------------------------------------------------------------------------------------------ override-at-pnts [155] 0.4% ??? [79] 1.4% composed [196] 2.5% option-on-expr [204] 2.5% best-and-tied-at-points [154] 17.4% loop [192] 30.9% ??? [6] 44.8% [26] 91162(11.8%) 0(0.0%) eval-errors ...regime-testing/src/points.rkt:271:0 for-loop [44] 100.0% ------------------------------------------------------------------------------------------ loop [4] 100.0% [27] 51216(6.6%) 656(0.1%) pass ...regime-testing/src/core/simplify.rkt:227:2 for-loop [45] 98.7% ------------------------------------------------------------------------------------------ approximate9 [5] 100.0% [28] 40804(5.3%) 0(0.0%) loop ...ie/regime-testing/src/core/taylor.rkt:86:4 ??? [174] 61.1% hash-ref! [216] 38.9% ------------------------------------------------------------------------------------------ ??? [9] 100.0% [29] 35372(4.6%) 324(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:285:52 ??? [47] 61.7% hash-ref! [216] 34.5% simplify [46] 2.9% ------------------------------------------------------------------------------------------ ??? [10] 100.0% [30] 32676(4.2%) 0(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:298:27 hash-ref! [216] 99.6% simplify [46] 0.4% ------------------------------------------------------------------------------------------ ??? [11] 100.0% [31] 29660(3.8%) 1010(0.1%) for-loop ...ime-testing/src/core/taylor.rkt:324:46 hash-ref! [216] 96.6% ------------------------------------------------------------------------------------------ run-improve50 [2] 100.0% [32] 29582(3.8%) 0(0.0%) get-final-combination ...ng/src/mainloop.rkt:266:0 split-table [48] 97.8% extract-alt [51] 2.2% ------------------------------------------------------------------------------------------ ??? [12] 100.0% [33] 16154(2.1%) 0(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:396:33 for-loop [49] 100.0% ------------------------------------------------------------------------------------------ iterate-egraph!13 [3] 100.0% [34] 4918(0.6%) 0(0.0%) map-enodes ...me-testing/src/core/egraph.rkt:140:0 loop [192] 86.9% for-loop [16] 13.1% ------------------------------------------------------------------------------------------ for-loop [16] 100.0% [35] 2572(0.3%) 960(0.1%) setfindf ...ie/regime-testing/src/common.rkt:187:0 for-loop [50] 50.6% ??? [132] 12.1% ------------------------------------------------------------------------------------------ taylor-sqrt [177] 16.0% taylor-invert [17] 84.0% [36] 2076(0.3%) 0(0.0%) first-nonzero-exp ...ing/src/core/taylor.rkt:260:0 hash-ref! [216] 100.0% ------------------------------------------------------------------------------------------ for-loop [16] 100.0% [37] 696(0.1%) 0(0.0%) eval-const-expr ...-testing/src/programs.rkt:168:0 ??? [165] 51.7% f45 [162] 48.3% ------------------------------------------------------------------------------------------ run-improve50 [2] 100.0% [38] 660(0.1%) 0(0.0%) setup-alt-simplified ...-testing/src/glue.rkt:49:0 simplify-alt [52] 100.0% ------------------------------------------------------------------------------------------ ??? [20] 100.0% [39] 654(0.1%) 0(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:487:35 parse-loop12 [187] 100.0% ------------------------------------------------------------------------------------------ ??? [18] 100.0% [40] 326(0.0%) 0(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:341:46 hash-ref! [216] 100.0% ------------------------------------------------------------------------------------------ for-loop [24] 100.0% [41] 469684(60.6%) 0(0.0%) run-iter! ...regime-testing/src/mainloop.rkt:215:0 simplify! [53] 85.5% gen-series! [57] 9.1% finalize-iter! [59] 4.1% gen-rewrites! [63] 0.6% choose-best-alt! [66] 0.3% ------------------------------------------------------------------------------------------ one-iter [25] 100.0% [42] 322100(41.5%) 968(0.1%) for-loop ...me-testing/src/core/simplify.rkt:166:2 for-loop [54] 86.2% match-e [56] 11.3% update-leader! [83] 2.2% ------------------------------------------------------------------------------------------ for-loop [43] 48.7% one-iter [25] 51.3% [43] 119822(15.5%) 31392(4.0%) for-loop ...e-testing/src/core/simplify.rkt:126:12 for-loop [43] 48.7% match-e [56] 36.9% ------------------------------------------------------------------------------------------ eval-errors [26] 100.0% [44] 91162(11.8%) 3366(0.4%) for-loop ...ie/regime-testing/src/points.rkt:273:2 ??? [55] 89.7% ??? [174] 3.7% ulp-difference [62] 2.1% ??? [77] 0.3% ------------------------------------------------------------------------------------------ pass [27] 100.0% [45] 50560(6.5%) 978(0.1%) for-loop ...e-testing/src/core/simplify.rkt:230:29 argmin [58] 71.9% loop [60] 18.3% for-loop [61] 7.2% make-sequence [145] 0.7% ------------------------------------------------------------------------------------------ parse-loop12 [187] 0.8% for-loop [30] 0.8% for-loop [29] 4.1% taylor-invert [17] 5.1% hash-ref! [216] 88.2% [46] 40506(5.2%) 0(0.0%) simplify ...egime-testing/src/core/reduce.rkt:16:0 f45 [162] 55.0% parse-loop12 [187] 32.1% debug7 [65] 4.1% debug-print [64] 4.0% simplify-node [175] 3.1% simplify* [195] 1.6% ------------------------------------------------------------------------------------------ ??? [8] 40.4% for-loop [29] 59.6% [47] 35676(4.6%) 0(0.0%) ??? ...e/regime-testing/src/core/taylor.rkt:288:19 hash-ref! [216] 100.0% ------------------------------------------------------------------------------------------ get-final-combination [32] 100.0% [48] 28920(3.7%) 0(0.0%) split-table ...e/regime-testing/src/glue.rkt:162:0 f45 [162] 100.0% ------------------------------------------------------------------------------------------ for-loop [33] 100.0% [49] 16154(2.1%) 0(0.0%) for-loop ...ime-testing/src/core/taylor.rkt:398:39 hash-ref! [216] 100.0% ------------------------------------------------------------------------------------------ setfindf [35] 100.0% [50] 1302(0.2%) 1302(0.2%) for-loop ...ie/regime-testing/src/common.rkt:188:2 ------------------------------------------------------------------------------------------ get-final-combination [32] 100.0% [51] 662(0.1%) 0(0.0%) extract-alt ...ie/regime-testing/src/glue.rkt:55:0 loop [188] 50.5% argmins [67] 49.5% ------------------------------------------------------------------------------------------ setup-alt-simplified [38] 100.0% [52] 660(0.1%) 0(0.0%) simplify-alt ...e/regime-testing/src/glue.rkt:76:0 ??? [180] 100.0% ------------------------------------------------------------------------------------------ run-iter! [41] 100.0% [53] 401750(51.8%) 0(0.0%) simplify! ...regime-testing/src/mainloop.rkt:155:0 for-loop [68] 100.0% ------------------------------------------------------------------------------------------ for-loop [42] 100.0% [54] 277526(35.8%) 936(0.1%) for-loop ...e-testing/src/core/simplify.rkt:148:10 mk-enode! [80] 28.5% merge-egraph-nodes! [71] 25.1% loop! [203] 21.4% substitute-e [72] 19.0% list-member? [220] 5.7% ------------------------------------------------------------------------------------------ ??? [238] 0.2% loop [127] 0.2% ??? [257] 0.6% jloop [15] 2.4% loop [270] 3.3% ??? [7] 20.0% loop [192] 28.8% for-loop [44] 44.6% [55] 183244(23.6%) 5334(0.7%) ??? ...erbie/regime-testing/src/programs.rkt:156:4 ??? [69] 70.7% ??? [174] 14.7% loop [192] 6.3% ??? [207] 5.1% ??? [77] 0.4% ------------------------------------------------------------------------------------------ for-loop [42] 18.2% for-loop [81] 23.6% for-loop [43] 58.3% [56] 124684(16.1%) 41522(5.4%) match-e ...regime-testing/src/core/ematch.rkt:46:0 for-loop [70] 51.1% for-loop [81] 7.5% ??? [132] 6.4% list-cartesian-product [87] 2.8% variable? [14] 2.8% loop [192] 1.4% curry* [93] 0.3% ------------------------------------------------------------------------------------------ run-iter! [41] 100.0% [57] 42836(5.5%) 0(0.0%) gen-series! ...gime-testing/src/mainloop.rkt:129:0 for-loop [73] 100.0% ------------------------------------------------------------------------------------------ for-loop [45] 100.0% [58] 36360(4.7%) 0(0.0%) argmin ...re/racket/collects/racket/list.rkt:743:0 for-loop [74] 92.5% expression-cost [75] 7.5% ------------------------------------------------------------------------------------------ run-iter! [41] 100.0% [59] 19442(2.5%) 0(0.0%) finalize-iter! ...e-testing/src/mainloop.rkt:169:0 ??? [132] 100.0% ------------------------------------------------------------------------------------------ for-loop [45] 100.0% [60] 9258(1.2%) 0(0.0%) loop ...hare/racket/collects/racket/list.rkt:729:4 for-loop [74] 82.1% expression-cost [75] 17.9% ------------------------------------------------------------------------------------------ for-loop [45] 100.0% [61] 3630(0.5%) 3630(0.5%) for-loop ...e-testing/src/core/simplify.rkt:217:13 ------------------------------------------------------------------------------------------ loop [192] 19.2% ??? [7] 20.4% for-loop [44] 60.4% [62] 3236(0.4%) 2186(0.3%) ulp-difference ...egime-testing/src/float.rkt:19:0 ??? [235] 22.3% return/no-unsupplied [237] 10.1% ------------------------------------------------------------------------------------------ run-iter! [41] 100.0% [63] 3014(0.4%) 0(0.0%) gen-rewrites! ...me-testing/src/mainloop.rkt:142:0 for-loop [76] 100.0% ------------------------------------------------------------------------------------------ taylor [21] 16.7% simplify [46] 83.3% [64] 1942(0.3%) 1280(0.2%) debug-print .../regime-testing/src/debug.rkt:107:0 for-loop [78] 34.1% ------------------------------------------------------------------------------------------ simplify [46] 100.0% [65] 1676(0.2%) 1026(0.1%) debug7 ...erbie/regime-testing/src/debug.rkt:102:0 ??? [235] 38.8% ------------------------------------------------------------------------------------------ run-iter! [41] 100.0% [66] 1626(0.2%) 0(0.0%) choose-best-alt! ...testing/src/mainloop.rkt:115:0 ??? [132] 100.0% ------------------------------------------------------------------------------------------ extract-alt [51] 49.2% best-alt [182] 50.8% [67] 666(0.1%) 0(0.0%) argmins ...bie/regime-testing/src/common.rkt:153:0 composed [196] 100.0% ------------------------------------------------------------------------------------------ simplify! [53] 94.0% [68] 427388(55.1%) 0(0.0%) for-loop .../regime-testing/src/mainloop.rkt:159:6 ??? [79] 100.0% ------------------------------------------------------------------------------------------ ??? [55] 100.0% [69] 129468(16.7%) 83768(10.8%) ??? ...acket/collects/racket/private/kw.rkt:444:14 ??? [85] 12.0% ??? [86] 11.8% ??? [102] 5.9% bf* [91] 1.9% ??? [224] 1.6% for-loop [92] 1.3% parse-loop12 [187] 0.2% ------------------------------------------------------------------------------------------ match-e [56] 100.0% [70] 81154(10.5%) 27218(3.5%) for-loop ...gime-testing/src/core/ematch.rkt:59:11 for-loop [81] 52.3% list-cartesian-product [87] 10.0% loop [192] 7.1% curry* [93] 1.0% foldl [209] 0.4% filter [256] 0.4% ------------------------------------------------------------------------------------------ for-loop [88] 3.7% merge-egraph-nodes! [71] 5.1% for-loop [54] 91.2% [71] 69762(9.0%) 2032(0.3%) merge-egraph-nodes! ...g/src/core/egraph.rkt:153:0 update-leader! [83] 41.6% for-loop [84] 26.7% loop! [203] 16.7% merge-egraph-nodes! [71] 5.1% for-loop [88] 4.9% enode-merge! [94] 1.0% make-sequence [145] 1.0% for-loop [197] 0.5% ------------------------------------------------------------------------------------------ substitute-e [72] 6.3% for-loop [54] 93.7% [72] 52684(6.8%) 2450(0.3%) substitute-e ...e-testing/src/core/ematch.rkt:71:0 mk-enode! [80] 86.5% substitute-e [72] 6.3% ??? [132] 2.7% variable? [14] 1.6% ------------------------------------------------------------------------------------------ gen-series! [57] 100.0% [73] 42836(5.5%) 0(0.0%) for-loop .../regime-testing/src/mainloop.rkt:135:7 taylor-alt [82] 100.0% ------------------------------------------------------------------------------------------ loop [60] 18.4% argmin [58] 81.6% [74] 41232(5.3%) 2552(0.3%) for-loop .../regime-testing/src/programs.rkt:210:2 ??? [180] 66.7% ...sting/src/common.rkt:48:2 [135] 21.5% ??? [133] 5.7% ------------------------------------------------------------------------------------------ loop [60] 37.8% argmin [58] 62.2% [75] 4386(0.6%) 314(0.0%) expression-cost ...-testing/src/programs.rkt:209:0 compile [89] 92.8% ------------------------------------------------------------------------------------------ gen-rewrites! [63] 100.0% [76] 3014(0.4%) 0(0.0%) for-loop ...regime-testing/src/mainloop.rkt:147:11 ??? [90] 100.0% ------------------------------------------------------------------------------------------ for-loop [44] 30.1% ??? [55] 69.9% [77] 1004(0.1%) 302(0.0%) ??? ...tract/private/arrow-higher-order.rkt:145:21 ??? [224] 35.3% ??? [252] 34.7% ------------------------------------------------------------------------------------------ debug-print [64] 100.0% [78] 662(0.1%) 662(0.1%) for-loop ...bie/regime-testing/src/debug.rkt:113:2 ------------------------------------------------------------------------------------------ for-loop [68] 20.8% run [23] 34.1% [79] 767438(99.0%) 0(0.0%) ??? ...lects/racket/private/more-scheme.rkt:261:28 profile-thunk16 [1] 33.9% ??? [180] 30.6% run-improve50 [2] 24.2% baseline-error [96] 4.3% oracle-error [97] 4.0% loop [100] 1.6% prepare-points [101] 1.4% eval-errors [26] 0.1% loop [192] 0.0% ------------------------------------------------------------------------------------------ mk-egraph [189] 0.3% mk-enode-rec! [201] 0.5% substitute-e [72] 36.3% for-loop [54] 62.9% [80] 125628(16.2%) 7584(1.0%) mk-enode! ...ime-testing/src/core/egraph.rkt:102:0 new-enode [95] 92.4% hash-has-key? [139] 1.5% ------------------------------------------------------------------------------------------ match-e [56] 25.2% for-loop [70] 74.8% [81] 60270(7.8%) 6218(0.8%) for-loop ...gime-testing/src/core/ematch.rkt:65:37 match-e [56] 88.0% ??? [104] 2.8% ------------------------------------------------------------------------------------------ for-loop [73] 100.0% [82] 42836(5.5%) 0(0.0%) taylor-alt ...ie/regime-testing/src/glue.rkt:134:0 for-loop [98] 100.0% ------------------------------------------------------------------------------------------ for-loop [42] 17.9% merge-egraph-nodes! [71] 82.1% [83] 39222(5.1%) 3960(0.5%) update-leader! ...esting/src/core/egraph.rkt:218:0 for-loop [99] 83.3% custom-set-union! [106] 3.4% hash-ref! [216] 1.6% make-sequence [145] 0.8% mutable-set [119] 0.8% ------------------------------------------------------------------------------------------ merge-egraph-nodes! [71] 100.0% [84] 18994(2.4%) 12212(1.6%) for-loop ...gime-testing/src/core/egraph.rkt:233:6 update-en-expr [218] 35.7% ------------------------------------------------------------------------------------------ ??? [69] 100.0% [85] 15534(2.0%) 15534(2.0%) ??? ...rivate/unconstrained-domain-arrow.rkt:40:13 ------------------------------------------------------------------------------------------ ??? [69] 100.0% [86] 15232(2.0%) 9568(1.2%) ??? ...regime-testing/src/syntax/syntax.rkt:156:17 ??? [235] 24.6% return/no-unsupplied [237] 12.6% ------------------------------------------------------------------------------------------ list-cartesian-product [87] 22.8% match-e [56] 29.8% for-loop [70] 47.4% [87] 13186(1.7%) 11516(1.5%) list-cartesian-product ...src/core/ematch.rkt:20:0 list-cartesian-product [87] 22.8% for-loop [105] 11.4% ------------------------------------------------------------------------------------------ merge-egraph-nodes! [71] 100.0% [88] 6278(0.8%) 0(0.0%) for-loop ...ime-testing/src/core/egraph.rkt:201:10 merge-egraph-nodes! [71] 84.0% loop! [203] 16.0% ------------------------------------------------------------------------------------------ expression-cost [75] 100.0% [89] 4072(0.5%) 322(0.0%) compile ...e/regime-testing/src/programs.rkt:186:0 hash-ref! [216] 92.1% ------------------------------------------------------------------------------------------ for-loop [76] 100.0% [90] 3014(0.4%) 0(0.0%) ??? ...bie/regime-testing/src/alternative.rkt:88:0 rewriter [103] 100.0% ------------------------------------------------------------------------------------------ ??? [69] 100.0% [91] 2482(0.3%) 0(0.0%) bf* ...ath/private/bigfloat/bigfloat-mpfr.rkt:95:0 ??? [102] 100.0% ------------------------------------------------------------------------------------------ ??? [69] 100.0% [92] 1710(0.2%) 376(0.0%) for-loop ...me-testing/src/syntax/syntax.rkt:531:2 ...higher-order.rkt:342:33 [108] 40.5% bf< [109] 37.5% ------------------------------------------------------------------------------------------ match-e [56] 25.0% for-loop [70] 75.0% [93] 1338(0.2%) 638(0.1%) curry* ...racket/collects/racket/function.rkt:44:2 procedure-arity [107] 52.3% ------------------------------------------------------------------------------------------ merge-egraph-nodes! [71] 100.0% [94] 684(0.1%) 360(0.0%) enode-merge! ...e-testing/src/core/enode.rkt:129:0 adopt-enode! [110] 47.4% ------------------------------------------------------------------------------------------ mk-enode! [80] 100.0% [95] 116116(15.0%) 998(0.1%) new-enode ...egime-testing/src/core/enode.rkt:97:0 type-of-enode-expr [111] 98.0% mutable-set [119] 0.6% set [219] 0.6% ------------------------------------------------------------------------------------------ ??? [79] 100.0% [96] 60784(7.8%) 0(0.0%) baseline-error ...ime-testing/src/points.rkt:291:0 loop [192] 100.0% ------------------------------------------------------------------------------------------ ??? [79] 100.0% [97] 58506(7.5%) 0(0.0%) oracle-error ...egime-testing/src/points.rkt:286:0 for-loop [112] 100.0% ------------------------------------------------------------------------------------------ taylor-alt [82] 100.0% [98] 42836(5.5%) 0(0.0%) for-loop ...rbie/regime-testing/src/glue.rkt:139:6 ??? [174] 100.0% ------------------------------------------------------------------------------------------ update-leader! [83] 100.0% [99] 32672(4.2%) 1022(0.1%) for-loop ...gime-testing/src/core/egraph.rkt:223:6 for-loop [113] 96.9% ------------------------------------------------------------------------------------------ run-improve50 [2] 5.6% ??? [79] 94.4% [100] 24892(3.2%) 0(0.0%) loop ...herbie/regime-testing/src/points.rkt:238:2 make-exacts* [126] 30.5% loop [114] 29.6% loop [127] 27.5% for-loop [117] 12.4% ------------------------------------------------------------------------------------------ run-improve50 [2] 1.7% ??? [79] 98.3% [101] 21034(2.7%) 0(0.0%) prepare-points ...ime-testing/src/points.rkt:214:0 make-exacts* [126] 39.3% loop [114] 28.7% loop [127] 28.5% for-loop [117] 1.7% ------------------------------------------------------------------------------------------ bf* [91] 24.5% ??? [69] 75.5% [102] 10144(1.3%) 654(0.1%) ??? ...-racket/utils/simple-result-arrow.rkt:39:12 bfmul [115] 54.9% bfneg [116] 38.7% ------------------------------------------------------------------------------------------ ??? [90] 28.6% matcher [129] 71.4% [103] 3014(0.4%) 664(0.1%) rewriter ...ime-testing/src/core/matcher.rkt:131:2 for-loop [118] 73.9% *rules* [19] 10.0% f45 [162] 2.8% expression->type [122] 2.4% ------------------------------------------------------------------------------------------ for-loop [81] 100.0% [104] 2014(0.3%) 1032(0.1%) ??? ...bie/regime-testing/src/core/ematch.rkt:50:5 for-loop [120] 48.8% ------------------------------------------------------------------------------------------ list-cartesian-product [87] 100.0% [105] 1670(0.2%) 1670(0.2%) for-loop ...egime-testing/src/core/ematch.rkt:24:8 ------------------------------------------------------------------------------------------ update-leader! [83] 100.0% [106] 1324(0.2%) 1016(0.1%) custom-set-union! ...t/private/set-types.rkt:245:0 for-loop [123] 23.3% ------------------------------------------------------------------------------------------ curry* [93] 100.0% [107] 700(0.1%) 700(0.1%) procedure-arity ...ket/private/norm-arity.rkt:7:27 ------------------------------------------------------------------------------------------ for-loop [92] 100.0% [108] 692(0.1%) 330(0.0%) ...higher-order.rkt:342:33 (unknown source) ??? [194] 52.3% ------------------------------------------------------------------------------------------ for-loop [92] 100.0% [109] 642(0.1%) 0(0.0%) bf< ...th/private/bigfloat/bigfloat-mpfr.rkt:127:4 ??? [121] 100.0% ------------------------------------------------------------------------------------------ enode-merge! [94] 100.0% [110] 324(0.0%) 0(0.0%) adopt-enode! ...e-testing/src/core/enode.rkt:105:0 custom-set-union [217] 100.0% ------------------------------------------------------------------------------------------ new-enode [95] 100.0% [111] 113770(14.7%) 2030(0.3%) type-of-enode-expr ...ting/src/core/enode.rkt:68:0 get-sigs [124] 97.9% parse-loop12 [187] 0.3% ------------------------------------------------------------------------------------------ oracle-error [97] 100.0% [112] 58506(7.5%) 334(0.0%) for-loop ...ie/regime-testing/src/points.rkt:288:2 loop [192] 65.7% map [168] 33.8% ------------------------------------------------------------------------------------------ for-loop [99] 100.0% [113] 31650(4.1%) 312(0.0%) for-loop ...gime-testing/src/core/egraph.rkt:224:8 hash-update! [125] 99.0% ------------------------------------------------------------------------------------------ prepare-points [101] 45.0% loop [100] 55.0% [114] 13408(1.7%) 0(0.0%) loop ...herbie/regime-testing/src/points.rkt:158:2 make-exacts* [126] 54.3% loop [127] 45.7% ------------------------------------------------------------------------------------------ ??? [102] 100.0% [115] 5564(0.7%) 3124(0.4%) bfmul ...-lib/math/private/bigfloat/mpfr.rkt:819:4 ??? [239] 43.9% ------------------------------------------------------------------------------------------ ??? [102] 100.0% [116] 3926(0.5%) 1864(0.2%) bfneg ...-lib/math/private/bigfloat/mpfr.rkt:616:4 ??? [239] 52.5% ------------------------------------------------------------------------------------------ prepare-points [101] 10.5% loop [100] 89.5% [117] 3442(0.4%) 0(0.0%) for-loop ...e/regime-testing/src/points.rkt:255:26 ??? [128] 100.0% ------------------------------------------------------------------------------------------ rewriter [103] 100.0% [118] 3014(0.4%) 0(0.0%) for-loop ...me-testing/src/core/matcher.rkt:134:10 matcher [129] 100.0% ------------------------------------------------------------------------------------------ update-leader! [83] 30.3% new-enode [95] 69.7% [119] 984(0.1%) 686(0.1%) mutable-set .../racket/private/set-types.rkt:999:0 for-loop [131] 30.3% ------------------------------------------------------------------------------------------ ??? [104] 100.0% [120] 982(0.1%) 650(0.1%) for-loop ...egime-testing/src/core/ematch.rkt:51:7 ??? [132] 33.8% ------------------------------------------------------------------------------------------ bf< [109] 100.0% [121] 642(0.1%) 642(0.1%) ??? ...tract/private/arrow-higher-order.rkt:346:33 ------------------------------------------------------------------------------------------ for-loop [130] 50.0% rewriter [103] 50.0% [122] 358(0.0%) 0(0.0%) expression->type ...esting/src/type-check.rkt:45:0 for-loop [130] 50.0% ??? [132] 50.0% ------------------------------------------------------------------------------------------ custom-set-union! [106] 100.0% [123] 308(0.0%) 308(0.0%) for-loop ...cts/racket/private/set-types.rkt:253:4 ------------------------------------------------------------------------------------------ type-of-enode-expr [111] 100.0% [124] 111426(14.4%) 3094(0.4%) get-sigs .../regime-testing/src/type-check.rkt:5:0 ??? [133] 42.4% ??? [180] 29.3% ??? [132] 13.0% ...sting/src/common.rkt:48:2 [135] 12.5% ------------------------------------------------------------------------------------------ for-loop [113] 100.0% [125] 31338(4.0%) 672(0.1%) hash-update! ...ket/private/more-scheme.rkt:362:13 ??? [134] 97.9% ------------------------------------------------------------------------------------------ loop [114] 31.5% loop [100] 32.8% prepare-points [101] 35.8% [126] 23132(3.0%) 0(0.0%) make-exacts* ...egime-testing/src/points.rkt:138:0 loop [192] 86.4% eval-prog [161] 13.6% ------------------------------------------------------------------------------------------ prepare-points [101] 31.6% loop [114] 32.3% loop [100] 36.1% [127] 18984(2.4%) 0(0.0%) loop ...herbie/regime-testing/src/points.rkt:141:4 loop [192] 97.9% ??? [55] 2.1% ------------------------------------------------------------------------------------------ for-loop [117] 100.0% [128] 3442(0.4%) 1214(0.2%) ??? .../herbie/regime-testing/src/points.rkt:230:7 ??? [174] 35.0% for-loop [137] 29.7% ------------------------------------------------------------------------------------------ for-loop [136] 11.7% matcher [129] 37.6% for-loop [118] 50.7% [129] 3014(0.4%) 370(0.0%) matcher ...gime-testing/src/core/matcher.rkt:167:2 rewriter [103] 49.3% matcher [129] 37.6% for-loop [136] 11.7% ------------------------------------------------------------------------------------------ expression->type [122] 100.0% [130] 358(0.0%) 0(0.0%) for-loop ...regime-testing/src/type-check.rkt:52:5 expression->type [122] 100.0% ------------------------------------------------------------------------------------------ mutable-set [119] 100.0% [131] 298(0.0%) 298(0.0%) for-loop ...cts/racket/private/set-types.rkt:943:2 ------------------------------------------------------------------------------------------ setfindf [35] 0.5% taylor [21] 0.5% f45 [162] 0.5% for-loop [120] 0.5% expression->type [122] 0.5% free-variables [272] 1.5% choose-best-alt! [66] 2.4% ??? [252] 2.4% substitute-e [72] 4.0% variable? [14] 15.7% get-sigs [124] 21.4% match-e [56] 21.8% finalize-iter! [59] 28.3% [132] 67596(8.7%) 20780(2.7%) ??? ...contract/private/arrow-val-first.rkt:357:18 hash-has-key? [139] 37.9% for-loop [141] 27.8% ??? [146] 2.4% ??? [252] 0.9% ------------------------------------------------------------------------------------------ f45 [162] 0.7% ??? [174] 0.7% for-loop [74] 4.6% get-sigs [124] 94.0% [133] 50302(6.5%) 4728(0.6%) ??? ...ts/racket/contract/private/arr-i.rkt:804:10 ??? [140] 50.6% for-loop [142] 29.8% get/build-late-neg-projection [159] 8.8% coerce-contract [147] 0.7% ??? [210] 0.7% ------------------------------------------------------------------------------------------ hash-update! [125] 100.0% [134] 30666(4.0%) 650(0.1%) ??? ...e/regime-testing/src/core/egraph.rkt:226:24 for-loop [138] 96.8% make-sequence [145] 1.1% ------------------------------------------------------------------------------------------ ??? [174] 1.4% for-loop [74] 38.3% get-sigs [124] 60.3% [135] 23124(3.0%) 6572(0.8%) ...sting/src/common.rkt:48:2 ...e/arr-i.rkt:906:19 ??? [143] 40.3% ??? [144] 28.4% coerce-contract [147] 2.9% ------------------------------------------------------------------------------------------ matcher [129] 100.0% [136] 1704(0.2%) 0(0.0%) for-loop ...me-testing/src/core/matcher.rkt:183:21 matcher [129] 100.0% ------------------------------------------------------------------------------------------ ??? [128] 100.0% [137] 1022(0.1%) 0(0.0%) for-loop ...ie/regime-testing/src/points.rkt:231:9 ??? [174] 100.0% ------------------------------------------------------------------------------------------ ??? [134] 100.0% [138] 29684(3.8%) 23722(3.1%) for-loop ...ime-testing/src/core/egraph.rkt:227:26 update-en-expr [218] 20.1% ------------------------------------------------------------------------------------------ mk-enode! [80] 7.0% ??? [132] 93.0% [139] 27690(3.6%) 8654(1.1%) hash-has-key? ...et/private/more-scheme.rkt:368:13 ??? [186] 58.1% ??? [150] 10.6% ------------------------------------------------------------------------------------------ ??? [133] 100.0% [140] 25476(3.3%) 2108(0.3%) ??? ...ects/racket/contract/private/hash.rkt:217:4 ??? [148] 91.7% ------------------------------------------------------------------------------------------ ??? [132] 100.0% [141] 19108(2.5%) 0(0.0%) for-loop ...me-testing/src/core/alt-table.rkt:51:2 atab-add-altn [149] 100.0% ------------------------------------------------------------------------------------------ for-loop [193] 18.0% ??? [133] 82.0% [142] 18306(2.4%) 3684(0.5%) for-loop ...racket/contract/private/hash.rkt:239:6 ??? [194] 79.9% ------------------------------------------------------------------------------------------ for-loop [208] 8.8% ??? [194] 12.4% ...sting/src/common.rkt:48:2 [135] 78.9% [143] 9312(1.2%) 2330(0.3%) ??? ...ects/racket/contract/private/list.rkt:342:4 ??? [194] 61.1% ??? [210] 17.8% ??? [252] 8.6% ------------------------------------------------------------------------------------------ ...sting/src/common.rkt:48:2 [135] 100.0% [144] 6560(0.8%) 1676(0.2%) ??? ...s/racket/contract/private/arr-i.rkt:1268:39 ??? [235] 34.5% return/no-unsupplied [237] 29.5% assoc-ref [151] 10.5% ------------------------------------------------------------------------------------------ update-leader! [83] 19.4% ??? [134] 20.0% for-loop [45] 20.2% merge-egraph-nodes! [71] 40.3% [145] 1656(0.2%) 990(0.1%) make-sequence ...ects/racket/private/for.rkt:509:2 ??? [152] 40.2% ------------------------------------------------------------------------------------------ ??? [132] 100.0% [146] 1626(0.2%) 0(0.0%) ??? .../regime-testing/src/core/alt-table.rkt:54:0 ??? [174] 100.0% ------------------------------------------------------------------------------------------ ??? [133] 33.6% ...sting/src/common.rkt:48:2 [135] 66.4% [147] 1024(0.1%) 1024(0.1%) coerce-contract ...contract/private/guts.rkt:257:0 ------------------------------------------------------------------------------------------ ??? [140] 100.0% [148] 23368(3.0%) 1786(0.2%) ??? ...ects/racket/contract/private/list.rkt:169:2 ??? [153] 92.4% ------------------------------------------------------------------------------------------ for-loop [141] 98.3% [149] 19442(2.5%) 0(0.0%) atab-add-altn ...ting/src/core/alt-table.rkt:208:0 best-and-tied-at-points [154] 95.0% override-at-pnts [155] 5.0% ------------------------------------------------------------------------------------------ hash-has-key? [139] 100.0% [150] 2936(0.4%) 2936(0.4%) ??? ...ects/racket/contract/private/hash.rkt:245:7 ------------------------------------------------------------------------------------------ ??? [144] 100.0% [151] 686(0.1%) 686(0.1%) assoc-ref ...collects/racket/private/dict.rkt:56:0 ------------------------------------------------------------------------------------------ make-sequence [145] 100.0% [152] 666(0.1%) 332(0.0%) ??? ...acket/collects/racket/private/for.rkt:429:7 custom-in-set [156] 50.2% ------------------------------------------------------------------------------------------ ??? [148] 100.0% [153] 21582(2.8%) 1358(0.2%) ??? ...ects/racket/contract/private/list.rkt:678:5 for-loop [157] 93.7% ------------------------------------------------------------------------------------------ atab-add-altn [149] 100.0% [154] 18466(2.4%) 626(0.1%) best-and-tied-at-points ...ore/alt-table.rkt:122:0 eval-errors [26] 86.1% alt-errors [158] 10.5% ------------------------------------------------------------------------------------------ atab-add-altn [149] 100.0% [155] 976(0.1%) 0(0.0%) override-at-pnts ...g/src/core/alt-table.rkt:145:0 alt-errors [158] 65.8% eval-errors [26] 34.2% ------------------------------------------------------------------------------------------ ??? [152] 100.0% [156] 334(0.0%) 334(0.0%) custom-in-set ...acket/private/set-types.rkt:577:0 ------------------------------------------------------------------------------------------ ??? [153] 100.0% [157] 20224(2.6%) 5744(0.7%) for-loop ...racket/contract/private/list.rkt:680:9 add-list-context [160] 45.2% get/build-late-neg-projection [159] 26.4% ------------------------------------------------------------------------------------------ override-at-pnts [155] 13.1% option-on-expr [204] 13.8% loop [192] 33.4% best-and-tied-at-points [154] 39.7% [158] 4894(0.6%) 332(0.0%) alt-errors ...ime-testing/src/alternative.rkt:59:0 eval-prog [161] 93.2% ------------------------------------------------------------------------------------------ listof-late-neg-projection [166] 2.3% ??? [164] 14.6% ??? [133] 26.5% for-loop [157] 54.8% [159] 9748(1.3%) 4674(0.6%) get/build-late-neg-projection ...te/guts.rkt:691:0 ??? [163] 43.3% ??? [164] 18.0% listof-late-neg-projection [166] 2.3% ------------------------------------------------------------------------------------------ for-loop [157] 100.0% [160] 9136(1.2%) 9136(1.2%) add-list-context ...ontract/private/list.rkt:752:0 ------------------------------------------------------------------------------------------ loop [192] 3.8% option-on-expr [204] 4.0% make-exacts* [126] 37.7% alt-errors [158] 54.5% [161] 8368(1.1%) 2660(0.3%) eval-prog ...regime-testing/src/programs.rkt:150:0 f45 [162] 35.5% hash-ref! [216] 21.2% ??? [165] 7.6% ??? [167] 3.9% ------------------------------------------------------------------------------------------ for-loop [251] 0.0% composed [196] 0.2% for-loop [199] 0.2% rewriter [103] 0.5% eval-const-expr [37] 0.5% simplify-node [175] 0.6% hash-ref! [216] 1.2% approximate9 [5] 1.2% append-map [181] 1.3% eval-prog [161] 1.5% loop [192] 4.4% parse-loop12 [187] 6.6% f45 [162] 7.1% simplify [46] 13.5% map [168] 16.3% split-table [48] 44.7% [162] 62280(8.0%) 6110(0.8%) f45 ...t/collects/racket/match/compiler.rkt:507:40 loop [192] 38.2% map [168] 35.1% f45 [162] 7.1% option-on-expr [204] 6.5% append-map [181] 1.7% ??? [174] 1.5% taylor-invert [17] 0.8% combine-mterms [170] 0.7% ??? [133] 0.5% ??? [132] 0.5% parse-loop12 [187] 0.4% compose [244] 0.3% composed [196] 0.2% taylor [21] 0.1% loop [172] 0.0% ------------------------------------------------------------------------------------------ get/build-late-neg-projection [159] 100.0% [163] 4738(0.6%) 0(0.0%) ??? ...cts/racket/contract/private/prop.rkt:314:12 build-compound-type-name [169] 100.0% ------------------------------------------------------------------------------------------ get/build-late-neg-projection [159] 100.0% [164] 3410(0.4%) 336(0.0%) ??? ...ects/racket/contract/private/hash.rkt:211:2 get/build-late-neg-projection [159] 90.1% ------------------------------------------------------------------------------------------ eval-const-expr [37] 36.0% eval-prog [161] 64.0% [165] 1000(0.1%) 310(0.0%) ??? ...cket/collects/racket/private/kw.rkt:1633:36 unpack245 [171] 69.0% ------------------------------------------------------------------------------------------ get/build-late-neg-projection [159] 100.0% [166] 682(0.1%) 0(0.0%) listof-late-neg-projection ...ivate/list.rkt:162:0 get/build-late-neg-projection [159] 100.0% ------------------------------------------------------------------------------------------ eval-prog [161] 100.0% [167] 324(0.0%) 0(0.0%) ??? ...collects/racket/private/qq-and-or.rkt:155:9 ??? [173] 100.0% ------------------------------------------------------------------------------------------ loop [192] 0.7% parse-loop12 [187] 2.2% for-loop [112] 41.6% f45 [162] 54.2% [168] 47496(6.1%) 328(0.0%) map ...acket/collects/racket/private/map.rkt:20:13 loop [192] 39.4% f45 [162] 26.9% parse-loop12 [187] 20.7% ??? [7] 4.2% simplify-node [175] 1.7% simplify* [195] 1.6% taylor-sqrt [177] 1.5% make-multiplication-node [212] 0.5% ??? [174] 0.3% ------------------------------------------------------------------------------------------ contract-struct-name [176] 10.6% ??? [163] 89.4% [169] 4738(0.6%) 4034(0.5%) build-compound-type-name ...private/guts.rkt:448:0 contract-struct-name [176] 25.4% ------------------------------------------------------------------------------------------ f45 [162] 100.0% [170] 1668(0.2%) 998(0.1%) combine-mterms ...esting/src/core/reduce.rkt:191:0 for-loop [179] 40.2% ------------------------------------------------------------------------------------------ ??? [165] 100.0% [171] 690(0.1%) 0(0.0%) unpack245 ...private/arrow-higher-order.rkt:354:44 maybe-cons-kwd [178] 100.0% ------------------------------------------------------------------------------------------ f45 [162] 100.0% [172] 332(0.0%) 0(0.0%) loop ...ie/regime-testing/src/core/reduce.rkt:18:2 return/no-unsupplied [237] 100.0% ------------------------------------------------------------------------------------------ ??? [167] 100.0% [173] 324(0.0%) 324(0.0%) ??? ...ollects/racket/private/qq-and-or.rkt:164:21 ------------------------------------------------------------------------------------------ for-loop [185] 0.2% ??? [257] 0.3% jloop [15] 0.3% map [168] 0.4% loop [22] 0.4% loop [270] 0.4% for-loop [137] 1.1% ??? [128] 1.3% ??? [146] 1.4% ??? [7] 2.8% f45 [162] 3.2% for-loop [44] 3.6% ??? [13] 5.7% loop [28] 8.9% loop [192] 10.0% for-loop [98] 15.6% ??? [180] 15.8% ??? [55] 28.6% [174] 93478(12.1%) 6820(0.9%) ??? ...tract/private/arrow-higher-order.rkt:342:33 ??? [180] 58.1% approximate9 [5] 15.6% hash-ref! [216] 14.6% ??? [252] 2.1% best-alt [182] 1.4% ??? [133] 0.4% ...sting/src/common.rkt:48:2 [135] 0.3% arrow-higher-order:lnp [211] 0.2% ------------------------------------------------------------------------------------------ simplify [46] 42.2% map [168] 57.8% [175] 3000(0.4%) 0(0.0%) simplify-node ...-testing/src/core/reduce.rkt:50:0 f45 [162] 55.7% combine-aterms [183] 21.7% gather-additive-terms6 [184] 12.0% append-map [181] 10.7% ------------------------------------------------------------------------------------------ build-compound-type-name [169] 100.0% [176] 1704(0.2%) 704(0.1%) contract-struct-name ...ract/private/prop.rkt:89:0 build-compound-type-name [169] 58.7% ------------------------------------------------------------------------------------------ map [168] 100.0% [177] 694(0.1%) 0(0.0%) taylor-sqrt ...e-testing/src/core/taylor.rkt:346:0 hash-ref! [216] 52.2% first-nonzero-exp [36] 47.8% ------------------------------------------------------------------------------------------ unpack245 [171] 100.0% [178] 690(0.1%) 0(0.0%) maybe-cons-kwd ...ate/arrow-higher-order.rkt:498:0 successfully-got-the-right-kind-of-function [200] 52.2% arrow-higher-order:lnp [211] 47.8% ------------------------------------------------------------------------------------------ combine-mterms [170] 100.0% [179] 670(0.1%) 346(0.0%) for-loop ...gime-testing/src/core/reduce.rkt:195:5 hash-ref! [216] 48.4% ------------------------------------------------------------------------------------------ simplify-alt [52] 0.1% for-loop [74] 1.6% get-sigs [124] 1.9% ??? [180] 5.5% ??? [174] 13.0% for-loop [185] 36.8% ??? [79] 41.0% [180] 588030(75.8%) 36966(4.8%) ??? ...llects/racket/private/norm-define.rkt:53:83 iterate-egraph!13 [3] 37.5% for-loop [185] 36.8% ??? [174] 6.2% ??? [180] 5.5% loop [4] 3.6% ??? [186] 2.4% ??? [235] 1.7% return/no-unsupplied [237] 1.2% mk-egraph [189] 0.1% *simplify-rules* [191] 0.1% ??? [194] 0.0% dict? [268] 0.0% ------------------------------------------------------------------------------------------ simplify-node [175] 12.0% f45 [162] 88.0% [181] 2658(0.3%) 0(0.0%) append-map ...acket/collects/racket/list.rkt:565:2 f45 [162] 51.4% parse-loop12 [187] 31.3% free-variables [272] 17.3% ------------------------------------------------------------------------------------------ ??? [174] 100.0% [182] 1626(0.2%) 0(0.0%) best-alt ...erbie/regime-testing/src/glue.rkt:71:0 loop [188] 79.2% argmins [67] 20.8% ------------------------------------------------------------------------------------------ simplify-node [175] 100.0% [183] 650(0.1%) 0(0.0%) combine-aterms ...esting/src/core/reduce.rkt:183:0 for-loop [190] 100.0% ------------------------------------------------------------------------------------------ simplify-node [175] 100.0% [184] 360(0.0%) 360(0.0%) gather-additive-terms6 ...src/core/reduce.rkt:72:0 ------------------------------------------------------------------------------------------ ??? [180] 100.0% [185] 450628(58.1%) 0(0.0%) for-loop ...me-testing/src/core/simplify.rkt:66:12 ??? [180] 99.9% ??? [174] 0.1% ------------------------------------------------------------------------------------------ hash-has-key? [139] 28.0% ??? [180] 72.0% [186] 57598(7.4%) 19102(2.5%) ??? ...ects/racket/contract/private/list.rkt:737:4 for-loop [193] 48.9% ??? [205] 14.4% arrow-higher-order:lnp [211] 1.8% ??? [194] 1.2% successfully-got-the-right-kind-of-function [200] 0.5% ------------------------------------------------------------------------------------------ type-of-enode-expr [111] 1.0% ??? [69] 1.0% for-loop [39] 1.6% f45 [162] 2.2% append-map [181] 3.3% hash-ref! [216] 3.3% loop [192] 4.3% simplify [46] 20.9% parse-loop12 [187] 23.9% map [168] 38.4% [187] 30340(3.9%) 18428(2.4%) parse-loop12 ...s/racket/match/compiler.rkt:418:15 parse-loop12 [187] 23.9% loop [192] 12.6% f45 [162] 11.0% parse-app [198] 3.7% map [168] 3.1% for-loop [199] 2.2% simplify* [195] 1.7% ??? [202] 1.3% simplify [46] 1.1% make-multiplication-node [212] 1.1% loop [22] 0.6% ------------------------------------------------------------------------------------------ extract-alt [51] 20.6% best-alt [182] 79.4% [188] 1622(0.2%) 0(0.0%) loop ...herbie/regime-testing/src/common.rkt:154:2 composed [196] 100.0% ------------------------------------------------------------------------------------------ ??? [180] 100.0% [189] 996(0.1%) 0(0.0%) mk-egraph ...ime-testing/src/core/egraph.rkt:131:0 mk-enode-rec! [201] 66.9% mk-enode! [80] 33.1% ------------------------------------------------------------------------------------------ combine-aterms [183] 100.0% [190] 650(0.1%) 326(0.0%) for-loop ...gime-testing/src/core/reduce.rkt:185:4 hash-ref! [216] 49.8% ------------------------------------------------------------------------------------------ ??? [180] 100.0% [191] 330(0.0%) 0(0.0%) *simplify-rules* ...ing/src/syntax/rules.rkt:570:0 for-loop [197] 100.0% ------------------------------------------------------------------------------------------ ??? [79] 0.0% make-exacts* [126] 0.2% loop [127] 0.2% map-enodes [34] 0.4% ??? [240] 0.5% match-e [56] 0.6% ??? [55] 1.1% option-on-expr [204] 1.6% for-loop [70] 2.2% map [168] 2.3% for-loop [112] 2.3% loop! [203] 2.9% parse-loop12 [187] 3.1% baseline-error [96] 5.1% f45 [162] 5.3% loop [192] 72.3% [192] 255558(33.0%) 8400(1.1%) loop ...cket/collects/racket/private/map.rkt:25:19 loop [192] 72.3% option-on-expr [204] 5.3% ??? [6] 3.4% eval-errors [26] 2.9% ??? [7] 2.8% loop! [203] 2.8% parse-loop12 [187] 1.7% f45 [162] 1.7% ??? [55] 1.4% ??? [174] 1.1% foldl [209] 1.0% curried [271] 0.6% hash-ref! [216] 0.5% for-loop [16] 0.4% alt-errors [158] 0.3% ulp-difference [62] 0.0% ??? [207] 0.0% eval-prog [161] 0.0% ??? [257] 0.0% map [168] 0.0% ------------------------------------------------------------------------------------------ ??? [186] 100.0% [193] 28194(3.6%) 15120(2.0%) for-loop ...acket/contract/private/list.rkt:743:12 ??? [205] 27.7% for-loop [142] 11.7% ??? [210] 4.6% arrow-higher-order:lnp [211] 2.4% ------------------------------------------------------------------------------------------ ...higher-order.rkt:342:33 [108] 1.5% ??? [186] 2.8% ??? [180] 2.9% ??? [143] 31.4% for-loop [142] 60.1% [194] 24332(3.1%) 3984(0.5%) ??? ...cts/racket/contract/private/list.rkt:187:10 ??? [206] 57.5% for-loop [208] 13.9% ??? [143] 9.5% ??? [224] 1.5% ??? [252] 1.3% ------------------------------------------------------------------------------------------ simplify [46] 17.7% parse-loop12 [187] 18.4% map [168] 64.0% [195] 3780(0.5%) 2018(0.3%) simplify* ...gime-testing/src/core/reduce.rkt:31:0 hash-ref! [216] 28.3% return/no-unsupplied [237] 18.4% ------------------------------------------------------------------------------------------ argmins [67] 22.3% f45 [162] 23.4% loop [188] 54.3% [196] 2986(0.4%) 0(0.0%) composed ...ollects/racket/private/list.rkt:321:16 eval-errors [26] 76.6% f45 [162] 23.4% ------------------------------------------------------------------------------------------ *simplify-rules* [191] 16.9% merge-egraph-nodes! [71] 17.2% *rules* [19] 66.0% [197] 1956(0.3%) 646(0.1%) for-loop ...ie/regime-testing/src/common.rkt:100:9 loop [213] 33.7% ormap [214] 17.1% flag-set? [231] 16.2% ------------------------------------------------------------------------------------------ parse-loop12 [187] 100.0% [198] 1780(0.2%) 1780(0.2%) parse-app .../collects/racket/private/kw.rkt:951:2 ------------------------------------------------------------------------------------------ parse-loop12 [187] 100.0% [199] 996(0.1%) 0(0.0%) for-loop ...gime-testing/src/core/reduce.rkt:99:13 f45 [162] 70.1% make-multiplication-node [212] 29.9% ------------------------------------------------------------------------------------------ ??? [186] 46.4% maybe-cons-kwd [178] 53.6% [200] 672(0.1%) 672(0.1%) successfully-got-the-right-kind-of-function ...9:4 ------------------------------------------------------------------------------------------ mk-egraph [189] 35.2% mk-enode-rec! [201] 64.8% [201] 666(0.1%) 0(0.0%) mk-enode-rec! ...testing/src/core/egraph.rkt:122:0 mk-enode-rec! [201] 64.8% mk-enode! [80] 35.2% ------------------------------------------------------------------------------------------ parse-loop12 [187] 100.0% [202] 662(0.1%) 0(0.0%) ??? ...bie/regime-testing/src/core/reduce.rkt:61:5 pattern-match [232] 51.1% for-loop [215] 48.9% ------------------------------------------------------------------------------------------ for-loop [88] 1.1% merge-egraph-nodes! [71] 12.7% loop [192] 30.4% for-loop [54] 55.8% [203] 74142(9.6%) 14430(1.9%) loop! ...e/regime-testing/src/core/enode.rkt:192:2 custom-set-union [217] 36.7% loop [192] 33.5% set [219] 13.3% update-en-expr [218] 5.0% ------------------------------------------------------------------------------------------ f45 [162] 14.0% loop [192] 86.0% [204] 28920(3.7%) 0(0.0%) option-on-expr ...esting/src/core/regimes.rkt:83:0 loop [192] 39.9% sort-context-on-expr [222] 39.5% eval-errors [26] 8.0% err-lsts->split-indices [228] 4.5% pick-errors [229] 3.6% alt-errors [158] 2.3% eval-prog [161] 1.2% sindices->spoints [233] 1.1% ------------------------------------------------------------------------------------------ for-loop [193] 48.5% ??? [186] 51.5% [205] 16100(2.1%) 8240(1.1%) ??? ...private/unconstrained-domain-arrow.rkt:74:4 make-keyword-procedure [223] 48.8% ------------------------------------------------------------------------------------------ ??? [194] 100.0% [206] 13990(1.8%) 322(0.0%) ??? ...ects/racket/contract/private/list.rkt:686:7 ??? [252] 93.0% ??? [224] 4.7% ------------------------------------------------------------------------------------------ loop [192] 29.2% ??? [55] 70.8% [207] 13206(1.7%) 0(0.0%) ??? ...contract/private/../../private/kw.rkt:761:9 real->bigfloat [221] 100.0% ------------------------------------------------------------------------------------------ ??? [194] 100.0% [208] 3378(0.4%) 1746(0.2%) for-loop ...acket/contract/private/list.rkt:190:15 ??? [143] 48.3% ------------------------------------------------------------------------------------------ for-loop [70] 9.7% loop [192] 90.3% [209] 3364(0.4%) 300(0.0%) foldl ...et/collects/racket/private/list.rkt:229:4 merge2 [225] 91.1% ------------------------------------------------------------------------------------------ ??? [133] 10.2% for-loop [193] 39.5% ??? [143] 50.2% [210] 3304(0.4%) 1338(0.2%) ??? ...ects/racket/contract/private/hash.rkt:222:6 check-hash/c [226] 59.5% ------------------------------------------------------------------------------------------ ??? [174] 13.8% maybe-cons-kwd [178] 14.1% for-loop [193] 28.5% ??? [186] 43.6% [211] 2340(0.3%) 1352(0.2%) arrow-higher-order:lnp ...w-higher-order.rkt:597:7 ??? [230] 42.2% ------------------------------------------------------------------------------------------ for-loop [199] 18.1% parse-loop12 [187] 19.4% map [168] 62.5% [212] 1646(0.2%) 0(0.0%) make-multiplication-node .../core/reduce.rkt:230:0 make-multiplication-subnode [227] 100.0% ------------------------------------------------------------------------------------------ for-loop [197] 100.0% [213] 660(0.1%) 0(0.0%) loop ...ket/collects/racket/private/map.rkt:112:23 list-member? [220] 50.0% flag-set? [231] 50.0% ------------------------------------------------------------------------------------------ for-loop [197] 100.0% [214] 334(0.0%) 0(0.0%) ormap ...et/collects/racket/private/map.rkt:105:13 flag-set? [231] 100.0% ------------------------------------------------------------------------------------------ ??? [202] 100.0% [215] 324(0.0%) 0(0.0%) for-loop ...egime-testing/src/core/reduce.rkt:62:7 pattern-match [232] 100.0% ------------------------------------------------------------------------------------------ for-loop [179] 0.1% for-loop [40] 0.1% for-loop [190] 0.1% simplify* [195] 0.4% ??? [18] 0.6% taylor-sqrt [177] 0.7% ??? [8] 0.9% update-leader! [83] 1.3% ??? [13] 1.5% eval-prog [161] 1.8% loop [192] 1.8% first-nonzero-exp [36] 2.1% for-loop [49] 3.0% ??? [240] 3.7% compile [89] 3.9% for-loop [31] 6.0% loop [28] 6.3% for-loop [29] 6.7% ??? [174] 11.7% for-loop [30] 16.4% ??? [47] 28.6% [216] 49016(6.3%) 5744(0.7%) hash-ref! ...racket/private/more-scheme.rkt:373:13 ??? [9] 28.8% ??? [10] 16.5% simplify [46] 9.6% ??? [8] 9.3% ??? [13] 7.9% ??? [11] 6.3% ??? [240] 5.9% ??? [12] 3.0% f45 [162] 2.6% parse-loop12 [187] 1.8% bigfloat-hash [242] 1.8% ??? [18] 0.7% ??? [20] 0.2% ------------------------------------------------------------------------------------------ adopt-enode! [110] 1.1% loop! [203] 98.9% [217] 30162(3.9%) 8660(1.1%) custom-set-union ...et/private/set-types.rkt:168:0 for-loop [234] 63.1% for-loop [241] 8.2% ------------------------------------------------------------------------------------------ for-loop [138] 29.1% for-loop [84] 33.1% loop! [203] 37.8% [218] 20484(2.6%) 20484(2.6%) update-en-expr ...esting/src/core/egraph.rkt:212:0 ------------------------------------------------------------------------------------------ new-enode [95] 3.2% loop! [203] 96.8% [219] 20446(2.6%) 20446(2.6%) set ...collects/racket/private/set-types.rkt:981:0 ------------------------------------------------------------------------------------------ loop [213] 2.0% for-loop [54] 98.0% [220] 16118(2.1%) 690(0.1%) list-member? ...llects/racket/private/set.rkt:24:0 loop [236] 85.1% member [243] 10.6% ------------------------------------------------------------------------------------------ ??? [207] 100.0% [221] 13206(1.7%) 9694(1.3%) real->bigfloat .../private/bigfloat/mpfr.rkt:451:0 ??? [239] 26.6% ------------------------------------------------------------------------------------------ option-on-expr [204] 100.0% [222] 11412(1.5%) 0(0.0%) sort-context-on-expr ...sting/src/points.rkt:115:0 ??? [238] 100.0% ------------------------------------------------------------------------------------------ ??? [205] 100.0% [223] 7860(1.0%) 7860(1.0%) make-keyword-procedure ...ket/private/kw.rkt:260:4 ------------------------------------------------------------------------------------------ ??? [77] 10.2% ??? [194] 10.3% ??? [206] 19.1% ??? [69] 60.4% [224] 3458(0.4%) 2426(0.3%) ??? ...ects/racket/contract/private/prop.rkt:493:4 ??? [247] 29.8% ------------------------------------------------------------------------------------------ foldl [209] 100.0% [225] 3064(0.4%) 3064(0.4%) merge2 .../regime-testing/src/core/ematch.rkt:31:0 ------------------------------------------------------------------------------------------ ??? [210] 100.0% [226] 1966(0.3%) 1332(0.2%) check-hash/c ...ket/contract/private/hash.rkt:83:0 flat-contract? [249] 32.2% ------------------------------------------------------------------------------------------ make-multiplication-node [212] 100.0% [227] 1646(0.2%) 298(0.0%) make-multiplication-subnode ...re/reduce.rkt:240:0 for-loop [248] 39.7% compose [244] 22.6% group-by68 [250] 19.6% ------------------------------------------------------------------------------------------ option-on-expr [204] 100.0% [228] 1302(0.2%) 0(0.0%) err-lsts->split-indices .../core/regimes.rkt:239:0 add-splitpoint [245] 100.0% ------------------------------------------------------------------------------------------ option-on-expr [204] 100.0% [229] 1036(0.1%) 0(0.0%) pick-errors ...-testing/src/core/regimes.rkt:201:0 for-loop [246] 100.0% ------------------------------------------------------------------------------------------ arrow-higher-order:lnp [211] 100.0% [230] 988(0.1%) 988(0.1%) ??? ...et/contract/private/arity-checking.rkt:19:2 ------------------------------------------------------------------------------------------ for-loop [197] 32.2% loop [213] 33.7% ormap [214] 34.1% [231] 980(0.1%) 0(0.0%) flag-set? ...ie/regime-testing/src/config.rkt:34:0 return/no-unsupplied [237] 65.9% ??? [235] 34.1% ------------------------------------------------------------------------------------------ for-loop [215] 48.9% ??? [202] 51.1% [232] 662(0.1%) 662(0.1%) pattern-match ...testing/src/core/matcher.rkt:48:0 ------------------------------------------------------------------------------------------ option-on-expr [204] 100.0% [233] 316(0.0%) 0(0.0%) sindices->spoints ...ng/src/core/regimes.rkt:136:0 for-loop [251] 100.0% ------------------------------------------------------------------------------------------ custom-set-union [217] 100.0% [234] 19040(2.5%) 3534(0.5%) for-loop ...cts/racket/private/set-types.rkt:174:3 for-loop [253] 81.4% ------------------------------------------------------------------------------------------ flag-set? [231] 1.9% debug7 [65] 3.6% ulp-difference [62] 4.0% ??? [144] 12.5% ??? [86] 20.7% ??? [180] 57.3% [235] 18040(2.3%) 5010(0.6%) ??? ...racket/collects/racket/private/kw.rkt:761:9 ??? [254] 61.3% ??? [252] 11.0% ------------------------------------------------------------------------------------------ list-member? [220] 100.0% [236] 13716(1.8%) 13716(1.8%) loop (unknown source) ------------------------------------------------------------------------------------------ ulp-difference [62] 2.4% loop [172] 2.5% flag-set? [231] 4.8% simplify* [195] 5.2% ??? [86] 14.3% ??? [144] 14.4% ??? [180] 56.4% [237] 13426(1.7%) 9210(1.2%) return/no-unsupplied ...ct/private/arr-i.rkt:533:0 filter [256] 31.4% ------------------------------------------------------------------------------------------ sort-context-on-expr [222] 100.0% [238] 11412(1.5%) 0(0.0%) ??? ...cket/collects/racket/private/sort.rkt:403:3 generic-sort/key [255] 82.4% loop [270] 14.6% ??? [55] 2.9% ------------------------------------------------------------------------------------------ bfneg [116] 25.7% bfmul [115] 30.4% real->bigfloat [221] 43.8% [239] 8014(1.0%) 8014(1.0%) ??? .../share/racket/collects/ffi/unsafe.rkt:861:4 ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [240] 4182(0.5%) 368(0.0%) ??? ...erbie/regime-testing/src/programs.rkt:194:5 hash-ref! [216] 61.5% loop [192] 34.1% ------------------------------------------------------------------------------------------ custom-set-union [217] 100.0% [241] 2462(0.3%) 2462(0.3%) for-loop ...cts/racket/private/set-types.rkt:152:2 ------------------------------------------------------------------------------------------ hash-ref! [216] 100.0% [242] 1772(0.2%) 0(0.0%) bigfloat-hash ...h/private/bigfloat/mpfr.rkt:165:0 bfcanonicalize [259] 57.3% sig+exp->bigfloat [261] 23.3% bigfloat->sig+exp [262] 19.4% ------------------------------------------------------------------------------------------ list-member? [220] 100.0% [243] 1712(0.2%) 1712(0.2%) member (unknown source) ------------------------------------------------------------------------------------------ make-multiplication-subnode [227] 21.9% for-loop [16] 37.9% f45 [162] 40.2% [244] 1696(0.2%) 1696(0.2%) compose ...collects/racket/private/list.rkt:385:12 ------------------------------------------------------------------------------------------ err-lsts->split-indices [228] 100.0% [245] 1302(0.2%) 0(0.0%) add-splitpoint ...sting/src/core/regimes.rkt:253:2 for-loop [258] 100.0% ------------------------------------------------------------------------------------------ pick-errors [229] 100.0% [246] 1036(0.1%) 0(0.0%) for-loop ...ime-testing/src/core/regimes.rkt:203:2 ??? [257] 100.0% ------------------------------------------------------------------------------------------ ??? [224] 100.0% [247] 1032(0.1%) 706(0.1%) ??? ...lects/racket/contract/private/orc.rkt:83:14 ??? [263] 31.6% ------------------------------------------------------------------------------------------ make-multiplication-subnode [227] 100.0% [248] 654(0.1%) 0(0.0%) for-loop ...gime-testing/src/core/reduce.rkt:242:3 make-multiplication-subsubsubnode [260] 100.0% ------------------------------------------------------------------------------------------ check-hash/c [226] 100.0% [249] 634(0.1%) 634(0.1%) flat-contract? .../contract/private/guts.rkt:111:0 ------------------------------------------------------------------------------------------ make-multiplication-subnode [227] 100.0% [250] 322(0.0%) 322(0.0%) group-by68 ...acket/collects/racket/list.rkt:749:0 ------------------------------------------------------------------------------------------ sindices->spoints [233] 100.0% [251] 316(0.0%) 0(0.0%) for-loop ...ime-testing/src/core/regimes.rkt:137:2 f45 [162] 100.0% ------------------------------------------------------------------------------------------ ??? [194] 1.6% ??? [77] 1.8% ??? [132] 3.2% ??? [143] 8.1% ??? [174] 9.9% ??? [235] 10.0% ??? [206] 65.5% [252] 19856(2.6%) 15638(2.0%) ??? ...ects/racket/contract/private/guts.rkt:644:8 ??? [132] 9.9% contract? [265] 8.1% dict? [268] 3.2% ------------------------------------------------------------------------------------------ for-loop [234] 100.0% [253] 15506(2.0%) 15506(2.0%) for-loop ...cts/racket/private/set-types.rkt:178:5 ------------------------------------------------------------------------------------------ ??? [235] 100.0% [254] 11054(1.4%) 10030(1.3%) ??? ...s/racket/contract/private/arr-i.rkt:1233:27 ??? [267] 9.3% ------------------------------------------------------------------------------------------ ??? [238] 100.0% [255] 9408(1.2%) 0(0.0%) generic-sort/key .../racket/private/sort.rkt:185:2 copying-mergesort [264] 96.4% loop [270] 3.6% ------------------------------------------------------------------------------------------ for-loop [70] 6.9% return/no-unsupplied [237] 93.1% [256] 4528(0.6%) 4528(0.6%) filter ...t/collects/racket/private/list.rkt:256:2 ------------------------------------------------------------------------------------------ loop [192] 23.7% for-loop [246] 76.3% [257] 1358(0.2%) 0(0.0%) ??? ...e/regime-testing/src/core/regimes.rkt:194:2 ??? [55] 76.3% ??? [174] 23.7% ------------------------------------------------------------------------------------------ add-splitpoint [245] 100.0% [258] 1302(0.2%) 0(0.0%) for-loop ...ime-testing/src/core/regimes.rkt:255:4 for-loop [266] 100.0% ------------------------------------------------------------------------------------------ bigfloat-hash [242] 100.0% [259] 1016(0.1%) 1016(0.1%) bfcanonicalize .../private/bigfloat/mpfr.rkt:155:0 ------------------------------------------------------------------------------------------ for-loop [248] 100.0% [260] 654(0.1%) 0(0.0%) make-multiplication-subsubsubnode ...uce.rkt:262:0 mterm->expr [269] 100.0% ------------------------------------------------------------------------------------------ bigfloat-hash [242] 100.0% [261] 412(0.1%) 412(0.1%) sig+exp->bigfloat ...ivate/bigfloat/mpfr.rkt:385:0 ------------------------------------------------------------------------------------------ bigfloat-hash [242] 100.0% [262] 344(0.0%) 344(0.0%) bigfloat->sig+exp ...ivate/bigfloat/mpfr.rkt:370:0 ------------------------------------------------------------------------------------------ ??? [247] 100.0% [263] 326(0.0%) 326(0.0%) ??? ...ects/racket/contract/private/list.rkt:141:5 ------------------------------------------------------------------------------------------ generic-sort/key [255] 41.3% copying-mergesort [264] 58.7% [264] 9074(1.2%) 0(0.0%) copying-mergesort ...racket/private/sort.rkt:130:8 copying-mergesort [264] 58.7% loop [270] 28.4% jloop [15] 12.9% ------------------------------------------------------------------------------------------ ??? [252] 100.0% [265] 1604(0.2%) 1604(0.2%) contract? ...acket/contract/private/guts.rkt:109:0 ------------------------------------------------------------------------------------------ for-loop [258] 100.0% [266] 1302(0.2%) 982(0.1%) for-loop ...ime-testing/src/core/regimes.rkt:259:8 curried [271] 24.6% ------------------------------------------------------------------------------------------ ??? [254] 100.0% [267] 1024(0.1%) 1024(0.1%) ??? ...ects/racket/contract/private/prop.rkt:256:4 ------------------------------------------------------------------------------------------ ??? [180] 32.2% ??? [252] 67.8% [268] 950(0.1%) 950(0.1%) dict? ...ollects/racket/private/generic.rkt:146:11 ------------------------------------------------------------------------------------------ make-multiplication-subsubsubnode [260] 100.0% [269] 654(0.1%) 0(0.0%) mterm->expr ...e-testing/src/core/reduce.rkt:270:0 free-variables [272] 100.0% ------------------------------------------------------------------------------------------ generic-sort/key [255] 5.2% ??? [238] 26.1% copying-mergesort [264] 68.7% [270] 6396(0.8%) 0(0.0%) loop ...ket/collects/racket/private/sort.rkt:97:12 ??? [55] 94.4% ??? [174] 5.6% ------------------------------------------------------------------------------------------ for-loop [266] 13.2% for-loop [16] 14.7% loop [192] 72.1% [271] 2430(0.3%) 2430(0.3%) curried ...cket/collects/racket/function.rkt:71:18 ------------------------------------------------------------------------------------------ mterm->expr [269] 39.2% append-map [181] 60.8% [272] 1668(0.2%) 338(0.0%) free-variables ...e-testing/src/programs.rkt:102:0 ??? [132] 59.7% variable? [14] 20.0% ------------------------------------------------------------------------------------------