Logistic regression 2

Average Error: 0.4 → 0.5

Time: 15.5s

Precision: 64

• could not determine a ground truth for program body

  1. x = 9.137530169746875e+200
  2. y = 1.4011542472758516e-71

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -1.4952609597112308 \cdot 10^{-05}:\\ \;\;\;\;\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \left(\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}\right) - x \cdot y\\ \mathbf{elif}\;x \le 8.156195176767745 \cdot 10^{-07}:\\ \;\;\;\;\log \left(\left(2 + x\right) \cdot \left(2 + x\right) - \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right) \cdot \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right)\right) - \left(x \cdot y + \log \left(\left(2 + x\right) - \frac{1}{2} \cdot \left(x \cdot x\right)\right)\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \left(\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}\right) - x \cdot y\\ \end{array}\]

C

double f(double x, double y) {
        double r6417546 = 1.0;
        double r6417547 = x;
        double r6417548 = exp(r6417547);
        double r6417549 = r6417546 + r6417548;
        double r6417550 = log(r6417549);
        double r6417551 = y;
        double r6417552 = r6417547 * r6417551;
        double r6417553 = r6417550 - r6417552;
        return r6417553;
}

↓

double f(double x, double y) {
        double r6417554 = x;
        double r6417555 = -1.4952609597112308e-05;
        bool r6417556 = r6417554 <= r6417555;
        double r6417557 = 1.0;
        double r6417558 = exp(r6417554);
        double r6417559 = r6417557 + r6417558;
        double r6417560 = log(r6417559);
        double r6417561 = cbrt(r6417560);
        double r6417562 = r6417561 * r6417561;
        double r6417563 = r6417561 * r6417562;
        double r6417564 = y;
        double r6417565 = r6417554 * r6417564;
        double r6417566 = r6417563 - r6417565;
        double r6417567 = 8.156195176767745e-07;
        bool r6417568 = r6417554 <= r6417567;
        double r6417569 = 2.0;
        double r6417570 = r6417569 + r6417554;
        double r6417571 = r6417570 * r6417570;
        double r6417572 = 0.5;
        double r6417573 = r6417554 * r6417554;
        double r6417574 = r6417572 * r6417573;
        double r6417575 = r6417574 * r6417574;
        double r6417576 = r6417571 - r6417575;
        double r6417577 = log(r6417576);
        double r6417578 = r6417570 - r6417574;
        double r6417579 = log(r6417578);
        double r6417580 = r6417565 + r6417579;
        double r6417581 = r6417577 - r6417580;
        double r6417582 = r6417568 ? r6417581 : r6417566;
        double r6417583 = r6417556 ? r6417566 : r6417582;
        return r6417583;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.4
Target	0.0
Herbie	0.5

\[\begin{array}{l} \mathbf{if}\;x \le 0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -1.4952609597112308e-05 or 8.156195176767745e-07 < x

   1. Initial program 1.5

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied add-cube-cbrt 1.5

      \[\leadsto \color{blue}{\left(\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}\right) \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}} - x \cdot y\]

   if -1.4952609597112308e-05 < x < 8.156195176767745e-07

   1. Initial program 0.0

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Taylor expanded around 0 0.0

      \[\leadsto \log \color{blue}{\left(x + \left(\frac{1}{2} \cdot {x}^{2} + 2\right)\right)} - x \cdot y\]

   3. Simplified 0.0

      \[\leadsto \log \color{blue}{\left(\left(2 + x\right) + \frac{1}{2} \cdot \left(x \cdot x\right)\right)} - x \cdot y\]
   4. Using strategy rm

   5. Applied flip-+ 0.0

      \[\leadsto \log \color{blue}{\left(\frac{\left(2 + x\right) \cdot \left(2 + x\right) - \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right) \cdot \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right)}{\left(2 + x\right) - \frac{1}{2} \cdot \left(x \cdot x\right)}\right)} - x \cdot y\]

   6. Applied log-div 0.0

      \[\leadsto \color{blue}{\left(\log \left(\left(2 + x\right) \cdot \left(2 + x\right) - \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right) \cdot \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right)\right) - \log \left(\left(2 + x\right) - \frac{1}{2} \cdot \left(x \cdot x\right)\right)\right)} - x \cdot y\]

   7. Applied associate--l- 0.0

      \[\leadsto \color{blue}{\log \left(\left(2 + x\right) \cdot \left(2 + x\right) - \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right) \cdot \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right)\right) - \left(\log \left(\left(2 + x\right) - \frac{1}{2} \cdot \left(x \cdot x\right)\right) + x \cdot y\right)}\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.5

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -1.4952609597112308 \cdot 10^{-05}:\\ \;\;\;\;\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \left(\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}\right) - x \cdot y\\ \mathbf{elif}\;x \le 8.156195176767745 \cdot 10^{-07}:\\ \;\;\;\;\log \left(\left(2 + x\right) \cdot \left(2 + x\right) - \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right) \cdot \left(\frac{1}{2} \cdot \left(x \cdot x\right)\right)\right) - \left(x \cdot y + \log \left(\left(2 + x\right) - \frac{1}{2} \cdot \left(x \cdot x\right)\right)\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \left(\sqrt[3]{\log \left(1 + e^{x}\right)} \cdot \sqrt[3]{\log \left(1 + e^{x}\right)}\right) - x \cdot y\\ \end{array}\]

Reproduce

herbie shell --seed 2019165 
(FPCore (x y)
  :name "Logistic regression 2"

  :herbie-target
  (if (<= x 0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))