Logistic regression 2

Average Error: 0.6 → 0.4

Time: 14.0s

Precision: 64

• could not determine a ground truth for program body

  1. x = 1.957382309183684e+68
  2. y = -1.4673813471923265e-106

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -9.413547838898711118287877808175068139462 \cdot 10^{-6}:\\ \;\;\;\;\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)} - x \cdot y\\ \end{array}\]

C

double f(double x, double y) {
        double r128577 = 1.0;
        double r128578 = x;
        double r128579 = exp(r128578);
        double r128580 = r128577 + r128579;
        double r128581 = log(r128580);
        double r128582 = y;
        double r128583 = r128578 * r128582;
        double r128584 = r128581 - r128583;
        return r128584;
}

↓

double f(double x, double y) {
        double r128585 = x;
        double r128586 = -9.413547838898711e-06;
        bool r128587 = r128585 <= r128586;
        double r128588 = 1.0;
        double r128589 = exp(r128585);
        double r128590 = r128588 + r128589;
        double r128591 = sqrt(r128590);
        double r128592 = log(r128591);
        double r128593 = y;
        double r128594 = r128585 * r128593;
        double r128595 = r128592 - r128594;
        double r128596 = r128592 + r128595;
        double r128597 = 2.0;
        double r128598 = log(r128597);
        double r128599 = 2.0;
        double r128600 = pow(r128598, r128599);
        double r128601 = pow(r128585, r128599);
        double r128602 = r128600 * r128601;
        double r128603 = pow(r128597, r128599);
        double r128604 = r128602 / r128603;
        double r128605 = -1.5;
        double r128606 = r128604 * r128605;
        double r128607 = 1.5;
        double r128608 = r128585 * r128607;
        double r128609 = r128608 + r128598;
        double r128610 = r128600 * r128609;
        double r128611 = 0.75;
        double r128612 = r128611 * r128601;
        double r128613 = r128598 + r128600;
        double r128614 = r128612 * r128613;
        double r128615 = r128610 + r128614;
        double r128616 = r128606 + r128615;
        double r128617 = cbrt(r128616);
        double r128618 = r128617 - r128594;
        double r128619 = r128587 ? r128596 : r128618;
        return r128619;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.6
Target	0.1
Herbie	0.4

\[\begin{array}{l} \mathbf{if}\;x \le 0.0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -9.413547838898711e-06

   1. Initial program 0.2

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied add-sqr-sqrt 0.2

      \[\leadsto \log \color{blue}{\left(\sqrt{1 + e^{x}} \cdot \sqrt{1 + e^{x}}\right)} - x \cdot y\]

   4. Applied log-prod 0.2

      \[\leadsto \color{blue}{\left(\log \left(\sqrt{1 + e^{x}}\right) + \log \left(\sqrt{1 + e^{x}}\right)\right)} - x \cdot y\]

   5. Applied associate--l+ 0.2

      \[\leadsto \color{blue}{\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)}\]

   if -9.413547838898711e-06 < x

   1. Initial program 0.7

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Taylor expanded around 0 0.4

      \[\leadsto \log \color{blue}{\left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)} - x \cdot y\]
   3. Using strategy rm

   4. Applied add-cbrt-cube 0.4

      \[\leadsto \color{blue}{\sqrt[3]{\left(\log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right) \cdot \log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)\right) \cdot \log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)}} - x \cdot y\]

   5. Simplified 0.4

      \[\leadsto \sqrt[3]{\color{blue}{{\left(\log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)\right)}^{3}}} - x \cdot y\]

   6. Taylor expanded around 0 0.4

      \[\leadsto \sqrt[3]{\color{blue}{\left(0.75 \cdot \left({\left(\log 2\right)}^{2} \cdot {x}^{2}\right) + \left({\left(\log 2\right)}^{3} + \left(1.5 \cdot \left({\left(\log 2\right)}^{2} \cdot x\right) + 0.75 \cdot \left(\log 2 \cdot {x}^{2}\right)\right)\right)\right) - \frac{3}{2} \cdot \frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}}}} - x \cdot y\]

   7. Simplified 0.4

      \[\leadsto \sqrt[3]{\color{blue}{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)}} - x \cdot y\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.4

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -9.413547838898711118287877808175068139462 \cdot 10^{-6}:\\ \;\;\;\;\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)} - x \cdot y\\ \end{array}\]

Reproduce

herbie shell --seed 2019325 
(FPCore (x y)
  :name "Logistic regression 2"
  :precision binary64

  :herbie-target
  (if (<= x 0.0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))