Logistic regression 2

Average Error: 0.5 → 0.5

Time: 8.9s

Precision: 64

• Falling back on racket egraph because egg-herbie package not installed

• could not determine a ground truth for program body

  1. x = 2.4112655861716263e+231
  2. y = -5.670122129476186e-07

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -1.694724557776266 \cdot 10^{-5}:\\ \;\;\;\;\log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \mathsf{fma}\left(x, y, \log \left(1 - e^{x}\right)\right)\\ \mathbf{else}:\\ \;\;\;\;\mathsf{fma}\left(x, 0.5 - y, \log 2\right)\\ \end{array}\]

C

double f(double x, double y) {
        double r162495 = 1.0;
        double r162496 = x;
        double r162497 = exp(r162496);
        double r162498 = r162495 + r162497;
        double r162499 = log(r162498);
        double r162500 = y;
        double r162501 = r162496 * r162500;
        double r162502 = r162499 - r162501;
        return r162502;
}

↓

double f(double x, double y) {
        double r162503 = x;
        double r162504 = -1.694724557776266e-05;
        bool r162505 = r162503 <= r162504;
        double r162506 = 1.0;
        double r162507 = r162506 * r162506;
        double r162508 = exp(r162503);
        double r162509 = r162508 * r162508;
        double r162510 = r162507 - r162509;
        double r162511 = log(r162510);
        double r162512 = y;
        double r162513 = r162506 - r162508;
        double r162514 = log(r162513);
        double r162515 = fma(r162503, r162512, r162514);
        double r162516 = r162511 - r162515;
        double r162517 = 0.5;
        double r162518 = r162517 - r162512;
        double r162519 = 2.0;
        double r162520 = log(r162519);
        double r162521 = fma(r162503, r162518, r162520);
        double r162522 = r162505 ? r162516 : r162521;
        return r162522;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.5
Target	0.1
Herbie	0.5

\[\begin{array}{l} \mathbf{if}\;x \le 0.0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -1.694724557776266e-05

   1. Initial program 0.2

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied flip-+ 0.2

      \[\leadsto \log \color{blue}{\left(\frac{1 \cdot 1 - e^{x} \cdot e^{x}}{1 - e^{x}}\right)} - x \cdot y\]

   4. Applied log-div 0.2

      \[\leadsto \color{blue}{\left(\log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \log \left(1 - e^{x}\right)\right)} - x \cdot y\]

   5. Applied associate--l- 0.2

      \[\leadsto \color{blue}{\log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \left(\log \left(1 - e^{x}\right) + x \cdot y\right)}\]

   6. Simplified 0.2

      \[\leadsto \log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \color{blue}{\mathsf{fma}\left(x, y, \log \left(1 - e^{x}\right)\right)}\]

   if -1.694724557776266e-05 < x

   1. Initial program 0.7

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied pow1 0.7

      \[\leadsto \log \color{blue}{\left({\left(1 + e^{x}\right)}^{1}\right)} - x \cdot y\]

   4. Applied log-pow 0.7

      \[\leadsto \color{blue}{1 \cdot \log \left(1 + e^{x}\right)} - x \cdot y\]

   5. Applied prod-diff 0.7

      \[\leadsto \color{blue}{\mathsf{fma}\left(1, \log \left(1 + e^{x}\right), -y \cdot x\right) + \mathsf{fma}\left(-y, x, y \cdot x\right)}\]

   6. Simplified 0.7

      \[\leadsto \color{blue}{\mathsf{fma}\left(y, -x, \log \left(e^{x} + 1\right)\right)} + \mathsf{fma}\left(-y, x, y \cdot x\right)\]

   7. Simplified 0.7

      \[\leadsto \mathsf{fma}\left(y, -x, \log \left(e^{x} + 1\right)\right) + \color{blue}{0}\]

   8. Taylor expanded around 0 0.6

      \[\leadsto \color{blue}{\left(\left(\log 2 + 0.5 \cdot x\right) - x \cdot y\right)} + 0\]

   9. Simplified 0.6

      \[\leadsto \color{blue}{\mathsf{fma}\left(x, 0.5 - y, \log 2\right)} + 0\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.5

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -1.694724557776266 \cdot 10^{-5}:\\ \;\;\;\;\log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \mathsf{fma}\left(x, y, \log \left(1 - e^{x}\right)\right)\\ \mathbf{else}:\\ \;\;\;\;\mathsf{fma}\left(x, 0.5 - y, \log 2\right)\\ \end{array}\]

Reproduce

herbie shell --seed 2020043 +o rules:numerics
(FPCore (x y)
  :name "Logistic regression 2"
  :precision binary64

  :herbie-target
  (if (<= x 0.0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))