Logistic regression 2

Average Error: 0.4 → 0.8

Time: 19.1s

Precision: 64

• could not determine a ground truth for program body

  1. x = 9.137530169746875e+200
  2. y = 1.4011542472758516e-71

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -16179825.06434172:\\ \;\;\;\;\mathsf{fma}\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, y \cdot \left(-x\right)\right)\\ \mathbf{elif}\;x \le 8.70864822876016 \cdot 10^{-07}:\\ \;\;\;\;\mathsf{fma}\left(x, \mathsf{fma}\left(x, \frac{1}{8}, \frac{1}{2}\right), \log 2\right) - y \cdot x\\ \mathbf{else}:\\ \;\;\;\;\mathsf{fma}\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, y \cdot \left(-x\right)\right)\\ \end{array}\]

C

double f(double x, double y) {
        double r6138098 = 1.0;
        double r6138099 = x;
        double r6138100 = exp(r6138099);
        double r6138101 = r6138098 + r6138100;
        double r6138102 = log(r6138101);
        double r6138103 = y;
        double r6138104 = r6138099 * r6138103;
        double r6138105 = r6138102 - r6138104;
        return r6138105;
}

↓

double f(double x, double y) {
        double r6138106 = x;
        double r6138107 = -16179825.06434172;
        bool r6138108 = r6138106 <= r6138107;
        double r6138109 = exp(r6138106);
        double r6138110 = log1p(r6138109);
        double r6138111 = cbrt(r6138110);
        double r6138112 = r6138111 * r6138111;
        double r6138113 = y;
        double r6138114 = -r6138106;
        double r6138115 = r6138113 * r6138114;
        double r6138116 = fma(r6138112, r6138111, r6138115);
        double r6138117 = 8.70864822876016e-07;
        bool r6138118 = r6138106 <= r6138117;
        double r6138119 = 0.125;
        double r6138120 = 0.5;
        double r6138121 = fma(r6138106, r6138119, r6138120);
        double r6138122 = 2.0;
        double r6138123 = log(r6138122);
        double r6138124 = fma(r6138106, r6138121, r6138123);
        double r6138125 = r6138113 * r6138106;
        double r6138126 = r6138124 - r6138125;
        double r6138127 = r6138118 ? r6138126 : r6138116;
        double r6138128 = r6138108 ? r6138116 : r6138127;
        return r6138128;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.4
Target	0.0
Herbie	0.8

\[\begin{array}{l} \mathbf{if}\;x \le 0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -16179825.06434172 or 8.70864822876016e-07 < x

   1. Initial program 1.5

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Simplified 1.5

      \[\leadsto \color{blue}{\mathsf{log1p}\left(e^{x}\right) - y \cdot x}\]
   3. Using strategy rm

   4. Applied add-cube-cbrt 1.5

      \[\leadsto \color{blue}{\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}\right) \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}} - y \cdot x\]

   5. Applied fma-neg 1.5

      \[\leadsto \color{blue}{\mathsf{fma}\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, -y \cdot x\right)}\]

   if -16179825.06434172 < x < 8.70864822876016e-07

   1. Initial program 0.0

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Simplified 0.0

      \[\leadsto \color{blue}{\mathsf{log1p}\left(e^{x}\right) - y \cdot x}\]

   3. Taylor expanded around 0 0.5

      \[\leadsto \color{blue}{\left(\log 2 + \left(\frac{1}{2} \cdot x + \frac{1}{8} \cdot {x}^{2}\right)\right)} - y \cdot x\]

   4. Simplified 0.5

      \[\leadsto \color{blue}{\mathsf{fma}\left(x, \mathsf{fma}\left(x, \frac{1}{8}, \frac{1}{2}\right), \log 2\right)} - y \cdot x\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.8

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -16179825.06434172:\\ \;\;\;\;\mathsf{fma}\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, y \cdot \left(-x\right)\right)\\ \mathbf{elif}\;x \le 8.70864822876016 \cdot 10^{-07}:\\ \;\;\;\;\mathsf{fma}\left(x, \mathsf{fma}\left(x, \frac{1}{8}, \frac{1}{2}\right), \log 2\right) - y \cdot x\\ \mathbf{else}:\\ \;\;\;\;\mathsf{fma}\left(\sqrt[3]{\mathsf{log1p}\left(e^{x}\right)} \cdot \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, \sqrt[3]{\mathsf{log1p}\left(e^{x}\right)}, y \cdot \left(-x\right)\right)\\ \end{array}\]

Reproduce

herbie shell --seed 2019165 +o rules:numerics
(FPCore (x y)
  :name "Logistic regression 2"

  :herbie-target
  (if (<= x 0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))