Logistic regression 2

Average Error: 0.7 → 0.7

Time: 4.8s

Precision: 64

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\mathsf{expm1}\left(\mathsf{log1p}\left(\log \left(1 + e^{x}\right)\right)\right) - x \cdot y\]

C

double f(double x, double y) {
        double r200790 = 1.0;
        double r200791 = x;
        double r200792 = exp(r200791);
        double r200793 = r200790 + r200792;
        double r200794 = log(r200793);
        double r200795 = y;
        double r200796 = r200791 * r200795;
        double r200797 = r200794 - r200796;
        return r200797;
}

↓

double f(double x, double y) {
        double r200798 = 1.0;
        double r200799 = x;
        double r200800 = exp(r200799);
        double r200801 = r200798 + r200800;
        double r200802 = log(r200801);
        double r200803 = log1p(r200802);
        double r200804 = expm1(r200803);
        double r200805 = y;
        double r200806 = r200799 * r200805;
        double r200807 = r200804 - r200806;
        return r200807;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.7
Target	0.1
Herbie	0.7

\[\begin{array}{l} \mathbf{if}\;x \le 0.0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Initial program 0.7

   \[\log \left(1 + e^{x}\right) - x \cdot y\]
2. Using strategy rm

3. Applied expm1-log1p-u 0.7

   \[\leadsto \color{blue}{\mathsf{expm1}\left(\mathsf{log1p}\left(\log \left(1 + e^{x}\right)\right)\right)} - x \cdot y\]

4. Final simplification 0.7

   \[\leadsto \mathsf{expm1}\left(\mathsf{log1p}\left(\log \left(1 + e^{x}\right)\right)\right) - x \cdot y\]

Reproduce

herbie shell --seed 2020081 +o rules:numerics
(FPCore (x y)
  :name "Logistic regression 2"
  :precision binary64

  :herbie-target
  (if (<= x 0.0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))