Logistic regression 2

Average Error: 0.4 → 0.4

Time: 20.1s

Precision: 64

• could not determine a ground truth for program body

  1. x = 1.3315749489519957e+219
  2. y = -145.89507053458865

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -0.00706318010460099:\\ \;\;\;\;\left(\log \left(1 - e^{x} \cdot e^{x}\right) - \log \left(1 - e^{x}\right)\right) - y \cdot x\\ \mathbf{else}:\\ \;\;\;\;\left(x \cdot \left(\frac{1}{8} \cdot x + \frac{1}{2}\right) + \log 2\right) - y \cdot x\\ \end{array}\]

C

double f(double x, double y) {
        double r6527660 = 1.0;
        double r6527661 = x;
        double r6527662 = exp(r6527661);
        double r6527663 = r6527660 + r6527662;
        double r6527664 = log(r6527663);
        double r6527665 = y;
        double r6527666 = r6527661 * r6527665;
        double r6527667 = r6527664 - r6527666;
        return r6527667;
}

↓

double f(double x, double y) {
        double r6527668 = x;
        double r6527669 = -0.00706318010460099;
        bool r6527670 = r6527668 <= r6527669;
        double r6527671 = 1.0;
        double r6527672 = exp(r6527668);
        double r6527673 = r6527672 * r6527672;
        double r6527674 = r6527671 - r6527673;
        double r6527675 = log(r6527674);
        double r6527676 = r6527671 - r6527672;
        double r6527677 = log(r6527676);
        double r6527678 = r6527675 - r6527677;
        double r6527679 = y;
        double r6527680 = r6527679 * r6527668;
        double r6527681 = r6527678 - r6527680;
        double r6527682 = 0.125;
        double r6527683 = r6527682 * r6527668;
        double r6527684 = 0.5;
        double r6527685 = r6527683 + r6527684;
        double r6527686 = r6527668 * r6527685;
        double r6527687 = 2.0;
        double r6527688 = log(r6527687);
        double r6527689 = r6527686 + r6527688;
        double r6527690 = r6527689 - r6527680;
        double r6527691 = r6527670 ? r6527681 : r6527690;
        return r6527691;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.4
Target	0.0
Herbie	0.4

\[\begin{array}{l} \mathbf{if}\;x \le 0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -0.00706318010460099

   1. Initial program 0.1

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied flip-+ 0.1

      \[\leadsto \log \color{blue}{\left(\frac{1 \cdot 1 - e^{x} \cdot e^{x}}{1 - e^{x}}\right)} - x \cdot y\]

   4. Applied log-div 0.1

      \[\leadsto \color{blue}{\left(\log \left(1 \cdot 1 - e^{x} \cdot e^{x}\right) - \log \left(1 - e^{x}\right)\right)} - x \cdot y\]

   5. Simplified 0.1

      \[\leadsto \left(\color{blue}{\log \left(1 - e^{x} \cdot e^{x}\right)} - \log \left(1 - e^{x}\right)\right) - x \cdot y\]

   if -0.00706318010460099 < x

   1. Initial program 0.5

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Taylor expanded around 0 0.5

      \[\leadsto \color{blue}{\left(\log 2 + \left(\frac{1}{2} \cdot x + \frac{1}{8} \cdot {x}^{2}\right)\right)} - x \cdot y\]

   3. Simplified 0.5

      \[\leadsto \color{blue}{\left(x \cdot \left(\frac{1}{2} + x \cdot \frac{1}{8}\right) + \log 2\right)} - x \cdot y\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.4

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -0.00706318010460099:\\ \;\;\;\;\left(\log \left(1 - e^{x} \cdot e^{x}\right) - \log \left(1 - e^{x}\right)\right) - y \cdot x\\ \mathbf{else}:\\ \;\;\;\;\left(x \cdot \left(\frac{1}{8} \cdot x + \frac{1}{2}\right) + \log 2\right) - y \cdot x\\ \end{array}\]

Reproduce

herbie shell --seed 2019146 
(FPCore (x y)
  :name "Logistic regression 2"

  :herbie-target
  (if (<= x 0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))