Logistic regression 2

Average Error: 0.6 → 0.4

Time: 14.6s

Precision: 64

• could not determine a ground truth for program body

  1. x = 1.957382309183684e+68
  2. y = -1.4673813471923265e-106

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\begin{array}{l} \mathbf{if}\;x \le -9.413547838898711118287877808175068139462 \cdot 10^{-6}:\\ \;\;\;\;\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)} - x \cdot y\\ \end{array}\]

C

double f(double x, double y) {
        double r154204 = 1.0;
        double r154205 = x;
        double r154206 = exp(r154205);
        double r154207 = r154204 + r154206;
        double r154208 = log(r154207);
        double r154209 = y;
        double r154210 = r154205 * r154209;
        double r154211 = r154208 - r154210;
        return r154211;
}

↓

double f(double x, double y) {
        double r154212 = x;
        double r154213 = -9.413547838898711e-06;
        bool r154214 = r154212 <= r154213;
        double r154215 = 1.0;
        double r154216 = exp(r154212);
        double r154217 = r154215 + r154216;
        double r154218 = sqrt(r154217);
        double r154219 = log(r154218);
        double r154220 = y;
        double r154221 = r154212 * r154220;
        double r154222 = r154219 - r154221;
        double r154223 = r154219 + r154222;
        double r154224 = 2.0;
        double r154225 = log(r154224);
        double r154226 = 2.0;
        double r154227 = pow(r154225, r154226);
        double r154228 = pow(r154212, r154226);
        double r154229 = r154227 * r154228;
        double r154230 = pow(r154224, r154226);
        double r154231 = r154229 / r154230;
        double r154232 = -1.5;
        double r154233 = r154231 * r154232;
        double r154234 = 1.5;
        double r154235 = r154212 * r154234;
        double r154236 = r154235 + r154225;
        double r154237 = r154227 * r154236;
        double r154238 = 0.75;
        double r154239 = r154238 * r154228;
        double r154240 = r154225 + r154227;
        double r154241 = r154239 * r154240;
        double r154242 = r154237 + r154241;
        double r154243 = r154233 + r154242;
        double r154244 = cbrt(r154243);
        double r154245 = r154244 - r154221;
        double r154246 = r154214 ? r154223 : r154245;
        return r154246;
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.6
Target	0.1
Herbie	0.4

\[\begin{array}{l} \mathbf{if}\;x \le 0.0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Split input into 2 regimes

2. if x < -9.413547838898711e-06

   1. Initial program 0.2

      \[\log \left(1 + e^{x}\right) - x \cdot y\]
   2. Using strategy rm

   3. Applied add-sqr-sqrt 0.2

      \[\leadsto \log \color{blue}{\left(\sqrt{1 + e^{x}} \cdot \sqrt{1 + e^{x}}\right)} - x \cdot y\]

   4. Applied log-prod 0.2

      \[\leadsto \color{blue}{\left(\log \left(\sqrt{1 + e^{x}}\right) + \log \left(\sqrt{1 + e^{x}}\right)\right)} - x \cdot y\]

   5. Applied associate--l+ 0.2

      \[\leadsto \color{blue}{\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)}\]

   if -9.413547838898711e-06 < x

   1. Initial program 0.7

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

   2. Taylor expanded around 0 0.4

      \[\leadsto \log \color{blue}{\left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)} - x \cdot y\]
   3. Using strategy rm

   4. Applied add-cbrt-cube 0.4

      \[\leadsto \color{blue}{\sqrt[3]{\left(\log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right) \cdot \log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)\right) \cdot \log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)}} - x \cdot y\]

   5. Simplified 0.4

      \[\leadsto \sqrt[3]{\color{blue}{{\left(\log \left(\frac{1}{2} \cdot {x}^{2} + \left(x + 2\right)\right)\right)}^{3}}} - x \cdot y\]

   6. Taylor expanded around 0 0.4

      \[\leadsto \sqrt[3]{\color{blue}{\left(0.75 \cdot \left({\left(\log 2\right)}^{2} \cdot {x}^{2}\right) + \left({\left(\log 2\right)}^{3} + \left(1.5 \cdot \left({\left(\log 2\right)}^{2} \cdot x\right) + 0.75 \cdot \left(\log 2 \cdot {x}^{2}\right)\right)\right)\right) - \frac{3}{2} \cdot \frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}}}} - x \cdot y\]

   7. Simplified 0.4

      \[\leadsto \sqrt[3]{\color{blue}{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)}} - x \cdot y\]
3. Recombined 2 regimes into one program.

4. Final simplification 0.4

   \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -9.413547838898711118287877808175068139462 \cdot 10^{-6}:\\ \;\;\;\;\log \left(\sqrt{1 + e^{x}}\right) + \left(\log \left(\sqrt{1 + e^{x}}\right) - x \cdot y\right)\\ \mathbf{else}:\\ \;\;\;\;\sqrt[3]{\frac{{\left(\log 2\right)}^{2} \cdot {x}^{2}}{{2}^{2}} \cdot \frac{-3}{2} + \left({\left(\log 2\right)}^{2} \cdot \left(x \cdot 1.5 + \log 2\right) + \left(0.75 \cdot {x}^{2}\right) \cdot \left(\log 2 + {\left(\log 2\right)}^{2}\right)\right)} - x \cdot y\\ \end{array}\]

Reproduce

herbie shell --seed 2019325 
(FPCore (x y)
  :name "Logistic regression 2"
  :precision binary64

  :herbie-target
  (if (<= x 0.0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))