Logistic regression 2

Average Error: 0.6 → 0.6

Time: 5.8s

Precision: binary64

Math

\[\log \left(1 + e^{x}\right) - x \cdot y\]

↓

\[\left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \left(\log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right) + 0.5 \cdot \log \left(1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)\right)\right)\right) - x \cdot y\]

FPCore

(FPCore (x y) :precision binary64 (- (log (+ 1.0 (exp x))) (* x y)))

↓

(FPCore (x y)
 :precision binary64
 (-
  (-
   (log (+ 1.0 (pow (exp x) 3.0)))
   (+
    (log (sqrt (+ 1.0 (- (pow (exp x) 2.0) (exp x)))))
    (* 0.5 (log (+ 1.0 (- (pow (exp x) 2.0) (exp x)))))))
  (* x y)))

C

double code(double x, double y) {
	return log(1.0 + exp(x)) - (x * y);
}

↓

double code(double x, double y) {
	return (log(1.0 + pow(exp(x), 3.0)) - (log(sqrt(1.0 + (pow(exp(x), 2.0) - exp(x)))) + (0.5 * log(1.0 + (pow(exp(x), 2.0) - exp(x)))))) - (x * y);
}

Error

Bits error versus x

Bits error versus y

Target

Original	0.6
Target	0.1
Herbie	0.6

\[\begin{array}{l} \mathbf{if}\;x \leq 0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

1. Initial program 0.6

   \[\log \left(1 + e^{x}\right) - x \cdot y\]
2. Using strategy rm

3. Applied flip3-+_binary64_2127 0.6

   \[\leadsto \log \color{blue}{\left(\frac{{1}^{3} + {\left(e^{x}\right)}^{3}}{1 \cdot 1 + \left(e^{x} \cdot e^{x} - 1 \cdot e^{x}\right)}\right)} - x \cdot y\]

4. Applied log-div_binary64_2211 0.6

   \[\leadsto \color{blue}{\left(\log \left({1}^{3} + {\left(e^{x}\right)}^{3}\right) - \log \left(1 \cdot 1 + \left(e^{x} \cdot e^{x} - 1 \cdot e^{x}\right)\right)\right)} - x \cdot y\]

5. Simplified 0.6

   \[\leadsto \left(\color{blue}{\log \left(1 + {\left(e^{x}\right)}^{3}\right)} - \log \left(1 \cdot 1 + \left(e^{x} \cdot e^{x} - 1 \cdot e^{x}\right)\right)\right) - x \cdot y\]

6. Simplified 0.6

   \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \color{blue}{\log \left(1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)\right)}\right) - x \cdot y\]
7. Using strategy rm

8. Applied add-sqr-sqrt_binary64_2146 0.6

   \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \log \color{blue}{\left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)} \cdot \sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right)}\right) - x \cdot y\]

9. Applied log-prod_binary64_2210 0.6

   \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \color{blue}{\left(\log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right) + \log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right)\right)}\right) - x \cdot y\]
10. Using strategy rm

11. Applied pow1/2_binary64_2204 0.6

    \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \left(\log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right) + \log \color{blue}{\left({\left(1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)\right)}^{0.5}\right)}\right)\right) - x \cdot y\]

12. Applied log-pow_binary64_2213 0.6

    \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \left(\log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right) + \color{blue}{0.5 \cdot \log \left(1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)\right)}\right)\right) - x \cdot y\]

13. Final simplification 0.6

    \[\leadsto \left(\log \left(1 + {\left(e^{x}\right)}^{3}\right) - \left(\log \left(\sqrt{1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)}\right) + 0.5 \cdot \log \left(1 + \left({\left(e^{x}\right)}^{2} - e^{x}\right)\right)\right)\right) - x \cdot y\]

Reproduce

herbie shell --seed 2020308 
(FPCore (x y)
  :name "Logistic regression 2"
  :precision binary64

  :herbie-target
  (if (<= x 0.0) (- (log (+ 1.0 (exp x))) (* x y)) (- (log (+ 1.0 (exp (- x)))) (* (- x) (- 1.0 y))))

  (- (log (+ 1.0 (exp x))) (* x y)))