Average Error: 0.5 → 0.3
Time: 16.5s
Precision: 64
Internal Precision: 128
\[\log \left(1 + e^{x}\right) - x \cdot y\]
↓
\[\begin{array}{l} \mathbf{if}\;x \le -0.0037090083770916294:\\ \;\;\;\;(\left(\frac{\log_* (1 + e^{x})}{y \cdot x}\right) \cdot \left(y \cdot x\right) + \left(\left(-1 + \frac{y \cdot x}{y \cdot x}\right) - y \cdot x\right))_*\\ \mathbf{else}:\\ \;\;\;\;e^{\log \left((x \cdot \left((\frac{1}{8} \cdot x + \frac{1}{2})_*\right) + \left(\log 2\right))_*\right)} - y \cdot x\\ \end{array}\]

Error

Bits error versus x

Bits error versus y

Target

Original0.5
Target0.1
Herbie0.3
\[\begin{array}{l} \mathbf{if}\;x \le 0:\\ \;\;\;\;\log \left(1 + e^{x}\right) - x \cdot y\\ \mathbf{else}:\\ \;\;\;\;\log \left(1 + e^{-x}\right) - \left(-x\right) \cdot \left(1 - y\right)\\ \end{array}\]

Derivation

  1. Split input into 2 regimes

  2. if x < -0.0037090083770916294

    1. Initial program 0.3

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

    2. Initial simplification0.0

      \[\leadsto \log_* (1 + e^{x}) - y \cdot x\]
    3. Using strategy rm

    4. Applied expm1-log1p-u24.6

      \[\leadsto \color{blue}{(e^{\log_* (1 + \left(\log_* (1 + e^{x}) - y \cdot x\right))} - 1)^*}\]

    5. Taylor expanded around inf 62.5

      \[\leadsto \color{blue}{e^{\log -1 - \left(\log \left(\frac{1}{x}\right) + \log \left(\frac{1}{y}\right)\right)} - \left(\frac{\log \left(e^{x} + 1\right) \cdot e^{\log -1 - \left(\log \left(\frac{1}{x}\right) + \log \left(\frac{1}{y}\right)\right)}}{x \cdot y} + \left(\frac{e^{\log -1 - \left(\log \left(\frac{1}{x}\right) + \log \left(\frac{1}{y}\right)\right)}}{x \cdot y} + 1\right)\right)}\]

    6. Simplified0.0

      \[\leadsto \color{blue}{(\left(\frac{\log_* (1 + e^{x})}{x \cdot y}\right) \cdot \left(x \cdot y\right) + \left(\left(-1 + \frac{x \cdot y}{x \cdot y}\right) - x \cdot y\right))_*}\]

    if -0.0037090083770916294 < x

    1. Initial program 0.5

      \[\log \left(1 + e^{x}\right) - x \cdot y\]

    2. Initial simplification0.5

      \[\leadsto \log_* (1 + e^{x}) - y \cdot x\]

    3. Taylor expanded around 0 0.4

      \[\leadsto \color{blue}{\left(\log 2 + \left(\frac{1}{2} \cdot x + \frac{1}{8} \cdot {x}^{2}\right)\right)} - y \cdot x\]

    4. Simplified0.4

      \[\leadsto \color{blue}{(x \cdot \left((\frac{1}{8} \cdot x + \frac{1}{2})_*\right) + \left(\log 2\right))_*} - y \cdot x\]
    5. Using strategy rm

    6. Applied add-exp-log0.4

      \[\leadsto \color{blue}{e^{\log \left((x \cdot \left((\frac{1}{8} \cdot x + \frac{1}{2})_*\right) + \left(\log 2\right))_*\right)}} - y \cdot x\]
  3. Recombined 2 regimes into one program.

  4. Final simplification0.3

    \[\leadsto \begin{array}{l} \mathbf{if}\;x \le -0.0037090083770916294:\\ \;\;\;\;(\left(\frac{\log_* (1 + e^{x})}{y \cdot x}\right) \cdot \left(y \cdot x\right) + \left(\left(-1 + \frac{y \cdot x}{y \cdot x}\right) - y \cdot x\right))_*\\ \mathbf{else}:\\ \;\;\;\;e^{\log \left((x \cdot \left((\frac{1}{8} \cdot x + \frac{1}{2})_*\right) + \left(\log 2\right))_*\right)} - y \cdot x\\ \end{array}\]

Runtime

Time bar (total: 16.5s)Debug logProfile

BaselineHerbieOracleSpan%
Regimes11.10.30.211.098.8%
herbie shell --seed 2018351 +o rules:numerics
(FPCore (x y)
  :name "Logistic regression 2"

  :herbie-target
  (if (<= x 0) (- (log (+ 1 (exp x))) (* x y)) (- (log (+ 1 (exp (- x)))) (* (- x) (- 1 y))))

  (- (log (+ 1 (exp x))) (* x y)))