Exponentials in Python: math.exp, **, and the Overflow You Will Hit

math.exp(x) computes e raised to x. x ** y is the general power operator, and math.e ** x is a slower, less accurate way of writing math.exp(x). The thing that actually matters in real code: math.exp raises OverflowError at around x = 710, and NumPy’s version returns inf with a warning instead — which is why every softmax implementation subtracts the maximum before exponentiating.

The functions are trivial. The numerical behaviour at the edges is where the bugs live, and it shows up the first time real data goes through a model.

Table of contents

• The functions
• The overflow, and where it bites
• The log-sum-exp trick
• Arrays and performance
• Exponential decay and growth in practice
• How this fits the rest of the stack
• FAQ

The functions

import math

math.exp(1)          # 2.718281828459045
math.exp(0)          # 1.0
math.exp(-1)         # 0.36787944117144233

# General powers
2 ** 10              # 1024
math.pow(2, 10)      # 1024.0  -- always float

# Do not do this for e**x
math.e ** 2          # 7.3890560989306495
math.exp(2)          # 7.38905609893065   -- correct, and faster

math.e ** x computes a float power of a rounded constant; math.exp(x) calls the C library’s dedicated routine. The results differ in the last digits and exp is the accurate one. Use it.

The related functions are worth knowing because they exist precisely to fix precision problems:

math.expm1(1e-10)    # exp(x) - 1, accurate for tiny x
math.exp(1e-10) - 1  # loses most of the significant digits

math.log1p(1e-10)    # log(1 + x), same reasoning
math.log(x)          # natural log
math.log(x, 10)      # log base 10 -- math.log10(x) is more accurate
math.log2(x)

expm1 matters whenever x is small: exp(1e-10) is 1.0000000001, and subtracting 1 from a float that close to 1 discards nearly all the precision you wanted. Compound interest and continuous rate calculations hit this constantly.

The overflow, and where it bites

import math

math.exp(709)     # 8.218407461554972e+307  -- fine
math.exp(710)     # OverflowError: math range error

import numpy as np
np.exp(710)       # inf, with a RuntimeWarning -- does not raise

The limit is where the result exceeds the largest representable float. The two libraries differ in how they tell you, and NumPy’s silence is the more dangerous of the two — inf propagates quietly through subsequent arithmetic and turns into nan the moment it meets a division or a subtraction.

Underflow is the mirror image and equally consequential:

math.exp(-745)    # 5e-324, the smallest subnormal
math.exp(-746)    # 0.0 -- underflows silently, no error

math.log(math.exp(-746))   # ValueError: math domain error

Underflow to zero raises nothing. The failure surfaces later as a log(0), a division by zero, or a probability of exactly zero where the maths says it should be very small but positive.

The log-sum-exp trick

This is the single most useful thing in the article. Softmax and normalised probabilities both compute a ratio of exponentials, and the ratio is unchanged if you subtract a constant from every input — so subtract the maximum, and the largest exponent becomes exp(0) = 1.

import numpy as np

# Overflows on large logits
def softmax_naive(x):
    e = np.exp(x)
    return e / e.sum()

# Stable: shift so the maximum is 0
def softmax(x):
    shifted = x - np.max(x)
    e = np.exp(shifted)
    return e / e.sum()

logits = np.array([1000.0, 1001.0, 1002.0])
# softmax_naive(logits) -> nan, nan, nan
softmax(logits)          # [0.09003057, 0.24472847, 0.66524096]

The maths is identical; the arithmetic is representable. This is why every framework’s softmax does the subtraction internally, and why a hand-rolled one is a reliable source of nan in training logs.

For summing in log space directly, use the library function rather than writing it:

from scipy.special import logsumexp
logsumexp([1000, 1001, 1002])   # 1002.4076059644443

# scipy also has a stable expit (logistic sigmoid)
from scipy.special import expit
expit(-1000)                     # 0.0, no warning, no overflow

Arrays and performance

import numpy as np

arr = np.array([1.0, 2.0, 3.0])
np.exp(arr)          # elementwise, in compiled code
np.exp2(arr)         # 2 ** x
np.expm1(arr)        # exp(x) - 1, elementwise

# math.exp does not accept arrays
# math.exp(arr)      -> TypeError

# Looping in Python is the slow way
[math.exp(v) for v in arr]

math.exp is faster than np.exp for a single scalar, because NumPy has array-handling overhead. For anything with more than a handful of values, np.exp wins by a wide margin. The crossover is small — a few dozen elements.

To turn NumPy’s silent inf into something you notice:

np.seterr(over='raise', under='warn')
# or scope it
with np.errstate(over='raise'):
    result = np.exp(large_values)

Setting over='raise' during development is a good default. An inf that surfaces at its origin is a five-minute fix; one that surfaces as nan in a loss value three layers later is an afternoon.

Exponential decay and growth in practice

import math

# Exponential backoff with a ceiling
def backoff(attempt, base=0.5, cap=60.0):
    return min(cap, base * (2 ** attempt))

# Continuous compounding
def compound(principal, rate, years):
    return principal * math.exp(rate * years)

# Half-life decay
def remaining(initial, elapsed, half_life):
    return initial * math.exp(-math.log(2) * elapsed / half_life)

The cap in the backoff function is not optional. Without it, 2 ** attempt grows past any sensible retry delay within about twenty attempts, and past the float range not long after. A retry loop that has been running long enough to overflow has other problems, but the cap is one line and prevents a strange failure on top of an existing one.

Add jitter too — random.uniform(0, delay) — or every client that failed at the same moment retries at the same moment, and your backoff has built a synchronised thundering herd.

That retry behaviour is where exponentials meet operations. A backoff that is too aggressive turns a brief upstream blip into a self-inflicted outage; one that is too gentle means every client hammers a service that is trying to recover. Getting it right is easier to see when the runtime logs show the retry pattern next to the failing upstream, which is the view per-deploy logs give you rather than reconstructing it from two systems.

How this fits the rest of the stack

math.exp(x) for e to the x, x ** y for general powers, math.expm1 when x is small. Remember the limits: math.exp raises OverflowError above about 710, NumPy returns inf silently, and underflow to zero is never reported at all.

Subtract the maximum before exponentiating in any softmax or normalised-probability calculation, use logsumexp for log-space sums, and set np.seterr(over='raise') in development. Cap and jitter your exponential backoffs. If you are sizing the service doing the numeric work, the RunxBuild hosting calculator shows service, database, storage, and bandwidth as separate line items.

Useful related references:

• Python Not Equal: != vs is not, and Why the Difference Bites
• Python Integer Division: Why // Floors and Why -7 // 2 Is -4
• Python for Websites: Where It Fits and Where It Does Not
• Python services on RunxBuild

FAQ

How do I calculate e to the power of x in Python?

math.exp(x). Do not use math.e ** x — it raises a rounded float constant to a power instead of calling the dedicated C routine, so it is both slower and less accurate in the final digits.

Why does math.exp raise OverflowError?

Because the result exceeds the largest representable double, which happens at roughly x = 710. NumPy’s np.exp returns inf with a warning instead of raising, which is more dangerous because the infinity propagates quietly and becomes nan later in the calculation.

How do I avoid overflow in a softmax function?

Subtract the maximum value from every input before exponentiating. The ratio of exponentials is unchanged by a constant shift, so the result is mathematically identical while the largest exponent becomes exp(0) = 1 and cannot overflow. Every framework’s built-in softmax does this internally.

What is math.expm1 for?

It computes exp(x) - 1 accurately when x is very small. Calculating math.exp(1e-10) - 1 directly gives a float extremely close to 1 and then subtracts 1, discarding nearly all the significant digits. expm1 avoids that cancellation, which matters in interest and rate calculations.

Should I use math.exp or numpy.exp?

math.exp for a single scalar, since NumPy carries array-handling overhead that dominates for one value. np.exp for arrays, where it operates elementwise in compiled code and is dramatically faster. The crossover is only a few dozen elements.