Python re.sub(): Replace With a Function, Not Just a String

re.sub(pattern, replacement, string) replaces every place a regex matches with a replacement, and the thing most people never learn is that the replacement can be a function. When it is a string, you get find-and-replace with backreferences. When it is a function, re.sub calls it once per match and substitutes whatever it returns - which means you can compute the replacement from the matched text. That single feature is the difference between escaping a string with a regex and transforming it.

If you only ever pass a literal string as the replacement, you are using a fraction of the function, and probably reaching for a loop when re.sub already does the job.

The basic form and backreferences
The function replacement is the real feature
count and flags
re.sub vs str.replace
A practical one: collapsing whitespace
How this fits the rest of the stack
FAQ

The basic form and backreferences

import re

re.sub(r"apple", "grape", "apple orange apple")
# 'grape orange grape' - every match replaced

# Backreferences: \1 is the first capture group
re.sub(r"(\w+)@(\w+)", r"\2.\1", "user@example")
# 'example.user'

\1, \2 and so on refer to capture groups from the pattern. Named groups work too: (?P<user>\w+) is referenced as \g<user> in the replacement. Use raw strings (r"...") for both the pattern and the replacement so the backslashes mean what you think.

The function replacement is the real feature

Pass a function and re.sub calls it with each match object, using the return value as the replacement.

import re

def shout(m):
    return m.group(0).upper()

re.sub(r"\b\w{4,}\b", shout, "the quick brown fox")
# 'the QUICK BROWN fox' - only words of 4+ letters, uppercased

def bump(m):
    return str(int(m.group(0)) + 1)

re.sub(r"\d+", bump, "v1 to v2 to v9")
# 'v2 to v3 to v10'

This is where re.sub stops being find-and-replace. You can look at the match, decide, compute, and return - templating, number bumping, currency formatting, redaction - all in one pass and without building the output string by hand.

count and flags

re.sub(r"x", "y", "xxxx", count=2)     # 'yyxx' - only first 2
re.sub(r"hello", "hi", text, flags=re.IGNORECASE)

count limits how many replacements happen (0 means all, the default). flags takes the usual re flags - re.IGNORECASE, re.MULTILINE, re.DOTALL. If you are calling the same substitution in a loop, re.compile the pattern once and call .sub on the compiled object; it avoids recompiling the regex on every call.

re.sub vs str.replace

If your replacement is a fixed literal with no pattern, str.replace is faster and clearer:

"a-b-c".replace("-", "_")     # 'a_b_c' - no regex needed

Reach for re.sub when the thing you are matching is a pattern (any digit, any whitespace run, a word boundary), when you need backreferences, or when you want a function replacement. Using a regex to replace a plain fixed string is over-engineering, and it is slower. Match the tool to the shape of the problem.

A practical one: collapsing whitespace

import re

messy = "  too    many\n\n spaces "
re.sub(r"\s+", " ", messy).strip()
# 'too many spaces'

\s+ matches any run of whitespace - spaces, tabs, newlines - and replacing it with a single space is the canonical way to normalize scraped or user-entered text. It is two lines, it is correct on every whitespace character, and it is the example that makes the case for regex replacement better than any explanation.

How this fits the rest of the stack

A regex that transforms text in one pass is the same instinct that keeps infrastructure honest: do the work once, in the right place, instead of scattering it. When a service does text processing at request time, that work shows up as CPU, and CPU shows up on the bill. The RunxBuild hosting calculator puts compute, memory, and bandwidth on the table as separate numbers so you can see the cost of the work before it runs, and the RunxBuild dashboard shows the real usage once it is deployed.

Useful related references:

FAQ

How do I use a function as the replacement in re.sub?

Pass a function instead of a string as the second argument. re.sub calls it once per match with the match object, and substitutes the function’s return value. For example def bump(m): return str(int(m.group(0)) + 1) used with re.sub(r’\d+’, bump, text) increments every number in the string. This lets you compute each replacement from the matched text.

What is the difference between re.sub and str.replace?

str.replace swaps a fixed literal substring and needs no regex, so it is faster and clearer when the target is a plain string. re.sub matches a regular-expression pattern and supports backreferences, flags, a replacement count, and function replacements. Use re.sub when you are matching a pattern rather than a fixed string; use str.replace otherwise.

How do backreferences work in re.sub?

In the replacement string, \1, \2, and so on refer to the corresponding capture groups from the pattern; \g refers to a named group. re.sub(r’(\w+)@(\w+)’, r’\2.\1’, ‘user@example’) yields ‘example.user’. Use raw strings for both the pattern and replacement so the backslashes are not interpreted by Python first.

How do I limit how many replacements re.sub makes?

Pass the count argument. re.sub(pattern, repl, text, count=2) replaces only the first two matches; the default of 0 replaces all of them. Combine it with flags such as re.IGNORECASE or re.MULTILINE as needed. If you run the same substitution repeatedly, compile the pattern once with re.compile for efficiency.

How do I collapse multiple spaces into one with regex?

Use re.sub(r’\s+’, ’ ’, text).strip(). The \s+ pattern matches any run of whitespace including tabs and newlines, replaces it with a single space, and strip removes leading and trailing whitespace. It is the standard way to normalize messy scraped or user-entered text in one pass.