Python Print to stderr: And Why Logging Is Usually Better

Q: How do I print to stderr in Python?

Pass the file argument: print("message", file=sys.stderr). This is preferred over sys.stderr.write because it handles multiple arguments, separators, and the trailing newline.

Q: Does logging write to stdout or stderr by default?

stderr. logging.basicConfig defaults to a StreamHandler on sys.stderr, which is correct: log records are diagnostics, not program output.

Pass file=sys.stderr to print. The more useful question is when stderr is the right destination at all, and why your output disappears when the program runs in a container.

The mechanics take one line. What is worth more attention is the reasoning: stdout and stderr exist as separate streams so that a program’s actual output can be piped somewhere while its diagnostics remain visible.

Get that separation right and your program composes with other tools. Get it wrong and you have written something that cannot be used in a pipeline without mangling the data.

The ways to do it
Which stream should get what
Buffering, and where the output went
Why logging is usually the better tool
Exit codes belong with error messages
How this fits the rest of the stack
FAQ

The ways to do it

import sys

# The idiomatic form
print("could not open config file", file=sys.stderr)

# Direct write -- no automatic newline, no separator handling
sys.stderr.write("could not open config file\n")

# Force a flush immediately
print("progress: 40%", file=sys.stderr, flush=True)

# A small helper worth having in any CLI
def warn(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

warn("falling back to defaults")

print(file=sys.stderr) is the version to use. It handles multiple arguments, separators, and the trailing newline, where sys.stderr.write takes exactly one string and adds nothing.

One detail that catches people: sys.stderr should be looked up at call time, not captured at import. Test frameworks and capture utilities replace sys.stderr at runtime, and a module that stored a reference to the original object writes past the capture.

Which stream should get what

The convention is old, widely honoured, and worth following precisely because tools depend on it.

stdout — the program’s actual output. The data another program would consume. Results, generated content, the answer.
stderr — everything else. Errors, warnings, progress indicators, diagnostics, prompts, and anything a human reads while the program runs.

The test is whether the output would be right in the middle of a pipe. If someone runs yourtool | grep foo, everything on stdout becomes grep’s input. A progress message on stdout corrupts that data; on stderr it stays on the terminal where it belongs.

# stderr keeps diagnostics visible while stdout is redirected
python report.py > results.csv
# progress and warnings still print to the terminal

# Discard diagnostics, keep the data
python report.py 2>/dev/null > results.csv

# Keep only the diagnostics
python report.py > /dev/null

This is also why a program that prints its errors to stdout is genuinely broken rather than merely untidy — the caller has no way to separate failure messages from results.

Buffering, and where the output went

The single most common confusion with these streams. Python buffers stdout and stderr differently, and the behaviour changes depending on whether the output is a terminal.

Terminal — stdout is line-buffered, flushing on each newline. Output appears as you expect.
Pipe or file — stdout switches to block buffering, typically 8 KB. Nothing appears until the buffer fills or the program exits.
stderr — unbuffered in Python 2, line-buffered in Python 3, and importantly it is not switched to block buffering when redirected.

This is why output vanishes from Docker logs. Your container’s stdout is a pipe, so Python block-buffers it, and a long-running process shows nothing for minutes — then dumps everything at once, or loses it entirely if the container is killed.

# Fix it at the process level
python -u script.py

# Or with an environment variable, which is the Dockerfile-friendly form
PYTHONUNBUFFERED=1 python script.py

# In a Dockerfile -- one line that prevents a lot of confusion
ENV PYTHONUNBUFFERED=1

That single environment variable belongs in essentially every Python Dockerfile. Without it, logs are delayed and unreliable in exactly the situation where you most need them.

Because stdout and stderr are buffered differently, interleaved writes can also appear out of order when redirected to the same file. If ordering matters, write to one stream or use logging with a single handler.

Why logging is usually the better tool

For anything longer-lived than a small script, print to stderr is a step on the way to logging rather than a destination.

import logging
import sys

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(name)s %(message)s",
    stream=sys.stderr,
)

log = logging.getLogger(__name__)

log.debug("connection pool size is 12")
log.info("processed 4210 records")
log.warning("retrying after timeout")
log.error("could not reach the upstream service")

try:
    risky()
except Exception:
    log.exception("unhandled failure")   # includes the traceback

What logging gives you that print does not: severity levels you can filter at runtime, timestamps and module names without repeating yourself, per-module configuration, multiple destinations at once, and structured output for log aggregation.

logging.exception deserves special mention. Called inside an exception handler it records the full traceback automatically, which is the difference between an alert that says something failed and one that tells you where.

Note that logging.basicConfig already defaults to stderr, which is the correct choice and reinforces the convention.

The reasonable rule: print to stderr in short scripts and CLI tools where the message is genuinely for the person running it. Logging for anything that runs unattended, in a container, or as a service.

Exit codes belong with error messages

Writing to stderr tells a human something went wrong. The exit code tells the calling program, and only one of those is machine-readable.

import sys

def main():
    try:
        config = load_config()
    except FileNotFoundError as exc:
        print(f"config not found: {exc}", file=sys.stderr)
        return 1
    except ValueError as exc:
        print(f"invalid config: {exc}", file=sys.stderr)
        return 2

    run(config)
    return 0

if __name__ == "__main__":
    sys.exit(main())

A main() that returns an integer passed to sys.exit is clean and testable — you can call main() in a test and assert on the return value without catching SystemExit.

The convention is 0 for success and non-zero for failure. Distinct codes for distinct failure modes are useful in scripts, and by convention codes above 125 are reserved for the shell’s own signalling.

A program that prints an error to stderr and exits 0 is a genuine bug. Every && chain and every CI pipeline treats it as a success, which means the failure propagates silently — the worst possible outcome.

How this fits the rest of the stack

Stream discipline stops being cosmetic the moment your code runs somewhere you cannot attach a terminal. Log collectors read stdout and stderr, and a service that buffers its output or reports failures on the wrong stream is one you cannot debug from the outside. RunxBuild captures both streams from deployed services into the same log view as builds and deploys, so a stderr line from a crashed worker is where you would look for it — and the RunxBuild hosting calculator prices those services with that observability included.

Useful related references:

FAQ

How do I print to stderr in Python?

Pass the file argument: print("message", file=sys.stderr). This is preferred over sys.stderr.write because it handles multiple arguments, separators, and the trailing newline.

What is the difference between stdout and stderr?

stdout carries the program’s actual output — the data another program would consume. stderr carries diagnostics: errors, warnings, and progress. The separation means output can be redirected while messages stay visible.

Why don’t my Python logs appear in Docker?

Python block-buffers stdout when it is a pipe rather than a terminal, so output is held until the buffer fills. Set PYTHONUNBUFFERED=1 in the Dockerfile, or run with python -u.

Should I use print or logging for errors?

print to stderr is fine for short scripts and CLI tools where a human reads the message. Use logging for anything unattended or long-running — you get levels, timestamps, module names, and configurable destinations.

Does logging write to stdout or stderr by default?

stderr. logging.basicConfig defaults to a StreamHandler on sys.stderr, which is correct: log records are diagnostics, not program output.