Python Secrets: How to Stop Hardcoding Them, Where to Put Them, and the Pattern That Scales

Q: How do I load environment variables in Python?

os.environ[VAR_NAME] for required, os.environ.get(VAR_NAME, default) for optional. For local development, use python-dotenv's load_dotenv() to load a dotenv file. For typed config, use pydantic-settings.

Q: Is dotenv secure for secrets?

For local development, yes — the file is gitignored and not on the production server. For production, no — use the platform's secret store.

Q: How do I recover a secret that I committed to git?

Rotate the secret immediately, then use git filter-repo or bfg-repo-cleaner to rewrite the git history and remove the old value. Assume the old value is permanently compromised and audit the access logs.

The three layers of Python secret management: a dotenv file for local development (loaded by python-dotenv), environment variables in production (set by the platform), and a secrets manager (Doppler, 1Password, AWS Secrets Manager) for cross-service secrets and audit logs. The wrong pattern is hardcoded secrets in the source. The right pattern is dotenv locally, env vars in production, and a secret manager when secrets are shared across services or need rotation. The reason “python secrets” is still a top search is that the pattern is the same shape as it was a decade ago (os.environ is still the right answer) but the layers around it have changed.

This post is the three layers, the libraries that load them, the rotation pattern, and the one rule that prevents every “I committed my API key” incident.

The direct answer: three layers, one source at a time
Layer 1: local development with dotenv
Layer 2: production with environment variables
Layer 3: shared secrets with a secret manager
The pattern: pydantic-settings or a 20-line wrapper
The rotation story
The “I committed my secret” recovery
The audit and the principle of least privilege
FAQ

The direct answer: three layers, one source at a time

The right structure for a Python application that reads secrets:

# config.py — the one place secrets are read
import os

# Layer 1: dotenv file (loaded at import time, only in dev)
try:
    from dotenv import load_dotenv
    load_dotenv()
except ImportError:
    pass  # python-dotenv is a dev dependency, not a runtime one

# Layer 2: environment variables (the runtime source of truth)
DATABASE_URL = os.environ["DATABASE_URL"]
DEBUG = os.environ.get("DEBUG", "false") == "true"
API_KEY=os.env...Y")  # required at runtime

# Layer 3: secret manager (only for cross-service secrets)
def get_shared_secret(name):
    import boto3
    client = boto3.client("secretsmanager")
    return client.get_secret_value(SecretId=name)["SecretString"]

The structure is the same regardless of the deployment: secrets are read from os.environ (or a wrapper), and the wrapper decides which layer to pull from. The application code never has a hardcoded secret.

Layer 1: local development with dotenv

For local development, the secret is in a dotenv file at the project root. The file is loaded into the environment by python-dotenv at import time. The library is a dev dependency, not a runtime one (production deploys should not need it).

The setup:

# Install
python3 -m pip install python-dotenv

# Add to requirements-dev.txt, NOT requirements.txt

The dotenv file format is KEY=value, one per line, with comments starting with #. The file should be in .gitignore and never committed. A typical file has the local database URL, the local API key, and a few feature flags:

# .env (NEVER commit this; add to .gitignore)
DATABASE_URL=postgresql://localhost/mydb
API_KEY=local-test-key-here
DEBUG=true

The .env.example pattern: commit a .env.example file with the same shape as .env but with empty or placeholder values. Every developer copies it to .env and fills in the real local values. The .env.example is the documentation; the .env is the local override. Both belong in the repo workflow.

The python-dotenv library has a few options worth knowing:

from dotenv import load_dotenv

# Load the dotenv file (default)
load_dotenv()

# Load a specific file
load_dotenv(".env.production")

# Override existing env vars (default: dont override)
load_dotenv(override=True)

# Load by encoding (for international teams)
load_dotenv(encoding="utf-8")

The override=True option is the right answer when the application is running in a container and the container’s env vars should win over the dotenv file. The default behavior is “env vars from the process win”, which is usually what you want.

For a Docker-based local dev setup, the pattern is to mount the dotenv file as a volume:

# docker-compose.yml
services:
  app:
    build: .
    env_file:
      - .env
    volumes:
      - .:/app

The env_file directive loads the dotenv file into the container’s environment, and the application reads os.environ as usual.

Layer 2: production with environment variables

In production, the env var is set by the platform. The shape is the same regardless of the platform:

# Render / Fly.io / Railway: set in the dashboard, NOT in a dotenv file
# (each platform has a Settings > Environment Variables page)

# Kubernetes: set in the Deployment YAML
env:
  - name: DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: myapp-secrets
        key: database-url

# Docker: pass at run time
docker run -e DATABASE_URL -e API_KEY myapp

# systemd: set in the unit file
Environment=DATABASE_URL=postgresql://...

The application code does not change. The same os.environ["DATABASE_URL"] works in every case, because the env var is set by whatever is starting the process.

The right production pattern:

Set the secret in the platform’s secret store (Render’s env, Fly’s secrets, K8s Secrets, HashiCorp Vault). The platform encrypts the value at rest and injects it into the process’s environment.
Reference the env var in the application with os.environ["..."] (required) or os.environ.get("...", default) (optional). The application never sees the secret value at build time, only at runtime.
Do not log the env var. This is the rule. A print() of the env var in a debug log is a leak. A traceback that includes the env var is a leak. A health check that returns the env var as JSON is a leak. The secret is read once, at startup, and held in memory; it is never echoed.

For a Docker image, the right shape is to inject the env var at run time, not at build time. The docker build should not have access to the production secret; only the docker run should. The pattern is “build the image in CI with no production secrets, then docker run with the secrets injected.”

Layer 3: shared secrets with a secret manager

For secrets that are shared across services (the database password used by three different applications, the API key used by the main service and a background worker), the right answer is a secret manager, not platform env vars. The reason: platform env vars are per-service, and the secret becomes inconsistent when each service has its own copy.

The common choices:

AWS Secrets Manager — the right answer for AWS-heavy stacks. Integrates with IAM, audit logs via CloudTrail, automatic rotation via Lambda.
HashiCorp Vault — the right answer for multi-cloud or self-hosted. Operates as a separate service, with policies for who can read what.
Doppler — the SaaS version of the same idea. The right answer for a team that does not want to operate Vault themselves.
1Password CLI — the right answer for a team that already uses 1Password. The secrets are in the same place as the team’s other credentials.

The integration pattern in Python:

import boto3
import json

def get_database_credentials():
    client = boto3.client("secretsmanager")
    response = client.get_secret_value(SecretId="prod/myapp/db")
    return json.loads(response["SecretString"])

creds = get_database_credentials()
database_url = f"postgresql://{creds['username']}:{creds['password']}@{creds['host']}:5432/{creds['dbname']}"

The secret is fetched at startup, the credentials are held in memory, and the application code never sees the secret value in a log or a config file.

For a team that has not yet adopted a secret manager, the right starting point is the platform’s built-in (Render’s env, Fly’s secrets, K8s Secrets, ECS task definitions). The third-party tool is for when the team’s secret footprint grows past what the platform can comfortably manage.

The pattern: pydantic-settings or a 20-line wrapper

For an application with a few secrets and a few config values, the modern pattern is pydantic-settings:

from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    database_url: str
    api_key: str
    debug: bool = False

    class Config:
        env_file = ".env"

settings = Settings()

The BaseSettings class reads the env vars, validates the types, and raises a clear error if a required var is missing. The env_file = ".env" line tells it to fall back to the dotenv file if the var is not in the process environment. The right answer for an application, where the dependency on pydantic is acceptable.

For a library or a tool where the dependency footprint matters, the 20-line wrapper is the right answer:

import os
from dataclasses import dataclass

@dataclass(frozen=True)
class Settings:
    database_url: str = os.environ["DATABASE_URL"]
    api_key: str = os.environ["API_KEY"]
    debug: bool = os.environ.get("DEBUG", "false").lower() == "true"

settings = Settings()

The decision: use pydantic-settings for an application, use the wrapper for a library or a small tool. The library should not force a pydantic dependency on its users.

The rotation story

Secrets leak. The right response to a leak is rotation, not scrubbing. The rotation story:

Generate a new value in the secret store. For a database password, this is a SQL ALTER USER ... WITH PASSWORD '...'. For an API key, this is a new key in the third-party’s dashboard.
Update the secret store with the new value. The platform (Render, Fly.io, AWS Secrets Manager) updates the value, and the next process restart reads the new value.
Restart the application to pick up the new value. The restart is a redeploy on a managed platform, a kubectl rollout restart on K8s, or a systemctl restart on systemd.
Revoke the old value once the application is confirmed to be using the new one. For a database password, this is the same ALTER USER with the new password; the old password is no longer accepted. For an API key, the third-party’s dashboard has a “revoke” button.

The frequency: for a high-value secret, rotate every 90 days. For a low-value secret, rotate annually. The rotation cadence is a policy; the automated rotation is a Lambda or a cron job that does the steps.

For a hosted platform, the rotation is usually a one-click operation in the dashboard. The platform keeps a version history, the new value is injected at the next deploy, and the old value is retired. The complexity is in the application: it has to be restartable without downtime, and the rotation has to happen in a way that does not require a maintenance window.

The “I committed my secret” recovery

The first response to a committed secret is not scrubbing the commit. It is rotation. The old value is compromised, period; the only fix is a new value.

The recovery sequence:

Rotate the secret immediately. New value in the secret store, application restarted, old value revoked.
Audit the access logs for the service that used the secret. Look for unusual activity in the period since the commit.
Clean the git history with git filter-repo or bfg-repo-cleaner. The cleanup is for the secret value in code, not for the secret’s validity. The cleanup is also risky (it rewrites history for every contributor), and is sometimes skipped in favor of “the secret is rotated, the history is irrelevant.”
Add a pre-commit hook to catch the next one. Tools like gitleaks or trufflehog scan every commit for known secret patterns and reject the commit if a match is found.

The discipline that prevents the next one: a pre-commit hook on every developer machine, plus a CI step that scans every push. The hooks are the two layers of defense. Either alone is bypassable; both together are not.

The audit and the principle of least privilege

The audit: every secret in the codebase, every secret in the deploy config, every secret in the secret store, and which application has access to which. The right cadence is quarterly, with the audit run by someone who did not write the code (a different developer, a security team, an external auditor).

The principle of least privilege: every secret is accessible only to the service that needs it, with the minimum set of permissions. A read-only secret for a reporting service, a write-only secret for an ingestion service, a read-write secret for the main application. The granularity is per-secret, not per-service.

The right pattern for a team that has outgrown “set the secret in the platform’s dashboard”:

Secrets in a secret manager. AWS Secrets Manager, Vault, or Doppler.
Access via IAM or service accounts. Not “everyone in the team can read every secret.”
Audit log of every secret access. Who read what, when, and from where.
Rotation on a schedule. 90 days for high-value, 365 days for low-value.
Pre-commit hook plus CI scan. The defense-in-depth that catches the next mistake.

The hosting calculator at RunxBuild hosting calculator is the way to estimate the cost of a managed platform that includes the secret store, the audit log, and the rotation policy in the price. For a team that wants to own the secret management, the third-party tools above are the path. The wrong answer is “everyone has the production database password in their dotenv file because it is easier.”

FAQ

How do I load environment variables in Python?

os.environ["VAR_NAME"] for required, os.environ.get("VAR_NAME", default) for optional. For local development, use python-dotenv’s load_dotenv() to load a dotenv file. For typed config, use pydantic-settings.

Is dotenv secure for secrets?

For local development, yes (the file is gitignored and not on the production server). For production, no (use the platform’s secret store).

How do I keep API keys out of source code?

Use environment variables, set by the platform or loaded from a dotenv file in development. The application reads os.environ["API_KEY"], not a hardcoded string. The CI scans every commit for known secret patterns.

What is the best secret manager for Python?

For AWS-heavy stacks, AWS Secrets Manager. For multi-cloud, HashiCorp Vault. For a small team that does not want to operate Vault, Doppler or 1Password CLI. For a Python-specific project, pydantic-settings is the modern config library.

How do I rotate a Python secret?

Generate a new value in the secret store, update the application to read the new value (a redeploy is the simplest path), and remove the old value. For frequent rotation, the application can fetch the secret at startup.

How do I recover a secret that I committed to git?

Rotate the secret immediately, then use git filter-repo or bfg-repo-cleaner to rewrite the git history and remove the old value. Assume the old value is permanently compromised and audit the access logs.