OpenRouter with Docker Compose: Wiring an LLM Gateway Into Your Stack

OpenRouter is a hosted routing API, not something you run yourself, so there is no image to add to your compose file. What you do add is a service that calls it - and the interesting decisions are about keys, timeouts, and where the gateway logic lives.

People searching this phrase generally want one of two things: a compose stack where an application talks to a model-routing API without hardcoding a provider, or a self-hosted equivalent that does the routing locally. Both are reasonable and they lead to different setups.

Table of contents

• The shape that actually works
• Calling it from the application
• Self-hosted alternatives
• Keys, and not leaking them
• From compose to deployed
• How this fits the rest of the stack
• FAQ

The shape that actually works

Your application calls a routing API over HTTP. The API is OpenAI-compatible, which is the practical reason to use one - you keep the same client library and change a base URL.

services:
  api:
    build: .
    ports:
      - "3000:3000"
    environment:
      OPENAI_BASE_URL: "https://openrouter.ai/api/v1"
      OPENAI_API_KEY: "${OPENROUTER_API_KEY}"
      LLM_MODEL: "${LLM_MODEL:-openai/gpt-4o-mini}"
      DATABASE_URL: "postgres://app:app@db:5432/app"
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:16.2-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: app
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app"]
      interval: 5s
      retries: 5
    volumes:
      - pgdata:/var/lib/postgresql/data

volumes:
  pgdata:

The key is read from the shell environment rather than written into the file, so the compose file is safe to commit. Put the actual value in a gitignored env file that compose reads automatically, or export it in your shell.

# .env - gitignored
OPENROUTER_API_KEY=sk-or-v1-xxxxxxxx
LLM_MODEL=anthropic/claude-sonnet-4

The model is a variable, not a constant in code. That is the whole point of routing through a gateway: switching models is a configuration change and a restart, not a code change and a deploy.

Calling it from the application

Because the API is OpenAI-compatible, the standard client works with a base URL override.

import OpenAI from 'openai'

const client = new OpenAI({
  baseURL: process.env.OPENAI_BASE_URL,
  apiKey: process.env.OPENAI_API_KEY,
})

export async function summarise(text) {
  const res = await client.chat.completions.create({
    model: process.env.LLM_MODEL,
    messages: [
      { role: 'system', content: 'Summarise in two sentences.' },
      { role: 'user', content: text },
    ],
  })
  return res.choices[0].message.content
}

Three production concerns that tutorials skip.

Timeouts. Model calls can take tens of seconds. Without an explicit timeout, a slow upstream holds a request open and your connection pool fills up. Set one on the client and make it shorter than whatever your load balancer allows.

const client = new OpenAI({
  baseURL: process.env.OPENAI_BASE_URL,
  apiKey: process.env.OPENAI_API_KEY,
  timeout: 30_000,
  maxRetries: 2,
})

Failure handling. Upstream providers return rate limits and transient errors. Retry with exponential backoff and jitter on 429 and 5xx responses, and cap the attempts - retrying forever turns a provider blip into your outage.

Cost visibility. Token usage is returned on each response. Log it with a request identifier. Without that, the first sign of a runaway prompt loop is the monthly invoice.

Self-hosted alternatives

If the requirement is genuinely to run the routing yourself - for data handling reasons, or to route to local models - there are open-source gateways that do this, and they do have compose setups.

They generally present an OpenAI-compatible endpoint, accept a configuration of upstream providers with credentials, and route by model name or a policy. Some also route to local inference servers, which is the case where self-hosting genuinely earns its keep: prompts never leave your infrastructure.

services:
  gateway:
    image: your-chosen-gateway:pinned-version
    ports:
      - "18800:18800"
    environment:
      UPSTREAM_KEYS_FILE: /config/keys.yaml
    volumes:
      - ./gateway-config:/config:ro
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:18800/health"]
      interval: 10s

  api:
    build: .
    environment:
      OPENAI_BASE_URL: "http://gateway:18800/v1"
      OPENAI_API_KEY: "local-not-used"
    depends_on:
      gateway:
        condition: service_healthy

Note the base URL uses the compose service name, which resolves on the internal network. The gateway does not need a published port at all unless you want to reach it from the host - removing the ports block keeps it internal, which is the safer default.

Be clear-eyed about what self-hosting buys. If the gateway still calls hosted providers, your prompts still leave your infrastructure - you have added an availability dependency without gaining the privacy property. Self-hosting pays off when it fronts local models, or when you genuinely need routing policy that a hosted service will not implement.

Keys, and not leaking them

An API key with billing attached is the most valuable thing in a small stack, and the ways it escapes are consistent.

• Never in the compose file. Reference an environment variable; keep values in a gitignored env file or the shell.
• Never in a built image. A key in a Dockerfile lives in the image layers and in whatever registry holds it.
• Never in browser-reachable code. A key in frontend JavaScript is public the moment it ships. Calls must go through your backend.
• Never in logs. Do not dump the environment or full request headers on error.
• Rotate on exposure. If a key reaches a repository, rotate it - removing the commit does not un-publish it.

Use compose secrets for anything beyond a development stack, or better, let the deployment platform hold the value. A key injected into the process at start is never on disk and never in an image, and rotating it is an edit and a restart.

On RunxBuild, a service takes environment variables in the dashboard and they are injected at process start, so the key is not in the repository or the image. Deploy logs and runtime logs sit together, which matters when a model call fails and you need to see the error and the deploy that introduced it in one place.

From compose to deployed

Compose is a local development tool. When the stack goes to production, three things change and they are the same three every time.

The database. A Postgres container with a local volume is fine on your machine and is not a production database. It has no backups you have tested, no connection limits you configured, and nobody patching it. A managed instance replaces it - on RunxBuild, managed Postgres or MySQL with backups, connection limits, user management, and private networking.

Configuration. Environment variables move from an env file to the platform, injected at start.

Scaling. Compose runs one of everything. A deployed service needs to handle a traffic spike, which means autoscaling between a floor and a ceiling rather than a fixed container - and model-backed endpoints are exactly the kind that spike, because each request is slow and holds a connection.

What does not change is the application. It still reads a base URL, a key, and a model name from the environment, and still calls an OpenAI-compatible endpoint. That is the payoff for keeping the gateway behind configuration rather than in code.

How this fits the rest of the stack

A model-backed service is an ordinary service with slow requests and a key it must not leak, which makes the deployment questions the familiar ones. Push the repo, hold the key in the platform, read the build and runtime logs in one place, and scale between plans you pick. The RunxBuild hosting calculator shows the service, the managed Postgres, storage, and bandwidth as separate line items so the stack has a number attached.

Useful related references:

• n8n Docker Compose Example: A Setup That Survives Restarts
• docker compose up —build: When You Need It and Why It Sometimes Changes Nothing
• Docker Compose Environment Variables: env_file, .env, and Substitution
• Docker services on RunxBuild

FAQ

Can I self-host OpenRouter with Docker Compose?

No - it is a hosted API, so there is no image to run. What goes in your compose file is a service that calls it, with the base URL and key supplied as environment variables. If you need local routing, use one of the open-source OpenAI-compatible gateways instead.

How do I keep my API key out of the compose file?

Reference an environment variable in the compose file and put the real value in a gitignored env file that compose reads automatically, or export it in your shell. Never bake a key into an image - it persists in the layers and in any registry holding it.

Why does my application hang on model calls?

Almost certainly a missing timeout. Model responses can take tens of seconds, and without an explicit client timeout a slow upstream holds the request open until your connection pool fills. Set a timeout shorter than your load balancer’s limit and cap retries.

Is a self-hosted gateway more private?

Only if it routes to local models. A self-hosted gateway that still calls hosted providers sends your prompts to exactly the same places, so you have added an availability dependency without gaining the privacy property you were after.

What changes when moving this stack from compose to production?

Three things: the database container becomes a managed instance with real backups and connection limits, environment variables move from an env file to the platform, and the single fixed container becomes something that scales - which matters for model-backed endpoints because each request is slow.