HTTP 504 Gateway Timeout: Reading the Error as a Map of Your Stack

A 504 Gateway Timeout means a server acting as a proxy asked something upstream for a response and gave up waiting. The important part is who is reporting it: the proxy is alive and healthy enough to write you an error page, so the problem is behind it, not in front of it.

That single fact does most of the diagnostic work. A 504 is not a vague failure — it is a specific claim about which layer of your stack is responding and which one is not. Read it that way and the search space collapses fast.

What the proxy is telling you
The four things that actually cause it
Narrowing it down in about five minutes
Raising the timeout is usually the wrong fix
Getting long work out of the request path
Timeouts worth setting deliberately
How this fits the rest of the stack
FAQ

What the proxy is telling you

Almost every production request path has at least two hops: something at the edge, and something running your code. A 504 is the edge saying it could not get an answer from your code in time.

Contrast it with the neighbouring codes, because the distinction is the whole diagnosis:

502 Bad Gateway — the upstream answered, but with something the proxy could not parse, or the connection was refused or reset. The app is usually crashed or not listening.
504 Gateway Timeout — the upstream accepted the connection and then said nothing within the timeout. The app is usually running and stuck.
503 Service Unavailable — the proxy itself is declining, typically because no healthy backends exist or a rate limit tripped.

So 502 points at a dead process and 504 points at a live one that is blocked. Those are completely different investigations, and mixing them up costs an hour.

The four things that actually cause it

In practice nearly every 504 is one of these, roughly in order of how often they turn up:

A slow database query. The request handler is waiting on a query that is doing a sequential scan, or waiting on a lock held by another transaction. The app is healthy and idle-looking; it is blocked on I/O.
Connection pool exhaustion. Every pooled database connection is checked out, so new requests queue for a connection that never frees up. This one is nasty because the symptom is timeouts on every endpoint, including ones that barely touch the database.
A slow third-party call. Your handler calls an external API with no timeout set, that API hangs, and your request hangs with it. The default timeout in most HTTP clients is far longer than your proxy’s.
Genuinely slow work in the request path. Report generation, image processing, a large export. The work is legitimate; doing it inside an HTTP request is the mistake.

Notice that three of the four are waiting on something else. A 504 is rarely a CPU problem — a pegged CPU usually shows up as uniformly slow responses rather than a clean timeout at the proxy’s limit.

Narrowing it down in about five minutes

The fastest triage is to work out whether the timeouts are on everything or on one route.

Start by hitting a route that does no database work at all — a health check or a static asset served by the app:

curl -w '\ntotal: %{time_total}s status: %{http_code}\n' -o /dev/null -s https://example.com/health

If the health check is fast and one endpoint times out, you have a slow query or a slow external call on that specific path. If the health check also times out, you have pool exhaustion or a saturated worker pool, and the individual endpoint is a red herring.

Next, check whether requests are even arriving at the application. Your runtime logs should show the request starting. If the proxy reports a 504 and the app never logged the request, the connection queued at the proxy and never got a worker — that is a concurrency limit, not a slow handler.

Then look at the database side. In Postgres, the query that tells you the most:

SELECT pid, state, wait_event_type, wait_event,
       now() - query_start AS duration, left(query, 80) AS query
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY duration DESC
LIMIT 10;

Long durations with a wait_event_type of Lock means transactions are blocking each other. Long durations with no wait event means the queries themselves are slow and want an index.

Raising the timeout is usually the wrong fix

The first instinct on a 504 is to increase proxy_read_timeout and move on. Sometimes that is correct. Usually it converts a fast failure into a slow one.

Raising it is right when the work is legitimately long and bounded — a report that honestly takes 45 seconds and always has. Raising it is wrong when the timeout is a symptom of a query that should take 20 milliseconds and takes 40 seconds because it lost an index.

The test worth applying: do you know the number this operation should complete in? If yes and you are above it, fix the operation. If yes and the timeout is below it, raise the timeout. If you do not know the number, find that out before touching config.

There is also a real cost to a long timeout. Every request waiting on it holds a worker and a connection. Raising the timeout from 30 seconds to 300 means a slow upstream can tie up your entire worker pool for five minutes instead of thirty seconds, turning a slow endpoint into a total outage. Short timeouts are a bulkhead.

Getting long work out of the request path

For the fourth cause — work that is genuinely slow — the fix is architectural and not complicated. Stop doing it inside the request.

The shape that works:

The request handler validates the input, writes a job row, and returns 202 Accepted with a job id. This takes milliseconds.
A background worker picks the job up and does the actual work.
The client polls a status endpoint, or you send a webhook when the job finishes.

This is more moving parts than a single slow endpoint, and it is worth it the moment the work exceeds a few seconds. It also gives you something a synchronous endpoint never can: the job survives a deploy, a client disconnect, and a proxy timeout, because its state is in a row rather than in a hanging TCP connection.

The polling loop is the part people over-engineer. For a job that takes under a minute, polling every two seconds is fine and costs almost nothing. Reach for webhooks when the work runs for minutes and the client is another server rather than a browser.

Timeouts worth setting deliberately

Most 504s trace back to a timeout that was never chosen — it was inherited from a default. Four worth setting on purpose:

Proxy read timeout. How long the edge waits for your app. Set slightly above your slowest legitimate endpoint.
Database statement timeout. A per-query ceiling. Set this and a runaway query kills itself instead of holding a connection until the proxy gives up.
HTTP client timeout for outbound calls. Most clients default to no timeout or a very long one. Set it below your proxy’s timeout so a hanging third party fails inside your code, where you can catch it and return something useful.
Connection pool acquisition timeout. How long a request waits for a free database connection before failing. Without it, pool exhaustion presents as a mysterious hang.

The ordering principle: each layer’s timeout should be shorter than the layer in front of it. When the innermost thing fails first, you get a specific error you can log. When the outermost fails first, you get a 504 and no information.

How this fits the rest of the stack

The reason a 504 is frustrating is rarely the timeout itself — it is that the proxy error and the application logs live in different places, so you are correlating timestamps across two systems to work out what was slow. Having the deploy log, the runtime log, and the database on one platform removes that step: the failing request and the query it was waiting on are in the same view. Managed Postgres and MySQL on RunxBuild come with connection limits you can see rather than discover during an incident, and if you are sizing the service and database together, the RunxBuild hosting calculator shows them as separate line items.

Useful related references:

FAQ

What is the difference between a 502 and a 504?

A 502 means the upstream answered with something invalid, or refused the connection — usually a crashed process. A 504 means the upstream accepted the connection and never responded in time — usually a running process blocked on something. Different investigations entirely.

Is a 504 error my fault or the server’s?

Almost always the server side. A 504 is generated by a proxy in front of the application, which means the proxy is healthy and something behind it is not. As a visitor there is nothing to fix beyond retrying.

Should I just increase the timeout to fix a 504?

Only if you know the operation is legitimately long. Otherwise you convert a fast failure into a slow one and let a single slow upstream tie up your whole worker pool. Find out what the operation should take first.

Why do all my endpoints 504 at once?

That pattern points at connection pool exhaustion or a saturated worker pool rather than a slow endpoint. Every request is queuing for a resource that never frees up, so even endpoints that do no database work time out.

How do I stop long jobs from causing 504s?

Move them out of the request path. Have the handler write a job row and return 202 immediately, run the work in a background worker, and let the client poll a status endpoint or receive a webhook.