A load balancer is honest machinery: it will tell you what is wrong if you read the right log line. The hard part is knowing which log line is the right one, because the load balancer, the targets, and the network between them each produce their own. Once you learn to read all three in order, a 502 stops being a mystery and starts being a checklist.
The shape of the problem in 2026 has not changed much from the shape it had in 2018: the load balancer returns a 502, the user sees a “bad gateway” page, the engineer pings the target directly and it works fine. The reason the target works but the load balancer returns 502 is almost always one of: the security group blocks the load balancer’s traffic, the target’s health check is failing, or the target is closing the connection before it finishes the response. Each one has a different fix.
Table of contents
- Step 0: pull the load balancer’s access log
- Step 1: verify the target is healthy
- Step 2: check the security group and the network path
- Step 3: test the target directly from inside the VPC
- Step 4: check the application’s response
- Step 5: look at the long tail
- FAQ
Step 0: pull the load balancer’s access log
Before you do anything else, get the load balancer’s access log. The log records the source IP, the destination port, the target’s response code, the target’s connection time, and the request URL. Filtering by elb_status_code = "502" narrows the search to the requests that actually failed. From there, group by target_status_code - if the target returned a 200, the problem is in the network path; if the target returned nothing, the problem is the target itself or the connection between them.
The most common pattern: the load balancer returns 502 because the target closed the connection (TCP RST) before it could complete the request. The access log records this as target_status_code = "-" and a connection error. The fix is almost always in the application - the request is taking too long, or the container is restarting on the request.
Step 1: verify the target is healthy
The load balancer’s health check is the single most common source of 502s, because the health check can lie. It can pass when the application is broken (because the health check only tests /health, which is a different code path), and it can fail when the application is fine (because the health check times out at 5 seconds but the application genuinely needs 7 to warm up).
The fix is not “make the health check more lenient.” The fix is to make the health check actually test the application: hit a real endpoint, get a real response, and confirm the response time is under the load balancer’s timeout. The two mistakes I see most often:
- Health check is too cheap. A
/healthendpoint that returns 200 if the process is up - but does not check the database connection - will pass while the application is unable to serve a real request. - Health check is too expensive. A
/healthendpoint that runs a full database query can hang the health check under load, which causes the load balancer to mark the target unhealthy, which removes the target from rotation, which makes the load spike on the remaining targets, which causes them to fail their health checks too. The cascade ends when the load balancer has no healthy targets and returns 503 to every request.
A good health check is between the two. It tests a real endpoint, but a cheap one - the kind that returns 200 in under 100ms even at peak load.
Step 2: check the security group and the network path
This is the second most common source of 502s, and it is the one most engineers get wrong on the first try. The pattern: the target’s security group allows SSH from the engineer’s IP, so the engineer can curl the target from their laptop, and the request works. But the load balancer’s security group (or the security group on the target’s subnet) does not allow traffic from the load balancer’s IP range, so the load balancer’s requests get silently dropped at the network layer.
The fix is to allow traffic from the load balancer’s CIDR (or, better, from the load balancer’s security group, if your cloud supports that) to the target’s port. The mental model: the security group on the target has to allow the load balancer’s traffic, not just the engineer’s traffic. The engineer’s laptop is not the load balancer, and treating the two as equivalent is a common mistake.
Same goes for the network ACL at the subnet level. A DENY rule on the subnet can block the load balancer’s traffic even if the security group allows it.
Step 3: test the target directly from inside the VPC
Once the security group and the network path look right, test the target from inside the VPC. A common pattern: spin up a small “test” instance in the same subnet, ssh into it, and curl the target’s port directly. This confirms the target is reachable from inside the VPC at all - independent of the load balancer.
If the curl from inside the VPC works, the problem is the load balancer (or the network path between the load balancer and the target). If the curl from inside the VPC fails, the problem is the target (or the security group on the target). The two failure modes have different fixes and the test is the fastest way to know which one you are in.
A useful extra: time the request. A curl -w '%{time_total}' tells you how long the request actually takes. If the target’s response time is approaching the load balancer’s idle timeout (default 60s on most providers), the request is going to fail with a 504, not a 502, but the symptom is similar enough that the time measurement is the only reliable way to tell them apart.
Step 4: check the application’s response
Half the time the load balancer is the messenger, not the cause. The application is returning a 500, the load balancer forwards it, and the user sees a 502 because the load balancer’s error code does not match the upstream’s error code. The fix is to look at the application’s logs, not the load balancer’s.
The pattern that catches this: filter the application logs for the same timestamp as the 502 in the load balancer’s access log. If the application logged a 500 for that request, the load balancer is just relaying the failure; the real bug is in the application.
The most common application-level causes for this:
- Database connection pool exhausted. The application is blocking on a database query, the request times out, the load balancer sees a closed connection.
- Memory leak. The application is using more memory than the container allows, the OOM killer fires, the container restarts mid-request.
- Slow GC. The application’s garbage collector is taking 10+ seconds, the request times out, the load balancer sees a closed connection.
- External service dependency. The application is waiting on a downstream service (a payment API, an email service, a third-party SDK) that is slow or down.
The fix is in the application, not the load balancer. The load balancer is the messenger, and the fix is to address the underlying issue.
Step 5: look at the long tail
Most load balancer 502s are not the long tail. The long tail is the small fraction of requests that fail for reasons nobody can reproduce, and the long tail is what makes load balancer debugging feel like guesswork.
The fix for the long tail is not “more dashboards.” The fix is to capture the request and the response for the failing requests, including the headers, the body, the time, the target, the connection state, and the application’s internal state at the moment of failure. Distributed tracing - OpenTelemetry, Jaeger, Zipkin, Datadog APM - does this automatically. If you do not have distributed tracing, the long tail is going to be a recurring problem.
A useful pattern: when the long tail shows up, the fastest fix is to add a 1% sample of failed requests to a log stream that captures the full request and response. The 1% sample is enough to see the pattern; the 100% sample will cost more than the bug is worth.
Once you have the long tail captured, the pattern usually emerges: it is a specific kind of request (a long URL, a specific header, a specific user agent, a specific cookie), and the fix is targeted. The long tail almost never has a single cause - it has a family of causes, all related to the application doing something specific under specific conditions.
FAQ
What causes a 502 Bad Gateway on a load balancer?
Almost always one of three things: the target’s security group blocks the load balancer’s traffic, the target’s health check is failing, or the target is closing the connection before it can complete the response. Each has a different fix. The access log tells you which.
How do I check the load balancer’s access log?
Enable access logging on the load balancer. The log records the source IP, the destination port, the target’s response code, and the request URL. Filter by the status code you are investigating (e.g., 502) and group by the target’s response code to see whether the target is responding or not.
Why does the target work directly but the load balancer returns 502?
The target is reachable from your laptop (because your IP is allowed in the security group), but the load balancer is not. The fix is to allow the load balancer’s CIDR or security group in the target’s security group, not just your IP. The most common cause of 502s is a security group rule that allows the engineer’s IP but not the load balancer’s.
How do I know if the 502 is in the application or the load balancer?
Check the application’s logs for the same timestamp as the 502 in the load balancer’s access log. If the application logged a 500, the load balancer is just relaying the failure; the fix is in the application. If the application logged nothing, the failure is between the load balancer and the target.
What is the long tail of 502s?
The small fraction of requests that fail for reasons nobody can reproduce. The fix is distributed tracing - capture the request, the response, the headers, the body, the time, and the application’s internal state at the moment of failure. Without tracing, the long tail is going to be a recurring problem.
If you are sizing the load balancer tier as part of a new deployment, the RunxBuild hosting calculator is the right place to model the throughput and the per-request cost before committing to a region. The dashboard at dashboard.runxbuild.com is where you can see the same metrics for any of your services in one place.