502 Bad Gateway: Error Communicating with Target

Membrane API Gateway returns 502 Bad Gateway with the detail Error communicating with target ... Reason: ... when the HTTP client that forwards to the target hits a low-level network error while it talks to the backend. The backend or the network is to blame.

The Error Response

The response contains the status 502 Bad Gateway, the title Bad Gateway, the detail Error communicating with target <target>. Reason: <reason>, the problem type https://membrane-api.io/problems/internal and a link to this page. The reason is the message reported by the operating system's network stack. The type internal means that the gateway could not complete the processing of the request. In production mode, the title is replaced by Internal Server Error and the details are left out.

A refused connection is not reported here but under its own URL, gateway/http-client/connect.

Common Causes

Connection Reset

The backend, or a device in between, reset the TCP connection while the request was being sent or the response was being read.

Broken Pipe or Socket Closed

The backend closed the connection while Membrane was still writing the request.

No Route to Host

The network has no route to the target host, for example because of a routing or firewall problem.

Other Messages from the Network Stack

Connection aborted or similar messages come from the local network stack of the gateway host.

How to Fix It

1. Operator: use the reason in the detail and the backend's log to find which side ended the connection.
2. Operator: check routing, firewalls and security groups between Membrane and the target host.
3. Operator: for resets on reused connections, set keepAliveTimeout below the backend's keep-alive timeout (see below).
4. Client: retry the request. Membrane already retries idempotent requests, as configured in retries.

configuration:
  httpClientConfig:
    connection:
      keepAliveTimeout: 4000
    retries:
      retries: 3
      delay: 100

configuration:
  httpClientConfig:
    connection:
      keepAliveTimeout: 4000
    retries:
      retries: 3
      delay: 100

Hide Details in Production

By default, Membrane runs in development mode and adds details to error responses that help you find the problem. These details may reveal information about your setup. In production, turn them off:

configuration:
  production: true

configuration:
  production: true

Add the configuration document to your apis.yaml and separate it from the APIs with ---.

Related Documentation

Configuration reference:

• httpClient
• httpClientConfig
• target

Tutorials with runnable examples:

• First API

← All Membrane error messages