How Fast Is Membrane API Gateway?

Benchmark numbers are one thing. What counts is how fast an API gateway is when it does real work. What a gateway does with each request, meaning the plugins it runs, affects its performance more than anything else.

We therefore ran three tests. The first two are classic benchmark setups: a plain proxy, and a secured API with cheap per-request checks. The third shows Membrane doing real, demanding work. It validates every request against an OpenAPI specification and still handles almost 90,000 requests per second. Every run finished with zero errors.

Each test sent 1,000,000 requests through the gateway, with 175 requests in flight at once.

The Test Setup

We measured the gateway alone, without any interference from the other parts of the test. Client, gateway and backend each ran on their own virtual machine, connected by a real network:

• Machines: three Azure VMs with 16 vCPUs each, in one virtual network (Sweden Central). Gateway: Standard_FX16mds_v2, backend: Standard_F16as_v7, client: Standard_F16as_v6.
• Gateway: the standard Membrane distribution, started with membrane.sh just like in production. Java 21 (Eclipse Temurin) with a 32 GB heap and the Parallel garbage collector.
• Backend: a minimal service that answers every request with 200 OK. It does almost no work, so the numbers show what the gateway itself can do.
• Client: a Java load generator that sends POST /shop/v2/products with a JSON body.

Test 1: Proxying

The gateway receives each request and forwards it unchanged to the backend. This is the classic benchmark number: it shows the raw speed of the gateway engine. It is not a realistic use of an API gateway, though, because a gateway that only forwards requests does nothing a plain reverse proxy couldn't do.

api:
  port: 2000
  target:
    host: backend
    port: 2010

api:
  port: 2000
  target:
    host: backend
    port: 2010

Test 2: Secured API (Basic Auth + Rate Limiting + TLS)

This is a setup commonly used in published API gateway benchmarks, so we included it to make the results comparable. For every request, the gateway:

1. decrypts it (TLS; the client connects via HTTPS),
2. checks the user name and password (HTTP Basic Authentication),
3. counts it against a rate limit,
4. forwards it to the backend over a second, encrypted TLS connection.

api:
  port: 2000
  ssl:
    keystore:
      location: gateway.p12
      password: "..."
      keyPassword: "..."
  flow:
    - basicAuthentication:
        users:
          - username: loadtest
            password: "..."
    - rateLimiter:
        requestLimit: 2000000
        requestLimitDuration: PT1H
  target:
    host: backend
    port: 2011
    ssl:
      truststore:
        location: truststore.p12
        password: "..."

api:
  port: 2000
  ssl:
    keystore:
      location: gateway.p12
      password: "..."
      keyPassword: "..."
  flow:
    - basicAuthentication:
        users:
          - username: loadtest
            password: "..."
    - rateLimiter:
        requestLimit: 2000000
        requestLimitDuration: PT1H
  target:
    host: backend
    port: 2011
    ssl:
      truststore:
        location: truststore.p12
        password: "..."

We set the rate limit high enough that no request is rejected. A rejected request is cheaper than a forwarded one, so rejections would make the result look better than it is. Zero errors means every request went through all four steps.

TLS, a password check and a rate-limit counter are cheap for the gateway, which is why the result is close to plain proxying.

Test 3: OpenAPI Validation

This test shows how the gateway behaves in a real setup. Validation is a typical task for an API gateway, and one a plain reverse proxy cannot do. The gateway checks every request against the Fruitshop OpenAPI specification: method, path, content type, and the JSON body against the Product schema. It only forwards valid requests to the backend. Learn more about deploying APIs from OpenAPI.

api:
  port: 2000
  openapi:
    - location: fruitshop-v2-2-0.oas.yml
      validateRequests: true
  target:
    host: backend
    port: 2010

api:
  port: 2000
  openapi:
    - location: fruitshop-v2-2-0.oas.yml
      validateRequests: true
  target:
    host: backend
    port: 2010

The request body is a valid product, {"name":"Mangos","price":2.79}:

Validation is the most demanding of the three tasks: for every request, the gateway parses the JSON body and checks it against the schema. Throughput drops the most here, even though the body is much smaller than the ~1 KB body in tests 1 and 2. This is the key result: the plugins you use matter far more for performance than the gateway's raw speed. Membrane stays fast even under this kind of load.

Run the Tests Yourself

The whole test is scripted. Log in to Azure and run the scripts. They create the VMs, install Membrane and run the tests. Afterwards, run the teardown script to delete the VMs again, because they cost money while they are running. The README explains the steps.

The YAML samples on this page show the essential part of each configuration. The complete configurations are in the conf folder.

What These Numbers Mean

• Single runs. Each number comes from one run, not an average. Cloud VMs vary by a few percent from run to run. Expect the same order of magnitude when you repeat the tests, not the exact same figures.
• Minimal backend. The backend does no real work. In a real deployment, your backend and its database will almost always be the bottleneck long before the gateway is.
• Hardware matters. The numbers apply to the 16-vCPU machines described above. On other machine sizes, results will differ, and the ideal number of requests in flight may too.