Gateway in front of an LLM provider's chat API (OpenAI, Anthropic/Claude, or Google Gemini). It can share a single provider API key among many clients, authenticate clients and enforce per-user token limits through a store, apply usage policies, and inject a system prompt; token usage is recorded when the response returns. Place it in the flow of an api whose target is the provider's API. Without a store the gateway is stateless and just forwards requests with the configured key and policies. See the tutorials under tutorials/llm-gateway.
llmGateway:
[ apiKey: <provider-key> ]
claude | openai | google # the provider (required)
[ policies: ... ] # token limits, model rules
[ systemPrompt: ... ]
[ simpleStore ] # per-user auth and limits
[ jdbcAiApiUsageStore ] # records usage in a database
API key for the LLM provider, used for all upstream calls and overriding any key sent by the client. Use the key issued by OpenAI, Anthropic, or Google.
Talks to any service that serves the OpenAI Chat Completions API, so that a gateway can front a provider Membrane has no element of its own for. The api key travels as a Bearer token in the Authorization header, and the target url decides which service is called. Known to work with Azure OpenAI, the OpenAI compatible endpoint of Google Gemini, TogetherAI, Fireworks AI, DeepSeek AI, OpenRouter, Mistral AI, DeepInfra, SiliconFlow, NVIDIA NIM, ML Studio, vLLM and Ollama.
(Experimental) Talks to the Anthropic Claude Messages API under /v1/messages. The api key travels in the x-api-key header, and the system prompt in the top level system field. See tutorials/llm-gateway/claude/10-Basic-LLM-Gateway.yaml.
(Experimental) Talks to the Google Gemini API, where the model is named in the path rather than in the request body, as in /v1beta/models/gemini-2.5-pro:generateContent. The api key travels in the x-goog-api-key header. See tutorials/llm-gateway/google/10-Basic-LLM-Gateway.yaml.
Records the token usage of every request in a database table (experimental). It neither authenticates clients nor enforces token limits, so use it where the gateway should only account for what was used. The table ai_api_usage is created at startup if it does not exist, and usage that cannot be attributed to a user is recorded as anonymous.
Talks to the OpenAI API. Both the Responses API under /v1/responses and the Chat Completions API under /v1/chat/completions are supported, and the endpoint the client calls decides which one is used. The api key travels as a Bearer token in the Authorization header. See tutorials/llm-gateway/openai/10-Basic-LLM-Gateway.yaml.
Authenticates clients of the LLM Gateway by their api key and enforces a token limit per user, with the users configured inline and their usage kept in memory. A request whose key is unknown is rejected; a user who has spent the tokens of the current period is rejected until the period resets. Usage counts are lost on restart, so this fits a small, fixed set of clients rather than a large or changing one. See tutorials/llm-gateway/openai/20-Sharing-API-Keys.yaml.