Talks to any service that serves the OpenAI Chat Completions API, so that a gateway can front a provider Membrane has no element of its own for. The api key travels as a Bearer token in the Authorization header, and the target url decides which service is called. Known to work with Azure OpenAI, the OpenAI compatible endpoint of Google Gemini, TogetherAI, Fireworks AI, DeepSeek AI, OpenRouter, Mistral AI, DeepInfra, SiliconFlow, NVIDIA NIM, ML Studio, vLLM and Ollama.

Example Configuration

api:
   port: 2000
   flow:
     - llmGateway:
         chatCompletions: {}
         policies:
           maxOutputTokens: 200
   target:
     url: http://localhost:11434

api:
   port: 2000
   flow:
     - llmGateway:
         chatCompletions: {}
         policies:
           maxOutputTokens: 200
   target:
     url: http://localhost:11434

Syntax

chatCompletions: {}

chatCompletions: {}

Can be used in

components llmGateway