Gateway in front of an LLM provider's chat API (OpenAI, Anthropic/Claude, or Google Gemini). It can share a single provider API key among many clients, authenticate clients and enforce per-user token limits through a store, apply usage policies, and inject a system prompt; token usage is recorded when the response returns. Place it in the flow of an api whose target is the provider's API. Without a store the gateway is stateless and just forwards requests with the configured key and policies. See the tutorials under tutorials/ai/llm-gateway.

 llmGateway:
   [ apiKey: <provider-key> ]
   claude | openai | google              # the provider (required)
   [ policies: ... ]                     # token limits, model rules
   [ systemPrompt: ... ]
   [ simpleStore | jdbcAiApiUsageStore ] # enables per-user auth and limits

Example Configuration

api:
   port: 2000
   flow:
     - llmGateway:
         claude: {}
         policies:
           maxInputTokens: 100
           maxOutputTokens: 200
   target:
     url: https://api.anthropic.com

api:
   port: 2000
   flow:
     - llmGateway:
         claude: {}
         policies:
           maxInputTokens: 100
           maxOutputTokens: 200
   target:
     url: https://api.anthropic.com

Syntax

llmGateway:
  '$ref': <string>
  apiKey: <string>
  chatCompletions: {}
  claude: {}
  google: {}
  jdbcAiApiUsageStore: {}
  openai: {}
  policies: {}
  simpleStore: {}
  systemPrompt: {}

llmGateway:
  '$ref': <string>
  apiKey: <string>
  chatCompletions: {}
  claude: {}
  google: {}
  jdbcAiApiUsageStore: {}
  openai: {}
  policies: {}
  simpleStore: {}
  systemPrompt: {}

Attributes

Name	Required	Default	Description	Examples
$ref	false	-	Reference a component defined under components.	-
apiKey	false	-	API key for the LLM provider, used for all upstream calls and overriding any key sent by the client. Use the key issued by OpenAI, Anthropic, or Google.	sk-...

Child Structure

Element	Cardinality	Description
chatCompletions	0..1	OpenAI Chat Completions API compatible provider. Can be used for the following providers:
claude	0..1	(Experimental) Anthropic Claude provider configuration Use to configure a LLM gateway to use the anthropic API
google	0..1	(Experimental)Google AI provider configuration Use to configure a LLM gateway to use the Google LLM API
jdbcAiApiUsageStore	0..1	Stores AI API usage in a database (experimental).
openai	0..1	OpenAI provider configuration Use to configure a LLM gateway to use the OpenAI API
policies	0..1	Usage policies applied to each request, such as maximum input and output tokens or allowed models.
simpleStore	0..1	Simple store for the LLM Gateway that stores limits in memory. Users and keys can be configured in the configuration file.
systemPrompt	0..1	System prompt injected into every request before it is forwarded to the provider.

Can be used in

abort api case chain components for global if interceptor internal otherwise proxy registration request response serviceProxy soapProxy stompProxy time transport wsStompReassembler