Authenticates clients of the LLM Gateway by their api key and enforces a token limit per user, with the users configured inline and their usage kept in memory. A request whose key is unknown is rejected; a user who has spent the tokens of the current period is rejected until the period resets. Usage counts are lost on restart, so this fits a small, fixed set of clients rather than a large or changing one. See tutorials/llm-gateway/openai/20-Sharing-API-Keys.yaml.

 simpleStore:
   users:
     - name: <name>
       apiKey: <key>
       [ tokens: <limit> ]
     ...
   [ limitResetPeriod: <seconds> ]
   [ logUsage: true | false ]

Example Configuration

api:
   port: 2000
   flow:
     - llmGateway:
         apiKey: sk-...
         openai: {}
         simpleStore:
           users:
             - name: alice
               apiKey: abc123
               tokens: 500
             - name: bob
               apiKey: qwertz
               tokens: 10000
           limitResetPeriod: 60
   target:
     url: https://api.openai.com

api:
   port: 2000
   flow:
     - llmGateway:
         apiKey: sk-...
         openai: {}
         simpleStore:
           users:
             - name: alice
               apiKey: abc123
               tokens: 500
             - name: bob
               apiKey: qwertz
               tokens: 10000
           limitResetPeriod: 60
   target:
     url: https://api.openai.com

Syntax

simpleStore:
  limitResetPeriod: <int>
  logUsage: <boolean>
  users:
    - <users>

simpleStore:
  limitResetPeriod: <int>
  logUsage: <boolean>
  users:
    - <users>

Attributes

Name	Required	Default	Description	Examples
limitResetPeriod	false	60	The period in seconds after which the tokens used by every user are set back to zero and their full limit is available again.	3600
logUsage	false	true	Whether the token usage of every request is written to the log.	false

Child Structure

Element	Cardinality	Description
users	0..*	The users the gateway accepts, each with the api key it authenticates by and its token limit. A user without an api key is a configuration error and fails the startup.

Can be used in

llmGateway