Roll out an LLM gateway for your organization

​

Prerequisites

• A gateway deployed on your infrastructure, serving HTTPS at the exact address you’ll distribute to developers, not an address that redirects to it, and configured to route Claude model names to your provider
• A provider credential for the gateway to forward with:
  • For the Anthropic API: an API key from the Claude Console
  • For a cloud provider: cloud credentials with model access. See the prerequisites on the Amazon Bedrock, Google Vertex AI, or Microsoft Foundry page
• A way to deliver settings files to developer machines, such as MDM or configuration management
  • If you don’t have one yet, how settings reach devices compares the options

​

Gateway requirements

• Accept a supported API format: one of the formats in the API formats table. The rollout steps below assume the Anthropic Messages API at POST /v1/messages, which most gateways serve
• Stream responses: pass server-sent events through as they arrive instead of buffering the whole response
• Route Claude model names: map each name developers use to an upstream model. Claude Code sends a model name such as claude-sonnet-4-6 in each request; in most gateway products the mapping is a model list or routing table in the gateway’s own configuration
• Forward headers and body unchanged: pass anthropic-beta, anthropic-version, and the request body through in both directions; the feature pass-through table maps each to the feature that breaks without it
• Return upstream errors unmodified: Claude Code’s automatic recovery matches on error wording, so wrapping errors in the gateway’s own envelope breaks it
• Exempt the path from request-body WAF inspection: Claude Code prompts carry source code and XML-style tags that match cross-site-scripting body rules; a WAF in front of the gateway returns 403 on real sessions while short test requests pass

​

Rollout steps

1. Confirm the gateway routes your models
2. Issue each developer a credential
3. Test Claude Code against the gateway
4. Distribute the base URL and credentials
5. Verify from a developer machine

​

Confirm the gateway routes your models

• <gateway-key> is whatever credential lets you call the gateway right now: an administrative key, a test key, or your own developer key if you’ve already issued one. Not every gateway product has a separate admin credential; if yours doesn’t, issue yourself a developer key in Issue developer credentials first
• model is a Claude model name your gateway is configured to route. The example uses claude-sonnet-4-6; substitute a name you’ve configured

curl -X POST "https://llm-gateway.example.com/v1/messages" \
  -H "Authorization: Bearer <gateway-key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
  -Headers @{ "Authorization" = "Bearer <gateway-key>"; "anthropic-version" = "2023-06-01" } `
  -ContentType "application/json" `
  -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

​

Issue developer credentials

curl -X POST "https://llm-gateway.example.com/v1/messages" \
  -H "Authorization: Bearer <developer-key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

Invoke-RestMethod -Method Post -Uri "https://llm-gateway.example.com/v1/messages" `
  -Headers @{ "Authorization" = "Bearer <developer-key>"; "anthropic-version" = "2023-06-01" } `
  -ContentType "application/json" `
  -Body '{"model": "claude-sonnet-4-6", "max_tokens": 1, "messages": [{"role": "user", "content": "."}]}'

​

Test Claude Code against the gateway

export ANTHROPIC_BASE_URL=https://llm-gateway.example.com
export ANTHROPIC_AUTH_TOKEN="<developer-key>"

$env:ANTHROPIC_BASE_URL = "https://llm-gateway.example.com"
$env:ANTHROPIC_AUTH_TOKEN = "<developer-key>"

claude -p "Reply with one word: connected"

• Not logged in: check the gateway log to tell the two causes apart. If it’s empty, no credential reached the session and no request left the machine; re-run the exports in the shell you’re testing from. If it shows a rejected request with x-api-key in the 401 body, the gateway expects keys in that header instead; switch to ANTHROPIC_API_KEY
• Failed to authenticate. API Error: 401 means a credential was sent and rejected, and the gateway log says where: a 401 naming api.anthropic.com or your provider’s endpoint means the gateway reached the upstream but its provider credential was rejected, so the developer key worked and the provider credential the gateway holds is wrong or a placeholder

​

Distribute the configuration

​

What to distribute

​

Distribute through managed settings

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://llm-gateway.example.com"
  },
  "apiKeyHelper": "/usr/local/bin/get-gateway-key"
}

• The desktop app reads gateway routing only from its MDM-delivered third-party inference configuration; deploy that file alongside managed settings so desktop sessions route through the gateway too. See the desktop third-party configuration docs and the desktop gateway docs
• CI runners need ANTHROPIC_BASE_URL and the credential set in the runner’s environment
• WSL on managed Windows machines reads the Windows managed settings only when wslInheritsWindowsSettings is true

​

Hand developers the values to set themselves

• The gateway URL
• Their personal credential
• Which variable to put the credential in: ANTHROPIC_AUTH_TOKEN for a bearer-token gateway, or ANTHROPIC_API_KEY for an x-api-key gateway. Telling developers which one saves them the trial-and-error described on the connect page
• Any conditional variables from the What to distribute table, with their values

​

Verify the rollout

curl -N -X POST "https://llm-gateway.example.com/v1/messages" \
  -H "Authorization: Bearer <developer-key>" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'

$body = '{"model": "claude-sonnet-4-6", "max_tokens": 16, "stream": true, "messages": [{"role": "user", "content": "count to 3"}]}'
$body | curl.exe -N -X POST "https://llm-gateway.example.com/v1/messages" `
  -H "Authorization: Bearer <developer-key>" `
  -H "anthropic-version: 2023-06-01" `
  -H "content-type: application/json" `
  --data-binary '@-'

• A login prompt means a credential gap. Run /status and open the Status tab: when the Setting sources line doesn’t include managed settings, the distribution didn’t reach the machine; when it does, the developer credential wasn’t delivered, so set ANTHROPIC_AUTH_TOKEN or the apiKeyHelper
• Failed to authenticate errors mean the gateway is rejecting requests; its log says which credential failed. A rejection the gateway logs itself names the developer key, while a 401 from api.anthropic.com or your provider’s endpoint means the provider credential the gateway holds was rejected
• A one-time approval prompt for the key is expected on first use when the gateway expects keys in the x-api-key header, set as ANTHROPIC_API_KEY. With ANTHROPIC_AUTH_TOKEN, no prompt appears and the variable takes over silently; a previously saved claude.ai login is inactive for that session

​

Maintain the gateway

​

Related resources

• Connect Claude Code to an LLM gateway: the developer-facing setup steps, with per-surface configuration and a troubleshooting table you can hand to developers
• Gateway protocol reference: the wire contract for gateway operators, covering endpoints, headers to forward, and the feature pass-through table
• Settings files and precedence: how managed, project, and user settings combine, and where the managed file goes on each platform
• Set up Claude Code for your organization: the wider rollout this gateway is one part of, including policy enforcement, usage visibility, and data handling