Third-Party Client Configuration

Connect AI Commons to supported third-party clients using the OpenAI-compatible or Anthropic-compatible APIs.

On This Page

Using Claude Code with the AI Commons Anthropic Endpoint

Audience: Anyone with access to an API key who wants to run Claude Code, or another Anthropic SDK client, against the Campus AI Commons account.

The OpenAI-compatible API also implements the Anthropic Messages API on the same host and behind the same API key. Claude Code speaks that protocol natively, so setup requires only a base URL and key.

What You Get

• Claude Code on the campus account: Model calls go to Bedrock through AI Commons, are metered to your user, and count against your monthly token cap. No Anthropic account or Anthropic API key is required.
• Day-to-day Claude Code features: Streaming, parallel tool calls, thinking, images, PDFs read by tools, prompt caching, subagents, local MCP servers, /model switching, --print mode, and hooks.
• Account model selection: Select Claude, Amazon Nova, Llama 4, Qwen 3, or GPT-OSS models using the literals returned by GET /v1/models.
• Stateless requests: This route does not write conversations. Only token usage is recorded in your usage ledger.

1. Before You Start

1. Be in the openai group. This is the same access that gates the OpenAI-compatible API. If the API Keys page says the feature is unavailable, group membership is the likely cause.
2. Mint an API key. Open My API Keys in AI Commons. Copy the key when it is shown and note the base URL displayed on the page.
3. Install Claude Code. The guide was verified with Claude Code 2.1 or later.

npm install -g @anthropic-ai/claude-code

2. Point Claude Code at AI Commons

Set ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY. Put them in .claude/settings.local.json for one project, .claude/settings.json for a shared project configuration, or ~/.claude/settings.json for all projects.

{
  "env": {
    "ANTHROPIC_BASE_URL": "https://<api-id>.execute-api.<region>.amazonaws.com",
    "ANTHROPIC_API_KEY": "<the key you minted>"
  },
  "model": "sonnet"
}

For one shell session:

export ANTHROPIC_BASE_URL=https://<api-id>.execute-api.<region>.amazonaws.com
export ANTHROPIC_API_KEY=<the key you minted>
claude

Base URL Behavior

Claude Code appends /v1/messages automatically. The bare deployed host works, and the base URL shown on the API Keys page ending in /v1 is also accepted by the deployed API. A local backend has no /v1 stage and requires the bare host; see Local Development.

Two startup warnings are expected and harmless:

• claude.ai connectors are disabled because ANTHROPIC_API_KEY is set
• An unrecognized_model telemetry line when using a Bedrock literal Claude Code does not recognize. The request still runs.

3. Choosing Models

Claude Code has one main model and several secondary model settings. Each may use a direct Bedrock model literal exactly as returned by GET /v1/models. A retired or unrecognized Claude literal falls back to claude-v5-sonnet.

Recommended Model Configuration

{
  "env": {
    "ANTHROPIC_DEFAULT_SONNET_MODEL": "claude-v5-sonnet",
    "ANTHROPIC_DEFAULT_OPUS_MODEL": "claude-v5-opus",
    "ANTHROPIC_DEFAULT_HAIKU_MODEL": "amazon-nova-micro",
    "CLAUDE_CODE_SUBAGENT_MODEL": "claude-v4.5-haiku"
  },
  "model": "claude-v5-sonnet"
}

• model: Main model. Use an alias (sonnet, opus, or haiku) or a direct literal.
• ANTHROPIC_DEFAULT_*_MODEL: Defines what each alias means. The Haiku variable also selects the background model used for titles, summaries, and frequent small internal calls. The older ANTHROPIC_SMALL_FAST_MODEL name is deprecated.
• CLAUDE_CODE_SUBAGENT_MODEL: Model used by Agent tool subagents.
• One session: Run claude --model qwen3-32b, set ANTHROPIC_MODEL, or use /model amazon-nova-pro during a session.

Default Translation

If no model mapping is configured, Claude Code sends its own Anthropic model IDs. The endpoint translates recognized prefixes to campus models. An ID it cannot place runs as claude-v5-sonnet.

Measured Model Behavior

Using Bots

model may also be a bot ID or <bot-id>/<literal>. The bot instruction leads the system prompt, its model runs, and an enabled guardrail applies. Claude Code supplies its own tools, so the bot's agent tools do not run. Bots with knowledge bases are rejected with a 400 because this route cannot search the knowledge base.

Maximum Output Tokens

If Claude Code requests more output tokens than the selected model supports, the endpoint lowers the value and reports the adjustment in the X-Max-Tokens-Clamped response header instead of failing the request.

4. Differences from Anthropic's API

• Server-side web search: Claude Code's built-in WebSearch tool does not work because Bedrock cannot host Anthropic's server-side search. The endpoint drops it and reports it in X-Dropped-Server-Tools. WebFetch, which runs locally, works.
• Accepted but not implemented: metadata, mcp_servers, container, service_tier, and context_management. These appear in X-Unimplemented-Parameters.
• Stop sequences: stop_sequence identifies the matched sequence only when exactly one sequence was configured. With several sequences, it is null.
• Token counting: count_tokens returns an estimate based on approximately four characters per token. Actual counts appear in each response's usage.
• Prompt caching: Client cache markers are read but not applied. The endpoint places its own cache points, so caching still works but is not controlled by the client.
• Images and PDFs in tool results: Claude models receive them as images and documents. Nova, Llama, Qwen, and GPT-OSS receive a short note identifying omitted media.

Replacing Web Search with Firecrawl

A client-side MCP search tool can replace WebSearch. The following setup was verified with Firecrawl; Brave, Tavily, Exa, and DuckDuckGo MCP servers can be connected similarly.

1. Disable WebSearch and Permit Read-Only Firecrawl Tools

{
  "permissions": {
    "deny": ["WebSearch"],
    "allow": ["mcp__firecrawl__firecrawl_search", "mcp__firecrawl__firecrawl_scrape"]
  }
}

Omit the allow list if you prefer to approve each call.

2. Register the MCP Server

claude mcp add --scope local firecrawl -e FIRECRAWL_API_KEY=fc-… -- npx -y firecrawl-mcp
claude mcp list        # firecrawl: npx -y firecrawl-mcp - ✔ Connected

--scope local stores the key in your local Claude Code project configuration outside the repository. For a project .mcp.json, add "enabledMcpjsonServers": ["firecrawl"] to settings and either write the key directly in the protected, uncommitted .mcp.json or export it in the shell. Values from a settings file's env block are not used for ${FIRECRAWL_API_KEY} expansion in .mcp.json.

5. Limits You Will Hit

Monthly Token Cap

The cap counts raw input and output tokens equally across models. Claude Code resends its context on each turn, so an agent session can consume a standard one-million-token allowance quickly. When the cap is exhausted, requests return 429 rate_limit_error; Claude Code treats 429 as transient and retries with backoff.

1. Set ENABLE_TOOL_SEARCH=force to load tool schemas on demand.
2. Disconnect MCP servers the project does not need.
3. Keep file reads, command output, and scraped content short.
4. Use /clear after large tool outputs, while recognizing that tool schemas may dominate usage.
5. Ask an AI Commons administrator for a higher cap if Claude Code is a regular tool.

Response Duration and Streaming

• A request and its backend execution may run for up to 15 minutes.
• Model output normally stops at max_tokens before the request timeout.
• Five minutes without a streamed event closes the connection as an idle timeout.
• MAX_THINKING_TOKENS does not lower Claude 5 adaptive thinking because thinking and text share the same max_tokens ceiling.
• There is no finer request throttle before the monthly cap; interrupt a thrashing agent early with Ctrl+C.

6. Errors and What They Mean

Errors use Anthropic's envelope, allowing Claude Code to apply its normal retry behavior.

Quick Authentication Check

curl -si \
  -H "x-api-key: <your key>" \
  https://<api-id>.execute-api.<region>.amazonaws.com/v1/models

7. Watching Usage

• AI Commons UI: The token-usage page shows the monthly total used for quota enforcement.
• Claude Code telemetry: Send OpenTelemetry metrics to your own Prometheus instance for model, token, cost-estimate, session, and active-time tracking.

{
  "env": {
    "CLAUDE_CODE_ENABLE_TELEMETRY": "1",
    "OTEL_METRICS_EXPORTER": "otlp",
    "OTEL_EXPORTER_OTLP_PROTOCOL": "http/json",
    "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT": "http://localhost:9090/api/v1/otlp/v1/metrics",
    "OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE": "cumulative",
    "OTEL_METRIC_EXPORT_INTERVAL": "5000"
  }
}

Prometheus 3 must run with --web.enable-otlp-receiver. The cumulative temporality setting is required because Prometheus rejects the default delta counters. Useful series include:

• claude_code.token.usage_tokens_total with model and type labels
• claude_code.cost.usage_USD_total
• claude_code.session.count_total
• claude_code.active_time.total_seconds_total

sum by (model, type) ({"claude_code.token.usage_tokens_total"})

8. Developers: Running It Locally

From a bedrock-chat checkout with the backend environment and current AWS credentials:

cd backend
set -a; . ./.env; set +a
poetry run uvicorn app.openai_compat.main:app --port 8001

Local authentication uses the real key table and Cognito, so your user needs a real API key row. Mint a key in the development UI or use put_user_api_key in a Python shell. Because the local app has no /v1 stage, use the bare host:

ANTHROPIC_BASE_URL=http://127.0.0.1:8001 \
ANTHROPIC_API_KEY=<key> \
claude

The server log emits an anthropic(stream) model-resolution line and a per-turn input breakdown for system prompt, tools, history, and current input.

9. Reference

• Current model IDs: AI Commons GET /models reference
• Endpoint request and response shapes: docs/OPENAI_COMPATIBLE_API_REFERENCE.md, sections 6.10–6.11, in the application repository
• Design and known limits: docs/anthropic-messages-api.md in the application repository
• Implementation: docs/openai-compat-api-implementation.md, section 8, in the application repository
• Claude Code model settings: Model configuration
• Claude Code environment variables: Environment variables

AnythingLLM

Continue.dev

{
  "models": [
    {
      "title": "UCSB AI Commons",
      "provider": "openai",
      "model": "claude-v4.6-sonnet",
      "apiBase": "YOUR_BASE_URL",
      "apiKey": "your-api-key"
    }
  ]
}

GitHub Copilot (VS Code)

Prerequisites

Before beginning the setup, ensure your development environment meets the following requirements:

• VS Code Version: 1.122 or later (which natively supports the custom endpoint vendor and Bring Your Own Key functionality).
• Extensions: The official GitHub Copilot and GitHub Copilot Chat extensions must be installed and active.
• Network Access: Ensure your machine has network connectivity to the AWS API Gateway URL specified in your endpoint configuration.

Configuration Steps

1. Open the Command Palette in VS Code:
   • Windows/Linux: Ctrl + Shift + P
   • macOS: Cmd + Shift + P
2. Type and select: Chat: Manage Language Models.
3. In the Language Models editor view, click on Add Models.
4. Select Custom Endpoint from the provider options list.
5. Enter a Group Name for this endpoint (e.g., UCSB AI).
6. Paste your custom internal API Key when prompted by the input box.

7. Select Chat Completions (chat-completions) when prompted for the API schema type.
8. VS Code will automatically open your global or workspace-specific chatLanguageModels.json configuration file.
9. Replace the contents of the opened chatLanguageModels.json file with your verified JSON configuration below:

[
    {
        "name": "UCSB AI",
        "vendor": "customendpoint",
        "apiKey": "${input:chat.lm.secret.-149527b5}",
        "apiType": "chat-completions",
        "models": [
            {
                "id": "claude-v4.6-sonnet",
                "name": "UCSB-claude-v4.6-sonnet",
                "url": "YOUR_BASE_URL",
                "toolCalling": true,
                "vision": true,
                "maxInputTokens": 128000,
                "maxOutputTokens": 16000
            }
        ]
    }
]