Provider adapters

Provider adapters translate normalized IBEX chat requests into upstream LLM API calls and stream responses back to clients. The registry ships with a mock adapter (default), an OpenAI adapter, an Anthropic adapter (ADR-0040), and a self-hosted OpenAI-compatible adapter (ADR-0042).

Why adapters exist

IBEX Harness must support multiple LLM vendors without leaking provider specifics into middleware. Adapters isolate:

Upstream URL and authentication (API keys, Azure deployment IDs)
Request/response dialect differences (tool calls, streaming chunk format)
Retry and circuit-breaker policy per provider

The proxy critical path stays provider-agnostic: normalize once, delegate to the registry, stream the response. Target overhead remains under 20ms p99 excluding upstream LLM latency — see Architecture overview.

+---------------------------+     +-------------------+     +--------------------------+     +---------------------+
|                           |     |                   |     |                          |     |                     |
| ProviderRouter middleware |---->| Provider registry |---->|      OpenAI adapter      |---->|    api.openai.com   |
|                           |     |                   |     |                          |     |                     |
+---------------------------+     +-------------------+     +--------------------------+     +---------------------+
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |               +--------------------------+     +---------------------+
                                            |               |                          |     |                     |
                                            +-------------->|    Anthropic adapter     |---->|  api.anthropic.com  |
                                            |               |                          |     |                     |
                                            |               +--------------------------+     +---------------------+
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |               +--------------------------+     +---------------------+
                                            |               |                          |     |                     |
                                            +-------------->| openaicompatible adapter |---->| vLLM / TGI / Ollama |
                                                            |                          |     |                     |
                                                            +--------------------------+     +---------------------+

Self-hosted OpenAI-compatible (vLLM-first)

Enable with IBEX_SELFHOSTED_ENABLED=true, a BaseURL ending in /v1, and IBEX_SELFHOSTED_MODELS. Capability overlays use provider: "openai" (wire dialect) while the runtime provider name is openaicompatible. Bootstrap probes GET /models before registration; /ready exposes an advisory selfhosted_llm check. Queue-full (HTTP 503) and circuit-open failures return distinct PROVIDER_UNAVAILABLE details — see ADR-0042.

Planned adapter contract → Anthropic shipped

Anthropic is implemented in packages/provider/anthropic/. It translates Messages API requests/responses to OpenAI-compatible JSON and SSE so the public POST /v1/chat/completions contract and dual-write forwarder stay unchanged. Built-in models: claude-sonnet-4-5, claude-haiku-4-5, claude-opus-4-5 (plus ANTHROPIC_EXTRA_MODELS). Tool-use / image blocks are not passthrough in this milestone.

Register at startup

Each configured adapter is registered into provider.Registry by SupportedModels(). Live mode registers OpenAI, Anthropic, and/or self-hosted (openaicompatible) when configured.

Receive normalized payload

Input is the validated OpenAI chat JSON plus org_id, agent_id, and request_id from middleware context.

Call upstream

Adapter applies provider credentials from process env (never from the client request body). Anthropic uses x-api-key + anthropic-version; self-hosted omits Authorization when the key is empty.

Stream response

Return OpenAI-compatible JSON or SSE chunks to the client; accumulate for async trace emission.

Adapters must not read org_id from the request body. Tenant context comes from verified auth middleware only — Tenant isolation.

Mock vs live behavior

Registered models in mock mode return HTTP 200. Probe status (not .error.code):

bash

curl -s -o /tmp/chat.json -w "%{http_code}\n" http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer ${IBEX_DEV_TOKEN}" \
  -H "X-IBEX-Agent-ID: ${IBEX_DEV_AGENT_ID}" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Expected output: 200.

Error envelope (unknown model)

JSON

{
  "error": {
    "code": "PROVIDER_NOT_CONFIGURED",
    "message": "No LLM provider adapter is configured for this request",
    "request_id": "0192a3b4-c5d6-7890-abcd-ef1234567890",
    "timestamp": "2026-06-14T12:00:00Z"
  }
}

This is distinct from 503 dependency failures — the proxy itself is healthy; the requested model is not in the active registry.

What still evolves

Registry (shipped)

Model-to-adapter resolution for mock and OpenAI-compatible live modes.

OpenAI adapter (shipped)

Non-streaming JSON and streaming SSE dual-write (ADR-0027).

Circuit breaker

Provider 5xx triggers breaker; clients receive 503 with Retry-After (hardening continues).

Memory injection

Parallel memory retrieval before upstream call remains Phase 3 — directives already inject today.

Provider API keys are server-side (OPENAI_API_KEY / future org vault) — never accepted from client headers. Rotation guidance: Secrets and keys.

Security boundaries

Control	Today
Client supplies provider API key	Forbidden
Org from token	Enforced
Audit log per upstream call	ClickHouse traces when `CLICKHOUSE_DSN` set
PII in upstream prompts	Validated locally; memory redaction is Phase 3

Verify adapters

bash

make compose-dev-up
make db-migrate && make db-seed
make dev-smoke   # asserts mock chat success without OPENAI_API_KEY

Integration tests in services/proxy/ cover mock success and PROVIDER_NOT_CONFIGURED for unknown models.

Request routing — normalization before adapter handoff
Overview — middleware and failure modes
Architecture services — proxy role in the system map
Roadmap Phase 2 — exit-gate timeline

Was this page helpful?

Why adapters exist

IBEX Harness must support multiple LLM vendors without leaking provider specifics into middleware. Adapters isolate:

Upstream URL and authentication (API keys, Azure deployment IDs)
Request/response dialect differences (tool calls, streaming chunk format)
Retry and circuit-breaker policy per provider

+---------------------------+     +-------------------+     +--------------------------+     +---------------------+
|                           |     |                   |     |                          |     |                     |
| ProviderRouter middleware |---->| Provider registry |---->|      OpenAI adapter      |---->|    api.openai.com   |
|                           |     |                   |     |                          |     |                     |
+---------------------------+     +-------------------+     +--------------------------+     +---------------------+
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |               +--------------------------+     +---------------------+
                                            |               |                          |     |                     |
                                            +-------------->|    Anthropic adapter     |---->|  api.anthropic.com  |
                                            |               |                          |     |                     |
                                            |               +--------------------------+     +---------------------+
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |                                                                       
                                            |               +--------------------------+     +---------------------+
                                            |               |                          |     |                     |
                                            +-------------->| openaicompatible adapter |---->| vLLM / TGI / Ollama |
                                                            |                          |     |                     |
                                                            +--------------------------+     +---------------------+

Self-hosted OpenAI-compatible (vLLM-first)

Planned adapter contract → Anthropic shipped

Register at startup

Each configured adapter is registered into provider.Registry by SupportedModels(). Live mode registers OpenAI, Anthropic, and/or self-hosted (openaicompatible) when configured.

Receive normalized payload

Input is the validated OpenAI chat JSON plus org_id, agent_id, and request_id from middleware context.

Call upstream

Adapter applies provider credentials from process env (never from the client request body). Anthropic uses x-api-key + anthropic-version; self-hosted omits Authorization when the key is empty.

Stream response

Return OpenAI-compatible JSON or SSE chunks to the client; accumulate for async trace emission.

Adapters must not read org_id from the request body. Tenant context comes from verified auth middleware only — Tenant isolation.

Mock vs live behavior

Registered models in mock mode return HTTP 200. Probe status (not .error.code):

bash

curl -s -o /tmp/chat.json -w "%{http_code}\n" http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer ${IBEX_DEV_TOKEN}" \
  -H "X-IBEX-Agent-ID: ${IBEX_DEV_AGENT_ID}" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"hi"}]}'

Expected output: 200.

Error envelope (unknown model)

JSON

{
  "error": {
    "code": "PROVIDER_NOT_CONFIGURED",
    "message": "No LLM provider adapter is configured for this request",
    "request_id": "0192a3b4-c5d6-7890-abcd-ef1234567890",
    "timestamp": "2026-06-14T12:00:00Z"
  }
}

This is distinct from 503 dependency failures — the proxy itself is healthy; the requested model is not in the active registry.

What still evolves

Registry (shipped)

Model-to-adapter resolution for mock and OpenAI-compatible live modes.

OpenAI adapter (shipped)

Non-streaming JSON and streaming SSE dual-write (ADR-0027).

Circuit breaker

Provider 5xx triggers breaker; clients receive 503 with Retry-After (hardening continues).

Memory injection

Parallel memory retrieval before upstream call remains Phase 3 — directives already inject today.

Provider API keys are server-side (OPENAI_API_KEY / future org vault) — never accepted from client headers. Rotation guidance: Secrets and keys.

Security boundaries

Control	Today
Client supplies provider API key	Forbidden
Org from token	Enforced
Audit log per upstream call	ClickHouse traces when `CLICKHOUSE_DSN` set
PII in upstream prompts	Validated locally; memory redaction is Phase 3

Verify adapters

bash

make compose-dev-up
make db-migrate && make db-seed
make dev-smoke   # asserts mock chat success without OPENAI_API_KEY

Integration tests in services/proxy/ cover mock success and PROVIDER_NOT_CONFIGURED for unknown models.

Request routing — normalization before adapter handoff
Overview — middleware and failure modes
Architecture services — proxy role in the system map
Roadmap Phase 2 — exit-gate timeline

Was this page helpful?