ADR-0042: Self-hosted OpenAI-compatible LLM adapter

• Status: Accepted
• Date: 2026-08-21
• Authors: IBEX Harness team
• Milestone: 2.5.G1.M3 Self-hosted LLM adapter

Context

Air-gapped and GPU-local deployments need the proxy to talk to OpenAI-compatible inference servers (vLLM, TGI, Ollama, llama.cpp) without overloading hosted OPENAI_BASE_URL. Hosted OpenAI and self-hosted backends differ in auth (optional bearer), readiness (model load can take minutes), and failure modes (queue-full 503 vs breaker-open).

Capability overlays already require ExtraModels metadata (ADR-0041). Self-hosted model IDs participate in that same overlay contract using the openai vendor family (wire dialect), while the runtime adapter name is distinct.

Decision

1) Second Provider: openaicompatible

Register a separate provider.Provider named openaicompatible when IBEX_SELFHOSTED_ENABLED=true. Do not steal OPENAI_BASE_URL from the hosted OpenAI client.

Shared HTTP/chat/SSE logic lives in packages/provider/openaicompatible. Hosted OpenAI (packages/provider/openai) is a thin wrapper with curated built-ins and always-on Bearer auth.

2) Config surface

Live mode may start with only self-hosted (no cloud keys). Mock mode ignores self-hosted registration.

3) Overlays use openai vendor family

IBEX_SELFHOSTED_MODELS IDs are active ExtraModels keyed as provider: "openai" for overlay validation (wire dialect). Runtime routing still resolves through the openaicompatible adapter's SupportedModels().

tokenizer_family: unknown is acceptable until 2.5.G2.M1.

4) Bootstrap readiness + advisory /ready

Before registry registration, poll GET {base}/models until 2xx or timeout (fail-closed at boot). After registration, /ready exposes an advisory selfhosted_llm checker so temporary GPU saturation does not fail critical readiness.

5) Circuit breaker outside retries

packages/circuitbreaker wraps each Complete (retries happen inside one breaker attempt). Distinct client-facing details:

• HTTP 503 from upstream → queue_full reason → “Self-hosted LLM backend queue is full”
• Breaker open → circuit_open reason → “Self-hosted LLM circuit breaker is open”

Both map to PROVIDER_UNAVAILABLE (HTTP 503) without leaking upstream bodies or keys.

6) SSRF exception

Self-hosted BaseURL may target private/loopback hosts only when explicitly enabled. Hosted OpenAI BaseURL policy is unchanged. See SECURITY.md §8.3.

Consequences

Positive: Air-gapped deployments work without cloud keys; hosted OpenAI path stays behavior-preserving; operators get distinct queue vs breaker signals.

Negative / follow-ups: Tokenizer accuracy for open-weight models waits on G2.M1; multi-backend self-hosted pools are out of scope.