Milestone 2.1.4 — Provider Routing Middleware

Status: Completed
Goal: 2.1 — LLM provider abstraction and OpenAI forwarding
Phase: 2 — Single Provider End-to-End
Estimated effort: 1–2 days

Why This Milestone Exists

With the provider interface and OpenAI client in place, the proxy needs middleware that selects the correct provider for each request and attaches it to context. M1.2.2 already enforces body-level gates (size, Content-Type). This milestone adds ChatParseMiddleware for typed chat JSON parse/validation onto context, then ProviderRoutingMiddleware for registry lookup. If no provider is registered for the model, return 501 PROVIDER_NOT_CONFIGURED.

This is a small but architecturally important milestone: it is the single place where the model-to-provider mapping is enforced. Phase 4 adds Anthropic and Azure providers by registering them in the registry — this middleware requires no changes.

DirectiveResolve middleware (later milestones)
Org-scoped provider overrides
Multi-provider registration beyond OpenAI (Phase 4)
Provider error-mapping package (2.1.5)
Changes to IBEX_LLM_MODE / buildProviderRegistry

Deliverables

Middleware

// ChatParseMiddleware parses/validates chat JSON once and attaches llm.ChatCompletionRequest.
// ProviderRoutingMiddleware selects the LLM provider for the request based on
// the `model` field and attaches provider.Provider to context.
//
// Returns 501 PROVIDER_NOT_CONFIGURED if no provider supports the requested model.
// Missing/invalid model returns 400 VALIDATION_ERROR from ChatParse (unchanged Phase 1 contract).
//
// Required position: AFTER RateLimitMiddleware, BEFORE the forwarding handler.
//   … → RateLimit → ChatParse → ProviderRouter → [handler]
func ChatParseMiddleware(opts chatParseOpts) func(http.Handler) http.Handler
func ProviderRoutingMiddleware(opts providerRoutingOpts) func(http.Handler) http.Handler
 
// ProviderFromContext returns the selected provider when routing succeeded.
func ProviderFromContext(ctx context.Context) (provider.Provider, bool)

Full middleware chain (Phase 2)

RequestID → OTel Span → Auth (LRU+gRPC) → AgentVerify → RateLimit → ChatParse → ProviderRouter → Handler

DirectiveResolve is a future insertion (between RateLimit and ChatParse/ProviderRouter) — not implemented in this milestone.

Testing Requirements

TestUnit_ProviderRouting_KnownModelAttachesProvider: known model → provider on context → next handler runs
TestUnit_ProviderRouting_UnknownModel501: unknown model → 501; handler spy not called
TestUnit_ChatParse_MissingModel400: empty model → 400 VALIDATION_ERROR with field model
TestUnit_ChatCompletions_missingProviderReturnsInternalError: handler without provider context → 500 INTERNAL_ERROR

Acceptance Criteria

Known model (e.g. gpt-4o) → provider attached to context → handler executes
Unknown model → 501 PROVIDER_NOT_CONFIGURED before reaching handler
Missing model field → 400 VALIDATION_ERROR (field model; unchanged Phase 1 contract)
Phase 4: adding a new provider requires only registering it in main.go — no middleware changes