Milestone 2.1.4 — Provider Routing Middleware

Status: Planned
Goal: 2.1 — LLM provider abstraction and OpenAI forwarding
Phase: 2 — Single Provider End-to-End
Estimated effort: 1–2 days

Why This Milestone Exists

With the provider interface and OpenAI client in place, the proxy needs a middleware that selects the correct provider for each request and attaches it to the request context. The selection logic: read the model field from the request body (already parsed by M1.2.2), look up the provider registry, and attach the selected provider to context. If no provider is registered for the model, return 501 PROVIDER_NOT_CONFIGURED.

This is a small but architecturally important milestone: it is the single place where the model-to-provider mapping is enforced. Phase 4 adds Anthropic and Azure providers by registering them in the registry — this middleware requires no changes.

Branch

feature/m2-1-4-provider-routing

PR Title

feat(proxy): provider routing middleware — selects provider by model (m2.1.4)

Deliverables

Middleware

// ProviderRoutingMiddleware selects the LLM provider for the request based on
// the `model` field extracted during request normalisation (M1.2.2).
// Attaches the selected provider.Provider to the request context.
//
// Returns 501 PROVIDER_NOT_CONFIGURED if no provider supports the requested model.
// Returns 400 INVALID_REQUEST if the model field is absent from the request.
//
// Required position: AFTER RateLimitMiddleware, BEFORE the forwarding handler.
//   RequestID → Span → Auth → AgentVerify → RateLimit → ProviderRouter → [handler]
func ProviderRoutingMiddleware(registry *provider.Registry, log *logger.Logger) func(http.Handler) http.Handler
 
// ProviderFromContext retrieves the selected provider from ctx.
// Panics if called outside a handler that ran ProviderRoutingMiddleware.
func ProviderFromContext(ctx context.Context) provider.Provider

Full middleware chain (Phase 2)

RequestID → OTel Span → Auth (LRU+gRPC) → AgentVerify → RateLimit → DirectiveResolve → ProviderRouter → Handler

Acceptance Criteria

• Known model (e.g. gpt-4o) → provider attached to context → handler executes
• Unknown model → 501 PROVIDER_NOT_CONFIGURED before reaching handler
• Missing model field → 400 INVALID_REQUEST
• Phase 4: adding a new provider requires only registering it in main.go — no middleware changes