Phase 2.5 — Provider Generalization & Foundation

What Phase 2.5 Delivers

Phase 2 proved an OpenAI-compatible proxy works end-to-end. Phase 2.5 widens the foundation so that Phase 3's memory engine, Phase 3.5's context assembly, and Phase 4's multi-provider routing all build on proven abstractions instead of inventing them under pressure. Every track in this phase creates a seam that a later phase fills with real logic — the same "build the plumbing before the thing that uses it" pattern that made Phase 1 (auth/proxy skeleton) a clean foundation for Phase 2 (real forwarding).

Phase 2.5 Scope

Seven tracks

Track	Theme	Key Deliverable
A	Multi-provider adapters	Anthropic adapter proving `Provider` interface generalizes; model capability registry
B	Tokenizer registry	Per-model token counting via Hugging Face `tokenizers` (CGo in proxy, Python service for context assembly)
C	Response-side processing	`ResponsePipeline` stage interface; typed decode/re-encode of upstream responses; streaming design ADR
D	Pluggable embedding service	`Embedder` interface + registry; TEI GPU backend, hosted OpenAI/Cohere backends, content-hash cache (CPU MiniLM stub)
E	Schema pre-work	Temporal validity columns, multi-label category support, relationship graph index — migrations only, no application code
F	MCP server skeleton	Auth-boundary-scoped MCP server exposing stub memory tools; resource-server auth via `AuthService.ValidateToken`
G	Exit / ops pack-up	Local LGTM observability, `verify-phase25`, thin K8s observability chart, roadmap sign-off

Recommended Execution Order

Track A (Multi-provider — start immediately):
 2.5.G1.M1 (Anthropic adapter) → 2.5.G1.M2 (capability registry) → 2.5.G1.M3 (self-hosted vLLM adapter)

Track B (Tokenizer — blocks context assembly):
 2.5.G2.M1 (tokenizer registry)

Track C (Response pipeline — seam for Phase 3 guardrails):
 2.5.G3.M1 (non-streaming response middleware) → 2.5.G3.M2 (streaming design spike / ADR)

Track D (Embedding — blocks memory writes):
 2.5.G4.M1 (Embedder interface + registry) → 2.5.G4.M2 (TEI backend)
 → 2.5.G4.M3 (hosted-API backend) → 2.5.G4.M4 (content-hash cache)

Track E (Schema — parallel with A/B):
 2.5.G5.M1 (temporal validity) → 2.5.G5.M2 (multi-label categories)
 → 2.5.G5.M3 (relationship graph index)

Track F (MCP skeleton — after A):
 2.5.G6.M1 (MCP server with stub tools)

Track G (Exit / ops — after A–F):
 2.5.G7 (exit gate, LGTM observability, verify-phase25)

Tech Stack Additions

Concern	Choice	Reason
Response pipeline typed model	Go structs mirroring OpenAI chat response shape, in `packages/responsepipeline`	Keeps decode/re-encode in the proxy's hot language
Embedding backend abstraction	Python ABC (`EmbeddingBackend`), profile selected by `EMBEDDER_PROFILE` env at startup	Mirrors the proven `provider.Provider`/`Registry` pattern
Default production embedding model	`BAAI/bge-m3` (1024-dim)	Best available open retrieval-tuned model; multilingual + long-context
CPU MiniLM profile	Stub only (ADR-0046); real CPU path deferred	Avoid shipping an unfinished local transformer path as a “real” backend
Embedding serving runtime (GPU)	Hugging Face TEI (Apache-2.0)	Eliminates hand-rolled batching code entirely
Hosted embeddings	OpenAI and/or Cohere HTTP backends	Second real backend family for contract parity without GPU
MCP SDK	Official `mcp` Python SDK	Tracks an evolving spec
MCP auth	Reuse existing `AuthService.ValidateToken` gRPC	One identity system, not two
Local observability	Prometheus + Grafana + Tempo + Loki + OTel Collector (compose)	Pull-forward for Phase 2.5 exit (ADR-0051); HA platform remains later

Exit Criteria

Anthropic adapter passes the same streaming/non-streaming/error-mapping test depth as the OpenAI adapter
Registry.For(model) resolves both OpenAI and Anthropic models without handler/middleware changes
ModelCapability struct queryable for context window, tool support, and tokenizer family
Response pipeline decode/re-encode round trip is byte-for-byte identical to passthrough (zero stages) with p99 overhead < 2ms
Embedder interface has at least two real backends (TEI + hosted OpenAI or Cohere) passing the same contract test suite (CPU MiniLM remains stub)
All three schema migrations (temporal validity, multi-label, relationship graph index) apply cleanly via make db-migrate
MCP server passes official MCP conformance against stub handlers with auth validated via gRPC
All Phase 1 and Phase 2 tests still pass with zero regressions
Local LGTM observability stack is operable (make observability-up / make observability-smoke) with Proxy Critical Path dashboard
make verify-phase25 exits 0 (phase pack-up gate)