With the session store in place (2.4.2), the proxy needs to wire session lifecycle into the request handler — after authentication and before LLM forwarding for session creation, and after the LLM response for checkpoint creation. Session writes are non-blocking (async after response) to avoid adding to proxy overhead.
Milestone 2.4.3 — Proxy Session Lifecycle Management
Status: Completed
Goal: 2.4 — Session tracking infrastructure
Phase: 2 — Single Provider End-to-End
Estimated effort: 2–3 days
Contract note: Response
X-IBEX-Session-IDechoes the sticky client key stored assessions.external_id(minted UUID when absent). It is notsessions.id. Multi-turn reuse sends that same external id back on subsequent requests.
Why This Milestone Exists
With the session store in place (2.4.2), the proxy needs to wire session lifecycle into the request handler — after authentication and before LLM forwarding for session creation, and after the LLM response for checkpoint creation. Session writes are non-blocking (async after response) to avoid adding to proxy overhead.
Branch
feature/m2-4-3-proxy-session-lifecycle
PR Title
feat(proxy): session lifecycle management in LLM request handler (m2.4.3)
Session ID propagation
The client optionally sends X-IBEX-Session-ID (sticky external_id) to associate a request with an existing session. The proxy returns X-IBEX-Session-ID on every LLM response when session lifecycle is active — including GetOrCreate fail-open (minted or client-supplied sticky id echoed for reuse). Oversized client values are discarded and replaced with a minted sticky id. Fail-open cases are surfaced via warn logs (session get_or_create failed) and ibex_proxy_session_get_or_create_total{result="error"}.
Client → Proxy: X-IBEX-Session-ID: <uuid> (optional)
Proxy → Client: X-IBEX-Session-ID: <external_id sticky key> (when resolved/sticky; not sessions.id)The non-blocking session write pattern
1. [Hot path] Resolve session (cache: Redis session state, miss: Postgres GetOrCreate)
2. [Hot path] Forward request to LLM provider
3. [Hot path] Stream/return response to client — session ID in response header
4. [Async, after response] AppendCheckpoint (Postgres write — does NOT block client)
5. [Async, after response] Update session statsThe async post-response processing uses a service-owned context (not the
HTTP request context) with a bounded worker pool tracked by sync.WaitGroup.
Each task gets its own 5-second deadline. Startup registers the pool with
packages/shutdown; shutdown drains queued tasks (does not cancel in-flight
work mid-write) then waits for the WaitGroup.
Backpressure (required)
Unbounded post-response goroutines will OOM under load. Bound async work:
- Checkpoints (required): fixed-concurrency worker pool or a non-dropping durable queue / outbox. Checkpoint tasks must not be silently discarded; if the queue is full, apply backpressure to the submitter or spill to durable outbox and recover when capacity returns. Expired (5s) or processing errors must be logged/metrics-reported — never swallowed without a signal.
- Telemetry / traces (advisory): may use a fixed-capacity drop-oldest queue; only non-critical telemetry may be dropped when full.
- Emit metrics via
packages/metrics:ibex_proxy_async_queue_depth— current queue depth (checkpoints + telemetry)ibex_proxy_async_dropped_total— tasks dropped when a telemetry queue is full (checkpoint drops must be zero under the durable-queue contract)
Acceptance Criteria
-
X-IBEX-Session-IDreturned when a sticky external id is available (minted or client-supplied), including GetOrCreate fail-open; oversized client values are discarded and replaced with a minted sticky id - Checkpoint written after every completed turn with a durable session (streaming and non-streaming); skipped on sticky-only fail-open (no SessionID)
- Checkpoint write failure does NOT affect LLM response (async)
- Session created automatically when
X-IBEX-Session-IDabsent from request - Session reused when
X-IBEX-Session-IDmatches existing session in same org/agent - Cross-org session ID reuse is impossible (session lookup always includes org_id)
- Post-response work is bounded (service-owned pool + WaitGroup; drain on shutdown); checkpoints use non-dropping path; metrics
ibex_proxy_async_queue_depthandibex_proxy_async_dropped_totalregistered viapackages/metrics; fail-open surfaced via warn logs +ibex_proxy_session_get_or_create_total{result="error"}
Last updated on