Milestone 2.5.3 — Async Trace Emitter and Handler Integration

Status: Complete
Goal: 2.5 — Async trace emission to ClickHouse
Phase: 2 — Single Provider End-to-End
Estimated effort: 2 days

Why This Milestone Exists

With the ClickHouse schema (2.5.1) and typed writer (2.5.2) in place, this milestone wires trace emission into the proxy request handler. The trace is assembled after the LLM response is complete (token counts are only known then) and submitted to the ClickHouse writer's non-blocking queue.

The key constraint: trace emission must never add latency to the LLM response path. The implementation uses the same post-response goroutine pattern as session checkpoints (2.4.3).

Branch

feature/m2-5-3-async-trace-emitter

PR Title

feat(proxy): async trace emitter — ClickHouse integration in LLM handler (m2.5.3)

Deliverables

Trace assembly function

// assembleTrace builds a TraceRecord from the request context and response data.
// Called after the LLM response is complete (streaming: after [DONE]; non-streaming: after decode).
// Content (prompt text, completion text) is intentionally excluded — never stored.
func assembleTrace(
    ctx context.Context,
    req provider.Request,
    resp provider.Response,
    timings requestTimings,
    outcome requestOutcome,
) clickhouse.TraceRecord

Post-response async (bounded)

// After response is written to client — submit to bounded pool/queue (see Backpressure).
// Do NOT spawn unbounded go func() per request.
// WithoutCancel keeps request-scoped logger/trace values; timeout detaches cancel.
asyncPool.Submit(func() {
    postCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 5*time.Second)
    defer cancel()
 
    // 1. Append checkpoint (from 2.4.3)
    if err := sessionStore.AppendCheckpoint(postCtx, checkpointParams); err != nil {
        log.WarnCtx(postCtx, "session checkpoint failed", "error", err)
    }
 
    // 2. Emit trace
    trace := assembleTrace(postCtx, req, resp, timings, outcome)
    if err := traceWriter.Write(trace); err != nil {
        log.WarnCtx(postCtx, "trace emit failed", "error", err)
        // Both failures are advisory; the LLM response was already sent successfully
    }
})

Backpressure (required)

Same contract as 2.4.3: bound post-response work with a worker pool or drop-oldest queue. Register via packages/metrics:

• ibex_proxy_async_queue_depth
• ibex_proxy_async_dropped_total

Acceptance Criteria

• Trace record in ClickHouse for every completed LLM request within 500ms of response
• Trace includes all fields from the schema (no nil/zero fields for non-optional data)
• Trace write failure does NOT affect LLM response (logged at WARN, not propagated)
• No LLM content (prompt, completion text) in trace record
• proxy_overhead_ms (total_latency_ms) in trace matches measured proxy overhead
• Async emission uses bounded pool/queue; ibex_proxy_async_queue_depth and ibex_proxy_async_dropped_total metrics present

Implementation notes (complete)

• assembleTrace + post-response enqueuePostResponse on the existing non-dropping checkpoint asyncpool
• Success (stream/non-stream), incomplete stream, and provider-failure emit paths
• Auth/directive stage latencies on context; checkpoint_id left nil (store does not return IDs yet)
• Optional when CLICKHOUSE_DSN unset (TraceWriter nil)