Milestone 1.2.6 — Request ID Generation and Correlation Middleware

Status: Complete
Goal: 1.2 — Proxy platform integration
Phase: 1 — Core Platform
Estimated effort: 2 days
ADR required: ADR-0017 — Request ID and trace context strategy

Why This Milestone Exists

ARCHITECTURE.md mandates a structured log format with request_id and trace_id fields in every log line. Without a middleware that generates a canonical request ID on every inbound request, log lines from different stages of the same request (auth gRPC call, rate limit check, handler) cannot be correlated. This makes debugging in production effectively impossible.

M1.3.1 mentions "propagate trace_id in logs" as a single bullet point. That bullet depends on a request ID existing in context — which this milestone provides.

Status note (post-M1.2.3): ADR-0013 and M1.2.3 already introduced RequestContextMiddleware, ResponseHeadersMiddleware, and X-Request-ID/X-Trace-ID/X-Response-Time headers on all proxy responses, and the request_id field in the error envelope is now populated. This milestone is therefore partially complete. The remaining work is limited to formalizing the request ID strategy and closing the gaps listed below; it must not re-implement middleware that already exists.

This milestone now delivers the remaining gaps only:

• packages/reqid (UUID v7 generation and context helpers; today the request ID is a UUID v4)
• gRPC client interceptor propagating x-request-id to the auth service
• ADR-0017 documenting the relationship between request ID and trace ID, and its interaction with ADR-0013

Non-Goals

• W3C traceparent header propagation (that is OTel's job, covered in M1.3.1)
• Request ID in ClickHouse trace records (Phase 2)
• Request ID in gRPC calls beyond auth (no other gRPC calls exist in Phase 1)

Branch

feature/m1-2-6-request-id-middleware

PR Title

feat(proxy): request ID generation and context correlation middleware (m1.2.6)

Prerequisites

• 1.2.3 merged — stable error envelope in place; request_id field already appears in the envelope but is never populated

Deliverables

1. ADR-0017 — Request ID strategy

Write docs/adr/ADR-0017-request-id-strategy.md covering:

• UUID v7 vs v4: UUID v7 is time-ordered (first 48 bits are a millisecond timestamp). This makes log entries sortable by request ID without a timestamp field, and makes IDs identifiable as being from a given time window. UUID v4 is random and provides no ordering.
• Header name: X-Request-ID is the de facto standard (Heroku, AWS API Gateway, GCP Load Balancer). We honour an inbound X-Request-ID from upstream gateways and generate one if absent.
• gRPC metadata key: x-request-id (lowercase, per gRPC metadata conventions)
• Relationship to OTel trace ID: The request ID is our internal correlation token. The OTel trace ID (added in M1.3.1) is the distributed tracing token. They coexist. The request ID appears in all log lines; the trace ID appears only when the OTel tracer is active.

2. packages/reqid — UUID v7 generation

// Package reqid provides request ID generation and context propagation.
//
// Request IDs are UUID v7 (RFC 9562): time-ordered, monotonically
// increasing within a millisecond, globally unique across instances.
// They are safe to expose to callers in response headers and error bodies.
package reqid
 
import (
    "context"
    "net/http"
 
    "github.com/google/uuid"
)
 
// contextKey is the unexported type for reqid context values.
type contextKey struct{}
 
// New generates a new UUID v7 request ID.
// Falls back to UUID v4 if the system clock is unavailable.
func New() string {
    id, err := uuid.NewV7()
    if err != nil {
        return uuid.New().String() // v4 fallback
    }
    return id.String()
}
 
// WithRequestID returns a context with the given request ID attached.
func WithRequestID(ctx context.Context, id string) context.Context {
    return context.WithValue(ctx, contextKey{}, id)
}
 
// FromContext retrieves the request ID from ctx.
// Returns ("", false) if not set.
func FromContext(ctx context.Context) (string, bool) {
    v, ok := ctx.Value(contextKey{}).(string)
    return v, ok
}
 
// MustFromContext retrieves the request ID or panics.
// Use only in handlers that are guaranteed to run after RequestIDMiddleware.
func MustFromContext(ctx context.Context) string {
    id, ok := FromContext(ctx)
    if !ok {
        panic("reqid: request ID not in context; is RequestIDMiddleware wired?")
    }
    return id
}
 
// Header is the canonical HTTP header name for the request ID.
const Header = "X-Request-ID"
 
// GRPCMetadataKey is the gRPC metadata key for request ID propagation.
const GRPCMetadataKey = "x-request-id"

3. HTTP middleware

// RequestIDMiddleware is the first middleware in the proxy chain.
// It generates or honours a UUID v7 request ID and attaches it to
// the request context and response headers.
//
// Inbound X-Request-ID is honoured if it is a valid UUID (v4 or v7)
// to support tracing across upstream gateways. Invalid inbound values
// are discarded and a fresh ID is generated.
//
// Required middleware position: FIRST — before auth, rate limit, handlers.
//   RequestID → Auth → AgentVerification → RateLimit → [handler]
func RequestIDMiddleware() func(http.Handler) http.Handler {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            id := r.Header.Get(reqid.Header)
            if _, err := uuid.Parse(id); err != nil {
                id = reqid.New()
            }
            ctx := reqid.WithRequestID(r.Context(), id)
            w.Header().Set(reqid.Header, id)
            next.ServeHTTP(w, r.WithContext(ctx))
        })
    }
}

4. gRPC interceptor for auth client

Propagate request_id in gRPC metadata on every call from proxy to auth:

// RequestIDUnaryInterceptor is a gRPC UnaryClientInterceptor that
// injects the request ID from the Go context into gRPC call metadata.
// Attach to the auth gRPC client connection in proxy main.go.
func RequestIDUnaryInterceptor() grpc.UnaryClientInterceptor {
    return func(
        ctx context.Context,
        method string,
        req, reply any,
        cc *grpc.ClientConn,
        invoker grpc.UnaryInvoker,
        opts ...grpc.CallOption,
    ) error {
        if id, ok := reqid.FromContext(ctx); ok {
            ctx = metadata.AppendToOutgoingContext(ctx, reqid.GRPCMetadataKey, id)
        }
        return invoker(ctx, method, req, reply, cc, opts...)
    }
}

5. Error envelope — populate request_id field

M1.2.3 introduced the error envelope with a request_id field always set to "". This milestone populates it:

// In proxy error handlers, replace:
//   requestID: ""
// With:
//   requestID: reqid.MustFromContext(r.Context())

Files Affected

Testing Requirements

• TestRequestIDMiddleware_Generates: no inbound header → response has X-Request-ID that is a valid UUID v7
• TestRequestIDMiddleware_Honours: valid UUID v4 in inbound header → same UUID in response
• TestRequestIDMiddleware_RejectsInvalid: garbage string in inbound header → fresh UUID v7 in response (not the garbage string)
• TestRequestIDMiddleware_ContextPropagation: ID is accessible via reqid.FromContext inside handler
• TestRequestIDInErrorEnvelope: error response body contains request_id matching X-Request-ID header

Acceptance Criteria

• Every response includes X-Request-ID header with a valid UUID
• Valid inbound X-Request-ID is honoured; invalid is replaced
• request_id field in error envelope matches X-Request-ID response header
• Request ID propagated to auth gRPC calls via metadata
• reqid.FromContext available in all middleware and handlers
• ADR-0017 written and indexed

Risks