Phase 4.5 intelligence layer

10-case integration test matrix proving fingerprinting, drift detection, and directive regression work as one composed pipeline — not just in isolation.

Milestone 4.5.D.1 — Cross-Track Integration Suite

Status: Planned
Goal: Track D — Exit Gate
Phase: 4.5 — Intelligence Layer
Estimated effort: 3 days
Track: Track D — Exit Gate


Why This Milestone Exists

Phase 1's audit proved individual milestone checkboxes don't guarantee composed behavior works. This suite proves the full pipeline: drifted agent fingerprint → dashboard alert → linked regression run → gated promotion. Uses real Postgres + Redis via testcontainers.


INT-4.5 Test Matrix

IDScenarioAssertion
INT-4.5.1Fresh agent, first 20 sessionsis_baseline=True fingerprint, no drift alert
INT-4.5.2Stable agent, sessions 21–40Low/absent severity, no alert
INT-4.5.3Injected drift at session 25Alert fires within one fingerprinting cycle
INT-4.5.4Drift alert → regression from suggested-actionsubmit-review triggers against current active directive
INT-4.5.52 critical-scenario failurespromote rejects with 403
INT-4.5.6Regression passes, gradual rolloutAdvances only if drift/error within threshold
INT-4.5.7Fingerprinting disabled for orgZero fingerprint rows written
INT-4.5.8Auth service outage during fingerprintWorker retries, no corrupt rows
INT-4.5.9Cross-tenant: Org A alert invisible to Org B403 on direct API call
INT-4.5.10Rollout paused manuallyResumes from paused percentage, not 0

Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

  • open the live tree and follow existing patterns before inventing new ones
  • research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
  • advance the design beyond the sketch where measurement or code reality says so
  • land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

  • Evaluation / benchmarks
  • Fingerprinting / drift
  • Directive regression / promotion

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

  • All 10 INT-4.5 cases pass with pytest -m integration, real Postgres + Redis
  • Suite runtime < 90s in CI
  • CI job intelligence-integration added to required checks on main

Prerequisites

  • All Track A–C milestones complete
Edit on GitHub

Last updated on

On this page

0%