Milestone 3.5.C.5 — Context Formatter

Status: Planned
Goal: Track C — Context Assembly Engine
Phase: 3.5 — Extraction & Context Assembly
Estimated effort: 2 days
Track: Track C — Context Assembly Engine
Depends on: Milestone 3.5.C.4 (packer v2)

Why This Milestone Exists

The existing formatter design is good: XML-style delimiters, category grouping, session nonce to prevent prompt-injection via fake <memory> tags. However, three source-of-truth documents disagree on where conversation history goes in the ordering. This milestone locks a single ordering and captures it in an ADR.

Final Locked Ordering

1. Directive (always first — sets behavior before any content)
2. Conversation history (recent turns — benefits from recency-in-context positioning)
3. Memories by category (procedural → factual → preference → behavioral → episodic)
4. Tool schemas (if applicable)

Rationale for History in position 2: the "lost in the middle" phenomenon means LLMs attend most strongly to the start and end of context. Conversation history is what the current turn is about — it needs to be positioned where attention is reliably high. Putting it last would bury the most immediately relevant content in the "middle" once memories are injected.

Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

open the live tree and follow existing patterns before inventing new ones
research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
advance the design beyond the sketch where measurement or code reality says so
land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

Context assembly service
Tokenizer registry / counting

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

CATEGORY_ORDER unchanged from original design (already correct)
History injected as position 2, formatted as native role: user/assistant messages (not XML-wrapped)
ADR-0042 amended with the final locked ordering and explicit rationale, superseding the conflicting statements
Golden-output test: fixed input (directive + 2 history turns + 3 memories across categories) → exact expected formatted output, committed as a regression fixture
Repo guards / CI checks still pass

Prerequisites

Milestone 3.5.C.4 (packer v2)