Ranking and packing

Context assembly is a bounded selection problem, not “retrieve top-k and concatenate.” The service has to reserve room for the model's response, the directive, recent messages, tool schemas, formatter markup, and safety margin before it can spend tokens on memory.

Retrieval

The service runs three branches concurrently:

There is no fourth history branch. Recent messages arrive in the request and are token-counted directly; the session checkpoint table stores message hashes, not message text, so the context service cannot read history from it.

The outer retrieval wall is min(IBEX_CONTEXT_TIMEOUT, IBEX_CONTEXT_DEADLINE_MS) — by default min(45ms, 40ms). A slow branch is cancelled, while already completed branches are kept. Each branch records success, timeout, or error and can degrade independently.

Cold search embeds the query inside the memory service. The context service must not call the embedder itself; doing so would pay the embedding latency twice inside the deadline.

Budget calculation

BudgetCalculator uses the model capability catalog and a tokenizer-family policy:

usable_budget =
  context_window
  − response_reserve
  − safety_buffer
  − directive_tokens
  − messages_tokens
  − tool_schema_tokens
  − formatter_overhead_tokens

Key properties:

• The response reserve is bounded: at least 500 tokens, at most 4,096, and by default 15% of the context window, not exceeding the model's max output.
• The safety buffer is a fraction of the context window from the tokenizer policy.
• Formatter overhead is estimated before packing: fixed section joins, the nonce-bearing memory wrapper, and the directive safety sentence. This keeps the packer's total_tokens ≤ token_budget invariant meaningful after formatting.
• Usable budget is clamped at zero. A budget below 256 tokens is marked constrained.
• Counts are estimates, not exact tokenizer output. The catalog records the estimate kind and tokenizer family so mixed assumptions fail instead of silently combining.

Memory ranking

There are two different scores, and confusing them produces incorrect expectations.

Memory-service composite

The durable read path ranks hydrated candidates with a weighted sum:

Recency uses category-specific half-lives. Before the weighted sum, rank_hydrated_hits applies a scoring-time relevance floor. The default is 0.15; values at or above 0.5 are rejected so full-text sentinel hits are not mass-excluded.

This is distinct from the retrieval-time cosine cutoff (min_similarity, default 0.70 in the production vector path). The retrieval cutoff runs before hydration; the relevance floor runs after hydration and before composite scoring.

Context packer score

The context service consumes only similarity, confidence, and rank from the memory HTTP response. Until memory exposes its numeric composite on that wire, the packer uses a labeled interim score:

interim = 0.85 × similarity + 0.15 × confidence

Both inputs must be finite and in [0, 1]. The field is named composite_score on the internal ScoredMemory type for formatter/proto alignment, but it is not the memory-service composite. Per-request AssemblyOptions weights are a forward contract and do not yet override this formula.

Packing

The packer solves a 0/1 knapsack under the usable token budget.

1. Sort candidates by descending interim score, then memory_id for deterministic ties.
2. Discretize the budget into 16-token buckets.
3. Run a numpy-vectorized dynamic program when n × (buckets + 1) is within the cell ceiling (default 70 × 6251).
4. If the table is too large, fall back to score-descending greedy with a bounded consecutive-skip limit.
5. Repair the DP result so exact per-memory estimates never exceed the budget.
6. Refill and locally improve after repair instead of accepting floor-bucket stranding.
7. Emit packed memories in descending score, then memory_id.

The result records whether the DP or greedy path ran, how many candidates were evaluated, how many were excluded for budget, and whether the budget was reached. An empty candidate set is not a truncation; a non-positive budget with candidates present is.

Formatting

The formatter is deterministic and treats memory as untrusted data:

• Directive, history, memories, and tools have a fixed order.
• Memory blocks use a per-assembly nonce in the opening tag and an explicit safety sentence telling the model to treat the block as data.
• Attribute values and bodies are escaped so a memory cannot forge a closing tag or inject an element boundary.
• The formatter serializes; it never parses memory markup or executes instructions found in a memory.

Degradation

The proxy may proceed without assembled context when context assembly is disabled, the client is absent, memory is skipped, retrieval fails, or the deadline is exceeded. The fallback is directive-only or history-only context, with headers and metadata recording that assembly was attempted and degraded.

Degradation is quality loss, not an authorization change. A failed retrieval must not widen visibility or invent a memory.

Known limits

• Caller authentication at the context gRPC boundary is not implemented; non-loopback use trusts caller-supplied org_id and agent_id.
• Token counts are conservative estimates, not an exact model tokenizer.
• The packer optimizes the interim score, not the memory-service composite.
• Per-request weight fields are not wired.
• No hosted p99 or end-to-end assembly SLO exists; the deadlines are local budgets.

Related

• Assembly pipeline — service surface and degradation
• Search and lifecycle — memory-side retrieval and filters
• Write path — how candidates are produced
• Request lifecycle — where assembly sits in the proxy path