Milestone 3.D.1 — Semantic search read path

Status: Planned
Goal: Track D — Read/Ranking Pipeline
Phase: 3 — Core Memory Substrate
Estimated effort: 2 days

Why This Milestone Exists

Semantic search should use cosine distance with org/agent filtering and a confidence threshold. Preferred starting shape: route through a pluggable vector-store interface (HNSW search settings benchmarked rather than copied from IVFFlat), and keep a GIN full-text fallback when vector results are sparse. Exact SQL and helper layout should follow live repository patterns.

Non-Goals

Composite scoring (Milestone 3.D.2)
Hot cache (Milestone 3.D.3)
Context assembly integration (Phase 3.5)

Orientation (indicative)

Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.

During implementation, expect to:

open the live tree and follow existing patterns before inventing new ones
research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
advance the design beyond the sketch where measurement or code reality says so
land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it

Prefer outcomes over matching any particular file tree or command sequence.

Areas that may be involved (situational — not a checklist):

Memory service / repositories
Vector store / search
Caching / Redis

Suggested naming (provisional)

Rename freely to match the change that actually lands.

Branch: feature/m3-d-1-semantic-search
PR title: feat(memory): semantic search read path with HNSW and GIN fallback (m3.D.1)

Design

Primary path: HNSW vector search

find_similar delegates to VectorStore.search(). The repository layer does not embed raw SQL vector queries — all vector operations go through the VectorStore interface.

Python

# Illustrative — exact path may differ
 
VECTOR_FALLBACK_THRESHOLD = 5 # if vector results < K, supplement with full-text
 
class MemoryRepository:
 def __init__(
 self,
 session_factory: AsyncSessionFactory,
 vector_store: VectorStore,
 ) -> None: ...
 
 async def find_similar(
 self,
 org_id: UUID,
 agent_id: UUID,
 query_embedding: list[float],
 limit: int = 10,
 min_confidence: float = 0.0,
 ) -> list[MemorySearchResult]:
 vector_results = await self.vector_store.search(
 org_id=org_id,
 agent_id=agent_id,
 query_embedding=query_embedding,
 limit=limit,
 min_confidence=min_confidence,
 )
 
 if len(vector_results) < VECTOR_FALLBACK_THRESHOLD:
 # Hybrid search: supplement with GIN full-text results
 full_text_results = await self._full_text_search(org_id, agent_id, limit)
 vector_ids = {mid for mid, _ in vector_results}
 supplemental = [r for r in full_text_results if r.id not in vector_ids]
 # Merge and return up to limit
 ...
 
 return await self._fetch_memories(org_id, [mid for mid, _ in vector_results])

Fallback path: GIN full-text search

When vector search returns fewer than K results (sparse memory store for a new agent), supplement with GIN full-text search using the search_vector column (created in Milestone 3.1.1):

SQL

SELECT id FROM ibex_core.memories
WHERE org_id = :org_id
 AND agent_id = :agent_id
 AND status = 'active'
 AND deleted_at IS NULL
 AND search_vector @@ plainto_tsquery('english', :query_text)
ORDER BY ts_rank_cd(search_vector, plainto_tsquery('english', :query_text)) DESC
LIMIT :limit

Success signals

Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.

Semantic search uses the HNSW path (confirmed via query plans in tests, not assumed)
Cross-org result leakage is covered by an integration isolation test
Full-text fallback triggers when vector results are sparse
Search calls are always org/agent scoped — no unscoped queries

Prerequisites

Milestone 3.1.1 merged (HNSW index and search_vector GIN index in schema)
Milestone 3.2.1 merged (VectorStore ABC and PgVectorStore implementation available)
Track C milestones (write path should be in place for test fixtures to be loadable)