RRF combiner for dense and sparse result lists — rank-position-only fusion immune to score scale incomparability, with memories in both lists near the top dominating.
Milestone 5.A.2 — Reciprocal Rank Fusion
Status: Planned
Goal: Track A — Hybrid Retrieval
Phase: 5 — Advanced Retrieval
Estimated effort: 2 days
Track: Track A — Hybrid Retrieval
Why This Milestone Exists
RRF combines dense and sparse ranked lists using only rank position (not raw scores), making it immune to the cosine-similarity vs. ts_rank_cd scale incomparability problem. It requires no tuning of weight coefficients per model and is the empirically dominant choice in production hybrid-search systems.
Orientation (indicative)
Named paths, package layouts, libraries, schemas, env vars, and commands anywhere on this page are rough sketches for orientation — inspiration and a baseline, not a required change list.
During implementation, expect to:
- open the live tree and follow existing patterns before inventing new ones
- research current constraints (latency, tenancy, deploy shape, libraries) more deeply than this page can
- advance the design beyond the sketch where measurement or code reality says so
- land work in different filenames, merged packages, deferred docs, or new surfaces when the situation calls for it
Prefer outcomes over matching any particular file tree or command sequence.
Areas that may be involved (situational — not a checklist):
- Hybrid retrieval
- Memory service / repositories
Success signals
Outcome-oriented signals that the milestone is in good shape. Exact filenames, package layouts, and commands may differ from any sketches above.
-
reciprocal_rank_fusionunit-tested against hand-computed expected scores -
hybrid_search.search()orchestratesfind_similar+find_lexicalconcurrently viaasyncio.gather, fuses via RRF - Retrieval-quality benchmark extended with exact-match query subset; hybrid beats pure-dense baseline
Prerequisites
- Milestone 5.A.1 (BM25 sparse retrieval)
Last updated on