Embedder
The embedder owns the embedding contract, profile registry, geometry validation, and backend selection. The CPU stub, TEI, and hosted backends are implemented. Readiness depends on the selected profile's external service and credentials.
Profiles and geometry
| Profile | Default model | Default dimension | Backend | What readiness proves |
|---|---|---|---|---|
cpu | all-MiniLM-L6-v2 | 384 | Deterministic stub | Contract plumbing only |
gpu | BAAI/bge-m3 | 1024 | TEI HTTP | TEI health, model ID, and observed dimensions |
hosted | text-embedding-3-large | 3072 | OpenAI or Cohere API | Credentials and a successful probe at the configured dimension |
A profile name is not a quality claim. The CPU stub is deterministic and useful for tests; it is not a semantic model.
Hosted geometry
| Provider | Models and dimensions |
|---|---|
| OpenAI | text-embedding-3-large 1–3072 via Matryoshka; text-embedding-3-small 1–1536; text-embedding-ada-002 fixed 1536 |
| Cohere | embed-english-v3.0 and embed-multilingual-v3.0 fixed 1024; light variants fixed 384 |
| Voyage | Accepted in settings but fail-closed; no client is implemented |
Base URLs must use HTTPS and must not contain userinfo. The hosted API key is a secret and is never logged. The canonical key variable is IBEX_EMBEDDING_HOSTED_API_KEY; the OpenAI-only OPENAI_EMBEDDING_API_KEY is a migration alias.
Selection and fallback
app/factory.py maps the profile to exactly one backend. There is deliberately no silent fallback from gpu or hosted to the stub: that would produce wrong-geometry vectors and corrupt the index.
| Misconfiguration | Result |
|---|---|
| Unknown profile | Startup validation failure |
gpu without TEI base URL | BackendUnavailableError; not ready |
hosted without API key | BackendUnavailableError; not ready |
hosted with Voyage | Fail closed; unsupported client |
| Cache enabled without Redis URL | Startup validation failure |
Readiness is an active probe
GET /health is liveness. GET /ready is only true after startup checks complete:
- TEI: poll
/healthwithinIBEX_EMBEDDING_TEI_HEALTH_TIMEOUT_SECONDS; require a model ID from/info; reject a reported dimension mismatch; then run one bounded embed probe and require the configured dimension. - Hosted: run one bounded embed probe and require the configured dimension. A mismatch is reported as a geometry failure, not a generic vector error.
- Cache: ping Redis when caching is enabled.
Dimension mismatch is a startup/readiness failure, never a silent truncation. Never write a vector whose dimension does not match the index.
Configuration
| Variable | Meaning |
|---|---|
IBEX_EMBEDDING_PROFILE | cpu, gpu, or hosted; default cpu |
IBEX_EMBEDDING_DIM | Expected vector dimension; profile default when unset |
IBEX_EMBEDDING_MODEL | Model identifier; profile default when unset |
IBEX_EMBEDDING_TEI_BASE_URL | TEI origin for gpu |
IBEX_EMBEDDING_TEI_ALLOW_INSECURE | Development-only cleartext escape hatch |
IBEX_EMBEDDING_TEI_API_KEY | Optional TEI bearer token; forbidden with insecure HTTP |
IBEX_EMBEDDING_HOSTED_PROVIDER | openai or cohere (Voyage fails closed) |
IBEX_EMBEDDING_HOSTED_BASE_URL | Optional HTTPS origin override |
IBEX_EMBEDDING_HOSTED_API_KEY | Hosted secret; never log |
IBEX_EMBEDDING_CACHE_ENABLED | Wrap the backend in the Redis cache |
IBEX_EMBEDDING_CACHE_TTL_SECONDS | Cache TTL; default 86400 |
IBEX_EMBEDDING_CACHE_REDIS_URL | Cache Redis; falls back to REDIS_URL |
IBEX_EMBEDDING_API_TOKEN | Internal bearer for POST /v1/embed |
Tenant-scoped cache
The optional cache is a content-addressed Redis decorator. Keys are:
{org_id}:embed:v1:{sha256(length-prefixed model_id, dimensions, utf-8 text)}
The digest is over exact UTF-8 bytes; there is no Unicode normalization or case folding, because the provider inference input must match. The cache requires organization context from the request; without it the backend fails closed rather than using a global key.
Cache reads and writes fail open on Redis errors: the request falls through to the embedding backend instead of failing inference. Hits are validated for dimension and sentinel shape before use. Duplicate texts within a batch share one digest and one backend call.
Operational rules
- Changing model or dimension is an index migration, not a configuration tweak. Re-embed and verify the pgvector index before switching traffic.
- Do not log texts, vectors, tokens, or API keys. Log backend, batch size, latency, and error class only.
- A readiness probe that passes at startup does not prove the provider stays healthy; monitor backend errors and latency.
- Keep embedder credentials server-side and out of Console/browser configuration.
- A CPU profile is not an acceptable production embedding quality claim.
Run locally
cd services/embedder
uv sync --frozen --extra dev
IBEX_EMBEDDING_PROFILE=cpu .venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8004
The compose and observability examples use port 8004; use the port you actually bind.
Limits
- No hosted embedding quality benchmark is published.
- TEI capacity and GPU scheduling are environment-specific.
- The CPU stub does not model real semantic neighborhoods.
- Cache fail-open means a Redis outage increases provider load rather than failing requests.
- Voyage is not implemented.
Related
- Extraction — where embeddings enter the write path
- Memory writes — embedding and index persistence
- Memory search — vector retrieval behavior
- Extraction evaluation — quality evidence boundaries