3.8 KiB
ADR 0059: Isolated RC embedding comparison and consumer cutover
Status: proposed; implementation and runtime qualification pending.
Context
The RC product contract requires a disposable validation-canary node with
production_eligible=false. Stable serving promotion and the approved lifecycle
transition require a production-eligible node and production approval. Relaxing
those checks for RC testing would grant a LAB deployment production authority.
The selected-comparison workflow currently compares a stable deployment with one LAB experiment route. An isolated RC stack needs two LAB routes, one for its current embedding identity and one for the candidate. Different embedding-space IDs require separately pinned Qdrant collections even when vector dimensions match. The experiment routes are capability-wide; changing their targets would also affect any other client using the same key.
Decision
- Keep stable deployment selection and production lifecycle rules unchanged.
- In the isolated RC stack, create two distinct LAB_READY deployments with approved LAB execution and opaque, immutable experiment routes. Give the designated ExampleRAG project one scoped service client credential.
- Permit a selected comparison with an explicit current experiment route ID. Pin and revalidate both route-to-deployment mappings, full embedding identities, reviewed suite, corpus, retrieval settings, consumer and distinct index references. A stale route, credential, approval or identity fails closed.
- ExampleRAG builds the candidate collection separately and executes both legs through the pinned ModelForge routes. The current collection is never overwritten during comparison. Gateway request IDs and retrieved chunk IDs are retained as evidence.
- For a different embedding space, the RC cutover authority is a single durable ExampleRAG selection record that names both the active embedding profile (including its experiment route key and embedding-space ID) and the Qdrant collection. Ingest and query resolve that same record. Activation and rollback use its monotonically increasing generation and compare-and-swap. They do not retarget a ModelForge experiment route or its production stable route.
- Activation requires the completed, reviewed ModelForge comparison and an explicit operator decision covering impact and reindex. The candidate collection must have a complete authoritative reindex, correct identity pins, and a successful consumer smoke check before the selection changes. A retry with the same operation ID returns the recorded result; a stale generation or different payload conflicts. Rollback must verify the retained previous collection/profile and current generation before restoring the prior selection. Calls already admitted may finish against the old selection; subsequent calls use the new generation.
- A failed or interrupted operation leaves the previous selection usable or enters an explicit blocked state with a recovery journal. It never silently falls back to Ollama or an unpinned collection.
Qualification boundary
This is an RC consumer transition on disposable state. It does not claim production promotion, a stable route change or a production rollout. The final candidate must prove the full comparison, activation, real request, rollback, second real request, duplicate/retry behavior, stale CAS, concurrent operators, and process-restart recovery on the same exact images and source revision.
Consequences
ModelForge remains the source of model identity, LAB execution, gateway, scheduler and comparison evidence. ExampleRAG owns the atomic choice of its embedding route and matching index. A global experiment-route remap is not used as a shortcut for consumer cutover. The RC cannot be marked qualified until the selection record and its runtime tests are implemented and verified.