67 lines
3.8 KiB
Markdown
67 lines
3.8 KiB
Markdown
# ADR 0059: Isolated RC embedding comparison and consumer cutover
|
|
|
|
Status: proposed; implementation and runtime qualification pending.
|
|
|
|
## Context
|
|
|
|
The RC product contract requires a disposable `validation-canary` node with
|
|
`production_eligible=false`. Stable serving promotion and the approved lifecycle
|
|
transition require a production-eligible node and production approval. Relaxing
|
|
those checks for RC testing would grant a LAB deployment production authority.
|
|
|
|
The selected-comparison workflow currently compares a stable deployment with
|
|
one LAB experiment route. An isolated RC stack needs two LAB routes, one for its
|
|
current embedding identity and one for the candidate. Different embedding-space
|
|
IDs require separately pinned Qdrant collections even when vector dimensions
|
|
match. The experiment routes are capability-wide; changing their targets would
|
|
also affect any other client using the same key.
|
|
|
|
## Decision
|
|
|
|
- Keep stable deployment selection and production lifecycle rules unchanged.
|
|
- In the isolated RC stack, create two distinct LAB_READY deployments with
|
|
approved LAB execution and opaque, immutable experiment routes. Give the
|
|
designated ExampleRAG project one scoped service client credential.
|
|
- Permit a selected comparison with an explicit current experiment route ID.
|
|
Pin and revalidate both route-to-deployment mappings, full embedding
|
|
identities, reviewed suite, corpus, retrieval settings, consumer and distinct
|
|
index references. A stale route, credential, approval or identity fails closed.
|
|
- ExampleRAG builds the candidate collection separately and executes both legs
|
|
through the pinned ModelForge routes. The current collection is never
|
|
overwritten during comparison. Gateway request IDs and retrieved chunk IDs
|
|
are retained as evidence.
|
|
- For a different embedding space, the RC cutover authority is a single durable
|
|
ExampleRAG selection record that names **both** the active embedding profile
|
|
(including its experiment route key and embedding-space ID) and the Qdrant
|
|
collection. Ingest and query resolve that same record. Activation and rollback
|
|
use its monotonically increasing generation and compare-and-swap. They do not
|
|
retarget a ModelForge experiment route or its production stable route.
|
|
- Activation requires the completed, reviewed ModelForge comparison and an
|
|
explicit operator decision covering impact and reindex. The candidate
|
|
collection must have a complete authoritative reindex, correct identity pins,
|
|
and a successful consumer smoke check before the selection changes. A retry
|
|
with the same operation ID returns the recorded result; a stale generation or
|
|
different payload conflicts. Rollback must verify the retained previous
|
|
collection/profile and current generation before restoring the prior
|
|
selection. Calls already admitted may finish against the old selection;
|
|
subsequent calls use the new generation.
|
|
- A failed or interrupted operation leaves the previous selection usable or
|
|
enters an explicit blocked state with a recovery journal. It never silently
|
|
falls back to Ollama or an unpinned collection.
|
|
|
|
## Qualification boundary
|
|
|
|
This is an RC consumer transition on disposable state. It does not claim
|
|
production promotion, a stable route change or a production rollout. The final
|
|
candidate must prove the full comparison, activation, real request, rollback,
|
|
second real request, duplicate/retry behavior, stale CAS, concurrent operators,
|
|
and process-restart recovery on the same exact images and source revision.
|
|
|
|
## Consequences
|
|
|
|
ModelForge remains the source of model identity, LAB execution, gateway,
|
|
scheduler and comparison evidence. ExampleRAG owns the atomic choice of its
|
|
embedding route and matching index. A global experiment-route remap is not used
|
|
as a shortcut for consumer cutover. The RC cannot be marked qualified until the
|
|
selection record and its runtime tests are implemented and verified.
|