Publish curated ITWorx-ModelForge source from 990401737f96c8d46a9f7b0d84994516cadcb6a0

docs/architecture/adr/0048-transactional-serving-configuration.md

@@ -0,0 +1,32 @@
+# ADR-0048: Transactional serving configuration
+
+Status: accepted for RC-01, 2026-09-07.
+
+The operator override registry lives in PostgreSQL, alongside its revision history and audit.
+Only explicitly enumerated, non-secret numeric settings consumed by `ServingService` may be
+overridden. A serving request resolves one immutable snapshot on its first policy access, after
+authentication. In-flight requests retain their snapshot. API middleware, process startup, agents and
+workers are not hot-reloaded by this contract; their settings remain deployment-managed.
+
+Precedence is product defaults, Pydantic deployment sources, then operator overrides. The catalog
+does not guess provenance that Pydantic discarded. Preview binds the complete override set,
+effective numeric baseline and active revision to a SHA-256 digest. Apply repeats validation,
+requires that exact preview, and advances the singleton using compare-and-swap. Revision, pointer
+and attributable audit commit together. Audit failure rolls everything back. Concurrent or stale
+previews fail with conflict. Rollback is a new previewed revision, never deletion of history.
+
+Acknowledgement means the control-plane serving settings consumer accepted the typed snapshot for
+new requests. It does not mean an existing request or private worker restarted. Export contains
+portable safe overrides, compatibility metadata and names of secret slots, never credentials or
+credential-bearing connection strings. Import follows the same preview/apply path. Bootstrap
+secrets and deployment settings cannot enter override persistence through import or ordinary edits.
+
+Configuration state and history are authoritative recovery state. A new migration is required;
+qualification must exercise PostgreSQL migration round-trips and concurrent writes. The remaining
+write-only secret lifecycle and interactive identity work are separate RC-01 acceptance criteria.
+
+Implementation note (2026-09-24): the fixed Hugging Face control-plane secret slot was delivered
+under PR #32 and passed exact-source managed full/security run 9327. Together with the safe
+configuration transfer and browser evidence above, this closes RC-CFG-03's functional contract.
+The implementation and its limits are documented in [P01 secret-slot evidence](../../quality/RC_P01_SECRET_SLOTS.md).
+Interactive identity and isolated-RC live provider/root-key qualification remain separate gates.

docs/architecture/adr/0049-named-operator-keys.md

@@ -0,0 +1,30 @@
+# ADR-0049: Named operator keys and bounded Console access
+
+Status: accepted for RC-01, 2026-09-08. Online managed identities extend this deployment-owned
+provider under ADR-0052; its distinct ownership and recovery contract supersede only the online
+identity-administration limitation below. The deployment registry contract remains unchanged.
+
+Support a small deployment-owned registry of named human credentials as the first attributable
+viewer/operator/admin authentication provider. Each immutable deployment record identifies a
+credential, subject, role, SHA-256 digest and required UTC-aware expiry. Generate keys with 256 bits
+of randomness and the distinct `mfop_` namespace. Validate the complete bounded registry at startup;
+reject duplicate JSON fields, credential IDs, digests, conflicting roles and reserved subjects.
+Rotation may temporarily retain two records for the same subject and role. Removing a record takes
+effect when the API is recreated with the replacement configuration. This is a deployment operation,
+not an online secret-management API. There is no new database state or migration.
+
+Keep the legacy administrator header and bootstrap key compatible. Named credentials use that header
+but never fall back to bootstrap privileges, including after expiry/revocation. Machine keys and
+single-use enrollment tokens cannot authenticate as humans. The existing role dependencies remain
+the authorization authority, and audit writers receive the authenticated subject and correlation ID.
+An installation may disable bootstrap access once it has an unexpired named administrator.
+
+The Console verifies identity before loading protected data, exposes subject, role and key expiry,
+and clears its memory-only key on manual lock, reload, credential refusal or after at most fifteen
+minutes. This local lock deadline is distinct from server-enforced key expiry. Protected responses
+carry `Cache-Control: no-store`; requests do not use cookies. Stale responses cannot restore data or
+invalidate a newly unlocked session. A dependency outage and a forbidden operation are not logout.
+
+OIDC/Authentik, server-issued browser sessions, online identity administration and an audited
+write-only deployment-secret lifecycle remain explicit RC limitations. This provider establishes
+real least privilege and attribution without claiming enterprise IAM or SSO completion.

docs/architecture/adr/0050-project-onboarding-and-credential-overlap.md

@@ -0,0 +1,33 @@
+# ADR-0050: Persisted project onboarding and bounded service-key overlap
+
+Status: accepted for RC-02, 2026-09-08.
+
+Persist operator-created projects in the existing registry. A separate immutable onboarding record
+owns owner, environment and labels; existing manifest-backed projects retain their identities and
+bindings. Startup reconciliation must never overwrite operator bindings. New requirements target
+the stable capability gateway, not a model or candidate. Require explicit reindex/schema migration
+support when the contract requires it. A production environment label never grants promotion.
+
+Use the existing hash-only service credential domain for online project-key administration. This
+does not depend on a generic secret vault or change the deployment-owned human identity provider.
+Credential administration remains administrator-only. Project creation and requirements require
+operator; viewers can inspect non-secret metadata and readiness.
+
+Add `superseded_at` and enforce one current non-revoked credential per client with a partial unique
+index. Rotation conditionally claims the exact current credential and creates its replacement in
+the same transaction as audit. A bounded overlap (zero to 24 hours) may retain one predecessor;
+it never extends the predecessor's existing expiry. A subsequent rotation retires older overlaps.
+Typed revocation claims the same current credential and revokes every key of that service identity.
+Concurrent rotation/revocation has one winner. Historical secrets cannot be restored by rotation.
+
+Readiness is derived, not a writable boolean: current credential authentication plus a successful
+gateway request from that exact credential, the current stable deployment and the last 24 hours.
+Persist the credential ID in gateway decision evidence; no plaintext key or input is added. A test
+performed with the previous overlapping key cannot qualify the new key. Missing deployment or
+fresh evidence remains a visible blocker. Engineering readiness never substitutes for lifecycle
+production approval or real-consumer certification.
+
+Migration 0026 preserves existing projects and credential hashes. A mechanical downgrade is
+allowed only before onboarding or rotation history exists. Thereafter recover the verified
+pre-migration backup. Recovery permits exactly an empty onboarding table and null supersession
+metadata during an older-schema upgrade; current-schema restore preserves both exactly.

docs/architecture/adr/0051-manual-canary-observations-are-rehearsal-only.md

@@ -0,0 +1,30 @@
+# ADR-0051: Manual canary observations are rehearsal-only
+
+Status: accepted for P00 containment, 2026-09-08.
+
+The operational Console previously offered fixed successful and failed canary observations.
+The lifecycle service accepted those counters and health fields as canary results. Reaching the
+target count could satisfy the canary review gate without a measured request. Other promotion
+and approval checks remained; this finding does not imply unrestricted production promotion.
+
+Remove both synthetic controls from the Console. Keep manual observation compatibility only for
+an explicitly persisted LAB rehearsal: a LAB_REHEARSAL subject in LAB, a consistent LAB approval
+with LAB_PROMOTION policy, and a plan targeting LAB_STABLE. Client labels and observation fields
+cannot grant that scope. Refuse other manual observations before changing counters, status or
+audit state. Attribute accepted manual observations to the authenticated operator and request
+correlation, and label their result as manual rehearsal evidence.
+
+Also refuse non-rehearsal canary commits, including existing READY records with no authoritative
+evidence. Do not invent a trusted source flag that a client can submit to bypass the restriction.
+This deliberately leaves real canary qualification unavailable until P05 adds an authoritative
+evidence ingestion contract. Existing non-canary transitions retain their independent checks.
+
+P05 must consume deduplicated real gateway requests for the exact candidate/deployment and time
+window, combine observed runtime/scheduler health without converting unknown to healthy, and use
+transactional concurrency control. Production qualification requires that follow-on implementation
+and isolated real-workload evidence. Manual LAB rehearsal proves only the lifecycle mechanics.
+
+No schema change is needed for containment; existing result metadata can label the evidence.
+Rollback to the old implementation would restore the unverified-observation path and is not a
+safe remedy for blocked production canaries. Existing explicit abort/rollback operations remain
+available for reconciliation.

docs/architecture/adr/0052-managed-personal-keys-and-recovery-admission.md

@@ -0,0 +1,63 @@
+# ADR-0052: Managed personal keys and recovery admission
+
+Status: accepted design for P01; implementation qualification in progress.
+
+## Decision
+
+The bounded personal-header-key provider is the supported RC interactive provider. Preserve
+ADR-0049 deployment keys and the bootstrap administrator unchanged. Add administrator-managed
+identities under `managed:<slug>` with a distinct `mfopm_` plus 43 URL-safe-character credential
+namespace. Existing deployment subjects cannot contain `:`; authentication of deployment keys
+does not require a managed-identity database lookup. A managed key never falls back to deployment
+or machine privileges. The existing header, role boundaries and memory-only Console remain.
+This decision does not claim OIDC, SSO logout, cookie sessions or enterprise IAM.
+
+Store only the SHA-256 digest of a random 256-bit key, never plaintext or recoverable ciphertext.
+No encryption root key or additional dependency is needed. Issue plaintext exactly once after
+commit. A lost response requires a new rotation. Bound expiry to 1–365 days, rotation overlap to
+0–900 seconds, identities to 100, and displayed credential history to the latest ten records.
+History remains authoritative; retired keys cannot be restored by another rotation. Metadata
+exposes role, source, expiry, version, last use, activation and usability, never digests. Mutation
+reasons cannot contain ModelForge credential namespaces and validation errors omit rejected inputs.
+
+Keep deployment identities read-only. Production startup still requires a secure deployment
+administrator/recovery anchor; this PR does not introduce managed-only bootstrap or automatic
+environment migration. First managed identity must be an administrator. A singleton transaction
+lock serializes every managed mutation; expected registry/identity versions reject stale requests.
+Under that lock recheck the initiating managed administrator's credential, role, activation and
+expiry. Once initialized, every successful transaction leaves an enabled administrator with an
+activated, unrevoked, unexpired key. Expiry can still cause future loss of managed administration;
+the deployment recovery anchor remains necessary. Audit and identity mutation commit atomically.
+
+## Recovery
+
+`managed_operator_state`, `managed_operator_identities` and `managed_operator_credentials` are
+AUTHORITATIVE. Migration 0027 introduces the exact initial singleton and empty identity/credential
+tables. Older-schema restore permits only that exact empty introduction; same-schema restoration
+preserves all authoritative metadata. Credential digests stay excluded from public fingerprints
+under the established secret-column policy. Mechanical downgrade refuses existing identity history;
+recover the verified pre-migration backup instead.
+
+`managed_operator_credential_activations` is CURRENT_TRUTH admission state. Normal issue/rotation
+adds an activation only for the new key. Authentication requires it in addition to authoritative
+credential validity. Supported restore clears activations alongside other stale current truth;
+the independent READY validation rejects any retained activation. This quarantines every restored
+managed key, including keys revoked after the backup, without rewriting historical credentials
+or permitting fingerprint drift. Interrupted restore targets remain isolated and must not serve.
+
+After supported recovery, an external deployment administrator rotates an enabled managed
+administrator or creates a new administrator to restore admission. The new key alone gains
+activation; old keys remain quarantined even during requested overlap. Reconciliation never
+rehydrates activations automatically. A raw `pg_restore` bypassing ModelForge reconciliation and
+READY validation is not a supported recovery procedure and must not be attached to the serving API.
+
+## Evidence boundary
+
+Pre-body managed authentication uses the existing bounded thread-pool/session pattern and returns
+a typed 503 on database failure. No cached successful credential can override revocation.
+Already-authorized ordinary requests may finish; managed administration additionally rechecks its
+initiator while holding the mutation lock. Tests must cover real PostgreSQL cross-admin races,
+rotation/revoke CAS, last-admin protection, migration downgrade/recovery, restored admission
+quarantine, source namespace separation, audit rollback, and Console one-time secret handling.
+Local SQLite tests are not PostgreSQL qualification. Exact-head managed and browser gates remain
+required before this ADR's implementation is marked verified.

docs/architecture/adr/0053-huggingface-control-plane-secret-slot.md

@@ -0,0 +1,51 @@
+# ADR-0053: Bounded Hugging Face control-plane secret lifecycle
+
+Status: accepted design; implementation and qualification in progress.
+
+Only `huggingface_control_plane` is API-managed. It supplies the control-plane Hugging Face
+discovery and repository metadata adapter. Node-agent downloads retain their independently
+deployed credential; no operation here acknowledges node credential activation. Database,
+migration, operator/recovery, backup encryption and Redis credentials remain deployment-managed.
+
+Deployment mode remains until the first explicit set. Thereafter no missing, revoked, restored,
+undecryptable or inactive managed state falls back to deployment or ambient HF credentials.
+Set persists a new encrypted candidate and switches to managed mode; an existing active version
+continues serving during rotation. Test calls only the fixed HTTPS Hugging Face whoami-v2 endpoint
+with bounded time/body, no redirects or environment proxies, and redacted results. It proves token
+authentication, not repository entitlement or node downloads. Activation requires a successful
+test within five minutes for the exact candidate version and current revision. Each mutation
+uses CAS and atomic attributable audit. Network testing occurs outside the mutation transaction;
+the result cannot bless a concurrently replaced/revoked candidate. Revocation disables all managed
+versions; recovery requires setting a new candidate, testing and activating it.
+
+Use AES-256-GCM from the existing cryptography dependency, fresh 96-bit nonces and associated data
+binding slot, version and key ID. `MODELFORGE_SECRET_SLOT_ROOT_KEY` is strictly a base64 32-byte key,
+supplied externally, independent of backup encryption and operator credentials. Its explicit ID
+is stored on every version. This bounded first implementation supports one mounted/deployment
+key at a time; replacing that key does not silently rewrap history. Retain old external keys for
+historical restore, or set/test/activate a new version before retiring the prior deployment key.
+The root key cannot be managed through the slot API. No plaintext reads, exports or audit fields.
+
+State and encrypted versions are authoritative. Test results and active admission are current
+truth, cleared by supported restore and independently checked before READY. Restored candidates
+require a fresh test and explicit activation. Old-schema restore permits only the exact initial
+empty state introduced by migration 0028. A history-bearing downgrade requires verified backup
+recovery. Retired ciphertext remains historical; provider revocation cannot be undone locally.
+
+New provider instances resolve one snapshot when upstream access starts. Persisted history and
+node job progress, completion and cleanup do not require upstream credential resolution; a revoked
+slot must not strand an existing job's bookkeeping. Activation acknowledges this
+local resolver, not remote downloads or running requests. Provider failure never becomes a
+successful test. Missing root-key disables the managed consumer without blocking deployment
+administrator access needed for recovery. SQLite qualification is not PostgreSQL race/restore
+qualification; live HF account checks and isolated RC provisioning remain external gates.
+
+Persistence contract: migration 0028 introduces two authoritative tables and two current-truth
+admission tables. The runtime role can insert encrypted versions but can update only their
+`revoked_at` column; it cannot rewrite version numbers, key IDs, nonces, ciphertext or creation
+timestamps, nor delete/truncate historical rows. The recovery owner retains the isolated
+maintenance authority required by supported restore. Metadata fingerprints explicitly exclude
+ciphertext and nonces; encrypted archive authentication and AES-GCM validate secret contents.
+`SECRET_STRICT` retains all secret metadata while delegating the existing verified reconciliation
+marker exception to `MANAGED_STRICT`. Earlier schemas accept only the exact uninitialized seed
+and empty version history introduced by 0028, never arbitrary newly introduced secret state.

docs/architecture/adr/0054-lab-preflight-before-persistent-execution.md

@@ -0,0 +1,42 @@
+# ADR 0054: Model Lab preflight and execution are separate contracts
+
+Status: accepted for P03A; persistent execution remains outstanding.
+
+## Context
+
+The existing serving gateway has private worker jobs, GPU leases and transient payloads.
+Its invoke methods synchronously await completion, use service-client authorization and,
+for several modalities, resolve an experiment rather than an explicitly chosen candidate.
+Wrapping these methods in an operator endpoint would conflate credential domains and would
+not provide a durable, restart-safe Model Lab run.
+
+## Decision
+
+First expose an operator-authorized, read-only `POST /api/v1/lab/preflight`. Requests bind to
+exact candidate, artifact, runtime profile, node and LAB approval IDs. Inputs discriminate
+the five supported capability families and reuse existing typed serving validators. Execution,
+queue, retention and output budgets are bounded; concurrency is one.
+
+Recompute current assessment and approval evidence through the runtime service. Check candidate,
+completed probe, profile, node and contract coherence. Responses contain only typed binding,
+input digest/count/size, bounded limits and blocker codes. No raw content, arbitrary review text,
+service credential, payload-store reference or inferred GPU measurement is returned or persisted.
+Validation errors for this namespace also omit caller-controlled input, locations and messages.
+
+Preflight does not create an experiment, approval, gateway request, job, payload or lease. Both
+`execution_started` and `permission_granted` are always false. Passing is a current read-only
+observation; a future start operation must repeat admission and evidence checks atomically.
+The existing control-plane HTTP body cap applies, so initial inline inputs must fit that cap.
+No service-key privilege or machine credential is converted into an operator principal.
+
+## Subsequent implementation
+
+Persist LabRun identity, actor, immutable provenance and idempotency separately from transient
+input/output content. Extract explicit-target dispatch into the existing scheduler/private-worker
+pipeline rather than create another scheduler or fabricate a ServiceClient. Add leases, bounded
+retry, deadline/retention cleanup and restart reconciliation with PostgreSQL migration/recovery proof.
+
+Running cancellation must distinguish requested cancellation from worker acknowledgement.
+An API or database status alone cannot establish that GPU execution or reclamation has completed.
+Then expose run history/results and connect the Console. Real hardware, consumer journeys and
+the frozen RC artifactset remain separate qualification gates.

docs/architecture/adr/0055-selected-model-update-inbox.md

@@ -0,0 +1,42 @@
+# ADR 0055: Selected-model update inbox
+
+Status: accepted for selected-model tracking; replacement execution remains separate.
+
+The isolated integration introduces schema `20260909_0029` directly after `20260908_0028`.
+Watch and notice history are authoritative recovery state. An older backup may gain these tables
+only in their exact empty initial state; a current backup must retain their complete semantic
+fingerprints. Temporary check tokens and deadlines are excluded and cleared during restore,
+with an independent readiness check refusing remaining restored claims.
+
+ModelForge is a central hub for selected models and project capabilities. Upstream monitoring
+is opt-in, bounded to 100 explicit model/ref watches. A watch records an existing exact revision
+as its comparison baseline. It does not imply that this revision is deployed or that
+a subsequently observed commit is newer or better. Alternative repositories require their own
+explicit selection; this is not a global model-ranking system.
+
+Manual checks reuse the authenticated Hugging Face metadata and immutable revision boundary.
+They create no download plan, worker execution, approval or deployment. Changed commits produce
+one durable notice per watch/commit. Dismissal preserves history and repeated checks do not revive
+dismissed notices. Lists are bounded and paginated. Provider failure records an error while
+preserving the baseline and existing notices.
+
+An atomic database claim bounds concurrent checks. A five-minute lease allows restart recovery;
+token fencing prevents expired requests from overwriting a successor's state. The audit chain
+serializes creation admission so the 100-watch bound also applies to concurrent creators.
+Reads require viewer and mutations require operator. Nested provider audit events use the same
+authenticated actor and correlation as the watch operation.
+
+Production processes check enabled watches daily by default, at most one due watch per minute
+per API process. Atomic due checks avoid a duplicate scheduled call after another process finishes
+the same watch. This is a per-process throughput limit, not a global cross-process scheduler.
+Development/test lifespans do not start this monitor. Errors are recorded or logged without
+credential content; a monitor configuration flag is not proof of a successful provider check.
+
+Tracking changes use an explicit audited compare-and-swap on the observed version and baseline.
+Baseline changes validate the same model and refuse an active check. Pausing fences in-flight
+check publication; already-started metadata retrieval may finish, but cannot produce a notice
+or overwrite successor state. Historical notice provenance is unchanged. Changing tracking never
+changes a deployment. Broad model search, automatic tests, automatic replacement and model-quality
+claims are not introduced. Project-bound comparison and evidence-backed replacement
+continue through existing capability, evaluation and lifecycle boundaries, including embedding
+migration and rollback. This tranche does not complete that replacement workflow.

docs/architecture/adr/0056-selected-model-comparison-intents.md

@@ -0,0 +1,84 @@
+# ADR 0056: Durable selected-model comparison intents
+
+Status: accepted for the persisted operator/consumer protocol. Consumer execution integration and
+release qualification remain open.
+
+Migration note: the original canonical 0030 implementation below is retained as
+design history. [ADR 0057](0057-deployed-lineage-convergence.md) integrates these
+tables through additive 0033 while preserving the separately deployed lineage.
+
+## Context
+
+[ADR 0055](0055-selected-model-update-inbox.md) separates an upstream notice from model quality and
+replacement. [ADR 0054](0054-lab-preflight-before-persistent-execution.md) separates admission from durable execution.
+The selected-project hub needs one current-versus-candidate comparison with attributable evidence,
+without introducing a model tournament, another retrieval engine or another promotion engine.
+
+## Decision
+
+Schema `20260909_0030`, after canonical `20260909_0029`, adds `selected_model_comparisons`.
+An operator creates an intent for one stable project binding, one designated service client,
+one immutable project suite and two distinct exact capability deployments. The current leg must
+be a production `stable` ModelForge deployment; the candidate must be a nonproduction `candidate`
+with an active experiment route for the same `rag.embedding@1` contract. Distinct index references
+prevent silent reuse of the current vector collection. Different embedding spaces are allowed;
+this does not authorize reindex, cutover or replacement.
+
+Creation records immutable revision, artifact-set, runtime-profile, embedding-space, configuration,
+suite, corpus and index identities. Every reviewed case must name the requested corpus digest.
+The retrieval configuration digest must match the immutable suite settings. Claim and result
+publication revalidate the selected identities and route. A suite contains 1–100 reviewed cases.
+One active intent per project binding and a unique binding/idempotency key bound admission.
+Identical retries return the existing intent; changing its payload conflicts.
+
+The operator designates an existing active project-bound `rag.embedding@1` client. Consumer
+endpoints authenticate its service credential and require that exact client and binding on every
+operation. A claim token alone grants no authority. Operator credentials are not converted into
+service credentials. Assigned discovery is bounded and scoped to the designated client. Suite
+case definitions accompany its successful claim; this protocol adds no general consumer suite API.
+
+There is one claim, with a 120-second renewable lease and a fixed one-hour deadline from claim.
+No automatic reclaim or replay occurs. Progress counts are bounded and monotonic. Expired claims
+become `interrupted`; history, assignment reads and subsequent creation reconcile expiration.
+Admission retains its database locks until the decision commits. An explicit new intent is required
+for another attempt, and does not prove an earlier external process stopped.
+
+Pending cancellation becomes `cancelled`. Running cancellation becomes `cancel_requested` until
+the designated consumer acknowledges it. Results cannot publish after cancellation, expiry or
+loss of the claim. Expiry and interruption describe coordination state, not GPU reclamation.
+
+The consumer submits both legs together with ranked per-case results, its implementation digest
+and mappings from suite case IDs to public gateway `request_id` values. Successful cases require
+unique completed gateway requests for the designated client, exact leg deployment, capability and
+unchanged single-case query hash. Server-owned request timestamps must lie within this claim and
+deadline. Failed cases may omit a request, but cannot make the comparison eligible. Request evidence
+proves the matching embedding invocation; retrieval rankings remain explicitly attributed to the
+designated consumer rather than represented as independently observed hub retrieval.
+
+`EvaluationService(commit=False)` creates and completes the existing two evaluation runs and
+comparison inside the locked intent transaction. The server recomputes metrics and eligibility;
+caller aggregate scores are not accepted. Any publication failure rolls back the pair and links.
+An identical result retry returns the existing result, while a changed retry conflicts. The
+response derives a compact outcome from the linked comparison and fails closed on missing evidence.
+Eligibility is evaluation evidence, never production authorization or a promotion operation.
+
+## Recovery and boundaries
+
+Following [ADR 0041](0041-authoritative-versus-rebuildable-recovery-state.md), pins, operation history
+and linked results are authoritative. Claim hashes and expiry are transient. Restore clears these
+claims and transforms `pending`, `running` and `cancel_requested` intents to `interrupted` with
+`COMPARISON_RESTORE_INTERRUPTED`; it does not replay work. A deterministic fingerprint projection
+permits precisely that transformation while retaining the other provenance. An older schema can
+gain the new table only in its exact empty initial state. The readiness gate refuses residual
+restored claims or active intents. Populated comparison history blocks downgrade.
+
+The API does not launch a background consumer, run the ExampleRAG CLI, invoke a provider, claim GPU
+capacity or promote a model. The current ExampleRAG CLI dynamically selects its legacy CURRENT profile
+and does not yet implement this claim/result protocol. Its historical evaluation cannot satisfy a
+fresh intent. A consumer adapter must first validate the actual stable/candidate collection and
+profile identities, execute through the existing retrieval engine, and retain matching public
+gateway request IDs. Real inference, managed PostgreSQL concurrency, upgrade/restore and the frozen
+release artifact set remain qualification gates; this ADR does not assert those gates passed.
+
+The actionable protocol is documented in
+[Selected model comparisons](../../operations/SELECTED_MODEL_COMPARISONS.md).

docs/architecture/adr/0057-deployed-lineage-convergence.md

@@ -0,0 +1,64 @@
+# ADR 0057: Preserve the deployed migration lineage when integrating the model hub
+
+Status: accepted for implementation; deployment qualification pending.
+
+## Context
+
+The canonical hub and a separately deployed creative branch diverged after schema
+`20260908_0028`. Read-only inspection on 2026-09-09 found image
+`modelforge-api:dfec816` and production schema `20260909_0032`. In that deployed
+lineage, 0029 creates creative allocations, 0030 creates model watches, 0031 adds
+creative governance, and 0032 widens audio workload names. Canonical hub 0029 and
+0030 instead represented newer watches and selected comparisons. The shared
+revision strings do not establish compatible database state.
+
+## Decision
+
+Preserve the deployed 0029–0032 migration artifacts byte-for-byte. Add 0033 after
+0032 to introduce watch `enabled=true` and `version=1`, runtime history protections,
+and selected-comparison storage. Do not stamp, rename applied migrations, or
+downgrade populated production data to resolve the conflict. New installations
+and upgrades from the common 0028-or-earlier lineage follow this same linear graph.
+
+The earlier canonical comparison schema was development qualification, not an
+alternative supported production upgrade source. An isolated database built with
+that conflicting graph must retain its original migration bundle; it cannot be
+upgraded using a revision number alone. Convergence does not authorize discarding
+its data. Any required migration of that separate lineage needs its own explicit,
+verified transfer procedure.
+
+Retain canonical project-first navigation, model-watch pause/CAS and bounded
+comparison execution alongside deployed creative history. ModelForge remains the
+central capability hub. This integration adds no model tournament or external
+creative tool execution.
+
+Native inference and external grants share accelerator admission locks and account
+for all granted or uncertain reservations. Capacity reads inside admission do not
+commit the owning transaction. Unattributed GPU telemetry is not proof that a
+particular external grant materialized; retain its full reservation until explicit
+release. Queued work must revalidate current client, project, priority and approved
+resource before receiving a grant. Expiry cannot be reversed by a heartbeat.
+
+The governance tables belong to the typed persistence model, retaining the exact
+deployed constraints. Downgrading 0033 refuses populated hub or creative history,
+preventing traversal into older destructive downgrade paths.
+
+## Recovery and acceptance
+
+Recovery preserves creative provenance and explicitly marks restored queued/granted
+allocations uncertain. That state does not claim an external process stopped.
+Selected comparisons remain interrupted with their claims cleared. Deterministic
+projections computed from the source rows must prove these exact changes; missing,
+changed or unclassified history blocks readiness.
+
+A deployed watch schema without the new control columns requires a fresh source
+fingerprint with the explicit default-column projection. An old opaque fingerprint
+cannot be rewritten to remove execution-claim fields or invent the new controls.
+Unprojectable archived history remains a recovery blocker requiring its original
+qualified artifact; no hash exception is introduced.
+
+Qualify populated 0032→0033 upgrade and actual dump/restore, audit integrity,
+runtime privileges, competing admission transactions, revoked queued authority,
+expiry and UI stale-response handling. All four immutable images, the actual
+deployment upgrade/rollback plan and the release soak remain separate gates.
+Merging this source does not deploy it or establish RC readiness.

docs/architecture/adr/0058-approved-serving-transitions.md

@@ -0,0 +1,81 @@
+# ADR 0058: Execute stable route replacement through the approved lifecycle engine
+
+Status: implemented for identical embedding spaces; isolated PostgreSQL qualification passed; production deployment pending.
+
+The project comparison establishes evidence. It grants no production authority. Existing
+lifecycle subjects, plans and rollback snapshots previously recorded state without switching
+the gateway. The new transaction-only serving adapter connects those existing operations to
+the actual capability deployments; there is no second promotion engine.
+
+An approval's typed `evidence.serving_transition` selects current and candidate deployments,
+the exact production execution approval, expected capability `serving_generation`, and one
+selected-comparison ID for every affected stable project binding. The server derives the
+complete binding set and identity snapshots. Missing projects, reused comparisons, changed
+bindings, stale comparison pins, incomplete/errorful runs and revoked production evidence
+block execution. Creating a new comparison supersedes the older binding evidence for this
+purpose. The existing production validation engine rechecks the probe and supply chain.
+Unlike a capability-only readiness record, an operational transition requires the approval's
+declared `production_validation=SATISFIED`; explicitly deferred, required or not-required
+validation cannot activate a project-serving route. This requirement is also carried in the
+immutable serving contract and rechecked at commit.
+
+For each affected binding, the latest ProjectFitEvidence for that exact candidate is included
+in the approved contract when present. Negative/deferred conclusions, incomplete integration,
+blockers or critical errors stop the operation. A later record changes the contract and needs
+fresh review. Evidence for another deployment is not silently applied to this candidate.
+Absence of a separate project-fit record does not fabricate one: reviewed per-binding selected
+comparisons and explicit satisfied production validation remain required. This adapter does
+not impose ExampleVision's OWNER_PHOTO evidence class on RAG projects, or create separate
+PROJECT_PRODUCTION approvals. Generic capability-only/LAB policy interpretation remains intact.
+
+The reviewed contract is stored in the approval's resolved evidence and copied into the
+immutable promotion plan. Production scope, exact candidate/rollback targets and an explicit
+plan approval are required. This first operational adapter supports only `transparent` or
+`behavioral` plans targeting STABLE from a LAB_READY/PROMOTION_ELIGIBLE deployment subject.
+Manual canary observations do not acquire operational authority.
+
+Execution captures actual deployment/routing identities in the existing immutable rollback
+snapshot. The separate explicit lifecycle commit locks the capability contract, revalidates
+all evidence, compares/increments its durable route generation, drains the old route and
+activates the next route in one transaction with lifecycle state and audit. The capability
+endpoint and project credentials do not change. Existing admitted calls retain their old
+deployment; changing the route neither kills workers nor claims memory was reclaimed.
+
+Rollback checks the exact installed route generation, both deployment snapshots and the
+affected binding set. It restores the actual route and subject in one transaction and
+increments generation again. A later route/configuration change produces a conflict; the
+old operation never overwrites it. The previous artifact set, runtime profile and deployment
+receive explicit retention records. Nonzero route generations block schema downgrade.
+Rollback rechecks the retained target's actual production approval, supply-chain/probe facts,
+node availability and runtime readiness. A later revocation or quarantine is an explicit
+blocker, not permission to reactivate unsafe assets. Same-operation commit retries verify the
+exact installed state and return the persisted result without another generation increment.
+Activation timestamps come from the actual commit journal, not the earlier preparation time.
+
+Embedding identity equality means the exact existing embedding-space ID, not matching vector
+dimensions or apparently identical model files. A changed identity still requires a trusted
+consumer reindex/cutover adapter and is blocked by this packet. The old bootstrap promotion
+and newer experiment identity serializers differ; those existing identities are not rewritten
+or silently treated as equivalent. The same-space path therefore does not make arbitrary
+new-model replacement available and does not imply that consumer cutover is complete.
+
+The contract generation starts at zero for existing schemas and is authoritative thereafter.
+All stable-route writers, including initial promotion and explicit drain, share the same
+contract lock and generation increment. Recovery must preserve this value and exact rollback
+provenance; an opaque legacy populated fingerprint is not a substitute for a qualified
+default-zero source projection.
+Operators obtain `expected_route_generation` from `route_generation` on the existing deployment
+response. They must refresh and review after a conflict rather than guessing a generation.
+
+Validation includes real offline lifecycle commit/rollback, stale authority/binding/identity
+negative tests, transaction rollback on subject conflict, and an optional disposable PostgreSQL
+two-committer race with runtime-role writes. No inference, external index operation or production
+deployment is performed by this implementation qualification.
+
+The project comparison view offers a read-only replacement preview. It shows every affected
+stable binding, current/candidate identities, route generation, valid production approval
+choices and actionable blockers. Stale historical production approvals are excluded without
+invalidating another valid choice. The preview neither declares production validation satisfied
+nor creates lifecycle authority. Historical lifecycle approvals remain readable after a route
+change; expected qualification failures mark them STALE with an audited reason, and previously
+approved stale authority remains explicitly revocable.

docs/architecture/adr/0059-isolated-rc-embedding-cutover.md

@@ -0,0 +1,66 @@
+# ADR 0059: Isolated RC embedding comparison and consumer cutover
+
+Status: proposed; implementation and runtime qualification pending.
+
+## Context
+
+The RC product contract requires a disposable `validation-canary` node with
+`production_eligible=false`. Stable serving promotion and the approved lifecycle
+transition require a production-eligible node and production approval. Relaxing
+those checks for RC testing would grant a LAB deployment production authority.
+
+The selected-comparison workflow currently compares a stable deployment with
+one LAB experiment route. An isolated RC stack needs two LAB routes, one for its
+current embedding identity and one for the candidate. Different embedding-space
+IDs require separately pinned Qdrant collections even when vector dimensions
+match. The experiment routes are capability-wide; changing their targets would
+also affect any other client using the same key.
+
+## Decision
+
+- Keep stable deployment selection and production lifecycle rules unchanged.
+- In the isolated RC stack, create two distinct LAB_READY deployments with
+  approved LAB execution and opaque, immutable experiment routes. Give the
+  designated ExampleRAG project one scoped service client credential.
+- Permit a selected comparison with an explicit current experiment route ID.
+  Pin and revalidate both route-to-deployment mappings, full embedding
+  identities, reviewed suite, corpus, retrieval settings, consumer and distinct
+  index references. A stale route, credential, approval or identity fails closed.
+- ExampleRAG builds the candidate collection separately and executes both legs
+  through the pinned ModelForge routes. The current collection is never
+  overwritten during comparison. Gateway request IDs and retrieved chunk IDs
+  are retained as evidence.
+- For a different embedding space, the RC cutover authority is a single durable
+  ExampleRAG selection record that names **both** the active embedding profile
+  (including its experiment route key and embedding-space ID) and the Qdrant
+  collection. Ingest and query resolve that same record. Activation and rollback
+  use its monotonically increasing generation and compare-and-swap. They do not
+  retarget a ModelForge experiment route or its production stable route.
+- Activation requires the completed, reviewed ModelForge comparison and an
+  explicit operator decision covering impact and reindex. The candidate
+  collection must have a complete authoritative reindex, correct identity pins,
+  and a successful consumer smoke check before the selection changes. A retry
+  with the same operation ID returns the recorded result; a stale generation or
+  different payload conflicts. Rollback must verify the retained previous
+  collection/profile and current generation before restoring the prior
+  selection. Calls already admitted may finish against the old selection;
+  subsequent calls use the new generation.
+- A failed or interrupted operation leaves the previous selection usable or
+  enters an explicit blocked state with a recovery journal. It never silently
+  falls back to Ollama or an unpinned collection.
+
+## Qualification boundary
+
+This is an RC consumer transition on disposable state. It does not claim
+production promotion, a stable route change or a production rollout. The final
+candidate must prove the full comparison, activation, real request, rollback,
+second real request, duplicate/retry behavior, stale CAS, concurrent operators,
+and process-restart recovery on the same exact images and source revision.
+
+## Consequences
+
+ModelForge remains the source of model identity, LAB execution, gateway,
+scheduler and comparison evidence. ExampleRAG owns the atomic choice of its
+embedding route and matching index. A global experiment-route remap is not used
+as a shortcut for consumer cutover. The RC cannot be marked qualified until the
+selection record and its runtime tests are implemented and verified.