43 lines
2.6 KiB
Markdown
43 lines
2.6 KiB
Markdown
# ADR 0054: Model Lab preflight and execution are separate contracts
|
|
|
|
Status: accepted for P03A; persistent execution remains outstanding.
|
|
|
|
## Context
|
|
|
|
The existing serving gateway has private worker jobs, GPU leases and transient payloads.
|
|
Its invoke methods synchronously await completion, use service-client authorization and,
|
|
for several modalities, resolve an experiment rather than an explicitly chosen candidate.
|
|
Wrapping these methods in an operator endpoint would conflate credential domains and would
|
|
not provide a durable, restart-safe Model Lab run.
|
|
|
|
## Decision
|
|
|
|
First expose an operator-authorized, read-only `POST /api/v1/lab/preflight`. Requests bind to
|
|
exact candidate, artifact, runtime profile, node and LAB approval IDs. Inputs discriminate
|
|
the five supported capability families and reuse existing typed serving validators. Execution,
|
|
queue, retention and output budgets are bounded; concurrency is one.
|
|
|
|
Recompute current assessment and approval evidence through the runtime service. Check candidate,
|
|
completed probe, profile, node and contract coherence. Responses contain only typed binding,
|
|
input digest/count/size, bounded limits and blocker codes. No raw content, arbitrary review text,
|
|
service credential, payload-store reference or inferred GPU measurement is returned or persisted.
|
|
Validation errors for this namespace also omit caller-controlled input, locations and messages.
|
|
|
|
Preflight does not create an experiment, approval, gateway request, job, payload or lease. Both
|
|
`execution_started` and `permission_granted` are always false. Passing is a current read-only
|
|
observation; a future start operation must repeat admission and evidence checks atomically.
|
|
The existing control-plane HTTP body cap applies, so initial inline inputs must fit that cap.
|
|
No service-key privilege or machine credential is converted into an operator principal.
|
|
|
|
## Subsequent implementation
|
|
|
|
Persist LabRun identity, actor, immutable provenance and idempotency separately from transient
|
|
input/output content. Extract explicit-target dispatch into the existing scheduler/private-worker
|
|
pipeline rather than create another scheduler or fabricate a ServiceClient. Add leases, bounded
|
|
retry, deadline/retention cleanup and restart reconciliation with PostgreSQL migration/recovery proof.
|
|
|
|
Running cancellation must distinguish requested cancellation from worker acknowledgement.
|
|
An API or database status alone cannot establish that GPU execution or reclamation has completed.
|
|
Then expose run history/results and connect the Console. Real hardware, consumer journeys and
|
|
the frozen RC artifactset remain separate qualification gates.
|