# Controlled legacy deployment migration `plan.py` is a **pure planner**. `execute.py prepare` is an explicit opt-in, build-only preparation command. Separate `cutover` and `recover` modes require `--apply` and the exact verified preparation identity. Tests use fixtures/offline Compose rendering. The first verified production migration is recorded below; test results alone must never be presented as proof of a production cutover. Authoritative broker code: `Unraid-AppOps-Gateway/ops/gitea-deploy-broker/deploy.py`. Production installs it separately at `/mnt/pool/appdata/gitea-deploy-runners/controller/deploy.py`. Production projects configuration is intentionally outside Git. Compare normalized source before changes. ## Why standard retry/bootstrap cannot migrate these two VacatureRadar and LumaOps have healthy legacy containers but schema3 failed state with no known live commit or artifacts. `capture_compose_rollback` rightly refuses unattested rollback. `bootstrap-legacy` requires exact historical successful legacy state and refuses schema3. Never invent a revision or a successful receipt. Standard preflight overlays live appdata, and standard migration cleanup removes retained legacy containers. Neither is suitable for a rollback-retaining migration with unknown original provenance. Use a reviewed separate one-shot transaction. ## Required transaction design 1. Pin/validate exact Gitea commit and tree. Prepare a separate clean build context; external credentials remain outside it. Build before disrupting production. Require immutable OCI source/revision/tree/version/time labels, not container labels alone. Existing configurations do not enable these checks yet. 2. Acquire the same project then global locks as the broker. Revalidate original container and immutable image IDs, healthy state, exact mount inventory, free disk space, candidate network/ports, and absence of attempt targets. Protect the original image with an immutable rollback reference. Persist a root-only journal and original configuration snapshot without exposing secrets in output. 3. Stop the original, cold-copy and verify all writable data into new attempt-owned paths. VacatureRadar runs PostgreSQL17 and Django migrations on startup: restarting the old image on migrated data is NOT rollback. Its nested `local`, logs and media bind mounts share one cold copy. LumaOps has bind mounts plus an anonymous `/config` volume; include both with their nested destination semantics preserved. 4. Retain/rename the exact original container. Start only the candidate under a distinct Compose project (`-managed`), preserving production name/ports. The candidate mounts clones exclusively. A distinct project prevents Compose from adopting/removing retained original containers or failing exact-container-set checks. 5. Verify immutable built image IDs, provenance and Docker health plus representative HTTP behavior. Only then obtain the real broker receipt and commit broker config (exact source, external env, explicit cloned data paths, new project identity, OCI verification) and state. Keep the originals and cold-copy journal available. 6. On any failure before commit, remove only recorded attempt-created container IDs after identity checks, restore broker configuration, and rename/restart the exact original container with original untouched data. Verify old health. A failed restore is reported as unknown/failed, never as healthy. Keep partial artifacts for recovery. Process-crash recovery needs a durable phase journal and idempotent identity checks; exception handlers alone are insufficient. Source-tree, data, broker configuration, runtime and receipt transitions must be separately recorded. Do not automatically clean originals after success. Retention cleanup needs separate authorization. `build_plan` accepts a strict sanitized runtime projection: `name`, `containerId`, `imageId`, `running`, `health`, `composeProject`, and `mounts` (type/source/destination/rw). Never feed or print full Docker inspect/environment output. This planner validates paths lexically; the executor also resolves roots, rejects symlinks/special files, verifies backup checksums/ownership and revalidates state while locks are held. Run: `python -m unittest discover -s scripts/legacy-deployment-migration -p 'test_*.py'`. ## Build-only preparation After the two tracked `docker-compose.migration.yml` files are reviewed and merged, `execute.py prepare --container-id --image-id --apply` prepares on the controller. Without `--apply` it performs no work. It validates the reviewed broker artifact, current branch head and original healthy container identity, then holds the normal project/global locks while extracting an isolated exact source tree and building an candidate image through the exact merged Compose configuration. OCI labels are checked on its immutable image ID. Source, original runtime and mounts are rechecked after build. Runtime environment is captured privately for Compose interpolation; credentials remain outside the build context and are not supplied as build arguments. Root-only journal/logs retain interrupted/failed attempts; a repeated attempt refuses to overwrite them. No production configuration/state/data is written. The build-only journal is NOT a live receipt. `cutover` starts that same image using Compose `up --no-build --pull never`, then normal broker verification binds the observed runtime to its prepared immutable image ID. No rebuild occurs during downtime. Only a genuine normal broker receipt can be committed. Tracked Compose override field contract (all required, no live-data defaults): - Shared: `MIGRATION_ATTEMPT`, `MIGRATION_IMAGE`, `MIGRATION_SOURCE_REVISION`, `MIGRATION_SOURCE_TREE`, `MIGRATION_BUILD_ID`, `MIGRATION_BUILD_DATE`. - VacatureRadar: `MIGRATION_LOCAL_ROOT`, `VACATURERADAR_ENV_FILE`. - LumaOps: `MIGRATION_CONFIG_ROOT` (anonymous `/config` clone), `MIGRATION_OPENRGB_ROOT`, `MIGRATION_LUMA_CONFIG_ROOT`, `MIGRATION_DATA_ROOT`, `MIGRATION_LOGS_ROOT`, `MIGRATION_ENV_FILE`. Pass the protected external runtime env file as Compose `--env-file` as well, preserving base-file interpolation values. Before cutover validate rendered absolute mounts against the plan, reject originals, and verify `create_host_path:false` so a missing copy cannot silently become an empty data directory. The normal broker requires Compose files to exist in the committed tree, which is why these overrides are tracked app files rather than a generated external override. ## Cutover and recovery Use the same slug, revision, attempt and expected original IDs as preparation: `execute.py cutover ... --apply`. Selected project configuration/state hashes, source, image, runtime env, original IDs/health/mounts and rendered clone mounts/ports are revalidated before stopping. Both normal deployment locks remain held throughout. The durable journal records intent before `stopping`, `copying`, `renaming`, `deploying`, `verifying`, and `committing`. Cold copies use `cp -a --reflink=auto`, filesystem sync and SHA256/ownership/mode/xattr/hardlink verification. The original is retained under a distinct name and immutable image tag. Broker config/state and original container configuration are saved root-only. The candidate never mounts original data. The new broker project uses dynamic revision/tree/build placeholders so subsequent normal deployments do not reuse frozen provenance labels. An ordinary failure attempts verified recovery automatically. After abrupt process termination use `execute.py recover ... --apply`: it accepts only the saved original or candidate config/state, checks rollback snapshot integrity, and removes only the exact owned candidate (nonce, Compose project, recorded ID where available). It restores original config/state, canonical name and original untouched data, then verifies original immutable image/mounts, Docker health and HTTP readiness. Candidate data, logs and all originals are retained. Unrelated containers/config/state or changed original data block automatic recovery rather than be overwritten/deleted. Recovery after the final `committed` journal marker reverifies production and resumes activation; it does not roll back potentially newer application writes. Recovery before that marker restores the original, including a crash after receipt/config write but before final journal commit. Failed recovery remains explicitly `recovery_failed`, never healthy. Independent review and actual prepared-image/maintenance-window checks are required before first invocation. Health paths were probed read-only and returned HTTP 200 on 2026-09-09: VacatureRadar port1226 `/health/ready/`, LumaOps port1223 `/health/ready`. These endpoints supplement, not replace, immutable image and normal receipt checks. VacatureRadar's reviewed source must contain both the supervised Celery startup hold and first-position HTTP maintenance middleware. The helper writes an attempt-bound `.migration-worker-hold` only into the verified cold clone before candidate start. Ordinary traffic receives 503; exact GET/HEAD readiness/liveness probes remain available. After durable commit, activation journals intent, removes only the matching marker, fsyncs its directory, and records completion. Activation failure stays committed: explicit recovery resumes release and never rolls back after workers can run. A missing marker without durable activation intent fails closed. Both tracked overrides require `MIGRATION_NETWORK`, bound to the exact inspected original `_default` network. Different/additional original networks fail closed; the rendered candidate must retain that network. Network/device compatibility and app-specific migration behavior still require operator review; the helper is not a generalized transaction system for external services. LumaOps has no application maintenance gate in this migration. Its candidate can serve requests and interact with RGB hardware before the receipt is committed. Cold-data rollback restores the original application/container/data, but cannot undo external hardware effects or preserve candidate writes from that interval. Perform its cutover in the approved maintenance window with no concurrent user changes; do not describe this as a zero-effect or zero-data-loss transaction. ## Handoff safety boundary (2026-09-09) The guarded broker artifact expected by `execute.py` has normalized SHA256 `b5a0467f53acba5c783781a22bf81cf99f8b96ae99535460ad6236fac038330d`. This reviewed broker revision was installed atomically on 2026-09-09 after AppOps PR30 passed both validations and merged as `219184267972c596e7b61bbb29f0fab1b6472c9b`. The previous controller is retained as `deploy.py.before-single-runtime-rollback-20260909`. The helper refuses to run until the host artifact matches. Candidate devices, capabilities, read-only root, privilege, security options and tmpfs must match the inspected original exactly. It rejects stale selected-project configuration after acquiring the project lock. Preparation also records selected-project configuration and original state hashes; cutover compares both before making any production change. Simulated failure tests cover interruption after original stop, after rename and after receipt/config write before journal commit. Negative adapter tests cover unrelated candidate nonce/project/IDs and binding broker verification to the prepared immutable image. Offline Compose tests verify effective clone mounts and missing-input failures. These tests do not claim a live migration or an end-to-end Docker rollback. The protected-main approval/release gate for the app overrides and other prerequisite branches must be satisfied before selecting a preparation SHA. Never run preparation against an unmerged workaround or claim these local tests prove a live migration. ## Verified production outcome, 2026-09-09 VacatureRadar was prepared and committed from merged revision `38e87678e1c5b6aa6de2cb49e49366fce8a3cbf2`, image `sha256:c410c5ec26deb0cbd6a2992d4741e6c356f537d087de7fade74f214a747cca5a`. The protected journal is under the controller's `legacy-migrations/vacatureradar/legacy-20260909-a/` directory and records `committed`, `activationIntent:true`, `activated:true`. The original container `VacatureRadar-legacy-legacy-20260909-a` remains stopped with its original data. Docker health, HTTP readiness, the login page and running Celery processes were checked after activation. Normal Gitea deployment run9748 succeeded against the same verified revision. No post-commit rollback was attempted. The initial helper's Docker JSON template incorrectly grouped `index` arguments. Preparation rejected this before production changes. Revision `4d1fc0d` fixed it, with a regression test and successful read-only qualification against real Docker. The actual cutover used that reviewed helper revision. Three additional Linux storage tests used isolated temporary fixtures, not application data. LumaOps has **not** been migrated. Its original runtime has `AUTH_ENABLED=false` and no admin token; the merged target requires authentication on its production network binding. Preparation stopped before building or changing production. Provisioning secure login needs an explicit operator decision; never disable the target's authentication guard or describe the old runtime as a verified new build. Failed preparation artifacts are retained; no LumaOps cutover journal exists.