Initial curated projectbrain source from 8a7b1e0651dbdab97d744cbd2e653ab3970ac6c5

This commit is contained in:
Public Source Publisher committed 2026-10-01 01:43:26 +02:00
commit 5aa3e8d685
700 files changed
+132801

No files matched your search

@@ -0,0 +1 @@
__pycache__/
@@ -0,0 +1,206 @@
# Controlled legacy deployment migration
`plan.py` is a **pure planner**. `execute.py prepare` is an explicit opt-in,
build-only preparation command. Separate `cutover` and `recover` modes require
`--apply` and the exact verified preparation identity. Tests use fixtures/offline
Compose rendering. The first verified production migration is recorded below;
test results alone must never be presented as proof of a production cutover.
Authoritative broker code: `Unraid-AppOps-Gateway/ops/gitea-deploy-broker/deploy.py`.
Production installs it separately at
`/mnt/pool/appdata/gitea-deploy-runners/controller/deploy.py`. Production projects
configuration is intentionally outside Git. Compare normalized source before changes.
## Why standard retry/bootstrap cannot migrate these two
VacatureRadar and LumaOps have healthy legacy containers but schema3 failed state
with no known live commit or artifacts. `capture_compose_rollback` rightly refuses
unattested rollback. `bootstrap-legacy` requires exact historical successful legacy
state and refuses schema3. Never invent a revision or a successful receipt.
Standard preflight overlays live appdata, and standard migration cleanup removes
retained legacy containers. Neither is suitable for a rollback-retaining migration
with unknown original provenance. Use a reviewed separate one-shot transaction.
## Required transaction design
1. Pin/validate exact Gitea commit and tree. Prepare a separate clean build context;
external credentials remain outside it. Build before disrupting production.
Require immutable OCI source/revision/tree/version/time labels, not container
labels alone. Existing configurations do not enable these checks yet.
2. Acquire the same project then global locks as the broker. Revalidate original
container and immutable image IDs, healthy state, exact mount inventory, free
disk space, candidate network/ports, and absence of attempt targets. Protect the
original image with an immutable rollback reference. Persist a root-only journal
and original configuration snapshot without exposing secrets in output.
3. Stop the original, cold-copy and verify all writable data into new attempt-owned
paths. VacatureRadar runs PostgreSQL17 and Django migrations on startup: restarting
the old image on migrated data is NOT rollback. Its nested `local`, logs and media
bind mounts share one cold copy. LumaOps has bind mounts plus an anonymous `/config`
volume; include both with their nested destination semantics preserved.
4. Retain/rename the exact original container. Start only the candidate under a
distinct Compose project (`<slug>-managed`), preserving production name/ports.
The candidate mounts clones exclusively. A distinct project prevents Compose from
adopting/removing retained original containers or failing exact-container-set checks.
5. Verify immutable built image IDs, provenance and Docker health plus representative
HTTP behavior. Only then obtain the real broker receipt and commit broker config
(exact source, external env, explicit cloned data paths, new project identity,
OCI verification) and state. Keep the originals and cold-copy journal available.
6. On any failure before commit, remove only recorded attempt-created container IDs
after identity checks, restore broker configuration, and rename/restart the exact
original container with original untouched data. Verify old health. A failed restore
is reported as unknown/failed, never as healthy. Keep partial artifacts for recovery.
Process-crash recovery needs a durable phase journal and idempotent identity checks;
exception handlers alone are insufficient. Source-tree, data, broker configuration,
runtime and receipt transitions must be separately recorded. Do not automatically
clean originals after success. Retention cleanup needs separate authorization.
`build_plan` accepts a strict sanitized runtime projection: `name`, `containerId`,
`imageId`, `running`, `health`, `composeProject`, and `mounts` (type/source/destination/rw).
Never feed or print full Docker inspect/environment output. This planner validates
paths lexically; the executor also resolves roots, rejects symlinks/special files,
verifies backup checksums/ownership and revalidates state while locks are held.
Run: `python -m unittest discover -s scripts/legacy-deployment-migration -p 'test_*.py'`.
## Build-only preparation
After the two tracked `docker-compose.migration.yml` files are reviewed and merged,
`execute.py prepare <slug> <full-current-branch-sha> <unique-attempt> --container-id
<expected-id> --image-id <expected-immutable-id> --apply` prepares on the controller.
Without `--apply` it performs no work. It validates the reviewed broker artifact,
current branch head and original healthy container identity, then holds the normal
project/global locks while extracting an isolated exact source tree and building an
candidate image through the exact merged Compose configuration. OCI labels are
checked on its immutable image ID. Source, original runtime and mounts are rechecked
after build. Runtime environment is captured privately for Compose interpolation;
credentials remain outside the build context and are not supplied as build arguments.
Root-only journal/logs retain interrupted/failed attempts; a repeated
attempt refuses to overwrite them. No production configuration/state/data is written.
The build-only journal is NOT a live receipt. `cutover` starts that same image using
Compose `up --no-build --pull never`, then normal broker verification binds the
observed runtime to its prepared immutable image ID. No rebuild occurs during downtime.
Only a genuine normal broker receipt can be committed.
Tracked Compose override field contract (all required, no live-data defaults):
- Shared: `MIGRATION_ATTEMPT`, `MIGRATION_IMAGE`, `MIGRATION_SOURCE_REVISION`, `MIGRATION_SOURCE_TREE`,
`MIGRATION_BUILD_ID`, `MIGRATION_BUILD_DATE`.
- VacatureRadar: `MIGRATION_LOCAL_ROOT`, `VACATURERADAR_ENV_FILE`.
- LumaOps: `MIGRATION_CONFIG_ROOT` (anonymous `/config` clone),
`MIGRATION_OPENRGB_ROOT`, `MIGRATION_LUMA_CONFIG_ROOT`, `MIGRATION_DATA_ROOT`,
`MIGRATION_LOGS_ROOT`, `MIGRATION_ENV_FILE`.
Pass the protected external runtime env file as Compose `--env-file` as well,
preserving base-file interpolation values. Before cutover validate rendered absolute
mounts against the plan, reject originals, and verify `create_host_path:false` so a
missing copy cannot silently become an empty data directory. The normal broker
requires Compose files to exist in the committed tree, which is why these overrides
are tracked app files rather than a generated external override.
## Cutover and recovery
Use the same slug, revision, attempt and expected original IDs as preparation:
`execute.py cutover ... --apply`. Selected project configuration/state hashes,
source, image, runtime env, original IDs/health/mounts and rendered clone mounts/ports
are revalidated before stopping. Both normal deployment locks remain held throughout.
The durable journal records intent before `stopping`, `copying`, `renaming`,
`deploying`, `verifying`, and `committing`. Cold copies use `cp -a --reflink=auto`,
filesystem sync and SHA256/ownership/mode/xattr/hardlink verification. The original
is retained under a distinct name and immutable image tag. Broker config/state and
original container configuration are saved root-only. The candidate never mounts
original data. The new broker project uses dynamic revision/tree/build placeholders
so subsequent normal deployments do not reuse frozen provenance labels.
An ordinary failure attempts verified recovery automatically. After abrupt process
termination use `execute.py recover ... --apply`: it accepts only the saved original
or candidate config/state, checks rollback snapshot integrity, and removes only the
exact owned candidate (nonce, Compose project, recorded ID where available). It
restores original config/state, canonical name and original untouched data, then
verifies original immutable image/mounts, Docker health and HTTP readiness. Candidate
data, logs and all originals are retained. Unrelated containers/config/state or
changed original data block automatic recovery rather than be overwritten/deleted.
Recovery after the final `committed` journal marker reverifies production and resumes activation; it
does not roll back potentially newer application writes. Recovery before that marker
restores the original, including a crash after receipt/config write but before final
journal commit. Failed recovery remains explicitly `recovery_failed`, never healthy.
Independent review and actual prepared-image/maintenance-window checks are required
before first invocation. Health paths were probed read-only and returned HTTP 200 on
2026-09-09: VacatureRadar port1226 `/health/ready/`, LumaOps port1223 `/health/ready`.
These endpoints supplement, not replace, immutable image and normal receipt checks.
VacatureRadar's reviewed source must contain both the supervised Celery startup hold
and first-position HTTP maintenance middleware. The helper writes an attempt-bound
`.migration-worker-hold` only into the verified cold clone before candidate start.
Ordinary traffic receives 503; exact GET/HEAD readiness/liveness probes remain available.
After durable commit, activation journals intent, removes only the matching marker,
fsyncs its directory, and records completion. Activation failure stays committed:
explicit recovery resumes release and never rolls back after workers can run.
A missing marker without durable activation intent fails closed.
Both tracked overrides require `MIGRATION_NETWORK`, bound to the exact inspected
original `<slug>_default` network. Different/additional original networks fail closed;
the rendered candidate must retain that network. Network/device
compatibility and app-specific migration behavior still require operator review;
the helper is not a generalized transaction system for external services.
LumaOps has no application maintenance gate in this migration. Its candidate can
serve requests and interact with RGB hardware before the receipt is committed.
Cold-data rollback restores the original application/container/data, but cannot
undo external hardware effects or preserve candidate writes from that interval.
Perform its cutover in the approved maintenance window with no concurrent user
changes; do not describe this as a zero-effect or zero-data-loss transaction.
## Handoff safety boundary (2026-09-09)
The guarded broker artifact expected by `execute.py` has normalized SHA256
`b5a0467f53acba5c783781a22bf81cf99f8b96ae99535460ad6236fac038330d`.
This reviewed broker revision was installed atomically on 2026-09-09 after
AppOps PR30 passed both validations and merged as `219184267972c596e7b61bbb29f0fab1b6472c9b`.
The previous controller is retained as `deploy.py.before-single-runtime-rollback-20260909`.
The helper refuses to run until the host artifact matches. Candidate devices, capabilities, read-only root,
privilege, security options and tmpfs must match the inspected original exactly.
It rejects stale selected-project configuration after acquiring the project lock.
Preparation also records selected-project configuration and original state hashes;
cutover compares both before making any production change.
Simulated failure tests cover interruption after original stop, after rename and
after receipt/config write before journal commit. Negative adapter tests cover
unrelated candidate nonce/project/IDs and binding broker verification to the prepared
immutable image. Offline Compose tests verify effective clone mounts and missing-input
failures. These tests do not claim a live migration or an end-to-end Docker rollback.
The protected-main approval/release gate for the app overrides and other prerequisite
branches must be satisfied before selecting a preparation SHA. Never run preparation
against an unmerged workaround or claim these local tests prove a live migration.
## Verified production outcome, 2026-09-09
VacatureRadar was prepared and committed from merged revision
`38e87678e1c5b6aa6de2cb49e49366fce8a3cbf2`, image
`sha256:c410c5ec26deb0cbd6a2992d4741e6c356f537d087de7fade74f214a747cca5a`.
The protected journal is under the controller's
`legacy-migrations/vacatureradar/legacy-20260909-a/` directory and records
`committed`, `activationIntent:true`, `activated:true`. The original container
`VacatureRadar-legacy-legacy-20260909-a` remains stopped with its original data.
Docker health, HTTP readiness, the login page and running Celery processes were
checked after activation. Normal Gitea deployment run9748 succeeded against the
same verified revision. No post-commit rollback was attempted.
The initial helper's Docker JSON template incorrectly grouped `index` arguments.
Preparation rejected this before production changes. Revision `4d1fc0d` fixed it,
with a regression test and successful read-only qualification against real Docker.
The actual cutover used that reviewed helper revision. Three additional Linux
storage tests used isolated temporary fixtures, not application data.
LumaOps has **not** been migrated. Its original runtime has `AUTH_ENABLED=false`
and no admin token; the merged target requires authentication on its production
network binding. Preparation stopped before building or changing production.
Provisioning secure login needs an explicit operator decision; never disable the
target's authentication guard or describe the old runtime as a verified new build.
Failed preparation artifacts are retained; no LumaOps cutover journal exists.
@@ -0,0 +1,221 @@
"""Explicit fixed-target preparation, journaled cutover and crash recovery.
Run on the reviewed Unraid controller host. Builds a verified exact-source candidate
Preparation leaves live config/state/containers/data untouched. Cutover is a
separate --apply operation requiring a reviewed, verified preparation. Originals
and failed candidate data are retained; recover never undoes a committed migration.
"""
import argparse
from contextlib import ExitStack
import hashlib
import importlib.util
import json
import os
from pathlib import Path
import re
import subprocess
from plan import ATTEMPT, ID, SHA, TARGETS
BROKER_PATH = Path("/mnt/pool/appdata/gitea-deploy-runners/controller/deploy.py")
BROKER_SHA = "b5a0467f53acba5c783781a22bf81cf99f8b96ae99535460ad6236fac038330d"
JOURNAL_ROOT = Path("/mnt/pool/appdata/gitea-deploy-runners/legacy-migrations")
DOCKERFILES = {"vacatureradar": "Dockerfile.unraid", "lumaops": "Dockerfile"}
def durable_json(path, payload):
temporary = path.with_suffix(".tmp")
descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "w") as stream:
json.dump(payload, stream, sort_keys=True)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, path)
descriptor = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def require_real_child(path, parent):
resolved = path.resolve()
root = parent.resolve()
if path.is_symlink() or resolved == root or not resolved.is_relative_to(root):
raise ValueError("Candidate path is not a private child of the fixed root")
return resolved
def checked(command, *, log=None, timeout=30, env=None, cwd=None):
# Do not inherit application credentials into builds or print raw subprocess output.
environment = {key: os.environ[key] for key in ("PATH", "HOME", "DOCKER_CONFIG") if key in os.environ}
result = subprocess.run(command, env=environment if env is None else env, cwd=cwd, stdout=log or subprocess.PIPE,
stderr=log or subprocess.PIPE, text=log is None,
timeout=timeout, check=False)
if result.returncode:
raise RuntimeError("Bounded preparation command failed; inspect the protected attempt log")
return result.stdout if log is None else ""
def docker_json_template(fields):
# Go templates otherwise pass index/map/key as three separate json args.
return "\n".join("{{json (" + field + ")}}" for field in fields)
def original_identity(name, runner=checked):
fields = [".Id", ".Image", ".State.Running", ".State.Health.Status",
'index .Config.Labels "com.docker.compose.project"']
template = docker_json_template(fields)
values = runner(["docker", "inspect", "--format", template, name]).splitlines()
if len(values) != len(fields):
raise ValueError("Unexpected sanitized container inspection")
return dict(zip(("containerId", "imageId", "running", "health", "composeProject"),
(json.loads(value) for value in values)))
def assert_original(observed, slug, container_id, image_id):
if observed != {"containerId": container_id, "imageId": image_id,
"running": True, "health": "healthy", "composeProject": slug}:
raise ValueError("Original runtime identity/health drift; preparation refused")
def mount_inventory(target, runner=checked):
raw = json.loads(runner(["docker", "inspect", "--format", "{{json .Mounts}}", target]))
return sorted(({"type": item["Type"], "source": item["Source"],
"destination": item["Destination"], "rw": item["RW"]} for item in raw),
key=lambda item: item["destination"])
def expected_labels(metadata):
return {
"org.opencontainers.image.source": metadata["repository"],
"org.opencontainers.image.revision": metadata["revision"],
"org.opencontainers.image.source-tree": metadata["tree"],
"org.opencontainers.image.version": metadata["id"],
"org.opencontainers.image.created": metadata["builtAt"],
}
def verify_image(image, metadata, runner=checked):
image_id = runner(["docker", "image", "inspect", "--format", "{{.Id}}", image]).strip()
if not re.fullmatch(r"sha256:[0-9a-f]{64}", image_id):
raise ValueError("Candidate immutable image identity is invalid")
for key, expected in expected_labels(metadata).items():
actual = runner(["docker", "image", "inspect", "--format",
'{{index .Config.Labels "' + key + '"}}', image_id]).strip()
if actual != expected:
raise ValueError("Candidate immutable provenance did not verify")
return image_id
def prepare(broker, slug, revision, attempt, container_id, image_id, runner=checked):
if (slug not in TARGETS or not SHA.fullmatch(revision) or not ATTEMPT.fullmatch(attempt)
or not ID.fullmatch(container_id) or not re.fullmatch(r"sha256:[0-9a-f]{64}", image_id)):
raise ValueError("Invalid fixed target, commit, attempt or expected runtime identity")
repository = f"jens/{slug}"
original_name = TARGETS[slug][0]
workspace = require_real_child(JOURNAL_ROOT / slug / attempt, JOURNAL_ROOT)
source = require_real_child(broker.EXACT_SOURCE_ROOT / f"{slug}-migration-{attempt}", broker.EXACT_SOURCE_ROOT)
with ExitStack() as locks:
locks.enter_context(broker.exclusive_lock(broker.STATE_ROOT / "locks" / f"{slug}.lock", "migration project"))
locks.enter_context(broker.exclusive_lock(broker.GLOBAL_DEPLOY_LOCK, "migration preparation build"))
# Read config only after lock acquisition, unlike starting another broker process.
project = dict(broker.load_projects()[repository])
repo = broker.repository_path(repository, project)
broker.validate_commit(repo, str(project["branch"]), revision)
head = broker.output(["git", f"--git-dir={repo}", "rev-parse", f"refs/heads/{project['branch']}"])
if head != revision:
raise ValueError("Migration candidate must be the current configured branch head")
assert_original(original_identity(original_name, runner), slug, container_id, image_id)
if workspace.exists() or source.exists():
raise ValueError("Attempt artifacts already exist; retain them and review before a new attempt")
metadata = broker.deployment_metadata(repo, revision, repository)
if not broker.git_path_exists(repo, revision, "docker-compose.migration.yml"):
raise ValueError("Candidate commit does not contain the reviewed migration Compose contract")
if not broker.git_path_exists(repo, revision, DOCKERFILES[slug]):
raise ValueError("Candidate Dockerfile missing")
workspace.mkdir(parents=True, mode=0o700)
workspace.chmod(0o700)
journal_path = workspace / "prepare.json"
journal = {"schema": "legacy-prepare.v1", "phase": "preparing", "repository": repository,
"revision": revision, "metadata": metadata, "source": str(source),
"originalContainerId": container_id, "originalImageId": image_id,
"originalMounts": mount_inventory(container_id, runner),
"projectConfigSha256": hashlib.sha256(json.dumps(project, sort_keys=True).encode()).hexdigest(),
"stateSha256": hashlib.sha256((broker.STATE_ROOT / f"{slug}.json").read_bytes()).hexdigest(),
"originalProvenance": "unknown-legacy", "productionMutated": False}
durable_json(journal_path, journal)
try:
broker.sync_source(repo, revision, source, slug, exact=True)
if not broker.exact_source_matches_revision(repo, revision, source):
raise ValueError("Candidate source is not the exact committed tree")
from transaction import prepare_compose
candidate, command, cwd, environment, plan, nonce = prepare_compose(
broker, slug, revision, attempt, project, journal, workspace, runner)
image = environment["MIGRATION_IMAGE"]
journal.update(candidateProject=candidate, plan=plan, nonce=nonce)
journal["runtimeEnvSha256"] = hashlib.sha256((workspace / "runtime.env").read_bytes()).hexdigest()
journal.update(phase="building", candidateImage=image)
durable_json(journal_path, journal)
descriptor = os.open(workspace / "build.log", os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(descriptor, "wb") as log:
runner(command, cwd=cwd, env=environment, log=log, timeout=3600)
log.flush()
os.fsync(log.fileno())
immutable = verify_image(image, metadata, runner)
if not broker.exact_source_matches_revision(repo, revision, source):
raise ValueError("Candidate source changed during build")
assert_original(original_identity(original_name, runner), slug, container_id, image_id)
if mount_inventory(container_id, runner) != journal["originalMounts"]:
raise ValueError("Original mount inventory changed during preparation")
journal.update(phase="build_verified", candidateImageId=immutable)
durable_json(journal_path, journal)
return {"phase": journal["phase"], "repository": repository, "revision": revision,
"candidateImageId": immutable, "journal": str(journal_path),
"productionMutated": False, "cutoverAvailable": True}
except Exception:
journal["phase"] = "preparation_failed"
durable_json(journal_path, journal)
raise
def load_broker():
if hashlib.sha256(BROKER_PATH.read_bytes().replace(b"\r\n", b"\n")).hexdigest() != BROKER_SHA:
raise ValueError("Live broker source differs from the reviewed artifact")
spec = importlib.util.spec_from_file_location("reviewed_deploy_broker", BROKER_PATH)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("mode", choices=["prepare", "cutover", "recover"])
parser.add_argument("slug", choices=sorted(TARGETS))
parser.add_argument("revision")
parser.add_argument("attempt")
parser.add_argument("--container-id", required=True)
parser.add_argument("--image-id", required=True)
parser.add_argument("--apply", action="store_true")
args = parser.parse_args()
if not args.apply:
print(json.dumps({"apply": False, "mode": args.mode, "productionMutated": False}))
return
if not hasattr(os, "geteuid") or os.geteuid() != 0:
raise SystemExit("Run only as root on the configured Unraid controller host")
try:
broker = load_broker()
if args.mode == "prepare":
result = prepare(broker, args.slug, args.revision, args.attempt, args.container_id, args.image_id)
else:
from transaction import operate
result = operate(broker, args.mode, args.slug, args.revision, args.attempt, args.container_id, args.image_id)
except Exception:
raise SystemExit("Operation failed. Inspect the protected attempt journal; do not assume production or rollback succeeded. Use explicit recover for an unfinished cutover.") from None
print(json.dumps(result))
if __name__ == "__main__":
main()
+103
View File
@@ -0,0 +1,103 @@
"""Pure, fail-closed migration planning; this module cannot mutate Docker or files.
Accept only the sanitized inspect projection documented in README.md. A plan is
not deployment authorization, a snapshot, a receipt, or proof of successful backup.
"""
import re
from pathlib import PurePosixPath
TARGETS = {
"vacatureradar": ("VacatureRadar", "/mnt/pool/appdata/VacatureRadar"),
"lumaops": ("lumaops", "/mnt/pool/appdata/lumaops"),
}
SHA = re.compile(r"[0-9a-f]{40}")
ID = re.compile(r"[0-9a-f]{64}")
ATTEMPT = re.compile(r"[a-z0-9][a-z0-9-]{7,47}")
def _under(path, parent):
value = PurePosixPath(path)
return (
value.is_absolute()
and ".." not in value.parts
and value != PurePosixPath(parent)
and value.is_relative_to(parent)
)
def build_plan(slug, revision, attempt, observed):
"""Bind one target to retained runtime identity and isolated candidate data."""
if slug not in TARGETS or not SHA.fullmatch(revision) or not ATTEMPT.fullmatch(attempt):
raise ValueError("Invalid fixed target, full revision, or attempt identifier")
expected_name, appdata = TARGETS[slug]
keys = {"name", "containerId", "imageId", "running", "health", "composeProject", "mounts"}
if set(observed) != keys:
raise ValueError("Expected only the sanitized runtime projection")
if (
observed["name"] != expected_name
or observed["composeProject"] != slug
or not ID.fullmatch(observed["containerId"])
or not re.fullmatch(r"sha256:[0-9a-f]{64}", observed["imageId"])
or observed["running"] is not True
or observed["health"] != "healthy"
):
raise ValueError("Legacy runtime identity or health does not match")
mounts = observed["mounts"]
if not mounts or len(mounts) > 16:
raise ValueError("Expected bounded persistent mounts")
destinations = set()
sources = set()
for mount in mounts:
if set(mount) != {"type", "source", "destination", "rw"}:
raise ValueError("Unexpected mount fields")
destination = mount["destination"]
if not _under(destination, "/") or destination in destinations:
raise ValueError("Invalid or duplicate mount destination")
destinations.add(destination)
source = mount["source"]
if mount["type"] == "bind":
valid = _under(source, appdata)
elif mount["type"] == "volume":
valid = bool(re.fullmatch(r"/var/lib/docker/volumes/[0-9a-f]{64}/_data", source))
else:
valid = False
if not valid or mount["rw"] is not True:
raise ValueError("Unreviewed source, mount type, or access mode")
sources.add(source)
# Clone a parent only once; preserve nested bind destinations against the
# corresponding cloned subdirectory, not unrelated independent snapshots.
roots = sorted(source for source in sources if not any(
other != source and _under(source, other) for other in sources
))
candidate_root = f"/mnt/pool/appdata/gitea-deploy-runners/legacy-migrations/{slug}/{attempt}"
clones = {source: f"{candidate_root}/data/{index}" for index, source in enumerate(roots)}
candidate_mounts = []
for mount in mounts:
root = next(source for source in roots if mount["source"] == source or _under(mount["source"], source))
relative = PurePosixPath(mount["source"]).relative_to(root)
candidate_mounts.append({
"source": str(PurePosixPath(clones[root]) / relative),
"destination": mount["destination"],
"type": "bind", "rw": True,
})
return {
"schema": "legacy-migration-plan.v1", "executable": False,
"repository": f"jens/{slug}", "candidateRevision": revision,
"originalProvenance": "unknown-legacy", "originalRevision": None,
"originalContainerId": observed["containerId"], "originalImageId": observed["imageId"],
"originalName": expected_name, "retainedName": f"{expected_name}-legacy-{attempt}",
"candidateComposeProject": f"{slug}-managed",
"workspace": candidate_root,
"coldCopyRoots": [{"original": root, "candidate": clones[root]} for root in roots],
"candidateMounts": candidate_mounts,
"requiredGates": [
"exact-commit-and-tree-validated", "candidate-built-before-cutover",
"immutable-oci-provenance-verified", "project-and-global-locks-held",
"original-identity-revalidated", "legacy-stopped-before-data-copy",
"cold-copy-checksums-verified", "original-data-never-mounted-by-candidate",
"candidate-container-ids-recorded", "docker-and-http-health-verified",
"real-broker-receipt-verified", "broker-config-and-state-committed",
],
}
@@ -0,0 +1,96 @@
"""Private durable journal and fail-closed cold-copy verification primitives."""
import hashlib
import json
import os
from pathlib import Path
import stat
import tempfile
def digest(data):
return hashlib.sha256(data).hexdigest()
def private_write(path, data):
"""Unique temporary files let explicit recovery proceed after process crashes."""
path = Path(path)
if path.is_symlink():
raise ValueError("Refusing a symlink journal/configuration target")
descriptor, temporary = tempfile.mkstemp(prefix=f".{path.name}-", dir=path.parent)
with os.fdopen(descriptor, "wb") as stream:
stream.write(data)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, path)
descriptor = os.open(path.parent, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
def private_json(path, value):
private_write(path, json.dumps(value, sort_keys=True).encode())
def dotenv(entries):
"""Do not guess escaping for unsupported environment values; never log them."""
import re
result = {}
for entry in entries:
key, separator, value = entry.partition("=")
if not separator or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", key):
raise ValueError("Unsupported runtime environment entry")
if key in result or any(character in value for character in "\x00\n\r'\\"):
raise ValueError("Runtime environment needs reviewed dotenv escaping")
result[key] = value
return "".join(f"{key}='{value}'\n" for key, value in sorted(result.items())).encode()
def tree_manifest(root):
"""Hash content plus ownership/modes/xattrs; reject symlinks and special files.
Preserving an unknown symlink could let the candidate write original data.
Hardlink equivalence groups are compared without depending on inode numbers.
"""
root = Path(root)
if root.is_symlink() or not root.is_dir():
raise ValueError("Cold-copy root must be a real directory")
records = {}
inodes = {}
for path in [root, *sorted(root.rglob("*"))]:
info = path.lstat()
relative = str(path.relative_to(root))
if not stat.S_ISREG(info.st_mode) and not stat.S_ISDIR(info.st_mode):
raise ValueError("Cold-copy tree contains an unreviewed symlink/special file")
record = {"mode": stat.S_IMODE(info.st_mode), "uid": info.st_uid, "gid": info.st_gid}
if stat.S_ISREG(info.st_mode):
hasher = hashlib.sha256()
with path.open("rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
hasher.update(block)
record.update(kind="file", size=info.st_size, sha256=hasher.hexdigest())
group = (info.st_dev, info.st_ino)
record["hardlink"] = inodes.setdefault(group, relative)
else:
record["kind"] = "directory"
if hasattr(os, "listxattr"):
record["xattrs"] = {name: digest(os.getxattr(path, name)) for name in sorted(os.listxattr(path))}
records[relative] = record
return records
def clone_cold(source, target, runner):
source, target = Path(source), Path(target)
if target.exists() or target.is_symlink():
raise ValueError("Cold-copy target already exists")
before = tree_manifest(source)
target.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
# Destination does not exist: cp -a copies root metadata as well as children.
runner(["cp", "-a", "--reflink=auto", "--", str(source), str(target)], timeout=3600)
runner(["sync", "-f", "--", str(target)], timeout=3600)
copied = tree_manifest(target)
if before != copied or before != tree_manifest(source):
raise ValueError("Cold copy content/metadata does not match unchanged original")
return digest(json.dumps(before, sort_keys=True).encode())
@@ -0,0 +1,84 @@
"""Offline Compose rendering; never builds images or starts containers."""
import json
import os
from pathlib import Path
import shutil
import subprocess
import unittest
PROJECTS = Path(__file__).resolve().parents[3]
CASES = {
"VacatureRadar": ("app", ["docker-compose.unraid.yml"], {
"MIGRATION_LOCAL_ROOT": "/migration-fixture/local",
"VACATURERADAR_ENV_FILE": "/migration-fixture/runtime.env",
}, {"/app/local", "/app/media", "/app/logs"}),
"LumaOps": ("lumaops", ["docker-compose.yml", "docker-compose.unraid.yml"], {
"MIGRATION_CONFIG_ROOT": "/migration-fixture/config",
"MIGRATION_OPENRGB_ROOT": "/migration-fixture/openrgb",
"MIGRATION_LUMA_CONFIG_ROOT": "/migration-fixture/luma-config",
"MIGRATION_DATA_ROOT": "/migration-fixture/data",
"MIGRATION_LOGS_ROOT": "/migration-fixture/logs",
"MIGRATION_ENV_FILE": "/migration-fixture/runtime.env",
}, {"/config", "/config/openrgb", "/config/lumaops", "/data", "/logs"}),
}
@unittest.skipUnless(shutil.which("docker"), "Docker Compose CLI is required; no daemon used")
class ComposeTests(unittest.TestCase):
def render(self, name, missing=None):
_, files, variables, _ = CASES[name]
environment = dict(os.environ)
for key in list(environment):
if key.startswith("MIGRATION_") or key.startswith("VACATURERADAR_"):
environment.pop(key)
environment.update({
"MIGRATION_IMAGE": "migration-fixture:candidate",
"MIGRATION_NETWORK": name.lower() + "_default",
"MIGRATION_ATTEMPT": "fixture-migration-attempt",
"MIGRATION_SOURCE_REVISION": "a" * 40,
"MIGRATION_SOURCE_TREE": "b" * 40,
"MIGRATION_BUILD_ID": "a" * 12,
"MIGRATION_BUILD_DATE": "2026-09-09T00:00:00.000Z",
"LUMAOPS_ADMIN_TOKEN": "fixture-only-not-a-production-token-000000",
**variables,
})
if missing:
environment.pop(missing)
command = ["docker", "compose", "--env-file", os.devnull]
for file in [*files, "docker-compose.migration.yml"]:
command.extend(["-f", file])
command.extend(["config", "--no-env-resolution", "--no-path-resolution", "--format", "json"])
return subprocess.run(command, cwd=PROJECTS / name, env=environment,
capture_output=True, text=True, timeout=20)
def test_merged_mounts_are_exclusively_explicit_clones(self):
for name, (service, _, _, destinations) in CASES.items():
with self.subTest(name=name):
result = self.render(name)
self.assertEqual(result.returncode, 0, "Fixture Compose rendering failed")
config = json.loads(result.stdout)
runtime = config["services"][service]
self.assertEqual(runtime["network_mode"], name.lower() + "_default")
self.assertEqual(config["name"], name.lower() + "-managed")
self.assertEqual({m["target"] for m in runtime["volumes"]}, destinations)
for mount in runtime["volumes"]:
self.assertEqual(mount["type"], "bind")
self.assertTrue(mount["source"].startswith("/migration-fixture/"))
self.assertFalse(mount["bind"]["create_host_path"])
labels = runtime["build"]["labels"]
self.assertEqual(labels["org.opencontainers.image.revision"], "a" * 40)
self.assertEqual(labels["org.opencontainers.image.source-tree"], "b" * 40)
self.assertEqual(labels["org.opencontainers.image.source"], "jens/" + name.lower())
def test_missing_required_inputs_fail_closed(self):
for name, (_, _, variables, _) in CASES.items():
for variable in [*variables, "MIGRATION_NETWORK", "MIGRATION_IMAGE", "MIGRATION_ATTEMPT", "MIGRATION_SOURCE_REVISION",
"MIGRATION_SOURCE_TREE", "MIGRATION_BUILD_ID", "MIGRATION_BUILD_DATE"]:
with self.subTest(name=name, variable=variable):
self.assertNotEqual(self.render(name, variable).returncode, 0)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,49 @@
import unittest
from execute import assert_original, docker_json_template, expected_labels, original_identity, verify_image
class ExecuteTests(unittest.TestCase):
def test_template_groups_function_arguments_before_json_encoding(self):
self.assertEqual(
docker_json_template(['.Id', 'index .Config.Labels "com.docker.compose.project"']),
'{{json (.Id)}}\n{{json (index .Config.Labels "com.docker.compose.project")}}',
)
def test_inspection_never_requests_environment(self):
calls = []
def runner(command):
calls.append(command)
return '"' + "a" * 64 + '"\n"sha256:' + "b" * 64 + '"\ntrue\n"healthy"\n"lumaops"'
observed = original_identity("lumaops", runner)
assert_original(observed, "lumaops", "a" * 64, "sha256:" + "b" * 64)
self.assertNotIn(".Config.Env", str(calls))
self.assertEqual(calls[0][:2], ["docker", "inspect"])
def test_runtime_drift_blocks(self):
with self.assertRaises(ValueError):
assert_original({}, "lumaops", "a" * 64, "sha256:" + "b" * 64)
def test_image_labels_are_checked_on_immutable_id(self):
metadata = {"repository": "jens/lumaops", "revision": "a" * 40,
"tree": "b" * 40, "id": "a" * 12, "builtAt": "2026-09-09T00:00:00.000Z"}
labels = expected_labels(metadata)
calls = []
def runner(command):
calls.append(command)
if command[-2] == "{{.Id}}":
return "sha256:" + "c" * 64
key = command[-2].split('"')[1]
return labels[key]
self.assertEqual(verify_image("candidate", metadata, runner), "sha256:" + "c" * 64)
self.assertTrue(all(command[-1] == "sha256:" + "c" * 64 for command in calls[1:]))
with self.assertRaises(ValueError):
verify_image("candidate", metadata, lambda command: "invalid")
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,61 @@
import unittest
from plan import build_plan
def observation():
return {
"name": "VacatureRadar", "containerId": "a" * 64,
"imageId": "sha256:" + "b" * 64, "running": True,
"health": "healthy", "composeProject": "vacatureradar",
"mounts": [
{"type": "bind", "source": "/mnt/pool/appdata/VacatureRadar/local",
"destination": "/app/local", "rw": True},
{"type": "bind", "source": "/mnt/pool/appdata/VacatureRadar/local/logs",
"destination": "/app/logs", "rw": True},
],
}
class PlanTests(unittest.TestCase):
def test_nested_mounts_share_one_cold_copy(self):
plan = build_plan("vacatureradar", "c" * 40, "migration-20260909", observation())
self.assertEqual(len(plan["coldCopyRoots"]), 1)
self.assertTrue(plan["candidateMounts"][1]["source"].endswith("/data/0/logs"))
self.assertFalse(plan["executable"])
self.assertIsNone(plan["originalRevision"])
self.assertEqual(plan["originalProvenance"], "unknown-legacy")
self.assertNotEqual(plan["candidateComposeProject"], "vacatureradar")
def test_rejects_runtime_drift_and_unsafe_paths(self):
for key, value in [("name", "other"), ("containerId", "short"),
("running", False), ("health", "starting")]:
observed = observation()
observed[key] = value
with self.subTest(key=key), self.assertRaises(ValueError):
build_plan("vacatureradar", "c" * 40, "migration-20260909", observed)
for source in ["/", "/mnt/pool/appdata", "/mnt/pool/appdata/VacatureRadar/../other"]:
observed = observation()
observed["mounts"][0]["source"] = source
with self.subTest(source=source), self.assertRaises(ValueError):
build_plan("vacatureradar", "c" * 40, "migration-20260909", observed)
def test_rejects_secret_bearing_full_inspect(self):
observed = observation()
observed["Config"] = {"Env": ["fixture-only"]}
with self.assertRaises(ValueError):
build_plan("vacatureradar", "c" * 40, "migration-20260909", observed)
def test_anonymous_volume_is_cloned_not_reused(self):
observed = observation()
observed.update(name="lumaops", composeProject="lumaops", mounts=[
{"type": "volume", "source": "/var/lib/docker/volumes/" + "d" * 64 + "/_data",
"destination": "/config", "rw": True},
])
plan = build_plan("lumaops", "c" * 40, "migration-20260909", observed)
self.assertEqual(plan["candidateMounts"][0]["type"], "bind")
self.assertNotIn("/var/lib/docker/volumes/", plan["candidateMounts"][0]["source"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,62 @@
"""Linux filesystem qualification with isolated temporary data, never appdata."""
import os
from pathlib import Path
import stat
import subprocess
import tempfile
import unittest
from storage import clone_cold, private_write, tree_manifest
@unittest.skipUnless(os.name == "posix", "Linux ownership/fsync qualification")
class StorageTests(unittest.TestCase):
def test_private_replacement_is_durable_and_private(self):
with tempfile.TemporaryDirectory(prefix="migration-storage-test-") as directory:
path = Path(directory) / "journal.json"
private_write(path, b"first")
private_write(path, b"second")
self.assertEqual(path.read_bytes(), b"second")
self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600)
target = Path(directory) / "linked"
target.symlink_to(path)
with self.assertRaises(ValueError):
private_write(target, b"must not replace")
self.assertEqual(path.read_bytes(), b"second")
def test_real_cold_copy_preserves_content_modes_and_hardlinks(self):
with tempfile.TemporaryDirectory(prefix="migration-storage-test-") as directory:
source = Path(directory) / "original"
source.mkdir(mode=0o750)
payload = source / "database"
payload.write_bytes(b"representative fixture data")
payload.chmod(0o640)
os.link(payload, source / "hardlink")
destination = Path(directory) / "candidate"
def runner(command, **kwargs):
subprocess.run(command, check=True, capture_output=True, **kwargs)
checksum = clone_cold(source, destination, runner)
self.assertEqual(tree_manifest(source), tree_manifest(destination))
self.assertEqual(len(checksum), 64)
self.assertNotEqual(payload.stat().st_ino, (destination / "database").stat().st_ino)
(destination / "database").write_bytes(b"candidate mutation")
self.assertEqual(payload.read_bytes(), b"representative fixture data")
with self.assertRaises(ValueError):
clone_cold(source, destination, runner)
def test_special_files_are_rejected_before_copy(self):
with tempfile.TemporaryDirectory(prefix="migration-storage-test-") as directory:
root = Path(directory)
(root / "link").symlink_to("/tmp")
with self.assertRaises(ValueError):
tree_manifest(root)
(root / "link").unlink()
os.mkfifo(root / "fifo")
with self.assertRaises(ValueError):
tree_manifest(root)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,289 @@
"""No Docker/network calls: exercise crash phases and fail-closed recovery gates."""
import json
from pathlib import Path
import tempfile
import unittest
from unittest.mock import patch
from storage import clone_cold, digest, dotenv, tree_manifest
from transaction import (Production, STEPS, candidate_project,
recover_transaction, run_transaction, validate_runtime_safety)
class Crash(BaseException):
pass
class FakeOperations:
def __init__(self, fail=None, crash=False):
self.journal = {"phase": "snapshotted"}
self.fail, self.crash = fail, crash
self.original_running, self.renamed, self.candidate = True, False, False
self.state, self.config = "original", "original"
self.clones = False
self.foreign = False
self.saved = []
def save(self, phase):
self.saved.append(phase)
self.journal["phase"] = phase
def event(self, name):
if self.fail == name:
raise Crash() if self.crash else ValueError("fixture failure")
def stopping(self):
self.original_running = False
self.event("after_stop")
def copying(self):
assert not self.original_running
self.clones = True
self.event("after_copy")
def renaming(self):
self.renamed = True
self.event("after_rename")
def deploying(self):
self.candidate = True
self.event("after_deploy")
def verifying(self):
self.event("during_verify")
def committing(self):
self.state = "candidate"
self.event("after_receipt")
self.config = "candidate"
self.event("after_config")
def recover(self):
if self.foreign:
raise ValueError("foreign canonical container")
self.candidate = False
self.state = self.config = "original"
self.renamed = False
self.original_running = True
def verify_committed(self):
assert self.state == self.config == "candidate"
def original(self, healthy=False):
assert self.original_running
def http_health(self):
pass
def activate(self):
assert self.journal["phase"] == "committed"
self.event("during_activation")
self.journal["activated"] = True
class TransactionTests(unittest.TestCase):
def test_runtime_safety_parity_rejects_each_security_or_device_drift(self):
host = {"Devices": [{"PathOnHost": "/dev/example", "PathInContainer": "/dev/example", "CgroupPermissions": "rwm"}],
"CapAdd": ["CAP_CHOWN"], "CapDrop": ["ALL"], "ReadonlyRootfs": True,
"SecurityOpt": ["no-new-privileges:true"], "Tmpfs": {"/tmp": "size=128m"}}
service = {"devices": [{"source": "/dev/example", "target": "/dev/example", "permissions": "rwm"}],
"cap_add": ["CHOWN"], "cap_drop": ["ALL"], "read_only": True,
"security_opt": ["no-new-privileges:true"], "tmpfs": ["/tmp:size=128m"]}
validate_runtime_safety(host, service)
for key, value in (("devices", []), ("cap_add", []), ("cap_drop", []), ("read_only", False),
("privileged", True), ("security_opt", []), ("tmpfs", [])):
with self.subTest(key=key), self.assertRaises(ValueError):
validate_runtime_safety(host, {**service, key: value})
def test_activation_failure_never_rolls_back_and_recovery_resumes(self):
ops = FakeOperations(fail="during_activation")
with self.assertRaises(ValueError):
run_transaction(ops)
self.assertEqual(ops.journal["phase"], "committed")
self.assertTrue(ops.candidate)
self.assertFalse(ops.original_running)
ops.fail = None
self.assertEqual(recover_transaction(ops), "committed")
self.assertTrue(ops.journal["activated"])
def test_real_marker_release_requires_matching_nonce_and_resumes_after_unlink(self):
with tempfile.TemporaryDirectory() as temporary:
ops = Production.__new__(Production)
ops.workspace = Path(temporary).resolve()
root = ops.workspace / "local"
root.mkdir()
ops.slug = "vacatureradar"
ops.plan = {"candidateMounts": [{"destination": "/app/local", "source": str(root)}]}
ops.journal = {"phase": "committed", "nonce": "correct", "candidateContainerId": "candidate"}
ops.verify_committed = lambda: None
ops.assert_candidate = lambda value: None
ops.save = lambda phase: None
marker = ops.hold_path()
marker.write_bytes(b"wrong")
with patch("transaction.private_read", side_effect=lambda path: path.read_bytes()):
with self.assertRaises(ValueError):
ops.activate()
self.assertTrue(marker.exists())
marker.write_bytes(ops.hold_bytes())
with patch("transaction.os.open", return_value=123), patch("transaction.os.fsync"), patch("transaction.os.close"):
ops.activate()
self.assertFalse(marker.exists())
self.assertTrue(ops.journal["activationIntent"])
ops.activate()
self.assertTrue(ops.journal["activated"])
ops.journal.pop("activationIntent")
with self.assertRaises(ValueError):
ops.activate()
def test_intent_precedes_each_successful_phase(self):
ops = FakeOperations()
run_transaction(ops)
self.assertEqual(ops.saved, [*STEPS, "committed"])
self.assertFalse(ops.original_running)
self.assertTrue(ops.renamed and ops.clones and ops.candidate)
self.assertEqual(recover_transaction(ops), "committed")
self.assertTrue(ops.candidate, "Recovery cannot discard writes after a committed migration")
def test_each_failure_restores_original_and_keeps_clones(self):
for failure in ("after_stop", "after_copy", "after_rename", "after_deploy",
"during_verify", "after_receipt", "after_config"):
with self.subTest(failure=failure):
ops = FakeOperations(fail=failure)
with self.assertRaisesRegex(RuntimeError, "original runtime restored"):
run_transaction(ops)
self.assertTrue(ops.original_running)
self.assertEqual(ops.state, "original")
self.assertEqual(ops.config, "original")
self.assertFalse(ops.candidate)
self.assertEqual(ops.clones, failure != "after_stop")
self.assertEqual(recover_transaction(ops), "rolled_back")
def test_abrupt_crash_after_stop_rename_and_receipt_requires_explicit_recovery(self):
for failure in ("after_stop", "after_rename", "after_receipt", "after_config"):
with self.subTest(failure=failure):
ops = FakeOperations(fail=failure, crash=True)
with self.assertRaises(Crash):
run_transaction(ops)
self.assertNotEqual(ops.journal["phase"], "committed")
self.assertEqual(recover_transaction(ops), "rolled_back")
self.assertTrue(ops.original_running)
def test_foreign_runtime_blocks_automatic_recovery(self):
ops = FakeOperations(fail="during_verify")
ops.foreign = True
with self.assertRaisesRegex(RuntimeError, "requires explicit recovery"):
run_transaction(ops)
self.assertEqual(ops.journal["phase"], "recovery_failed")
self.assertTrue(ops.candidate)
def test_production_candidate_gate_rejects_wrong_nonce_project_and_replaced_id(self):
ops = Production.__new__(Production)
ops.prepared = {"originalContainerId": "original"}
ops.plan = {"candidateComposeProject": "lumaops-managed"}
ops.journal = {"nonce": "expected", "candidateContainerId": "candidate"}
ops.runner = object()
for fields in (["candidate", "image", "lumaops-managed", "wrong"],
["candidate", "image", "foreign-project", "expected"],
["original", "image", "lumaops-managed", "expected"],
["replacement", "image", "lumaops-managed", "expected"]):
with self.subTest(fields=fields), patch("transaction.inspect_fields", return_value=fields):
with self.assertRaises(ValueError):
ops.assert_candidate(fields[0])
def test_future_broker_metadata_remains_dynamic(self):
original = {"source_dir": "/old", "compose_files": ["docker-compose.unraid.yml"],
"clean_untracked_paths": ["old"], "deploy_script": "old.sh"}
prepared = {"source": "/exact/new"}
plan = {"repository": "jens/vacatureradar", "candidateComposeProject": "vacatureradar-managed",
"candidateMounts": [{"source": "/clone/local", "destination": "/app/local"}]}
project = candidate_project(original, prepared, plan, "nonce", Path("/private/runtime.env"))
self.assertTrue(project["exact_source"] and project["verify_oci_revision"])
self.assertNotIn("deploy_script", project)
self.assertEqual(project["environment"]["MIGRATION_SOURCE_REVISION"], "{revision}")
self.assertEqual(project["environment"]["MIGRATION_SOURCE_TREE"], "{tree}")
self.assertEqual(project["environment"]["MIGRATION_BUILD_ID"], "{build_id}")
self.assertEqual(project["environment"]["MIGRATION_BUILD_DATE"], "{build_date}")
def test_dotenv_is_literal_and_rejects_unsupported_escaping(self):
self.assertEqual(dotenv(["KEY=has $literal #value"]), b"KEY='has $literal #value'\n")
for entries in (["KEY=one", "KEY=two"], ["BAD KEY=value"], ["KEY=has'quote"], ["KEY=line\nbreak"]):
with self.assertRaises(ValueError):
dotenv(entries)
def test_manifest_detects_content_or_mode_change(self):
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
file = root / "data"
file.write_bytes(b"fixture-only")
before = digest(json.dumps(tree_manifest(root), sort_keys=True).encode())
file.write_bytes(b"changed")
after = digest(json.dumps(tree_manifest(root), sort_keys=True).encode())
self.assertNotEqual(before, after)
def test_cold_copy_uses_archive_mode_and_rejects_metadata_mismatch(self):
with tempfile.TemporaryDirectory() as temporary:
source, target = Path(temporary) / "source", Path(temporary) / "target"
source.mkdir()
calls = []
identical = {".": {"mode": 448, "uid": 999, "gid": 999}}
with patch("storage.tree_manifest", return_value=identical):
clone_cold(source, target, lambda command, **kwargs: calls.append(command))
self.assertEqual(calls[0][:4], ["cp", "-a", "--reflink=auto", "--"])
with patch("storage.tree_manifest", side_effect=[identical, {}]):
with self.assertRaisesRegex(ValueError, "content/metadata"):
clone_cold(source, target, lambda *args, **kwargs: None)
def test_real_recovery_adapter_refuses_foreign_container_before_remove(self):
with tempfile.TemporaryDirectory() as temporary:
workspace = Path(temporary)
config, state = b"{}", b'{"status":"failed"}'
for name, value in [("original-projects.json", config), ("original-state.json", state),
("projects.json", config), ("state.json", state)]:
(workspace / name).write_bytes(value)
ops = Production.__new__(Production)
ops.workspace = workspace
ops.broker = type("Broker", (), {"CONFIG_PATH": workspace / "projects.json"})()
ops.state_path = workspace / "state.json"
ops.prepared = {"originalContainerId": "original"}
ops.plan = {"originalName": "lumaops", "candidateComposeProject": "lumaops-managed"}
ops.journal = {"originalConfigSha256": digest(config), "candidateConfigSha256": "other",
"originalStateSha256": digest(state), "nonce": "expected"}
ops.original = lambda **kwargs: {"running": False}
ops.canonical_id = lambda name: "foreign"
calls = []
ops.runner = lambda command, **kwargs: calls.append(command)
with patch("transaction.inspect_fields", return_value=["foreign", "image", "lumaops-managed", "wrong"]):
with self.assertRaisesRegex(ValueError, "unrelated"):
ops.recover()
self.assertEqual(calls, [])
def test_verification_binds_normal_broker_receipt_to_prepared_image(self):
ops = Production.__new__(Production)
ops.journal = {"candidateContainerId": "candidate"}
ops.slug = "lumaops"
ops.plan = {"originalName": "lumaops", "candidateMounts": []}
ops.prepared = {"metadata": {"fixture": "metadata"}, "candidateImageId": "immutable", "revision": "rev"}
ops.project, ops.repository, ops.runner = {}, "jens/lumaops", None
ops.assert_candidate = lambda value: None
ops.http_health = lambda: None
ops.save = lambda value: None
seen = []
class Broker:
def verify_deployment(self, project, metadata, **kwargs):
seen.append(kwargs)
return {"fixture": "real-broker-receipt"}
def validate_recorded_receipt(self, *args):
return None
ops.broker = Broker()
with patch("transaction.mounts", return_value=[]), patch("transaction.inspect_fields", return_value=[{"lumaops_default": {}}]):
ops.verifying()
self.assertEqual(seen, [{"expected_image_ids": {"lumaops": "immutable"}}])
self.assertEqual(ops.journal["receipt"], {"fixture": "real-broker-receipt"})
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,517 @@
"""Journaled legacy cutover through normal broker functions; fixed two targets only.
No provenance is written for the original runtime. Recovery retains every data
clone and original image/container. Completed migrations are never automatically
rolled back: new writes may have reached candidate data after commit.
"""
from contextlib import ExitStack
import json
import os
from pathlib import Path
import secrets
import shutil
import time
from urllib.request import urlopen
from execute import (JOURNAL_ROOT, assert_original, checked, docker_json_template, mount_inventory, original_identity,
require_real_child, verify_image)
from plan import ATTEMPT, SHA, TARGETS, build_plan
from storage import clone_cold, digest, dotenv, private_json, private_write, tree_manifest
STEPS = ("stopping", "copying", "renaming", "deploying", "verifying", "committing")
def run_transaction(ops):
"""Write intent before every side effect, allowing recovery after abrupt exit."""
try:
for phase in STEPS:
ops.save(phase)
getattr(ops, phase)()
ops.save("committed")
except Exception:
try:
ops.recover()
ops.save("rolled_back")
except Exception:
ops.save("recovery_failed")
raise RuntimeError("Migration failed and rollback requires explicit recovery") from None
raise RuntimeError("Migration failed; original runtime restored and candidate data retained") from None
# Outside rollback scope: releasing workers permits new external effects/data.
ops.activate()
def recover_transaction(ops):
if ops.journal["phase"] == "committed":
ops.verify_committed()
ops.activate()
return "committed"
if ops.journal["phase"] == "rolled_back":
ops.original(healthy=True)
ops.http_health()
return "rolled_back"
ops.recover()
ops.save("rolled_back")
return "rolled_back"
def inspect_fields(runner, target, expressions):
template = docker_json_template(expressions)
return [json.loads(line) for line in runner(["docker", "inspect", "--format", template, target]).splitlines()]
def mounts(runner, target):
raw = inspect_fields(runner, target, [".Mounts"])[0]
return sorted(({"type": item["Type"], "source": item["Source"],
"destination": item["Destination"], "rw": item["RW"]} for item in raw),
key=lambda item: item["destination"])
def candidate_project(original, prepared, plan, nonce, env_path):
slug = plan["repository"].split("/")[1]
by_target = {item["destination"]: item["source"] for item in plan["candidateMounts"]}
names = ({"MIGRATION_LOCAL_ROOT": "/app/local"} if slug == "vacatureradar" else {
"MIGRATION_CONFIG_ROOT": "/config", "MIGRATION_OPENRGB_ROOT": "/config/openrgb",
"MIGRATION_LUMA_CONFIG_ROOT": "/config/lumaops", "MIGRATION_DATA_ROOT": "/data",
"MIGRATION_LOGS_ROOT": "/logs"})
environment = {**dict(original.get("environment", {})),
**{key: by_target[target] for key, target in names.items()},
"MIGRATION_ATTEMPT": nonce, "MIGRATION_IMAGE": f"gitea-managed/{slug}:{{revision}}",
"MIGRATION_NETWORK": slug + "_default",
"MIGRATION_SOURCE_REVISION": "{revision}", "MIGRATION_SOURCE_TREE": "{tree}",
"MIGRATION_BUILD_ID": "{build_id}", "MIGRATION_BUILD_DATE": "{build_date}",
"MIGRATION_ENV_FILE": str(env_path), "VACATURERADAR_ENV_FILE": str(env_path)}
result = dict(original)
for key in ("deploy_script", "archive_deploy_script", "manual", "clean_untracked_paths", "legacy_containers"):
result.pop(key, None)
result.update(source_dir=prepared["source"], compose_project=plan["candidateComposeProject"],
exact_source=True, verify_oci_revision=True, env_source=str(env_path),
environment=environment,
compose_files=[*original["compose_files"], "docker-compose.migration.yml"])
return result
def render_contract(broker, project, prepared, plan, runner):
cwd, command = broker.compose_command(project)
environment = broker.deployment_environment(project, prepared["revision"], plan["repository"], prepared["metadata"])
rendered = json.loads(runner(command + ["config", "--format", "json"], cwd=cwd, env=environment))
services = rendered.get("services", {})
if len(services) != 1:
raise ValueError("Migration supports exactly one reviewed application container")
service = next(iter(services.values()))
if service.get("container_name") != plan["originalName"]:
raise ValueError("Candidate canonical name differs from the plan")
network = plan["repository"].split("/")[1] + "_default"
original_networks = inspect_fields(runner, prepared["originalContainerId"], [".NetworkSettings.Networks"])[0]
if set(original_networks) != {network} or service.get("network_mode") != network:
raise ValueError("Candidate network must exactly preserve the reviewed original network")
if prepared.get("originalNetwork") not in {None, network}:
raise ValueError("Original network changed since preparation")
host = inspect_fields(runner, prepared["originalContainerId"], [".HostConfig"])[0]
validate_runtime_safety(host, service)
expected = {item["destination"]: item["source"] for item in plan["candidateMounts"]}
observed = {}
for item in service.get("volumes", []):
if item.get("type") != "bind" or item.get("read_only") or item.get("bind", {}).get("create_host_path") is not False:
raise ValueError("Candidate contains an unreviewed writable mount")
observed[item["target"]] = item["source"]
if observed != expected or len(observed) != len(service.get("volumes", [])):
raise ValueError("Rendered candidate mounts do not exactly match cold clones")
bindings = inspect_fields(runner, prepared["originalContainerId"], [".HostConfig.PortBindings"])[0]
old_ports = sorted((port, item.get("HostIp") or "0.0.0.0", str(item["HostPort"]))
for port, items in bindings.items() for item in items or [])
new_ports = sorted((f"{item['target']}/{item.get('protocol', 'tcp')}", item.get("host_ip") or "0.0.0.0", str(item["published"]))
for item in service.get("ports", []))
if old_ports != new_ports:
raise ValueError("Candidate published ports differ from the original runtime")
if service.get("labels", {}).get("io.itworx.migration-attempt") != environment["MIGRATION_ATTEMPT"]:
raise ValueError("Candidate ownership correlation label is absent")
return cwd, command, environment
def validate_runtime_safety(host, service):
"""Compare nonsecret hardware/security bindings; unsupported shapes fail closed."""
old_devices = sorted((item["PathOnHost"], item["PathInContainer"], item["CgroupPermissions"])
for item in host.get("Devices") or [])
new_devices = sorted((item["source"], item["target"], item.get("permissions", "rwm"))
for item in service.get("devices", []))
if old_devices != new_devices or host.get("DeviceRequests") or host.get("DeviceCgroupRules"):
raise ValueError("Candidate device contract differs or is unsupported")
for actual, desired in (("CapAdd", "cap_add"), ("CapDrop", "cap_drop")):
normalize = lambda values: sorted(value.upper().removeprefix("CAP_") for value in values or [])
if normalize(host.get(actual)) != normalize(service.get(desired)):
raise ValueError("Candidate capability contract differs")
for actual, desired in (("ReadonlyRootfs", "read_only"), ("Privileged", "privileged")):
if bool(host.get(actual)) != bool(service.get(desired)):
raise ValueError("Candidate filesystem/privilege contract differs")
if sorted(host.get("SecurityOpt") or []) != sorted(service.get("security_opt") or []):
raise ValueError("Candidate security options differ")
new_tmpfs = {}
for item in service.get("tmpfs") or []:
target, separator, options = item.partition(":")
if target in new_tmpfs:
raise ValueError("Duplicate candidate tmpfs target")
new_tmpfs[target] = options if separator else ""
if (host.get("Tmpfs") or {}) != new_tmpfs:
raise ValueError("Candidate tmpfs contract differs")
def prepare_compose(broker, slug, revision, attempt, original, prepared, workspace, runner):
if slug == "vacatureradar":
source = Path(prepared["source"])
required = {
"scripts/migration_worker_gate.py": (".migration-worker-hold", "while check():"),
"deployment/unraid/supervisord.conf": ("migration_worker_gate.py worker", "migration_worker_gate.py scheduler"),
"config/settings.py": ("apps.core.middleware.MigrationMaintenanceMiddleware",),
"apps/core/middleware.py": ("class MigrationMaintenanceMiddleware", "migration_maintenance"),
}
for filename, fragments in required.items():
content = (source / filename).read_text()
if any(fragment not in content for fragment in fragments):
raise ValueError("Exact source lacks the reviewed maintenance contract")
observed = {"name": TARGETS[slug][0], **original_identity(prepared["originalContainerId"], runner),
"mounts": prepared["originalMounts"]}
plan = build_plan(slug, revision, attempt, observed)
if Path(plan["workspace"]).resolve() != workspace:
raise ValueError("Plan workspace differs from the protected preparation directory")
nonce = secrets.token_hex(16)
env_path = workspace / "runtime.env"
entries = inspect_fields(runner, prepared["originalContainerId"], [".Config.Env"])[0]
private_write(env_path, dotenv(entries))
prepared["originalNetwork"] = slug + "_default"
candidate = candidate_project(original, prepared, plan, nonce, env_path)
repo = broker.repository_path(plan["repository"], original)
broker.validate_project_config(repo, candidate, revision)
cwd, command, environment = render_contract(broker, candidate, prepared, plan, runner)
return candidate, command + ["build"], cwd, environment, plan, nonce
class Production:
def __init__(self, broker, slug, prepared, workspace, journal, runner=checked):
self.broker, self.slug, self.prepared = broker, slug, prepared
self.workspace, self.journal, self.runner = workspace, journal, runner
self.repository = f"jens/{slug}"
self.plan = journal["plan"]
self.project = dict(journal["candidateProject"])
self.project.update(_repository_name=self.repository,
_repository_path=journal["repositoryPath"],
_deployment_metadata=prepared["metadata"])
self.state_path = broker.STATE_ROOT / f"{slug}.json"
def save(self, phase):
self.journal["phase"] = phase
private_json(self.workspace / "cutover.json", self.journal)
def canonical_id(self, name):
result = self.runner(["docker", "container", "ls", "--all", "--quiet", "--no-trunc",
"--filter", f"name=^/{name}$"]).strip().splitlines()
if len(result) > 1:
raise ValueError("Ambiguous container identity")
return result[0] if result else None
def original(self, healthy=False):
container_id = self.prepared["originalContainerId"]
observed = original_identity(container_id, self.runner)
if observed["imageId"] != self.prepared["originalImageId"] or observed["composeProject"] != self.slug:
raise ValueError("Original immutable runtime identity changed")
if mounts(self.runner, container_id) != self.journal["originalMounts"]:
raise ValueError("Original mounts changed")
networks = inspect_fields(self.runner, container_id, [".NetworkSettings.Networks"])[0]
if set(networks) != {self.slug + "_default"}:
raise ValueError("Original network changed")
if healthy:
assert_original(observed, self.slug, container_id, self.prepared["originalImageId"])
return observed
def stopping(self):
self.original(healthy=True)
self.runner(["docker", "stop", "--time", "60", self.prepared["originalContainerId"]], timeout=90)
if self.original()["running"]:
raise ValueError("Original did not stop")
def copying(self):
if self.original()["running"]:
raise ValueError("Cold copy requires stopped original")
checksums = {}
for item in self.plan["coldCopyRoots"]:
original = Path(item["original"])
if original.is_symlink() or original.resolve() != original:
raise ValueError("Unreviewed symlink in original cold-copy root")
target = require_real_child(Path(item["candidate"]), self.workspace)
checksums[item["original"]] = clone_cold(original, target, self.runner)
self.journal["coldCopyChecksums"] = checksums
self.save("copying")
if self.slug == "vacatureradar":
marker = self.hold_path()
if marker.exists() or marker.is_symlink():
raise ValueError("Cold clone already contains a maintenance hold")
private_write(marker, self.hold_bytes())
def hold_path(self):
roots = [item["source"] for item in self.plan["candidateMounts"]
if item["destination"] == "/app/local"]
if len(roots) != 1:
raise ValueError("Maintenance hold requires one exact cloned local root")
return require_real_child(Path(roots[0]) / ".migration-worker-hold", self.workspace)
def hold_bytes(self):
return ("legacy-migration.v1:" + self.journal["nonce"] + "\n").encode()
def verify_hold(self):
if self.slug == "vacatureradar" and private_read(self.hold_path()) != self.hold_bytes():
raise ValueError("Maintenance hold identity does not match this attempt")
def activate(self):
if self.journal.get("phase") != "committed":
raise ValueError("Activation requires a durable committed transaction")
self.verify_committed()
self.assert_candidate(self.journal["candidateContainerId"])
if self.slug == "vacatureradar":
marker = self.hold_path()
if marker.exists() or marker.is_symlink():
self.verify_hold()
self.journal["activationIntent"] = True
self.save("committed")
marker.unlink()
descriptor = os.open(marker.parent, os.O_RDONLY)
try:
os.fsync(descriptor)
finally:
os.close(descriptor)
elif not self.journal.get("activationIntent"):
raise ValueError("Maintenance hold disappeared before activation intent")
self.journal["activated"] = True
self.save("committed")
def renaming(self):
if self.canonical_id(self.plan["retainedName"]):
raise ValueError("Retained container name is already occupied")
if self.canonical_id(self.plan["originalName"]) != self.prepared["originalContainerId"]:
raise ValueError("Canonical container identity changed")
self.runner(["docker", "rename", self.prepared["originalContainerId"], self.plan["retainedName"]])
def deploying(self):
self.verify_hold()
if self.canonical_id(self.plan["originalName"]):
raise ValueError("Canonical name was claimed before candidate deployment")
# True first deploy relative to the distinct managed Compose project.
if self.broker.capture_compose_rollback(self.project) is not None:
raise ValueError("Candidate project is not a first deployment")
self.broker.revalidate_deployment_candidate(Path(self.journal["repositoryPath"]), self.project,
self.prepared["revision"], self.repository, self.prepared["metadata"])
cwd, command, environment = render_contract(self.broker, self.project, self.prepared, self.plan, self.runner)
image_id = verify_image(environment["MIGRATION_IMAGE"], self.prepared["metadata"], self.runner)
if image_id != self.prepared["candidateImageId"]:
raise ValueError("Prepared image tag changed before candidate start")
self.runner(command + ["up", "--detach", "--no-build", "--pull", "never"],
cwd=cwd, env=environment, timeout=180)
container_id = self.canonical_id(self.plan["originalName"])
if not container_id:
raise ValueError("Candidate deployment produced no canonical container")
self.assert_candidate(container_id)
self.journal["candidateContainerId"] = container_id
self.save("deploying")
def assert_candidate(self, container_id):
values = inspect_fields(self.runner, container_id, [".Id", ".Image",
'index .Config.Labels "com.docker.compose.project"',
'index .Config.Labels "io.itworx.migration-attempt"'])
if (values[0] != container_id or container_id == self.prepared["originalContainerId"]
or values[2] != self.plan["candidateComposeProject"] or values[3] != self.journal["nonce"]):
raise ValueError("Refusing an unrelated candidate container")
recorded = self.journal.get("candidateContainerId")
if recorded and recorded != container_id:
raise ValueError("Recorded candidate container was replaced")
def http_health(self):
path = "http://127.0.0.1:1226/health/ready/" if self.slug == "vacatureradar" else "http://127.0.0.1:1223/health/ready"
with urlopen(path, timeout=10) as response:
if response.status != 200 or response.geturl() != path:
raise ValueError("Representative readiness probe failed")
def verifying(self):
self.verify_hold()
self.assert_candidate(self.journal["candidateContainerId"])
expected = sorted(self.plan["candidateMounts"], key=lambda item: item["destination"])
if mounts(self.runner, self.journal["candidateContainerId"]) != expected:
raise ValueError("Candidate mounts differ from the isolated cold-copy plan")
networks = inspect_fields(self.runner, self.journal["candidateContainerId"], [".NetworkSettings.Networks"])[0]
if set(networks) != {self.slug + "_default"}:
raise ValueError("Candidate runtime network differs from the original")
receipt = self.broker.verify_deployment(self.project, self.prepared["metadata"],
expected_image_ids={self.plan["originalName"]: self.prepared["candidateImageId"]})
self.broker.validate_recorded_receipt(self.project, receipt, self.repository, self.prepared["revision"])
self.http_health()
self.journal["receipt"] = receipt
self.save("verifying")
def committing(self):
self.broker.write_state(self.repository, self.project, self.prepared["revision"], "success",
"Controlled legacy migration passed immutable image and readiness verification",
live_receipt=self.journal["receipt"], attempt_metadata=self.prepared["metadata"])
# Never alter another project's configuration. Both deployment locks held.
current = self.broker.CONFIG_PATH.read_bytes()
if digest(current) != self.journal["originalConfigSha256"]:
raise ValueError("Broker configuration changed during cutover")
private_write(self.broker.CONFIG_PATH, (self.workspace / "candidate-projects.json").read_bytes())
self.verify_committed()
def verify_committed(self):
if digest(self.broker.CONFIG_PATH.read_bytes()) != self.journal["candidateConfigSha256"]:
raise ValueError("Committed broker configuration no longer matches")
receipt = self.broker.verified_current_receipt(self.repository, self.project,
self.prepared["revision"], self.prepared["metadata"]["tree"])
if receipt is None:
raise ValueError("Committed runtime receipt does not reverify")
self.http_health()
def recover(self):
if self.journal.get("phase") == "committed" or self.journal.get("activationIntent"):
raise ValueError("Committed activation must never roll back")
for filename, key in (("original-projects.json", "originalConfigSha256"),
("original-state.json", "originalStateSha256")):
if digest((self.workspace / filename).read_bytes()) != self.journal[key]:
raise ValueError("Protected rollback snapshot integrity failed")
config = self.broker.CONFIG_PATH.read_bytes()
if digest(config) not in {self.journal["originalConfigSha256"], self.journal["candidateConfigSha256"]}:
raise ValueError("Unrelated broker configuration change blocks automatic recovery")
state = self.state_path.read_bytes()
if digest(state) != self.journal["originalStateSha256"]:
payload = json.loads(state)
if payload.get("live_receipt") != self.journal.get("receipt") or not self.journal.get("receipt"):
raise ValueError("Unrelated deployment state blocks automatic recovery")
self.original()
if not self.journal.get("originalRestartIntent"):
for source, expected in self.journal.get("coldCopyChecksums", {}).items():
if digest(json.dumps(tree_manifest(source), sort_keys=True).encode()) != expected:
raise ValueError("Original data changed after verified cold copy")
container_id = self.canonical_id(self.plan["originalName"])
if container_id and container_id != self.prepared["originalContainerId"]:
self.assert_candidate(container_id)
# Persist discovered ownership before removing a crash-created container.
self.journal["candidateContainerId"] = container_id
self.save("recovering")
self.runner(["docker", "rm", "--force", container_id], timeout=90)
if self.canonical_id(self.plan["originalName"]) not in {None, self.prepared["originalContainerId"]}:
raise ValueError("Canonical name is occupied by unrelated runtime")
current_name = inspect_fields(self.runner, self.prepared["originalContainerId"], [".Name"])[0].lstrip("/")
if current_name not in {self.plan["originalName"], self.plan["retainedName"]}:
raise ValueError("Original runtime was renamed by another operator")
if current_name != self.plan["originalName"]:
self.runner(["docker", "rename", self.prepared["originalContainerId"], self.plan["originalName"]])
private_write(self.broker.CONFIG_PATH, (self.workspace / "original-projects.json").read_bytes())
private_write(self.state_path, (self.workspace / "original-state.json").read_bytes())
self.journal["originalRestartIntent"] = True
self.save("restoring_original")
if not self.original()["running"]:
self.runner(["docker", "start", self.prepared["originalContainerId"]], timeout=90)
deadline = time.monotonic() + 240
while time.monotonic() < deadline:
try:
self.original(healthy=True)
self.http_health()
return
except (ValueError, OSError):
time.sleep(3)
raise ValueError("Original runtime recovery health did not verify")
def private_read(path):
import os
import stat
info = path.lstat()
if not stat.S_ISREG(info.st_mode) or info.st_uid != os.geteuid() or stat.S_IMODE(info.st_mode) & 0o077:
raise ValueError("Expected a root-only regular transaction file")
return path.read_bytes()
def operate(broker, mode, slug, revision, attempt, container_id, image_id, runner=checked):
if mode not in {"cutover", "recover"} or slug not in TARGETS or not SHA.fullmatch(revision) or not ATTEMPT.fullmatch(attempt):
raise ValueError("Invalid fixed migration operation")
workspace = require_real_child(JOURNAL_ROOT / slug / attempt, JOURNAL_ROOT)
repository = f"jens/{slug}"
with ExitStack() as locks:
locks.enter_context(broker.exclusive_lock(broker.STATE_ROOT / "locks" / f"{slug}.lock", "migration project"))
locks.enter_context(broker.exclusive_lock(broker.GLOBAL_DEPLOY_LOCK, "migration production"))
prepared_bytes = private_read(workspace / "prepare.json")
prepared = json.loads(prepared_bytes)
if (prepared.get("phase") != "build_verified" or prepared.get("repository") != repository
or prepared.get("revision") != revision or prepared.get("originalContainerId") != container_id
or prepared.get("originalImageId") != image_id):
raise ValueError("Preparation identity does not match the requested operation")
journal_path = workspace / "cutover.json"
if mode == "recover":
journal = json.loads(private_read(journal_path))
if journal.get("preparedSha256") != digest(prepared_bytes):
raise ValueError("Prepared artifact changed after transaction snapshot")
ops = Production(broker, slug, prepared, workspace, journal, runner)
phase = recover_transaction(ops)
return {"phase": phase, "repository": repository, "candidateDataRetained": True}
if journal_path.exists():
raise ValueError("Transaction already exists; use explicit recover, never repeat cutover")
projects_bytes = broker.CONFIG_PATH.read_bytes()
projects = json.loads(projects_bytes)
original = projects[repository]
state_bytes = (broker.STATE_ROOT / f"{slug}.json").read_bytes()
if (digest(json.dumps(original, sort_keys=True).encode()) != prepared.get("projectConfigSha256")
or digest(state_bytes) != prepared.get("stateSha256")):
raise ValueError("Original broker config/state changed since preparation")
if broker.state_live_revision(json.loads(state_bytes)):
raise ValueError("This transaction only supports unattested legacy first deployments")
repo = broker.repository_path(repository, original)
broker.validate_commit(repo, str(original["branch"]), revision)
if broker.output(["git", f"--git-dir={repo}", "rev-parse", f"refs/heads/{original['branch']}"]) != revision:
raise ValueError("Prepared revision is no longer the configured branch head")
source = require_real_child(Path(prepared["source"]), broker.EXACT_SOURCE_ROOT)
if not broker.exact_source_matches_revision(repo, revision, source):
raise ValueError("Prepared source changed before cutover")
if verify_image(prepared["candidateImage"], prepared["metadata"], runner) != prepared["candidateImageId"]:
raise ValueError("Prepared immutable image changed")
assert_original(original_identity(container_id, runner), slug, container_id, image_id)
if mount_inventory(container_id, runner) != prepared["originalMounts"]:
raise ValueError("Original mounts changed after preparation")
if digest(private_read(workspace / "runtime.env")) != prepared["runtimeEnvSha256"]:
raise ValueError("Private runtime environment changed after preparation")
plan = build_plan(slug, revision, attempt, {"name": TARGETS[slug][0],
**original_identity(container_id, runner), "mounts": prepared["originalMounts"]})
if plan != prepared["plan"]:
raise ValueError("Cold-copy plan changed after preparation")
candidate = candidate_project(original, prepared, plan, prepared["nonce"], workspace / "runtime.env")
if candidate != prepared["candidateProject"]:
raise ValueError("Candidate project differs from the build configuration")
broker.validate_project_config(repo, candidate, revision)
render_contract(broker, candidate, prepared, plan, runner)
required = 8 * 1024**3
for item in plan["coldCopyRoots"]:
root = Path(item["original"])
if root.is_symlink() or root.resolve() != root or not root.is_dir():
raise ValueError("Cold-copy source is not an unchanged real directory")
required += sum(path.stat().st_size for path in root.rglob("*") if path.is_file())
if Path(item["candidate"]).exists():
raise ValueError("Cold-copy target already exists")
if shutil.disk_usage(workspace).free < required:
raise ValueError("Insufficient space for retained cold clones plus safety reserve")
updated = {**projects, repository: candidate}
candidate_bytes = json.dumps(updated, sort_keys=True).encode()
private_write(workspace / "original-projects.json", projects_bytes)
private_write(workspace / "original-state.json", state_bytes)
private_write(workspace / "candidate-projects.json", candidate_bytes)
# Configuration may contain secrets: stored privately, never in user output.
snapshot = inspect_fields(runner, container_id, [".Config", ".HostConfig", ".NetworkSettings.Networks"])
private_json(workspace / "original-container.json", snapshot)
journal = {"schema": "legacy-cutover.v1", "phase": "snapshotted", "plan": plan,
"preparedSha256": digest(prepared_bytes), "candidateProject": candidate,
"repositoryPath": str(repo), "nonce": prepared["nonce"], "originalMounts": prepared["originalMounts"],
"originalConfigSha256": digest(projects_bytes), "originalStateSha256": digest(state_bytes),
"candidateConfigSha256": digest(candidate_bytes)}
ops = Production(broker, slug, prepared, workspace, journal, runner)
ops.http_health()
if ops.canonical_id(plan["retainedName"]) or broker.compose_project_container_names(plan["candidateComposeProject"]):
raise ValueError("Attempt runtime/project is already occupied")
runner(["docker", "image", "tag", image_id, f"gitea-legacy-rollback/{slug}:{prepared['nonce']}"])
ops.save("snapshotted")
run_transaction(ops)
return {"phase": "committed", "repository": repository, "revision": revision,
"originalContainerRetained": True, "originalDataRetained": True,
"receiptVerified": True, "journal": str(journal_path)}