Jens/Unraid-AppOps-Gateway-Public
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Initial curated Unraid-AppOps-Gateway source from 3bd6f6fee310125bfbb4d4650389a817f498724d

Browse Source
This commit is contained in:
Public Source Publisher committed 2026-10-01 01:35:57 +02:00
commit 1dfbca08a1
528 files changed
+131054
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

CHANGELOG.md
+313
View File
@@ -0,0 +1,313 @@
# Changelog
## Unreleased
### Added
- A root-only local `--revalidate-delivery` controller operation can create one
additive, fresh exact-SHA validation attempt for an immutable
`validation_failed` or `validation_timeout` webhook record. It preserves the
original audit bytes, refuses other terminal states and changed policy or
branch head, exposes no HTTP mutation route, and rechecks exact branch head
immediately before deployment.
### Fixed
- The fixed two-runner Gitea Actions fleet now has a guarded, rollbackable
in-place upgrade from the affected 2.0.0 runner baseline to the immutable
3.5.0 image. Future fleet bootstrap also uses that reviewed pin; the upgrade
preserves separate runner credentials, labels and private-DinD topology and
recreates no production application, Gitea or DinD container.
- Gitea Actions scheduler recovery now proves it has a complete, bounded runner
inventory and fails closed on malformed, truncated or duplicate fixed-runner
state. Any disabled or busy registration vetoes recovery of that runner.
- Gitea Actions fleet recovery now treats the fixed private `gitea-runner-dind`
daemon as a bounded recovery target, persists `unless-stopped`, counts
start/restart attempts against one recovery budget and avoids querying DinD
networks while the daemon is down.
- A controller restart while a durable webhook delivery is `deploying` now
reconciles the exact repository/revision receipt. A verified receipt closes
the delivery as `deployed`; missing proof closes it as
`deployment_interrupted` without starting an unsafe duplicate deploy.
- Deployment-controller, enrollment and reconciliation writers now serialize
exclusively on the physical cache lock. This prevents an Unraid `shfs`
alias from making one process lock the same inode twice and deadlock itself.
- Controller promotion stops the old writer before acquiring the cache lock,
then holds that lock throughout backup, state reconciliation, controller
replacement and health verification.
## 1.8.0 - 2026-09-14
### Added
- Capability-based CI runner fleet telemetry, queue explanations and the
AppOps Runner Fleet view. The view joins Gitea scheduler facts with a
private, read-only controller projection; no Docker socket or token reaches
AppOps.
- A root-owned, fixed-identity CI fleet service with health, storage and
address-pool telemetry, bounded stale Gitea-task-network collection and a
three-attempt recovery budget per runner over 30 minutes.
- A rolling runner-fleet promotion procedure: two single-slot general
validation runners, separate workspaces, a private DinD daemon and a
`198.19.0.0/16` pool allocated as `/27` task networks (2,048 possible
networks).
### Changed
- Managed validation now asks for `linux-validation`; `unraid-deploy` is no
longer configured as an AppOps CI restart/deployment target.
- Automatic cache cleanup is restricted to dangling or expired cache inside
the dedicated CI daemon and only under its critical disk watermark. Generic
host Docker, network and volume prunes remain forbidden.
- The root-managed 30-minute exact-SHA validation window explicitly covers
scheduler queueing on the bounded two-slot CI fleet. A terminal delivery
record remains immutable; recovery always starts with a new protected
revision rather than bypassing the gate.
### Security
- CI code retains no production Docker socket or deployment credential. Safe
cleanup accepts only empty, older, unlabelled Gitea task networks in the
dedicated CI daemon; unknown, attached and production resources fail closed.
## 1.7.0 - 2026-09-14
### Added
- Runnerless Validated Delivery: signed Gitea main pushes are durably queued,
await explicit exact-SHA push validation, and call the existing receipt-bound
controller only after every required context succeeds.
- Durable `awaiting_validation`, validation failure/timeout and superseded
delivery states, bounded Gitea status polling, crash-safe resume and a
secret-free Delivery Reliability view in AppOps.
- Root-owned validation-token/config checks, controller backup manifests and
delivery inspection tooling.
### Changed
- Removed AppOps' `runs-on: unraid-deploy` production workflow. Managed
validation remains an ordinary `linux-validation` CI job and now runs for
protected-main pushes as well as pull requests.
### Security
- The validation credential stays controller-side, read-only and mode `0600`.
Exact repository identity, branch, SHA, HMAC and receipt checks remain
authoritative; missing policy/status/API reachability never deploys open.
## 1.6.0 - 2026-09-14
### Added
- Enrollment & Provisioning Plane v1 with strict `EnrollmentPlanV1`, trusted
server-side profiles, durable enrollment jobs, machine-readable readiness,
typed operator APIs and typed MCP tools.
- Controller bootstrap that registers one validated target through HMAC, creates
its bounded appdata layout and per-target credential, and seals only
profile-owned runtime values.
- Exact-target `deployment.apply` and `deployment.rollback` policies with
enrollment provenance; policy defaults remain deny-by-default.
### Security
- New enrollments refuse container, hostname, Authentik and storage takeover;
no arbitrary Compose, environment, callback, Docker, Portainer-exec or shell
capability was added.
- Enrollment credentials and OIDC client secrets are generated server-side,
encrypted at rest, write-only, and excluded from evidence, UI and MCP.
- Failed enrollment compensates only resources that the same execution proved
it created; incomplete compensation stays explicitly visible.
- Controller HMAC requests now bind exact method and endpoint, use a
single-use durable nonce, and fail closed for the legacy request format.
- Enrollment rejects the development secret-store fallback, mismatched OIDC
role claims/models, unsafe controller registries and unverified NPM ownership
markers.
- Changed local fallback operator authentication from reload-fragile frontend memory to a revocable, process-local HttpOnly browser session. Raw operator keys are exchanged once, never stored in JavaScript/browser storage, and cookie mutations require an explicit CSRF header.
- Fixed managed deployment provenance by forwarding the broker's exact revision, repository, tree,
build ID and build time into immutable OCI image labels while retaining the application version
in a separate vendor label.
## 1.5.2 - 2026-08-27
- Measure active Gitea Actions age from `started_at`, so an old queued job that
just acquired a runner is no longer reported as a stale active execution.
## 1.5.1 - 2026-08-27
### Fixed
- Canonical Compose now binds the persistent `/data/secret-key` through
`APPOPS_SECRET_KEY_FILE`, keeping encrypted secret storage production-ready
after the isolated data-mount migration.
- The MCP handshake now derives its advertised version from the same central
application version used by the API and runtime status.
## 1.5.0 - 2026-08-27
### Added
- Cached, read-only Gitea Actions runner and queue observability in the Overview, private API and MCP surface.
- Explicit runner-label-to-container guidance for stalled queues; restart remains an approval-gated Command Center action and never executes automatically.
### Changed
- SBOM generation now uses pnpm 11's native CycloneDX 1.6 command, removing the incompatible npm-specific generator and its transitive dependency tree.
- Canonical Compose now mounts the dedicated `APPOPS_DATA_PATH` instead of using the Git checkout as `/data`; Gitea observer configuration references only the mounted token file.
### Security
- Gitea admin credentials support mounted secret files and are never returned by status, UI or MCP responses.
- Fixed Gitea admin endpoints, bounded job samples and a cache that clients cannot bypass prevent generic admin access and polling amplification.
## Unreleased - 2026-08-11
### Added
- Governed MCP server operations: ChatGPT can request an exact typed action, follow approval/job state, execute only an operator-approved fingerprint and cancel a queued/running job record.
- Server-side execution concurrency is capped at one operation; work and post-checks run in AppOps rather than in the browser.
- Private MCP 2.0 Streamable HTTP server with authenticated read-only AppOps tools and a capability resource that explicitly excludes mutation, shell and Docker socket access.
- Persistent local, extractive RAG retrieval with citations across allowlisted documentation and redacted live AppOps context; AI Chat and MCP share the retrieval service.
- Dedicated n8n machine authentication plus cursor events, policy preflight, idempotent approval requests and approval/job status reads. No n8n execution endpoint is exposed.
- n8n can query the shared local RAG service with citations through its dedicated machine-authenticated integration path.
- An `/integrations` cockpit for readiness, endpoint discovery, RAG indexing and cited query verification.
### Security
- Every MCP mutation requires a separate expiring human approval even when the ordinary manual policy would allow it. MCP cannot approve requests, change the approved context, invoke an unknown action, open a shell or access the Docker socket.
- MCP, n8n, operator and voice identities remain separate. Production bootstraps distinct mode-`0600` MCP and n8n token files and never exposes their values in the UI or status API.
- Hardened the private machine interfaces with recursive cross-boundary redaction, pre-parse body limits, n8n middleware authentication and mutation throttling, SQL-bounded event cursors, serialized idempotency and bounded reservation retention.
- Destructive action defaults remain authoritative; automation can request governance but cannot approve or execute actions.
### Fixed
- The integrations cockpit no longer reports healthy MCP/n8n services as stopped or RAG as empty when an operator tab is locked; unavailable state is explicit and protected controls remain disabled until re-authentication.
- Unraid parity monitoring now uses the live `array.parityCheckStatus` and `array.parities` GraphQL fields instead of unsupported `parityStatus`/`system` queries.
- The parity endpoint distinguishes an active or paused check from healthy and unhealthy parity disks and exposes progress, speed, correction mode and sync-error metadata.
- The Unraid connector safely follows the appliance's same-host HTTP-to-HTTPS API redirect while retaining the API key; redirects to other hosts or non-HTTPS targets are refused.
## 1.3.0 - 2026-07-29
### Added
- Private operator authentication for all mutations and sensitive administrative reads, with an in-memory-only frontend unlock session.
- Persistent server-side operator-token bootstrap through a mode-`0600` file in `/data`, avoiding any workstation runtime dependency.
- Approval fingerprints and actor attribution, centralized audit redaction, connector URL SSRF controls and strict high-risk request contracts.
- Backup checksums, retention, free-space protection and disposable restore verification.
- CI gates for frontend/backend tests, safety validation, container smoke testing and a CycloneDX 1.6 SBOM artifact.
### Changed
- TLS verification is secure by default and production secret/token strength is enforced.
- Mutation requests have bounded bodies and rate limiting; production images require operator authentication.
- Responsive dashboard layouts, dependency compatibility and repository hygiene were hardened.
### Security
- The private deployment model explicitly separates operator and voice credentials and keeps destructive actions blocked by policy.
- Tracked test databases and the duplicate Dockerfile were removed; the root Dockerfile is canonical.
### Verified
- Backend regression suite: `381 passed`; 28 remaining warnings are deprecated timestamp calls in test fixtures.
- Frontend type-check, lint, all `243` tests and the Vite production build are green.
- Ruff, dependency audits, diff validation, secret-pattern scanning, CycloneDX SBOM generation and private safety validation are green.
- Docker image verification remains delegated to CI/a Docker-capable private host because Docker is unavailable on the current workstation.
## Unreleased - 2026-07-19
### Added
- Guarded Unraid VM start/resume and approval-aware pause with live state prechecks, JobRuns, post-checks and Audit; destructive stop/reboot/reset/force-stop remain blocked.
- Live Home Assistant/Gemini Assist readiness for API reachability, installed AppOps scripts and loaded conversation components.
- Read-only re-verification and `no-store` download of existing SQLite backups.
### Changed
- Authentik outpost status now distinguishes API-reported health from credential-free reachability and never forwards the Authentik bearer token to the configured outpost origin.
- Live audit copy now distinguishes saved Voice bridge configuration from actual Home Assistant reachability, and the route smoke explicitly covers the VM manager surface.
- Vite and the ESLint toolchain now use patched compatible releases; the remaining transitive `flatted` advisory is resolved with a scoped workspace override, and pnpm is pinned for reproducible local/container builds.
- The production container is now presented as `Unraid-AppOps-Gateway`, with a stable hostname, OCI metadata, Unraid WebUI/icon labels and matching SVG/PNG/ICO/PWA assets. The existing `/srv/example/appdata/unraid-appops-gateway-1216` data path remains unchanged.
- Obsolete admin-token examples and the unused browser voice-token bridge are removed; `APP_OPS_VOICE_TOKEN` remains server-side and is only supplied by Home Assistant to protected voice endpoints.
- Patched compatible transitive versions are pinned for the dependency-audit findings in `lodash`, `glob`, `minimatch`, `brace-expansion`, `picomatch`, `postcss` and `yaml`.
- Backend security pins move to FastAPI 0.139.0/Starlette 1.3.1, cryptography 48.0.1 and pytest 9.0.3 to clear the Python dependency audit.
- Active setup, deployment, API and media-pipeline documentation now consistently describes the live-only frontend instead of the removed demo/mock runtime.
- Canonical upgrade post-checks allow 30 seconds for the one-time cold migration read, preventing a false rollback while an oversized legacy inventory is being pruned.
- The one-time container migration strips obsolete admin/browser token variables while preserving all current connector, Unraid, Portainer and Home Assistant voice configuration.
- Upgrade post-check failures now propagate correctly through Bash conditionals and retry three times instead of logging a timed-out endpoint as successful.
- Production discovery retains detailed rows for the newest 48 successful runs. Unchanged missing containers are recorded once instead of being copied into every run, preventing unbounded SQLite growth and multi-second setup/runtime reads.
- Synchronous runtime and setup status aggregation now runs in FastAPI's worker pool instead of blocking the async event loop during cold inventory reads.
- The alternate Dockerfile now defaults the frontend to live API mode, matching the root production Dockerfile. Compose requires externally supplied voice and encryption keys instead of runnable placeholder secrets.
- Inventory discovery reconciles existing same-name/same-app mappings when Docker recreates a container with a new ID, removes duplicate stale mappings and writes a metadata-only audit event. Genuine unknown containers remain in review.
- The latest-discovery cache now uses weak Session object keys instead of reusable numeric object IDs, preventing detached ORM rows from leaking into later requests.
- Launchpad and Media Codec Dashboard receive evidence-based classifications from their live names/images instead of remaining generic unknowns.
- Action Registry renders 30 rows per page with filter-aware pagination instead of mounting all 181 actions at once.
- Exposure rows now include concrete risk evidence such as missing Authentik/access-list coverage, missing certificates and non-forced SSL; high-risk rows sort first in the UI.
- Inventory change detection canonicalizes Portainer port, mount and network ordering, so identical runtime snapshots no longer reopen reviewed apps as false drift. Historical missing containers remain visible in the ledger but are no longer presented as bulk import/edit work.
- Setup status now coalesces discovery history by current container name, preventing recreated Docker IDs from inflating discovered/review counts. `needsReview` reports the actual current workload while `reviewBlockers` separately indicates whether that workload still blocks first-run completion.
- Completed setup labels the accepted first-run review step as `Reviewbaseline vastgelegd`; later/current review workload is shown separately as `open inventory review`.
- Apps without a registered AppOps connector now report connector status `none`; Jellyfin no longer advertises missing connector configuration or an unavailable scan executor.
- Action Registry responses now distinguish real Command Center execution, approval execution, dedicated surfaces, governance-only actions and blocked actions.
- Approval decisions use the correct `PATCH` API and a strict persisted state machine. Approved Deluge pause-all/resume-all and Plex scan-all requests can be executed separately through audited JobRuns; unsupported approvals stay non-executable.
- Overview and TopStatusBar use shared React Query caching with independent fail-soft polling intervals, focus refresh and no background-tab polling.
- Inventory review now coalesces recreated Docker container IDs by unique container name, preferring the current non-missing instance while retaining per-run history for audit and drift. The review grid renders 24 cards per page instead of mounting the complete ledger at once.
- Overview action cards use a balanced two-column desktop layout before expanding to four columns on ultrawide screens.
- Overview starts its primary runtime, app, activity, chart and storage snapshot in parallel with secondary Safety/setup/notification intelligence, but no longer blocks first paint on the connector-heavy secondary reads.
- Inventory latest-state and drift fallback queries now use indexed SQLite window functions, the mapping UI loads the compact canonical app catalog, and no-op startup classification avoids full history scans.
- FastAPI database initialization now uses the supported application lifespan hook.
- Backend persistence timestamps now use one Python-forward-compatible UTC helper while preserving existing SQLite semantics; audit listing no longer uses SQLAlchemy's legacy query API.
- Successful canonical upgrades retain the newest 20 timestamped backups and remove only scoped legacy AppOps image tags after health validation.
- Canonical upgrades automatically migrate the former `unraid-appops-gateway-1216` name to `Unraid-AppOps-Gateway`, roll back from the previous immutable image ID and never retain versioned `pre`/`failed` containers. Production uses the stable `unraid-appops-gateway:production` tag with one `rollback` image tag.
- Runtime and OpenAPI version reporting now share release version `1.2.1`.
- Safety refreshes expired persistent snapshots before falling back to stale data.
- Notification evaluation prefers the most specific matching rule and retires exact duplicates plus generic media umbrella overlaps from the active inbox.
- Conservative app classification patterns reduce canonical `Unknown` noise without overriding explicit categories.
- Canonical classification now uses app IDs, mapped container names and discovered images, and normalizes humanized names consistently.
- Historical AppOps rollback registrations and the superseded unqualified deployment record no longer pollute active app, inventory, setup, operations-map, Safety or AI views; history remains stored.
- AI Security's unavailable Media Pipeline fallback now captures and redacts its initialization error safely instead of retaining an expired exception reference.
- The secondary top-bar activity chip now waits for a true wide-desktop viewport, preventing it from colliding with infrastructure status around 1280px.
- AI Security excludes current and rollback AppOps deployment containers from discovery-review findings and invalidates pre-fix cached reviews.
- Managed canonical apps can now be normalized from App Detail through an audited metadata-only editor for name, category and criticality.
- Locally documented custom apps now receive evidence-based categories and protection defaults; canonical `Unknown` coverage reaches zero without generic guessing.
- App category filters are generated from the live catalog, and App Detail derives credential-free Open links from configured connector URLs or mapped published ports.
- Inventory baseline updates now persist the current live ledger, reset stale acknowledgements, drive subsequent drift comparisons and write a real audit event instead of returning a placeholder success.
- Discovery's short-lived latest-container cache is scoped to the active database session so ORM rows cannot leak across requests as detached or stale runtime data.
- The configured Home Assistant notification channel was validated end to end against the live API and a delivered test notification; the existing Home Assistant container now uses `unless-stopped` restart policy.
- Overview formats Unraid boot timestamps as compact uptime durations so Server Health remains readable at five-column desktop widths.
- Inventory loads runs, review records, apps, mappings and drift independently; a slow source now produces a partial warning without clearing successful live ledger data or fabricating an empty-discovery state.
- Large inventory ledger reads use a dedicated 30-second request budget, and the empty-discovery prompt only appears after review records successfully load as empty.
- The read-only container list also uses a 30-second cold-start budget, keeping Inventory's optional runtime comparison from timing out behind simultaneous ledger reads.
- Exposure Center and Containers now progressively enrich canonical live data with optional connector details and Portainer telemetry.
- Exposure Center no longer flashes false `0` or `Unknown` detail metrics while NPM and Authentik enrichment is still loading; canonical exposure counts remain available immediately and optional values are labelled as loading or unavailable.
- Authentik outposts that exist but provide no runtime health field are labelled `Unverified` instead of being overstated as unavailable.
- Overview quick actions use a two-column mobile grid and expand to a wrapped desktop row, preventing clipped controls on narrow screens.
### Verified
- Backend regression suite: `378 passed`; 28 remaining warnings are deprecated timestamp calls in test fixtures.
- Frontend type-check, lint and Vite production build are green on the patched toolchain.
- Ruff, Python compileall, diff validation and tracked secret-pattern scanning are green.
- Python dependency audit reports no known vulnerabilities; pnpm reports no high-severity advisories.
## 1.2.1 - 2026-07-02
### Added
- SQLite-safe online backup flow through `POST /api/admin/backups/sqlite` and Settings > Backups.
- Server-side backup listing for `/data/backups/appops-db-*.db`.
- Backup audit event `runtime.sqlite_backup` with integrity-check metadata and no secret values.
- Root `SECURITY.md` with no-secrets, local-first and exposure guidance.
### Changed
- `stack.redeploy` is documented as approval-only V1 behavior.
- Connector binding lookup now computes latest discovery state once per request, fixing live `/connectors` loading timeouts.
- Legacy AppOps deployment artifacts are filtered from operator inventory lists.
- Current roadmap and implementation status now distinguish active 1216 production status from historical validation notes.
### Removed
- Un-routed legacy AI pages with static/demo controls under `src/pages/ai`.
### Verified
- Live deployment on canonical `http://198.18.10.150:1216`.
- `scripts/validate_live_deployment.ps1` result: `PASS=39 WARN=1 FAIL=0`.
- Production container image: `unraid-appops-gateway:connector-audit-34cc5a6-1216`.
## 1.2.0 - 2026-07-02
### Added
- Live route hardening for AppDetail, Apps, Containers, Stacks and Overview action controls.
- Policy action fallback handling so missing handlers no longer silently no-op.
- Active roadmap source at `docs/ROADMAP_CURRENT.md`.
### Changed
- App lifecycle controls now route to real restart, approval or review flows.
- Read-only or unsupported actions are explicitly disabled in V1.
## Historical Notes
Detailed sprint history, validation snapshots and superseded deployment notes remain in `docs/IMPLEMENTATION_STATUS.md` and `docs/LIVE_VALIDATION_REPORT.md`. Those files are archives unless repeated in `docs/ROADMAP_CURRENT.md` or this changelog.