ModelForge-Public/docs/CONFIGURATION.md

Configuration reference

Generated from the typed settings by scripts/generate_configuration_docs.py for ITWorx ModelForge 1.3.0-rc.1. Every setting the control plane reads appears here; a setting added without documentation fails the build.

All settings are environment variables with the MODELFORGE_ prefix, read from the process environment or from .env.

Required in production

MODELFORGE_ENV=production turns on fail-closed startup validation. With it set, the control plane refuses to start unless each of these is present and sound:

• MODELFORGE_DATABASE_URL — SQLAlchemy URL for the API's non-owner modelforge_runtime role. It may read the audit trail and execute the canonical append function, but cannot mutate audit tables directly.
• MODELFORGE_REDIS_URL — Redis URL for transient request payloads and queues.
• MODELFORGE_BACKUP_ENCRYPTION_KEY — Base64 AES-256 key for backup encryption. Without it no backup can be produced, and without the same key no backup can be restored — store it outside this deployment.

Production additionally refuses: a well-known development database password, a wildcard CORS origin, remote model code execution, an unwritable storage root, an unsupported schema revision, a PostgreSQL major below 16, and three policy combinations that cannot all hold at once.

Generating secrets

ModelForge never mints its own credentials — a platform that generates its own admin secret has no way to tell you it did. Generate them yourself and store them outside the deployment:

# Operator API key (at least 32 characters)
python -c "import secrets; print(secrets.token_urlsafe(48))"

# Backup encryption key (base64 AES-256)
python -c "import base64, os; print(base64.b64encode(os.urandom(32)).decode())"

# Database password
python -c "import secrets; print(secrets.token_urlsafe(32))"

Losing the backup encryption key makes every existing backup unrecoverable. It is the one value that must be stored somewhere the deployment cannot take down with it.

Sensitivity

9 settings are credentials. They are never logged, never written to a release artefact and never echoed in an error response:

• MODELFORGE_DATABASE_URL
• MODELFORGE_MIGRATION_DATABASE_URL
• MODELFORGE_OPERATOR_IDENTITIES
• MODELFORGE_OPERATOR_PASSWORD_HASH
• MODELFORGE_OPERATOR_API_KEY
• MODELFORGE_HF_TOKEN
• MODELFORGE_SECRET_SLOT_ROOT_KEY
• MODELFORGE_BACKUP_ENCRYPTION_KEY
• MODELFORGE_ALERT_WEBHOOK_URL

Every setting

Variable	Type	Default	Required	Sensitivity	Description
MODELFORGE_ENV	development, test, production	development	no	public	Deployment profile. 'production' turns on every fail-closed startup rule; 'development' and 'test' report the same problems without refusing to start.
MODELFORGE_API_HOST	string	0.0.0.0	no	public	Interface the API binds inside its container. Leave at 0.0.0.0.
MODELFORGE_API_PORT	integer	8000	no	public	Port the API listens on inside its container.
MODELFORGE_CONTROL_PLANE_MAX_PAYLOAD_BYTES	integer	1048576	no	public	Pre-parser request-body limit for public and operator control-plane routes.
MODELFORGE_SERVICE_NAME	string	modelforge-api	no	public	Name this process reports in logs and audit events.
MODELFORGE_LOG_LEVEL	string	INFO	no	public	Structured log level: DEBUG, INFO, WARNING or ERROR.
MODELFORGE_DATABASE_URL	string	—	yes	secret	SQLAlchemy URL for the API's non-owner modelforge_runtime role. It may read the audit trail and execute the canonical append function, but cannot mutate audit tables directly.
MODELFORGE_MIGRATION_DATABASE_URL	secret, optional	—	no	secret	SQLAlchemy URL for the non-superuser modelforge schema-owner role. Set only in the one-shot migration process; production API startup refuses when this secret is present.
MODELFORGE_REDIS_URL	string	redis://localhost:6379/0	yes	internal	Redis URL for transient request payloads and queues.
MODELFORGE_CORS_ORIGINS	string	http://localhost:3000	no	public	Comma-separated exact origins allowed to call the API from a browser. A wildcard is refused in production because requests are credentialed.
MODELFORGE_OPERATOR_IDENTITIES	secret, optional	—	no	secret	Optional JSON array of named human identities with role, credential ID, SHA-256 key digest and timezone-aware expiry. Provision with scripts/operator_identity.py. Replace this deployment setting and recreate the API to rotate or revoke access.
MODELFORGE_OPERATOR_PASSWORD_HASH	secret, optional	—	no	secret	Optional salted scrypt verifier generated by scripts/operator_password.py. Enables password login with 15-minute sessions; requires the separate strong operator API key. Changing either value invalidates password sessions.
MODELFORGE_OPERATOR_API_KEY	secret, optional	—	no	secret	Legacy bootstrap administrator key. Generate at least 32 random characters. Production requires this key or an unexpired named administrator in operator_identities.
MODELFORGE_HF_TOKEN	secret, optional	—	no	secret	Optional deployment Hugging Face token for control-plane metadata. Compose may also provision it independently to node-agent downloads; API secret activation does not update nodes.
MODELFORGE_SECRET_SLOT_ROOT_KEY	secret, optional	—	no	secret	External base64 32-byte AES-256 key for the managed Hugging Face control-plane slot. Keep independent of backup and operator keys; retain it externally for historical restore.
MODELFORGE_SECRET_SLOT_ROOT_KEY_ID	string	hf-control-plane-1	no	public	External key identifier bound to encrypted HF slot versions. Replacing a key does not rewrap history.
MODELFORGE_BACKUP_ENCRYPTION_KEY	secret, optional	—	yes	secret	Base64 AES-256 key for backup encryption. Without it no backup can be produced, and without the same key no backup can be restored — store it outside this deployment.
MODELFORGE_BACKUP_ENCRYPTION_KEY_ID	string	modelforge-backup-key-1	no	public	Identifier recorded in each backup manifest so a restore can name the key it needs.
MODELFORGE_HF_HOME	string	/data/hf-cache	no	public	Hugging Face cache root inside the container.
MODELFORGE_ARTIFACT_ROOT	string	/data/artifacts	no	public	Verified model artifact root. Must exist and be writable.
MODELFORGE_QUARANTINE_ROOT	string	/data/quarantine	no	public	Where acquired artifacts are held until their checks pass.
MODELFORGE_RUNTIME_ARTIFACT_ROOT	string	/models/model-registry	no	public	Artifact root as a runtime worker sees it on a compute node.
MODELFORGE_CONFIG_ROOT	path	PydanticUndefined	no	public	Directory holding the capability, project and policy manifests.
MODELFORGE_BACKUP_ROOT	path	/data/backups	no	public	Backup destination. Must exist and be writable, or backups fail closed.
MODELFORGE_BACKUP_RESTORE_ROOT	path	/data/restore	no	public	Working directory a restore stages into before it commits.
MODELFORGE_ALEMBIC_DIRECTORY	path, optional	—	no	public	Override for the migration directory. Leave empty in a container.
MODELFORGE_HF_TIMEOUT_SECONDS	number	30	no	public	Per-request timeout for Hugging Face metadata calls.
MODELFORGE_HF_SNAPSHOT_TTL_SECONDS	integer	3600	no	public	How long a resolved upstream snapshot stays cached.
MODELFORGE_ALLOW_REMOTE_CODE	boolean	false	no	public	Whether model repositories may execute their own Python. Always false in production; startup refuses any other value there.
MODELFORGE_ENABLE_GPU_TELEMETRY	boolean	true	no	public	Collect GPU telemetry on this host.
MODELFORGE_HARDWARE_REFRESH_ON_STARTUP	boolean	false	no	public	Run a hardware inventory pass when the process starts.
MODELFORGE_HARDWARE_POLL_INTERVAL_SECONDS	integer	30	no	public	Interval between hardware inventory passes.
MODELFORGE_NODE_IDENTITY	string, optional	—	no	internal	Explicit node identity. Leave empty to use the persisted file.
MODELFORGE_NODE_IDENTITY_MODE	auto, persisted	auto	no	public	'persisted' keeps a node's identity across restarts; 'auto' derives it.
MODELFORGE_NODE_IDENTITY_FILE	path	/data/state/node-id	no	public	Where a persisted node identity is stored.
MODELFORGE_NODE_STALE_AFTER_SECONDS	integer	30	no	public	Silence after which a node is considered stale.
MODELFORGE_NODE_OFFLINE_AFTER_SECONDS	integer	90	no	public	Silence after which a node is considered offline. Must exceed the stale threshold.
MODELFORGE_LIVENESS_POLL_INTERVAL_SECONDS	integer	5	no	public	How often node liveness is re-evaluated.
MODELFORGE_AGENT_MAX_CLOCK_SKEW_SECONDS	integer	300	no	public	Clock skew tolerated on an agent report before refusal.
MODELFORGE_NODE_AGENT_MAX_PAYLOAD_BYTES	integer	4194304	no	public	Pre-parser request-body limit for enrollment and authenticated Node Agent reports.
MODELFORGE_NODE_LIVENESS_MONITOR_ENABLED	boolean	false	no	public	Run the node liveness monitor in this process.
MODELFORGE_AGENT_PROTOCOL_VERSION	integer	1	no	public	Agent protocol version this control plane speaks.
MODELFORGE_GATEWAY_MAX_BATCH_SIZE	integer	8	no	public	Maximum inputs accepted in a single capability invocation.
MODELFORGE_GATEWAY_MAX_INPUT_CHARACTERS	integer	8192	no	public	Maximum characters per input item.
MODELFORGE_GATEWAY_MAX_PAYLOAD_BYTES	integer	65536	no	public	Maximum accepted request body size.
MODELFORGE_GATEWAY_REQUEST_TIMEOUT_SECONDS	integer	45	no	public	Total time a capability invocation may take. Must exceed the queue timeout.
MODELFORGE_GATEWAY_QUEUE_TIMEOUT_SECONDS	integer	30	no	public	How long a request may wait for capacity before rejection.
MODELFORGE_SERVING_JOB_LEASE_SECONDS	integer	120	no	public	Lease held by a serving job before it is reclaimed.
MODELFORGE_SERVING_PAYLOAD_TTL_SECONDS	integer	120	no	public	How long a request payload survives in Redis.
MODELFORGE_SCHEDULER_SAFETY_RESERVE_BYTES	integer	1073741824	no	public	VRAM never offered to a placement, as an absolute floor.
MODELFORGE_SCHEDULER_SAFETY_RESERVE_PERCENTAGE	number	0.05	no	public	VRAM never offered to a placement, as a fraction. Half a device leaves nothing schedulable.
MODELFORGE_SCHEDULER_RUNTIME_MARGIN_BYTES	integer	268435456	no	public	Headroom reserved for runtime overhead per node.
MODELFORGE_SCHEDULER_DEPLOYMENT_MARGIN_BYTES	integer	134217728	no	public	Absolute headroom added to each deployment estimate.
MODELFORGE_SCHEDULER_DEPLOYMENT_MARGIN_PERCENTAGE	number	0.1	no	public	Proportional headroom added to each estimate.
MODELFORGE_SCHEDULER_GLOBAL_QUEUE_LIMIT	integer	128	no	public	Queued requests accepted before capacity rejection begins.
MODELFORGE_SCHEDULER_TELEMETRY_STALE_SECONDS	integer	90	no	public	Telemetry age past which admission is blocked rather than extrapolated.
MODELFORGE_SCHEDULER_PRESSURE_STABLE_SECONDS	integer	30	no	public	How long pressure must hold before the state changes.
MODELFORGE_SCHEDULER_EVICTION_COOLDOWN_SECONDS	integer	60	no	public	Minimum interval between evictions on a node.
MODELFORGE_SCHEDULER_PLACEMENT_HISTORY_LIMIT	integer	500	no	public	Placement decisions retained for inspection.
MODELFORGE_REGISTRY_SEED_ON_STARTUP	boolean	false	no	public	Seed the candidate and project registries from manifests.
MODELFORGE_SERVING_RECONCILIATION_ENABLED	boolean	false	no	public	Reconcile abandoned serving work in this process.
MODELFORGE_SERVING_RECONCILIATION_INTERVAL_SECONDS	integer	5	no	public	Interval between serving reconciliation passes.
MODELFORGE_LIFECYCLE_RECONCILIATION_ENABLED	boolean	false	no	public	Roll back incomplete lifecycle operations at startup.
MODELFORGE_MIGRATION_RECONCILIATION_ENABLED	boolean	false	no	public	Report interrupted migration cutovers at startup. They are never auto-resolved: external alias truth cannot be inferred after a crash.
MODELFORGE_OBSERVABILITY_MONITOR_ENABLED	boolean	false	no	public	Run SLO and alert evaluation in this process.
MODELFORGE_OBSERVABILITY_POLL_INTERVAL_SECONDS	integer	60	no	public	Interval between observability evaluation passes.
MODELFORGE_ALERT_NOTIFICATIONS_ENABLED	boolean	false	no	public	Queue and dispatch the deployment-managed operational alert webhook.
MODELFORGE_ALERT_WEBHOOK_URL	secret, optional	—	no	secret	Deployment-managed HTTPS notification destination. Its exact hostname must also appear in alert_webhook_allowed_hosts; the URL is never returned by the API.
MODELFORGE_ALERT_WEBHOOK_ALLOWED_HOSTS	string	—	no	public	Comma-separated exact destination host allowlist for the alert webhook. Wildcards and redirects are not accepted.
MODELFORGE_ALERT_WEBHOOK_MIN_SEVERITY	INFO, WARNING, CRITICAL	WARNING	no	public	Minimum alert severity sent to the single webhook route: INFO, WARNING or CRITICAL.
MODELFORGE_ALERT_NOTIFICATION_TIMEOUT_SECONDS	number	5.0	no	public	Total timeout for one webhook delivery.
MODELFORGE_ALERT_NOTIFICATION_MAX_ATTEMPTS	integer	8	no	public	Maximum webhook attempts before a delivery fails permanently.
MODELFORGE_ALERT_NOTIFICATION_BATCH_SIZE	integer	20	no	public	Maximum outbox rows claimed by one notification dispatch pass.
MODELFORGE_ALERT_NOTIFICATION_LEASE_SECONDS	integer	60	no	public	Lease duration before an interrupted delivery can be reclaimed.
MODELFORGE_ALERT_NOTIFICATION_RETRY_BASE_SECONDS	integer	15	no	public	Base delay for bounded exponential webhook retries.
MODELFORGE_MODEL_UPDATE_MONITOR_ENABLED	boolean	true	no	public	In production, check explicitly followed model references for changed revisions. Metadata only; never downloads, tests or promotes a model.
MODELFORGE_MODEL_UPDATE_CHECK_INTERVAL_SECONDS	integer	86400	no	public	Minimum interval between automatic checks of each followed model reference; default one day. Each API process checks at most one due reference per minute.
MODELFORGE_RECOVERY_RECONCILIATION_ENABLED	boolean	false	no	public	Reconcile interrupted backups and restores at startup.
MODELFORGE_BACKUP_PG_DUMP_PATH	string	pg_dump	no	public	pg_dump executable. Must match the server major version.
MODELFORGE_BACKUP_PG_RESTORE_PATH	string	pg_restore	no	public	pg_restore executable.
MODELFORGE_BACKUP_PSQL_PATH	string	psql	no	public	psql executable.
MODELFORGE_BACKUP_COMMAND_TIMEOUT_SECONDS	integer	1800	no	public	Timeout for a dump or restore command.
MODELFORGE_BACKUP_STALE_AFTER_SECONDS	integer	93600	no	public	Age past which the newest verified backup raises BACKUP_STALE.
MODELFORGE_BACKUP_MINIMUM_FREE_BYTES	integer	1073741824	no	public	Free space below which a backup refuses to start.
MODELFORGE_BACKUP_CAPACITY_HEADROOM_RATIO	number	3.0	no	public	Required free space as a multiple of the estimated size.
MODELFORGE_RESTORE_ALLOW_PRODUCTION_TARGET	boolean	false	no	public	Whether a restore may overwrite the live database. Keep false outside a rehearsal.
MODELFORGE_BUILD_COMMIT	string, optional	—	no	public	Source commit stamped into the image at build time. Never set by hand.
MODELFORGE_BUILD_TIMESTAMP	string, optional	—	no	public	Build time stamped into the image. Never set by hand.
MODELFORGE_BUILD_IMAGE_DIGEST	string, optional	—	no	public	Image digest recorded at deployment. Never set by hand.

Deployment variables

Read by Compose, the Node Agent and the Runtime Worker rather than by the control-plane process. An operator still has to set them, so they are documented here too.

Variable	Required	Sensitivity	Description
MODELFORGE_POSTGRES_BIND	no	public	Host address the control-plane database is published on. Defaults to 127.0.0.1; publishing it more widely exposes provenance, credential hashes and the audit trail.
MODELFORGE_REDIS_BIND	no	public	Host address Redis is published on. Defaults to 127.0.0.1.
MODELFORGE_API_BIND	no	public	Host address the API is published on. Defaults to 0.0.0.0 deliberately: the console and compute nodes need it, and every admin route is operator-authenticated.
MODELFORGE_WEB_BIND	no	public	Host address the console is published on. Defaults to 127.0.0.1.
MODELFORGE_POSTGRES_PORT	no	public	Host port the database is published on. Defaults to 5432.
MODELFORGE_REDIS_PORT	no	public	Host port Redis is published on. Defaults to 6379.
MODELFORGE_API_PUBLISHED_PORT	no	public	Host port the API is published on. Defaults to 8000.
MODELFORGE_WEB_PORT	no	public	Host port the console is published on. Defaults to 3000.
MODELFORGE_DR_POSTGRES_BIND	no	public	Host address for the DR rehearsal database. Loopback only.
MODELFORGE_DR_API_BIND	no	public	Host address for the DR rehearsal API. Loopback only.
VITE_API_BASE_URL	no	public	API base URL compiled into the console. Vite inlines it at build time, so changing it requires rebuilding the console image, not restarting it.
MODELFORGE_POSTGRES_DB	no	public	Production database name. Required by the production overlay.
MODELFORGE_POSTGRES_ADMIN_USER	no	internal	Bootstrap/admin role used only by PostgreSQL provisioning; defaults to postgres.
MODELFORGE_POSTGRES_ADMIN_PASSWORD	yes	secret	Bootstrap/admin password; never passed to the migration or API container.
MODELFORGE_MIGRATION_DB_PASSWORD	yes	secret	Raw password supplied to provisioning for the non-superuser modelforge owner role.
MODELFORGE_RUNTIME_DB_PASSWORD	yes	secret	Raw password supplied to provisioning for the non-owner modelforge_runtime role.
MODELFORGE_RUNTIME_DATABASE_URL	yes	secret	Non-owner runtime-role URL passed only to the API container.
MODELFORGE_VERSION	no	public	Exact version tag applied to built images and required when the production overlay is not given explicit API and web image references.
MODELFORGE_COMMIT	no	public	Source commit stamped into images at build time.
MODELFORGE_BUILT_AT	no	public	Build timestamp stamped into images.
MODELFORGE_API_IMAGE	no	public	Exact tag or digest the production overlay runs for the API; never use latest.
MODELFORGE_WEB_IMAGE	no	public	Exact tag or digest the production overlay runs for the console; never use latest.
MODELFORGE_NODE_AGENT_IMAGE	no	public	Exact release tag or digest for the standalone Node Agent. The local-build fallback is named local and never resolves to latest.
MODELFORGE_API_IMAGE_DIGEST	no	public	Digest recorded as the running API build identity.
MODELFORGE_BACKUP_VOLUME	no	public	Volume or bind path backing the backup root.
MODELFORGE_RESTORE_VOLUME	no	public	Volume or bind path backing the restore staging root.
MODELFORGE_AGENT_STATE_VOLUME	no	public	Volume or bind path holding the agent's persisted identity.
MODELFORGE_AGENT_HF_CACHE_VOLUME	no	public	Volume or bind path for the agent's Hugging Face cache.
MODELFORGE_AGENT_ARTIFACT_VOLUME	no	public	Volume or bind path for verified artifacts on a node.
MODELFORGE_AGENT_QUARANTINE_VOLUME	no	public	Volume or bind path for the node's quarantine area.
MODELFORGE_AGENT_CONTROL_PLANE_URL	no	public	URL the agent reports to. Outbound only; the control plane never dials a node.
MODELFORGE_AGENT_ENROLLMENT_TOKEN	no	secret	Single-use enrolment token. Consumed atomically: a storm against one token produces exactly one identity.
MODELFORGE_AGENT_HOSTNAME	no	public	Hostname the agent enrols under.
MODELFORGE_AGENT_ACCELERATOR_MODE	no	public	Accelerator contract: nvidia fails closed unless NVML inventory and telemetry are valid; cpu permits a legitimate CPU-only node; auto requires NVIDIA when injected devices are observed. Canonical GPU Compose deployments set nvidia explicitly.
MODELFORGE_AGENT_TLS_VERIFY	no	public	Whether the agent verifies the control plane's certificate. True wherever TLS is real.
MODELFORGE_AGENT_CONTROL_PLANE_HOST_ADDRESS	no	internal	Host address mapped for a private-CA deployment.
MODELFORGE_AGENT_CA_CERT_PATH	no	public	Path to the private CA certificate the agent trusts.
MODELFORGE_RUNTIME_WORKER_ARTIFACT_ROOT	no	public	Artifact root as the runtime worker sees it.
MODELFORGE_RUNTIME_WORKER_POLL_INTERVAL_SECONDS	no	public	Worker poll interval, in seconds.
MODELFORGE_SOURCE_COMMIT	no	public	Source commit reported by the deployment.
MODELFORGE_SOURCE_REFERENCE	no	public	Git reference reported by the deployment.
MODELFORGE_SOURCE_REPOSITORY	no	public	Repository URL reported by the deployment.

Restarts

Deployment environment settings are read at process start; changing an environment value requires a restart. The Console also supports a closed set of numeric serving overrides which take effect for new requests after preview and explicit apply. See the configuration workflow for scope, acknowledgement, rollback and secret-safe transfer. Other settings remain deployment-managed.