7.2 KiB
ITWorx ModelForge
One stable capability interface for models running on your own hardware.
Console tour · Workflow · Capabilities · Quick start · Architecture
ModelForge is a self-hosted control plane for running AI models on hardware you control. Applications
ask for a stable capability such as rag.embedding@1; ModelForge selects an approved model revision,
places it on an eligible compute node, and retains the evidence needed to explain and roll back that
decision.
Console tour
Overview: the operator's starting point for model and infrastructure status.
Model library: inspect registered revisions and their details before planning a deployment.
Both images show the actual web console with synthetic API responses. No real operator, model, project or infrastructure data appear in them, and they do not imply a validated deployment. The current console uses Dutch interface labels.
The current source line is 1.3.0-rc.1. A source version does not by itself identify a qualified release or a running deployment; use the exact release evidence and deployment revision when operating an instance. The publication guide explains the release boundary.
What you get
- A web console for models, hardware, capabilities, deployments, incidents and recovery.
- A FastAPI control plane with capability-scoped service credentials.
- Outbound-only node agents: compute nodes connect to the control plane and need no inbound agent port.
- Quarantined model acquisition with immutable revisions, SHA-256 evidence and static inspection.
- GPU-aware placement, leases, load-on-demand residency and typed runtime workers.
- Approval-gated promotion, canary and rollback workflows with an append-only audit chain.
- Backup, restore rehearsal, observability and operational runbooks.
ModelForge is designed for a small self-hosted fleet. It uses Docker Compose rather than Kubernetes, and a GPU is needed only on nodes that serve GPU-backed capabilities.
Quick start
Requirements: Docker Engine 24+, Docker Compose 2+, Python 3.12 for the host-side tools, and at least 50 GiB free plus room for model weights.
python scripts/preflight.py
cp .env.example .env
# Generate and enter the distinct secrets described in docs/INSTALLATION.md.
docker compose -f docker-compose.yml -f docker-compose.production.yml up -d --build
Then verify the control plane and open the console:
curl http://127.0.0.1:8000/api/v1/health/live
curl http://127.0.0.1:8000/api/v1/health/ready
- Console:
http://127.0.0.1:3000 - API documentation:
http://127.0.0.1:8000/docs
docker-compose.yml by itself is for development. The production overlay requires separate
bootstrap, migration-owner and runtime database credentials and applies the production startup
guards. Follow the complete installation guide before exposing an instance.
First useful workflow
- Enrol a compute node with a short-lived, single-use token.
- Discover a model and resolve it to an immutable upstream commit.
- Approve a download plan; the node agent quarantines, verifies and atomically promotes the files.
- Register a versioned capability contract and promote an evidence-backed deployment.
- Give an application a credential scoped only to the capability it needs.
The application calls a stable endpoint such as:
curl -X POST \
-H "Authorization: Bearer $MODELFORGE_CLIENT_CREDENTIAL" \
-H "Content-Type: application/json" \
-d '{"input":["text to embed"]}' \
http://127.0.0.1:8000/api/v1/capabilities/rag.embedding@1/invoke
The caller does not need to know the model name, artifact path, runtime or GPU node. See First run and Project integration.
Architecture
flowchart LR
Apps[Applications] --> Gateway[Capability gateway]
Gateway --> Scheduler[Placement scheduler]
Scheduler --> Workers[Typed runtime workers]
Agents[Outbound node agents] --> Control[Control plane]
Control --> Scheduler
Control --> DB[(PostgreSQL and Redis)]
Control --> Artifacts[Verified model artifacts]
Important boundaries:
- exact model commits and container digests replace mutable names;
trust_remote_coderemains disabled;- production changes require evidence and explicit approval;
- compute agents publish observations but cannot act as operators;
- runtime workers have no external network access by default;
- destructive and recovery actions are planned, bounded and audited.
The system architecture, threat model and model supply-chain policy describe these guarantees in detail.
Documentation
| Goal | Guide |
|---|---|
| Install or upgrade | Installation · Upgrade |
| Enrol a compute node | Node Agent |
| Configure the platform | Configuration |
| Integrate an application | Project integration |
| Operate and recover it | Operations · Troubleshooting |
| Review security | Security · Threat model |
| Understand compatibility | Compatibility |
| See release changes | Changelog · 1.2.2 release notes |
Repository layout
backend/ FastAPI control plane, persistence and migrations
frontend/ React operator console
node-agent/ Outbound compute-node observer and artifact acquirer
runtime-worker/ Isolated typed model runtime
config/ Example capability, policy and model manifests
docs/ User, architecture, operations and security documentation
scripts/ Preflight, bootstrap, release and recovery tools
Development
Each component keeps its own pinned dependencies and tests. The protected-branch validation installs all four components, runs linting, strict type checks, tests and the frontend build, verifies Compose projections, scans for secrets, and audits Python and production npm dependencies.
python -m pytest backend/tests -q
python -m pytest node-agent/tests -q
python -m pytest runtime-worker/tests -q
cd frontend && npm ci && npm test -- --run && npm run build
See CONTRIBUTING.md before proposing a change and SECURITY.md for private vulnerability reporting.
License status
ModelForge is licensed under GNU AGPL-3.0-or-later. If you run a modified version for users over a network, the AGPL requires that those users can obtain the corresponding source. See LICENSE. Downloaded models, model weights and datasets retain their own upstream license terms and are not relicensed by ModelForge.