ModelForge is a self-hosted control plane for running AI models on hardware you control. Applications ask for a stable capability such as rag.embedding@1; ModelForge selects an approved model revision, places it on an eligible compute node, and retains the evidence needed to explain and roll back that decision.

Console tour

Overview: the operator's starting point for model and infrastructure status.

Model library: inspect registered revisions and their details before planning a deployment.

Both images show the actual web console with synthetic API responses. No real operator, model, project or infrastructure data appear in them, and they do not imply a validated deployment. The current console uses Dutch interface labels.

The current source line is 1.3.0-rc.1. A source version does not by itself identify a qualified release or a running deployment; use the exact release evidence and deployment revision when operating an instance. The publication guide explains the release boundary.

What you get

A web console for models, hardware, capabilities, deployments, incidents and recovery.
A FastAPI control plane with capability-scoped service credentials.
Outbound-only node agents: compute nodes connect to the control plane and need no inbound agent port.
Quarantined model acquisition with immutable revisions, SHA-256 evidence and static inspection.
GPU-aware placement, leases, load-on-demand residency and typed runtime workers.
Approval-gated promotion, canary and rollback workflows with an append-only audit chain.
Backup, restore rehearsal, observability and operational runbooks.

ModelForge is designed for a small self-hosted fleet. It uses Docker Compose rather than Kubernetes, and a GPU is needed only on nodes that serve GPU-backed capabilities.

Quick start

Requirements: Docker Engine 24+, Docker Compose 2+, Python 3.12 for the host-side tools, and at least 50 GiB free plus room for model weights.

python scripts/preflight.py
cp .env.example .env
# Generate and enter the distinct secrets described in docs/INSTALLATION.md.
docker compose -f docker-compose.yml -f docker-compose.production.yml up -d --build

Then verify the control plane and open the console:

curl http://127.0.0.1:8000/api/v1/health/live
curl http://127.0.0.1:8000/api/v1/health/ready

Console: http://127.0.0.1:3000
API documentation: http://127.0.0.1:8000/docs

docker-compose.yml by itself is for development. The production overlay requires separate bootstrap, migration-owner and runtime database credentials and applies the production startup guards. Follow the complete installation guide before exposing an instance.

First useful workflow

Enrol a compute node with a short-lived, single-use token.
Discover a model and resolve it to an immutable upstream commit.
Approve a download plan; the node agent quarantines, verifies and atomically promotes the files.
Register a versioned capability contract and promote an evidence-backed deployment.
Give an application a credential scoped only to the capability it needs.

The application calls a stable endpoint such as:

curl -X POST \
  -H "Authorization: Bearer $MODELFORGE_CLIENT_CREDENTIAL" \
  -H "Content-Type: application/json" \
  -d '{"input":["text to embed"]}' \
  http://127.0.0.1:8000/api/v1/capabilities/rag.embedding@1/invoke

The caller does not need to know the model name, artifact path, runtime or GPU node. See First run and Project integration.

Architecture

flowchart LR
    Apps[Applications] --> Gateway[Capability gateway]
    Gateway --> Scheduler[Placement scheduler]
    Scheduler --> Workers[Typed runtime workers]
    Agents[Outbound node agents] --> Control[Control plane]
    Control --> Scheduler
    Control --> DB[(PostgreSQL and Redis)]
    Control --> Artifacts[Verified model artifacts]

Important boundaries:

exact model commits and container digests replace mutable names;
trust_remote_code remains disabled;
production changes require evidence and explicit approval;
compute agents publish observations but cannot act as operators;
runtime workers have no external network access by default;
destructive and recovery actions are planned, bounded and audited.

The system architecture, threat model and model supply-chain policy describe these guarantees in detail.

Documentation

Goal	Guide
Install or upgrade	Installation · Upgrade
Enrol a compute node	Node Agent
Configure the platform	Configuration
Integrate an application	Project integration
Operate and recover it	Operations · Troubleshooting
Review security	Security · Threat model
Understand compatibility	Compatibility
See release changes	Changelog · 1.2.2 release notes

Repository layout

backend/          FastAPI control plane, persistence and migrations
frontend/         React operator console
node-agent/       Outbound compute-node observer and artifact acquirer
runtime-worker/   Isolated typed model runtime
config/           Example capability, policy and model manifests
docs/             User, architecture, operations and security documentation
scripts/          Preflight, bootstrap, release and recovery tools

Diagnose readiness before invoking a model

Symptom	Check first
Console opens but a capability is unavailable	Approved artifact revision, capability binding and runtime readiness.
A node is visible but cannot host the model	Hardware inventory, placement requirements and available capacity.
Invocation is denied	Caller credentials and the granted capability scope.
A replacement model is not promoted	Required validation evidence and the promotion/canary decision.

Keep a working revision available for rollback. A downloaded weight file, a registered node or an accessible console alone does not establish a qualified serving route. Check the model's own license and resource requirements before acquiring its artifacts.

Development

Each component keeps its own pinned dependencies and tests. The protected-branch validation installs all four components, runs linting, strict type checks, tests and the frontend build, verifies Compose projections, scans for secrets, and audits Python and production npm dependencies.

python -m pytest backend/tests -q
python -m pytest node-agent/tests -q
python -m pytest runtime-worker/tests -q
cd frontend && npm ci && npm test -- --run && npm run build

See CONTRIBUTING.md before proposing a change and SECURITY.md for private vulnerability reporting.

License status

ModelForge is licensed under GNU AGPL-3.0-or-later. If you run a modified version for users over a network, the AGPL requires that those users can obtain the corresponding source. See LICENSE. Downloaded models, model weights and datasets retain their own upstream license terms and are not relicensed by ModelForge.