3.3 KiB
PrivateMuse audio resource control
ModelForge coordinates PrivateMuse audio providers through the existing
creative.resource@1 capability. It does not generate media and it does not infer that a model is
usable merely because its workload type is supported.
Workload vocabulary
| Workload | Product meaning |
|---|---|
speech_generation |
Narration and character dialogue synthesis |
voice_clone |
Consent-bound synthesis using an authorized reference voice |
music_generation |
Background music and score generation |
sound_generation |
Text-directed ambience and isolated sound effects |
foley_generation |
Audio derived from an accepted video take |
transcription |
Speech-to-text verification and timing |
audio_enhancement |
Cleanup or source separation that may use the shared GPU |
lip_sync |
A derived video aligned to an accepted speech asset |
The existing image, video, and chat values remain compatible. CPU-only utilities do not need
a GPU allocation. They may still be represented by a resource binding if a deployment uses the
shared accelerator; callers must request a positive measured or configured VRAM budget.
Admission invariant
An allocation is admitted only when the operator has approved the exact tuple of project, resource revision, workload, and provider resource identity. Supporting a workload name never creates a resource record and never approves an uninstalled or unqualified model. PrivateMuse must send a distinct resource route for ordinary speech synthesis and voice cloning even when both are served by one model, because cloned-voice use has separate product governance.
The first research candidates are Chatterbox Multilingual V3 for speech and authorized cloning, ACE-Step 1.5 for music, and Dasheng-AudioGen for ambience and effects. These names are planning inputs only. They must not appear in the live ModelForge catalog until their exact revisions, runtime manifests, local measurements, licenses, and provider receipts have passed qualification.
Lifecycle behavior
Audio allocations use the same durable behavior as image and video work:
- idempotency covers the complete request, including workload;
- a timeout after submission is reconciled as uncertain and never fails over blindly;
- a heartbeat extends only a granted lease;
- drain blocks new work while admitted work finishes or reconciles;
- measured peak VRAM is attached to the exact approved route;
- releasing one route may promote the next queued job without interrupting ComfyUI or Ollama.
An allocation granted from already observed schedulable free VRAM carries no automatic unload instruction. A resident model is not discarded just because a new PrivateMuse job starts. Queued work can be promoted only after capacity is actually observed; PrivateMuse may request a guarded idle ComfyUI cache release when its own queue is empty. Residency reuse and restoration of another application's default model require an explicit, measured warm lease protocol; an allocation grant alone is not proof that ComfyUI is holding the same model.
Director can orchestrate several independent PrivateMuse jobs, but ModelForge observes each provider allocation separately. A failed music request therefore cannot revoke accepted dialogue, and an uncertain sound-effect submission keeps only its own capacity fenced.