ITWorx_MCP_Hub-Public/docs/04-mcp/KNOWLEDGE_INTAKE.md

ChatGPT to Knowledge Hub document intake

The separate MCP Hub exposes ten focused tools at /mcp/knowledge-hub/prod: search and project context; scanned attachment text extraction; workspace roots, children and current page reads; attachment and caller-supplied Gitea text proposals; review submission; and status. analyze_knowledge_files and extract_knowledge_files advertise _meta.openai/fileParams: ["files"]. ContextForge omits connector tool metadata, so the dataplane restores this exact binding only for these two vetted attachment schemas on knowledge-hub/prod. The connector scans every source within fixed limits. Extraction returns real text in bounded pages with per-file coverage and an actor-bound authenticated continuation cursor; it has no proposal side effects. A proposal includes the selected create or update publication, and only a human reviewer in Knowledge Hub can approve or publish it.

For early attachment validation failures, connector audit may include a fixed input_reason label such as attachment_file_id_invalid. The label identifies the rejected check only; attachment IDs, URLs, names, and content are never recorded.

Access boundaries

• The ChatGPT OAuth identity must be bound in MCP Hub to the exact knowledge-hub project and prod environment. MCP Hub signs the connector identity assertion and applies the tool policy. A supplied target Knowledge Hub project_id must be in KNOWLEDGE_HUB_ALLOWED_PROJECT_IDS. Project context always requires this ID. When project_id is omitted, search, extraction, proposal creation, review submission and status use the token-scoped workspace only if KNOWLEDGE_HUB_WORKSPACE_READ_ENABLED=true and the signed connector identity is exactly knowledge-hub/prod. This setting defaults to false; an empty target project allowlist is accepted only when workspace access is enabled.
• Workspace roots and children are cursor paged. Current page Markdown is character paged and carries an ETag; later slices require the same ETag. Select a parent page or workspace root for a create, or compare the complete current page and include its ETag for an update.
• For Gitea sources, first read the complete file through the separate Gitea MCP. Pass the bounded UTF-8 text and caller-supplied itworx-git:// URI, ref, commit and full Git blob SHA to create_text_source_proposal. The intake connector checks that the supplied text matches that SHA and scans it. source_complete=true and repository provenance remain caller assertions; the connector does not fetch a repository or independently verify its contents.
• A publication Markdown body is limited to 32,000 characters and analysis to 32,000 characters. An attachment or PDF with no extractable text is rejected before a proposal can be created; OCR is outside this path. DOCX extraction reads the main document text and may omit header/footer content, so review those sections separately before submitting a proposal.
• The connector uses a separate Knowledge Hub service account with exactly knowledge:search, projects:read, pages:read, and ingestion:write. It cannot approve or publish proposals. Its identity assertion uses a dedicated knowledge_hub_ingestion_identity_key secret mounted only in the dataplane and intake connector, separate from the other connector HMAC key.
• The Knowledge Hub token is loaded only from /run/secrets/knowledge_hub_ingestion_token. Signed attachment download URLs and token values must not appear in logs, audit events, or persisted proposal analysis.
• An internal retry of a failed proposal is available only with the current ETag, retryEligible=true, and a fresh operation key. It is not exposed as an MCP tool or a publication action.
• The connector and scanner are opt in. The default INGESTION_CONNECTOR_ENABLED=false excludes the new project and gateway from reconciliation, leaving existing MCP routes available during staging. The default dataplane does not mount or require the new identity key; the final intake production overlay adds that mount when the intake route is activated.

Production activation

The live wiki.example.com Authentik proxy requires a narrow unauthenticated-path exception for the connector's API routes. Its proxy provider uses:

^/api/v1/(?:search|projects/[0-9a-fA-F-]{36}/context|pages(?:/roots|/[0-9a-fA-F-]{36}(?:/children)?)|ingestion-proposals(?:/[0-9a-fA-F-]{36}(?:/(?:sources|retry))?)?)$

This only passes requests to Knowledge Hub's own bearer authorization. An anonymous JSON request to each allowed route must return 401; / and unrelated API routes must continue to redirect to Authentik. Keep the exception aligned with the connector's fixed route allowlist.

1. Deploy the Knowledge Hub admin preset that issues the four-scope intake token, including pages:read. Create the dedicated token in Beheer and place it in the operator-owned secret directory as knowledge_hub_ingestion_token with ownership matching the MCP Hub service UID/GID. Generate a separate random 32-byte-or-longer knowledge_hub_ingestion_identity_key in that directory. Keep both secrets out of Git and deployment logs.
2. Verify a current backup and the pinned images. Set KNOWLEDGE_HUB_ENDPOINT to the actual HTTPS Knowledge Hub origin. Set KNOWLEDGE_HUB_ALLOWED_PROJECT_IDS to comma-separated UUIDs if project-scoped intake is needed; for workspace-only courses, keep it empty and explicitly set KNOWLEDGE_HUB_WORKSPACE_READ_ENABLED=true after reviewing the workspace-wide permission. Pin CONNECTOR_INGESTION_IMAGE to an immutable digest. Render Compose with docker-compose.ingestion.prod.yml last, after docker-compose.prod.yml and docker-compose.autodeploy.yml; inspect that the final connector image is the digest and build is absent before starting it.
3. Start ingestion-clamd and connector-ingestion with the knowledge-intake Compose profile while INGESTION_CONNECTOR_ENABLED=false. Wait for current scanner signatures and connector readiness. Verify the four token scopes, a project context read when project intake is configured, and roots/current-page reads when workspace access is enabled.
4. Add an enabled, exact OAuth client binding for knowledge-hub/prod. Set INGESTION_CONNECTOR_ENABLED=true on control API and worker, then restart only those services and verify gateway tool parity, project route, and all ten advertised tool schemas.
5. Connect the MCP URL in ChatGPT. Exercise search, roots/children and a complete current-page read; attach a benign test PDF and read every extraction page; then create a complete proposal and submit it. Confirm it reaches ReadyForReview; approve/publish through the Knowledge Hub UI only. Check that a rejected file leaves no proposal or unscanned persisted source.

If staging fails, disable the knowledge-hub/prod OAuth client binding, set INGESTION_CONNECTOR_ENABLED=false, and reconcile again before stopping the connector. This disables an already created intake virtual server as well. Retain proposal IDs and audit evidence for human review. Do not delete Knowledge Hub data to roll back the connector.