Initial curated itworx-knowledge-hub source from 3b1751d0d9dee551aa6994eee0093704dc334817

docs/13-attachments.md

@@ -0,0 +1,132 @@
+# Attachments
+
+## Goals
+
+- Preserve school and project files.
+- Avoid filename/path vulnerabilities.
+- Deduplicate identical content.
+- Maintain authorization and provenance.
+- Detect corruption.
+- Never execute uploaded content.
+
+## Storage model
+
+Physical blobs are addressed by lowercase SHA-256:
+
+```text
+/var/lib/knowledge-hub/attachments/ab/cd/<full-sha256>
+```
+
+The physical path never contains the user's filename. A logical attachment reference
+stores the original display filename and metadata in PostgreSQL.
+
+Temporary uploads use a separate same-filesystem quarantine directory so a validated file
+can be atomically moved.
+
+## Upload pipeline
+
+1. Authenticate and authorize.
+2. Enforce request and streaming limits.
+3. Sanitize display filename; reject path separators/control characters.
+4. Stream once to quarantine while calculating SHA-256.
+5. Verify actual byte length.
+6. Detect known signatures.
+7. Compare declared type, extension and detected type.
+8. Apply policy:
+   - safe raster image: eligible for inline preview;
+   - PDF/Office/text/archive/config: download by default;
+   - HTML/SVG/script/executable: download-only or reject according to policy;
+   - unknown: `application/octet-stream`, download-only.
+9. Atomically commit blob and logical reference.
+10. Generate preview asynchronously where supported.
+11. Remove quarantine data on every failure/cancellation.
+
+Default maximum: 100 MiB per file, configurable by administrator. The reverse proxy,
+Kestrel and application limits must agree.
+
+## Supported initial categories
+
+- PNG, JPEG, GIF, WebP;
+- PDF;
+- plain text and common configuration files;
+- Office documents;
+- common archives as opaque downloads;
+- other opaque files by administrator policy.
+
+SVG is not rendered inline in v1. Archives are not extracted as ordinary attachments.
+
+## Serving
+
+Every content request re-authorizes the logical reference.
+
+Headers:
+
+- `X-Content-Type-Options: nosniff`;
+- safe `Content-Disposition`;
+- bounded cache policy for private content;
+- CSP/sandbox where inline documents are later enabled;
+- no user-controlled response headers.
+
+Inline rendering is initially restricted to known safe raster images. Range requests may
+be added for large downloads with authorization tests.
+
+## Deduplication
+
+Two logical references may point to one blob.
+
+- Deduplication is global inside the installation storage, but authorization always uses
+  the logical reference.
+- A caller cannot discover whether a matching private blob already exists.
+- Content hash is not sufficient permission to retrieve.
+- Concurrent uploads with the same SHA-256 serialize the blob lookup, atomic commit
+  and database insert. Every upload receives its own logical reference while exactly
+  one physical blob is retained; uploads with different hashes remain concurrent.
+- Blob cleanup occurs only when no active or retained reference remains.
+- Reference counts are derived or transactionally maintained and periodically verified.
+
+## Image metadata
+
+Logical references may store:
+
+- alt text;
+- caption;
+- width/height;
+- rotation-normalized preview;
+- uploader and upload time.
+
+Strip unnecessary metadata from generated previews. Preserve the original bytes as the
+blob unless policy explicitly rejects them.
+
+## Integrity
+
+- New blobs are verified after atomic move.
+- Scheduled sampling/full scan recalculates hashes.
+- Missing/corrupt blobs mark references degraded and alert administrators.
+- Backups include a blob manifest and hashes.
+- Restore validates before enabling normal write traffic.
+
+## Deletion
+
+Deleting an attachment use/reference is logical first. Retention protects accidental
+deletion. Orphan physical blobs are cleaned in a background job after the configured
+window and a final transactional reference check.
+
+## Security tests
+
+Include:
+
+- double extensions;
+- path traversal names;
+- null/control characters;
+- oversized/chunked uploads;
+- declared/detected MIME mismatch;
+- HTML/SVG stored XSS;
+- polyglot samples where feasible;
+- executable/script files;
+- authorization ID substitution;
+- interrupted upload cleanup;
+- duplicate content;
+- concurrent duplicate uploads;
+- malicious archive names without extraction;
+- race between deletion and download;
+- corruption detection.