44 lines
2.7 KiB
Markdown
44 lines
2.7 KiB
Markdown
# Durable background jobs
|
|
|
|
Course tagging and manual backup creation run as PostgreSQL-backed jobs. The admin
|
|
workspace shows persisted progress and supports cancellation. SSE sends the same
|
|
safe snapshot as the polling endpoint and closes on a terminal state; the browser
|
|
falls back to polling after an interrupted stream.
|
|
|
|
Only an enabled workspace administrator can create or execute a job. Mutation
|
|
endpoints require cookie authentication and CSRF validation. The worker rechecks
|
|
the actor's membership before work and before each course page. An idempotency key
|
|
is scoped to the workspace and actor; reusing it with a different payload returns
|
|
409. Retrying the same payload returns the existing job, including at capacity.
|
|
The job row and its `job.queued` audit event commit together. Cancellation requests
|
|
and their `job.cancel_requested` audit event also commit together; repeating a
|
|
running-job cancellation returns the current request without adding another audit
|
|
event.
|
|
|
|
The database serializes admission per workspace. Defaults under `KnowledgeHub:Jobs`
|
|
are `MaxPendingPerWorkspace=10`, `MaxPendingPerActor=3`, and
|
|
`MaxCreatedPerDay=100` (a rolling 24-hour window). Capacity rejection returns 429
|
|
before inserting a job or page snapshot. HTTP creation also has a shared budget
|
|
of ten requests per effective remote IP per minute. Only configured trusted
|
|
proxies can affect that address. These limits bound distinct-key floods across
|
|
application restarts and replicas as well as ordinary bursts.
|
|
|
|
Course tagging snapshots at most `MaxCoursePages=100000` page IDs in PostgreSQL.
|
|
The worker updates one current page at a time with its ETag, preserves content,
|
|
skips existing tags, and records safe per-page failures. A lease is renewed while
|
|
work is active; expired leases can be reclaimed with bounded retries. Default
|
|
worker concurrency is two. Cancellation stops future items and preserves the
|
|
already completed outcomes.
|
|
|
|
Terminal jobs retain their counters, bounded errors, result, actor and audit trail.
|
|
Their page-ID recovery snapshots are removed in the same transaction as the final
|
|
state. This limits large item-table growth without discarding the job history.
|
|
The daily admission budget bounds growth of that compact history. Source data is
|
|
never removed by this maintenance.
|
|
|
|
The job `Version` is a monotonic optimistic-concurrency token for persisted worker
|
|
updates and SSE event IDs. If a cancellation update wins a race with a tracked worker
|
|
save, the worker retries against the current version while keeping the cancellation
|
|
request. SSE always emits a terminal event and closes when it observes a terminal
|
|
job, including when that snapshot has the same version as the last progress event.
|