209 lines
4.9 KiB
Markdown
209 lines
4.9 KiB
Markdown
# Testing strategy
|
|
|
|
## Test pyramid and ownership
|
|
|
|
### Domain/unit tests
|
|
|
|
Fast deterministic tests for:
|
|
|
|
- domain invariants;
|
|
- status transitions;
|
|
- slug/title normalization;
|
|
- link parsing;
|
|
- import planning rules;
|
|
- token/password-independent utility behavior;
|
|
- context bounding;
|
|
- retention rules.
|
|
|
|
### Application tests
|
|
|
|
Use cases with controlled ports/fakes where appropriate:
|
|
|
|
- authorization requirements;
|
|
- commands/queries;
|
|
- conflict behavior;
|
|
- audit/outbox intent;
|
|
- validation and error types.
|
|
|
|
### PostgreSQL integration tests
|
|
|
|
Use Testcontainers with the supported PostgreSQL major/minor family:
|
|
|
|
- EF migrations from empty and previous schema fixtures;
|
|
- constraints and transactionality;
|
|
- authorization-aware queries;
|
|
- search ranking/leakage;
|
|
- import idempotency/rollback;
|
|
- job leasing;
|
|
- backup metadata/integrity logic;
|
|
- optimistic concurrency.
|
|
|
|
SQLite is not an acceptable substitute for application persistence tests. SQLite is used
|
|
only as a legacy import source fixture.
|
|
|
|
### Frontend component tests
|
|
|
|
Vitest and Testing Library:
|
|
|
|
- component behavior and semantics;
|
|
- forms/validation;
|
|
- loading/error/conflict states;
|
|
- navigation tree interactions;
|
|
- API contract adapters;
|
|
- draft state;
|
|
- project/ADR/runbook views.
|
|
|
|
### Browser end-to-end tests
|
|
|
|
Playwright against the real app and PostgreSQL:
|
|
|
|
- first-run setup;
|
|
- login/logout/recovery;
|
|
- create/edit/move/trash/restore page;
|
|
- internal links and backlinks;
|
|
- revision compare/restore;
|
|
- search and filters;
|
|
- attachment upload/download;
|
|
- synthetic import and report;
|
|
- project/ADR/runbook flow;
|
|
- service-account lifecycle through admin UI;
|
|
- backup status/restore rehearsal entry points;
|
|
- desktop and mobile;
|
|
- axe checks.
|
|
|
|
### Operational tests
|
|
|
|
- production image build;
|
|
- non-root identity;
|
|
- read-only root filesystem;
|
|
- required mount permissions;
|
|
- health/readiness behavior;
|
|
- restart recovery;
|
|
- database migration rehearsal;
|
|
- full backup/clean restore;
|
|
- source immutability;
|
|
- upgrade from prior release.
|
|
|
|
## Test projects
|
|
|
|
Intended structure:
|
|
|
|
```text
|
|
tests/backend/
|
|
ItWorx.KnowledgeHub.Domain.Tests
|
|
ItWorx.KnowledgeHub.Application.Tests
|
|
ItWorx.KnowledgeHub.Architecture.Tests
|
|
ItWorx.KnowledgeHub.IntegrationTests
|
|
|
|
tests/e2e/
|
|
Playwright tests and fixtures
|
|
|
|
tools/leafwiki-inspector/tests/
|
|
Inspector unit tests
|
|
```
|
|
|
|
## Test data
|
|
|
|
Use deterministic builders and synthetic fixtures.
|
|
|
|
Never use:
|
|
|
|
- real LeafWiki data;
|
|
- real emails/passwords;
|
|
- production URLs/tokens;
|
|
- copied private project documents.
|
|
|
|
Synthetic fixture names should make their fictional nature clear.
|
|
|
|
## Database lifecycle
|
|
|
|
- One PostgreSQL Testcontainer per test collection/suite, with isolated database/schema
|
|
per test where needed.
|
|
- Apply real migrations.
|
|
- Reset state using a safe test-only reset strategy.
|
|
- Parallelize only after proving isolation.
|
|
- Capture container logs on failure.
|
|
- Pin the test image to the supported PostgreSQL version family.
|
|
|
|
## Time and IDs
|
|
|
|
- Inject `TimeProvider`.
|
|
- Use deterministic or recorded generated IDs.
|
|
- Avoid sleeps; wait on observable state with bounded timeouts.
|
|
- Control locale/timezone in tests.
|
|
|
|
## Security test corpus
|
|
|
|
- stored/reflected Markdown XSS;
|
|
- unsafe links;
|
|
- Mermaid injection;
|
|
- CSRF;
|
|
- session fixation/invalidation;
|
|
- role/scope escalation;
|
|
- object-ID substitution;
|
|
- search leakage;
|
|
- upload signatures/path names/polyglots;
|
|
- import traversal/symlinks/corrupt DB;
|
|
- secret logging;
|
|
- service-token revoke/rotation.
|
|
|
|
## Scale suites
|
|
|
|
Separate normal CI and heavier release suites.
|
|
|
|
Normal CI:
|
|
|
|
- representative hierarchy;
|
|
- hundreds/thousands of pages;
|
|
- fast search correctness.
|
|
|
|
Release:
|
|
|
|
- 100,000 pages;
|
|
- 1,000,000 revisions;
|
|
- large project/update relations;
|
|
- attachment manifest scale;
|
|
- query plans and p95 timings.
|
|
|
|
Store generator seed and environment details with results.
|
|
|
|
## Flake policy
|
|
|
|
- No blind retry loops.
|
|
- One infrastructure retry may be configured only for a documented external startup race.
|
|
- A flaky product test blocks release until root cause is fixed or an explicit accepted
|
|
issue proves it is not product risk.
|
|
- Quarantined/skipped tests require owner, reason and expiry.
|
|
|
|
## Coverage
|
|
|
|
Coverage percentage is diagnostic, not the completion definition. Critical branches need
|
|
explicit tests regardless of total percentage.
|
|
|
|
Minimum release expectations:
|
|
|
|
- all authorization policies covered;
|
|
- all domain invariants covered;
|
|
- all API error contracts covered;
|
|
- all migration/restore destructive boundaries covered;
|
|
- primary user journeys covered end to end.
|
|
|
|
## Verification entry points
|
|
|
|
`node scripts/verify.mjs` orchestrates:
|
|
|
|
- structural/bundle validation;
|
|
- format/lint/type check;
|
|
- .NET build/tests;
|
|
- frontend tests;
|
|
- Python inspector tests;
|
|
- optional E2E;
|
|
- optional container/release gates.
|
|
|
|
Flags are documented in the script. CI uses the same entry point.
|
|
|
|
## Evidence
|
|
|
|
Record exact commands, commit SHA, versions and counts in `EVIDENCE.md`. Generated test
|
|
reports live in ignored `artifacts/`, with stable summary references.
|