Initial curated EventLens source from 0d80d3b38dfb817285481665b9e71e4b8a0656a0
This commit is contained in:
commit
663ea8c829
85 files changed
+12395
No files matched your search
@@ -0,0 +1,40 @@
|
||||
# Architecture
|
||||
|
||||
```text
|
||||
Browser .evtx
|
||||
|
|
||||
v
|
||||
ASGI request/body + filename gates -> bounded in-memory bytes
|
||||
|
|
||||
v
|
||||
bounded parse admission -> Rust-backed evtx parser (single thread, time/event/record limits)
|
||||
|
|
||||
v
|
||||
normalized events (bounded strings/fields, explicit timestamp validity/source, stable IDs)
|
||||
|
|
||||
v
|
||||
deterministic correlation rules -> incidents + reasons + evidence IDs
|
||||
|
|
||||
v
|
||||
React workspace shell (Analysis/Timeline/Hunting/Reports/Settings) -> virtualized timeline/filter/inspector -> cloned redacted JSON/Markdown export
|
||||
```
|
||||
|
||||
## Boundaries
|
||||
|
||||
- `backend/app/main.py` owns HTTP, CORS, receive-level request limits, bounded parse admission, privacy headers, upload reading and response contracts.
|
||||
- `backend/app/parser.py` owns the untrusted EVTX boundary and never emits raw XML to presentation code.
|
||||
- `backend/app/normalize.py` maps parser-native JSON into `NormalizedEvent` values, with bounded fields, UTC timestamps and explicit `event`/`record`/`missing`/`invalid` timestamp metadata.
|
||||
- `backend/app/correlate.py` consumes normalized events only. It never parses raw EVTX, requires provider/channel context for provider-scoped IDs, avoids unknown grouping keys and attaches stable rule/incident/evidence identifiers.
|
||||
- `backend/app/models.py` defines the API contract.
|
||||
- `frontend/src/filter.ts`, `redact.ts` and `export.ts` are pure browser-side workflow functions.
|
||||
- `frontend/src/main.tsx` is only the React bootstrap. `frontend/src/app/App.tsx` owns explicit workspace state (active hash workspace, selection, filters, settings and loading/error state); reusable shell/state components live in `frontend/src/components/`, and Analysis, Timeline, Hunting, Reports and Settings live in `frontend/src/workspaces/` over the same in-memory `Analysis` object. The browser UI does not implement parser or correlation rules.
|
||||
|
||||
## Resource and privacy model
|
||||
|
||||
Uploads are bounded while ASGI body messages are received, then read in bounded chunks and parsed from bytes in a worker thread so the FastAPI event loop remains responsive. Each parser record is checked before JSON expansion. Normalized output has a global character budget in addition to per-event limits. The multipart handle is closed after the request and no upload path is retained.
|
||||
|
||||
API and health responses are non-cacheable and carry defensive browser headers. The production container runs as an unprivileged user with a read-only root filesystem, dropped capabilities and a bounded temporary filesystem. The browser keeps only the current normalized analysis in memory; large timelines are virtualized and search documents are cached per event.
|
||||
|
||||
The deployment broker passes the validated main commit as `GITEA_COMMIT_SHA` into the Compose build. Docker embeds that exact SHA as `EVENTLENS_REVISION`; `/health` exposes it with the product version without running Git in the container. An unlabelled local build reports `local`.
|
||||
|
||||
The browser stores only one validated local-preferences record in `localStorage` (`eventlens.preferences.v1`); it never stores events, analysis results or EVTX-derived values. Settings writes that record automatically, exposes an accessible save confirmation and can reset it to validated defaults. A configured default workspace is used only when no URL hash is present, so a hash/deep link always wins. Workspace navigation uses a lightweight URL hash so browser back/forward works without a routing dependency. The static browser-brand package (SVG/ICO/PNG/manifest) is bundled locally, has no service worker and does not add a cache or network dependency. The bundled JSON demo exercises the same normalized, correlation and presentation contracts while deliberately skipping only raw EVTX parsing. The real parser is qualified in CI against a public, hash-verified seven-record EVTX fixture without committing that fixture or its event data to the repository.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Correlation design
|
||||
|
||||
Correlation is deterministic and explainable. EventLens says records are temporally or semantically related; it does not invent causality.
|
||||
|
||||
Every incident carries a stable `rule_id`, bounded total time window, explicit predicate, grouping key, human-readable reason and exact evidence event IDs. Incident IDs are SHA-256-derived from the rule and ordered evidence IDs, so repeated analysis of identical normalized evidence yields the same identifiers.
|
||||
|
||||
## Ordering and identity
|
||||
|
||||
- Parser timestamps are normalized to timezone-aware UTC values. A valid event timestamp wins; a valid parser-record timestamp is an explicit fallback. Invalid or missing timestamps remain `null` with `timestamp_source`/`timestamp_valid` metadata and are excluded from temporal correlation, so no synthetic 1970 evidence is created.
|
||||
- Correlation sorts by `(timestamp, event.id)` before grouping and returns incidents by `(start, rule_id, id)`.
|
||||
- EVTX record IDs are retained as stable event IDs. When a parser record has no record ID, the parser ordinal plus canonical event content produces a deterministic hash; duplicate records at the same timestamp do not silently share an ID.
|
||||
- Evidence IDs are copied exactly into every incident. Repeated signatures remain in the evidence timeline and are represented by `original_count` versus `compressed_count` in incident metadata.
|
||||
|
||||
## Rules and boundaries
|
||||
|
||||
- `windows.crash.application-report` — Application Error 1000 and/or Windows Error Reporting 1001 from the exact Application provider/channel within 120 seconds, grouped by known host/process. The word “sequence” is reserved for evidence containing both records.
|
||||
- `windows.service.failure` — abnormal Service Control Manager 7023/7024/7031/7034 events from the System channel within 120 seconds, grouped by known host/service.
|
||||
- `windows.network.dns-timeout` — DNS Client 1014 events from the System channel in a bounded 120-second window per known host.
|
||||
- `windows.auth.failed-logon` — Security Auditing 4625 events from the Security channel in a bounded 120-second window per known host/account. One event is recorded evidence; two are repeated failures; three or more are called a burst.
|
||||
- `windows.system.startup` — Kernel-General 12 or EventLog 6005 startup signals in a bounded 300-second window per known host.
|
||||
- `windows.system.shutdown.normal` — Kernel-General 13 or EventLog 6006 normal-shutdown evidence in a bounded 300-second window per known host.
|
||||
- `windows.system.shutdown.unexpected` — Kernel-Power 41 or EventLog 6008 unexpected-shutdown/restart evidence in a bounded 300-second window per known host. The reason describes evidence, not a cause.
|
||||
- `windows.generic.attention` — preserves unmatched warning/error evidence rather than hiding it.
|
||||
|
||||
A rule requires its provider and channel context; a coincidental Event ID from another provider is not classified as that rule. Missing/placeholder host, process, service or account values create an event-isolated group and the reason says that grouping was unavailable; they never become a shared `unknown-*` key. Events without a reliable timestamp remain evidence but do not enter temporal groups.
|
||||
|
||||
A group is split when adding the next event would make the total span from the first event exceed the rule window. Equality at the boundary is included; one millisecond beyond it is split. This prevents a long chain of near-adjacent records from creating an unbounded incident.
|
||||
|
||||
Repeated identical provider/event/summary signatures remain in the evidence timeline but are compressed in the incident summary; source records and evidence IDs are never removed. Events may support more than one specifically matching rule when their predicates independently match; no rule is treated as proof of root cause.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Privacy model
|
||||
|
||||
Windows Event Logs frequently contain usernames, hostnames, IP addresses, e-mail addresses, executable paths and application-specific fields.
|
||||
|
||||
## In-memory workflow
|
||||
|
||||
1. The browser sends the selected `.evtx` to the self-hosted API.
|
||||
2. A receive-level request gate rejects oversized body streams before multipart parsing; the API then reads bounded chunks, parses bytes off the event loop and returns normalized JSON.
|
||||
3. EventLens does not create an application-level upload file or database record.
|
||||
4. The multipart parser may transiently spool request data as part of the web framework; the upload handle is closed after every request and the app does not retain that path.
|
||||
5. The browser keeps the normalized analysis in component memory for the current session; switching workspaces does not create another copy or upload it elsewhere.
|
||||
6. Only display preferences (timestamp mode, defaults, density and preferred workspace) may be stored in the single `eventlens.preferences.v1` localStorage record; event text, EVTX-derived values and analysis results are never stored there. The Settings workspace labels this boundary directly, exposes its technical key only on request and can reset the record to defaults.
|
||||
7. Export is generated locally in the browser.
|
||||
8. The favicon, touch icons and web manifest are static files bundled with EventLens. They make no third-party request, do not enable a service worker and do not persist investigation data.
|
||||
|
||||
## Redaction before export
|
||||
|
||||
Redaction is enabled by default and operates on a cloned analysis object. It masks or replaces:
|
||||
|
||||
- computer/host values;
|
||||
- username/account/e-mail identity fields, including domain-qualified accounts, SIDs and machine accounts;
|
||||
- e-mail addresses in free text;
|
||||
- IPv4 and IPv6 addresses in free text or IP-address fields;
|
||||
- Windows and UNC paths (including spaces/parentheses) and path-like fields;
|
||||
- source-name fields and source values propagated into free-form text;
|
||||
- known structured identities propagated into summaries, incident titles and reasons;
|
||||
- the source filename in the exported report.
|
||||
|
||||
The UI previews replacement counts by type before export. The original in-memory analysis is never modified by the redaction function. The source name is replaced with `[REDACTED_SOURCE].evtx` when redaction is enabled. JSON and Markdown carry the same redacted normalized analysis and parse-completeness metadata.
|
||||
|
||||
Redaction can be disabled when a raw forensic export is explicitly required, but that is an explicit user choice and the warning remains visible.
|
||||
|
||||
## Limits
|
||||
|
||||
The API enforces a 50 MiB file limit, a bounded request body independent of `Content-Length`, parser event/time budgets, bounded normalized strings/fields/output and bounded parse concurrency. These controls reduce accidental resource exhaustion; they are not a guarantee against every parser implementation vulnerability.
|
||||
|
||||
Redaction is a best-effort privacy layer, not a formal DLP engine. Application-specific free-form values may contain sensitive data that cannot be inferred safely. Review exports before sharing them outside the intended context.
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 279 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 260 KiB |
Reference in new issue
Block a user