69 lines
2.5 KiB
YAML
69 lines
2.5 KiB
YAML
# Operations overview dashboard.
|
|
#
|
|
# Documents the panels a real dashboard (Grafana or equivalent) should
|
|
# expose for the representative metric set
|
|
# src/ragcore/platform/telemetry/metrics.py::TelemetryMetrics records, per
|
|
# docs/09_OPERATIONS/OBSERVABILITY.md's "Metrics" list. Tool-agnostic on
|
|
# purpose -- no specific dashboarding backend is assumed by this codebase
|
|
# yet.
|
|
#
|
|
# This is a representative subset (request latency/error, job queue
|
|
# depth/age, Qdrant/provider errors), not every metric that document
|
|
# names; wiring metrics for every existing subsystem (search candidate
|
|
# counts, zero-result rate, ingestion throughput, backup/restore age, ...)
|
|
# is future work, not required by this task's own acceptance criteria.
|
|
|
|
dashboard: RAGCore Operations Overview
|
|
|
|
panels:
|
|
- title: Request latency
|
|
metric: ragcore_request_duration_seconds
|
|
type: histogram
|
|
unit: seconds
|
|
description: HTTP request duration, by status_code.
|
|
|
|
- title: Request error rate
|
|
metric: ragcore_request_errors_total
|
|
type: counter
|
|
description: >-
|
|
HTTP requests resulting in a server error (5xx) or rate-limited
|
|
(429) status, by status_code.
|
|
|
|
- title: Job queue depth
|
|
metric: ragcore_job_queue_depth
|
|
type: up_down_counter
|
|
description: Number of jobs currently queued (enqueued minus claimed).
|
|
|
|
- title: Job age at claim
|
|
metric: ragcore_job_age_seconds
|
|
type: histogram
|
|
unit: seconds
|
|
description: >-
|
|
How long a job waited between being enqueued and being claimed by a
|
|
worker. Feeds the queue-age SLO alert (see
|
|
ops/alerts/queue_age_slo.yaml).
|
|
|
|
- title: Qdrant errors
|
|
metric: ragcore_qdrant_errors_total
|
|
type: counter
|
|
description: Qdrant operation failures, by operation.
|
|
|
|
- title: Provider errors
|
|
metric: ragcore_provider_errors_total
|
|
type: counter
|
|
description: >-
|
|
Model provider (embedding/reranker/generation) operation failures,
|
|
by provider. Compare against
|
|
ops/alerts/provider_circuit_open.yaml (M5-07) for when a provider's
|
|
circuit breaker actually opens.
|
|
|
|
correlation:
|
|
# How to find every span/log/metric for one request or job in whatever
|
|
# backend actually stores them: they all carry the same correlation_id
|
|
# attribute, and spans opened from different components (an API handler,
|
|
# a later job) for the same correlation_id share the literal same
|
|
# OpenTelemetry trace id (see
|
|
# src/ragcore/platform/telemetry/tracing.py::trace_id_from_correlation_id).
|
|
attribute: correlation_id
|
|
source: RequestCorrelationMiddleware's request_id / a job's own correlation id
|