# Operations overview dashboard. # # Documents the panels a real dashboard (Grafana or equivalent) should # expose for the representative metric set # src/ragcore/platform/telemetry/metrics.py::TelemetryMetrics records, per # docs/09_OPERATIONS/OBSERVABILITY.md's "Metrics" list. Tool-agnostic on # purpose -- no specific dashboarding backend is assumed by this codebase # yet. # # This is a representative subset (request latency/error, job queue # depth/age, Qdrant/provider errors), not every metric that document # names; wiring metrics for every existing subsystem (search candidate # counts, zero-result rate, ingestion throughput, backup/restore age, ...) # is future work, not required by this task's own acceptance criteria. dashboard: RAGCore Operations Overview panels: - title: Request latency metric: ragcore_request_duration_seconds type: histogram unit: seconds description: HTTP request duration, by status_code. - title: Request error rate metric: ragcore_request_errors_total type: counter description: >- HTTP requests resulting in a server error (5xx) or rate-limited (429) status, by status_code. - title: Job queue depth metric: ragcore_job_queue_depth type: up_down_counter description: Number of jobs currently queued (enqueued minus claimed). - title: Job age at claim metric: ragcore_job_age_seconds type: histogram unit: seconds description: >- How long a job waited between being enqueued and being claimed by a worker. Feeds the queue-age SLO alert (see ops/alerts/queue_age_slo.yaml). - title: Qdrant errors metric: ragcore_qdrant_errors_total type: counter description: Qdrant operation failures, by operation. - title: Provider errors metric: ragcore_provider_errors_total type: counter description: >- Model provider (embedding/reranker/generation) operation failures, by provider. Compare against ops/alerts/provider_circuit_open.yaml (M5-07) for when a provider's circuit breaker actually opens. correlation: # How to find every span/log/metric for one request or job in whatever # backend actually stores them: they all carry the same correlation_id # attribute, and spans opened from different components (an API handler, # a later job) for the same correlation_id share the literal same # OpenTelemetry trace id (see # src/ragcore/platform/telemetry/tracing.py::trace_id_from_correlation_id). attribute: correlation_id source: RequestCorrelationMiddleware's request_id / a job's own correlation id