Observability

How RustFS Exposes Telemetry

RustFS pushes metrics, traces, and logs over OTLP (OpenTelemetry Protocol). It does not expose a native Prometheus /metrics scrape endpoint — there is no HTTP route on port 9000 or 9001 that serves Prometheus text format. To get RustFS metrics into Prometheus you must run an OpenTelemetry Collector that receives OTLP from RustFS and re-exposes the data in Prometheus format:

RustFS (RUSTFS_OBS_ENDPOINT) --OTLP--> OpenTelemetry Collector --> Prometheus --> Grafana
                                                              \--> Loki (logs)
                                                              \--> Tempo / Jaeger (traces)

Point RustFS at the Collector with:

# OTLP over HTTP (the Collector's default HTTP receiver port is 4318; gRPC is 4317)
RUSTFS_OBS_ENDPOINT=http://otel-collector:4318

Related environment variables (all defined in the server configuration):

Reference Deployment (Docker Compose Observability Profile)

The upstream rustfs/rustfs repository ships a complete reference stack in docker-compose.yml behind the observability profile: otel-collector, Prometheus, Grafana, Loki (logs), Tempo and Jaeger (traces). Its Collector configuration receives OTLP on 4317 (gRPC) and 4318 (HTTP) and re-exports metrics for Prometheus on port 8889:

# OpenTelemetry Collector (excerpt from the upstream reference config)
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheus]

Prometheus then scrapes the Collector, not RustFS:

# prometheus.yml (excerpt)
scrape_configs:
  - job_name: "rustfs-app-metrics"
    static_configs:
      - targets: ["otel-collector:8889"]   # RustFS application metrics
  - job_name: "otel-collector"
    static_configs:
      - targets: ["otel-collector:8888"]   # Collector self-metrics

Metrics Reference

Every name below is verified against the server source. The list covers the operations-relevant subset exported via OTLP; internal and experimental instruments are omitted, and new releases may add metrics not listed here. Label sets are those recorded at the emission site. The Collector and Prometheus may attach additional resource labels, such as service_name and instance.

Readiness and Process

Scanner

Heal

API / IO Path

Backpressure

Capacity

Locking

Log Cleaner

Suggested Alerts

Log Collection

Where logs go is controlled by RUSTFS_OBS_LOG_DIRECTORY:

• Unset — logs go to stdout. In containers, use your log driver; on systemd hosts, stdout/stderr is captured by journald, so journalctl -u rustfs -f works without extra configuration.
• A local directory (e.g. /logs) — RustFS writes rotating log files there. Rotation is tuned with RUSTFS_OBS_LOG_FILENAME, RUSTFS_OBS_LOG_ROTATION_SIZE_MB, RUSTFS_OBS_LOG_ROTATION_TIME, and RUSTFS_OBS_LOG_KEEP_FILES.
• A URL (contains ://) — logs are shipped to a remote endpoint.

When RUSTFS_OBS_ENDPOINT is set and log export is enabled, logs are additionally exported via OTLP; the reference stack routes them into Loki for querying from Grafana.

RUSTFS_OBS_LOG_DIRECTORY=/var/log/rustfs
RUSTFS_OBS_LOGGER_LEVEL=info
RUSTFS_OBS_ENDPOINT=http://otel-collector:4318