Observability
This document describes how to configure observability mechanisms including metrics, logging, and tracing for Flipt v2.
Metrics
Prometheus
Flipt exposes Prometheus metrics by default at the /metrics
HTTP endpoint. To see which metrics are currently supported, point your browser
to FLIPT_HOST/metrics
(ex: localhost:8080/metrics
).
You should see a bunch of metrics being recorded such as:
You can disable the Prometheus metrics collection by setting the
metrics.enabled
configuration option to false
.
OTLP
Flipt supports sending metrics to an OTLP collector.
OTLP supports additional configuration such as specifying the protocol to use (gRPC or HTTP) as well as providing custom headers to send with the request.
Custom headers can be used to provide authentication information to the collector which may be required if you are using a hosted collector such as NewRelic, DataDog, or Honeycomb.
OpenTelemetry OTLP metrics are configured via the default OpenTelemetry Environment Variables. See the OpenTelemetry Environment Variables documentation for more details.
For example, to configure the OTLP metrics endpoint, you can set the OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
environment variable.
To configure the OTLP headers, you can set the OTEL_EXPORTER_OTLP_METRICS_HEADERS
environment variable.
Logging
Flipt writes logs to STDOUT in two formats:
The format can be configured via the log.encoding
configuration option.
For production deployments, we recommend using the JSON format as it’s easier to parse and ingest into log aggregation systems such as Elasticsearch, Splunk, Loki, or Datadog.
JSON
Log Key Descriptions
L
: Level (log level). Possible values include: debug, info, warn, error, fatal, and panic.T
: Timestamp. The timestamp is in ISO 8601 format, widely used for representing date and time. It includes the date, time, and time zone information. For example, “2024-01-20T21:59:49-05:00” represents the date and time in the Eastern Time Zone (UTC-5).M
: Message. The message describes the log event. It can include information about the operation, errors encountered, or other relevant details.
Console
More information about the available configuration options can be found in the Logging configuration section.
OTLP
Flipt v2 supports the new OpenTelemetry OTLP logging specification. To enable OTLP logging, set the OTLP_LOGS_EXPORTER
environment variable
OpenTelemetry logging is in addition to the existing logging configuration. It does not replace the ability to log to a file or stdout/stderr.
OpenTelemetry OTLP logging is configured via the default OpenTelemetry Environment Variables. See the OpenTelemetry Environment Variables documentation for more details.
For example, to configure the OTLP logging endpoint, you can set the OTEL_EXPORTER_OTLP_LOGS_ENDPOINT
environment variable.
Tracing
Flipt supports distributed tracing via the OpenTelemetry project using the OTLP protocol.
OTLP
OTLP supports additional configuration such as specifying the protocol to use (gRPC or HTTP) as well as providing custom headers to send with the request.
Custom headers can be used to provide authentication information to the collector which may be required if you are using a hosted collector such as NewRelic, Datadog, or Honeycomb.
OpenTelemetry OTLP tracing is configured via the default OpenTelemetry Environment Variables. See the OpenTelemetry Environment Variables documentation for more details.
For example, to configure the OTLP tracing endpoint, you can set the OTEL_EXPORTER_OTLP_TRACES_ENDPOINT
environment variable.
To configure the OTLP headers, you can set the OTEL_EXPORTER_OTLP_TRACES_HEADERS
environment variable.
Environment Variables
Flipt supports all OTLP environment variables that are part of the OTLP spec.
Here are a few of the most commonly used environment variables supported by Flipt:
OTEL_SERVICE_NAME
- Sets the value of theservice.name
resource attribute (default:flipt
)OTEL_RESOURCE_ATTRIBUTES
- Key-value pairs to be used as resource attributes.OTEL_EXPORTER_OTLP_ENDPOINT
- The OTLP endpoint to any signal data (metrics, traces, logs) toOTEL_EXPORTER_OTLP_HEADERS
- Key-value pairs to be used as OTLP headers for any signal data (metrics, traces, logs).OTEL_EXPORTER_OTLP_PROTOCOL
- The protocol to use for the OTLP endpoint (grpc, http/protobuf, http/json)