> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flipt.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Observability

> This document describes how to configure Flipt's observability mechanisms including metrics, logging, and tracing.

## Metrics

### Prometheus

Flipt exposes [Prometheus](https://prometheus.io/) metrics by default at the `/metrics`
HTTP endpoint. To see which metrics are currently supported, point your browser
to `FLIPT_HOST/metrics` (ex: `localhost:8080/metrics`).

You should see a bunch of metrics being recorded such as:

```yaml theme={null}
flipt_cache_hit_total{cache="memory",type="flag"} 1
flipt_cache_miss_total{cache="memory",type="flag"} 1
---
go_gc_duration_seconds{quantile="0"} 8.641e-06
go_gc_duration_seconds{quantile="0.25"} 2.499e-05
go_gc_duration_seconds{quantile="0.5"} 3.5359e-05
go_gc_duration_seconds{quantile="0.75"} 6.6594e-05
go_gc_duration_seconds{quantile="1"} 0.00026651 go_gc_duration_seconds_sum
0.000402094 go_gc_duration_seconds_count 5
```

An [example](https://github.com/flipt-io/flipt/tree/main/examples/metrics)
showing how to set up Flipt with Prometheus can be found in the GitHub repository.

<Tip>
  You can disable the Prometheus metrics collection by setting the
  `metrics.enabled` configuration option to `false`.
</Tip>

### OTLP

Flipt supports sending metrics to an [OTLP](https://opentelemetry.io/docs/concepts/data-collection/) collector.

OTLP supports additional configuration such as specifying the protocol to use (gRPC or HTTP) as well as providing custom headers to send with the request.

Custom headers can be used to provide authentication information to the collector which may be required if you are using a hosted collector such as [NewRelic](https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/get-started/opentelemetry-set-up-your-app/), [DataDog](https://docs.datadoghq.com/opentelemetry/otlp_ingest_in_the_agent/?tab=host), or [Honeycomb](https://docs.honeycomb.io/getting-data-in/opentelemetry-overview/#instrumenting-with-opentelemetry).

These can be configured via the `metrics.otlp` configuration section.

```yaml theme={null}
metrics:
  enabled: true
  exporter: "otlp"
  otlp:
    endpoint: "https://{your-collector-url}"
    headers:
      "X-Some-Header": "some-value"
```

### Dashboards

<img src="https://mintcdn.com/flipt/O1Bfdh4QkYyvK5vl/v1/images/configuration/grafana_flag_dashboard.png?fit=max&auto=format&n=O1Bfdh4QkYyvK5vl&q=85&s=31d9ac83cd9f48ca5e13d924bf1593b0" alt="Grafana Dashboard" width="3354" height="1926" data-path="v1/images/configuration/grafana_flag_dashboard.png" />

We provide a set of [Grafana](https://grafana.com/) dashboards that you can use to visualize the metrics collected by Flipt, including both server health and flag evaluation metrics.

You can find the dashboards in our [grafana-dashboards](https://github.com/flipt-io/grafana-dashboards) repository.

## Logging

Flipt writes logs to STDOUT in two formats:

* [JSON](#json)
* [Console](#console)

The format can be configured via the `log.encoding` configuration option.

```yaml theme={null}
log:
  encoding: json
```

For production deployments, we recommend using the JSON format as it's easier to parse and ingest into log aggregation systems such as Elasticsearch, Splunk, Loki, or Datadog.

We've prepared an [example](https://github.com/flipt-io/flipt/tree/main/examples/audit/log) showing how to set up Flipt with Grafana Loki and Promtail to ingest and query logs.

### JSON

```json theme={null}
{
  "L": "INFO",
  "T": "2024-01-20T21:59:49-05:00",
  "M": "finished unary call with code OK",
  "server": "grpc",
  "grpc.start_time": "2024-01-20T21:59:49-05:00",
  "system": "grpc",
  "span.kind": "server",
  "grpc.service": "flipt.evaluation.EvaluationService",
  "grpc.method": "Boolean",
  "peer.address": "127.0.0.1:52635",
  "grpc.code": "OK",
  "grpc.time_ms": 0.146
}
```

#### Log Key Descriptions

* `L`: Level (log level). Possible values include: debug, info, warn, error, fatal, and panic.
* `T`: Timestamp. The timestamp is in ISO 8601 format, widely used for representing date and time. It includes the date, time, and time zone information. For example, "2024-01-20T21:59:49-05:00" represents the date and time in the Eastern Time Zone (UTC-5).
* `M`: Message. The message describes the log event. It can include information about the operation, errors encountered, or other relevant details.

### Console

```text theme={null}
2024-01-20T22:04:18-05:00	INFO	finished unary call with code OK	{"server": "grpc", "grpc.start_time": "2024-01-20T22:04:18-05:00", "system": "grpc", "span.kind": "server", "grpc.service": "flipt.evaluation.EvaluationService", "grpc.method": "Boolean", "peer.address": "127.0.0.1:53714", "grpc.code": "OK", "grpc.time_ms": 0.373}
```

More information about the available configuration options can be found in the [Logging configuration](/v1/configuration/overview#logging) section.

## Tracing

Flipt supports distributed tracing via the [OpenTelemetry](https://opentelemetry.io/) project.

Currently, we support the following tracing backends:

* [Jaeger](https://www.jaegertracing.io/)
* [Zipkin](https://zipkin.io/)
* [OTLP](https://opentelemetry.io/docs/reference/specification/protocol/)

Enable tracing via the values described in the [Tracing configuration](/v1/configuration/overview#tracing) and point Flipt to your configured collector to record spans.

[Examples](https://github.com/flipt-io/flipt/tree/main/examples/tracing) showing how to set up Flipt with each of the supported tracing backends can be found in the main GitHub repository .

### OTLP

<img src="https://mintcdn.com/flipt/O1Bfdh4QkYyvK5vl/v1/images/configuration/datadog_otlp.png?fit=max&auto=format&n=O1Bfdh4QkYyvK5vl&q=85&s=718929362d720ff4120c1b2a008c0408" alt="Datadog OTLP" width="1691" height="746" data-path="v1/images/configuration/datadog_otlp.png" />

OTLP supports additional configuration such as specifying the protocol to use (gRPC or HTTP) as well as providing custom headers to send with the request.

Custom headers can be used to provide authentication information to the collector which may be required if you are using a hosted collector such as [NewRelic](https://docs.newrelic.com/docs/more-integrations/open-source-telemetry-integrations/opentelemetry/get-started/opentelemetry-set-up-your-app/), [Datadog](https://docs.datadoghq.com/opentelemetry/otlp_ingest_in_the_agent/?tab=host), or [Honeycomb](https://docs.honeycomb.io/getting-data-in/opentelemetry-overview/#instrumenting-with-opentelemetry).

These can be configured via the `tracing.otlp` configuration section.

```yaml theme={null}
tracing:
  enabled: true
  exporter: "otlp"
  otlp:
    endpoint: "https://{your-collector-url}"
    headers:
      "X-Some-Header": "some-value"
```

#### Environment Variables

Flipt supports OTLP specific resource environment variables that are part of the [OTLP spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/).

The following environment variables are supported:

* `OTEL_SERVICE_NAME` - Sets the value of the `service.name` resource attribute (default: `flipt`)
* `OTEL_RESOURCE_ATTRIBUTES` - Key-value pairs to be used as [resource attributes](https://opentelemetry.io/docs/specs/semconv/resource/#semantic-attributes-with-dedicated-environment-variable).
