Configuring Loki

    Configuration examples can be found in the Configuration Examples document.

    To specify which configuration file to load, pass the -config.file flag at thecommand line. The file is written in YAML format,defined by the scheme below. Brackets indicate that a parameter is optional. Fornon-list parameters the value is set to the specified default.

    Generic placeholders are defined as follows:

    • <boolean>: a boolean that can take the values true or false
    • <int>: any integer matching the regular expression [1-9]+[0-9]*
    • <duration>: a duration matching the regular expression [0-9]+(ms|[smhdwy])
    • <labelname>: a string matching the regular expression [a-zA-Z_][a-zA-Z0-9_]*
    • <labelvalue>: a string of unicode characters
    • <filename>: a valid path relative to current working directory or an absolute path.
    • <host>: a valid string consisting of a hostname or IP followed by an optional port number
    • <string>: a regular string
    • <secret>: a regular string that is a secret, such as a password

    Supported contents and default values of loki.yaml:

    server_config

    The server_config block configures Promtail’s behavior as an HTTP server:

    1. # HTTP server listen host
    2. [http_listen_host: <string>]
    4. # HTTP server listen port
    5. [http_listen_port: <int> | default = 80]
    7. # gRPC server listen host
    8. [grpc_listen_host: <string>]
    10. # gRPC server listen port
    11. [grpc_listen_port: <int> | default = 9095]
    13. # Register instrumentation handlers (/metrics, etc.)
    14. [register_instrumentation: <boolean> | default = true]
    16. # Timeout for graceful shutdowns
    17. [graceful_shutdown_timeout: <duration> | default = 30s]
    19. # Read timeout for HTTP server
    20. [http_server_read_timeout: <duration> | default = 30s]
    22. # Write timeout for HTTP server
    23. [http_server_write_timeout: <duration> | default = 30s]
    25. # Idle timeout for HTTP server
    26. [http_server_idle_timeout: <duration> | default = 120s]
    28. # Max gRPC message size that can be received
    29. [grpc_server_max_recv_msg_size: <int> | default = 4194304]
    31. # Max gRPC message size that can be sent
    32. [grpc_server_max_send_msg_size: <int> | default = 4194304]
    34. # Limit on the number of concurrent streams for gRPC calls (0 = unlimited)
    35. [grpc_server_max_concurrent_streams: <int> | default = 100]
    37. # Log only messages with the given severity or above. Supported values [debug,
    38. # info, warn, error]
    39. [log_level: <string> | default = "info"]
    41. # Base path to server all API routes from (e.g., /v1/).
    42. [http_path_prefix: <string>]

    querier_config

    The querier_config block configures the Loki Querier.

    1. # Timeout when querying ingesters or storage during the execution of a
    2. # query request.
    3. [query_timeout: <duration> | default = 1m]
    5. # Limit of the duration for which live tailing requests should be
    6. # served.
    7. [tail_max_duration: <duration> | default = 1h]
    9. # Configuration options for the LogQL engine.
    10. engine:
    11.   # Timeout for query execution
    12.   [timeout: <duration> | default = 3m]
    14.   # The maximum amount of time to look back for log lines. Only
    15.   # applicable for instant log queries.
    16.   [max_look_back_period: <duration> | default = 30s]
    1. # Configures how connections are pooled
    2. pool_config:
    3.   # Whether or not to do health checks.
    4.   [health_check_ingesters: <boolean> | default = false]
    6.   # How frequently to clean up clients for servers that have gone away after
    7.   # a health check.
    8.   [client_cleanup_period: <duration> | default = 15s]
    10.   # How quickly a dead client will be removed after it has been detected
    11.   # to disappear. Set this to a value to allow time for a secondary
    12.   # health check to recover the missing client.
    13.   [remotetimeout: <duration>]
    15. # The remote request timeout on the client side.
    16. [remote_timeout: <duration> | default = 5s]
    18. # Configures how the gRPC connection to ingesters work as a
    19. # client.
    20. [grpc_client_config: <grpc_client_config>]

    The grpc_client_config block configures a client connection to a gRPC service.

    1. # The maximum size in bytes the client can recieve
    2. [max_recv_msg_size: <int> | default = 104857600]
    4. # The maximum size in bytes the client can send
    5. [max_send_msg_size: <int> | default = 16777216]
    7. # Whether or not messages should be compressed
    8. [use_gzip_compression: <bool> | default = false]
    10. # Rate limit for gRPC client. 0 is disabled
    11. [rate_limit: <float> | default = 0]
    13. # Rate limit burst for gRPC client.
    14. [rate_limit_burst: <int> | default = 0]
    16. # Enable backoff and retry when a rate limit is hit.
    17. [backoff_on_ratelimits: <bool> | default = false]
    19. # Configures backoff when enbaled.
    20. backoff_config:
    21.   # Minimum delay when backing off.
    22.   [minbackoff: <duration> | default = 100ms]
    24.   # The maximum delay when backing off.
    25.   [maxbackoff: <duration> | default = 10s]
    27.   # Number of times to backoff and retry before failing.
    28.   [maxretries: <int> | default = 10]

    ingester_config

    The ingester_config block configures Ingesters.

    lifecycler_config

    The lifecycler_config is used by the Ingester to control how that ingesterregisters itself into the ring and manages its lifecycle during its stay in thering.

    1. # Configures the ring the lifecycler connects to
    2. [ring: <ring_config>]
    4. # The number of tokens the lifecycler will generate and put into the ring if
    5. # it joined without transfering tokens from another lifecycler.
    6. [num_tokens: <int> | default = 128]
    8. # Period at which to heartbeat to the underlying ring.
    9. [heartbeat_period: <duration> | default = 5s]
    11. # How long to wait to claim tokens and chunks from another member when
    12. # that member is leaving. Will join automatically after the duration expires.
    13. [join_after: <duration> | default = 0s]
    15. # Minimum duration to wait before becoming ready. This is to work around race
    16. # conditions with ingesters exiting and updating the ring.
    17. [min_ready_duration: <duration> | default = 1m]
    19. # Store tokens in a normalised fashion to reduce the number of allocations.
    20. [normalise_tokens: <boolean> | default = false]
    22. # Name of network interfaces to read addresses from.
    23. interface_names:
    24.   - [<string> ... | default = ["eth0", "en0"]]
    26. # Duration to sleep before exiting to ensure metrics are scraped.
    27. [final_sleep: <duration> | default = 30s]

    The ring_config is used to discover and connect to Ingesters.

    1. kvstore:
    2.   # The backend storage to use for the ring. Supported values are
    3.   # consul, etcd, inmemory
    4.   store: <string>
    6.   # The prefix for the keys in the store. Should end with a /.
    7.   [prefix: <string> | default = "collectors/"]
    9.   # Configuration for a Consul client. Only applies if store
    10.   # is "consul"
    11.   consul:
    12.     # The hostname and port of Consul.
    13.     [host: <string> | duration = "localhost:8500"]
    15.     # The ACL Token used to interact with Consul.
    16.     [acltoken: <string>]
    18.     [httpclienttimeout: <duration> | default = 20s]
    20.     # Whether or not consistent reads to Consul are enabled.
    21.     [consistentreads: <boolean> | default = true]
    23.   # Configuration for an ETCD v3 client. Only applies if
    24.   # store is "etcd"
    25.   etcd:
    26.     # The ETCD endpoints to connect to.
    27.     endpoints:
    28.       - <string>
    30.     # The Dial timeout for the ETCD connection.
    31.     [dial_tmeout: <duration> | default = 10s]
    33.     # The maximum number of retries to do for failed ops to ETCD.
    34.     [max_retries: <int> | default = 10]
    36. # The heartbeart timeout after which ingesters are skipped for
    37. # reading and writing.
    38. [heartbeart_timeout: <duration> | default = 1m]
    40. # The number of ingesters to write to and read from. Must be at least
    41. # 1.
    42. [replication_factor: <int> | default = 3]

    storage_config

    The storage_config block configures one of many possible stores for both theindex and chunks. Which configuration is read from depends on the schema_configblock and what is set for the store value.

    1. # Configures storing chunks in AWS. Required options only required when aws is
    2. # present.
    3. aws:
    4.   # S3 or S3-compatible URL to connect to. If only region is specified as a
    5.   # host, the proper endpoint will be deduced. Use inmemory:///<bucket-name> to
    6.   # use a mock in-memory implementation.
    7.   s3: <string>
    9.   # Set to true to force the request to use path-style addressing
    10.   [s3forcepathstyle: <boolean> | default = false]
    12.   # Configure the DynamoDB conection
    13.   dynamodbconfig:
    14.     # URL for DynamoDB with escaped Key and Secret encoded. If only region is specified as a
    15.     # host, the proper endpoint will be deduced. Use inmemory:///<bucket-name> to
    16.     # use a mock in-memory implementation.
    17.     dynamodb: <string>
    19.     # DynamoDB table management requests per-second limit.
    20.     [apilimit: <float> | default = 2.0]
    22.     # DynamoDB rate cap to back off when throttled.
    23.     [throttlelimit: <float> | default = 10.0]
    25.     # Application Autoscaling endpoint URL with escaped Key and Secret
    26.     # encoded.
    27.     [applicationautoscaling: <string>]
    29.     # Metics-based autoscaling configuration.
    30.     metrics:
    31.       # Use metrics-based autoscaling via this Prometheus query URL.
    32.       [url: <string>]
    34.       # Queue length above which we will scale up capacity.
    35.       [targetqueuelen: <int> | default = 100000]
    37.       # Scale up capacity by this multiple
    38.       [scaleupfactor: <float64> | default = 1.3]
    40.       # Ignore throttling below this level (rate per second)
    41.       [minthrottling: <float64> | default = 1]
    43.       # Query to fetch ingester queue length
    44.       [queuelengthquery: <string> | default = "sum(avg_over_time(cortex_ingester_flush_queue_length{job="cortex/ingester"}[2m]))"]
    46.       # Query to fetch throttle rates per table
    47.       [throttlequery: <string> | default = "sum(rate(cortex_dynamo_throttled_total{operation="DynamoDB.BatchWriteItem"}[1m])) by (table) > 0"]
    49.       # Quer to fetch write capacity usage per table
    50.       [usagequery: <string> | default = "sum(rate(cortex_dynamo_consumed_capacity_total{operation="DynamoDB.BatchWriteItem"}[15m])) by (table) > 0"]
    52.       # Query to fetch read capacity usage per table
    53.       [readusagequery: <string> | default = "sum(rate(cortex_dynamo_consumed_capacity_total{operation="DynamoDB.QueryPages"}[1h])) by (table) > 0"]
    55.       # Query to fetch read errors per table
    56.       [readerrorquery: <string> | default = "sum(increase(cortex_dynamo_failures_total{operation="DynamoDB.QueryPages",error="ProvisionedThroughputExceededException"}[1m])) by (table) > 0"]
    58.     # Number of chunks to group together to parallelise fetches (0 to disable)
    59.     [chunkgangsize: <int> | default = 10]
    61.     # Max number of chunk get operations to start in parallel.
    62.     [chunkgetmaxparallelism: <int> | default = 32]
    64. # Configures storing chunks in Bigtable. Required fields only required
    65. # when bigtable is defined in config.
    66. bigtable:
    67.   # BigTable project ID
    68.   project: <string>
    70.   # BigTable instance ID
    71.   instance: <string>
    73.   # Configures the gRPC client used to connect to Bigtable.
    74.   [grpc_client_config: <grpc_client_config>]
    76. # Configures storing index in GCS. Required fields only required
    77. # when gcs is defined in config.
    78. gcs:
    79.   # Name of GCS bucket to put chunks in.
    80.   bucket_name: <string>
    82.   # The size of the buffer that the GCS client uses for each PUT request. 0
    83.   # to disable buffering.
    84.   [chunk_buffer_size: <int> | default = 0]
    86.   # The duration after which the requests to GCS should be timed out.
    87.   [request_timeout: <duration> | default = 0s]
    89. # Configures storing chunks in Cassandra
    90. cassandra:
    91.   # Comma-separated hostnames or IPs of Cassandra instances
    92.   addresses: <string>
    94.   # Port that cassandra is running on
    95.   [port: <int> | default = 9042]
    97.   # Keyspace to use in Cassandra
    98.   keyspace: <string>
    100.   # Consistency level for Cassandra
    101.   [consistency: <string> | default = "QUORUM"]
    103.   # Replication factor to use in Cassandra.
    104.   [replication_factor: <int> | default = 1]
    106.   # Instruct the Cassandra driver to not attempt to get host
    107.   # info from the system.peers table.
    108.   [disable_initial_host_lookup: <bool> | default = false]
    110.   # Use SSL when connecting to Cassandra instances.
    111.   [SSL: <boolean> | default = false]
    113.   # Require SSL certificate validation when SSL is enabled.
    114.   [host_verification: <bool> | default = true]
    116.   # Path to certificate file to verify the peer when SSL is
    117.   # enabled.
    118.   [CA_path: <string>]
    120.   # Enable password authentication when connecting to Cassandra.
    121.   [auth: <bool> | default = false]
    123.   # Username for password authentication when auth is true.
    124.   [username: <string>]
    126.   # Password for password authentication when auth is true.
    127.   [password: <string>]
    129.   # Timeout when connecting to Cassandra.
    130.   [timeout: <duration> | default = 600ms]
    132.   # Initial connection timeout during initial dial to server.
    133.   [connect_timeout: <duration> | default = 600ms]
    135. # Configures storing index in BoltDB. Required fields only
    136. # required when boltdb is present in config.
    137. boltdb:
    138.   # Location of BoltDB index files.
    139.   directory: <string>
    141. # Configures storing the chunks on the local filesystem. Required
    142. # fields only required when filesystem is present in config.
    143. filesystem:
    144.   # Directory to store chunks in.
    145.   directory: <string>
    147. # Cache validity for active index entries. Should be no higher than
    148. # the chunk_idle_period in the ingester settings.
    149. [indexcachevalidity: <duration> | default = 5m]
    151. # The maximum number of chunks to fetch per batch.
    152. [max_chunk_batch_size: <int> | default = 50]
    154. # Config for how the cache for index queries should
    155. # be built.
    156. index_queries_cache_config: <cache_config>

    cache_config

    The cache_config block configures how Loki will cache requests, chunks, andthe index to a backing cache store.

    1. # Enable in-memory cache.
    2. [enable_fifocache: <boolean>]
    4. # The default validity of entries for caches unless overriden.
    5. # "defaul" is correct.
    6. [defaul_validity: <duration>]
    8. # Configures the background cache when memcached is used.
    9. background:
    10.   # How many goroutines to use to write back to memcached.
    11.   [writeback_goroutines: <int> | default = 10]
    13.   [writeback_buffer: <int> = 10000]
    15. # Configures memcached settings.
    16. memcached:
    17.   # Configures how long keys stay in memcached.
    18.   expiration: <duration>
    20.   # Configures how many keys to fetch in each batch request.
    21.   batch_size: <int>
    23.   # Maximum active requests to memcached.
    24.   [parallelism: <int> | default = 100]
    27. memcached_client:
    28.   # The hostname to use for memcached services when caching chunks. If
    29.   # empty, no memcached will be used. A SRV lookup will be used.
    30.   [host: <string>]
    32.   # SRV service used to discover memcached servers.
    33.   [service: <string> | default = "memcached"]
    35.   # Maximum time to wait before giving up on memcached requests.
    36.   [timeout: <duration> | default = 100ms]
    38.   # The maximum number of idle connections in the memcached client
    39.   # pool.
    40.   [max_idle_conns: <int> | default = 100]
    42.   # The period with which to poll the DNS for memcached servers.
    43.   [update_interval: <duration> | default = 1m]
    45.   # Whether or not to use a consistent hash to discover multiple memcached
    46.   # servers.
    47.   [consistent_hash: <bool>]
    49. fifocache:
    50.   # Number of entries to cache in-memory.
    51.   [size: <int> | default = 0]
    53.   # The expiry duration for the in-memory cache.
    54.   [validity: <duration> | default = 0s]

    schema_config

    The schema_config block configures schemas from given dates.

    1. # The configuration for chunk index schemas.
    2. configs:
    3.   - [<period_config>]

    The period_config block configures what index schemas should be usedfor from specific time periods.

    1. # The date of the first day that index buckets should be created. Use
    2. # a date in the past if this is your only period_config, otherwise
    3. # use a date when you want the schema to switch over.
    4. [from: <daytime>]
    6. # store and object_store below affect which <storage_config> key is
    7. # used.
    9. # Which store to use for the index. Either cassandra, bigtable, dynamodb, or
    10. # boltdb
    11. store: <string>
    13. # Which store to use for the chunks. Either gcs, s3, inmemory, filesystem,
    14. # cassandra. If omitted, defaults to same value as store.
    15. [object_store: <string>]
    17. # The schema to use. Set to v9 or v10.
    18. schema: <string>
    20. # Configures how the index is updated and stored.
    21. index:
    22.   # Table prefix for all period tables.
    23.   prefix: <string>
    24.   # Table period.
    25.   [period: <duration> | default = 168h]
    26.   # A map to be added to all managed tables.
    27.   tags:
    28.     [<string>: <string> ...]
    30. # Configured how the chunks are updated and stored.
    31. chunks:
    32.   # Table prefix for all period tables.
    33.   prefix: <string>
    34.   # Table period.
    35.   [period: <duration> | default = 168h]
    36.   # A map to be added to all managed tables.
    37.   tags:
    38.     [<string>: <string> ...]
    40. # How many shards will be created. Only used if schema is v10.
    41. [row_shards: <int> | default = 16]

    Where daytime is a value in the format of yyyy-mm-dd like 2006-01-02.

    limits_config

    The limits_config block configures global and per-tenant limits for ingestinglogs in Loki.

    1. # Per-user ingestion rate limit in samples per second.
    2. [ingestion_rate: <float> | default = 25000]
    4. # Per-user allowed ingestion burst size (in number of samples).
    5. [ingestion_burst_size: <int> | default = 50000]
    7. # Whether or not, for all users, samples with external labels
    8. # identifying replicas in an HA Prometheus setup will be handled.
    9. [accept_ha_samples: <boolean> | default = false]
    11. # Prometheus label to look for in samples to identify a
    12. # Prometheus HA cluster.
    13. [ha_cluster_label: <string> | default = "cluster"]
    15. # Prometheus label to look for in samples to identify a Prometheus HA
    16. # replica.
    17. [ha_replica_label: <string> | default = "__replica__"]
    19. # Maximum length of a label name.
    20. [max_label_name_length: <int> | default = 1024]
    22. # Maximum length of a label value.
    23. [max_label_value_length: <int> | default = 2048]
    25. # Maximum number of label names per series.
    26. [max_label_names_per_series: <int> | default = 30]
    28. # Whether or not old samples will be rejected.
    29. [reject_old_samples: <bool> | default = false]
    31. # Maximum accepted sample age before rejecting.
    32. [reject_old_samples_max_age: <duration> | default = 336h]
    34. # Duration for a table to be created/deleted before/after it's
    35. # needed. Samples won't be accepted before this time.
    36. [creation_grace_period: <duration> | default = 10m]
    38. # Enforce every sample has a metric name.
    39. [enforce_metric_name: <boolean> | default = true]
    41. # Maximum number of samples that a query can return.
    42. [max_samples_per_query: <int> | default = 1000000]
    44. # Maximum number of active series per user.
    45. [max_series_per_user: <int> | default = 5000000]
    47. # Maximum number of active series per metric name.
    48. [max_series_per_metric: <int> | default = 50000]
    50. # Maximum number of chunks that can be fetched by a single query.
    51. [max_chunks_per_query: <int> | default = 2000000]
    53. # The limit to length of chunk store queries. 0 to disable.
    54. [max_query_length: <duration> | default = 0]
    56. # Maximum number of queries that will be scheduled in parallel by the
    57. # frontend.
    58. [max_query_parallelism: <int> | default = 14]
    60. # Cardinality limit for index queries
    61. [cardinality_limit: <int> | default = 100000]
    63. # Filename of per-user overrides file
    64. [per_tenant_override_config: <string>]
    66. # Period with which to reload the overrides file if configured.
    67. [per_tenant_override_period: <duration> | default = 10s]

    The table_manager_config block configures how the table manager operatesand how to provision tables when DynamoDB is used as the backing store.

    1. # Master 'off-switch' for table capacity updates, e.g. when troubleshooting
    2. [throughput_updates_disabled: <boolean> | default = false]
    4. # Master 'on-switch' for table retention deletions
    5. [retention_deletes_enabled: <boolean> | default = false]
    7. # How far back tables will be kept before they are deleted. 0s disables
    8. # deletion. The retention period must be a multiple of the index / chunks
    9. # table "period" (see period_config).
    10. [retention_period: <duration> | default = 0s]
    12. # Period with which the table manager will poll for tables.
    13. [dynamodb_poll_interval: <duration> | default = 2m]
    15. # duration a table will be created before it is needed.
    16. [creation_grace_period: <duration> | default = 10m]
    18. # Configures management of the index tables for DynamoDB.
    19. index_tables_provisioning: <provision_config>
    21. # Configures management of the chunk tables for DynamoDB.
    22. chunk_tables_provisioning: <provision_config>

    provision_config

    The provision_config block configures provisioning capacity for DynamoDB.

    auto_scaling_config

    1. # Whether or not autoscaling should be enabled.
    2. [enabled: <boolean>: default = false]
    4. # AWS AutoScaling role ARN
    5. [role_arn: <string>]
    7. # DynamoDB minimum provision capacity.
    8. [min_capacity: <int> | default = 3000]
    10. # DynamoDB maximum provision capacity.
    11. [max_capacity: <int> | default = 6000]
    13. # DynamoDB minimum seconds between each autoscale up.
    14. [out_cooldown: <int> | default = 1800]
    16. # DynamoDB minimum seconds between each autoscale down.
    17. [in_cooldown: <int> | default = 1800]
    19. [target: <float> | default = 80]