Configuration

    The visual editor can assist inbuilding routing trees.

    To view all available command-line flags, run .

    Alertmanager can reload its configuration at runtime. If the new configurationis not well-formed, the changes will not be applied and an error is logged.A configuration reload is triggered by sending a SIGHUP to the process orsending a HTTP POST request to the /-/reload endpoint.

    To specify which configuration file to load, use the —config.file flag.

    The file is written in the ,defined by the scheme described below.Brackets indicate that a parameter is optional. For non-list parameters thevalue is set to the specified default.

    Generic placeholders are defined as follows:

    • <duration>: a duration matching the regular expression [0-9]+(ms|[smhdwy])
    • <labelname>: a string matching the regular expression [a-zA-Z][a-zA-Z0-9]*
    • <labelvalue>: a string of unicode characters
    • <filepath>: a valid path in the current working directory
    • <boolean>: a boolean that can take the values true or false
    • <string>: a regular string
    • <secret>: a regular string that is a secret, such as a password
    • <tmpl_string>: a string which is template-expanded before usage
    • <tmpl_secret>: a string which is template-expanded before usage that is a secretThe other placeholders are specified separately.

    A provided valid example fileshows usage in context.

    The global configuration specifies parameters that are valid in all otherconfiguration contexts. They also serve as defaults for other configurationsections.

    1. global:
    2.   # ResolveTimeout is the time after which an alert is declared resolved
    3.   # if it has not been updated.
    4.   [ resolve_timeout:  | default = 5m ]
    6.   # The default SMTP From header field.
    7.   [ smtp_from: <tmpl_string> ]
    8.   # The default SMTP smarthost used for sending emails, including port number.
    9.   # Port number usually is 25, or 587 for SMTP over TLS (sometimes referred to as STARTTLS).
    10.   # Example: smtp.example.org:587
    11.   [ smtp_smarthost:  ]
    12.   # The default hostname to identify to the SMTP server.
    13.   [ smtp_hello: <string> | default = "localhost" ]
    14.   # SMTP Auth using CRAM-MD5, LOGIN and PLAIN. If empty, Alertmanager doesn't authenticate to the SMTP server.
    15.   [ smtp_auth_username:  ]
    16.   # SMTP Auth using LOGIN and PLAIN.
    17.   [ smtp_auth_password: <secret> ]
    18.   # SMTP Auth using PLAIN.
    19.   [ smtp_auth_identity:  ]
    20.   # SMTP Auth using CRAM-MD5. 
    21.   [ smtp_auth_secret: <secret> ]
    22.   # The default SMTP TLS requirement.
    23.   [ smtp_require_tls: <bool> | default = true ]
    25.   # The API URL to use for Slack notifications.
    26.   [ slack_api_url:  ]
    27.   [ victorops_api_key: <secret> ]
    28.   [ victorops_api_url:  | default = "https://alert.victorops.com/integrations/generic/20131114/alert/" ]
    29.   [ pagerduty_url: <string> | default = "https://events.pagerduty.com/v2/enqueue" ]
    30.   [ opsgenie_api_key:  ]
    31.   [ opsgenie_api_url: <string> | default = "https://api.opsgenie.com/" ]
    32.   [ hipchat_api_url:  | default = "https://api.hipchat.com/" ]
    33.   [ hipchat_auth_token: <secret> ]
    34.   [ wechat_api_url:  | default = "https://qyapi.weixin.qq.com/cgi-bin/" ]
    35.   [ wechat_api_secret: <secret> ]
    36.   [ wechat_api_corp_id:  ]
    38.   # The default HTTP client configuration
    39.   [ http_config: <http_config> ]
    41. # Files from which custom notification template definitions are read.
    42. # The last component may use a wildcard matcher, e.g. 'templates/*.tmpl'.
    43. templates:
    44.   [ -  ... ]
    46. # The root node of the routing tree.
    47. route: <route>
    49. # A list of notification receivers.
    50. receivers:
    51.   -  ...
    53. # A list of inhibition rules.
    54. inhibit_rules:
    55.   [ - <inhibit_rule> ... ]

    <route>

    A route block defines a node in a routing tree and its children. Its optionalconfiguration parameters are inherited from its parent node if not set.

    1. [ receiver: <string> ]
    2. # The labels by which incoming alerts are grouped together. For example,
    3. # multiple alerts coming in for cluster=A and alertname=LatencyHigh would
    4. # be batched into a single group.
    5. #
    6. # To aggregate by all possible labels use the special value '...' as the sole label name, for example:
    7. # group_by: ['...'] 
    8. # This effectively disables aggregation entirely, passing through all 
    9. # alerts as-is. This is unlikely to be what you want, unless you have 
    10. # a very low alert volume or your upstream notification system performs 
    11. # its own grouping.
    12. [ group_by: '[' , ... ']' ]
    14. # Whether an alert should continue matching subsequent sibling nodes.
    15. [ continue: <boolean> | default = false ]
    17. # A set of equality matchers an alert has to fulfill to match the node.
    18. match:
    19.   [ : <labelvalue>, ... ]
    21. # A set of regex-matchers an alert has to fulfill to match the node.
    22. match_re:
    23.   [ : <regex>, ... ]
    25. # How long to initially wait to send a notification for a group
    26. # of alerts. Allows to wait for an inhibiting alert to arrive or collect
    27. # more initial alerts for the same group. (Usually ~0s to few minutes.)
    28. [ group_wait: <duration> | default = 30s ]
    30. # How long to wait before sending a notification about new alerts that
    31. # are added to a group of alerts for which an initial notification has
    32. # already been sent. (Usually ~5m or more.)
    33. [ group_interval:  | default = 5m ]
    35. # How long to wait before sending a notification again if it has already
    36. # been sent successfully for an alert. (Usually ~3h or more).
    37. [ repeat_interval: <duration> | default = 4h ]
    39. # Zero or more child routes.
    40. routes:
    41.   [ -  ... ]
    1. # The root route with all parameters, which are inherited by the child
    2. # routes if they are not overwritten.
    3. route:
    4.   receiver: 'default-receiver'
    5.   group_wait: 30s
    6.   group_interval: 5m
    7.   repeat_interval: 4h
    8.   group_by: [cluster, alertname]
    9.   # All alerts that do not match the following child routes
    10.   # will remain at the root node and be dispatched to 'default-receiver'.
    11.   routes:
    12.   # All alerts with service=mysql or service=cassandra
    13.   # are dispatched to the database pager.
    14.   - receiver: 'database-pager'
    15.     group_wait: 10s
    16.     match_re:
    17.       service: mysql|cassandra
    18.   # All alerts with the team=frontend label match this sub-route.
    19.   # They are grouped by product and environment rather than cluster
    20.   # and alertname.
    21.   - receiver: 'frontend-pager'
    22.     match:
    23.       team: frontend

    <inhibit_rule>

    An inhibition rule mutes an alert (target) matching a set of matcherswhen an alert (source) exists that matches another set of matchers.Both target and source alerts must have the same label values for the label names in the equal list.

    Semantically, a missing label and a label with an empty value are the samething. Therefore, if all the label names listed in equal are missing fromboth the source and target alerts, the inhibition rule will apply.

    To prevent an alert from inhibiting itself, an alert that matches both thetarget and the source side of a rule cannot be inhibited by alerts for whichthe same is true (including itself). However, we recommend to choose target andsource matchers in a way that alerts never match both sides. It is much easierto reason about and does not trigger this special case.

    1. # Matchers that have to be fulfilled in the alerts to be muted.
    2. target_match:
    3.   [ : <labelvalue>, ... ]
    4. target_match_re:
    7. # Matchers for which one or more alerts have to exist for the
    8. # inhibition to take effect.
    9. source_match:
    10.   [ : <labelvalue>, ... ]
    11. source_match_re:
    12.   [ : <regex>, ... ]
    14. # Labels that must have an equal value in the source and target
    15. # alert for the inhibition to take effect.
    16. [ equal: '[' <labelname>, ... ']' ]

    <http_config>

    A http_config allows configuring the HTTP client that the receiver uses tocommunicate with HTTP-based API services.

    1. # Note that `basic_auth`, `bearer_token` and `bearer_token_file` options are
    2. # mutually exclusive.
    4. # Sets the `Authorization` header with the configured username and password.
    5. # password and password_file are mutually exclusive.
    6. basic_auth:
    7.   [ username: <string> ]
    8.   [ password:  ]
    9.   [ password_file: <string> ]
    11. # Sets the `Authorization` header with the configured bearer token.
    12. [ bearer_token:  ]
    14. # Sets the `Authorization` header with the bearer token read from the configured file.
    15. [ bearer_token_file: <filepath> ]
    17. # Configures the TLS settings.
    18. tls_config:
    19.   [  ]
    21. # Optional proxy URL.
    22. [ proxy_url: <string> ]

    <tls_config>

    A tls_config allows configuring TLS connections.

    1. # CA certificate to validate the server certificate with.
    2. [ ca_file: <filepath> ]
    4. # Certificate and key files for client cert authentication to the server.
    5. [ cert_file:  ]
    6. [ key_file: <filepath> ]
    8. # ServerName extension to indicate the name of the server.
    9. # http://tools.ietf.org/html/rfc4366#section-3.1
    10. [ server_name:  ]
    12. # Disable validation of the server certificate.
    13. [ insecure_skip_verify: <boolean> | default = false]

    Receiver is a named configuration of one or more notification integrations.

    We're not actively adding new receivers, we recommend implementing custom notification integrations via the receiver.

    <email_config>

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved:  | default = false ]
    4. # The email address to send notifications to.
    5. to: <tmpl_string>
    7. # The sender address.
    8. [ from:  | default = global.smtp_from ]
    10. # The SMTP host through which emails are sent.
    11. [ smarthost: <string> | default = global.smtp_smarthost ]
    13. # The hostname to identify to the SMTP server.
    14. [ hello:  | default = global.smtp_hello ]
    16. # SMTP authentication information.
    17. [ auth_username: <string> | default = global.smtp_auth_username ]
    18. [ auth_password:  | default = global.smtp_auth_password ]
    19. [ auth_secret: <secret> | default = global.smtp_auth_secret ]
    20. [ auth_identity:  | default = global.smtp_auth_identity ]
    22. # The SMTP TLS requirement.
    23. [ require_tls: <bool> | default = global.smtp_require_tls ]
    25. # TLS configuration.
    26. tls_config:
    27.   [ <tls_config> ]
    29. # The HTML body of the email notification.
    30. [ html:  | default = '{{ template "email.default.html" . }}' ]
    31. # The text body of the email notification.
    32. [ text: <tmpl_string> ]
    34. # Further headers email header key/value pairs. Overrides any headers
    35. # previously set by the notification implementation.
    36. [ headers: { : <tmpl_string>, ... } ]

    <hipchat_config>

    HipChat notifications use a Build Your Own integration.

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved:  | default = false ]
    4. # The HipChat Room ID.
    5. room_id: <tmpl_string>
    6. # The auth token.
    7. [ auth_token:  | default = global.hipchat_auth_token ]
    8. # The URL to send API requests to.
    9. [ api_url: <string> | default = global.hipchat_api_url ]
    11. # See https://www.hipchat.com/docs/apiv2/method/send_room_notification
    12. # A label to be shown in addition to the sender's name.
    13. [ from:   | default = '{{ template "hipchat.default.from" . }}' ]
    14. # The message body.
    15. [ message:  <tmpl_string> | default = '{{ template "hipchat.default.message" . }}' ]
    16. # Whether this message should trigger a user notification.
    17. [ notify:   | default = false ]
    18. # Determines how the message is treated by the alertmanager and rendered inside HipChat. Valid values are 'text' and 'html'.
    19. [ message_format:  <string> | default = 'text' ]
    20. # Background color for message.
    21. [ color:   | default = '{{ if eq .Status "firing" }}red{{ else }}green{{ end }}' ]
    23. # The HTTP client's configuration.
    24. [ http_config: <http_config> | default = global.http_config ]

    <pagerduty_config>

    PagerDuty notifications are sent via the PagerDuty API.PagerDuty provides on how to integrate. There are important differences with Alertmanager's v0.11 and greater support of PagerDuty's Events API v2.

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved: <boolean> | default = true ]
    4. # The following two options are mutually exclusive.
    5. # The PagerDuty integration key (when using PagerDuty integration type `Events API v2`).
    6. routing_key: 
    7. # The PagerDuty integration key (when using PagerDuty integration type `Prometheus`).
    8. service_key: <tmpl_secret>
    10. # The URL to send API requests to
    11. [ url:  | default = global.pagerduty_url ]
    13. # The client identification of the Alertmanager.
    14. [ client:  <tmpl_string> | default = '{{ template "pagerduty.default.client" . }}' ]
    15. # A backlink to the sender of the notification.
    16. [ client_url:   | default = '{{ template "pagerduty.default.clientURL" . }}' ]
    18. # A description of the incident.
    19. [ description: <tmpl_string> | default = '{{ template "pagerduty.default.description" .}}' ]
    21. # Severity of the incident.
    22. [ severity:  | default = 'error' ]
    24. # A set of arbitrary key/value pairs that provide further detail
    25. # about the incident.
    26. [ details: { <string>: , ... } | default = {
    27.   firing:       '{{ template "pagerduty.default.instances" .Alerts.Firing }}'
    28.   resolved:     '{{ template "pagerduty.default.instances" .Alerts.Resolved }}'
    29.   num_firing:   '{{ .Alerts.Firing | len }}'
    30.   num_resolved: '{{ .Alerts.Resolved | len }}'
    31. } ]
    33. # Images to attach to the incident.
    34.   [ <image_config> ... ]
    36. # Links to attach to the incident.
    37. links:
    38.   [  ... ]
    40. # The HTTP client's configuration.
    41. [ http_config: <http_config> | default = global.http_config ]

    <image_config>

    1. href: <tmpl_string>
    2. source: 
    3. alt: <tmpl_string>

    The fields are documented in the .

    1. href: <tmpl_string>
    2. text:

    <pushover_config>

    Pushover notifications are sent via the .

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved: <boolean> | default = true ]
    4. # The recipient user’s user key.
    5. user_key: 
    7. # Your registered application’s API token, see https://pushover.net/apps
    8. token: <secret>
    10. # Notification title.
    11. [ title:  | default = '{{ template "pushover.default.title" . }}' ]
    13. # Notification message.
    14. [ message: <tmpl_string> | default = '{{ template "pushover.default.message" . }}' ]
    16. # A supplementary URL shown alongside the message.
    17. [ url:  | default = '{{ template "pushover.default.url" . }}' ]
    19. # Priority, see https://pushover.net/api#priority
    20. [ priority: <tmpl_string> | default = '{{ if eq .Status "firing" }}2{{ else }}0{{ end }}' ]
    22. # How often the Pushover servers will send the same notification to the user.
    23. # Must be at least 30 seconds.
    24. [ retry:  | default = 1m ]
    26. # How long your notification will continue to be retried for, unless the user
    27. # acknowledges the notification.
    28. [ expire: <duration> | default = 1h ]
    30. # The HTTP client's configuration.
    31. [ http_config:  | default = global.http_config ]

    Slack notifications are sent via Slackwebhooks. The notification containsan .

    <action_config>

    The fields are documented in the .

    1. type: <tmpl_string>
    2. text: 
    3. url: <tmpl_string>
    4. [ style:  [ default = '' ]

    The fields are documented in the Slack API documentation.

    1. title: 
    2. value: <tmpl_string>
    3. [ short:  | default = slack_config.short_fields ]

    <opsgenie_config>

    OpsGenie notifications are sent via the .

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved: <boolean> | default = true ]
    4. # The API key to use when talking to the OpsGenie API.
    5. [ api_key:  | default = global.opsgenie_api_key ]
    7. # The host to send OpsGenie API requests to.
    8. [ api_url: <string> | default = global.opsgenie_api_url ]
    10. # Alert text limited to 130 characters.
    11. [ message:  ]
    13. # A description of the incident.
    14. [ description: <tmpl_string> | default = '{{ template "opsgenie.default.description" . }}' ]
    16. # A backlink to the sender of the notification.
    17. [ source:  | default = '{{ template "opsgenie.default.source" . }}' ]
    19. # A set of arbitrary key/value pairs that provide further detail
    20. # about the incident.
    21. [ details: { <string>: , ... } ]
    23. # List of responders responsible for notifications.
    24. responders:
    25.   [ - <responder> ... ]
    27. # Comma separated list of tags attached to the notifications.
    28. [ tags:  ]
    30. # Additional alert note.
    31. [ note: <tmpl_string> ]
    33. # Priority level of alert. Possible values are P1, P2, P3, P4, and P5.
    34. [ priority:  ]
    36. # The HTTP client's configuration.
    37. [ http_config: <http_config> | default = global.http_config ]

    <responder>

    1. # Exactly one of these fields should be defined.
    2. [ id: <tmpl_string> ]
    3. [ name:  ]
    4. [ username: <tmpl_string> ]
    6. # "team", "user", "escalation" or schedule".
    7. type:

    <victorops_config>

    VictorOps notifications are sent out via the 

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved: <boolean> | default = true ]
    4. # The API key to use when talking to the VictorOps API.
    5. [ api_key:  | default = global.victorops_api_key ]
    7. # The VictorOps API URL.
    8. [ api_url: <string> | default = global.victorops_api_url ]
    10. # A key used to map the alert to a team.
    11. routing_key: 
    13. # Describes the behavior of the alert (CRITICAL, WARNING, INFO).
    14. [ message_type: <tmpl_string> | default = 'CRITICAL' ]
    16. # Contains summary of the alerted problem.
    17. [ entity_display_name:  | default = '{{ template "victorops.default.entity_display_name" . }}' ]
    19. # Contains long explanation of the alerted problem.
    20. [ state_message: <tmpl_string> | default = '{{ template "victorops.default.state_message" . }}' ]
    22. # The monitoring tool the state message is from.
    23. [ monitoring_tool:  | default = '{{ template "victorops.default.monitoring_tool" . }}' ]
    25. # The HTTP client's configuration.
    26. [ http_config: <http_config> | default = global.http_config ]

    <webhook_config>

    The webhook receiver allows configuring a generic receiver.

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved: <boolean> | default = true ]
    4. # The endpoint to send HTTP POST requests to.
    5. url: 
    7. # The HTTP client's configuration.
    8. [ http_config: <http_config> | default = global.http_config ]

    The Alertmanagerwill send HTTP POST requests in the following JSON format to the configuredendpoint:

    <wechat_config>

    WeChat notifications are sent via the WeChatAPI.

    1. # Whether or not to notify about resolved alerts.
    2. [ send_resolved:  | default = false ]
    4. # The API key to use when talking to the WeChat API.
    5. [ api_secret: <secret> | default = global.wechat_api_secret ]
    7. # The WeChat API URL.
    8. [ api_url:  | default = global.wechat_api_url ]
    10. # The corp id for authentication.
    11. [ corp_id: <string> | default = global.wechat_api_corp_id ]
    13. # API request data as defined by the WeChat API.
    14. [ message:  | default = '{{ template "wechat.default.message" . }}' ]
    15. [ agent_id: <string> | default = '{{ template "wechat.default.agent_id" . }}' ]
    16. [ to_user:  | default = '{{ template "wechat.default.to_user" . }}' ]
    17. [ to_party: <string> | default = '{{ template "wechat.default.to_party" . }}' ]