Manage subscriptions in InfluxDB

    As data is written to InfluxDB, writes are duplicated to subscriber endpoints viaHTTP, HTTPS, or UDP in line protocol.the InfluxDB subscriber service creates multiple “writers” ()which send writes to the subscription endpoints.

    The number of writer goroutines is defined by the configuration.

    As writes occur in InfluxDB, each subscription writer sends the written data to thespecified subscription endpoints.However, with a high write-concurrency (multiple writers) and a high ingest rate,nanosecond differences in writer processes and the transport layer can resultin writes being received out of order.

    InfluxQL subscription statements

    Use the following InfluxQL statements to manage subscriptions:

    CREATE SUBSCRIPTIONDROP SUBSCRIPTION

    Create subscriptions using the CREATE SUBSCRIPTION InfluxQL statement.Specify the subscription name, the database name and retention policy to subscribe to,and the URL of the host to which data written to InfluxDB should be copied.

    1. CREATE SUBSCRIPTION "sub0" ON "mydb"."autogen" DESTINATIONS ALL 'http://subscriber:secret@example.com:8086'

    SHOW SUBSCRIPTIONS outputs all subscriber URL in plain text, including those with authentication credentials.Any user with the privileges to run SHOW SUBSCRIPTIONS is able to see these credentials.

    The CREATE SUBSCRIPTION statement allows you to specify multiple hosts as endpoints for the subscription.In your DESTINATIONS clause, you can pass multiple host strings separated by commas.Using ALL or ANY in the DESTINATIONS clause determines how InfluxDB writes data to each endpoint:

    ALL: Writes data to all specified hosts.

    : Round-robins writes between specified hosts.

    Subscriptions with multiple hosts

    1. -- Write all data to multiple hosts
    2. CREATE SUBSCRIPTION "mysub" ON "mydb"."autogen" DESTINATIONS ALL 'http://host1.example.com:9090', 'http://host2.example.com:9090'
    4. -- Round-robin writes between multiple hosts
    5. CREATE SUBSCRIPTION "mysub" ON "mydb"."autogen" DESTINATIONS ANY 'http://host1.example.com:9090', 'http://host2.example.com:9090'

    Subscriptions can use HTTP, HTTPS, or UDP transport protocols.Which to use is determined by the protocol expected by the subscription endpoint.If creating a Kapacitor subscription, this is defined by the subscription-protocoloption in the [[influxdb]] section of your .

    kapacitor.conf

    For information regarding HTTPS connections and secure communication between InfluxDB and Kapacitor,view the Kapacitor security documentation.

    Show subscriptions

    1. SHOW SUBSCRIPTIONS

    Example output:

    1. name: _internal
    2. retention_policy name                                           mode destinations
    3. ---------------- ----                                           ---- ------------
    4. monitor          kapacitor-39545771-7b64-4692-ab8f-1796c07f3314 ANY  [http://localhost:9092]

    Remove or drop subscriptions using the DROP SUBSCRIPTION InfluxQL statement.

    In some cases, it may be necessary to remove all subscriptions.Run the following bash script that utilizes the influx CLI, loops through all subscriptions, and removes them.This script depends on the $INFLUXUSER and $INFLUXPASS environment variables.If these are not set, export them as part of the script.

    1. # Uncomment these if INFLUXUSER and INFLUXPASS are not already globally set.
    2. # export INFLUXUSER=influxdb-username
    3. # export INFLUXPASS=influxdb-password
    5. IFS=$'\n'; for i in $(influx -format csv -username $INFLUXUSER -password $INFLUXPASS -database _internal -execute 'show subscriptions' | tail -n +2 | grep -v name); do influx -format csv -username $INFLUXUSER -password $INFLUXPASS -database _internal -execute "drop subscription \"$(echo "$i" | cut -f 3 -d ',')\" ON \"$(echo "$i" | cut -f 1 -d ',')\".\"$(echo "$i" | cut -f 2 -d ',')\""; done

    Configure InfluxDB subscriptions

    InfluxDB subscription configuration options are available in the [subscriber]section of the influxdb.conf.In order to use subcriptions, the enabled option in the [subscriber] section must be set to true.Below is an example influxdb.conf subscriber configuration:

    1. [subscriber]
    2.   enabled = true
    3.   http-timeout = "30s"
    4.   insecure-skip-verify = false
    5.   ca-certs = ""
    6.   write-concurrency = 40

    Descriptions of configuration options are available in the documentation.

    Unless a subscription is dropped, InfluxDB assumes the endpointshould always receive data and will continue to attempt to send data.If an endpoint host is inaccessible or has been decommissioned, you will see errorssimilar to the following:

    In some cases, this may be caused by a networking error or something similarpreventing a successful connection to the subscription endpoint.In other cases, it’s because the subscription endpoint no longer exists andthe subscription hasn’t been dropped from InfluxDB.

    Because InfluxDB does not know if a subscription endpoint will or will not become accessible again,subscriptions are not automatically dropped when an endpoint becomes inaccessible.If a subscription endpoint is removed, you must manually from InfluxDB.