Quickly build an IT DevOps visualization system using TDengine + collectd/StatsD + Grafana

    IT DevOps metric data usually are time sensitive, for example:

    • System resource metrics: CPU, memory, IO, bandwidth, etc.
    • Software system metrics: health status, number of connections, number of requests, number of timeouts, number of errors, response time, service type, and other business-related metrics.

    The current mainstream IT DevOps visualization system usually contains a data collection module, a data persistence module, and a visual display module. collectd/StatsD, as an old-fashion open source data collection tool, has a wide user base. However, collectd/StatsD has limited functionality, and often needs to be combined with Telegraf, Grafana, and a time-series database to build a complete monitoring system. The new version of TDengine supports multiple data protocols and can accept data from collectd and StatsD directly, and provides Grafana dashboard for graphical display.

    This article introduces how to quickly build an IT DevOps visualization system based on TDengine + collectd / StatsD + Grafana without writing even a single line of code but by simply modifying a few lines in configuration files. The architecture is shown in the following figure.

    To install collectd, StatsD, Grafana, and TDengine, please refer to the official documentation.

    Please refer to the .

    Installing StatsD

    Please refer to the official documentation.

    Install TDengine

    Download the latest TDengine-server from the Downloads page on the TAOSData website and install it.

    Please refer to

    Configure collectd

    Add the following to the file, where the host and port should be the actual values of the TDengine and taosAdapter configurations.

    Start StatsD after adding the following to the config.js file, where the host and are the actual values of the TDengine and taosAdapter configurations.

    1. Add { host:'<TDengine server/cluster host>', port: <port for StatsD>} to the repeater section

    Importing the Dashboard

    Use a web browser to access the server running Grafana on port 3000 host:3000 to log into the Grafana interface with the initial system username and password of admin/admin. Click on the gear icon on the left and select , you should find the TDengine data source plugin icon.

    Importing the collectd dashboard

    TDengine Database IT-DevOps-Solutions-collectd-dashboard

    import collectd dashboard

    Download the dashboard json file from https://github.com/taosdata/grafanaplugin/blob/master/examples/collectd/grafana/dashboards/collect-metrics-with-tdengine-v0.1.0.json. Download the dashboard json file, click the plus icon on the left side and select Import, and follow the interface prompts to select the JSON file to import. After that, you can see dashboard with the following interface.

    Importing the StatsD dashboard

    Download the dashboard json from https://github.com/taosdata/grafanaplugin/blob/master/examples/statsd/dashboards/statsd-with-tdengine-v0.1.0.json. Click on the plus icon on the left and select , and follow the interface prompts to import the JSON file. You will then see the dashboard in the following screen. TDengine Database IT-DevOps-Solutions-statsd-dashboard

    TDengine, as an emerging time-series big data platform, has the advantages of high performance, high reliability, easy management and easy maintenance. Thanks to the new schemaless protocol parsing feature in TDengine and ability to integrate easily with a large software ecosystem, users can build an efficient and easy-to-use IT DevOps visualization system, or adapt an existing system, in just a few minutes.

    For TDengine’s powerful data writing and querying performance and other features, please refer to the official documentation and successful product implementation cases.