Troubleshoot image rendering

    To troubleshoot the image renderer, different kind of logs are available.

    You can enable debug log messages for rendering in the Grafana configuration file and inspect the Grafana server logs.

    You can also enable more logs in image renderer service itself by:

    The plugin and rendering service uses which depends on certain libraries. If you don’t have all of those libraries installed in your system you may encounter errors when trying to render an image, e.g.

    2. error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory\n\n\nTROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md

    In general you can use the ldd utility to figure out what shared libraries are not installed in your system:

    1. ldd chrome-linux/chrome
    2.       linux-vdso.so.1 (0x00007fff1bf65000)
    3.       libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2047945000)
    4.       libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2047924000)
    5.       librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f204791a000)
    7.       libX11-xcb.so.1 => not found
    8.       libxcb.so.1 => not found
    9.       libXcomposite.so.1 => not found

    Ubuntu:

    1. libx11-6 libx11-xcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrender1 libxtst6 libglib2.0-0 libnss3 libcups2  libdbus-1-3 libxss1 libxrandr2 libgtk-3-0 libgtk-3-0 libasound2 libxcb-dri3-0 libgbm1 libxshmfence1

    Debian:

    On Debian 9 (Stretch) the following dependencies have been confirmed as needed for the image rendering to function.

    On Debian 10 (Buster) the following dependencies have been confirmed as needed for the image rendering to function.

    1. libxdamage1 libxext6 libxi6 libxtst6 libnss3 libnss3 libcups2 libxss1 libxrandr2 libasound2 libatk1.0-0 libatk-bridge2.0-0 libpangocairo-1.0-0 libpango-1.0-0 libcairo2 libatspi2.0-0 libgtk3.0-cil libgdk3.0-cil libx11-xcb-dev libgbm1 libxshmfence1

    Centos:

    On a minimal Centos installation, the following dependencies have been confirmed as needed for the image rendering to function:

    1. libXcomposite libXdamage libXtst cups libXScrnSaver pango atk adwaita-cursor-theme adwaita-icon-theme at at-spi2-atk at-spi2-core cairo-gobject colord-libs dconf desktop-file-utils ed emacs-filesystem gdk-pixbuf2 glib-networking gnutls gsettings-desktop-schemas gtk-update-icon-cache gtk3 hicolor-icon-theme jasper-libs json-glib libappindicator-gtk3 libdbusmenu libdbusmenu-gtk3 libepoxy liberation-fonts liberation-narrow-fonts liberation-sans-fonts liberation-serif-fonts libgusb libindicator-gtk3 libmodman libproxy libsoup libwayland-cursor libwayland-egl libxkbcommon m4 mailx nettle patch psmisc redhat-lsb-core redhat-lsb-submod-security rest spax time trousers xdg-utils xkeyboard-config alsa-lib

    In many cases, Grafana runs on internal servers and uses certificates that have not been signed by a CA (Certificate Authority) known to Chrome, and therefore cannot be validated. Chrome internally uses NSS () for cryptographic operations such as the validation of certificates.

    2. t=2019-12-04T12:39:22+0000 lvl=error msg="Rendering failed." logger=context userId=1 orgId=1 uname=admin error="Rendering failed: Error: net::ERR_CERT_AUTHORITY_INVALID at https://192.168.106.101:3443/d-solo/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m&from=1575438321300&to=1575459921300&var-Host=master1&panelId=4&width=1000&height=500&tz=Europe%2FBerlin&render=1"
    3. t=2019-12-04T12:39:22+0000 lvl=error msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/render/d-solo/zxDJxNaZk/graphite-metrics status=500 remote_addr=192.168.106.101 time_ms=310 size=1722 referer="https://grafana.xxx-xxx/d/zxDJxNaZk/graphite-metrics?orgId=1&refresh=1m"

    If this happens, then you have to add the certificate to the trust store. If you have the certificate file for the internal root CA in the file internal-root-ca.crt.pem, then use these commands to create a user specific NSS trust store for the Grafana user (grafana for the purpose of this example) and execute the following steps:

    Linux:

    Windows:

    1. certutil –addstore "Root" <path>/internal-root-ca.crt.pem

    As a last resort, if you already have or Chromium installed on your system, then you can configure the to use this instead of the pre-packaged version of Chromium.

    To override the path to the Chrome/Chromium executable in plugin mode, set an environment variable and make sure that it’s available for the Grafana process. For example:

    1. export GF_PLUGIN_RENDERING_CHROME_BIN="/usr/bin/chromium-browser"
    1. CHROME_BIN="/usr/bin/chromium-browser"