Collecting Logs with Fluentbit

Setting up log collection consists of two pieces: running a log forwarding DaemonSet on each node, and running a collector somewhere in the cluster (in our example, we use a StatefulSet which stores logs on a Kubernetes PersistentVolumeClaim, but you could also use a HostPath).

It’s useful to set up the collector before the forwarders, because you’ll need the address of the collector when configuring the forwarders, and the forwarders may queue logs until the collector is ready.

The defines a StatefulSet as well as a Kubernetes Service which allows accessing and reading the logs from within the cluster. The supplied configuration will create the monitoring configuration in a namespace called logging. You can apply the configuration with:

The default configuration will classify logs into Knative, apps (pods with an label which aren’t Knative), and the default to logging with the pod name; this can be changed by updating the log-collector-config ConfigMap before or after installation. Once the ConfigMap is updated, you’ll need to restart Fluent Bit (for example, by deleting the pod and letting the StatefulSet recreate it).

And then visit .

You can also open a shell in the pod and search the logs using unix tools:

Setting up the forwarders

If you are using a different log collection infrastructure (Splunk, for example), on how to configure your forwarders.