Federation

There are different use cases for federation. Commonly, it is used to eitherachieve scalable Prometheus monitoring setups or to pull related metrics fromone service's Prometheus into another.

For example, a setup might consist of many per-datacenter Prometheus serversthat collect data in high detail (instance-level drill-down), and a set ofglobal Prometheus servers which collect and store only aggregated data(job-level drill-down) from those local servers. This provides an aggregateglobal view and detailed local views.

Cross-service federation

For example, a cluster scheduler running multiple services might exposeresource usage information (like memory and CPU usage) about service instancesrunning on the cluster. On the other hand, a service running on that clusterwill only expose application-specific service metrics. Often, these two sets ofmetrics are scraped by separate Prometheus servers. Using federation, thePrometheus server containing service-level metrics may pull in the clusterresource usage metrics about its specific service from the cluster Prometheus,so that both sets of metrics can be used within that server.

Configuring federation

To federate metrics from one server to another, configure your destinationPrometheus server to scrape from the /federate endpoint of a source server,while also enabling the honor_labels scrape option (to not overwrite anylabels exposed by the source server) and passing in the desired parameters. For example, the following scrape_configs federates any serieswith the label job="prometheus" or a metric name starting with job: fromthe Prometheus servers at into the scrapingPrometheus: