Skip to content
KloudMate Docs

Kafka Prometheus ServiceMonitor Integration

Use this integration when a Kafka cluster (for example, a Bitnami Helm release) runs in Kubernetes and an existing Prometheus setup already scrapes it through a ServiceMonitor. Instead of standing up a new scrape path, you point the KloudMate Agent’s Prometheus receiver at the same metrics endpoints.

Prerequisites

Section titled “Prerequisites”
  1. The KloudMate Agent is deployed and running in the cluster.
  2. The scrape details from the ServiceMonitor(s) already collecting Kafka metrics.

Step 1: Gather Details from the Existing ServiceMonitor

Section titled “Step 1: Gather Details from the Existing ServiceMonitor”
kubectl get servicemonitor -n <kafka-namespace> -o yaml

From the output, note:

  1. spec.selector.matchLabels → identifies which Service it targets.
  2. spec.endpoints[].port or targetPort → the metrics port (Bitnami defaults: 5556 for the JMX exporter, 9308 for kafka-exporter).
  3. spec.endpoints[].path → usually /metrics.
  4. spec.endpoints[].interval → match this in your scrape config to stay consistent with the existing setup.
  5. spec.namespaceSelector → confirms the namespace to target.

Step 2: Open the Collector Configuration

Section titled “Step 2: Open the Collector Configuration”
  1. Go to Settings → Agents.
  2. Find the agent you want to configure and select the three dots.
  3. Choose Configure.
  4. Go to Collector config.

Step 3: Add the Prometheus Receiver

Section titled “Step 3: Add the Prometheus Receiver”

Add the following configuration. Replace <release>, <kafka-namespace>, and <cluster-name> with your own values, and adjust the ports and interval if your ServiceMonitor differs from the Bitnami defaults.

  • If the agent runs in managed mode, add it to the Custom config override section. See Custom config override.
  • If the agent runs in manual mode, add it directly to the collector configuration.
receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: 'kafka-jmx'
          scrape_interval: 30s          # match the ServiceMonitor's interval
          metrics_path: /metrics
          static_configs:
            - targets: ['<release>-kafka-metrics.<kafka-namespace>.svc.cluster.local:5556']
              labels:
                cluster: '<cluster-name>'

        - job_name: 'kafka-exporter'
          scrape_interval: 30s
          metrics_path: /metrics
          static_configs:
            - targets: ['<release>-kafka-exporter.<kafka-namespace>.svc.cluster.local:9308']
              labels:
                cluster: '<cluster-name>'

service:
  pipelines:
    metrics/kafka:
      receivers:
        - prometheus
      processors:
        - batch
      exporters:
        - otlphttp

Step 4: Save the Configuration

Section titled “Step 4: Save the Configuration”

Click Save. The Agent restarts and applies the new configuration.

Step 5: Validate in KloudMate

Section titled “Step 5: Validate in KloudMate”

After the Agent restarts:

  • Open Explore in KloudMate.
  • Query one of the scraped metrics, for example:
kafka_server_brokertopicmetrics_messagesin_total

kafka_server_replicamanager_leadercount

kafka_consumergroup_lag
  • Build dashboards or alerts once the data is visible.

Standard Kafka Dashboards

Section titled “Standard Kafka Dashboards”

To import and start using a template, follow the steps described in Importing a Dashboard. Alternatively, use From Template when creating a new dashboard to pick from KloudMate’s built-in templates.

Metrics Collected

Section titled “Metrics Collected”

This integration collects:

  • Kafka broker metrics
  • Kafka topic metrics
  • Kafka consumer group metrics
  • Kafka Exporter metrics
  • JVM metrics
  • Broker health metrics
  • Replication metrics
  • Network metrics
  • Request metrics
  • Controller metrics
Section titled “Related”