Database monitoring on Kubernetes

On Kubernetes, the agent monitors the databases running in your cluster. The flow is the same as on a virtual machine, with two Kubernetes differences. Credentials come from a Kubernetes Secret, and database monitoring needs managed mode. For the general model, see the Database monitoring overview.

One monitor per cluster

On Kubernetes, one collector in the cluster monitors your databases. Each database is monitored once for the whole cluster, not once per node.

Discovery prefills the endpoint

The agent finds databases running in your cluster. It prefills the wizard with their in-cluster address, <service>.<namespace>.svc.cluster.local:<port>. Use this address. It is the name your applications connect to, so the database’s metrics and traces line up. Databases outside the cluster are not discovered, so add them manually.

Managed mode is required

Database monitoring needs the agent in managed mode. On Kubernetes, use the “Configure databases” action on the agent. If the agent is still in manual mode, it prompts you to switch. See the configuration model.

Available coverages

On Kubernetes, the agent collects database metrics and query coverages: standard metrics, query performance, locks, and query logs. Database log coverages are available on a virtual machine, not on Kubernetes. See Engines and coverages.

Provide credentials with a Kubernetes Secret

Credentials use an environment reference backed by a Kubernetes Secret you provide. Set it up before you configure any database, because the collector reads the Secret when its pod starts.

1. Create a Kubernetes Secret with a key of the form KM_SECRET_<name> for each database credential. For example, a key KM_SECRET_postgresql holding the monitoring user’s password.

   kubectl create secret generic km-db-creds \
     --namespace km-agent \
     --from-literal=KM_SECRET_postgresql='<password>'

2. Point the Helm release at the Secret with the dbMonitoring.secretName value, and upgrade the release. The cluster collector then reads the Secret through envFrom.

   helm upgrade kloudmate-release kloudmate/km-kube-agent \
     --namespace km-agent --reuse-values \
     --set dbMonitoring.secretName=km-db-creds

3. Configure the database in the wizard, referencing the key by name. The password stays in the cluster and never reaches KloudMate.

Caution

Because the collector reads the Secret at pod startup, create the Secret and run the Helm upgrade before you configure a database. If you rotate a password without changing the configuration, the collector keeps the old value until its pod restarts.

Worked example: PostgreSQL

1. Create a monitoring role in PostgreSQL and grant it the built-in pg_monitor role:

   CREATE ROLE kmmonitor LOGIN PASSWORD '<password>';
   GRANT pg_monitor TO kmmonitor;

2. Store the password in a Kubernetes Secret under KM_SECRET_postgresql and set dbMonitoring.secretName, as above.

3. Configure PostgreSQL in the wizard with the Service DNS endpoint (prefilled from discovery), the kmmonitor user, and an environment reference to the secret. Choose the coverages you want, such as standard metrics, locks, and query performance.

4. Enable query performance prerequisites, if used. Query performance reads pg_stat_statements. Add it to shared_preload_libraries and create the extension, which needs a PostgreSQL restart:

   -- postgresql.conf: shared_preload_libraries = 'pg_stat_statements'  (then restart)
   CREATE EXTENSION pg_stat_statements;

After you apply the configuration, the cluster collector starts collecting from PostgreSQL. Its metrics carry the same database identity as the application spans that hit it.

Next steps

Credentials How the credential modes work.

Kubernetes platform notes Fleet manager, collector pods, and ConfigMaps.

Telemetry identity Build dashboards that join metrics and traces.