MongoDB Monitoring

mongodb is a widely used nosql database designed for scalability and flexibility it stores data in a json like format (bson), enabling dynamic schema design and efficient handling of unstructured data mongodb is known for its high performance in read and write heavy environments and is commonly used for real time analytics, content management, and iot systems it supports horizontal scaling through sharding and provides built in replication for high availability mongodb monitoring in kloudmate helps you observe the health, performance, and behavior of your mongodb servers by collecting metrics and logs using the kloudmate agent powered by opentelemetry this allows you to monitor mongodb instances running on aws ec2, azure virtual machines, or on premise servers from a centralized view what this integration provides with mongodb monitoring enabled, kloudmate collects telemetry at multiple levels, including server level performance metrics database and collection level metrics connection, operation, and lock behavior network and storage usage mongodb server logs this visibility helps identify performance bottlenecks, abnormal query behavior, resource saturation, and availability issues pre requisites before configuring mongodb monitoring, ensure the following mongodb server is running supported mongodb versions 4 0+, 5 0, 6 0, 7 0 kloudmate agent installed on the mongodb host refer to agent installation for https //docs kloudmate com/linux agent or https //docs kloudmate com/windows agent mongodb user configured with least privilege access mongodb recommends creating a user with the https //www mongodb com/docs/v5 0/reference/built in roles/#mongodb authrole clustermonitor to collect metrics refer to https //github com/open telemetry/opentelemetry collector contrib/blob/main/receiver/mongodbreceiver/testdata/integration/scripts/lpu sh for an example of how to configure these permissions example mongosh use admin db createuser({ user "user", pwd "password", roles \[{ role "clustermonitor", db "admin" }] }) exit step 1 access agents and opentelemetry collector configuration log in to the kloudmate platform navigate to settings → agents select the agent running on the mongodb host open collector configuration from the agent actions the configuration editor opens, allowing you to modify the yaml directly step 2 add required extensions and receivers extensions add the following extensions to support health checks, debugging, and local storage extensions health check pprof endpoint 0 0 0 0 1777 zpages endpoint 0 0 0 0 55679 file storage create directory true receivers two receivers can be used with mongodb mongodb receiver – collects mongodb metrics file log receiver – collects mongodb logs if you only need metrics, configure only the mongodb receiver receivers mongodb hosts \ endpoint localhost 27017 username otel #change the correct username password passwd #change the correct password collection interval 60s initial delay 1s tls insecure true insecure skip verify true filelog include \[/var/log/mongodb/ log] storage file storage retry on failure enabled true step 3 configure processors set up the processor component to identify resource information from the host and either append or replace the resource values in the telemetry data with this information choose one configuration based on where mongodb is running server (on premise / non cloud / cloud) processors batch send batch size 5000 timeout 10s resourcedetection detectors \[system] system resource attributes host name enabled true host id enabled true os type enabled false resource attributes \ key service name action insert from attribute host name aws ec2 (optional) to collect ec2 tags, attach an iam role with ec2\ describetags permission processors batch send batch size 5000 timeout 10s resourcedetection/ec2 detectors \["ec2"] ec2 tags \ ^tag1$ \ ^tag2$ \ ^label $ \ ^name$ resource attributes \ key service name action insert from attribute ec2 tag name \ key service instance id action insert from attribute host id azure virtual machines processors batch send batch size 5000 timeout 10s resourcedetection/azure detectors \["azure"] azure tags \ ^tag1$ \ ^tag2$ \ ^label $ \ ^name$ resource attributes \ key service name action insert from attribute azure vm name \ key service instance id action insert from attribute host id step 4 configure exporter and pipelines configure the kloudmate backend exporter and define pipelines for metrics and logs exporters debug verbosity detailed otlphttp endpoint 'https //otel kloudmate com 4318' headers authorization xxxxxxxx # use the auth key service pipelines metrics receivers \[mongodb] processors \[batch, resourcedetection, resource] # apply the correct resourcedetection exporters \[otlphttp] logs receivers \[filelog] processors \[batch, resourcedetection, resource] # apply the correct resourcedetection exporters \[otlphttp] extensions \[health check, pprof, zpages, file storage] step 5 save configuration and restart agent after updating the configuration, click save configuration in the kloudmate ui you can also restart the agent from your server’s console for linux execute the following commands systemctl restart kmagent systemctl status kmagent these commands restart the kloudmate agent and display its current status for windows open the services window press win + r , type services msc , and press ok or search for services from the start menu locate the kloudmate agent service right click the service and select restart after restarting the agent, verify that mongodb metrics and logs are visible in the kloudmate dashboard post‑integration data validation verify that metrics are flowing into kloudmate using the explore view after the agent restarts log in to kloudmate navigate to explore select opentelemetry → metrics choose a mongodb metric and run the query seeing time series data confirms that mongodb telemetry is flowing successfully standard mongodb dashboards kloudmate provides prebuilt mongodb dashboards through https //templates kloudmate com/mongo db/index html these dashboards visualize mongodb server performance, operations, storage, and resource usage to import and start using these templates, follow the steps described in https //docs kloudmate com/creating a dashboard#sfk3g default mongodb metrics mongodb monitoring automatically collects commonly used mongodb metrics when enabled example metrics metric name description mongodb connection count the number of connections mongodb operation count the number of operations executed mongodb operation latency time the latency of operations mongodb data size the size of the collection data compression does not affect this value mongodb storage size the total amount of storage allocated to this collection mongodb network io receive the number of bytes received mongodb network io transmit the number of bytes transmitted mongodb lock acquire count number of times the lock was acquired in the specified mode mongodb lock acquire time cumulative wait time for the lock acquisitions mongodb health the health status of the server mongodb uptime the amount of time that the server has been running for the complete metrics list, refer to the https //github com/open telemetry/opentelemetry collector contrib/blob/main/receiver/mongodbreceiver/documentation md optional deployment methods vm/ec2 with fluent bit https //blog kloudmate com/streaming mongodb metrics logs to kloudmate using fluent bit 32b54e91fe93?postpublishedtype=initial docker compose https //blog kloudmate com/monitoring mongodb with kloudmate using docker compose 49a5dc59c660?postpublishedtype=initial